Innovation and market perspectives from leading IoT innovators
We spoke with Mike Freedman and Ajay Kulkarni of TimescaleDB. The team has created a database specifically targeted for IoT use cases that can be deployed on devices at the edge, on premise or in the cloud. Our conversation focused on how the unique scale of IoT implementations drove need for a different approach to manage data. Their focus aligns with Momenta’s view that the keys to meaningful success Connected Industry is about solving the digital transformation challenges of real businesses.
Could you share a bit of your backgrounds and what led you to found TimescaleDB?
We (Mike and Ajay) have over 20 years of history together. Ajay entered the tech industry, earned an MBA back at MIT, and started Sensobi, a communication data analysis startup, which later was acquired by GroupMe. (GroupMe was then acquired by Skype, which was then acquired by Microsoft).
Mike earned a PhD at NYU and Stanford, where he built CoralCDN, an open CDN which served millions of daily users, and helped create Software Defined Networking. While at Stanford, he also started Illuminics Systems, a company focused on network intelligence and geolocation, which was acquired by Quova/Neustar. He now doubles as a Full Professor of Computer Science at Princeton.
We teamed up again a few years ago to focus on a new opportunity around data and IoT. The first thing we built was a moderately successful IoT platform, with nearly 100,000 devices sending data to and using the platform. We were collecting a lot of sensor data and needed a place to store it. We tried a few time-series databases and were unhappy with them, so we decided to build a new database for our own needs.
We soon discovered there was more interest in the database than our IoT platform. Developers needed some place to store their IoT data and non-IoT time-series data, which they also wanted to join with their operational data. We realized that a better way to tackle the IoT data opportunity was a database that businesses could use as a fundamental building block. We re-branded and launched last April as TimescaleDB, and since then have gained over 4,000 stars on GitHub, and are now deployed on a variety of sites globally.
What are limitations of other types of databases that led you to build your own?
There are challenges in IoT that didn’t exist before. First, the scale and ingest volume are rapidly growing. In this last wave of online web services, the volume of user data led to the development of non-relational or NoSQL databases. Yet the volume of machine data in IoT demands an even higher insert volume and velocity. Second, the questions that we ask of time-series data are becoming quite complex. We’re not simply monitoring the latest metrics collected about a device; we’re also asking questions to help us understand the conditions of entire systems or even predict their future, often in real time. Third, you traditionally build a system with silos: business data on one side, and big data or time-series data on another side. With IoT, you need to consider this data together. If you are in manufacturing or agriculture, your IoT data has both operational and business components.
When we tried to use NoSQL options, we found that they would push complexity to our developers. When we wanted to ask questions, we would continually need to write application code to pull and join data across our two silo’d databases. This was slow, inefficient, and annoying.
We heard similar stories from other companies: If you were in manufacturing and wanted to compare machines from different manufacturers, you’d have to modify the code. Queries would be limited to what a data API exposes. Sometimes the ability to ask questions required for engineering to add new capabilities into the API service, which could take a couple of months. Yet all you really needed was a SQL join.
Where do databases fit with the cloud/edge computing?
When people ask whether the data should reside in the cloud or at the edge, our answer is always: Yes. Database deployments are heterogeneous, with intelligence varying from the cloud and edge. Data can live anywhere in the stack; its placement often depends on device and deployment constraints. We got into the database business so we could play at any point in the stack – on a Raspberry PI, an edge gateway, a server rack, or in the cloud. We are agnostic on the cloud/edge difference, because it’s all based on the specific use case.
Heterogeneity will persist. If you are trying to connect remote devices from a remote part of Australia, you are not using WiFi. Similar, when deploying sensors on a ship, power may not be an issue, but connectivity may need to come through satellites. Yet in the smart home, a wired device may want to send all its data to the cloud. So, we find that people deployed TimescaleDB at different places in the system depending on their needs.
What could help companies accelerate their adoption?
One of the biggest challenges in the space is the noise – there are so many platforms and different players. You need to start bottom up, what are the problems and build from this to address business problems. Then, what are the experiences and processes that this technology can enable? Technologies make an impact because they solve problems, not because they are part of a top down approach.
So in short, to accelerate the adoption of these new technologies, we need focus on solving problems, and stop using these buzzwords as communicational crutches.