Scroll Top

The Inevitable Coming Together of Big Data and IoT

When “wireless” is perfectly applied, the whole earth will be converted into a huge brain, which in fact it is, all things being particles of a real and rhythmic whole… and the instruments through which we shall be able to do this will be amazingly simple compared with our present telephone. A man will be able to carry one in his vest pocket.” : Nikola Tesla In Colliers Magazine in 1926.

The first signs of the connectedness Tesla spoke of became apparent when the boffins at Carnegie Mellon University started monitoring their Coke vending machine from the computer, and, also, memorably when John Romkey demonstrated the ability to turn on and off a toaster over the internet, for the Interop conference in Vegas in 1989. Several such demonstrations followed – clearly hooking up machines to the network and getting them to “do stuff” is not a new idea, and it is also apparent that the technology needed to make this connection happen has also been around for a while. But if that is the case then why is it that talk of the Internet of Things, or IoT, has only, really, started enveloping us in the last couple of years? The answer folks, lies in data, lots of data – so much data, in fact, that it’s called Big Data. Our view is that the real power of connected devices only started coming to the fore once the IoT movement synergistically dovetailed into the Big Data / Analytics movement.

Consider that the IoT comprises a truly staggering network of sensor-enabled devices of all shapes, sizes, and purpose. These devices are connected to the network through wired or wireless networks, through which humans and systems communicate with them. So, what’s the role of Big Data

The specific function of each device varies but each and every one of them has one common characteristic – they capture data all the time. In many ways, the revolution that has come about is a “data collection” revolution. Where earlier it was very hard, or expensive to collect the right kind of data – with the coming of the IoT, devices are getting smaller, cheaper, and more ubiquitous, and they are collecting all the data enterprises need – and more. In 2014, EMC & IDC published a report talking about “The Digital Universe”. They predicted that by 2020 this Digital Universe of things would generate as much as 44 trillion gigabytes of data.

All these devices then transmit data over the internet to central locations where data from many such devices is collected, stored, secured, categorized, and ultimately analyzed. It stands to reason that the true value of this entire software-driven construct will only come about when this data can drive impactful business decisions. For that, the data must be sifted through, combined in meaningful ways, delved into to identify hidden trends, and then presented in an easy-to-understand manner to those tasked with taking the decisions that matter. The sheer volume of “useful” data these devices generate is on the rise – the Digital Universe report mentioned earlier, predicted that from 22% “useful data” on 2013, this would rapidly reach 35% of all data generated, by 2020. Isn’t that a textbook case for the application of the practice of Big Data and Analytics?

Extending this argument even further, an even greater value can be derived when appropriate decisions can be taken in real-time to respond to environmental, market or system-driven conditions. When faced with such volume, it may not always be desirable to wait for human intervention. This is where the system can be empowered to take appropriate actions instantly. These decisions would be driven by data, the algorithms and code created by the data scientists could trigger the appropriate action. Machine Learning can also come into the picture, to help the system evolve over time. As the system becomes more intelligent, it also becomes nimble!

Bill Schmarzo, CTO of Dell EMC, called the “Dean of Big Data”, while commenting on the Digital Universe report said, “The real benefits of the Internet of Things will not, however, be realized until leading companies develop the next generation of applications that address specific business needs from this wealth of data.” In a similar vein, the Dresner Advisory in their 2016 “The IoT and BI Market Study” reported that advocates of IoT were 3 times as likely to believe that big data was critical to their initiative being successful. The point being made by all these worthies is the same – for IoT to deliver its promised potential to any enterprise, it has to be combined with a robust, considered, and visionary Big Data and Analytics initiative. Without laboring the point more, let’s leave the last word to Stephen Lawson of EMC, “IoT will stump IT until clouds and big data come aboard.”

Leave a comment