Industry executives and experts share their predictions for 2022. Read them in this 14th annual VMblog.com series exclusive.
Enterprises Recognize the Requirements of a “Stream-First” Organization
By Joe Witt, Corporate Vice President at Cloudera
In 2022, enterprises recognize what it means to become “stream-first.” We know that enterprises in manufacturing, supply chain management, cybersecurity, smart cities and buildings, and more are dealing with huge amounts of data generated in real-time by all the devices, systems, software and third parties involved in their operations.
These businesses need real-time insights into their streaming data, complete with dashboards and alarms, and they need custom applications and microservices they can build themselves. But to accomplish this, they must solve three key challenges.
First, they must ensure their entire approach and tooling ensures real-time data collection at all points, while enabling distribution and easy access to all this information. And they must achieve this across all data in any source system residing in any location, on-premises or in any cloud – with full data lifecycle management and governance capabilities – without needing to bring the data to rest.
Second, a business can’t be stream-first without solving the challenge of connecting streaming data with their legacy batch systems. The systems they deploy for streams and low latency must also be able to interoperate with batch systems, allowing users to easily query, enrich and join streams from within their at-rest and batch data sources.
Finally, because streaming data is rapidly becoming the system of record for companies, all this data now falls within the purview of compliance and governance teams, who must have insight into the full data lifecycle, including origin, provenance, sensitivity, location and transformations, as well as who has access to the data, when and from where.
In the coming year, as enterprises increasingly recognize the need to become stream-first, vendors will focus on ensuring their solutions and partner ecosystems deliver these essential stream-first capabilities, allowing customers to drive their new business models and use cases without the fear of compliance failures or having to gut their legacy systems.
##
ABOUT THE AUTHOR
Joe Witt, is Corporate Vice President at Cloudera, leading product and engineering for Cloudera’s family of Data in Motion products. Joe had joined Cloudera via the merger with Hortonworks where he was Vice President of Engineering. Prior to his six years at Cloudera and Hortonworks, Joe was the co-founder and CTO of Onyara Inc. and had spent 10 years as a software developer for the US Federal Government focused specifically on connecting a global network of systems delivering massive volumes of data. During that time Joe created NiFi, built a team and community around it, and helped navigate its release to the Apache Software Foundation as Apache NiFi.
