Virtualization Technology News and Information
Zoomdata 2018 Predictions: What We See Coming in 2018

VMblog Predictions 2018

Industry executives and experts share their predictions for 2018.  Read them in this 10th annual series exclusive.

Contributed by Ruhollah Farchtchi, CTO at Zoomdata

What We See Coming in 2018

In our predictions blog for 2017, we noted three trends that characterized the declining prevalence of standalone BI applications:

1. Interactive exploration on the freshest data

2. The end of static data models

3. Search-based analytics gaining greater traction

All three of those trends continued throughout the year and feed directly into our predictions for 2018.

1. For analytics, the traditional relational database is dying.

According to the results of an O'Reilly study commissioned by Zoomdata on the state of data analytics and visualization adoption, more than two-thirds of data analytics now run on modern non-relational data platforms like Hadoop, NoSQL, and cloud and search databases.

Of course, relational databases continue to be the core of online transactional processing (OLTP) systems. However, when asked about their organization's main data sources, less than one-third of participants listed the relational database, with two-thirds selecting non-relational sources. This is a clear indication that these non-relational data platforms have firmly crossed the chasm from early adopters into mainstream use.

And, of course, this makes sense. When a new generation of BI tools can query data without modeling it first (2017: the end of the static data model), who needs the relational database? But you still need a place to land and store the data. So, enter the modern data platform.

2. Data delivered through modern data platforms will be used by organizations through an ecosystem of applications and interfaces.

The cloud has changed the game for application development. Now that new types of databases leverage cloud-based storage and look to make compute more elastic, the relational database is being deconstructed into its essentials: data, processing, and metadata. This will give rise to an ecosystem of best-of-breed applications and interfaces that handle storage, processing, metadata, and security. Higher-level tools that cover analytics, exploration, and governance are being re-envisioned as well - and will replace the single, end-to-end toolset. This will be especially true around data management and governance, where new iterations of the data catalog and master data management (MDM) will guide users to the data they need. Instead of a single system managing data integrity, provenance or data lineage will be the key and made transparent for users to decide on what data to use in support of their analysis.

3. Working with streaming data will become the new "normal."

In the O'Reilly study, more than 15 percent of respondents said they needed their data to be less than one hour old; and over 10 percent needed it to be less than one minute old. That's why so many organizations have adopted or plan to adopt a streaming data platform.

Businesses need to work with all the data, not just real-time streams. The difference between a stream of data coming off a device versus a data supply chain where data is manipulated, managed, and refined as a stream, record-by-record rather than in batches is diminishing. In the latter, data is constantly ingested, refined, managed, manipulated, and explored, only to be fed back into that loop again and again.

In the past, a batch of data would land in a file system and at a predetermined time an ETL process kicked in to process it. But the need for fresh data is driving a reforming of those infrastructures. As soon as new data lands, it needs to be processed. Instead of extract, transform and load, we have push, enrich, and consume. Like the relational database, batch is dying.


About the Author

Ruhollah Farchtchi 

Ruhollah Farchtchi is the CTO at Zoomdata. He has over 15 years' experience in enterprise data management, architecture and systems integration. Prior to Zoomdata, he held management positions at BearingPoint, Booz-Allen and Unisys. Ruhollah holds an M.S. in Information Technology from George Mason University. 

Published Monday, December 11, 2017 7:13 AM by David Marshall
There are no comments for this post.
To post a comment, you must be a registered user. Registration is free and easy! Sign up now!
<December 2017>