Virtualization Technology News and Information
Unravel Data 2020 Predictions: Kubernetes and SREs reshape approach to data workloads in 2020, while organizations get a reality check on NoOps

VMblog Predictions 2020 

Industry executives and experts share their predictions for 2020.  Read them in this 12th annual series exclusive.

By Kunal Agarwal, CEO of Unravel Data

Kubernetes and SREs reshape approach to data workloads in 2020, while organizations get a reality check on NoOps

The conversation around big data has changed quite a bit over the past few years. These days, fewer people even use the term "big data" and instead talk about how this data is being put to use, particularly in artificial intelligence and machine learning applications. This reflects a positive shift: organizations have deployed big data and are now looking at how to get the most real-world value from it. 

This evolution will be further exemplified in 2020 as enterprises focus less on specific big data technologies and more on use cases. In addition, the conversation around modern data workloads will be impacted by the continued rise of Kubernetes, increasing use of SREs and a changing POV on NoOps.

Next year will be dominated by these data trends:

SREs change big data: Enterprises have been using SREs (site reliability engineers) for a few years to ensure their websites function properly by developing automated fixes for any web errors that may occur. This approach to automated troubleshooting is now moving to data apps, with DataOps teams both adopting the general approach of SREs as well as hiring specialized SREs to solve data analytics problems. The role of data scientist has made lots of buzz as one of the most in-demand jobs over the past few years. Expect more major enterprises, especially web companies, to increasingly hire SREs for data apps and to create SRE positions that will be focused on specific technologies like Hadoop, Spark, Kafka, etc. 

NoOps falls short: The concept of NoOps gained some steam in 2019. Using AI to automate does make operations far more efficient, but the notion that organizations can leverage cloud services and AI to eliminate all IT operations is a pipe dream. The reality is that you need DevOps and DataOps in the cloud just like you do on-premises. The cloud is an ideal destination for many workloads, especially data workloads, but the operational challenges from on-premises deployments don't just disappear when you get to cloud. In fact, new challenges emerge, such as re-configuring apps for improved performance in the cloud and monitoring workloads for overuse (and increased costs). There are AI tools that significantly simplify these efforts, but organizations will need human operations teams to leverage those tools correctly. The cloud is great, but there need to be guard rails in place to ensure it's delivering on cost and performance. The NoOps trend reminds me of a time 5-8 years ago when people thought the cloud be the panacea for everything. Instead, it's clear that a hybrid model has won out with the cloud becoming ideal for many apps but others remaining best left on-premises.

Kubernetes for everything: Kubernetes recently surpassed Docker as the most talked about container technology. In the future, every data technology will run on Kubernetes. We may not quite get there in 2020, but Kubernetes will continue to see rising adoption as more major vendors base their flagship platforms on it. There are still some kinks to be ironed out, such as issues with persistent storage, but those are currently being addressed with initiatives like BlueK8s. The entire big data community is behind Kubernetes, and its continued domination is assured.

Conversation moves from technologies to use cases: Organizations increasingly care about the use cases a vendor supports rather than the specific data technology that vendor employs. In the past, many organizations thought that in order to leverage big data they had to deploy technology X or technology Y. That's changing dramatically as we move into 2020, with these orgs paying less attention to the tech under the hood and instead focusing on delivering specialized use cases that advance their bottom line.

More mergers and acquisitions: Expect to see more major mergers and acquisitions in big data next year, such as HPE's acquisition of MapR or the Cloudera-Hortonworks merger. This activity is great for customers as it gives them more robust platforms and helps avoid vendor lock-in. Enterprises care very little today about which vendor they're buying from, they just want to make sure the solutions fit their use cases. Personally, I haven't come across a customer who just uses AWS or just uses Cloudera - they all mix and match. The ongoing rollup of these companies is a natural result of this trend.

Big data has hit an inflection point in which data workloads are being used in full production now and organizations are concerned with getting more value from those workloads and supporting new use cases. These trends reflect this evolution and will define the big data conversation in 2020.


About the Author

Kunal Agarwal 

Kunal Agarwal is the CEO and Co-Founder of Unravel Data. He also founded in 2010, a pioneer in personalized shopping recommendations. Prior to this, Kunal he was a business development manager for Oracle products. He has managed enterprise data for companies like United Technologies and Express Scripts. Kunal helped Sun Microsystems evaluate Big Data infrastructure like Sun's Grid Computing Engine. Kunal holds a Bachelors in Computer Engineering from Valparaiso University, Indiana and M.B.A from The Fuqua School of Business, Duke University, NC.

Published Thursday, December 05, 2019 7:30 AM by David Marshall
There are no comments for this post.
To post a comment, you must be a registered user. Registration is free and easy! Sign up now!
<December 2019>