Industry executives and experts share their predictions for 2020. Read them in this 12th annual VMblog.com series exclusive.
By Kunal Agarwal, CEO of
Unravel Data
Kubernetes and SREs reshape approach to data workloads in 2020, while organizations get a reality check on NoOps
The conversation around big data
has changed quite a bit over the past few years. These days, fewer people even
use the term "big data" and instead talk about how this data is being put to
use, particularly in artificial intelligence and machine learning applications.
This reflects a positive shift: organizations have deployed big data and are
now looking at how to get the most real-world value from it.
This evolution will be further
exemplified in 2020 as enterprises focus less on specific big data technologies
and more on use cases. In addition, the conversation around modern data
workloads will be impacted by the continued rise of Kubernetes, increasing use
of SREs and a changing POV on NoOps.
Next year will be dominated by
these data trends:
SREs change big data: Enterprises have been using SREs (site reliability
engineers) for a few years to ensure their websites function properly by
developing automated fixes for any web errors that may occur. This approach
to automated troubleshooting is now moving to data apps, with DataOps teams
both adopting the general approach of SREs as well as hiring specialized SREs
to solve data analytics problems. The role of data scientist has made lots
of buzz as one of the most in-demand jobs over the past few years. Expect more
major enterprises, especially web companies, to increasingly hire SREs for data
apps and to create SRE positions that will be focused on specific technologies
like Hadoop, Spark, Kafka, etc.
NoOps falls short: The concept of NoOps gained some steam in 2019. Using AI
to automate does make operations far more efficient, but the notion that
organizations can leverage cloud services and AI to eliminate all IT operations
is a pipe dream. The reality is that you need DevOps and DataOps in the
cloud just like you do on-premises. The cloud is an ideal destination for many
workloads, especially data workloads, but the operational challenges from
on-premises deployments don't just disappear when you get to cloud. In
fact, new challenges emerge, such as re-configuring apps for improved
performance in the cloud and monitoring workloads for overuse (and increased
costs). There are AI tools that significantly simplify these efforts, but
organizations will need human operations teams to leverage those tools
correctly. The cloud is great, but there need to be guard rails in place to
ensure it's delivering on cost and performance. The NoOps trend reminds me of a
time 5-8 years ago when people thought the cloud be the panacea for everything.
Instead, it's clear that a hybrid model has won out with the cloud becoming
ideal for many apps but others remaining best left on-premises.
Kubernetes for everything: Kubernetes recently surpassed Docker as the most talked
about container technology. In the future, every data technology will run on
Kubernetes. We may not quite get there in 2020, but Kubernetes will
continue to see rising adoption as more major vendors base their flagship
platforms on it. There are still some kinks to be ironed out, such as issues
with persistent storage, but those are currently being addressed with
initiatives like BlueK8s. The entire big data community is behind Kubernetes,
and its continued domination is assured.
Conversation moves from
technologies to use cases:
Organizations increasingly care about the use cases a vendor supports rather
than the specific data technology that vendor employs. In the past, many
organizations thought that in order to leverage big data they had to deploy
technology X or technology Y. That's changing dramatically as we move into
2020, with these orgs paying less attention to the tech under the hood and
instead focusing on delivering specialized use cases that advance their bottom
line.
More mergers and
acquisitions: Expect to see more major
mergers and acquisitions in big data next year, such as HPE's acquisition of
MapR or the Cloudera-Hortonworks merger. This activity is great for customers
as it gives them more robust platforms and helps avoid vendor lock-in. Enterprises
care very little today about which vendor they're buying from, they just want
to make sure the solutions fit their use cases. Personally, I haven't come
across a customer who just uses AWS or just uses Cloudera - they all mix and
match. The ongoing rollup of these companies is a natural result of this
trend.
Big data has hit an inflection point in which data workloads
are being used in full production now and organizations are concerned with
getting more value from those workloads and supporting new use cases. These
trends reflect this evolution and will define the big data conversation in
2020.
##
About the Author
Kunal Agarwal is the CEO and
Co-Founder of Unravel Data. He also founded Yuuze.com in 2010, a pioneer in
personalized shopping recommendations. Prior to this, Kunal he was a business
development manager for Oracle products. He has managed enterprise data for
companies like United Technologies and Express Scripts. Kunal helped Sun
Microsystems evaluate Big Data infrastructure like Sun's Grid Computing Engine.
Kunal holds a Bachelors in Computer Engineering from Valparaiso University,
Indiana and M.B.A from The Fuqua School of Business, Duke University, NC.