Industry executives and experts share their predictions for 2021. Read them in this 13th annual VMblog.com series exclusive.
2021 – The Year of the Data Time-Share
By Thomas LaRock, Head Geek, SolarWinds
For the past few years, the phrase “Data Estate” has entered buzzword bingo cards everywhere. The phrase is meant to describe the infrastructure, systems, and processes a company uses to manage their data. Though it’s not the worst data buzzword available (hello, Data Fabric), Data Estate is misleading. Three trends are combining to turn your Data Estate into a Data Time-Share.
Data Streams to the Cloud
The coming year will see an acceleration in data moving to the cloud. Corporations recognize the benefits of leveraging cloud services, but they often lack the resources necessary for successful migrations. Enter companies such as VMware and Cisco, who are partnering with AWS® and Microsoft® Azure® solutions to help corporations migrate and manage their data as it journeys upwards.
Whether you call it “lift and shift” or “move and improve,” this trend has given rise to the phrase “Data Estate.” This phrase is meant to temper the reality of sprawl. Rather than think about how your data is everywhere, it’s easier to picture yourself drinking a mint julep on the patio, overlooking your vast Data Estate.
But we all know an estate is built upon a foundation. And the foundation for any Data Estate is made up of the data pipelines needing to be rebuilt, upgraded, or installed as new.
Data Engineering Roles Grow
When you mention the job title data engineer, most people think about companies such as Netflix, where millions of data points are ingested and processed daily. But companies migrating their systems to the cloud need data engineers, too. After all, someone needs to build and maintain the foundation of data pipelines.
Data engineering blends a bit of database administration with database development. With AWS and Azure automating away many traditional administration tasks, your current staff of database administrators and developers will be able to slowly take on the role of data engineer. They already have the skills and the data domain knowledge, so this is the next logical progression for those careers to take on the role of a data janitor engineer.
Of course, this does leave one last piece-someone needs to monitor your Data Estate.
Monitoring Your Data Estate
Monitoring your Data Estate can be tricky, considering you don’t own the hardware in the cloud. Therefore, you won’t have all the shiny metrics you were used to seeing. Having less data can be a good thing, as you it allows you to focus on the most important metrics. And with cloud, the most important metric is simple: cost.
It’s no longer enough to know whether a service is up, down, or just slow. The CXOs want to know the costs associated with cloud services, right down to the cost of running specific queries. “How much did a query cost, and why was it executed millions of times a day?” are questions needing answers.
With this additional metric comes the realization that most of your Data Estate is being managed by someone else. You simply pay your bill and get all the benefits of having a data center you won’t ever visit.
This is exactly how vacation rentals-or time-shares-work, too. In real estate, a time-share is a property arrangement where you share the cost with others to guarantee time at the property. Well, in AWS and Azure, we share costs to get compute time (and storage) in the cloud as needed. In short, your Data Estate is really a Data Time-Share, located right next to a Data Lake.
Summary
Unlike a vacation time-share, it’s easier to get out of the cloud. And maintenance fees in the cloud don’t increase at the same pace as vacation time-shares.
But make no mistake about it. Your Data Estate is located on a Data Lake, fed by a Data Stream also powering the Data Factory, which stores their output in a Data Warehouse built with Data Bricks so folks can shop at the local Data Mart for all their data needs, big and small, paid for with Data Cache… This is what you get with your Data Time-Share.
Luckily, you don’t have to claim your week a year in advance.
##
About the Author
Thomas LaRock, Head Geek, SolarWinds
Thomas has over 20 years of experience in roles including programmer, developer, analyst, and DBA. He enjoys working with data-probably too much to be healthy.
