Industry executives and experts share their predictions for 2023.  Read them in this 15th annual VMblog.com series exclusive.

Rethinking Traditional Data Management for Scale, Performance

By Adi Gelvan, co-founder and CEO, Speedb

The world revolves around data, yet in 2022 it became apparent that existing database architectures are based on many components that were not built for today's massive, hyperscale data operations. As a result businesses are changing how they approach database management, and how they architect the AI, ML, and analytics applications that have become increasingly strategic. In 2023 we will see an accelerated pace of updating many fundamental constructs to support better end user performance and eliminate trade-offs between capacity, scale, and performance.

We predict that in 2023: 

1. Metadata will continue to play a bigger role in the data management space and enterprises will start prioritizing a fix. Ten years ago, the typical ratio between data and metadata was 1,000:1. Today, the ratio is often more like 1:10 when the object is small. The situation will only get worse as the amount of unstructured data continues to explode. We see today databases struggling with large capacities and companies are failing to provide the same SLA for data intensive applications. The fix is often a different storage engine, an embedded Key Value Store (KVS) that sorts and indexes data. It can be installed either between the application and the storage or, increasingly, as a software layer within the application to execute different activities on live data while in transit. New data engines have emerged in answer to the demands of modern applications to stay ahead of data growth. Next-generation data engines can be a key enabler when low-latency, data-intensive workloads require significant scalability and performance, as is common with metadata.


2. Unstructured data volume growth will force the rethinking of traditional databases and the way they interact with enterprise applications. Between 80% and 90% of data will be unstructured by 2025 - much of it generated by 55.7 billion connected devices worldwide, according to multiple analyst reports. Because unstructured data can't be effectively stored in traditional column-row databases, IT needs NoSQL or key value store technologies to be able to cope with this phenomena. The issue affects every major industry and every major enterprise in the world today, and it will only get more critical. VCs are likewise funding database-related innovations that can better meet the present and future data processing needs of modern enterprises.


3. More enterprises will embrace open source software & products despite fearmongering among the part of large vendors. Open source security issues in the supply chain have given open source a bad rap, when in fact 97% of apps and 90% of businesses use open source to some extent, according to a GitHub survey and the OpenUK 2021 State of Open report.  Open source is far from going away, and flourishing developer communities will continue to emerge around needed tools and applications, making them better, strong, and more widely adopted.


4. Big tech firm layoffs will not ease the hiring crisis among the still vibrant startup ecosystem. We, and most startups we know, can't find and fill positions fast enough as modern applications require a rethink to traditional IT architectures and need employees with a different mindset. These are people who are not looking to work for a big company with job security, but who are enthusiastic about creating change and taking risks while doing that.


5. The shift-left trend will continue, where software developers start testing performance far earlier, sometimes before code is even developed. The developer community will continue to be a rising force in software and platform adoption of companies, not just the IT decision-makers - and key factors in speeding time to business value.
   

It's well past the time when businesses have needed to change how they approach database management. By adopting new solutions designed for modern challenges, not ones that are built on the designs of the past, IT teams will enable the sizable efficiency and productivity gains that they have been promised - gains that will deliver a genuine competitive edge.

##

ABOUT THE AUTHOR

Adi Gelvan is co-founder and CEO of Speedb, a data management startup that provides a drop-in replacement for RocksDB embedded storage engine, both an enterprise and open source editions. A former IT infrastructure manager with over two decades of management, commercialization and executive sales positions, Adi specializes in leading global software technology companies like Infinidat and SQream to outstanding growth. He holds a double academic degree in mathematics & computer science.