Industry executives and experts share their predictions for 2023.  Read them in this 15th annual VMblog.com series exclusive.

Unstructured Data Management Predictions for 2023: Data Insights and Automation take Center Stage

By Krishna Subramanian, COO, President and Co-founder of Komprise

Complexity reins in the IT infrastructure world, during a time when simplicity and efficiency are paramount. As unstructured data growth shows no sign of abating, IT leaders will need to optimize and modernize their data and storage environment to manage costs, be agile and deliver a data-driven competitive advantage.

Smart strategies will depend upon three things: 1) holistic visibility across silos to plan for cloud data migrations, data modernization and data management; 2) mobility to transparently automate data movement at scale with data governance and security, and 3) value, to index data and facilitate data workflows to achieve better outcomes through AI/ML and data processing in the cloud. Rather than requiring IT to become experts in each new cloud or storage innovation, data management abstracts the infrastructure specifics so IT can focus on managing through policies, and not the intricacies of each system. Data-driven decision-making and automation are the name of the game as the right talent becomes harder to find. Here's our take on what IT infrastructure and storage managers should consider in 2023.

IT managers will be measured on new, business-oriented data services metrics. Data storage teams have traditionally measured infrastructure metrics for capacity and performance such as latency, I/O operations per second (IOPS) and throughput.  But given the massive data growth of unstructured data, data focused metrics are becoming critical as enterprises move away from managing storage to managing data services in hybrid cloud infrastructure. New data management metrics look at usage indicators such as top data owners, percentage of "cold" files which haven't been accessed in over a year, most common file size and type, and financial operations metrics such as storage costs per department, storage costs per vendor per TB, percentage of backups reduced, rate of data growth, chargeback metrics and more.

Cloud data migration complexities require networking skills and network-aware data management: Large-scale unstructured file data migrations to the cloud are problematic for many enterprises because these data sets can be petabytes of data and billions of files. The overlooked component is often the network. Migration issues-such as slow transmissions, data loss and errors--not only derail timelines and add costs to projects but can sour appetites to expand cloud spending.  Planning to avoid networking-related migration issues requires better collaboration between data storage and network teams. Unstructured data management tools that assess the data and configurations for network and security tools are also valuable to speed the planning process and optimize data transfer with WAN acceleration capabilities. 

Demand for simpler cloud data management tools. With cloud infrastructure, IT pros are still managing technology. Even though there is no hardware to manage, the cloud is still complex infrastructure, and the architecture differs from cloud to cloud. Cloud data management tools help by automating processes such as providing analytics on cloud data usage and growth, automating data lifecycle management, policy execution, spend management, data workflow orchestration and reporting. With the growing need for users to run AI/ML and data analytics on unstructured data, these tools are becoming easier for IT generalists to use, giving them the ability to assess their cloud data footprint and design workflows to leverage cloud data services. Given the ongoing labor shortage and skills gap, this will be a hot area for innovation and product development in the coming year. Tools that automate smart data workflows in the cloud without requiring coding or infrastructure expertise will be in high demand.

Industry specific data management will emerge as unique data needs arise.  Data-intensive sectors such as life sciences, financial services, media and entertainment, and the public sector are facing rapid data growth. This is costly and makes it difficult to leverage data for new R&D, customer intelligence, risk management and/or to improve operational processes. Solution providers will need to meet these needs with in-depth understanding of industry specific file types, data management requirements, and research initiatives.  As AI and ML technologies mature, enterprises in data-intensive sectors will require niche tools for managing unique data sets such as machine data from lab instruments, consumer products generating sensor data such as smart home systems and health wearables, along with the vast array of industry-specific manufacturing systems.  Another growing use case is governance and compliance. Storage and infrastructure leaders can help by enabling search and policy automation for identifying and moving PII, IP, legal and other regulated or sensitive

The storage architect/engineer will evolve to incorporate data services.  We'll see more experienced individuals in these roles move on to cloud architect and other engineering roles while IT generalists/junior cloud engineers inherit their responsibilities.  This is a challenging time for IT organizations in a hybrid model as there is still significant NAS expertise needed. Either way, the IT employees managing the storage function will need new skills beyond managing the storage hardware. These individuals must understand the concept of data services-including facilitating secure, reliable governance and access to data and making data searchable and available to business stakeholders for applications such as cloud-based machine learning and data lakes. The new storage architect will frequently analyze and interpret data characteristics, developing data management plans which factor in cost savings strategies and business demands to create new value from data. This individual will interact regularly with departments to create and execute ongoing data management processes and plans.

##

ABOUT THE AUTHOR

Krishna Subramanian, COO, President and Co-founder of Komprise