Virtualization Technology News and Information
Data Dynamics 2015 Prediction: Transformation to Software-Defined Storage - A Word of Caution


Virtualization and Cloud executives share their predictions for 2015.  Read them in this series exclusive.

Contributed article by Piyush Mehta, CEO of Data Dynamics

Transformation to Software-Defined Storage: A Word of Caution

There is a large transformation taking place in the storage marketplace. It is a fundamental shift, with storage companies moving from being proprietary providers of the physical disk to being the software-based management layer that will manage heterogeneous storage devices. This shift is changing the way companies position themselves and is disrupting the marketplace, with new entrants giving a true challenge to the legacy "goliaths" that dominate the marketplace.

The shift of focus to the software layer is a result of the vast amount of data that has been and continues to be generated across all aspects of storage. Specifically, unstructured and big data is seeing storage growth more than double every two years, with the file- and object-based storage market reaching $38 billion by 2017, according to IDC. The growth is seen across all verticals, from financials and pharma to retail and manufacturing. The reliance on data for applications to make intelligence-based decisions makes storage the most essential component of the infrastructure stack.

This level of data growth, combined with its importance, has led customers to look at means of reducing the aggregate cost of storage while ensuring there is no compromise on the performance demands of the applications that call on that storage. The balance of cost and performance has led to enterprises looking for innovative means of managing their storage spend.

An area of focus that has led to the transition we are witnessing is that the underlying disks are becoming more and more robust and stable. With the evolution and price reduction in solid state disks and more scale and stability in traditional SATA drives, performance can be garnered regardless of storage vendor for a large majority of the estate. This realization has led enterprises to a belief that "a disk is a disk is a disk," regardless of the front-end storage vendor, and as such there can be tremendous cost savings gained through moving from proprietary storage vendors providing the underlying disks/storage to a generic disk/storage solution.

In order to leverage generic storage, enterprises require a storage management plane that is able to meet all the operational requirements for storage management while being able to support and make optimal use of the heterogeneous storage beneath it.  In parallel to a shift from traditional proprietary storage silos to generic ones, most enterprises have embarked on a hybrid cloud strategy, leveraging external providers for burst, archival, or primary storage. The reduction in management personnel, floor space, power and cooling savings in the data center leads to cost savings that justify the shift to a hybrid on- and off-premise storage model.  Thus, the software management layer must be able to talk to the cloud providers and be able to integrate the management functionality of those providers.

Although this shift to a pure software world sounds like a great idea, and theoretically is one, it is easier said than done. There are years of operational processes put in place based on legacy storage infrastructure. This web of touchpoints across the enterprise, from links being embedded in a spreadsheet to applications that are hard-coded with specific hardware information, makes an immediate "rip and replace" impossible, as that would be disruptive and potentially detrimental to the business. Therefore, the transition has to be one that allows for utilizing the existing operational processes while overlaying those processes with a centralized management capability.

From the vendor standpoint, creating such intricate software is not simple.  On the front end, it needs to be simple and easy to use, providing for a single pane for everything from provisioning and allocation to ease of data mobility, replication, disaster recovery, backup, and archival management. On the back end, it needs to have API integration with all the various vendors while ensuring supportability across the traditional silos of storage, including block, file, object, Hadoop, and tape-based storage. The API integration is not trivial, as most vendors don't have robust API stacks. Although vendors are starting to mature their API and provide for greater functionality through API versus through the command line, there is still a long ways to go before this is robust and proven.

The other challenge is the ongoing interoperability against new releases by hardware vendors and assurance that the API that functioned across prior versions continues to do so with the new versions. Add to this the complexity of integrating with cloud providers to support a hybrid cloud environment. When you take into the account all the various layers to account for, the integration requires meticulous focus and ongoing maintenance.

Another key aspect to be cognizant of is "vendor lock-in". For years we have seen storage hardware vendors tout the value of virtualization and the ability to be non-disruptive. This virtualization layer never gained the market adoption that was envisioned at inception because it became a "lock-in", whereby the customer is unable to switch from the incumbent vendor to another due to the dependencies created and the lack of visibility to them. This same "lock-in" is a challenge with storage management software. The software must provide transparency as to where exactly the storage is residing and how it is deployed.

This information is vital to ensure that, should you choose to remove the management layer for any reason, you are not blocked from doing so due to the "black hole" of interdependencies on the back end of which you are not aware. Enterprises have experienced tremendous pain when virtualization needs to be removed and have seen the disruption lead to either running the virtualized layer in an end-of-life or unsupported state or going through a massive disruption that in most instances has taken several years to unwind.

There is no doubt that the transition from a traditional silo-based storage infrastructure to one that is centrally managed through intelligent software and can scale to meet today's massive data growth is taking place. The process is one that over the coming years will drive optimization within enterprise infrastructures and provide for an economical means of managing the massive amounts of data, but the transition will be one that requires caution and a tremendous level of planning and impact analysis and be done in phases that ensure the least amount of disruption.

If done correctly, the increase in storage capacity and its corresponding costs will be countered by the efficiencies generated and the business advantages provided by having access to a robust amount of data.


About the Author

Piyush Mehta brings over 19 years of entrepreneurial experience, with the last 16 focused on software and technology services, to his role as CEO of Data Dynamics.

Having founded one company and nurtured another from commercial inception, he had grown both entities to strong growth and poised them for further success. As CEO at SANpulse Technologies, Piyush established alliances and partnerships with key industry leaders within the storage industry, built a strong management team, and developed a culture that inspired and rewarded creativity and unity within the organization. Prior to SANpulse, Piyush founded an IT services firm, Integration International, which he built into a multinational entity and sold in 2006. He also held a senior-level executive position with a multinational manufacturing company. He has extensive global experience in treasury, investor relations, business strategy, acquisitions and divestitures, finance, and operations.
Published Wednesday, October 29, 2014 6:31 AM by David Marshall
@VMblog - (Author's Link) - February 10, 2015 7:00 AM

Once again, how great is it to be a part of the virtualization and cloud industries? 2014 was another banner year, and we witnessed a number of fantastic technologies take shape and skyrocket. And I, along with many industry experts and executives, media

To post a comment, you must be a registered user. Registration is free and easy! Sign up now!
<October 2014>