Virtualization Technology News and Information
Gridstore 2015 Predictions: Erasure Coding will be Recognized in 2015 as Superior to Replicas


Virtualization and Cloud executives share their predictions for 2015.  Read them in this series exclusive.

Contributed article by Kelly Murphy, Founder and CTO of Gridstore

Erasure Coding will be Recognized in 2015 as Superior to Replicas

When a company deploys a private cloud, efficiency, performance and of course, cost are of utmost importance.

For the ultimate in performance we’re seeing all-flash being used more and more in data centers, but only recently are we beginning to see them in hyper-converged systems.  One of the inhibitors to broad adoption of all-flash hyper-converged systems is the inefficient method being used to protect against faults that results in data being written three times, thereby wasting a very expensive resource.

We’re talking about systems that use replication between nodes to provide protection from one or two faults.  3-way Replication (3-way replica is the original data plus two full replicas allowing 2 faults without data loss or disruption) provides a fault tolerance level of two and is recommended when using commodity hardware components that do not have redundancy built into the individual systems.  Yes, all vendors allow you to use a replication factor of just one – but this is living close to the edge.  Generally accepted standards in IT are for a replication factor of at least 2.

But when you’re talking about the use of flash, now it becomes a highly inefficient and expensive way to provide for data protection.  With a 3-way replica – you are literally burning 2/3rds (66%) of one of the most expensive resources in the infrastructure. Worse, not only is it an extraordinary waste of a very expensive resource, but flash also wears out with the number of writes.  Now with this 3-way replication model, each write I/O is amplified 3X.  This costs CPU cycles, network traffic and most importantly wears the Flash resource 3X faster.  The result is you either pay up now by under-provisioning flash to account for the write amplification or pay later by replacing Flash that wore out long before its sell-by date.  So not only do you throw away 66% of that Flash – you need to replace it even faster than you thought you would.

Additionally, when replication is used in the context of Hyper-Converged Infrastructures, the costs literally explode.  With Hyper-Converged Infrastructure (all layers of infrastructure converged into a single component) – you are literally replicating the entire infrastructure stack 3X.  Storage – Network – Compute and a full stack of software licenses on every one of these nodes.  Replicating your entire infrastructure 3X makes no sense – it is an extraordinary cost and excessive waste.

The better answer here, and one that will be recognized as such in 2015 as all-flash hyper-converged systems become widely adopted, is Erasure Encoding. Erasure Encoding eliminates the need for three-way replicas and its inefficient use of a very expensive resource, namely flash.  Instead, the data is broken into chunks, expanded and encoded and stored across nodes.  Redundant data can be reconstructed in the event of a loss.  This mathematical process is fundamental to a good design for an all-flash hyper-converged system, allowing for efficient and cost-effectiveness not available otherwise.


About the Author

As a serial entrepreneur with a track record of bringing disruptive technologies to market, Kelly Murphy brings 15 years CEO experience with disruptive venture backed software companies. In 1998, almost a decade before the cloud became popular, Murphy founded Marrakech, the first software company that offered on-demand procurement and supply chain systems to over 30,000 trading partners including some of the world's largest retailers, consumer food producers, packaging companies and utilities. After selling Marrakech in 2007, he turned his sights onto what was his largest obstacle in growing his previous business - storage. In 2009, Murphy founded Gridstore - a pioneer of software-defined storage that is set to disrupt the traditional storage industry. Currently, he serves on Gridstore's Board of Directors and is also the Chief Technology Officer.  

Published Monday, November 24, 2014 6:28 AM by David Marshall
@VMblog - (Author's Link) - February 10, 2015 6:59 AM

Once again, how great is it to be a part of the virtualization and cloud industries? 2014 was another banner year, and we witnessed a number of fantastic technologies take shape and skyrocket. And I, along with many industry experts and executives, media

To post a comment, you must be a registered user. Registration is free and easy! Sign up now!
<November 2014>