Virtualization Technology News and Information
Adding State to Containers Needs Perspective

By Scott Morrison, CTO, NetApp

Lately I've been spending my free time reading about the birth of Cubism in Paris, just prior to the First World War. Two things in this story stood out for me. The first is how revolution is an iterative process which only appears spontaneous when viewed from the other side. This should come as no surprise to anyone who has lived through a few waves in tech industry. The second is how Picasso and Braque-the original cubists -insisted on making a complete break from the past. To move forward, they needed to eliminate hundreds of years of received wisdom.

Not all revolutions can afford the luxury of rejecting everything that came before, nor should they. Often, we need the past to inform and extend our path to the future. This is especially true in technology, where we have a terrible habit of running after the new and shiny while forgetting valuable lessons of the past.

Containerization is a revolution that's still playing out. Docker and Kubernetes aren't simply a better way of doing virtualization; rather, these technologies offer an entirely different way of building and running application. Lightweight, ephemeral containers, coming and going in response to need, is a radical idea that suddenly makes perfect sense. The rallying cry that containers should be cattle, not pets, neatly sums up an entirely new zeitgeist.

But real world challenges remain. Containers are stateless, and statelessness can only get us so far. Eventually, all non-trivial applications need to connect to some kind of persistent state, and this means non-volatile storage. The challenge for this new paradigm, as applications began to scale to the hundreds and even thousands of container instances, is how to efficiently connect static storage systems to a swarm of containers that come and go in the blink of an eye.

Fortunately, solutions have emerged from both commercial vendors and the open-source community. These offerings create a strong separation between the processing layer and the storage layer, emphasizing how distinct these two worlds really are. The processing layer is where the revolution is happening. It's challenging and uncompromising, and the technology is still moving very fast. The storage layer, in contrast, is where we need to harness the mature thinking and tooling of the enterprise. This isn't something to abandon in the headlong rush to modernity, as it offers so much value going forward.

If you are going to buy into a stateless, ephemeral world up front, you need to build on a solid foundation. Rock solid predictability and reliability in the storage layer is the right place to start. It's not enough to sit on some disks and hope for the best. Drives fail; power goes out; data is corrupted; systems get compromised. A mature data management solution is designed to handle the grim realities of an imperfect world.

What are the hallmarks of mature data management? Mature data management means data protection, to ensure that backups are regular and reliable. It means snapshots to version data sets in time. It means mirroring, to create development and test images or to setup blue/green deployments. It means migrating data safely to other systems-across the data center, across town, or to and from the cloud. It means protection from ransomware, high storage efficiency, de-duplication and compression, encryption, SLA guarantees, and a host of features that have evolved to support mission critical workloads.

These are attributes we take for granted in enterprise storage. But we must demand the same robustness and completeness everywhere we deploy modern, container-based applications-whether these are on premise or in running the cloud. As application designers, we need to recognize this difference in expectations between the compute and storage layers, and leverage the best practice for each.

This is the other side to revolution, and fortunately, this is how the containerization revolution is playing out. Like Cubism, the container revolution should change our perspective on what is already familiar and help us to see the value in ordinary things.




K. Scott Morrison is the CTO of NetApp’s Hybrid Cloud Engineering group. He is responsible for providing vision and technical leadership to a global team of engineering professionals building cutting edge products for cloud, data and storage management.  

Scott is a passionate, entertaining and highly sought-after keynote speaker at technical conferences. His quotes appear regularly across the media, including the New York Times, the Wall Street Journal and CNN. Scott holds 14 US patents, and has over 75 publications including book chapters, magazine articles, and papers in medical, physics, and engineering journals.

Published Tuesday, August 09, 2022 7:32 AM by David Marshall
Filed under: ,
There are no comments for this post.
To post a comment, you must be a registered user. Registration is free and easy! Sign up now!
<August 2022>