Industry executives and experts share their predictions for 2020. Read them in this 12th annual VMblog.com series exclusive.
By Andy Watson, CTO, WekaIO
Storage Class Memory, Redefining "Tier 0"
"The faster you go, the shorter you are."
- Albert Einstein
My prediction for 2020 is
that we'll be able to go a whole lot faster than ever before, at least in terms
of data access. So, given that Einstein was right about so many things, maybe
we should prepare ourselves to get a great deal shorter.
In 2020, persistent memory
will begin to deliver on a long-elusive promise. True SCM (Storage Class
Memory) has been bandied about for years, with nothing but trickle-down limited-functionality
Intel® OptaneTM placeholders available on PCIe like a teaser for a movie with no
fixed release date. But ever since at long last Intel delivered Optane DC
Persistent Memory in mid-2019, the industry's long-held breath could finally be
exhaled.
Now we can experience
Optane-based data access with latencies below 1 microsecond. (Actually, down
around 0.35 microseconds.) That's about 1000x faster than the sub-millisecond
latencies quoted for ordinary NAND-based flash storage alternatives, and 20x faster than a PCI-based
Optane SSD.
Some of you might be
getting stuck on my singling out Intel's Optane. Well, okay, it won't always be only Optane;
but for now, Intel is shipping and its rivals are lagging. How long will it
take for Toshiba's XL-Flash, Samsung's Z-SSD, or any other contender to evolve
into a meaningful frontside-bus alternative? Realistically, the alternatives
still appear to be more hypothetical than realistic at this stage. But Intel's
Optane is available, it works, and it's compatible with Intel's Cascade Lake
CPUs.
Some readers may be aware of
other technology contenders in the SCM race. MRAM (Magneto-resistive Random
Access Memory) has been around for more than 10 years and is both faster than
Optane and more durable (capable of a greater number of write cycles before
wearing out). But its storage capacities are tiny by comparison, making it
irrelevant to the sea change I see coming in 2020. And although other emerging
technologies such as ReRAM (Resistive RAM) and PCRAM (Phase Change RAM) also
have potential, none of the alternatives have thusfar yielded anything commercially
relevant. (N.B., Optane is based on technology related to PCRAM.)
Regardless of the way it's
implemented at the chip level, for years we've been warned that SCM will be
useless until somebody with a monstrously big brain figures out how to make
"Memory-Centric Computing" happen. Apparently, that's not the case. Companies
like MemVerge, Levyx, and others are already making their software available to
customers to explore how to use SCM. They're making it possible to run existing
applications on SCM without any code
changes whether harnessing this new sub-microsecond layer as extended
memory or while accessing it as insanely fast persistent storage.
My prediction for 2020 is
that next-generation software will move beyond the beta-release stage to enable
enterprise customers to unlock tremendous business value with SCM - without
waiting for major revisions to the von Neumann computing model. The monstrously
big brains are welcome to continue working on memory-centric computing, but
there's no reason to wait.
This will take two forms:
- using Optane to
extend the available memory footprint into the hundreds of terabytes (via
clustering); and
- using the
persistent memory capability of Optane to offer data storage with unprecedented
sub-microsecond latency characteristics.
In-Memory Databases like
SAP Hana, Apache Ignite, Kinetica, and even good old-fashioned Memcached - or
any other memory-intensive applications - will enjoy hitherto-impossibly-large
footprints to do hitherto-impossible things at unprecedented scale.
Checkpointing long-running tasks will become virtually instantaneous (as will
recovering from those checkpoints). So-called "big data" analytics will be
accelerated dramatically even while running at larger scale than has ever been previously
possible. All of these developments will make it more realistic and
cost-effective to apply AI to a wider range of practical problems.
As SCM is adopted and
deployed in production, you can bet that the performance expectations for the
next layer adjacent to this new "Tier Zero" will be difficult to satisfy.
Although dozens or hundreds of terabytes of SCM might be deployed to hold the
working set of data, the size of an entire Data Lake nowadays is typically
measured in petabytes. At WekaIO, we anticipate that the performance demands
associated with refreshing the SCM-based working set will be daunting. Luckily,
our flash-native Tier One solution is
widely known as "the world's fastest filesystem" and so we are confident that
our filesystem will be the obvious choice as the Tier One storage layer associated with SCM-based Tier Zero.
##
About the Author
Andy Watson has focused on the customer experience of high-performance
file storage since 1991. After joining NetApp in 1995 (serving as its second
CTO there 1998-2007), Andy led cross-functional teams to integrate
multiprotocol file server appliances into increasingly complex environments as
diverse as Oracle dbms and Rocketmail (the precursor to all webmail), or
spanning the spectrum from SAP production environments to seismic analysis
applications for the energy sector. Most recently Andy served as CTO at Minio,
a cloud-native open source object storage software company, where he confronted
cloud-vs-on-premises decisions and the challenges of digital transformation.
As CTO at WekaIO, Andy actively engages with customers and partners,
monitoring emerging trends and bringing actionable feedback to Engineering and
Product Management.
Andy has a Bachelor's degree in Physics from Bard College.