As AI moves from pilot to production, the bottleneck is shifting from compute to memory. Reasoning models and agents build context faster than GPU memory can hold it, and whatever doesn’t fit gets recomputed, burning GPU time with nothing new to show for it. In this VMblog pre-show Q&A ahead of SC26 in Chicago, Paul Speciale, Chief Strategy Officer and Field CTO at Scality, makes the case that KV cache is now a data infrastructure problem. He points to lab results from August 2026 in which reading a stored context back was 14x faster than recomputing it at 14K tokens and 72x faster at 439K tokens, and in which a shared cache held 8 TB with first-token time staying flat as it grew.
Speciale also discusses how Scality, which launched Scality ADI in May and Scality AI Inference Factory in October, helps customers cope with the memory and accelerator supply crunch. He covers running on standard servers from Dell, HPE, Lenovo and Supermicro, sovereignty and private AI, and where scale-out object storage fits alongside parallel file systems in modern HPC environments. Attendees can find Scality at booth #4959, and can catch Candida Valois presenting “What Production AI Demands from Data Infrastructure” in the Exhibitor Forum on Tuesday, November 17, at 10:30 a.m. CST in Room W194b.
++
VMblog: If you were giving an SC26 attendee a quick overview of your company, what would you say? How has your company evolved since SC25, and what has been the biggest change in your business over the past twelve months?
Paul Speciale: Scality builds software-defined data infrastructure for AI and for the data that feeds it. Our software runs in production at more than 1,000 enterprises in over 70 countries, holds more than 12 exabytes of data, and has been in continuous production since 2009. In research and life sciences, customers include the Leibniz Supercomputing Centre (LRZ) and SeqOIA.
The biggest change this year is where we sit in the AI stack. In May we launched Scality ADI (AI Data Infrastructure), one platform that serves every stage of the AI pipeline, from ingest and training through inference, protection and long-term retention. In October we built on it with Scality AI Inference Factory, an open-code software stack for running AI inference on infrastructure the enterprise owns. That moved the conversation with customers from where AI data lives to what keeps GPUs productive once AI is in production.
VMblog: Your company is participating in SC26 in Chicago. What does that presence look like, and if you were in St. Louis last year, how does it differ from your SC25 participation?
Paul Speciale: We’re in the exhibit hall at booth #4959 all week, and we’re on the program. Candida Valois, our Field CTO and Alliances Leader for the Americas, presents “What Production AI Demands from Data Infrastructure” in the Exhibitor Forum on Tuesday, November 17. I’ll be in Chicago too, meeting with HPC and AI architects. What’s different this year is our focus topic. The SC community has always cared about feeding compute with data. With inference now sitting alongside simulation and training, that problem has reached GPU memory itself, and that’s what we’ll be talking about at the booth and in the session.
VMblog: What key message will SC26 attendees hear from you this year? With AI infrastructure budgets under heavy scrutiny and hardware costs rising, what concrete value proposition can they take back to their CFO and CIO?
Paul Speciale: When AI goes into production, the bottleneck very quickly moves from compute to memory, since reasoning models and agents build context faster than GPU memory can hold it. This is especially true when you have teams of many simultaneous user sessions, each of whom want to retain a long context. In our lab, one long session with a 438,764-token context produced 189 GB of KV cache, which will not fit in the GPU cache (HBM), so it requires external storage. Whatever doesn’t fit gets recomputed, and recomputing is GPU time you’ve paid for and get nothing new from.
For the CFO, the value is super clear and direct: put the GPUs you already own to work. Reading a stored context back instead of recomputing it was 14x faster at 14K tokens (Gemma-3 27B) and 72x faster at 439K tokens (GLM-5.2) in our August 2026 lab tests. That matters because GPUs already sit idle most of the time, for example Cast AI measured average GPU utilization of just 5% across tens of thousands of Kubernetes clusters on AWS, GCP and Azure (2026 State of Kubernetes Optimization Report), and 79% of organizations running AI in production told a VentureBeat Intelligence survey their GPUs are 50% of the time or less (July–September 2026).
On top of that, Scality runs on standard servers from Dell, HPE, Lenovo and Supermicro, and performance and capacity scale independently, so you buy flash only where a workload proves it needs it. For the CIO, the value is one governed dataset that serves many teams and frameworks without a copy for every tool, with cyber resilience built in rather than bolted on.
VMblog: Do you have speaking sessions, panels, Birds of a Feather meetings, workshop appearances, or paper tie-ins at SC26? What topics will you address, and how can attendees join?
Paul Speciale: Yes, we have the session mentioned above (Candida Valois presents “What Production AI Demands from Data Infrastructure” in the Exhibitor Forum (AI and Data Analytics I) on Tuesday, November 17, in Room W194b at 10:30-11am CST. She’ll cover why AI projects that worked in the pilot stall in production, the six AI workloads that hit one dataset at the same time (KV cache, RAG, vector search, agents, multimodal models and knowledge systems), and four practical moves: reduce unnecessary data movement, keep the GPUs fed, eliminate disconnected storage silos, and read context back instead of rebuilding it. Candida will provide measured lab results and patterns from production deployments. Anyone who wants a deeper discussion afterwards can find us at booth #4959.
VMblog: The memory and accelerator supply crunch has pushed up prices and stretched lead times for DRAM, HBM, and NAND. How are you helping customers plan and get more from constrained hardware, whether through flexible configurations, software efficiency, disaggregation, or extending the life of existing systems?
Paul Speciale: Our answer starts with the fact that Scality’s solutions are software. Customers run it on their own choice of standard servers from Dell, HPE, Lenovo and Supermicro, so they buy what they can get, when they can get it, and they’re not waiting on one vendor’s appliance. Clusters grow a server at a time, so customers add capacity when they need it rather than buying years ahead in a constrained market.
Disaggregation is really the bigger lever for efficiency of hardware. When HBM is scarce, the worst thing to do with it is fill it with context you’ll throw away and recompute. Scality ADI acts as a shared KV cache that every GPU in the cluster can read. In our lab it held 8 TB of KV cache, more than 80x one GPU’s memory, with first-token time flat as the cache grew, and the storage tier was five servers a few generations old. Model weights are stored once, and any GPU can load any model. That significantly stretches the HBM and DRAM you already have.
VMblog: With a Chinese system taking the No. 1 spot on the TOP500 and nations investing in domestic compute stacks, sovereignty and export controls are shaping procurement. How do sovereign AI, data residency, and geopolitical risk influence your product design and go-to-market, and how do you support on-prem, cloud, neocloud, and hybrid deployments?
Paul Speciale: Sovereignty is built into how we design and sell, not added on. Scality is software that runs on infrastructure the customer owns and operates, in its own data centers, under its own jurisdiction. The same platform serves on-prem, hybrid, service provider and neocloud deployments, and Scality AI Inference Factory is available as a software license or as a fully managed service.
For AI, this is where demand is heading. In a 2026 Freeform Dynamics study, 81% of 504 medium to large enterprises active with private AI said private AI on infrastructure they control is critical to their success. When inference runs on your own infrastructure, the codebase, retrieved documents and tool outputs an agent sends stay inside your perimeter, and nobody outside your company has any ability to switch it off. We support that with open-weight models and an open-code stack, so customers aren’t trading one dependency for another.
VMblog: With memory scarce and expensive, and modern AI leaning heavily on KV cache, checkpointing, and massive datasets, what innovations in memory tiering, storage orchestration, or interconnect (Ethernet, InfiniBand, optical) are you unveiling to keep costly accelerators fed?
Paul Speciale: We treat KV cache as a data infrastructure problem. Today, each GPU server typically keeps its own cache on local drives. Only that server’s GPUs can read it, a returning session has to find its way back to the same server, and when the context no longer fits, it’s dropped and recomputed.
Scality ADI replaces that with one shared KV cache and model store with access for all GPUs in the whole cluster over RDMA. In our August 2026 lab tests, reading a context back was 14x faster than recomputing it at 14K tokens and 72x faster at 439K tokens, and the cache held 8 TB with first-token time flat as it grew. A shared cache also changes the serving architecture: because any GPU can pick up any session, prefill and decode can run on separate GPU pools that scale independently, which is how Scality AI Inference Factory serves models.
The same platform keeps training data, checkpoints and long-lived datasets on dense capacity, so the fast tier is reserved for the workloads that need it.
VMblog: SC is a parallel file system crowd. Where does object storage fit in a modern supercomputer, and why should HPC architects care?
Paul Speciale: Parallel file systems are excellent at what they do: feeding a running job at full speed. But most of the data an HPC center holds isn’t in a running job. It’s input datasets, results, checkpoints, model weights and years of research output that has to stay durable, governed and reachable. Keeping all of that on the performance tier is expensive (especially in all-flash), and that’s where scale-out object storage belongs, as the capacity tier underneath, and still offering the performance to deliver massive throughputs on both flash and HDD based servers.
This is also where the industry is converging. We partner with WEKA on a jointly validated solution that pairs WEKA NeuralMesh for active AI and HPC data with Scality RING as the cost-efficient object tier. For example, Leibniz Supercomputing Centre (LRZ) is a Scality customer. And as AI moves into scientific workflows, the capacity tier increasingly has to serve AI too, as a data lake for training, a model store and a shared KV cache for inference. That’s the architecture we’ll be discussing with HPC teams at booth #4959.
##
