PEAK:AIO unveiled the first dedicated solution to unify KVCache acceleration and GPU memory expansion for large-scale AI workloads, including inference, agentic systems, and model creation.
As AI workloads evolve beyond static prompts into dynamic context streams, model creation pipelines, and long-running agents, infrastructure must evolve too.
“Whether you are deploying agents that think across sessions or scaling toward million-token context windows, where memory demands can exceed 500GB per model, this appliance makes it possible by treating token history as memory, not storage,” said Eyal Lemberger, Chief AI Strategist and Co-Founder of PEAK:AIO “It is time for memory to scale like compute has.”
As transformer models grow in size and context, AI pipelines face two critical limitations: KVCache inefficiency and GPU memory saturation. Until now, vendors have retrofitted legacy storage stacks or overextended NVMe to delay the inevitable. PEAK:AIO’s new 1U Token Memory Platform changes that by building for memory, not files.
The First Token-Centric Architecture Built for Scalable AI
Powered by CXL memory and integrated with Gen5 NVMe and GPUDirect RDMA, PEAK:AIO’s platform delivers up to 150 GB/sec sustained throughput with sub-5 microsecond latency. It enables:
- KVCache reuse across sessions, models, and nodes
- Context-window expansion for longer LLM history
- GPU memory offload via true CXL tiering
- Ultra-low latency access using RDMA over NVMe-oF
This is the first platform that treats token memory as infrastructure rather than storage, allowing teams to cache token history, attention maps, and streaming data at memory-class latency.
Unlike passive NVMe-based storage, PEAK:AIO’s architecture aligns directly with NVIDIA’s KVCache reuse and memory reclaim models. This provides plug-in support for teams building on TensorRT-LLM or Triton, accelerating inference with minimal integration effort. By harnessing true CXL memory-class performance, it delivers what others cannot: token memory that behaves like RAM, not files.
“While others are bending file systems to act like memory, we built infrastructure that behaves like memory, because that is what modern AI needs,” continued Lemberger. “At scale, it is not about saving files; it is about keeping every token accessible in microseconds. That is a memory problem, and we solved it at embracing the latest silicon layer.”
The fully software-defined solution utilizes off-the-shelf servers is expected to enter production by Q3.
