Nvidia’s new technique cuts LLM reasoning costs by 8x without losing accuracy

Nvidia researchers developed dynamic memory sparsification (DMS), a technique that compresses the KV cache in large language models by up to 8x while maintaining reasoning accuracy — and it can be ...
WREG

KV Cache Offload to SSDs Will Produce Over $10 Billion in Revenue by 2030

Revolutionary Memory Management Technology Set to Transform AI Infrastructure Market as Demand for Efficient Large Language Model Deployment Soars. Model output requirements are soaring past the ...
Semiconductor Engineering

Dynamic KV Cache Scheduling in Heterogeneous Memory Systems for LLM Inference (Rensselaer Polytechnic Institute, IBM)

A new technical paper titled “Accelerating LLM Inference via Dynamic KV Cache Placement in Heterogeneous Memory System” was published by researchers at Rensselaer Polytechnic Institute and IBM. “Large ...
insideHPC

DDN Takes on GPU Waste with KV Cache Performance for AI Reasoning

CHATSWORTH, Calif. — July 18, 2025 DDN today unveiled performance benchmarks that the company said demonstrates how its AI-optimized DDN Infinia platform eliminates GPU waste and delivers the fastest ...

The $20 Billion Bet On Inference: What Every AI Infrastructure Team Needs To Get Right

Every ChatGPT query, every AI agent action, every generated video is based on inference. Training a model is a one-time ...
The Next Platform

KV cache

Generative AI is arguably the most complex application that humankind has ever created, and the math behind it is incredibly complex even if the results are simple enough to understand. … ...