SPIN Unprocessed October 22, 2026 ai_technology research
To Keep or Not to Keep: Learning KV Cache Retention in Disaggregated LLM Serving Systems
View original on arxiv.orgOverview
arXiv:2610.28637v1 Announce Type: new Abstract: Disaggregated \textsc{LLM} serving separates prefill and decode across node pools, placing a network fabric between \textsc{KV}-cache computation and consumption. This breaks a key assumption of classical cache policies: a miss no longer incurs only local recomputation, but also network transfer to the decode node. Likewise, admitting a block to the global cache requires an additional transfer, so poor retention decisions waste both memory and band
SpinGraph analysis pending — check back after processing.
Ask AI about this story
Opens with the SpinGraph .md URL and structured context — one click, prompt included.
More from arXiv Machine Learning
View all →- AcoustiPrint: Physics-Guided Neural Design of Manufacturable Acoustic Structures
- Escaping the Phylogenetic Trap: Persistent Homology Enables Cross-Species Ion Channel Classification via Multimodal Graph Neural Networks
- Bayesian Fixed-Confidence Best-Arm Identification with Abstention
- Probing and Steering for Safety-Relevant Behaviours in Large Language Models
- CTRL-FM: Inference-Time Structural Control for Flow-Based Virtual Sensing under Long-Horizon Node Outages
- Spike-Aware Gradient-Based Mixed-Precision Quantization of Spiking Neural Networks for Vision Applications
Markdown (.md) · JSON-LD schema (.json) · Machine-readable for AI & GEO