Search Spins

Routing Without Training: Controllable-Ratio LLM Offloading via Reliability Gating

Researchers propose CARGO, a training-free method for routing LLM inference tasks between local and cloud models using the local model's self-consistency signal, enabling controllable offloading ratios without additional training.