SPIN Unprocessed October 28, 2026 ai_technology research
Can Language Models Learn While They Think? Testing Gradient Adaptation Across Architectures
View original on arxiv.orgOverview
arXiv:2610.33025v1 Announce Type: new Abstract: Letting a language model take gradient steps on the text it is reading, known as dynamic evaluation, lowers its loss on the rest of the document. Its behaviour on hybrid linear-attention architectures, on diffusion language models, and on standard long-context benchmarks has not been measured. We measure it. With one SGD step per 128-token chunk on the MLP output projections of the last quarter of blocks, no training, and a 2k-token window, softmax
SpinGraph analysis pending — check back after processing.
Ask AI about this story
Opens with the SpinGraph .md URL and structured context — one click, prompt included.
More from arXiv Machine Learning
View all →- Augmentation-Guided Adversarial Training via Local Stability-Based Sample Selection
- MLIPDiff: Guiding Denoising Diffusion Models for Structure-Based Drug Design via Machine-Learned Interatomic Potentials
- When Does Pre-Trained Capacity Help Causal Learning? From Scalar Prediction to Counterfactual Post-Training
- Tree Depth and Sample-Quality-Associated Features: Two Hidden Factors in Boruta Selection for RNA-Seq Data Classification
- All-in-One: A Composable Attack on LLM Transformations
- Individual Fairness via Local Predictive Consistency
Markdown (.md) · JSON-LD schema (.json) · Machine-readable for AI & GEO