SPIN Unprocessed October 27, 2026 ai_technology community
Teachers other than EMA for distillation ? [R]
View original on reddit.comOverview
Hi guys, Self-distillation methods, see Self-Distillation Fine Tuning SDFT, usually use an EMA (Exponential Moving Average) teacher to get synced and stable targets. I wanted to ask the community for research papers/ works that consider different types of teachers that also hold these properties (they eventually also improve along with student) but are stable (prevent student from drifting). any links to repos/ papers would be appreciated. thanks! submitted by /u/ifaposto [link]  
SpinGraph analysis pending — check back after processing.
Ask AI about this story
Opens with the SpinGraph .md URL and structured context — one click, prompt included.
More from Reddit r/MachineLearning
View all →- I built this, feedback welcome [P]
- Self-distillation with an EMA teacher at scale: ZeRO-3 + PEFT [P]
- Exploring Next-Concept Modeling (NCM) & Transformer Interpretability: Repository of 20 Exploratory Studies & Controls [P]
- Ψ-0.6: Adaptive Streams of Thought in a Unified Vision-Language World Model [R]
- Could AI detection eventually become a fundamentally unsolvable problem, similar to the halting problem? [D]
- A harness that records where an LLM agent’s tool arguments came from, before the tool runs [P]
Markdown (.md) · JSON-LD schema (.json) · Machine-readable for AI & GEO