Mixture-of-Experts (MoE) language models

TEXAS: Task-Expert-Aware Supervision for Downstream Mixture-of-Experts LLM Adaptation

A new research method called TEXAS improves fine-tuning of Mixture-of-Experts (MoE) LLMs by using correctness-conditioned expert activation patterns to guide token-level supervision, yielding consistent performance gains across models and benchmarks.