PSA: Upscaling Gemma 4 requires a proportional layer_scalar adjustment

Summary

A lot of people seem to be confused or mystified about this so figured I'd spell it out. I played around with RYS and realized that it broke Gemma 4 models. Turns out there's a `layer_scalar` value that is applied at each layer. If you don't adjust that so that the resulting model gets "the same amount", you break it. Since it's multiplicative, you have to do `s^(1/N)`, where `s` is the original scalar and `N` is the number of times the layer occurs (duplications + 1 fo

SpinGraph analysis pending — check back after processing.

• Using "applications" to make a smaller model more effective at bigger tasks.
• Appreciation post!
• possible evidence of literal prompt injection by anthropic
• Qwen3.6 27B on a 5090, 6.4k sample tok/s distribution after tuning MTP/cache settings
• DGX Spark and Overtemps
• Gemma 4 12B - MLX Kernel