ships_vanilla_norm_weights

Function ships_vanilla_norm_weights 

Source
pub fn ships_vanilla_norm_weights(config: &ModelConfig) -> bool
Expand description

True when the checkpoint ships HF-vanilla RMSNorm weights — i.e. the norm weight is used as out = x * w / rms, not Qwen3-Next’s offset-from-1 out = x * (1 + w) / rms.

Such a model must load its norm weights exactly and dispatch rms_norm_vanilla. The alternative — pre-subtracting 1.0 and storing bf16(w - 1) for the offset kernel — is only lossless when w ≈ 1. DeepSeek-V4’s norm weights are ≈ 0.03, so w - 1 ≈ -0.97, and BF16’s rounding error there (~1.9e-3 absolute) becomes a 1.8-3.4 % relative error on the weight itself once 1 is added back — catastrophic cancellation. Measured over all 249 V4 norm tensors: up to 19 % on q_norm, and 100 % with sign flips on the compressor norms.

This is an explicit model dispatch, NOT an inference from weight statistics.