pub fn ships_vanilla_norm_weights(config: &ModelConfig) -> boolExpand description
True when the checkpoint ships HF-vanilla RMSNorm weights — i.e. the norm
weight is used as out = x * w / rms, not Qwen3-Next’s offset-from-1
out = x * (1 + w) / rms.
Such a model must load its norm weights exactly and dispatch
rms_norm_vanilla. The alternative — pre-subtracting 1.0 and storing
bf16(w - 1) for the offset kernel — is only lossless when w ≈ 1.
DeepSeek-V4’s norm weights are ≈ 0.03, so w - 1 ≈ -0.97, and BF16’s
rounding error there (~1.9e-3 absolute) becomes a 1.8-3.4 % relative error
on the weight itself once 1 is added back — catastrophic cancellation.
Measured over all 249 V4 norm tensors: up to 19 % on q_norm, and 100 %
with sign flips on the compressor norms.
This is an explicit model dispatch, NOT an inference from weight statistics.