Expand description
Glm5NextWeightLoader — assembles the 45-layer GLM-5.3 text stack from a WeightStore.
Everything it calls already existed and was gated: bind_kda_weights, build_dsa_weights,
glm5next_mlp::build. This is the wiring, plus the one thing wiring must do that the pieces
cannot — decide, per layer, WHICH pieces.
§Where the classification comes from
Not from tensor names, and not from modular arithmetic. Glm5NextTextSkeleton::from_config
derives the mixer and MLP kind of all 45 layers from the checkpoint’s own
linear_attn_config index lists and first_k_dense_replace, cross-checked against the
textual arrays, and refuses anything it was not taught. This loader iterates that.
§TP, on every half
DSA shards through DsaTpPlan, the MLP through Glm5NextMlpConfig, and KDA through
KdaShardedSource — an adapter that slices the host bytes before the proven
bind_kda_weights sees them, so TP=1 and TP=2 take the identical binder code path.
🪤 Both mixers end in a row-parallel o_proj, so the attention output is a partial sum
at TP>1 and Glm5NextLayer::mixer_all_reduce reduces it before the mHC highway sees it.
Half-applying the sharding — the state before this was wired — meant every rank computed a
WHOLE KDA block and the all-reduce double-counted it: no crash, no shape error.