Expand description
GLM-5.3 DSA checkpoint binding: the 14 self_attn tensors of a DSA block,
their expected dtype and shape, and a verifier that fails loudly.
Scoped to LibertAIDAI/GLM-5.3-Flash-NVFP4@9e0d74e3. Mirrors
crate::layers::glm5next_kda::binding and deliberately reuses its
RawTensor / TensorSource / dtype types instead of growing a parallel set.
Shapes are expressed against Glm5NextDsaConfig, so a geometry change fails
here rather than at kernel launch — the same contract the KDA binder holds.
§🪤 What this exists to catch
indexer.k_normhas abias. It is annn.LayerNorm, not an RMSNorm — the only other norm in GLM-5.3 with a bias. A binder that loads only.weightdrops mean-subtraction and the bias, and nothing about the shapes says so. The spec table lists the bias as REQUIRED so its absence is an error, not a silent zero.index_kpool_compress_apeis[kpool, index_head_dim]— indexed by pool SLOT, not by head and not by pool. Its 1 KB size makes a wrong-axis bind easy to miss.q_b_projandkv_b_projcarry different per-head widths —qk_head_dim(256) vsnope + v_head_dim(512). Both are[heads * w, lora], so a swapped width still yields a well-formed 2-D tensor.- NoPE: there is no
wkv_a_rope/wq_b_ropehere at all, andkv_a_proj_with_mqaiskv_lora_rankwide (512), not+ rope(576). A spec that expects 576 rejects the real checkpoint.
Structs§
- DsaBind
Report - What a successful bind saw.
- DsaSpec
- One expected tensor: layer-relative name, dtype, and full (unsharded) shape.
Functions§
- dsa_
tensor_ specs - Every
self_attntensor a DSA block has — and the complete list of what it may have.full_headsis the pre-shard head count; the checkpoint is never sharded on disk, so binding always validates against the full geometry and slices after. - verify_
dsa_ block - Verify one DSA block against the spec table.