Expand description
GLM-5.3-Flash 45-layer text-model skeleton (Slice 9 – topology, wiring, structural binding). GLM-5.3-Flash 45-layer text-model skeleton — Slice 9.
The topology, the residual/norm/mHC wiring, the structural weight contract and the state plumbing of the text stack, in one place that can be checked against the checkpoint without a GPU and without executing anything.
Scope is deliberate: no MoE, no dense FFN, no MTP speculation, no forward pass. The MLP
site is represented as a hole in the residual plan (ResidualStep::Mlp) with its kind
recorded, so the shape of what is missing is explicit rather than implied.
§Why a skeleton is its own artifact
“Every attention block executes” and “the model is assembled correctly” are different statements. Slices 1–8 proved the first for all 34 KDA and all 12 DSA blocks. A stack that binds every tensor and still orders its layers wrong, or hangs the hyper-connection off the wrong site, produces perfectly plausible output — the failure mode this campaign keeps paying for. So the ordering, the wiring and the binding are asserted as data.
§Measured facts this module encodes (checkpoint LibertAIDAI/GLM-5.3-Flash-NVFP4@9e0d74e3)
- The structural (non-MLP) surface is 1,047 tensors in exactly 3 signatures: 34 KDA layers × 23, 11 DSA layers × 22, layer 45 × 20, plus 3 non-layer tensors.
- 🪤 Layer 45 has NO hyper-connection. All 270
hc_*tensors live on layers 0..=44. A skeleton that gives the MTP layer anattn_hc/ffn_hclooks for six tensors that do not exist. - 🪤 The final collapse is an UNWEIGHTED MEAN.
Glm5NextTextHyperHeadhas no parameters and the checkpoint carries zerohc_headtensors — unlike DeepSeek-V4, whosehc_headis a learned sigmoid-weighted sum. Atlas’shc_headCUDA kernel is the DeepSeek one; for GLM it is ADAPT, not REUSE. - 🪤
hc_*_fnis BF16 on disk; onlybase/scaleare F32. Thehc_prekernel takesf32*, so binding must upcast. - Layers 0..=2 carry a dense MLP (
first_k_dense_replace = 3); 42 text layers and the MTP layer route to experts.
Structs§
- Glm5
Next Text Skeleton - Skeleton
Layer - One decoder layer’s structure.
- State
Budget - Per-sequence state contract (Slice 11 gate 3).
- Structural
Accounting
Enums§
- Final
Step - What the model does once the 45 text layers are done.
- Mixer
- Which mixer a layer runs. Narrower than
LayerTypeon purpose: the skeleton refuses the kinds GLM-5.3 does not have rather than carrying them as unreachable arms. - Mlp
- Which MLP a layer runs. Not executed by this slice; recorded so the hole is named.
- Residual
Step - One step of a layer’s residual path, in execution order.
- Site
- State
Kind - Which cache a layer needs. KDA and DSA are mutually exclusive here, and admission needs BOTH kinds satisfied — a KDA slot is not interchangeable with KV blocks.