Module glm5next_skeleton

Module glm5next_skeleton 

Source
Expand description

GLM-5.3-Flash 45-layer text-model skeleton (Slice 9 – topology, wiring, structural binding). GLM-5.3-Flash 45-layer text-model skeleton — Slice 9.

The topology, the residual/norm/mHC wiring, the structural weight contract and the state plumbing of the text stack, in one place that can be checked against the checkpoint without a GPU and without executing anything.

Scope is deliberate: no MoE, no dense FFN, no MTP speculation, no forward pass. The MLP site is represented as a hole in the residual plan (ResidualStep::Mlp) with its kind recorded, so the shape of what is missing is explicit rather than implied.

§Why a skeleton is its own artifact

“Every attention block executes” and “the model is assembled correctly” are different statements. Slices 1–8 proved the first for all 34 KDA and all 12 DSA blocks. A stack that binds every tensor and still orders its layers wrong, or hangs the hyper-connection off the wrong site, produces perfectly plausible output — the failure mode this campaign keeps paying for. So the ordering, the wiring and the binding are asserted as data.

§Measured facts this module encodes (checkpoint LibertAIDAI/GLM-5.3-Flash-NVFP4@9e0d74e3)

  • The structural (non-MLP) surface is 1,047 tensors in exactly 3 signatures: 34 KDA layers × 23, 11 DSA layers × 22, layer 45 × 20, plus 3 non-layer tensors.
  • 🪤 Layer 45 has NO hyper-connection. All 270 hc_* tensors live on layers 0..=44. A skeleton that gives the MTP layer an attn_hc/ffn_hc looks for six tensors that do not exist.
  • 🪤 The final collapse is an UNWEIGHTED MEAN. Glm5NextTextHyperHead has no parameters and the checkpoint carries zero hc_head tensors — unlike DeepSeek-V4, whose hc_head is a learned sigmoid-weighted sum. Atlas’s hc_head CUDA kernel is the DeepSeek one; for GLM it is ADAPT, not REUSE.
  • 🪤 hc_*_fn is BF16 on disk; only base/scale are F32. The hc_pre kernel takes f32*, so binding must upcast.
  • Layers 0..=2 carry a dense MLP (first_k_dense_replace = 3); 42 text layers and the MTP layer route to experts.

Structs§

Glm5NextTextSkeleton
SkeletonLayer
One decoder layer’s structure.
StateBudget
Per-sequence state contract (Slice 11 gate 3).
StructuralAccounting

Enums§

FinalStep
What the model does once the 45 text layers are done.
Mixer
Which mixer a layer runs. Narrower than LayerType on purpose: the skeleton refuses the kinds GLM-5.3 does not have rather than carrying them as unreachable arms.
Mlp
Which MLP a layer runs. Not executed by this slice; recorded so the hole is named.
ResidualStep
One step of a layer’s residual path, in execution order.
Site
StateKind
Which cache a layer needs. KDA and DSA are mutually exclusive here, and admission needs BOTH kinds satisfied — a KDA slot is not interchangeable with KV blocks.