Module glm5next_layer 

GLM-5.3-Flash composite decoder layer – mixer (KDA|DSA) + MLP (dense|MoE) + mHC. Glm5NextLayer — the composite GLM-5.3 decoder layer that implements TransformerLayer.

This is the piece that makes the model bind. Everything it dispatches to already existed and was numerically gated in Slices 1–13; what did not exist was a single type the loader can return 45 of, dispatching mixer (KDA | DSA) + MLP (dense | routed MoE) + mHC in the order crate::layers::glm5next_skeleton records as data.

The residual plan, executed

Per site, exactly ResidualStep’s order:

layer 0 only:  hc_expand(hidden) -> streams        [hc_mult, hidden] FP32 highway

attention site:  hc_pre(streams) -> y, post, comb
                 rms_norm_vanilla(y, input_layernorm) -> normed
                 mixer(normed) -> block_out
                 hc_post(block_out, residual = streams, post, comb) -> streams

FFN site:        the same, with post_attention_layernorm and the MLP

last layer:    hc_head_mean(streams) -> hidden     UNWEIGHTED mean, no parameters

🪤 hc_pre does not modify streams. That is what makes the skeleton’s ResidualStep::SaveResidual free here — hc_post reads the same buffer as its residual and writes back over it. Snapshotting is only needed if something overwrites the highway between the two calls; nothing here does, and the ordering below is the guard.

🪤 The traps this file holds

• GLM’s norms are PLAIN RMSNorm. rms_norm_vanilla is x * rms * w; the other rms_norm is x * rms * (1 + w). Identical signatures, identical shapes, and picking the wrong one is silent. Every norm here takes the vanilla entry point.
• The mHC head collapse is an UNWEIGHTED MEAN. GLM’s Glm5NextTextHyperHead has no parameters and the checkpoint carries zero hc_head tensors, unlike DeepSeek-V4’s learned sigmoid-weighted sum. Reaching for ops::hc_head would look for weights that do not exist.
• The highway is indexed by TOKEN. Prefill is overridden rather than left to the trait’s sequential default, because that default runs every token through layer 0 before layer 1 — which with a single-slot highway would leave only the LAST token’s streams alive. See Glm5NextLayer::prefill.
• Both MLP arms leave a PARTIAL SUM whenever TP or EP is on. The single all_reduce at the end of the FFN site covers both, and it must happen before hc_post mixes the output back into the highway.

Re-exports

pub use state::alloc_kda_ssm_state;

Modules

profile

ATLAS_GLM_PROFILE=1 — per-section decode timing for the GLM-5.3 stack.

state

One GLM-5.3 decoder layer’s per-sequence state.

Structs

Glm5NextLayer

One bound GLM-5.3 decoder layer.

Glm5NextMhc

This layer’s hyper-connection: both sites’ weights plus the kernels and the two scalars.

Enums

Glm5NextMixer

Which mixer this layer runs. Both halves already exist and are GPU-gated; this enum is the dispatch, not new math.

Glm5NextMlpSite

Which MLP this layer runs. Layers 0..first_k_dense_replace are dense; the rest route.