Struct Glm5NextKdaLayer 

pub struct Glm5NextKdaLayer {
    pub layer_idx: usize,
    pub cfg: Glm5NextKdaConfig,
    pub weights: Glm5NextKdaWeights,
    pub kernels: Glm5NextKdaKernels,
}

One bound KDA attention block.

Fields

layer_idx: usize

Index in the checkpoint’s 45-layer text stack, for diagnostics.

cfg: Glm5NextKdaConfig

weights: Glm5NextKdaWeights

kernels: Glm5NextKdaKernels

Implementations

impl Glm5NextKdaLayer

pub fn new( layer_idx: usize, cfg: Glm5NextKdaConfig, weights: Glm5NextKdaWeights, kernels: Glm5NextKdaKernels, ) -> Result<Self>

pub fn decode( &self, gpu: &dyn GpuBackend, hidden: DevicePtr, state: &KdaSeqState, ws: &Glm5NextKdaWorkspace, stream: u64, ) -> Result<()>

Single-token decode, carrying both states.

The conv fuses SiLU and L2, so q/k reach kda_recurrent already normalised — exactly the pre-normalised contract that kernel takes. Re-normalising here would silently restore the bf16 rounding the fused write destroyed and look like a kernel bug.

Result lands in ws.final_out; state is updated in place.

pub fn decode_k( &self, gpu: &dyn GpuBackend, hidden: DevicePtr, k: usize, state: &KdaSeqState, ws: &Glm5NextKdaWorkspace, snapshots: &[(DevicePtr, DevicePtr)], stream: u64, ) -> Result<()>

K tokens of ONE sequence: the projections batched, the recurrence NOT.

This is the speculative-verify body. The weight-heavy halves — Self::front_end’s q/k/v, both low-rank gate pairs and b_proj, and Self::back_end’s o_proj — run once over all K rows, so a K-token verify reads KDA’s 4.7 GB/rank/token ONCE instead of K times. That is the entire reason speculation can pay on this model.

🔴 Bit-identical to K serial Self::decode calls, which is not a nicety: an accepted draft token must be the token the unspeculated engine would have emitted, or speculation is silently lossy. It holds because dense_gemv_bf16_batchm reproduces each row’s exact K-iteration order and reduction tree (ops::dense_mm_bf16), the pack / gate / sigmoid / o_norm kernels are grid-parallel over the token axis, and Self::stateful_row walks the state one token at a time exactly as decode does.

snapshots[t] — (h_dst, conv_dst) — receives the state AFTER row t, which is what a partial accept rewinds to. Pass k - 1 of them (a full accept never rewinds) or none.

pub fn prefill( &self, gpu: &dyn GpuBackend, hidden: DevicePtr, t: usize, state: &KdaSeqState, ws: &Glm5NextKdaWorkspace, stream: u64, ) -> Result<()>

Chunked prefill over t tokens from the carried state.

pub fn prefill_with_pad_fill( &self, gpu: &dyn GpuBackend, hidden: DevicePtr, t: usize, state: &KdaSeqState, ws: &Glm5NextKdaWorkspace, pad_fill: f32, stream: u64, ) -> Result<()>

Self::prefill with the padded q/k/v tails primed to an arbitrary value.

Production passes zero. The numeric gate passes poison, because kda_chunk_* guard past T in-kernel and that guard needs a test with teeth: the Slice-5 bug wrote entirely correct outputs while leaving the carried recurrent state off by 1.623e13, so a pad tail the caller zeroes proves nothing.