kda_recurrent_prenorm

Function kda_recurrent_prenorm 

Source
pub fn kda_recurrent_prenorm(
    qn: &[f32],
    kn: &[f32],
    v: &[f32],
    gate: &[f32],
    beta: &[f32],
    dims: KdaDims,
    state: &mut [f32],
) -> Vec<f32>
Expand description

Same recurrence, but q/k are already L2-normalised.

This is Atlas’s contract, where causal_conv1d_update_l2norm fuses conv + SiLU + L2 upstream, and it is what the kda_recurrent GPU kernel consumes. Keep it separate rather than passing pre-normalised vectors into kda_recurrent: re-normalising an already-unit vector is nearly a no-op in fp32 (it scales by 1/sqrt(1+1e-6)), but on a bf16-rounded vector — whose norm is off by ~0.4% — it silently RESTORES the norm the rounding destroyed, which makes the reference disagree with the kernel by ~1e-5 and looks exactly like a kernel bug.