pub fn kda_recurrent_prenorm(
qn: &[f32],
kn: &[f32],
v: &[f32],
gate: &[f32],
beta: &[f32],
dims: KdaDims,
state: &mut [f32],
) -> Vec<f32>Expand description
Same recurrence, but q/k are already L2-normalised.
This is Atlas’s contract, where causal_conv1d_update_l2norm fuses conv + SiLU + L2
upstream, and it is what the kda_recurrent GPU kernel consumes. Keep it separate
rather than passing pre-normalised vectors into kda_recurrent: re-normalising an
already-unit vector is nearly a no-op in fp32 (it scales by 1/sqrt(1+1e-6)), but on a
bf16-rounded vector — whose norm is off by ~0.4% — it silently RESTORES the norm the
rounding destroyed, which makes the reference disagree with the kernel by ~1e-5 and
looks exactly like a kernel bug.