kda_recurrent

Function kda_recurrent 

Source
pub fn kda_recurrent(
    q: &[f32],
    k: &[f32],
    v: &[f32],
    gate: &[f32],
    beta: &[f32],
    dims: KdaDims,
    state: &mut [f32],
) -> Vec<f32>
Expand description

Decode formulation: one token at a time, carrying state.

S <- S * diag(exp(g_t))          decay along the KEY axis, per channel
delta <- (v_t - S^T k_t) * beta_t
S <- S + k_t (x) delta
o_t <- S^T q_t

q/k are l2-normalised and q scaled by 1/sqrt(head_dim) inside this function, matching use_qk_l2norm_in_kernel=True on the HF side. Pass raw post-conv q/k/v.