pub fn gdn_verify_fused_conv_kn_batched(
gpu: &dyn GpuBackend,
kernel: KernelHandle,
conv_state: DevicePtr,
new_input: DevicePtr,
weight: &DenseWeight,
output: DevicePtr,
conv_state_inter: DevicePtr,
num_tokens: u32,
d_inner: u32,
d_conv: u32,
qk_channels: u32,
head_dim: u32,
input_stride: u32,
output_stride: u32,
inter_stride: u32,
l2_eps: f32,
n_seq: u32,
conv_state_seq_stride: u32,
input_seq_stride: u32,
output_seq_stride: u32,
inter_seq_stride: u32,
stream: u64,
) -> Result<()>Expand description
Fused generic-K DFlash-verify conv1d+L2norm: ALL K draft positions in one
launch, with every per-token conv-state rollback snapshot written inline
to a strided intermediates array (replaces the per-token
conv1d_update_l2norm ×K + copy_d2d ×K sequence — 34 serialized ops at
K=17). conv_state is left holding the committed (post final-position)
window, which the kernel also duplicates as snapshot K-1, so the caller
issues NO copies.
Same numerics as the per-token path (identical accumulation order under –fmad=false; the K=2 twin is proven bit-identical by gdn_verify_fused_microtest).
Kernel: gdn_verify_fused_conv_kn(conv_state, new_input, weight, output, conv_state_inter, num_tokens, dim, d_conv, qk_channels, head_dim, input_stride, output_stride, inter_stride, l2_eps)
Grid: (ceil(dim/256), 1, 1) Block: (256, 1, 1)
BATCHED verify conv: n_seq sequences x num_tokens positions in ONE launch.
Step 1 of batched speculative decoding. spec_step.rs:94 currently calls
decode_verify one sequence at a time, so MTP at C=n runs n full model
forwards and re-reads ~9.6 GB of weights n times — measured as a 3.4% LOSS at
C=2 and a HALVING of C=4. One launch over gridDim.y = n_seq is the first
piece of making n*(K+1) rows share a single weight read.
Bit-identical to n separate gdn_verify_fused_conv_kn calls: each sequence’s
conv window is independent, so only the base addresses differ.
Grid: (ceil(d_inner/256), n_seq, 1) Block: (256, 1, 1)