Function q2_0_gemv_vec_batchm 

pub fn q2_0_gemv_vec_batchm(
    gpu: &dyn GpuBackend,
    kernel: KernelHandle,
    input: DevicePtr,
    weight: &PackedQ2Weight,
    output: DevicePtr,
    m: u32,
    stream: u64,
) -> Result<()>

Q2_0 batched GEMV (M>=1 decode), CANDIDATE B: C[M,N] = A[M,K] @ dequant(B).

Reads each weight word once and MAC’s it into all m accumulators (all m activation rows staged in smem). A BF16 [M,K] row-major, C BF16 [M,N] row-major. Bit-consistent with running the M=1 kernel M times.

The kernel itself is capped at Q2_BATCHM_MAX_M rows/launch, so m beyond that is served by CHUNKING: successive <=8-row launches with the [M,K] input and [M,N] output base pointers advanced by whole rows (BF16 = 2 B). Chunking a caller with m <= 8 costs one launch (identical to the direct call); it exists so a wide concurrent-decode step (max-num-seqs up to 16) can never drive the kernel into its OOB path.

Kernel: q2_0_gemv_vec_batchm(A, B, C, N, K, group, M).