pub fn q2_0_gemv(
gpu: &dyn GpuBackend,
kernel: KernelHandle,
input: DevicePtr,
weight: &PackedQ2Weight,
output: DevicePtr,
stream: u64,
) -> Result<()>Expand description
Q2_0 GEMV (M=1 decode): C[1,N] = A[1,K] @ dequant(B), weights kept packed.
A is BF16 [1, K]; B is the raw block_q2_0 buffer; C is BF16
[1, N]. Dequant (code-1)*d happens inside the dot-product.
Kernel: q2_0_gemv(A, B, C, N, K, group) Grid: (ceil(N/4),1,1) Block: (256,1,1)