q2_0_mmq_gemm

Function q2_0_mmq_gemm 

Source
pub fn q2_0_mmq_gemm(
    gpu: &dyn GpuBackend,
    kernel_nc: KernelHandle,
    kernel_wc: KernelHandle,
    a_q8: DevicePtr,
    w_q2_0: DevicePtr,
    out_bf16: DevicePtr,
    m: u32,
    n: u32,
    k: u32,
    stream: u64,
) -> Result<()>
Expand description

Q2_0 MMQ GEMM: C[m,n] (bf16) = A_q8[m,k] x W_q2_0[n,k]. Fused bf16 store.

a_q8 is the q8_1_mmq (DS4) activation produced by super::quantize_act_q8_1; w_q2_0 is the packed block_q2_0 weight [n, k] (the same buffer resident for the decode GEMV — no repack). Grid / block / smem mirror the Q4_K MMQ (same tile geometry, mmq_x=mmq_y=128).