w4a4_gemm

Function w4a4_gemm 

Source
pub fn w4a4_gemm(
    gpu: &dyn GpuBackend,
    kernel: KernelHandle,
    a_packed: DevicePtr,
    a_scale: DevicePtr,
    weight: &QuantizedWeight,
    output: DevicePtr,
    m: u32,
    n: u32,
    k: u32,
    stream: u64,
) -> Result<()>
Expand description

W4A4 NVFP4 prefill GEMM (native FP4 tensor cores, sm_121a). Activation is pre-quantized NVFP4 (a_packed/a_scale, scale2=1.0); weight is the native NVFP4 QuantizedWeight. Output BF16 [M, N]. See kernels/…/w4a4_gemm.cu. Grid: (ceil(N/128), ceil(M/128), 1) Block: (256, 1, 1).