pub fn w4a4_gemm(
gpu: &dyn GpuBackend,
kernel: KernelHandle,
a_packed: DevicePtr,
a_scale: DevicePtr,
weight: &QuantizedWeight,
output: DevicePtr,
m: u32,
n: u32,
k: u32,
stream: u64,
) -> Result<()>Expand description
W4A4 NVFP4 prefill GEMM (native FP4 tensor cores, sm_121a). Activation is
pre-quantized NVFP4 (a_packed/a_scale, scale2=1.0); weight is the native
NVFP4 QuantizedWeight. Output BF16 [M, N]. See kernels/…/w4a4_gemm.cu.
Grid: (ceil(N/128), ceil(M/128), 1) Block: (256, 1, 1).