nvfp4_mmq_gemm_tiled

Function nvfp4_mmq_gemm_tiled 

Source
pub fn nvfp4_mmq_gemm_tiled(
    gpu: &dyn GpuBackend,
    kernel_nc: KernelHandle,
    kernel_wc: KernelHandle,
    mmq_x: u32,
    a_fp4: DevicePtr,
    w_nvfp4: DevicePtr,
    out_bf16: DevicePtr,
    m: u32,
    n: u32,
    k: u32,
    stream: u64,
) -> Result<()>
Expand description

NVFP4 W4A4 MMQ GEMM with an M-SIZED TILE.

Same kernel family as nvfp4_mmq_gemm, but the caller picks the M tile. The 128-wide tile issues MMAs for all 128 tile columns regardless of m and discards the surplus in the write-back predicate, so decode at m=16 wasted 87.5% of its MMA slots. mmq_x must be one of {16, 32, 128} — the instantiated entries.

PREFILL MUST KEEP 128: grid.y = ceil(m / mmq_x), so a small tile re-streams the whole weight matrix once per M-tile. This is a decode-shape optimisation only.