nvfp4_mmq_repack

Function nvfp4_mmq_repack 

Source
pub fn nvfp4_mmq_repack(
    gpu: &dyn GpuBackend,
    kernel: KernelHandle,
    packed: DevicePtr,
    scales: DevicePtr,
    out_blocks: DevicePtr,
    n: u32,
    k: u32,
    stream: u64,
) -> Result<()>
Expand description

Repack a checkpoint NVFP4 weight (packed E2M1 [n, k/2] low=even/high=odd + E4M3 [n, k/16] scales) into llama block_nvfp4 rows [n][k/64]. Raw bit shuffle — the e2m1 codes and e4m3 scale bytes are reused verbatim (scale2 folded downstream).