requant_w_nvfp4_int8

Function requant_w_nvfp4_int8 

Source
pub fn requant_w_nvfp4_int8(
    gpu: &dyn GpuBackend,
    kernel: KernelHandle,
    w_packed: DevicePtr,
    w_e4m3: DevicePtr,
    scale2: f32,
    w_i8: DevicePtr,
    w_scale: DevicePtr,
    n: u32,
    k: u32,
    stream: u64,
) -> Result<()>
Expand description

Requant an NVFP4 weight (packed E2M1 + per-16 E4M3 block scales + per-tensor scale2) into an int8 weight + per-32 F32 block scale, for the int8 W4A8 prefill GEMM (int8_gemm_faith2). One-time conversion per weight at load (or lazily on first int8 prefill).

Reads W_packed[N, K/2], W_e4m3[N, K/16], scale2W_i8[N, K] (signed int8) + W_scale[N, K/32] (F32). The per-16 NVFP4 scales are re-blocked to per-32 int8 scales by the kernel.

Grid: (ceil(N*(K/32) / 128), 1, 1) Block: (128, 1, 1)