pub fn requant_w_nvfp4_int8(
gpu: &dyn GpuBackend,
kernel: KernelHandle,
w_packed: DevicePtr,
w_e4m3: DevicePtr,
scale2: f32,
w_i8: DevicePtr,
w_scale: DevicePtr,
n: u32,
k: u32,
stream: u64,
) -> Result<()>Expand description
Requant an NVFP4 weight (packed E2M1 + per-16 E4M3 block scales + per-tensor
scale2) into an int8 weight + per-32 F32 block scale, for the int8 W4A8
prefill GEMM (int8_gemm_faith2). One-time conversion per weight at load (or
lazily on first int8 prefill).
Reads W_packed[N, K/2], W_e4m3[N, K/16], scale2 → W_i8[N, K] (signed
int8) + W_scale[N, K/32] (F32). The per-16 NVFP4 scales are re-blocked to
per-32 int8 scales by the kernel.
Grid: (ceil(N*(K/32) / 128), 1, 1) Block: (128, 1, 1)