load_fp8_weight

Function load_fp8_weight 

Source
pub fn load_fp8_weight(
    store: &WeightStore,
    name: &str,
    gpu: &dyn GpuBackend,
) -> Result<Fp8Weight>
Expand description

Load an FP8 E4M3 checkpoint weight with per-row f32 scales.

Expects two tensors in the store:

  • {name}.weight: FP8E4M3 [N, K] (1 byte per element)
  • {name}.weight_scale: f32 [N] per-row dequant scale

Both are already on GPU from safetensors mmap — no conversion needed. Returns an Fp8Weight ready for the w8a16_gemv LUT kernel.