fp8_gemm_act_weight_t_rowwise

Function fp8_gemm_act_weight_t_rowwise 

Source
pub fn fp8_gemm_act_weight_t_rowwise(
    act_fp8: u64,
    act_scale: u64,
    weight_fp8: u64,
    weight_scale: u64,
    out: u64,
    m: u32,
    n: u32,
    k: u32,
    stream: u64,
) -> Result<()>
Expand description

Native FP8 (E4M3) out[M,N] = act[M,K] @ weight[N,K]ᵀ → BF16 with ROW-WISE scaling (OUTER_VEC): per-output-row weight scale weight_scale[N] and per-token activation scale act_scale[M]. This is the fp8 path GB10/sm_121 actually supports (128-block fp8 is B200-only). ~1.8× the bf16 path. cuBLAS folds A_scale[i]·B_scale[j] into the FP32 epilogue; with D=[N,M], i indexes weight rows (N) and j indexes tokens (M) — exactly row-wise.