grouped_gemm_mla

Function grouped_gemm_mla 

Source
pub fn grouped_gemm_mla(
    gpu: &dyn GpuBackend,
    kernel: KernelHandle,
    a: DevicePtr,
    b: DevicePtr,
    c: DevicePtr,
    m: u32,
    g: u32,
    k_g: u32,
    n_g: u32,
    a_stride: u32,
    c_stride: u32,
    stream: u64,
) -> Result<()>
Expand description

Grouped GEMM for MLA: G independent [M,K_g]@[N_g,K_g]^T→[M,N_g] in one launch. Grid: (M*G, ceil(N_g/4), 1) Block: (256, 1, 1)