transpose_u8

Function transpose_u8 

Source
pub fn transpose_u8(
    gpu: &dyn GpuBackend,
    kernel: KernelHandle,
    src: DevicePtr,
    dst: DevicePtr,
    rows: u32,
    cols: u32,
    stream: u64,
) -> Result<()>
Expand description

Batched per-expert uint8 transpose for MoE down_proj relayout.

Reads per-expert source pointers from src_ptrs and writes per-expert transposed [cols, rows] blocks via dst_ptrs. Both tables hold one device pointer per global expert; NULL entries (EP-remote experts) cause the kernel to exit early at block level.

Grid: (ceil(cols/32), ceil(rows/32), num_experts) Block: (32, 8) Single-matrix uint8 transpose [rows, cols] -> [cols, rows] on GPU (transpose_u8.cu, 32x32 shared-memory tiles). Load-time replacement for the host byte-loop in QuantizedWeight::transpose_for_gemm* — the old path bounced every packed weight D2H -> O(N*K) host loop -> H2D (~13.6 GB through host at 27B cold load).