pub fn transpose_u8(
gpu: &dyn GpuBackend,
kernel: KernelHandle,
src: DevicePtr,
dst: DevicePtr,
rows: u32,
cols: u32,
stream: u64,
) -> Result<()>Expand description
Batched per-expert uint8 transpose for MoE down_proj relayout.
Reads per-expert source pointers from src_ptrs and writes per-expert
transposed [cols, rows] blocks via dst_ptrs. Both tables hold one
device pointer per global expert; NULL entries (EP-remote experts)
cause the kernel to exit early at block level.
Grid: (ceil(cols/32), ceil(rows/32), num_experts) Block: (32, 8)
Single-matrix uint8 transpose [rows, cols] -> [cols, rows] on GPU
(transpose_u8.cu, 32x32 shared-memory tiles). Load-time replacement for
the host byte-loop in QuantizedWeight::transpose_for_gemm* — the old
path bounced every packed weight D2H -> O(N*K) host loop -> H2D
(~13.6 GB through host at 27B cold load).