moe_expert_gemv

Function moe_expert_gemv 

Source
pub fn moe_expert_gemv(
    gpu: &dyn GpuBackend,
    kernel: KernelHandle,
    input: DevicePtr,
    packed_ptrs: DevicePtr,
    scale_ptrs: DevicePtr,
    scale2_vals: DevicePtr,
    output: DevicePtr,
    expert_indices: DevicePtr,
    n: u32,
    k: u32,
    top_k: u32,
    input_stride: u32,
    stream: u64,
) -> Result<()>
Expand description

Batched MoE expert W4A16 GEMV: runs top_k expert GEMVs in one launch.

Uses device-side pointer tables for weight indirection. expert_indices come from GPU top-K (device memory).

input_stride: 0 = shared input (gate/up), K = per-expert input (down).

Kernel: moe_expert_gemv(A, packed_ptrs, scale_ptrs, scale2_vals, C, expert_indices, N, K, top_k, input_stride) Grid: (ceil(N/4), top_k, 1) Block: (128, 1, 1)