apply_router_lora

Function apply_router_lora 

Source
pub fn apply_router_lora(
    gpu: &dyn GpuBackend,
    kernels: &LoraKernels,
    pair: &LoraPair,
    router_in: DevicePtr,
    gate_logits: DevicePtr,
    n: u32,
    max_rows: u32,
    lora_xa: DevicePtr,
    lora_delta: DevicePtr,
    stream: u64,
) -> Result<()>
Expand description

Fold the router (mlp.gate) LoRA delta onto the routing logits in place, BEFORE top-k. router_in = [n, hidden], gate_logits = [n, num_experts] (modified in place). Chunked by max_rows (the scratch capacity in rows).