lmhead_overlay_routed

Function lmhead_overlay_routed 

Source
pub fn lmhead_overlay_routed(
    gpu: &dyn GpuBackend,
    kernel: KernelHandle,
    hidden: DevicePtr,
    seq_slot: DevicePtr,
    active: i32,
    rows_tab: DevicePtr,
    ids_tab: DevicePtr,
    n_tab: DevicePtr,
    logits: DevicePtr,
    m: u32,
    max_n_override: u32,
    h: u32,
    vocab: u32,
    stream: u64,
) -> Result<()>
Expand description

In-place recompute of overridden logit columns (BEFORE softcap). One warp per (row, j); j indexes the overridden-id slot of that row’s adapter. Picks the bf16 or f32 logits kernel per is_fp32.