embed_overlay_routed

Function embed_overlay_routed 

Source
pub fn embed_overlay_routed(
    gpu: &dyn GpuBackend,
    kernel: KernelHandle,
    ids: DevicePtr,
    seq_slot: DevicePtr,
    active: i32,
    slot_map_tab: DevicePtr,
    rows_tab: DevicePtr,
    n_tab: DevicePtr,
    out: DevicePtr,
    num_tokens: u32,
    h: u32,
    vocab: u32,
    stream: u64,
) -> Result<()>
Expand description

In-place row-replace of overridden vocab rows on the residual stream after the embed gather (BEFORE scale_embeddings). Per row r: s = seq_slot[r] (or active when NULL); s<0 skip; ids[r]>=vocab skip (no overlay entry for a token beyond the overlay’s served-vocab snapshot — CWE-125 guard); slot=slot_map_tab[s][ids[r]]; slot<0 or slot>=n_tab[s] skip; copy rows_tab[s][slot] over out[r].