Struct TokenOverlaySet 

pub struct TokenOverlaySet {
    pub overlays: Vec<Option<EmbedOverlay>>,
    pub embed_slot_map_table: DevicePtr,
    pub embed_rows_table: DevicePtr,
    pub embed_n_table: DevicePtr,
    pub vocab: u32,
    pub lmhead_rows_table: DevicePtr,
    pub lmhead_ids_table: DevicePtr,
    pub n_override_table: DevicePtr,
    pub max_n_override: u32,
}

The resolved overlay set for the whole adapter pool. Built once in set_lora_weights (Stage 2) from the per-slot EmbedOverlays. None on the model ⇒ overlay feature OFF ⇒ every forward hook early-returns (byte-identical to a no-overlay build).

Fields

overlays: Vec<Option<EmbedOverlay>>

Per-slot compact overlays (len == max_loras), retained so the device buffers they own outlive the tables that point into them.

embed_slot_map_table: DevicePtr

u64[max_loras] → i32*[vocab] embed slot_map (0 = slot has no overlay).

embed_rows_table: DevicePtr

u64[max_loras] → bf16*[n,h] embed override rows.

embed_n_table: DevicePtr

u32[max_loras] EMBED n_override per slot — the row count of each slot’s embed_rows_table cell, the embed kernel’s slot < n bound (CWE-125 guard). Distinct from n_override_table, which is the LM_HEAD count (0 for an untied slot with no lm_head overlay).

vocab: u32

slot_map length shared by every resident overlay (each slot records the served vocab it was built against; from_slots REFUSES a mix) — the embed kernel’s ids[r] < vocab bound (CWE-125 guard).

lmhead_rows_table: DevicePtr

u64[max_loras] → bf16*[n,h] lm_head override rows (== embed cell when tied).

lmhead_ids_table: DevicePtr

u64[max_loras] → u32*[n] lm_head override ids (== embed cell when tied).

n_override_table: DevicePtr

u32[max_loras] lm_head n_override per slot (0 ⇒ lm_head skip).

max_n_override: u32

max(lmhead n_override) across slots — the lm_head kernel’s grid.y.

Implementations

impl TokenOverlaySet

pub fn from_slots( gpu: &dyn GpuBackend, overlays: Vec<Option<EmbedOverlay>>, max_loras: usize, tied: bool, ) -> Result<Self>

Build the device tables from per-slot overlays. tied ⇒ the lm_head reuses each slot’s embed override rows/ids (tied vocab head, or a quantized head derived from embed): the logit recompute dot(hidden, embed_row) is exactly the tied output projection. An untied slot that ships its own lm_head overlay uses its distinct rows/ids; an untied slot that does NOT ⇒ n_override[k] = 0 (embed-only correction).

pub fn any_active(&self) -> bool

True when at least one slot has a resident overlay (else the set is inert and the caller can drop it to keep the hooks byte-identical to off).