Struct LoraWeights 

pub struct LoraWeights {

    pub name: String,
    pub adapter_config: PeftAdapterConfig,
    pub max_rank: usize,
    pub max_loras: usize,
    pub pool: DevicePtr,
    pub pool_bytes: usize,
    pub expert_pool: Option<DevicePtr>,
    pub expert_pool_bytes: usize,
    pub slots: Vec<AdapterSlot>,
    pub active: usize,
    pub tables: BTreeMap<(usize, LoraModule), (DevicePtr, DevicePtr)>,
    pub scale_table: DevicePtr,
    pub ref_counts: Vec<AtomicUsize>,
    pub pinned: usize,
    pub last_used: Vec<AtomicU64>,
    pub lru_tick: AtomicU64,
    pub overlay_raw: Vec<Option<OverlayRawSlot>>,
}

The loaded adapter set: one fixed-address rank-padded pool holding up to max_loras equal-size slots, one AdapterSlot per resident adapter, and per-module [max_loras] device u64 pointer tables (the frozen M2 BGMV contract — filled index k for each packed slot, NULL for the rest).

Single-adapter runs pack exactly one slot (slots.len() == 1, active == 0) — byte-identical to the pre-multi-adapter path. name/adapter_config mirror the ACTIVE slot for logs/status; the install walk reads Self::active_layers.

Fields

name: String

Name of the ACTIVE adapter (mirrors slots[active].name).

adapter_config: PeftAdapterConfig

Config of the ACTIVE adapter (mirrors slots[active].adapter_config).

max_rank: usize

max_loras: usize

pool: DevicePtr

One fixed-address allocation holding every padded A/B for every slot.

pool_bytes: usize

expert_pool: Option<DevicePtr>

Feature-1: separate fixed-address allocation for the router + routed- expert padded A/B (sized from the audited key set, NOT num_experts × num_layers). None when no adapter targets experts/router (byte- identical to the pre-Feature-1 pool). Deliberately outside the equal-size attention pool so pool_slot_bytes + the BGMV route tables are untouched.

expert_pool_bytes: usize

slots: Vec<AdapterSlot>

The resident adapters, slot-indexed (slots[k] lives at pool byte offset k * pool_slot_bytes). len() <= max_loras.

active: usize

Index into slots of the currently-active adapter (0 at load).

tables: BTreeMap<(usize, LoraModule), (DevicePtr, DevicePtr)>

key = (global_layer_idx, module) → (a_table, b_table); each table is a device [max_loras] u64 array, NULL (0) = base-only slot.

scale_table: DevicePtr

The parallel [max_loras] device f32 SCALE table the bgmv reads, indexed by slot: scale_table[k] = slots[k].adapter_config.scaling() (alpha/r, or alpha/√r under rsLoRA — the same per-adapter scale that rides each LoraPair), 0.0 for unpacked slots. Scale is per-ADAPTER (not per-module), so ONE table suffices. Built once at pool pack time alongside the a/b tables (load-time-fixed → graph-safe kernel arg).

ref_counts: Vec<AtomicUsize>

Task #25 (slot ref_count): per-slot in-flight-sequence count, one AtomicUsize per pool index (len() == max_loras, stable across swaps). A sequence acquires (+1) its resolved slot at prefill and releases (-1) at terminal free; a swap/rotate INTO a slot with ref_count > 0 is REFUSED (you cannot replace an adapter mid-decode — it would corrupt in-flight KV and replay a captured graph over swapped pool bytes). Kept as a parallel Vec here (not on AdapterSlot, which derives Clone and is cloned during install — AtomicUsize is not Clone); LoraWeights is deliberately non-Clone and already Send + Sync. Interior-mutable through &self (acquire/release run on the prefill/free &self paths); swaps read it under &mut self at a quiescent point.

pinned: usize

Task #27 (demand-driven promotion): the PINNED/CACHE boundary. Slots [0, pinned) are the startup --lora-adapter set — advertised by /v1/models, resolved by the position-based resolve_adapter_slot, and NEVER an eviction victim. Slots [pinned, max_loras) are the promotion HOT CACHE (empty placeholders at load): a demand-promoted adapter lands in one of these. pinned == slots-populated-at-load.

last_used: Vec<AtomicU64>

Task #27: per-slot last-used LRU tick, one AtomicU64 per pool index (len() == max_loras, parallel to ref_counts). Bumped in Self::acquire_slot on the RESOLVED index so victim selection ages the TRUE slot a request used (including -1 -> active). A cache slot with the smallest last_used among the ref_count == 0 idle slots is the LRU eviction victim. Interior-mutable through &self like ref_counts.

lru_tick: AtomicU64

Task #27: monotonic source for last_used ticks (never wraps in practice). Bumped once per acquire.

overlay_raw: Vec<Option<OverlayRawSlot>>

Feature-2 (token overlay): per-slot Stage-1 raw overlay upload, len == slots.len() (padding slots push None). Consumed + cleared by set_lora_weights (Stage 2 build_overlay), which needs the served embed/lm_head tables that only exist after weight load. Vec::new() / all-None ⇒ no overlay adapter ⇒ byte-identical to a no-overlay build.

Implementations

impl LoraWeights

pub fn acquire_slot(&self, slot: i32) -> i32

Task #25: resolve slot (>= 0 → that slot, -1 → active) to a concrete pool index and +1 its ref_count, returning the RESOLVED index so the caller can release EXACTLY that index later (immune to an intervening rotate changing active). Returns -1 — “nothing acquired” — when the resolved index is out of range (bad request slot); the active slot is always in range so -1 -> active never no-ops here for a loaded pool.

pub fn touch_slot(&self, slot: usize)

Task #27: stamp slot as most-recently-used WITHOUT taking a ref. Called right after a promote so a freshly-staged (ref_count==0) slot is NOT the immediate LRU victim of a back-to-back promote before its own request has acquired — otherwise two distinct cold adapters promoted in quick succession would collide on the same slot (the second evicting the first).

pub fn slot_last_used(&self, slot: usize) -> u64

Task #27: current LRU tick of pool slot (larger = more recently acquired). Out-of-range → 0 (never used).

pub fn refresh_slot_tables( &self, slot: usize, layers: &[Option<LoraLayerWeights>], scale: f32, gpu: &dyn GpuBackend, ) -> Result<()>

Task #26: refresh slot’s cell in the [max_loras] a/b pointer tables + the per-slot scale table from layers (the just-staged adapter’s actual per-module coverage). A re-staged adapter whose module coverage DIFFERS from the evicted one would otherwise keep a STALE table entry: the bgmv-routed path would SKIP a module the new adapter adds (a_table[slot] stale-NULL → missed delta), keep applying an evicted module (stale non-NULL → wrong delta), or use the wrong per-slot scale. Shared by BOTH the disk swap (pack_store_into_slot) and the RDMA swap (swap_lora_slot_from_peer). Only the [slot] cell of each fixed-address device array is rewritten.

pub fn cache_slot_views(&self) -> Vec<(usize, SlotView)>

Task #27: snapshot the CACHE region [pinned, max_loras) as (slot_index, SlotView) for select_victim_slot. filled = the slot holds a non-placeholder adapter (non-empty name). Read on the model thread at a quiescent point.

pub fn release_slot(&self, resolved: i32)

Task #25: release a ref previously taken by Self::acquire_slot, by the RESOLVED index it returned. -1 (nothing acquired) is a no-op. Saturating so a stray double-release can never wrap the counter below 0.

pub fn slot_ref_count(&self, slot: usize) -> usize

Task #25: current in-flight ref_count of pool slot (the exact read the swap busy-slot gate branches on). Out-of-range → 0.

impl LoraWeights

pub fn active_layers(&self) -> &[Option<LoraLayerWeights>]

The active slot’s per-layer pairs (GLOBAL-layer-indexed) — what the install walk copies onto the layer structs.

pub fn routed_prefill_slot(&self, adapter_slot: i32) -> Option<usize>

#30 (routed-prefill precision): resolve a request’s adapter_slot (>= 0 → that slot, -1 → active) and return Some(resolved) ONLY when it routes to a NON-active, in-range slot — the SINGLE source of truth for “this prefill must apply the request slot’s pair via the dense path”. Kept in exact lockstep with crate::model’s upload_seq_slot_uniform (resolved == active → DevicePtr(0) → installed-pair path). Returns None for an active/base request (byte-identical) and for out-of-range slots (bad request → installed active pair, never a panic).

pub fn slot_of(&self, name: &str) -> Option<usize>

Resolve an adapter NAME to its slot index (for runtime rotation).

pub fn adapter_names(&self) -> Vec<String>

All resident adapter names in slot order (for /v1/models).

pub fn adapter_id_for_slot(&self, slot: i32) -> u64

Stable adapter_id (Task #24) for a pool slot request selector. slot follows the SequenceState.adapter_slot convention: >= 0 selects that resident slot, -1 means “defer to the installed active adapter” (so a default request keys under whatever adapter is actually active — matching build_seq_slot_host’s -1 -> active resolution). The id is the NAME hash, resolved at prefill time (active may rotate between HTTP resolve and prefill). Out-of-range slots fall back to the base sentinel 0.