Struct RadixTree 

pub struct RadixTree { /* private fields */ }

Thread-safe radix tree prefix cache.

SSM snapshots are stored in a separate SsmSnapshotIndex, decoupled from tree node lifetime. This ensures snapshots survive KV cache eviction. Lock ordering: acquire inner first (then release), then snapshot_index.

Implementations

impl RadixTree

pub fn new() -> Self

Trait Implementations

impl Default for RadixTree

fn default() -> Self

Returns the “default value” for a type.

impl PrefixCache for RadixTree

fn lookup( &self, tokens: &[u32], block_size: usize, session_hash: u64, adapter_id: u64, ) -> PrefixMatch

Look up a token sequence and return cached KV blocks for the longest matching prefix (block-aligned).

fn peek_matched_tokens( &self, tokens: &[u32], block_size: usize, adapter_id: u64, ) -> usize

Read-only longest-prefix probe: number of tokens (block-aligned) lookup would match, WITHOUT taking refs, touching LRU state, or counting a hit/miss. Used by the prefill tail-checkpoint split to detect conversation reuse before deciding to pay the extra pass. Task #24: keyed by adapter_id so a cross-adapter peek reports a miss.

fn insert( &self, tokens: &[u32], block_table: &[u32], disk_block_ids: &[u32], block_size: usize, matched_tokens: usize, adapter_id: u64, ) -> InsertAcquired

Insert a completed prefill’s blocks into the cache.

fn insert_with_snapshot( &self, tokens: &[u32], block_table: &[u32], disk_block_ids: &[u32], block_size: usize, snapshot_id: usize, session_hash: u64, matched_tokens: usize, adapter_id: u64, ) -> (Option<usize>, InsertAcquired)

Insert blocks with an SSM state snapshot registered in the snapshot index.

fn insert_tail_snapshot( &self, tokens: &[u32], snapshot_id: usize, session_hash: u64, adapter_id: u64, ) -> Vec<usize>

Register the per-session TAIL snapshot in the index WITHOUT touching the radix tree (the final chunk’s insert covers those blocks). Supersedes this session’s previous tail; returns displaced snapshot ids to free.

fn insert_tail_sibling_snapshot( &self, tokens: &[u32], snapshot_id: usize, session_hash: u64, adapter_id: u64, ) -> Option<usize>

Register the tail’s EARLY sibling (tb - bs) in the index. Must be called after insert_tail_snapshot in the same finalize (the tail insert sweeps the session’s previous tail + sibling). Returns a displaced snapshot id to free, if the prefix was already registered.

fn insert_intermediate_snapshot( &self, tokens: &[u32], _block_table: &[u32], _disk_block_ids: &[u32], _block_size: usize, snapshot_id: usize, session_hash: u64, _matched_tokens: usize, adapter_id: u64, ) -> Option<usize>

Insert an SSM snapshot at an intermediate token boundary.

fn release(&self, tokens: &[u32], block_size: usize, adapter_id: u64)

Release ref_counts on blocks that were acquired via lookup.

fn release_matched( &self, tokens: &[u32], block_size: usize, matched_tokens: usize, adapter_id: u64, )

Release exactly the block-aligned prefix acquired by one lookup.

fn evict(&self, num_blocks: usize) -> EvictedBlocks

Evict up to num_blocks cached blocks, returning their physical indices and parallel disk-block IDs (Phase 6.1.e).

fn evict_snapshot_lru(&self) -> Option<usize>

Evict the least-recently-used SSM snapshot from the snapshot index. Returns the snapshot ID so the caller can free it in SsmSnapshotPool.

fn evict_snapshot_to_tier(&self, min_tokens: usize) -> Option<TierEvict>

Phase 1b spill tier: pick a spill victim (same policy as evict_snapshot_lru, HBM-resident only) and decide whether it is worth spilling — see TierEvict. A victim shallower than min_tokens cannot repay the spill’s fixed cost, so its entry is dropped outright rather than left findable-but-empty. min_tokens == 0 disables the gate. None when nothing resident remains; default None (no tier).

fn promote_snapshot(&self, key: u64, new_slot: usize) -> bool

Phase 1b spill tier: after the caller faulted a spilled snapshot’s bytes into new_slot, re-home its index entry to HBM. Returns false if the key is unknown. Default: false.

fn forget_snapshot_tier_key(&self, key: u64) -> bool

Phase 1b spill tier: the FAILED-fault-in twin of Self::promote_snapshot. The caller’s store.get(key) MISSED, so this entry is findable by lookup_tiered with no bytes behind it. Left in place, every warm turn on this prefix repeats the whole doomed cycle — spill a LIVE 66 MB victim D2H to free a slot, fault in, miss, free the slot — and then recomputes anyway; under ATLAS_SSM_TIER_DISK_GB that doomed spill evicts one MORE tier record, so the cap’s own pressure re-amplifies itself. Dropping the entry degrades the prefix to a plain recompute ONCE.

fn snapshot_count(&self) -> usize

Number of SSM snapshots currently stored in the snapshot index.

fn stats(&self) -> (usize, usize)

(entries, cached_blocks) for logging.

fn is_active(&self) -> bool

Whether this implementation is active (i.e., a real cache that actually inserts/holds refs). NoPrefixCaching returns false; RadixTree returns true. Callers use this to skip ref-bookkeeping that’s only meaningful when the cache holds refs (e.g., the manual kv_cache.inc_ref in cache_sequence that pairs with eviction’s return_evicted_block).