Module prefix_cache 

Prefix caching trait for KV block reuse (SDD).

When multiple requests share a common prompt prefix, previously-computed KV cache blocks can be reused instead of re-running prefill. The cache is indexed by token sequences at block granularity via a radix tree.

Two implementations:

• NoPrefixCaching: no-ops (zero overhead when disabled)
• RadixTree (see crate::radix_tree): full radix tree with LRU eviction

Structs

EvictedBlocks

Result of evicting LRU cached blocks (Phase 6.1.e).

InsertAcquired

What an insert newly took ownership of, so the caller can take the matching references.

NoPrefixCaching

No-op prefix cache (zero overhead when disabled).

PrefixMatch

Result of looking up a token sequence in the prefix cache.

Enums

TierEvict

Outcome of crate::prefix_cache::PrefixCache::evict_snapshot_to_tier.

Traits

PrefixCache

Trait for prefix caching strategies.

Functions

cache_hit_count

cache_hit_tokens_total

cache_miss_count

record_cache_hit

record_cache_miss