Module kv_cache 

Paged KV cache block allocator.

Manages a pool of fixed-size blocks for attention KV storage. Each block holds block_size token positions for all KV heads.

Structs

KvCacheConfig

Configuration for the paged KV cache.

PagedKvCache

Paged KV cache across all attention layers.

Enums

KvCacheDtype

KV cache quantization dtype.