Struct QsaIndexer 

pub struct QsaIndexer { /* private fields */ }

Implementations

impl QsaIndexer

pub fn prefill_select( &self, st: &mut QsaSeqState, normed: DevicePtr, q_roped: DevicePtr, attn_ctx: DevicePtr, k_pool: DevicePtr, v_pool: DevicePtr, seq_block_table: &[u32], seq_start: usize, num_tokens: usize, nq: u32, block_size: u32, inv_sqrt_d: f32, scratch: DevicePtr, gpu: &dyn GpuBackend, stream: u64, ) -> Result<()>

Stage 2: per-query prefill selection for ANY prefill chunk. Chunk rows whose GLOBAL position (seq_start + row) is at or past the inert bound get their ATTENTION CONTEXT rows (pre-gate, pre-o_proj) overwritten with attention over exactly their reference-selected set, read straight from the paged KV cache — which at this point holds every prior chunk plus this one (section-7 writes precede attention). Rows below the bound keep the dense output, which is provably identical there. Requires prefill_ingest to have run for this chunk (the ingest hook precedes the attention call).

impl QsaIndexer

pub fn snapshot_aux( &self, st: &QsaSeqState, gpu: &dyn GpuBackend, stream: u64, ) -> Result<Vec<u8>>

Marconi aux blob: [ingested u64][pooled u64][raw_keys bf16 bytes]. Raw keys are a deterministic function of the token prefix, so the snapshot IS the indexer state; block keys are re-pooled on restore (one kernel) rather than serialized.

pub fn restore_aux( &self, st: &mut QsaSeqState, blob: &[u8], gpu: &dyn GpuBackend, stream: u64, ) -> Result<()>

Restore the blob from Self::snapshot_aux on a prefix-cache hit: upload the raw keys, reset the counters, re-pool the block keys.

impl QsaIndexer

pub fn new( qk_proj_w: DevicePtr, q_norm_w: DevicePtr, k_norm_w: DevicePtr, n_heads: usize, hd: usize, ratio: usize, budget: usize, rot: usize, theta: f32, eps: f32, hidden: usize, nkv_attn: usize, hd_attn: usize, gpu: &dyn GpuBackend, ) -> Result<Self>

pub fn new_seq_state(&self, gpu: &dyn GpuBackend) -> Result<QsaSeqState>

The largest visible prefix whose selection is provably all-visible. One sequence’s indexer carry: counters + raw/pooled key buffers (per-seq CONTENT; launch scratch stays layer-owned — steps serialize).

pub fn release_seq_state( &self, st: &mut QsaSeqState, gpu: &dyn GpuBackend, ) -> Result<()>

Release one sequence’s indexer carry.

QsaSeqState holds bare DevicePtrs, so dropping the struct frees nothing. Without this every finished sequence left max_tokens * hd * 2 (raw) + max_tokens/ratio * hd * 2 (pooled) bytes on the device for EACH full-attention layer — at 200K context and 12 such layers, ~739 MB per request. On unified memory that is invisible to RSS and to nvidia-smi, so it surfaced only as the host running out of RAM with no process to blame.

Idempotent: each pointer is nulled as it is freed, so a second call (or a release after a partial failure) cannot double-free. A failure on the first buffer still attempts the second — leaking the rest because the first free failed is the bug this exists to prevent.

pub fn inert_bound(&self) -> usize

pub fn prefill_ingest( &self, st: &mut QsaSeqState, hidden: DevicePtr, num_tokens: usize, seq_start: usize, gpu: &dyn GpuBackend, stream: u64, ) -> Result<()>

Ingest num_tokens prefill tokens starting at seq_start: project qk, park the raw keys, pool freshly complete blocks. seq_start == 0 resets the sequence (single-seq v1, PLE-style).

pub fn decode_select( &self, st: &mut QsaSeqState, normed: DevicePtr, pos: usize, k_pool: DevicePtr, v_pool: DevicePtr, block_table_dev: DevicePtr, block_size: u32, gpu: &dyn GpuBackend, stream: u64, ) -> Result<Option<QsaSelection>>

Decode-step ingest + selection for the token at pos (0-based; pos + 1 visible). None inside the inert bound (dense is exact).