q12_batched_scratch_bytes_varlen

Function q12_batched_scratch_bytes_varlen 

Source
pub fn q12_batched_scratch_bytes_varlen(
    n: usize,
    total_tokens: usize,
    max_chunk_len: usize,
    top_k: usize,
    mrope: bool,
) -> usize
Expand description

Exact scratch footprint for a ragged Q12 batch.

total_tokens is the packed cu_seqlens total while max_chunk_len sizes the per-stream metadata slots. Uniform callers should use q12_batched_scratch_bytes. Keeping this separate prevents VARLEN admission from pessimistically charging every stream at the longest request’s length.