q12_batched_scratch_bytes

Function q12_batched_scratch_bytes 

Source
pub fn q12_batched_scratch_bytes(
    n: usize,
    chunk_len: usize,
    top_k: usize,
    mrope: bool,
) -> usize
Expand description

Exact scratch footprint (bytes) of the Q12 kernel-batched prefill staging for n streams of chunk_len tokens each. SSOT for both scratch sizing (BufferSizes::from_config) and the pre-flight eligibility check (check_kernel_batched_eligible), so the two can never disagree about whether a batch fits. Mirrors the staging layout in batch_kernel.rs (MoE topk area + N per-stream meta blocks) and stage_batched.rs (stacked positions ×(3 if MRoPE) + slots + block/seq_len pointer arrays) plus the per-SSM-layer h_state_ptrs JIT slot.