pub const DECODE_META_MAX_ROWS: usize = 128;Expand description
Layout ceiling, checked at serve time (serve.rs). The metadata gaps
themselves derive cleanly to any width; the binding constraints on this
tip are downstream row consumers sized at 96+ rows:
- the logits arena (
sizes.rs) — derivedmax(96, rows+1)rows, whererows+1covers the run_standard mixed path parking prefill logits at rowpadded_n; - the scratch block-table envelope (
sizes.rs) — derivedmax(verify 96-row overlay, decoderows-row layout).
Batched-decode kernels are row-count parametric (grid.y = n, smem per CTA constant, split-K workspace derived from the pinned max batch), so 128 is a policy cap for the widths validated by the enterprise- concurrency campaign, not an smem wall.