DECODE_META_MAX_ROWS

Constant DECODE_META_MAX_ROWS 

Source
pub const DECODE_META_MAX_ROWS: usize = 128;
Expand description

Layout ceiling, checked at serve time (serve.rs). The metadata gaps themselves derive cleanly to any width; the binding constraints on this tip are downstream row consumers sized at 96+ rows:

  • the logits arena (sizes.rs) — derived max(96, rows+1) rows, where rows+1 covers the run_standard mixed path parking prefill logits at row padded_n;
  • the scratch block-table envelope (sizes.rs) — derived max(verify 96-row overlay, decode rows-row layout).

Batched-decode kernels are row-count parametric (grid.y = n, smem per CTA constant, split-K workspace derived from the pinned max batch), so 128 is a policy cap for the widths validated by the enterprise- concurrency campaign, not an smem wall.