pub const DENSE_GEMV_BATCHM_DECODE_MAX_M: u32 = 8;Expand description
The band the batched GEMV is allowed to CLAIM on the decode paths: the MTP row dispatch and the BF16 lm_head arm.
🔴 Deliberately still 8, and NOT the same thing as the kernel’s MAX_M. Those two sites
pick between dense_gemv_bf16_batchm and a reassociating kernel (the pipelined /
tile GEMM), so the band’s upper edge decides which bits a decode of that width produces.
Widening the GEMV tier to 16 for prefill would silently move widths 9..=16 off the tile
GEMM they have always used — a numerics change on the MTP / DFlash γ>8 window, on a path
the prefill measurement says nothing about. Moving this edge needs its own A/B and its
own byte gate against the sealed decode reference; until then the decode band is frozen
where it was measured (+6 % at C=2, +24 % at C=4; NEGATIVE above 8 against the tile GEMM,
-14.4 % at C=16 — commits 84d5b763c / 78d276832).