pub fn q2_dequant_scratch_bytes(config: &ModelConfig) -> usizeExpand description
Bytes for the native keep-packed Q2_0 prefill transient-dequant scratch: the
LARGEST keep-packed projection [N, K] expanded to BF16 (2 bytes/elem). The
prefill dequant writes N*K BF16 elements into this buffer, which is then
consumed by the same-stream GEMM and reused by the next projection.
Every keep-packed projection has exactly one dimension equal to
hidden_size (FFN gate/up [inter, h], FFN down [h, inter], attention
q/k/v/o [·, h] or [h, ·], fused GDN in_proj_qkvz [qkvz, h]), so
N*K = max_other_dim * hidden — an EXACT bound, not an over-estimate, for
the covered families (the k/v and gated-q terms are safe upper bounds).
Independent of batch tokens (this dequants the WEIGHT, not activations).