max_dsa_context

Function max_dsa_context 

Source
pub fn max_dsa_context(cfg: &Glm5NextDsaConfig) -> usize
Expand description

Longest context DSA can select over, in tokens.

🟢 This used to be a KERNEL limit — dsa_topk_pools bitonic-sorted the whole padded pool axis in shared memory, capping the serve at 4,096 pools = 16,384 tokens whatever --max-seq-len claimed (ANOMALIES A62). The select is tiled now, so the kernel imposes nothing and this is purely an ALLOCATION decision: how many rows of indexer cache each sequence reserves, which is --max-seq-len rounded down to a whole pool.

🪤 It is charged per sequence per DSA layer at 2 · index_head_dim BF16 + 1 B a token — 5,643 B/token across GLM-5.3’s 11 text DSA layers, and the indexer is REPLICATED, so EP does not halve it. Glm5NextSkeleton::state_budget must carry the same number or the serve allocates past its own --gpu-memory-utilization (the A59 class of cliff).