pub fn marconi_snapshot_slots(
requested: usize,
prefix_caching_active: bool,
) -> MarconiSlotDecisionExpand description
Number of Marconi SSM-snapshot slots to RESERVE and ALLOCATE.
Two call sites MUST agree on this number, exactly as they must for the decode-rollback ring above, or a serve either under-reserves (runtime CUDA alloc failure after weights load) or over-reserves (preflight refuses a configuration the runtime could fund):
spark-serverpreflight_reserve— sizes the pre-load GPU reserve;TransformerModel::new(impl_a1.rs) — allocatesSsmSnapshotPool.
WHY a gate exists. The Marconi region’s ONLY consumer is the prefix
cache: a slot is written by prefill_b_save_checkpoint /
insert_*_snapshot and can only ever be READ BACK through a prefix-cache
lookup that returns an ssm_snapshot id (prefix_cache.rs, “SSM state
snapshot ID at the deepest matched node (Marconi caching)”). Without
--enable-prefix-caching, build_prefix_cache installs NoPrefixCaching
— no radix tree exists, no lookup can ever produce a snapshot id, and
every reserved slot is unreachable for the life of the process. Yet
--ssm-cache-slots defaults to 16 and was sized independently of the
flag, so a serve with prefix caching disabled still reserved
16 × num_ssm_layers × (h_state + conv_state) bytes that nothing can
restore from.
Measured on GLM-5.3-Flash NVFP4, 2× GB10, TP=2 EP=2, K=3, batch 1,
GMU 0.90: 2380 MiB per rank — 16 slots × 34 KDA layers ×
(h 4.000 MiB + conv 0.375 MiB). Both widths are FP32 by construction
(ModelConfig::ssm_h_state_bytes / ssm_conv_state_bytes each end in
* 4), and --ssm-h-dtype f16-pool is opt-in, so the FP32 figure is
what an ordinary serve reserves AND allocates: SsmStatePool reads the
same two accessors (ssm_pool.rs:182), so reserve and residency agree.
Confirmed by a paired A/B, same session, 90 s apart, identical flags
(2 131072 1 0.90): post-load requirement 13.58 → 11.25 GB, a
2.33 GB drop that matches 2380 MiB exactly. The gated default now needs
precisely what the same image required only when an operator passed
--ssm-cache-slots 0 by hand (ANOMALIES A68).
🪤 GLM53-MEMORY-LEDGER-20260830.md §2/§4 records this region as
“16 slots × 74.4 MB = 1190 MB”. That is the FP16-width arithmetic
(h 2.000 + conv 0.1875 MiB/layer) and is exactly half; the same halving
applies to its “SSM live state pool 1 slot × 34 layers = 74 MB” row.
Trust the FP32 figure — it is what the code allocates and what the live
A/B measured.
This is the same defect class the decode ring above already fixed: a pool reserved unconditionally while nothing could reach it.
Nothing degrades when the slots are dropped. prefill_b_save_checkpoint
early-returns on !ssm_snapshots.is_enabled(), so there is no work and
no warning spam on the prefill path; the only user-visible difference is
that prefix-cache hits would recompute SSM state — and with the cache
inactive there are no hits.
Env contract (read HERE and nowhere else):
ATLAS_SSM_MARCONI_FULL(PRESENCE, house convention —=0is NOT “off”): restore the old unconditional reservation. Accounting-safe over-reserve; the kill switch for this diet.