pub fn mtp_refeed_accepted_enabled() -> boolExpand description
Re-feed ACCEPTED draft rows with the target’s TRUE hidden state
(ATLAS_MTP_REFEED_ACCEPTED=1, default OFF). Requires ATLAS_MTP_CATCHUP=1.
WHY. The MTP head is one module run autoregressively. Draft 1 consumes the
TARGET’s verified hidden (mtp_hidden_save); every later draft consumes the
drafter’s OWN single-block residual (mtp_head.rs, current_hidden = ctx.buffers.hidden_states()). The drafter KV row written for draft d >= 2
therefore pairs the right token with the WRONG hidden — and on ACCEPT that
row is kept forever: after_verify trims only REJECTED rows. So every
accepted draft permanently contaminates the drafter’s own context.
Measured on dgx2 (W4A4 27B, gate disarmed, seq_len ~10k, n=700/config):
unconditional per-position acceptance 0.660 -> 0.485 -> 0.407, i.e. the
FIRST autoregressive step costs x0.735 while the second costs only x0.838 —
the loss is concentrated exactly at the hidden-state handoff. Neither
existing lever touches it: ATLAS_MTP_CATCHUP=1 alone is bit-identical
(its ring is only written on SERIAL decode steps, and with the throughput
gate disarmed there are none), and dropping ATLAS_MTP_DRAFTER_PREFILL
costs only 0.017/0.030 (~1 sd).
WHAT THIS DOES. After a verify, the target’s true hidden for every accepted
position is sitting in the verify hidden buffer. Ring those hiddens under
the same label convention the serial path uses, and have after_verify
additionally drop the num_accepted - 1 accepted rows that were written
with a drafter hidden. The next propose’s catch-up feed then rebuilds
exactly those rows from the ring, with the TARGET’s hidden, through the
already-exercised catchup_drafter batch path. No new kernel, no new
state machine — it reuses the gap-fill machinery for a gap that was never
being detected.
SAFETY. A wrong feed cannot corrupt output: verification rejects bad drafts. The stake is acceptance only.
§STATUS 2026-07-21 (SUPERSEDES the earlier “refuted” note). STAGED OFF.
The earlier note claimed the pair-key -> hidden mapping was wrong, inferred from a sign reversal between a 67%-delivery and a 99%-delivery arm at n=700 (+0.021 -> −0.023 on p2_uncond). That inference is withdrawn. The two arms differed by only ~1.7 sd, neither was more than 1 sd from the baseline, and they are not paired samples (each arm emits different text).
The mapping has since been VERIFIED DIRECTLY, with dumped hidden
fingerprints (ATLAS_MTP_REFEED_DEBUG=1, FNV-1a over each BF16 row), on
dgx2 / W4A4 27B / nd=2 / gate disarmed:
| check | result |
|---|---|
ring D2D landed (fp_src == fp_dst) | 658 / 658 |
| fed hidden == live ring content at that label | 422 / 422 |
label == key + 1 and RoPE == key + 1 on every feed | always |
fp(ring[position]) == fp(mtp_hidden_save) at each propose | 302 / 304 (the 2 are a run’s first propose) |
feed(key k) == mtp_hidden_save at the propose whose position was k+1 | 93 / 93 |
The last row is the non-tautological one: it compares the hidden this
feature feeds for pair key k against the hidden the drafter’s own
forward_one consumed as target_hidden when it wrote pair key k —
two different code paths, bit-identical on every checkable case. So the
convention “ring label n holds hidden_{n−1}, hence pair key k reads label
k+1” is confirmed against an independently-exercised consumer.
What the earlier session DID find is real and is now fixed: the exclusive
0..num_accepted bound left one label unwritten per step, collapsing the
ring’s contiguous window (458 fed / 231 missed = 67%). The bound is now
0..=num_accepted on both K=3 and K=4 (K=4 matters because
mtp_rows_to_trim’s extra trim is K-agnostic — without a K=4 ring write,
nd=3 would drop accepted drafter rows with nothing rebuilding them).
§POWERED A/B (2026-07-21, dgx2): the pre-registered threshold is MET.
nd=2, gate disarmed, 16 documents x 8 turns, ~10k verify steps per arm
(with ATLAS_MTP_GATE_FORCE=1 the engine is bit-reproducible, so n rises
only with NEW CONTENT, never with repetitions).
| arm | n | p1 | p2_uncond | tokens/verify step |
|---|---|---|---|---|
| OFF | 10,400 | 0.6100 | 0.4182 | 1.882 |
| ON | 10,100 | 0.6262 | 0.4452 | 1.926 |
| delta | +0.016 (2.4 sd) | +0.027 (3.9 sd) | +2.3% |
Criterion, pre-registered before the run: p2_uncond up by ≥ 0.015 at
≥ 3 sd. Met. At n=700 — the sample that produced the earlier “refuted”
verdict — this same effect is ~1.0 sd, i.e. invisible. That verdict was a
power problem, not a mapping problem.
Caveat kept deliberately: the arms emit different text, so the binomial sd understates the true variance. Content is matched (identical documents and questions in both arms) but this is one measurement, not a replication. STAYS DEFAULT OFF pending the standard gates (C2 smoke, A 35B webserver_ok, B/D ST-995).
§SIZE IT AGAINST THE REAL PRIZE BEFORE SPENDING ANY MORE TIME HERE
This lever is small BY CONSTRUCTION: it repairs at most
num_accepted − 1 drafter KV history rows per step, while the measured
p1->p2 cliff happens WITHIN a single propose, where a history repair
cannot act at all. Two larger effects were measured the same night:
- The drafter’s own INPUT hidden at draft position >= 2 (dgx1’s
teacher-forced oracle probe,
ATLAS_MTP_ORACLE_P2): feeding draft 2 the TARGET’s true hidden instead of the MTP head’s own takes p2_cond 0.5265 -> 0.7196, McNemar z = +18.4, recovering 1.40x the p1−p2 gap. “Exposure bias” is refuted — the drafter is not mis-calibrated, it is fed the wrong vector. That is +0.193, about 7x this flag’s +0.027. - Drafter context blindness on WARM turns (dgx2): the drafter holds
only 142 KV rows at sequence position 10,098, because
try_mtp_prefill_captureno-ops whenever a prefill starts at a reused-prefix boundary and the drafter prompt-prefill is then skipped. Prefilling it on every turn measured +0.086 p1 / +0.101 p2_uncond / +10.2% accepted tokens per verify step at n ~ 10k per arm, of which a de-confounding pair (drafter coverage held at zero, prefix caching the only variable) attributes +0.079 p1 / +0.089 p2_uncond — 92% / 88% — to drafter coverage and the small remainder to warm restore.
Both dwarf this flag, and (2) also changes what this flag is worth: a drafter that can actually see the prompt is a different drafter. Build (2) first, then re-measure this.
Force-off in multi-seq MTP mode: the refeed label space is single-sequence
(see mtp_multi_seq_mode).