mtp_refeed_accepted_enabled

Function mtp_refeed_accepted_enabled 

Source
pub fn mtp_refeed_accepted_enabled() -> bool
Expand description

Re-feed ACCEPTED draft rows with the target’s TRUE hidden state (ATLAS_MTP_REFEED_ACCEPTED=1, default OFF). Requires ATLAS_MTP_CATCHUP=1.

WHY. The MTP head is one module run autoregressively. Draft 1 consumes the TARGET’s verified hidden (mtp_hidden_save); every later draft consumes the drafter’s OWN single-block residual (mtp_head.rs, current_hidden = ctx.buffers.hidden_states()). The drafter KV row written for draft d >= 2 therefore pairs the right token with the WRONG hidden — and on ACCEPT that row is kept forever: after_verify trims only REJECTED rows. So every accepted draft permanently contaminates the drafter’s own context.

Measured on dgx2 (W4A4 27B, gate disarmed, seq_len ~10k, n=700/config): unconditional per-position acceptance 0.660 -> 0.485 -> 0.407, i.e. the FIRST autoregressive step costs x0.735 while the second costs only x0.838 — the loss is concentrated exactly at the hidden-state handoff. Neither existing lever touches it: ATLAS_MTP_CATCHUP=1 alone is bit-identical (its ring is only written on SERIAL decode steps, and with the throughput gate disarmed there are none), and dropping ATLAS_MTP_DRAFTER_PREFILL costs only 0.017/0.030 (~1 sd).

WHAT THIS DOES. After a verify, the target’s true hidden for every accepted position is sitting in the verify hidden buffer. Ring those hiddens under the same label convention the serial path uses, and have after_verify additionally drop the num_accepted - 1 accepted rows that were written with a drafter hidden. The next propose’s catch-up feed then rebuilds exactly those rows from the ring, with the TARGET’s hidden, through the already-exercised catchup_drafter batch path. No new kernel, no new state machine — it reuses the gap-fill machinery for a gap that was never being detected.

SAFETY. A wrong feed cannot corrupt output: verification rejects bad drafts. The stake is acceptance only.

§STATUS 2026-07-21 (SUPERSEDES the earlier “refuted” note). STAGED OFF.

The earlier note claimed the pair-key -> hidden mapping was wrong, inferred from a sign reversal between a 67%-delivery and a 99%-delivery arm at n=700 (+0.021 -> −0.023 on p2_uncond). That inference is withdrawn. The two arms differed by only ~1.7 sd, neither was more than 1 sd from the baseline, and they are not paired samples (each arm emits different text).

The mapping has since been VERIFIED DIRECTLY, with dumped hidden fingerprints (ATLAS_MTP_REFEED_DEBUG=1, FNV-1a over each BF16 row), on dgx2 / W4A4 27B / nd=2 / gate disarmed:

checkresult
ring D2D landed (fp_src == fp_dst)658 / 658
fed hidden == live ring content at that label422 / 422
label == key + 1 and RoPE == key + 1 on every feedalways
fp(ring[position]) == fp(mtp_hidden_save) at each propose302 / 304 (the 2 are a run’s first propose)
feed(key k) == mtp_hidden_save at the propose whose position was k+193 / 93

The last row is the non-tautological one: it compares the hidden this feature feeds for pair key k against the hidden the drafter’s own forward_one consumed as target_hidden when it wrote pair key k — two different code paths, bit-identical on every checkable case. So the convention “ring label n holds hidden_{n−1}, hence pair key k reads label k+1” is confirmed against an independently-exercised consumer.

What the earlier session DID find is real and is now fixed: the exclusive 0..num_accepted bound left one label unwritten per step, collapsing the ring’s contiguous window (458 fed / 231 missed = 67%). The bound is now 0..=num_accepted on both K=3 and K=4 (K=4 matters because mtp_rows_to_trim’s extra trim is K-agnostic — without a K=4 ring write, nd=3 would drop accepted drafter rows with nothing rebuilding them).

§POWERED A/B (2026-07-21, dgx2): the pre-registered threshold is MET.

nd=2, gate disarmed, 16 documents x 8 turns, ~10k verify steps per arm (with ATLAS_MTP_GATE_FORCE=1 the engine is bit-reproducible, so n rises only with NEW CONTENT, never with repetitions).

armnp1p2_uncondtokens/verify step
OFF10,4000.61000.41821.882
ON10,1000.62620.44521.926
delta+0.016 (2.4 sd)+0.027 (3.9 sd)+2.3%

Criterion, pre-registered before the run: p2_uncond up by ≥ 0.015 at ≥ 3 sd. Met. At n=700 — the sample that produced the earlier “refuted” verdict — this same effect is ~1.0 sd, i.e. invisible. That verdict was a power problem, not a mapping problem.

Caveat kept deliberately: the arms emit different text, so the binomial sd understates the true variance. Content is matched (identical documents and questions in both arms) but this is one measurement, not a replication. STAYS DEFAULT OFF pending the standard gates (C2 smoke, A 35B webserver_ok, B/D ST-995).

§SIZE IT AGAINST THE REAL PRIZE BEFORE SPENDING ANY MORE TIME HERE

This lever is small BY CONSTRUCTION: it repairs at most num_accepted − 1 drafter KV history rows per step, while the measured p1->p2 cliff happens WITHIN a single propose, where a history repair cannot act at all. Two larger effects were measured the same night:

  1. The drafter’s own INPUT hidden at draft position >= 2 (dgx1’s teacher-forced oracle probe, ATLAS_MTP_ORACLE_P2): feeding draft 2 the TARGET’s true hidden instead of the MTP head’s own takes p2_cond 0.5265 -> 0.7196, McNemar z = +18.4, recovering 1.40x the p1−p2 gap. “Exposure bias” is refuted — the drafter is not mis-calibrated, it is fed the wrong vector. That is +0.193, about 7x this flag’s +0.027.
  2. Drafter context blindness on WARM turns (dgx2): the drafter holds only 142 KV rows at sequence position 10,098, because try_mtp_prefill_capture no-ops whenever a prefill starts at a reused-prefix boundary and the drafter prompt-prefill is then skipped. Prefilling it on every turn measured +0.086 p1 / +0.101 p2_uncond / +10.2% accepted tokens per verify step at n ~ 10k per arm, of which a de-confounding pair (drafter coverage held at zero, prefix caching the only variable) attributes +0.079 p1 / +0.089 p2_uncond — 92% / 88% — to drafter coverage and the small remainder to warm restore.

Both dwarf this flag, and (2) also changes what this flag is worth: a drafter that can actually see the prompt is a different drafter. Build (2) first, then re-measure this.

Force-off in multi-seq MTP mode: the refeed label space is single-sequence (see mtp_multi_seq_mode).