Module speculative 

Speculative decoding abstraction (SDD).

Defines the DraftProposer trait for speculative decoding strategies. MTP implements this first; EAGLE-3 can implement later without engine changes.

Re-exports

pub use ladder::mtp_ladder_disabled;

pub use ladder::mtp_ladder_drafts;

pub use ladder::mtp_max_seqs;

Modules

ladder

K-vs-batch ladder (task #35): per-step draft count as a function of the number of active sequences.

tree_shape

Spine+hedge draft-tree shapes for tree speculative decoding (Phase 1 of the tree-spec plan).

verify_key

Batched-verify graph key: the canonical depth→slot assignment and the key bytes derived from it. ONE ordering rule, shared by the scheduler that dispatches the batch (mtp_dcut::plan, mtp_step) and the model that builds the CUDA-graph cache key (verify_e2::verify_batched_graph_key).

Constants

EP_CMD_MTP_PROPOSE

EP worker command: run one MTP propose in lockstep with rank 0. Payload after the code: last_token, position, num_drafts (3 x u32).

Traits

DraftProposer

ProposerState

Per-sequence state owned by a DraftProposer.

Functions

draft_conf_tau

A draft token proposer for speculative decoding.

hidden_fingerprint

FNV-1a over a BF16 GPU row, for mtp_refeed_debug fingerprints.

mtp_accept_debug

ATLAS_MTP_ACCEPT_DEBUG (PRESENCE): per-BATCH-WIDTH acceptance telemetry.

mtp_catchup_enabled

Drafter catch-up feed on serial->speculative transitions (ATLAS_MTP_CATCHUP=1, staged off). During serial-decode stretches the scheduler rings the per-step final hiddens; on the next propose the gap rows are batch-fed into the drafter KV so it never runs stale. Wrong feeds cannot corrupt output (verification rejects bad drafts) — the stake is acceptance only, which is the flip gate’s metric. Force-off in multi-seq MTP mode (single-sequence ring; see mtp_multi_seq_mode).

mtp_ep_propose_enabled

Run the drafter on EVERY rank with the communicator, instead of rank-0-only with comm: None. DEFAULT ON since 2026-08-29; kill switch ATLAS_NO_MTP_EP_PROPOSE=1 restores the rank-0-only path.

mtp_multi_seq_mode

True when the MTP cap is raised above one sequence. The catchup ring (types.rs mtp_catchup_ring + meta), the refeed label convention, and the carry slot (mtp_carry.rs) are SINGLE-SEQUENCE structures — one ring, one label range, one slot. Running them with n concurrently-verifying sequences interleaves unrelated hiddens under one label space and breaks after_verify’s env-keyed trim contract (mtp_rows_to_trim). Disabling them process-wide when the cap > 1 is the only consistent option; a slot-keyed ring is recorded follow-up work (acceptance debt at n>1).

mtp_refeed_accepted_enabled

Re-feed ACCEPTED draft rows with the target’s TRUE hidden state (ATLAS_MTP_REFEED_ACCEPTED=1, default OFF). Requires ATLAS_MTP_CATCHUP=1.

mtp_refeed_debug

ATLAS_MTP_REFEED_DEBUG=1: fingerprint every hidden that enters and leaves the catch-up ring, so the pair-key -> hidden mapping can be read off the serve log instead of argued about. Costs a D2H of one hidden row (h * 2 bytes) plus a stream sync per event — NEVER enable it in a timed leg. See mtp_refeed_shift for why the fingerprints alone cannot falsify the mapping, and what they DO establish (the ring’s slot arithmetic and the pair-key bookkeeping round-trip).

mtp_refeed_shift

Deliberate off-by-N perturbation of the re-feed’s ring LABEL (ATLAS_MTP_REFEED_SHIFT, default 0 = the derived mapping).

shadow_topk

Shadow top-k draft instrumentation (ATLAS_MTP_SHADOW_TOPK=k, default 0 = off, clamp k ≤ 8). Observational only — token selection untouched. Each drafter forward_one D2H’s its logits (same ~200 µs the conf path pays) and logs the top-k candidate ids + softmax probs per position; the verify steps log the target argmaxes under the same gate. Joining the two offline yields the per-depth conditional top-k coverage that gates the tree-speculation build (Phase 0 of the tree-spec plan). Value-parsed, not presence-checked (=0 really is off).