pub fn draft_conf_tau() -> f32Expand description
A draft token proposer for speculative decoding.
The engine calls propose() after each target decode to get draft tokens,
then verifies them with the target model. after_verify() lets the
proposer trim state (e.g., KV cache) based on how many drafts were accepted.
Confidence floor for submitting drafts to verification
(ATLAS_MTP_DRAFT_CONF, default 0.0 = disabled). When the drafter’s
chain confidence (min top-1 softmax prob across the drafts of one
propose) is below this, the drafts are discarded and the next step
decodes serially — skipping a verify that would most likely reject.
Economics at K=1 on the 35B MoE: verify ≈ 35 ms for 1+acc tokens vs
decode+propose ≈ 21 ms for 1, so a draft is only worth verifying when
p(accept) ≳ 0.66 — the threshold to calibrate around. Staged OFF until
its measured A/B (same discipline as ATLAS_SNAP_EVICT_ALPHA).