mtp_drafter_prefill_enabled

Function mtp_drafter_prefill_enabled 

Source
pub fn mtp_drafter_prefill_enabled(levers: &ModelLevers) -> bool
Expand description

Drafter context prefill — ON by default, cached once.

The target prefill captures every position’s final-layer hidden and the MTP drafter’s KV cache is batch-prefilled over the whole prompt before the first propose(), mirroring vLLM’s MTP proposer prefill. The drafter’s KV entries are pure functions of its input pair (embed(token_{i+1}), target_hidden_i) — a single-layer drafter’s K/V do not depend on its own attention outputs — so the prefill needs only the fc + k/v projections + norms + RoPE + cache write, no attention pass.

Policy, including the kill switch and the coupling to the cross-turn carry (which this half is useless without), lives in crate::model::drafter_context — the single source of truth.