Trait ProposerState

pub trait ProposerState: Send + Sync {
    // Required methods
    fn as_any(&self) -> &dyn Any;
    fn as_any_mut(&mut self) -> &mut dyn Any;
}

Expand description

Per-sequence state owned by a DraftProposer.

Stores KV cache, hidden states, or whatever the proposer needs across decode steps. Follows the same downcasting pattern as LayerState.

Required Methods§