pub fn per_sequence_state_bytes(
config: &ModelConfig,
max_seq_len: usize,
spec_on: bool,
) -> Result<PerSequenceState>Expand description
Per-sequence owned device state for config, at a context of max_seq_len tokens.
Returns zeros for any model that owns no per-sequence device state outside the pools —
which is every model except GLM-5.3 today. spec_on gates the proposer term: with
speculation off no proposer is constructed and no proposer state is ever allocated.