decode_rollback_ring_slots

Function decode_rollback_ring_slots 

Source
pub fn decode_rollback_ring_slots(
    num_ssm_layers: usize,
    use_speculative: bool,
) -> DecodeRingDecision
Expand description

Decide the per-sequence decode-rollback ring depth.

use_speculative MUST be the same flag factory::build_model receives (--speculative || --dflash as plumbed by spark-server) at every call site, or preflight and allocation diverge.