generate_speculative

spark_model::engine

Function generate_speculative 

Source 
pub fn generate_speculative(
    model: &dyn Model,
    prompt_tokens: &[u32],
    params: &SamplingParams,
    num_drafts: usize,
) -> Result<GenerateResult>
Expand description

Generate with speculative decoding (MTP).

Delegates the speculative decode loop to model.generate_speculative(), which has access to GPU/buffers needed for the MTP proposer.