generate

Function generate 

Source
pub fn generate(
    model: &dyn Model,
    prompt_tokens: &[u32],
    params: &SamplingParams,
) -> Result<GenerateResult>
Expand description

Generate response tokens from a prompt.

Runs prefill on the prompt tokens, then iteratively decodes up to params.max_tokens output tokens, stopping early on EOS or stop tokens.