pub fn generate_streaming<F>(
model: &dyn Model,
prompt_tokens: &[u32],
params: &SamplingParams,
on_token: F,
) -> Result<GenerateResult>Expand description
Generate response tokens with per-token callback.
Same as generate but calls on_token(token_id) after each token
is produced (including the first token from prefill). The callback
is synchronous — designed for the caller to send tokens through a
channel without pulling in an async runtime dependency.