Module sampler 

Token sampling strategies.

Phase 1: Greedy argmax (CPU-side D2H + argmax). Future: temperature, top-k, top-p, min-p, repetition penalty.

Structs

Sampler

Sampler that picks tokens from logits.

SamplingParams

Sampling parameters for a request.

Functions

apply_dry_penalty

DRY (Don’t Repeat Yourself) penalty. Ported from llama.cpp PR #9702.

apply_lz_penalty

Sampling pipeline: repetition_penalty → top-n-sigma → temperature → top-k → softmax → min-p → top-p → sample.

apply_penalties_and_bias

Apply repetition / presence / frequency / LZ / DRY penalties and per-token logit bias to logits IN PLACE, using token_history.

argmax_bf16

Legacy: argmax over BF16 values (still used by argmax_on_device fallback).

argmax_f32

Argmax over FP32 values stored as raw bytes (4 bytes per element, little-endian). First-index-wins, identical to argmax_first_wins_f32 — same two-pass shape, iterating the byte chunks directly so no Vec<f32> is materialised.

argmax_first_wins_f32

Argmax over an f32 slice with the strict-> FIRST-index-wins tie-break.

last_entropy

Read the most recent per-token entropy (nats).

low_entropy_token_count

Total tokens with entropy < 0.3 (potential degeneration).

sample_with_params

Convenience wrapper: sample without token history (no repetition penalty).

sample_with_params_history

sample_with_params_seeded

Core sampling pipeline with explicit seed control. seed overrides the RNG for deterministic sampling. None = thread_rng.

total_sampled_token_count

Total tokens sampled (for computing low-entropy ratio).