Skip to main content

Module sampler

Module sampler 

Source
Expand description

W2 — exact CPU sampling chain for decode.

The native decode loop selects tokens by pure greedy argmax, which is prone to the repetition collapse documented across the bench probes. This module adds a full, llama.cpp-compatible sampling chain that runs on the CPU over the full logit vector (read back once per token — acceptable at the W1 one-fence-per-token cost):

repetition/frequency/presence penalties → temperature → top-k → top-p → seeded draw

Design invariants:

  • Greedy is a hard short-circuit. temperature <= 0 returns the argmax BEFORE any penalty or filter is applied, so greedy decode is bit-identical to the pre-W2 path and the a1a/a1c/a1d guarantees are untouched.
  • Deterministic. The draw uses a self-contained SplitMix64 PRNG seeded from the config; the same seed + same logits + same context reproduce the same token, on native and wasm (no rand, no float transcendentals in the RNG, no platform entropy).
  • Exact, not top-K-approximated. The chain runs over the whole vocabulary; top-k / top-p are applied as masks, never as a lossy pre-reduction on the GPU.
  • Pure + zero-GPU. Everything here is testable without a device or a model.

Structs§

SamplerConfig
Sampling parameters. temperature <= 0.0 ⇒ greedy argmax (all other fields ignored).
SamplerState
Stateful sampler: owns the PRNG stream so successive tokens advance it deterministically.