Module inference_eval 

W1 — in-project quality oracle (no external libs): perplexity, KL-divergence, and a coherence ratio, measured by teacher-forcing an eval corpus through the engine and comparing a candidate model’s output distribution against a higher-fidelity reference’s.

Honesty note. ΔPPL and KL here are relative (candidate vs reference on identical text), so the comparison is self-consistent regardless of who authored the corpus. They measure engineering fidelity — does quantization preserve the reference model’s behaviour — not whether the model’s outputs are true or whether any direction is correct. The “reference” is the highest-fidelity model on disk (Q8_0 unless a real F16 is supplied); it is labelled as such, never silently called “FP16”.

This module is pure math + thresholds + a corpus loader; the engine forward pass that feeds it lives in the bench harness. Metric paths take slices and return scalars (heap only in the loader).

Structs

QualityVerdict

The three-tier verdict against the gate. hard_pass (coherence) must hold; ΔPPL/KL are evidence.

Constants

MAX_AVG_KL

Max average per-token KL-divergence (reference ‖ candidate) over the corpus.

MAX_DELTA_PPL

Max relative perplexity increase vs the reference (soft evidence).

MIN_UNIQ_WORD

Min unique-word ratio on a generation loop (hard gate — eliminates repetition collapse).

Functions

add_ppl

Add one position’s NLL (nats) + token to the accumulators.

delta_ppl

Relative perplexity increase of candidate over reference: (cand - ref) / ref.

kl_divergence

KL-divergence D(reference ‖ candidate) between the two softmax distributions, in nats. = Σ p_ref · (log p_ref − log p_cand), computed via log-softmax for stability. Non-negative.

load_corpus

Load the eval corpus (one passage per line; blank lines dropped). Searches the standard roots so it works from the crate dir or the repo root, mirroring the model/results lookups in the bench.

log_sum_exp

Numerically-stable log-sum-exp over logits (f64 accumulation).

perplexity

Perplexity from a summed NLL (nats) over n_tokens: exp(total_nll / n_tokens).

ppl_snapshot

Current (total_nll, token_count) snapshot.

reset_ppl

Reset the teacher-forced PPL accumulators before an eval pass.

token_nll

Negative log-likelihood (nats) of target under softmax(logits) = logsumexp - logits[target].

unique_word_ratio

Unique-word ratio (coherence proxy): distinct whitespace tokens / total. Repetition collapse → low.