Skip to main content

Module experiment

Module experiment 

Source
Expand description

Experiment execution engine — wraps the existing run_bench harness with quality verification, thermal sampling, and structured result collection.

Each experiment produces an ExperimentResult containing:

  • BenchResult (latency, tok/s, phase split)
  • VerifiedTurn (post-turn quality checks)
  • Thermal snapshot (GPU temp/power before + after)
  • VRAM usage from the GPU context ledger
  • CBOR config snapshot for reproduction

Structs§

BenchResultSerde
Serializable subset of BenchResult (the original only derives Serialize).
ExperimentConfig
Configuration for one experiment run.
ExperimentResult
The full result of one experiment run.
PhaseSnapshotSerde
Serializable phase snapshot (the atomic counters are copied into plain u64s).
QualityScore
Quality score derived from VerifiedTurn checks.
ThermalSnapshot
Thermal reading before and after an experiment.

Functions§

append_experiment_jsonl
Append-only JSONL experiment log.
load_experiment_log
Load all experiment results from a JSONL log.
run_experiment
Run one experiment end-to-end: apply toggles, run benchmark, verify quality, sample thermal, collect results.
run_experiment_with_quality
Run an experiment with quality verification (uses decode_with_metrics to get the actual generated text, then runs verify_and_heal_turn on it).