Expand description
Experiment execution engine — wraps the existing run_bench harness with
quality verification, thermal sampling, and structured result collection.
Each experiment produces an ExperimentResult containing:
BenchResult(latency, tok/s, phase split)VerifiedTurn(post-turn quality checks)- Thermal snapshot (GPU temp/power before + after)
- VRAM usage from the GPU context ledger
- CBOR config snapshot for reproduction
Structs§
- Bench
Result Serde - Serializable subset of
BenchResult(the original only derivesSerialize). - Experiment
Config - Configuration for one experiment run.
- Experiment
Result - The full result of one experiment run.
- Phase
Snapshot Serde - Serializable phase snapshot (the atomic counters are copied into plain u64s).
- Quality
Score - Quality score derived from
VerifiedTurnchecks. - Thermal
Snapshot - Thermal reading before and after an experiment.
Functions§
- append_
experiment_ jsonl - Append-only JSONL experiment log.
- load_
experiment_ log - Load all experiment results from a JSONL log.
- run_
experiment - Run one experiment end-to-end: apply toggles, run benchmark, verify quality, sample thermal, collect results.
- run_
experiment_ with_ quality - Run an experiment with quality verification (uses
decode_with_metricsto get the actual generated text, then runsverify_and_heal_turnon it).