Skip to main content

Module lab

Module lab 

Source
Expand description

Inference superiority lab — instruments for evidence-based methodology.

Plan: docs/plans/inference-superiority-lab-and-toolset-plan.md

Native-only. No Ollama product dependency; optional external A/B via CLI script.

Re-exports§

pub use ablate::run_ablation_matrix;
pub use audit_path::audit_hot_path;
pub use audit_path::HotPathAudit;
pub use auto_improve::default_search_space;
pub use auto_improve::run_auto_improve;
pub use auto_improve::LabConfig;
pub use auto_improve::LockInPackage;
pub use auto_improve::TrialResult;
pub use device_roof::calibrate_device_roof;
pub use micro::run_q4k_soa_microbench;
pub use operator_promotion::CapabilityMatrix;
pub use operator_promotion::OperatorPromotionReceipt;
pub use operator_promotion::PromotionError;
pub use operator_promotion::PromotionStatus;
pub use operator_promotion::SupportLevel;
pub use timeline::run_decode_timeline;
pub use campaign::run_optimization_campaign;
pub use experiment::run_experiment;
pub use experiment::run_experiment_with_quality;

Modules§

ablate
audit_path
Hot-path wiring auditor — makes unfinished integration visible.
auto_improve
campaign
config_space
device_roof
Device peak calibration — real roofline anchors (plan L1.2).
experiment
Experiment execution engine — wraps the existing run_bench harness with quality verification, thermal sampling, and structured result collection.
experiment_log
hypothesis
micro
Isolated Q4_K SoA GEMV microbench (CPU oracle + optional CUDA).
operator_compare
Canonical comparison execution with host-specific checkpoint discovery tests.
operator_metrics
Compatibility forwarding to canonical operator experiment receipts.
operator_promotion
Operator promotion receipts and capability support matrix (W9: EOS-090).
pareto
search
timeline
Decode timeline: host phase counters + optional GPU profile + tok/s.

Structs§

AblationRow
AutoImproveConfig
BatchComparisonResult
Batch scaling measurement row for EOS-042.
BeliefGraph
The typed belief graph.
BenchResultSerde
Serializable subset of BenchResult (the original only derives Serialize).
CampaignConfig
Configuration for an optimization campaign.
CampaignReport
The result of an optimization campaign.
Configuration
A concrete configuration: one point in the space.
ConfigurationSpace
A configuration space: named parameters with definitions.
DecodeTimeline
DeviceRoof
ExperimentConfig
Configuration for one experiment run.
ExperimentResult
The full result of one experiment run.
ExperimentRun
FfnBlockWeights
SwiGLU FFN block weights in packed GGML Q4_K format.
Hypothesis
A hypothesis: a testable claim about an optimization strategy.
KnnSurrogate
A simple k-nearest-neighbor surrogate model for the objective function. Predicts the objective at a new point as the distance-weighted average of the k nearest observed points. This is a lightweight alternative to a full Random Forest (SMAC3) or Gaussian Process, suitable for small budgets.
MicrobenchResult
OperatorExperimentReceipt
Structured lab experiment receipt recording operator conversion, execution, and fidelity.
OperatorMemoryBreakdown
Memory footprint breakdown across separate memory domains.
ParetoFrontier
A Pareto frontier: the set of non-dominated points from a collection of experiments.
ParetoPoint
The six Pareto dimensions. “Lower is better” dimensions are negated internally so that “higher is always better” in the dominance comparison.
PhaseSnapshotSerde
Serializable phase snapshot (the atomic counters are copied into plain u64s).
QualityScore
Quality score derived from VerifiedTurn checks.
SearchEngine
The search engine: combines Sobol initial design with k-NN + EI Bayesian optimization.
SobolSequence
A Sobol quasi-random sequence generator for uniform coverage of the [0,1]^d unit hypercube. Uses the Joe & Kuo direction numbers (compact implementation).
ThermalSnapshot
Thermal reading before and after an experiment.
TrackAndStopBandit
A multi-armed bandit using the Track-and-Stop algorithm for best-arm identification. Given a small set of pre-selected configurations, it adaptively allocates samples to identify the best one with statistical rigor.

Enums§

ApplicationProfileWeight
Score a result against an application profile to select the “best” point from the Pareto frontier.
ExperimentVerdict
The verdict of an experiment on a hypothesis.
ParameterDef
One parameter definition in the configuration space.
ParameterValue
A concrete parameter value.

Constants§

CSV_HEADER

Functions§

append_experiment_jsonl
append_run_csv
Append one run; create file with header if missing.
compare_batch_scaling
Run multi-batch scaling comparison across batch sizes [1, 2, 4, 8] (EOS-042).
compare_single_tensor
Run single tensor comparison and return an experiment receipt (EOS-041).
default_toggle_space
Build a default configuration space for inference toggle optimization. This covers the main toggle parameters that ExperimentConfig::apply_toggles supports.
evaluate_verdict
Determine the verdict of an experiment against a hypothesis. This compares the experiment result against a baseline.
execute_ffn_existing
Execute a full SwiGLU FFN block: down(silu(gate(x)) * up(x)) using existing Q4_K dequant.
execute_ffn_lookup
Execute a full SwiGLU FFN block using the new executable operator lookup runtime.
expected_improvement
Expected Improvement (EI) acquisition function. EI(x) = (mu - f_best) * Phi(Z) + sigma * phi(Z) where Z = (mu - f_best) / sigma. Higher EI = more promising to try.
format_lockin_summary
Renders a lock-in package summary.
load_experiment_log
save_campaign_report
Save a campaign report to a JSON file.