Skip to main content

Module llm_testing

Module llm_testing 

Source
Expand description

LLM Testing Integration for CLI

Simple LLM model testing functionality for the CLI.

Structs§

ExploreCandidateResult
One explore-row: layout (+ optional toggle label) → measured decode-proxy tok/s + coherence.
ExploreReport
TestResult
Test result

Functions§

benchmark_model
CLI command to benchmark a single model
list_models
CLI command to list available models
parse_explore_layouts 🔒
run_activate_qwen4exp_native
Cold-activate the native C:/E: package layout. This is deliberately separate from generation: it proves the model can be opened with a bounded metadata read before a decode runner is allowed to touch weights.
run_app_profile
Show / set application profile (interactive | live-fast | batch).
run_benchmark_models
CLI command to benchmark models
run_comprehensive_llm_test
Comprehensive LLM capability test (load + inference in one session)
run_convert_gguf_to_p64
Convert a GGUF import file to native .p64 + canonical .q42 model metadata.
run_cuda_tc_microbench
Dense CUDA WMMA microbench (mode-independent; reports whether TC path is live).
run_decode_proxy
Short resident decode for passport child processes. Machine-readable line on stdout.
run_explore_pipeline
Phase-0 explorer: convert missing layouts if source is GGUF, measure decode-proxy, rank, write JSON.
run_gather_qwen4exp_ple
Compute the checkpoint’s PLE n-gram row IDs from decode-order token IDs, then gather exactly those rows from the C:-resident table. This is an execution seam, not a text-generation benchmark: it proves that runtime token history selects and consumes trained PLE rows without mapping the table or using a synthetic hash.
run_generate_report
CLI command to generate test report
run_gpu_capability_campaign
Probe which native GPU tiers this host has, measure the backend × mode decode matrix in child processes, and write machine-gpu-profile.json + apply-machine-gpu.ps1.
run_ground_check
Quant-graph dry-run: ground a prompt+answer pair (forces mode logic via unconditional ground).
run_hardware_passport
Probe / load HardwarePassport and print the ranked circuit matrix.
run_inference_mode
Print or set multi-mode inference approach (portable / cuda / quant-graph).
run_lab
Inference superiority lab instruments.
run_optimize_pipeline
Remarkable one-shot: optional passport + auto-layout convert + activate knobs.
run_path_select
Print (and optionally apply) the device-optimal inference path plan.
run_prepare_qwen4exp_native
Extract the Qwen4Exp PLE table to NVMe and create its compact HMC contract. The source GGUF stays in place; only its row-addressable PLE span is copied.
run_probe_qwen4exp_expert_tile
Exercise a promoted C: expert tile using a true E: token embedding. This is one cached gate-plane operator, not a complete MoE or text decode.
run_probe_qwen4exp_gdn
Execute the trained recurrent token mixer with its real learned state. The two state vectors are explicitly bounded and live only for this CLI invocation; no model payload is copied from E: or C: into them.
run_probe_qwen4exp_hyper
Execute one trained attention- or FFN-side Hyper-Connection mix using the source checkpoint’s own weight rows. This is a forward component probe; it deliberately does not substitute an attention/GDN or MoE result.
run_probe_qwen4exp_moe
Execute one true Qwen4Exp routed-MoE operator. This is an operator validation seam, not text generation: it uses a token embedding directly instead of a fully evolved preceding token-mixer state.
run_probe_qwen4exp_ple
Exercise the actual C:-resident PLE reader without mapping the PLE table, opening the source GGUF, or allocating a row dynamically.
run_probe_qwen4exp_ple_block
Execute the complete trained PLE residual insertion for one token. This is a physical C:/E: forward component check, not token generation: all later GDN/QSA, four-stream and MoE layers remain to be executed.
run_probe_qwen4exp_trunk
Verify the external trunk reader against the exact source identified by an HMC. Only the bounded GGUF metadata prefix is read; the embedding row is consumed through TrunkNvmeReader from E:.
run_promote_qwen4exp_expert
Explicit cold-path promotion of one selected expert. This is deliberately not part of ordinary activation: the caller must name a C: cache directory and budget before any new artifact is made.
run_seed_grounding
Seed quant-graph facts from bundled TSV / QUALIA_GROUNDING_FACTS.
run_test_models
CLI command to run comprehensive LLM model tests
run_validate_models
CLI command to validate models
run_verify_qwen4exp_native
Audit an existing Qwen4Exp HMC’s external source before activation.
test_single_model 🔒
Test a single model
validate_model
CLI command to validate model structure