Skip to main content

awq_sweep_blocking

qualia_core_db::inference::inference_bench

Function awq_sweep_blocking 

Source 
pub fn awq_sweep_blocking(
    gguf_path: &str,
    alphas: &[f32],
    max_tok: usize,
    quant: FfnQuant,
) -> Result<(f64, Vec<(f32, f64, f64)>), String>
Expand description

AWQ α-sweep on the ternary FFN (AWQ steps 1–3 end to end): capture activation salience from the Q8 reference at gguf_path, then for each α compile an AWQ-scaled ternary .q42 (compile_gguf_to_q42_ternary_ffn_awq), evaluate its perplexity + unique-word coherence, and return (reference_ppl, [(alpha, ppl, uniq)]). α=0.0 is plain ternary (the baseline). max_tok caps tokens/passage to bound the sweep. Honest: this measures whether AWQ rescues ternary — it does not assume it does. Needs a GPU.