Skip to main content

decode_sampled_blocking

qualia_core_db::inference::inference_bench

Function decode_sampled_blocking 

Source 
pub fn decode_sampled_blocking(
    model_path: &str,
    prompt: &str,
    decode_tokens: u32,
    cfg: SamplerConfig,
) -> Result<(String, f64), String>
Expand description

W2: decode with the exact CPU sampler installed for the duration of the call. Returns (text, tok/s). Restores greedy (None) afterwards so it never leaks into other tests.