Skip to main content

Module chat_inference

Module chat_inference 

Source
Expand description

End-to-end Webizen-gated chat inference with retrieval, streaming, and provenance.

Structs§

ChatInferenceOptions
ChatInferenceResult

Functions§

clear_cancel_inference
Clears the cancellation flag.
is_inference_cancelled
true when inference-cancelled.
request_cancel_inference
Requests inference cancellation.
run_chat_inference_for_agent
Run a local turn using a named roster agent. A pinned model is activated on demand before inference; the lifecycle implementation owns any resident mapping replacement. This is a cold control-path operation and never runs inside the decode/evaluator hot path.
run_chat_inference_full
Runs full chat inference.
run_chat_inference_with_options
Runs chat inference with options.
stream_event_done
Done stream event.
stream_event_error
Error stream event.
stream_event_token
NDJSON stream events for Flutter FRB.
validate_axiom_preflight
Pre-flight axiom bounds check before KV prefill / orchestrator dispatch.