Skip to main content

Module inference_backend

Module inference_backend 

Source
Expand description

Persisted inference backend preference (Local / Remote / Hybrid / Ollama).

Local (GGUF in-process) is the primary Qualia engine. Ollama is an explicit opt-in harness for when native inference is not ready or the principal wants a local Ollama endpoint for chat / ETL prep.

Structs§

InferenceBackendSettings

Functions§

backend_label
backend label.
load_inference_backend_settings
Loads inference-backend-settings.
save_inference_backend_settings
Saves inference-backend-settings.
use_ollama_harness
True when chat should use the Ollama harness instead of LocalLlmAgent.