Struct LocalLlmAgent 

pub struct LocalLlmAgent {
    pub agent_did: String,
    pub backend: AgentBackend,
    pub memory_used_bytes: AtomicU64,
    /* private fields */
}

The concrete local inference agent. Uses a mock inference path for now; swap infer_local_model for an actual llama.cpp FFI call.

Fields

agent_did: String

backend: AgentBackend

memory_used_bytes: AtomicU64

Implementations

impl LocalLlmAgent

pub fn infer_local_model_streaming<F: FnMut(String) + Send + 'static>( &self, prompt: &str, graph_context: &str, on_token: Option<F>, ) -> (String, Vec<u64>, u32, Option<NQuin>)

Phase 8: Bifurcated Compute — SPSC Wait-Free Intercept.

On native targets: loads the GGUF model, tokenises the prompt, and runs an autoregressive decode loop via QTensorEngine::dispatch_fused_transformer_block. Logit summaries flow from the LLM engine thread to the Webizen Sentinel (this thread) over a wait-free SPSC ring. The Sentinel may inject DenyRollback for real governance signals; the old IEEE-754 0x99 mantissa check was removed (it fired randomly ~1/256 tokens and corrupted the stream).

On WASM / non-local backends: falls through to the original mock path. Run local inference, optionally streaming decoded text deltas to on_token.

impl LocalLlmAgent

pub fn new(agent_did: impl Into<String>, model_path: impl Into<String>) -> Self

pub fn with_local_backend( agent_did: impl Into<String>, backend: AgentBackend, ) -> Self

Construct an agent with a fully specified backend (e.g. catalog multimodal profile).

pub fn attach_lora_adapters(&self, adapter_dir: impl Into<PathBuf>)

Attach a LoRA adapter directory to this agent.

Adapters are loaded lazily on the first prompt that triggers a domain switch. The directory must contain *.lora files named after ContextType::adapter_filename() (e.g. medical_v1.lora).

pub fn attach_lora_adapters_with_dims( &self, adapter_dir: impl Into<PathBuf>, n_in: usize, n_out: usize, )

Attach a LoRA manager pre-configured with expected embedding dimensions.

n_in should match the model’s embedding dimension (e.g. 4096 for 7B models).

pub fn detach_lora_adapters(&self)

Remove the LoRA manager and revert to base-model-only inference.

pub fn warm_lora_for_prompt(&self, prompt: &str)

Detect context from prompt and pre-warm the LoRA adapter cache.

Call this before a batch of related prompts to avoid cold-load latency on the first inference.

pub fn active_lora_context(&self) -> Option<ContextType>

Return the currently active LoRA context type, if any.

pub fn configure_sieve_lex(&self, path: impl Into<String>)

Wire the .q42.lex sidecar used to populate FSM sieve masks at inference time.

pub fn agent_did_hash(&self) -> u64

impl LocalLlmAgent

pub fn validate_intent_frame(&self, frame: &AgentIntentFrame) -> WebizenVerdict

Zero-allocation pre-flight path for Core 1 (no active_profile heap lookup).

Trait Implementations

impl AgentRuntime for LocalLlmAgent

fn backend(&self) -> &AgentBackend

Returns the configured backend variant.

fn agent_did(&self) -> &str

Returns the name/DID of this agent instance for audit purposes.

fn validate_intent(&self, intent: &AgentIntent) -> WebizenVerdict

Submits an intent to the Webizen for pre-flight validation. This MUST be called before infer. Callers must not proceed if the verdict is Deny.

fn infer( &self, prompt: &str, graph_context: &str, ) -> Result<AgentOutput, AgentError>

Runs inference on the given prompt and graph context. graph_context is a serialised sub-graph slice provided by the Webizen.

fn validate_output(&self, output: &AgentOutput) -> WebizenVerdict

Submits the LLM output to the Webizen for post-flight grounding check. The Webizen verifies provenance citations exist in the live graph.

fn memory_budget_remaining(&self) -> u64

Returns remaining memory budget in bytes.