Skip to main content

LlmBackend

Trait LlmBackend 

Source
pub trait LlmBackend {
    // Required methods
    fn embed(&self, texts: &[String]) -> Result<Vec<Vec<f32>>>;
    fn generate(&self, system: &str, prompt: &str) -> Result<String>;
    fn embed_dim(&self) -> u32;
    fn model_id(&self) -> String;
}
Expand description

A swappable language-model backend.

Required Methods§

Source

fn embed(&self, texts: &[String]) -> Result<Vec<Vec<f32>>>

Embed a batch of texts. Returns one vector per input, all of the same dimensionality (embed_dim).

Source

fn generate(&self, system: &str, prompt: &str) -> Result<String>

Free-form completion used for structured extraction. system sets the instruction; prompt carries the chunk. Implementations should request non-streaming output.

Source

fn embed_dim(&self) -> u32

Embedding dimensionality, if known ahead of a call (0 = discover on first embed).

Source

fn model_id(&self) -> String

Identifier recorded in the container manifest (e.g. ollama/nomic-embed-text).

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§