pub trait LlmBackend {
// Required methods
fn embed(&self, texts: &[String]) -> Result<Vec<Vec<f32>>>;
fn generate(&self, system: &str, prompt: &str) -> Result<String>;
fn embed_dim(&self) -> u32;
fn model_id(&self) -> String;
}Expand description
A swappable language-model backend.
Required Methods§
Sourcefn embed(&self, texts: &[String]) -> Result<Vec<Vec<f32>>>
fn embed(&self, texts: &[String]) -> Result<Vec<Vec<f32>>>
Embed a batch of texts. Returns one vector per input, all of the same
dimensionality (embed_dim).
Sourcefn generate(&self, system: &str, prompt: &str) -> Result<String>
fn generate(&self, system: &str, prompt: &str) -> Result<String>
Free-form completion used for structured extraction. system sets the
instruction; prompt carries the chunk. Implementations should request
non-streaming output.
Dyn Compatibility§
This trait is dyn compatible.
In older versions of Rust, dyn compatibility was called "object safety".