pub struct CpuWasmEngine { /* private fields */ }Expand description
Cold-built, allocation-stable CPU decode plan.
Implementations§
Source§impl CpuWasmEngine
impl CpuWasmEngine
Sourcepub fn ingest_token(
&mut self,
token_id: u32,
position: usize,
) -> Result<(), CpuWasmError>
pub fn ingest_token( &mut self, token_id: u32, position: usize, ) -> Result<(), CpuWasmError>
Populate the KV cache for one prompt token without wasting a vocabulary projection. This is the CPU-WASM prefill primitive.
Sourcepub fn run_token(
&mut self,
token_id: u32,
position: usize,
) -> Result<CpuWasmStep, CpuWasmError>
pub fn run_token( &mut self, token_id: u32, position: usize, ) -> Result<CpuWasmStep, CpuWasmError>
Execute one complete autoregressive transformer step without allocation.
Source§impl CpuWasmEngine
impl CpuWasmEngine
Sourcepub fn new(model: Arc<[u8]>) -> Result<Self, CpuWasmError>
pub fn new(model: Arc<[u8]>) -> Result<Self, CpuWasmError>
Loads a GGUF model blob onto the WASM-CPU engine with the default context size.
Sourcepub fn new_with_context(
model: Arc<[u8]>,
max_context: usize,
) -> Result<Self, CpuWasmError>
pub fn new_with_context( model: Arc<[u8]>, max_context: usize, ) -> Result<Self, CpuWasmError>
Loads a GGUF model with an explicit max_context.
Sourcepub fn tokenizer(&self) -> &GgufTokenizer
pub fn tokenizer(&self) -> &GgufTokenizer
The model’s GGUF tokenizer.
Sourcepub fn max_context(&self) -> usize
pub fn max_context(&self) -> usize
Configured context-window length.
Sourcepub fn working_set_bytes(&self) -> usize
pub fn working_set_bytes(&self) -> usize
Current working-set size.
Auto Trait Implementations§
impl Freeze for CpuWasmEngine
impl RefUnwindSafe for CpuWasmEngine
impl Send for CpuWasmEngine
impl Sync for CpuWasmEngine
impl Unpin for CpuWasmEngine
impl UnsafeUnpin for CpuWasmEngine
impl UnwindSafe for CpuWasmEngine
Blanket Implementations§
§impl<S, A> Aggregate<Result<S, Error>> for Awhere
A: Aggregate<S>,
impl<S, A> Aggregate<Result<S, Error>> for Awhere
A: Aggregate<S>,
Aggregate shares in an MPC protocol.
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more