pub struct Session { /* private fields */ }Implementations§
Source§impl Session
impl Session
Sourcepub fn attention(
&mut self,
layer: usize,
hidden: &mut [f32],
) -> Result<(), SessionError>
pub fn attention( &mut self, layer: usize, hidden: &mut [f32], ) -> Result<(), SessionError>
Explicit lab operations do not commit a whole model token. Native decode must use step/step_controlled so its position advances exactly once.
pub fn ffn( &mut self, layer: usize, hidden: &mut [f32], ) -> Result<(), SessionError>
pub fn forward( &mut self, range: Range<usize>, hidden: &mut [f32], ) -> Result<usize, SessionError>
pub fn output_norm(&mut self, hidden: &mut [f32]) -> Result<(), SessionError>
pub fn argmax(&mut self, hidden: &[f32]) -> Result<(u32, f32), SessionError>
Source§impl Session
impl Session
pub fn plan(&self) -> &Arc<Plan> ⓘ
pub fn is_valid(&self) -> bool
Sourcepub fn new(plan: Arc<Plan>, byte_budget: usize) -> Result<Self, SessionError>
pub fn new(plan: Arc<Plan>, byte_budget: usize) -> Result<Self, SessionError>
The byte budget covers newly allocated state/scratch and owner metadata. Immutable admitted weights remain owned by the separately budgeted plan.
pub fn reserved_bytes(&self) -> usize
Sourcepub fn kv_cache_bytes(&self) -> usize
pub fn kv_cache_bytes(&self) -> usize
Allocated keys and values, excluding recurrent state and scratch.
pub fn position(&self) -> Result<usize, SessionError>
pub fn logits(&self) -> Result<&[f32], SessionError>
pub fn reset(&mut self)
pub fn embed(&self, token: u32, hidden: &mut [f32]) -> Result<(), SessionError>
Sourcepub fn step(
&mut self,
hidden: &mut [f32],
position: usize,
) -> Result<(u32, f32), SessionError>
pub fn step( &mut self, hidden: &mut [f32], position: usize, ) -> Result<(u32, f32), SessionError>
Hidden initially contains the embedding for this position. A partial error or unwind poisons the entire sequence; callers must reset or drop it.
pub fn step_controlled( &mut self, hidden: &mut [f32], position: usize, canceled: &dyn Fn() -> bool, ) -> Result<(u32, f32), SessionError>
Sourcepub fn prefill_controlled(
&mut self,
hidden: &mut [f32],
position: usize,
canceled: &dyn Fn() -> bool,
) -> Result<(), SessionError>
pub fn prefill_controlled( &mut self, hidden: &mut [f32], position: usize, canceled: &dyn Fn() -> bool, ) -> Result<(), SessionError>
Advance prompt state without projecting unused vocabulary logits.
Auto Trait Implementations§
impl Freeze for Session
impl RefUnwindSafe for Session
impl Send for Session
impl Sync for Session
impl Unpin for Session
impl UnsafeUnpin for Session
impl UnwindSafe for Session
Blanket Implementations§
§impl<S, A> Aggregate<Result<S, Error>> for Awhere
A: Aggregate<S>,
impl<S, A> Aggregate<Result<S, Error>> for Awhere
A: Aggregate<S>,
Aggregate shares in an MPC protocol.
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more