pub struct SamplerState {
pub cfg: SamplerConfig,
/* private fields */
}Expand description
Stateful sampler: owns the PRNG stream so successive tokens advance it deterministically.
Fields§
§cfg: SamplerConfigImplementations§
Source§impl SamplerState
impl SamplerState
Sourcepub fn new(cfg: SamplerConfig) -> Self
pub fn new(cfg: SamplerConfig) -> Self
Creates a default SamplerState.
Sourcepub fn sample(&mut self, logits: &mut [f32], ctx: &[u32]) -> u32
pub fn sample(&mut self, logits: &mut [f32], ctx: &[u32]) -> u32
Sample the next token id. logits is the full vocab (mutated in place as a scratch buffer);
ctx is the running token history (prompt + generated) for the penalty window.
Greedy (temperature <= 0) returns the argmax with no mutation of selection semantics.
Sourcepub fn sample_constrained(
&mut self,
logits: &mut [f32],
ctx: &[u32],
masker: &mut DominoMasker,
) -> u32
pub fn sample_constrained( &mut self, logits: &mut [f32], ctx: &[u32], masker: &mut DominoMasker, ) -> u32
Sample with a DOMINO constrained-decoding mask applied first.
This is the R9 integration point: the decode loop calls this instead
of sample when a DominoMasker() is active. The
masker sets disallowed token logits to -inf before the sampling
chain runs, so the sampler can only select grammar-valid tokens.
When the masker is inactive (!is_active()), this is equivalent to
calling sample directly — the mask is a no-op.
After sampling, the chosen token’s bytes should be fed back into the
masker via DominoMasker::feed_token() so the grammar state advances.
Trait Implementations§
Source§impl Clone for SamplerState
impl Clone for SamplerState
Auto Trait Implementations§
impl Freeze for SamplerState
impl RefUnwindSafe for SamplerState
impl Send for SamplerState
impl Sync for SamplerState
impl Unpin for SamplerState
impl UnsafeUnpin for SamplerState
impl UnwindSafe for SamplerState
Blanket Implementations§
§impl<S, A> Aggregate<Result<S, Error>> for Awhere
A: Aggregate<S>,
impl<S, A> Aggregate<Result<S, Error>> for Awhere
A: Aggregate<S>,
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more