Struct QwenExpertResidency
pub struct QwenExpertResidency { /* private fields */ }Expand description
Cold-created cache metadata. The runtime’s GPU uploader owns the actual payload buffers; this type only provides deterministic, zero-heap routing decisions during decode.
Implementations§
§impl ExpertResidency
impl ExpertResidency
pub fn new(capacity: usize) -> ExpertResidency
pub fn new(capacity: usize) -> ExpertResidency
Creates a default ExpertResidency.
pub fn resolve(&mut self, key: ExpertKey) -> ExpertAccess
pub fn resolve(&mut self, key: ExpertKey) -> ExpertAccess
Resolve a layer-specific expert. Equal LRU ages prefer the lower slot, which makes eviction receipts deterministic.
pub fn resolve_topk<const K: usize>(
&mut self,
layer: u16,
experts: &[usize; K],
out: &mut [ExpertAccess; K],
)
pub fn resolve_topk<const K: usize>( &mut self, layer: u16, experts: &[usize; K], out: &mut [ExpertAccess; K], )
Resolve the selected experts for one model layer into an output supplied by the caller. Repeated ids are still routed independently by the caller but hit the same residency slot.
Auto Trait Implementations§
impl Freeze for ExpertResidency
impl RefUnwindSafe for ExpertResidency
impl Send for ExpertResidency
impl Sync for ExpertResidency
impl Unpin for ExpertResidency
impl UnsafeUnpin for ExpertResidency
impl UnwindSafe for ExpertResidency
Blanket Implementations§
§impl<S, A> Aggregate<Result<S, Error>> for Awhere
A: Aggregate<S>,
impl<S, A> Aggregate<Result<S, Error>> for Awhere
A: Aggregate<S>,
Aggregate shares in an MPC protocol.
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more