Struct StreamedQsaBuffers
pub struct StreamedQsaBuffers<'a> {Show 16 fields
pub raw_row: &'a mut [u8],
pub projection_row: &'a mut [f32],
pub q_full: &'a mut [f32],
pub key: &'a mut [f32],
pub value: &'a mut [f32],
pub index_query: &'a mut [f32],
pub index_key: &'a mut [f32],
pub q_norm: &'a mut [f32],
pub k_norm: &'a mut [f32],
pub index_q_norm: &'a mut [f32],
pub index_k_norm: &'a mut [f32],
pub selected: &'a mut [usize],
pub scores: &'a mut [f32],
pub block_selected: &'a mut [usize],
pub block_scores: &'a mut [f32],
pub attention: &'a mut [f32],
}Expand description
Reused scratch for one QSA step. selected and scores are bounded by
the caller-owned cache capacity; no logits or cache-sized heap object is
created by the operator.
Fields§
§raw_row: &'a mut [u8]§projection_row: &'a mut [f32]§q_full: &'a mut [f32]§key: &'a mut [f32]§value: &'a mut [f32]§index_query: &'a mut [f32]§index_key: &'a mut [f32]§q_norm: &'a mut [f32]§k_norm: &'a mut [f32]§index_q_norm: &'a mut [f32]§index_k_norm: &'a mut [f32]§selected: &'a mut [usize]Expanded token list: all members of the selected four-token blocks.
Capacity bound is QWEN4EXP_QSA_MAX_SELECTED (8192), not the cache.
scores: &'a mut [f32]§block_selected: &'a mut [usize]Block-level ranking scratch: QWEN4EXP_QSA_TOP_K entries each.
block_scores: &'a mut [f32]§attention: &'a mut [f32]Auto Trait Implementations§
impl<'a> !UnwindSafe for StreamedQsaBuffers<'a>
impl<'a> Freeze for StreamedQsaBuffers<'a>
impl<'a> RefUnwindSafe for StreamedQsaBuffers<'a>
impl<'a> Send for StreamedQsaBuffers<'a>
impl<'a> Sync for StreamedQsaBuffers<'a>
impl<'a> Unpin for StreamedQsaBuffers<'a>
impl<'a> UnsafeUnpin for StreamedQsaBuffers<'a>
Blanket Implementations§
§impl<S, A> Aggregate<Result<S, Error>> for Awhere
A: Aggregate<S>,
impl<S, A> Aggregate<Result<S, Error>> for Awhere
A: Aggregate<S>,
Aggregate shares in an MPC protocol.
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more