pub struct SafetensorNvfp4Matrix<'a> {
pub packed: &'a [u8],
pub block_scales: &'a [u8],
pub global_scale: f32,
pub rows: usize,
pub cols: usize,
}Expand description
Zero-copy view of one NVFP4-quantized matrix inside a Safetensors mmap (U8 packed nibbles + F8_E4M3 block scales + f32 global scale).
Fields§
§packed: &'a [u8]§block_scales: &'a [u8]§global_scale: f32§rows: usize§cols: usizeImplementations§
Source§impl<'a> SafetensorNvfp4Matrix<'a>
impl<'a> SafetensorNvfp4Matrix<'a>
Sourcepub fn gemv_zero_heap(
&self,
input: &[f32],
out: &mut [f32],
) -> Result<(), Nvfp4Error>
pub fn gemv_zero_heap( &self, input: &[f32], out: &mut [f32], ) -> Result<(), Nvfp4Error>
Execute zero-heap NVFP4 matrix-vector multiplication $y = A \cdot x$ directly from the Safetensors mmap.
Trait Implementations§
Source§impl<'a> Clone for SafetensorNvfp4Matrix<'a>
impl<'a> Clone for SafetensorNvfp4Matrix<'a>
impl<'a> Copy for SafetensorNvfp4Matrix<'a>
Source§impl<'a> Debug for SafetensorNvfp4Matrix<'a>
impl<'a> Debug for SafetensorNvfp4Matrix<'a>
Source§impl<'a> PartialEq for SafetensorNvfp4Matrix<'a>
impl<'a> PartialEq for SafetensorNvfp4Matrix<'a>
impl<'a> StructuralPartialEq for SafetensorNvfp4Matrix<'a>
Auto Trait Implementations§
impl<'a> Freeze for SafetensorNvfp4Matrix<'a>
impl<'a> RefUnwindSafe for SafetensorNvfp4Matrix<'a>
impl<'a> Send for SafetensorNvfp4Matrix<'a>
impl<'a> Sync for SafetensorNvfp4Matrix<'a>
impl<'a> Unpin for SafetensorNvfp4Matrix<'a>
impl<'a> UnsafeUnpin for SafetensorNvfp4Matrix<'a>
impl<'a> UnwindSafe for SafetensorNvfp4Matrix<'a>
Blanket Implementations§
§impl<S, A> Aggregate<Result<S, Error>> for Awhere
A: Aggregate<S>,
impl<S, A> Aggregate<Result<S, Error>> for Awhere
A: Aggregate<S>,
Aggregate shares in an MPC protocol.
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more