pub struct FtwModelPackage {
pub manifest: FtwManifest,
pub shards: Vec<Arc<Mmap>>,
pub tensor_index: GgufTensorIndex,
pub expert_slices: HashMap<(u16, u16), FtwExpertBankSlice>,
/* private fields */
}Expand description
Container owning memory-mapped FTW shards and tensor index.
Fields§
§manifest: FtwManifest§shards: Vec<Arc<Mmap>>§tensor_index: GgufTensorIndex§expert_slices: HashMap<(u16, u16), FtwExpertBankSlice>Implementations§
Source§impl FtwModelPackage
impl FtwModelPackage
Sourcepub fn open_from_dir(dir: &Path) -> Result<Self, String>
pub fn open_from_dir(dir: &Path) -> Result<Self, String>
Load an FTW model package from a directory containing freetoken_weight.json and .ftw shards.
Sourcepub fn fetch_tensor_bytes(&self, name: &str) -> Option<&[u8]>
pub fn fetch_tensor_bytes(&self, name: &str) -> Option<&[u8]>
Read raw bytes for a shared weight tensor by name.
Sourcepub fn fetch_by_global_offset(
&self,
global_off: u64,
len: usize,
) -> Option<&[u8]>
pub fn fetch_by_global_offset( &self, global_off: u64, len: usize, ) -> Option<&[u8]>
Read raw bytes for a tensor by global offset and length across shards.
Sourcepub fn get_expert_data(
&self,
layer: u16,
expert_idx: u16,
) -> Option<FtwExpertData<'_>>
pub fn get_expert_data( &self, layer: u16, expert_idx: u16, ) -> Option<FtwExpertData<'_>>
Get zero-copy byte slices for one expert in a given layer.
Trait Implementations§
Source§impl Clone for FtwModelPackage
impl Clone for FtwModelPackage
Source§impl ExpertSource for FtwModelPackage
impl ExpertSource for FtwModelPackage
fn fetch_tensor_bytes(&self, name: &str) -> Option<&[u8]>
fn get_expert_data(&self, layer: u16, expert: u16) -> Option<FtwExpertData<'_>>
fn layer_tensor(&self, layer: u16, suffix: &[u8]) -> Option<&[u8]>
fn canonical_layer_tensor(&self, layer: u16, suffix: &[u8]) -> Option<&[u8]>
Auto Trait Implementations§
impl Freeze for FtwModelPackage
impl RefUnwindSafe for FtwModelPackage
impl Send for FtwModelPackage
impl Sync for FtwModelPackage
impl Unpin for FtwModelPackage
impl UnsafeUnpin for FtwModelPackage
impl UnwindSafe for FtwModelPackage
Blanket Implementations§
§impl<S, A> Aggregate<Result<S, Error>> for Awhere
A: Aggregate<S>,
impl<S, A> Aggregate<Result<S, Error>> for Awhere
A: Aggregate<S>,
Aggregate shares in an MPC protocol.
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more