Struct ExpertPlacementCalibration
#[repr(C)]pub struct ExpertPlacementCalibration {
pub transfer_bytes_per_second: u64,
pub cpu_experts_per_second: f32,
pub gpu_experts_per_second: f32,
pub gpu_launch_overhead_ns: u64,
pub max_inflight_transfers: u16,
}Expand description
Measured timings used to choose CPU evaluation or a GPU staging transfer.
All rates describe the same expert representation and batch shape. A zero rate means that lane was not measured and is never selected speculatively.
Fields§
§transfer_bytes_per_second: u64Sustained host-to-device transfer rate for compressed expert bytes.
cpu_experts_per_second: f32CPU evaluation rate for one routed expert, expressed as experts per second.
gpu_experts_per_second: f32GPU evaluation rate after an expert is resident, expressed as experts per second.
gpu_launch_overhead_ns: u64Fixed submission/synchronisation cost measured for this backend and batch shape.
max_inflight_transfers: u16Largest safely overlapped transfer queue observed by the calibrator.
Trait Implementations§
§impl Clone for ExpertPlacementCalibration
impl Clone for ExpertPlacementCalibration
§fn clone(&self) -> ExpertPlacementCalibration
fn clone(&self) -> ExpertPlacementCalibration
Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source. Read moreimpl Copy for ExpertPlacementCalibration
§impl Debug for ExpertPlacementCalibration
impl Debug for ExpertPlacementCalibration
§impl Default for ExpertPlacementCalibration
impl Default for ExpertPlacementCalibration
§fn default() -> ExpertPlacementCalibration
fn default() -> ExpertPlacementCalibration
Returns the “default value” for a type. Read more
§impl PartialEq for ExpertPlacementCalibration
impl PartialEq for ExpertPlacementCalibration
impl StructuralPartialEq for ExpertPlacementCalibration
Auto Trait Implementations§
impl Freeze for ExpertPlacementCalibration
impl RefUnwindSafe for ExpertPlacementCalibration
impl Send for ExpertPlacementCalibration
impl Sync for ExpertPlacementCalibration
impl Unpin for ExpertPlacementCalibration
impl UnsafeUnpin for ExpertPlacementCalibration
impl UnwindSafe for ExpertPlacementCalibration
Blanket Implementations§
§impl<S, A> Aggregate<Result<S, Error>> for Awhere
A: Aggregate<S>,
impl<S, A> Aggregate<Result<S, Error>> for Awhere
A: Aggregate<S>,
Aggregate shares in an MPC protocol.
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more§impl<T> NoneValue for Twhere
T: Default,
impl<T> NoneValue for Twhere
T: Default,
type NoneType = T
§fn null_value() -> T
fn null_value() -> T
The none-equivalent value.