pub struct RequestScheduler { /* private fields */ }Expand description
Request scheduler
Implementations§
Source§impl RequestScheduler
impl RequestScheduler
Sourcepub fn initialize(&mut self) -> Result<(), MLError>
pub fn initialize(&mut self) -> Result<(), MLError>
Initialize RequestScheduler; Ok when ready.
Sourcepub fn scheduling_policy(&self) -> &SchedulingPolicy
pub fn scheduling_policy(&self) -> &SchedulingPolicy
Return the current scheduling policy.
Sourcepub fn set_scheduling_policy(&mut self, policy: SchedulingPolicy)
pub fn set_scheduling_policy(&mut self, policy: SchedulingPolicy)
Set the scheduling policy.
Sourcepub fn queue_manager(&self) -> &QueueManager
pub fn queue_manager(&self) -> &QueueManager
Return a reference to the queue manager.
Sourcepub fn queue_manager_mut(&mut self) -> &mut QueueManager
pub fn queue_manager_mut(&mut self) -> &mut QueueManager
Return a mutable reference to the queue manager.
Sourcepub fn load_balancer(&self) -> &LoadBalancer
pub fn load_balancer(&self) -> &LoadBalancer
Return a reference to the load balancer.
Sourcepub fn load_balancer_mut(&mut self) -> &mut LoadBalancer
pub fn load_balancer_mut(&mut self) -> &mut LoadBalancer
Return a mutable reference to the load balancer.
Sourcepub fn schedule_request(
&mut self,
request: &InferenceRequest,
) -> Result<String, MLError>
pub fn schedule_request( &mut self, request: &InferenceRequest, ) -> Result<String, MLError>
Schedule an inference request to an available backend according to the configured load-balancing strategy and record it in the queue manager.
Auto Trait Implementations§
impl Freeze for RequestScheduler
impl RefUnwindSafe for RequestScheduler
impl Send for RequestScheduler
impl Sync for RequestScheduler
impl Unpin for RequestScheduler
impl UnsafeUnpin for RequestScheduler
impl UnwindSafe for RequestScheduler
Blanket Implementations§
§impl<S, A> Aggregate<Result<S, Error>> for Awhere
A: Aggregate<S>,
impl<S, A> Aggregate<Result<S, Error>> for Awhere
A: Aggregate<S>,
Aggregate shares in an MPC protocol.
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more