pub struct HeterogeneousDispatcher { /* private fields */ }Expand description
Routes compute jobs across GPU / NPU / CPU and degrades gracefully under VRAM pressure instead of hard-failing (the behaviour the CUDA bridge lacked).
Implementations§
Source§impl HeterogeneousDispatcher
impl HeterogeneousDispatcher
Sourcepub fn new(caps: HostCapabilities) -> Self
pub fn new(caps: HostCapabilities) -> Self
Creates a default HeterogeneousDispatcher.
Sourcepub fn select_backend(&self, vram_required: u64) -> ComputeBackend
pub fn select_backend(&self, vram_required: u64) -> ComputeBackend
Choose a backend for a job needing vram_required bytes: GPU if present and
it fits → else CPU. NPU discovery alone cannot select an execution path. When the GPU is
present but the job is larger than VRAM, the caller should GPU-tile (see
Self::gpu_tiles) rather than fall straight to CPU.
Sourcepub fn gpu_tiles(&self, total_bytes: u64) -> u32
pub fn gpu_tiles(&self, total_bytes: u64) -> u32
How many sequential tiles a total_bytes GPU job must be split into so each
tile fits in available VRAM — graceful degradation instead of an OOM hard
fail. Returns 1 when it already fits (or when there’s no GPU/VRAM to tile
into, in which case the job runs on the CPU as a single pass).
Sourcepub fn capabilities(&self) -> HostCapabilities
pub fn capabilities(&self) -> HostCapabilities
Dispatch capabilities.
Trait Implementations§
Source§impl Clone for HeterogeneousDispatcher
impl Clone for HeterogeneousDispatcher
impl Copy for HeterogeneousDispatcher
Auto Trait Implementations§
impl Freeze for HeterogeneousDispatcher
impl RefUnwindSafe for HeterogeneousDispatcher
impl Send for HeterogeneousDispatcher
impl Sync for HeterogeneousDispatcher
impl Unpin for HeterogeneousDispatcher
impl UnsafeUnpin for HeterogeneousDispatcher
impl UnwindSafe for HeterogeneousDispatcher
Blanket Implementations§
§impl<S, A> Aggregate<Result<S, Error>> for Awhere
A: Aggregate<S>,
impl<S, A> Aggregate<Result<S, Error>> for Awhere
A: Aggregate<S>,
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more