Struct HeterogeneousDispatcher 

pub struct HeterogeneousDispatcher { /* private fields */ }

Routes compute jobs across GPU / NPU / CPU and degrades gracefully under VRAM pressure instead of hard-failing (the behaviour the CUDA bridge lacked).

Implementations

impl HeterogeneousDispatcher

pub fn new(caps: HostCapabilities) -> Self

pub fn select_backend(&self, vram_required: u64) -> ComputeBackend

Choose a backend for a job needing vram_required bytes: GPU if present and it fits → else NPU if present → else CPU (always works). When the GPU is present but the job is larger than VRAM, the caller should GPU-tile (see Self::gpu_tiles) rather than fall straight to CPU.

pub fn gpu_tiles(&self, total_bytes: u64) -> u32

How many sequential tiles a total_bytes GPU job must be split into so each tile fits in available VRAM — graceful degradation instead of an OOM hard fail. Returns 1 when it already fits (or when there’s no GPU/VRAM to tile into, in which case the job runs on the CPU as a single pass).

pub fn capabilities(&self) -> HostCapabilities

Trait Implementations

impl Clone for HeterogeneousDispatcher

fn clone(&self) -> HeterogeneousDispatcher

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for HeterogeneousDispatcher

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Copy for HeterogeneousDispatcher