Skip to main content

compute_principled_slot_capacity

Function compute_principled_slot_capacity 

pub fn compute_principled_slot_capacity(
    available_accelerator_bytes: u64,
    backbone_bytes: u64,
    kv_cache_bytes: u64,
    bytes_per_expert: usize,
    total_experts: usize,
    is_unified_memory: bool,
) -> (usize, PersonalHardwareTier)
Expand description

Compute principled slot capacity given detected system memory and model footprint.

This principle-based calculation works across Vulkan/wgpu, CUDA, DirectML, and Apple Metal:

  • Determines the available accelerator headroom after subtracting backbone and KV cache budgets.
  • Slices headroom into slots of bytes_per_expert.
  • Selects the appropriate PersonalHardwareTier without hardcoding specific GPU models.