Skip to main content

DEFAULT_GPU_EXPERT_SLOTS

Constant DEFAULT_GPU_EXPERT_SLOTS 

pub const DEFAULT_GPU_EXPERT_SLOTS: usize = 32; // 32usize
Expand description

Maximum number of active expert slots in GPU VRAM simultaneously.