Skip to main content

dispatch_moe_step

Function dispatch_moe_step 

pub fn dispatch_moe_step<F>(
    input: &[f32],
    emb_dim: usize,
    router_weights: &[f32],
    num_experts: usize,
    topk: usize,
    expert_fetcher: F,
    shared_expert: Option<&ExpertWeightView<'_>>,
    shared_gate_weight: f32,
    scratch_accum: &mut [f32],
) -> Result<(), MoeDispatchError>
where F: FnMut(u16) -> Option<ExpertWeightView<'static>>,
Expand description

Dispatch MoE layer step:

  1. Compute router logits
  2. Select top-k experts
  3. Compute SwiGLU for each selected expert and accumulate weighted output