Function evaluate_swiglu_expert_nvfp4
pub fn evaluate_swiglu_expert_nvfp4(
input: &[f32],
weights: &ExpertWeightView<'_>,
gate_buf: &mut [f32],
up_buf: &mut [f32],
swiglu_buf: &mut [f32],
out: &mut [f32],
) -> Result<(), Nvfp4Error>Expand description
Zero-heap evaluation of one SwiGLU expert: y = (SiLU(x * W_gate) * (x * W_up)) * W_down