Skip to main content

gated_rms_norm

Function gated_rms_norm 

pub fn gated_rms_norm(
    input: &[f32],
    z: &[f32],
    weight: &[f32],
    epsilon: f32,
    out: &mut [f32],
) -> Result<(), QwenDeltaError>
Expand description

One head: RMS(input)stored_weightSiLU(z), with no zero-centered weight offset. Exact nonempty slices and positive finite epsilon; every error is atomic.