Skip to main content

rms_norm

Function rms_norm 

Source
pub fn rms_norm(x: &mut [f64], weight: &[f64], eps: f64)
Expand description

RMS normalization in place: x_i ← (x_i / sqrt(mean(x²) + eps)) · weight_i. No mean subtraction (the Llama/transformer RMSNorm). weight must match x in length; shorter is honoured up to the common length.