Skip to main content

normalize_qwen_qk

Function normalize_qwen_qk 

pub fn normalize_qwen_qk(
    layout: QwenHeadLayout,
    qkv: &[f32],
    q: &mut [f32],
    k: &mut [f32],
) -> Result<(), QwenDeltaError>
Expand description

L2 normalize global Q/K with sqrt(sum(x*x)+1e-6), repeat each key head Hv/Hk consecutive times, and scale Q by K^-0.5. Outputs have [Hv,K] shape. Every error leaves both output buffers unchanged.