Function normalize_qwen_qk
pub fn normalize_qwen_qk(
layout: QwenHeadLayout,
qkv: &[f32],
q: &mut [f32],
k: &mut [f32],
) -> Result<(), QwenDeltaError>Expand description
L2 normalize global Q/K with sqrt(sum(x*x)+1e-6), repeat each key head Hv/Hk consecutive times, and scale Q by K^-0.5. Outputs have [Hv,K] shape. Every error leaves both output buffers unchanged.