Skip to main content

rope_interleaved

Function rope_interleaved 

Source
pub fn rope_interleaved(
    vec: &mut [f64],
    n_heads: usize,
    head_dim: usize,
    pos: f64,
    base: f64,
    scale: f64,
)
Expand description

Apply interleaved (“normal”/llama) RoPE to vec, treated as n_heads consecutive blocks of head_dim elements. Each block’s adjacent pairs (2i, 2i+1) are rotated by θ_i. pos is the token position, base the RoPE frequency base (e.g. 10000), scale the position scaling (≤ 0 or non-finite is treated as 1).