Skip to main content

QWEN4EXP_QSA_ROTARY_DIM

Constant QWEN4EXP_QSA_ROTARY_DIM 

pub const QWEN4EXP_QSA_ROTARY_DIM: usize = 64; // 64usize
Expand description

Qwen4Exp rotates only the first 64 dims of each attention/indexer head (partial NeoX RoPE, qwen4exp.rope.dimension_count).