Skip to main content

QWEN4EXP_QSA_HEAD_DIM

Constant QWEN4EXP_QSA_HEAD_DIM 

pub const QWEN4EXP_QSA_HEAD_DIM: usize = 256; // 256usize
Expand description

QSA is 24×256 GQA, separate from the GatedDeltaNet 48×128 geometry.