Skip to main content

qwen_gates

Function qwen_gates 

pub fn qwen_gates(
    a: &[f32],
    b: &[f32],
    a_log: &[f32],
    dt_bias: &[f32],
    decay: &mut [f32],
    beta: &mut [f32],
) -> Result<(), QwenDeltaError>
Expand description

Produce explicit per-value-head decay=exp(-exp(A_log)*softplus(a+dt_bias)) and beta=sigmoid(b). Stable gates; any admission/arithmetic error is atomic.