Function step_gated_deltanet
pub fn step_gated_deltanet(
state: &mut [f32],
q: &[f32],
k: &[f32],
v: &[f32],
alpha: &[f32],
beta: &[f32],
d_state: usize,
d_head: usize,
out: &mut [f32],
) -> Result<(), QwenStateError>Expand description
Execute one step of the GatedDeltaNet recurrent state update and projection.
Mathematical formulation:
- Retrieve query projection: $v_{\text{pred}} = S_{t-1} \cdot k_t$
- Compute error / innovation: $e_t = v_t - v_{\text{pred}}$
- Update state with decay $\alpha$ and learning rate $\beta$: $S_t = S_{t-1} \odot \alpha_t + \beta_t (e_t \otimes k_t)$
- Output projection: $y_t = S_t \cdot q_t$
state: Persistent matrix of dimensions d_state * d_head.
q: Query vector of length d_state.
k: Key vector of length d_state.
v: Value vector of length d_head.
alpha: Decay factor per head (length d_head or 1).
beta: Update gate per head (length d_head or 1).
out: Output vector of length d_head.