Skip to main content

execute_streamed_gated_delta

Function execute_streamed_gated_delta 

pub fn execute_streamed_gated_delta(
    trunk: &mut TrunkNvmeReader,
    layer: &LayerTensors,
    input: &[f32],
    state: &mut GatedDeltaState<'_>,
    out: &mut [f32],
    buffers: &mut GatedDeltaBuffers<'_>,
) -> Result<(), GatedDeltaError>
Expand description

Execute one trained Qwen4Exp GatedDeltaNet recurrence.

The delta state update is the model’s column-wise gated delta rule: S <- decay*S + key*(value - Sᵀ*key)*beta. Query/key L2 normalization, learned per-head decay, causal convolution, output RMS norm, and the learned gate all run before the streamed output projection.