Module recurrent
Expand description
Caller-buffered convolution and recurrent state updates.
Modules§
- gated_
deltanet - Qwen hybrid linear-attention and GatedDeltaNet recurrent state execution (Work Package F11).
- qwen_
delta - Explicit Qwen decay-before-prediction recurrence, independent of the legacy API.
- qwen_
projection - Caller-buffered Qwen projection ordering and per-head numerical transforms.
Structs§
Enums§
Functions§
- gated_
rms_ norm - One head: RMS(input)stored_weightSiLU(z), with no zero-centered weight offset. Exact nonempty slices and positive finite epsilon; every error is atomic.
- normalize_
qwen_ qk - L2 normalize global Q/K with sqrt(sum(x*x)+1e-6), repeat each key head Hv/Hk consecutive times, and scale Q by K^-0.5. Outputs have [Hv,K] shape. Every error leaves both output buffers unchanged.
- qwen_
gates - Produce explicit per-value-head decay=exp(-exp(A_log)*softplus(a+dt_bias)) and beta=sigmoid(b). Stable gates; any admission/arithmetic error is atomic.
- silu_
into - Stable SiLU over one token’s channels; errors leave output unchanged.
- step_
qwen_ delta - One rectangular head:
state[key_lane * value_dim + value_lane]. Q/K are already L2-normalized and Q is already scaled bykey_dim^-0.5. Decay and beta are explicit scalars in [0,1]; decay isexp(g), notg. Every slice must have its exact logical length. Any error leaves all buffers unchanged. - unpack_
qwen_ projections - Unpack one token into global Q/K/V channels and head-major Z/B/A. Exact extents, finite inputs, and packing are required before any write.