Skip to main content

Module recurrent

Module recurrent 

Expand description

Caller-buffered convolution and recurrent state updates.

Modules§

gated_deltanet
Qwen hybrid linear-attention and GatedDeltaNet recurrent state execution (Work Package F11).
qwen_delta
Explicit Qwen decay-before-prediction recurrence, independent of the legacy API.
qwen_projection
Caller-buffered Qwen projection ordering and per-head numerical transforms.

Structs§

QwenHeadLayout

Enums§

QwenDeltaError
QwenProjectionPacking

Functions§

gated_rms_norm
One head: RMS(input)stored_weightSiLU(z), with no zero-centered weight offset. Exact nonempty slices and positive finite epsilon; every error is atomic.
normalize_qwen_qk
L2 normalize global Q/K with sqrt(sum(x*x)+1e-6), repeat each key head Hv/Hk consecutive times, and scale Q by K^-0.5. Outputs have [Hv,K] shape. Every error leaves both output buffers unchanged.
qwen_gates
Produce explicit per-value-head decay=exp(-exp(A_log)*softplus(a+dt_bias)) and beta=sigmoid(b). Stable gates; any admission/arithmetic error is atomic.
silu_into
Stable SiLU over one token’s channels; errors leave output unchanged.
step_qwen_delta
One rectangular head: state[key_lane * value_dim + value_lane]. Q/K are already L2-normalized and Q is already scaled by key_dim^-0.5. Decay and beta are explicit scalars in [0,1]; decay is exp(g), not g. Every slice must have its exact logical length. Any error leaves all buffers unchanged.
unpack_qwen_projections
Unpack one token into global Q/K/V channels and head-major Z/B/A. Exact extents, finite inputs, and packing are required before any write.