Skip to main content

Module qwen_delta

Module qwen_delta 

Expand description

Explicit Qwen decay-before-prediction recurrence, independent of the legacy API.

Enums§

QwenDeltaError

Functions§

step_qwen_delta
One rectangular head: state[key_lane * value_dim + value_lane]. Q/K are already L2-normalized and Q is already scaled by key_dim^-0.5. Decay and beta are explicit scalars in [0,1]; decay is exp(g), not g. Every slice must have its exact logical length. Any error leaves all buffers unchanged.