Expand description
Qwen hybrid architecture execution and state management (Work Package F11).
Implements GatedDeltaNet convolution, linear-attention recurrent updates, and dual-state checkpointing for Qwen3.6 MoE on the RTX A2000.
Re-exports§
pub use decode::decode_step;pub use decode::decode_tokens;pub use expert_tile::admit_expert_tile;pub use expert_tile::load_expert_tile_descriptor;pub use expert_tile::promote_expert_tile;pub use expert_tile::QwenExpertTileReader;pub use native_asset::load_qwen4exp_native_hmc;pub use native_asset::prepare_qwen4exp_native_hmc;pub use native_asset::ExternalGgufAsset;pub use native_asset::Qwen4ExpNativeAssetError;pub use native_asset::Qwen4ExpNativeDescriptor;pub use native_asset::Qwen4ExpPleSpan;pub use native_asset::QWEN4EXP_NATIVE_FORMAT;pub use native_runtime::Qwen4ExpActivationError;pub use native_runtime::Qwen4ExpNativeRuntime;pub use native_runtime::Qwen4ExpTrunkResidency;pub use ple_nvme::PleNvmeReader;pub use streamed_moe::execute_streamed_moe_with_tiles;
Modules§
- checkpoint
- Compatibility exports for the canonical Etalon Qwen engine.
- decode
- Explicit developer adapters for the canonical Etalon Qwen engine. Product generation uses NativeProductSession::decode_qwen_into with retained sources.
- expert_
residency - Compatibility exports for the canonical Etalon Qwen engine.
- expert_
tile - Bounded C:-resident HMC tiles for router-selected Qwen4Exp experts.
- gated_
deltanet - Compatibility exports for the canonical Etalon numerical implementation.
- graph_
contract - Compatibility exports for the canonical Etalon Qwen engine.
- hyper_
connection - Compatibility exports for the canonical Etalon numerical implementation.
- native_
asset - Native Qwen4Exp asset preparation.
- native_
runtime - Activation for a Qwen4Exp package with an NVMe-resident PLE payload.
- numerics
- Compatibility exports for the canonical Etalon numerical implementation.
- ple_
block - Compatibility exports for the canonical Etalon Qwen engine.
- ple_
ngram - ple_
nvme - HMC activation adapter for the canonical Etalon PLE reader.
- retained
- Cold immutable Qwen input closure for the authenticated native product route.
- storage_
plan - Compatibility exports for the canonical Etalon Qwen engine.
- streamed_
gdn - Compatibility exports for the canonical Etalon Qwen engine.
- streamed_
hyper - Compatibility exports for the canonical Etalon Qwen engine.
- streamed_
layer - Compatibility exports for the canonical Etalon Qwen engine.
- streamed_
moe - Host tile adapter for the canonical Etalon streamed MoE engine.
- streamed_
qsa - Compatibility exports for the canonical Etalon Qwen engine.
- trunk_
nvme - Compatibility exports for the canonical Etalon Qwen engine.
Structs§
- Gated
Delta Buffers - Scratch buffers reused for every GatedDeltaNet decode step.
- Gated
Delta State - Persistent recurrence state for one GatedDeltaNet layer. The runtime owner supplies these slices explicitly; no evaluator allocation occurs.
- PleBlock
Buffers - All hot-path PLE storage is provided by the decode owner. The block does not allocate or retain any of these vectors.
- PleGather
Receipt - Deterministic accounting for one selected-row PLE gather.
- PleIo
Stats - Cumulative physical I/O performed by a PLE reader.
- PleToken
History - QsaState
- Persistent cache for one QSA layer. Keys are RoPE-rotated before entry; indexer keys deliberately remain raw because pooling precedes its norm and positional rotation. Storage is supplied by the runtime owner.
- Qwen4
ExpDecode Receipt - Measurement receipt for a real multi-token decode run.
trunk_*andple_*counts come from the readers’ own counters, not estimates. - Qwen4
ExpDecode Scratch - All per-step scratch, allocated once and shared across every layer. The GDN and QSA paths run sequentially, so one set of buffers serves both.
- Qwen4
ExpGraph Report - Qwen4
ExpLayer Contract - Qwen4
ExpSession - Owned mutable state for a live Qwen4Exp sequence. Allocated once at session start; the per-token step only mutates slices inside it.
- Qwen4
ExpStorage Plan - Byte-precise placement receipt for Qwen4Exp activation.
- Qwen4
ExpTrace Record - One observed tensor inside a token step, for lab-style localization of a
divergent stage.
layerisu32::MAXfor the embedding/final stages. - Qwen
Checkpoint Registry - Registry of checkpoints for a multi-turn conversation or generation run.
- Qwen
Dual Checkpoint - Checkpoint of both Attention KV state and GatedDeltaNet recurrent state.
- Qwen
Expert Key - Qwen
Expert Plane - Qwen
Expert Residency - Cold-created cache metadata. The runtime’s GPU uploader owns the actual payload buffers; this type only provides deterministic, zero-heap routing decisions during decode.
- Qwen
Expert Tile Admission - Qwen
Expert Tile Descriptor - Streamed
Argmax - Exact winner from a streamed vocabulary projection. The token id is a flattened row number in the output tensor.
- Streamed
GdnMoe Layer Buffers - Streamed
Hyper Buffers - Streamed
MoeBuffers - Streamed
QsaBuffers - Reused scratch for one QSA step.
selectedandscoresare bounded by the caller-owned cache capacity; no logits or cache-sized heap object is created by the operator. - Streamed
QsaMoe Layer Buffers - Scratch for the QSA/MoE layer. The QSA state persists across decode tokens while all buffers are caller-owned and reused.
- Trunk
IoStats - Cumulative physical I/O accounting for one reader.
readscounts file read calls (chunked),rowscounts model rows consumed,bytesthe raw encoded weight bytes pulled from storage. - Trunk
Nvme Reader - One encoded-weight source cursor. It owns no model payload; caller buffers bound each individual read/dequantization operation. A fixed 4 MiB read chunk amortizes syscalls across consecutive rows without holding model weights; it carries raw bytes only.
Enums§
- Gated
Delta Error - PleBlock
Error - PleNgram
Error - PleNvme
Error - Qwen4
ExpDecode Error - Qwen4
ExpGraph Error - Qwen4
ExpStorage Error - Qwen
Expert Access - Qwen
Expert Tile Error - Qwen
Expert Tile Plane Kind - Qwen
Numeric Error - Qwen
State Error - Errors during Qwen recurrent state execution.
- Streamed
Hyper Error - Streamed
Layer Error - Streamed
MoeError - Streamed
QsaError - Trunk
Nvme Error
Constants§
- CAUSAL_
CONV_ KERNEL - DEFAULT_
QWEN4EXP_ HOST_ OS_ FLOOR - MAX_
QWEN_ EXPERT_ SLOTS - QWEN4EXP_
EXPERT_ COUNT - Qwen4Exp routed expert bank size (native streamed MoE geometry).
- QWEN4EXP_
GDN_ HEADS - QWEN4EXP_
GDN_ HEAD_ DIM - QWEN4EXP_
GDN_ KEY_ HEADS - QWEN4EXP_
HYPER_ STREAMS - QWEN4EXP_
LAYER_ GDN - QWEN4EXP_
LAYER_ HC_ ATTN - QWEN4EXP_
LAYER_ HC_ FFN - QWEN4EXP_
LAYER_ MOE - QWEN4EXP_
LAYER_ PLE - QWEN4EXP_
LAYER_ QSA - QWEN4EXP_
PLE_ HISTORY - QWEN4EXP_
QSA_ COMPRESS_ RATIO - QWEN4EXP_
QSA_ HEAD_ DIM - QSA is 24×256 GQA, separate from the GatedDeltaNet 48×128 geometry.
- QWEN4EXP_
QSA_ INDEXER_ HEADS - QWEN4EXP_
QSA_ INDEXER_ HEAD_ DIM - The lightweight indexer remains four 128-wide heads.
- QWEN4EXP_
QSA_ KV_ HEADS - QWEN4EXP_
QSA_ MAX_ SELECTED - Maximum tokens reachable per step:
token_topkblock members plus the open group’s causal tail (ratio - 1), matchingselect_width. - QWEN4EXP_
QSA_ QUERY_ HEADS - QWEN4EXP_
QSA_ ROTARY_ DIM - Qwen4Exp rotates only the first 64 dims of each attention/indexer head
(partial NeoX RoPE,
qwen4exp.rope.dimension_count). - QWEN4EXP_
QSA_ TOP_ BLOCKS - Complete blocks selectable per step (
index_budget // index_ratio). - QWEN4EXP_
QSA_ TOP_ K - Trained indexer budget in TOKENS (
qwen4exp.attention.indexer.top_k= 2048): complete four-token blocks compete forTOP_K / COMPRESS_RATIOslots and the members of the still-open trailing group are always visible. - QWEN4EXP_
TOP_ EXPERTS - Qwen4Exp MoE top-k width — shared by residency metadata (wasm-safe) and streamed MoE (native).
- QWEN4EXP_
TRACE_ META_ LAYER - Layer sentinel for records that do not belong to a numbered block.
- QWEN_
EXPERT_ TILE_ FORMAT - QWEN_
EXPERT_ TILE_ SUFFIX
Functions§
- add_
assign - Add
updateintostatein place, avoiding a temporary residual vector. - add_
into - Copy a supplied vector into an output buffer and add a second vector. This is useful where the decoder must preserve a pre-norm residual separately.
- execute_
ple_ block - Execute the trained PLE update inserted before the configured layer.
- execute_
streamed_ final_ hyper_ connection - Execute Qwen4Exp’s final global Hyper-Connection mixer. Unlike a layer
mixer it has no injection projection: the collapsed hidden vector feeds
the vocabulary projection directly in place of an
output_normtensor. - execute_
streamed_ gated_ delta - Execute one trained Qwen4Exp GatedDeltaNet recurrence.
- execute_
streamed_ gdn_ moe_ layer - Execute one non-QSA Qwen4Exp layer in trained order: Hyper-Connection mix → GatedDeltaNet → attention inject → Hyper-Connection mix → top-10 routed/shared MoE → FFN inject.
- execute_
streamed_ hyper_ connection - Stream one attention or FFN Hyper-Connection mix from the trunk.
- execute_
streamed_ moe - Source-only compatibility path. New native callers should use
[
execute_streamed_moe_with_tiles] to supply any validated C: tiles. - execute_
streamed_ qsa - Execute one causal QSA decode step.
rope_thetais explicit so the bounded operator cannot silently invent a positional encoding policy; the Qwen4Exp runtime passes the GGUF model value (10,000,000 for this model). - execute_
streamed_ qsa_ moe_ layer - Execute one full QSA Qwen4Exp layer in trained order: Hyper-Connection mix → sparse attention → attention inject → Hyper-Connection mix → top-10 routed/shared MoE → FFN inject.
- group_
rms_ norm_ in_ place - In-place variant for a transient stream buffer whose unnormalized values
are not needed by the caller after the operation. Same pre-folded gamma
semantics as
group_rms_norm_into. - group_
rms_ norm_ into - Normalize each Hyper-Connection stream independently with RMSNorm.
The GGUF converter pre-folds the zero-centered
(1 + weight)scale into the stored gamma, so the checkpoint bytes multiply directly — callers must not add a further bias (llama.cppbuild_hc_mix). - inject_
hyper_ update - Scatter a token-mixer/MoE branch back into each residual stream. Qwen4Exp uses one learned inject scalar per stream, divided by the stream count.
- inject_
stream_ update - Add a block update into each Hyper stream.
inject_gateis the sigmoid output of the streamed HC inject projection; Qwen scales it by two. - mix_
hyper_ streams - Form the hidden-width mixer input from the learned per-stream gates.
- mix_
streams_ into - Form the token-mixer or MoE input from the gated average of Hyper streams.
mix_gateis the sigmoid output of the streamed HC up-projection. - normalize_
top_ experts - plan_
qwen4exp_ storage - Create a storage placement receipt without probing hardware or allocating.
- rms_
norm_ into - Apply Qwen RMSNorm with a separately stored scale vector.
- select_
ple_ rows - Generate trained global row IDs; late row refusal may leave an output prefix.
- select_
top_ experts - sort_
dedup_ rows - Sort and deduplicate a selected row list in caller storage.
- step_
causal_ conv1d - Update causal 1D depthwise convolution state and produce convolved output.
- step_
gated_ deltanet - Execute one step of the GatedDeltaNet recurrent state update and projection.
- validate_
qwen4exp_ graph - Validate the exact tensor graph into caller-owned layer receipts.