Skip to main content

Module qwen

Module qwen 

Source
Expand description

Qwen hybrid architecture execution and state management (Work Package F11).

Implements GatedDeltaNet convolution, linear-attention recurrent updates, and dual-state checkpointing for Qwen3.6 MoE on the RTX A2000.

Re-exports§

pub use decode::decode_step;
pub use decode::decode_tokens;
pub use expert_tile::admit_expert_tile;
pub use expert_tile::load_expert_tile_descriptor;
pub use expert_tile::promote_expert_tile;
pub use expert_tile::QwenExpertTileReader;
pub use native_asset::load_qwen4exp_native_hmc;
pub use native_asset::prepare_qwen4exp_native_hmc;
pub use native_asset::ExternalGgufAsset;
pub use native_asset::Qwen4ExpNativeAssetError;
pub use native_asset::Qwen4ExpNativeDescriptor;
pub use native_asset::Qwen4ExpPleSpan;
pub use native_asset::QWEN4EXP_NATIVE_FORMAT;
pub use native_runtime::Qwen4ExpActivationError;
pub use native_runtime::Qwen4ExpNativeRuntime;
pub use native_runtime::Qwen4ExpTrunkResidency;
pub use ple_nvme::PleNvmeReader;
pub use streamed_moe::execute_streamed_moe_with_tiles;

Modules§

checkpoint
Compatibility exports for the canonical Etalon Qwen engine.
decode
Explicit developer adapters for the canonical Etalon Qwen engine. Product generation uses NativeProductSession::decode_qwen_into with retained sources.
expert_residency
Compatibility exports for the canonical Etalon Qwen engine.
expert_tile
Bounded C:-resident HMC tiles for router-selected Qwen4Exp experts.
gated_deltanet
Compatibility exports for the canonical Etalon numerical implementation.
graph_contract
Compatibility exports for the canonical Etalon Qwen engine.
hyper_connection
Compatibility exports for the canonical Etalon numerical implementation.
native_asset
Native Qwen4Exp asset preparation.
native_runtime
Activation for a Qwen4Exp package with an NVMe-resident PLE payload.
numerics
Compatibility exports for the canonical Etalon numerical implementation.
ple_block
Compatibility exports for the canonical Etalon Qwen engine.
ple_ngram
ple_nvme
HMC activation adapter for the canonical Etalon PLE reader.
retained
Cold immutable Qwen input closure for the authenticated native product route.
storage_plan
Compatibility exports for the canonical Etalon Qwen engine.
streamed_gdn
Compatibility exports for the canonical Etalon Qwen engine.
streamed_hyper
Compatibility exports for the canonical Etalon Qwen engine.
streamed_layer
Compatibility exports for the canonical Etalon Qwen engine.
streamed_moe
Host tile adapter for the canonical Etalon streamed MoE engine.
streamed_qsa
Compatibility exports for the canonical Etalon Qwen engine.
trunk_nvme
Compatibility exports for the canonical Etalon Qwen engine.

Structs§

GatedDeltaBuffers
Scratch buffers reused for every GatedDeltaNet decode step.
GatedDeltaState
Persistent recurrence state for one GatedDeltaNet layer. The runtime owner supplies these slices explicitly; no evaluator allocation occurs.
PleBlockBuffers
All hot-path PLE storage is provided by the decode owner. The block does not allocate or retain any of these vectors.
PleGatherReceipt
Deterministic accounting for one selected-row PLE gather.
PleIoStats
Cumulative physical I/O performed by a PLE reader.
PleTokenHistory
QsaState
Persistent cache for one QSA layer. Keys are RoPE-rotated before entry; indexer keys deliberately remain raw because pooling precedes its norm and positional rotation. Storage is supplied by the runtime owner.
Qwen4ExpDecodeReceipt
Measurement receipt for a real multi-token decode run. trunk_* and ple_* counts come from the readers’ own counters, not estimates.
Qwen4ExpDecodeScratch
All per-step scratch, allocated once and shared across every layer. The GDN and QSA paths run sequentially, so one set of buffers serves both.
Qwen4ExpGraphReport
Qwen4ExpLayerContract
Qwen4ExpSession
Owned mutable state for a live Qwen4Exp sequence. Allocated once at session start; the per-token step only mutates slices inside it.
Qwen4ExpStoragePlan
Byte-precise placement receipt for Qwen4Exp activation.
Qwen4ExpTraceRecord
One observed tensor inside a token step, for lab-style localization of a divergent stage. layer is u32::MAX for the embedding/final stages.
QwenCheckpointRegistry
Registry of checkpoints for a multi-turn conversation or generation run.
QwenDualCheckpoint
Checkpoint of both Attention KV state and GatedDeltaNet recurrent state.
QwenExpertKey
QwenExpertPlane
QwenExpertResidency
Cold-created cache metadata. The runtime’s GPU uploader owns the actual payload buffers; this type only provides deterministic, zero-heap routing decisions during decode.
QwenExpertTileAdmission
QwenExpertTileDescriptor
StreamedArgmax
Exact winner from a streamed vocabulary projection. The token id is a flattened row number in the output tensor.
StreamedGdnMoeLayerBuffers
StreamedHyperBuffers
StreamedMoeBuffers
StreamedQsaBuffers
Reused scratch for one QSA step. selected and scores are bounded by the caller-owned cache capacity; no logits or cache-sized heap object is created by the operator.
StreamedQsaMoeLayerBuffers
Scratch for the QSA/MoE layer. The QSA state persists across decode tokens while all buffers are caller-owned and reused.
TrunkIoStats
Cumulative physical I/O accounting for one reader. reads counts file read calls (chunked), rows counts model rows consumed, bytes the raw encoded weight bytes pulled from storage.
TrunkNvmeReader
One encoded-weight source cursor. It owns no model payload; caller buffers bound each individual read/dequantization operation. A fixed 4 MiB read chunk amortizes syscalls across consecutive rows without holding model weights; it carries raw bytes only.

Enums§

GatedDeltaError
PleBlockError
PleNgramError
PleNvmeError
Qwen4ExpDecodeError
Qwen4ExpGraphError
Qwen4ExpStorageError
QwenExpertAccess
QwenExpertTileError
QwenExpertTilePlaneKind
QwenNumericError
QwenStateError
Errors during Qwen recurrent state execution.
StreamedHyperError
StreamedLayerError
StreamedMoeError
StreamedQsaError
TrunkNvmeError

Constants§

CAUSAL_CONV_KERNEL
DEFAULT_QWEN4EXP_HOST_OS_FLOOR
MAX_QWEN_EXPERT_SLOTS
QWEN4EXP_EXPERT_COUNT
Qwen4Exp routed expert bank size (native streamed MoE geometry).
QWEN4EXP_GDN_HEADS
QWEN4EXP_GDN_HEAD_DIM
QWEN4EXP_GDN_KEY_HEADS
QWEN4EXP_HYPER_STREAMS
QWEN4EXP_LAYER_GDN
QWEN4EXP_LAYER_HC_ATTN
QWEN4EXP_LAYER_HC_FFN
QWEN4EXP_LAYER_MOE
QWEN4EXP_LAYER_PLE
QWEN4EXP_LAYER_QSA
QWEN4EXP_PLE_HISTORY
QWEN4EXP_QSA_COMPRESS_RATIO
QWEN4EXP_QSA_HEAD_DIM
QSA is 24×256 GQA, separate from the GatedDeltaNet 48×128 geometry.
QWEN4EXP_QSA_INDEXER_HEADS
QWEN4EXP_QSA_INDEXER_HEAD_DIM
The lightweight indexer remains four 128-wide heads.
QWEN4EXP_QSA_KV_HEADS
QWEN4EXP_QSA_MAX_SELECTED
Maximum tokens reachable per step: token_topk block members plus the open group’s causal tail (ratio - 1), matching select_width.
QWEN4EXP_QSA_QUERY_HEADS
QWEN4EXP_QSA_ROTARY_DIM
Qwen4Exp rotates only the first 64 dims of each attention/indexer head (partial NeoX RoPE, qwen4exp.rope.dimension_count).
QWEN4EXP_QSA_TOP_BLOCKS
Complete blocks selectable per step (index_budget // index_ratio).
QWEN4EXP_QSA_TOP_K
Trained indexer budget in TOKENS (qwen4exp.attention.indexer.top_k = 2048): complete four-token blocks compete for TOP_K / COMPRESS_RATIO slots and the members of the still-open trailing group are always visible.
QWEN4EXP_TOP_EXPERTS
Qwen4Exp MoE top-k width — shared by residency metadata (wasm-safe) and streamed MoE (native).
QWEN4EXP_TRACE_META_LAYER
Layer sentinel for records that do not belong to a numbered block.
QWEN_EXPERT_TILE_FORMAT
QWEN_EXPERT_TILE_SUFFIX

Functions§

add_assign
Add update into state in place, avoiding a temporary residual vector.
add_into
Copy a supplied vector into an output buffer and add a second vector. This is useful where the decoder must preserve a pre-norm residual separately.
execute_ple_block
Execute the trained PLE update inserted before the configured layer.
execute_streamed_final_hyper_connection
Execute Qwen4Exp’s final global Hyper-Connection mixer. Unlike a layer mixer it has no injection projection: the collapsed hidden vector feeds the vocabulary projection directly in place of an output_norm tensor.
execute_streamed_gated_delta
Execute one trained Qwen4Exp GatedDeltaNet recurrence.
execute_streamed_gdn_moe_layer
Execute one non-QSA Qwen4Exp layer in trained order: Hyper-Connection mix → GatedDeltaNet → attention inject → Hyper-Connection mix → top-10 routed/shared MoE → FFN inject.
execute_streamed_hyper_connection
Stream one attention or FFN Hyper-Connection mix from the trunk.
execute_streamed_moe
Source-only compatibility path. New native callers should use [execute_streamed_moe_with_tiles] to supply any validated C: tiles.
execute_streamed_qsa
Execute one causal QSA decode step. rope_theta is explicit so the bounded operator cannot silently invent a positional encoding policy; the Qwen4Exp runtime passes the GGUF model value (10,000,000 for this model).
execute_streamed_qsa_moe_layer
Execute one full QSA Qwen4Exp layer in trained order: Hyper-Connection mix → sparse attention → attention inject → Hyper-Connection mix → top-10 routed/shared MoE → FFN inject.
group_rms_norm_in_place
In-place variant for a transient stream buffer whose unnormalized values are not needed by the caller after the operation. Same pre-folded gamma semantics as group_rms_norm_into.
group_rms_norm_into
Normalize each Hyper-Connection stream independently with RMSNorm. The GGUF converter pre-folds the zero-centered (1 + weight) scale into the stored gamma, so the checkpoint bytes multiply directly — callers must not add a further bias (llama.cpp build_hc_mix).
inject_hyper_update
Scatter a token-mixer/MoE branch back into each residual stream. Qwen4Exp uses one learned inject scalar per stream, divided by the stream count.
inject_stream_update
Add a block update into each Hyper stream. inject_gate is the sigmoid output of the streamed HC inject projection; Qwen scales it by two.
mix_hyper_streams
Form the hidden-width mixer input from the learned per-stream gates.
mix_streams_into
Form the token-mixer or MoE input from the gated average of Hyper streams. mix_gate is the sigmoid output of the streamed HC up-projection.
normalize_top_experts
plan_qwen4exp_storage
Create a storage placement receipt without probing hardware or allocating.
rms_norm_into
Apply Qwen RMSNorm with a separately stored scale vector.
select_ple_rows
Generate trained global row IDs; late row refusal may leave an output prefix.
select_top_experts
sort_dedup_rows
Sort and deduplicate a selected row list in caller storage.
step_causal_conv1d
Update causal 1D depthwise convolution state and produce convolved output.
step_gated_deltanet
Execute one step of the GatedDeltaNet recurrent state update and projection.
validate_qwen4exp_graph
Validate the exact tensor graph into caller-owned layer receipts.