Module p64_weight

Expand description

Phase 4: AOT GGUF → .p64 LLM-weight container compiler.

A sibling of the semantic .q42 graph format — it carries an independent section magic b"p64\0" so the two never collide. Per the architectural decision, the weights are stored as opaque, cache-aligned (64-byte), contiguous quantized blobs.

The NQuin epistemic scaffold is removed from this probabilistic container. The 48-byte declarative q42 system manages truth, while this 64-byte aligned p64 system manages pure mathematical inference with zero-copy relative WASM pointers.

Output is little-endian on every host via explicit serialization.

Layout:

[ P64WeightHeader (64B) ]              magic, version, flags, 32-bit relative offsets
[ P64TensorEntry[] (64B each) ]        role, dtype, rank, dims, relative blob offsets
[ pad → 1<<page_log2 ]
[ tensor blob region ]                 quantized bytes; each tensor START page-aligned
                                       (default 16KB) for single-fetch mmap.

Structs§

P64HParams
P64LayerScheduleEntry: One row of the optional layer schedule table (64 B, cache-line DOD). Written when P64_FLAG_LAYER_SCHEDULE is set; offset in role_table_offset.
P64RoundTripReport
P64TensorEntry
P64TensorIndex: Runtime reader: parses a P64 container’s header + manifest in microseconds. Tensor blobs stay in the caller’s byte slice (zero-copy); only the small manifest is materialized. The role/layer/blob_offset fields map directly to the resident WebGPU weight arenas.
P64WeightHeader
RawVisionTensor: A raw vision tensor description for P64 weight container packing.
TranscodeReport: Outcome of a streaming transcode — the numbers that make the memory claim falsifiable.

Enums§

FfnQuant: Target quantization for the FFN tensors in an AWQ P64 compile.
IntegrityMode: How thoroughly P64TensorIndex::from_p64 verifies integrity.
P64ConvertLayout: Conversion-time weight layout policy for compile_gguf_to_p64_with_layout.

Constants§

FORMAT_FLAG_RAW_TRANSCODE: format_flags bit: container produced by the raw streaming transcode (safetensor/MLX → P64) — tensors are verbatim high-fidelity blobs not yet mapped to engine GEMM roles, and the GGUF hyperparameter block is absent. (Distinguishes it from a compile_gguf_to_p64 container.)
FORMAT_FLAG_TERNARY: format_flags bit: tensors were ternary-quantized (BitNet 1.58b) during transcode — each blob is [scale: f32][packed trits] (ggml_type = ternary::GGML_TYPE_TERNARY_158); decode via ternary::dequantize_blob.
P64_DEFAULT_PAGE_LOG2: 14 = 16 KB pages (default; minimizes page faults on large FFN blocks). 12 = 4 KB.
P64_FLAG_LAYER_MAJOR: Tensor blob region is layer-major (known roles ordered by layer, then role). Decode residency / CUDA slab fill SHOULD walk entries in table order.
P64_FLAG_LAYER_PACK: Blobs use layer-pack alignment: page-align at layer boundaries only; 256 B within layer.
P64_FLAG_LAYER_SCHEDULE: role_table_offset points at a layer schedule table (P64LayerScheduleEntry × n_layer).
P64_FLAG_LITTLE_ENDIAN
P64_FLAG_Q4K_SOA: At least one 2-D weight matrix was converted to GGML_TYPE_Q4_K_SOA (112).
P64_FLAG_RAW_TRANSCODE: Alias of FORMAT_FLAG_RAW_TRANSCODE (header-flag naming).
P64_FLAG_TERNARY: Alias of FORMAT_FLAG_TERNARY (header-flag naming).
P64_LAYER_GLOBAL: layer sentinel for non-layer (global) tensors.
P64_MAGIC
P64_MANIFOLD_ENTRY_BYTES: Ten little-endian f32 values plus 24 bytes of zero padding.
P64_ROLE_ATTN_K
P64_ROLE_ATTN_NORM
P64_ROLE_ATTN_OUTPUT
P64_ROLE_ATTN_Q
P64_ROLE_ATTN_SUBLN
P64_ROLE_ATTN_V
P64_ROLE_FFN_DOWN
P64_ROLE_FFN_GATE
P64_ROLE_FFN_NORM
P64_ROLE_FFN_SUBLN
P64_ROLE_FFN_UP
P64_ROLE_OUTPUT
P64_ROLE_OUTPUT_NORM
P64_ROLE_TOKEN_EMBD
P64_ROLE_UNKNOWN: A source GGUF tensor preserved byte-for-byte but not consumed by a known engine role. Its source offset and name hash remain in the entry so a validator can still prove complete model preservation.
P64_ROLE_VISION_BN
P64_ROLE_VISION_CONV2D
P64_ROLE_VISION_FC
P64_TENSOR_ENTRY_BYTES
P64_VERSION: Container format version written by the canonical compiler. Keep in lock-step with docs/manuals/standards/p64-weight-container-standard.md.
P64_VIEW_FLAG_BF16
P64_VIEW_FLAG_F16
P64_VIEW_FLAG_F32: Precision view flags for multi-precision .p64 containers.
P64_VIEW_FLAG_Q4_K
P64_VIEW_FLAG_Q8_0
P64_VIEW_FLAG_SOA
P64_VIEW_FLAG_TERNARY_158
P64_WEIGHT_HEADER_BYTES

Functions§

compile_gguf_to_p64: Compile a GGUF image into the cache-line-native P64 container (verbatim layout).
compile_gguf_to_p64_ffn_quant_awq: AWQ-aware FFN-quantized P64 compile. quant selects the FFN target (ternary or Q4_0). When awq_scales is Some (per-layer per-input-channel salience from crate::llm_awq::snapshot) the gate/up input channel i is scaled by s_i^alpha before packing and ffn_norm is divided by s_i^alpha — mathematically exact in f32 ((X·norm/s^a)·(W·s^a)=(X·norm)·W) — moving salient channels into a range the quant grid represents better. awq_scales = None / alpha == 0.0 reproduces the plain (un-calibrated) compile. The down projection is left un-scaled (no clean fold site — a v2 item). Everything outside the FFN passes through verbatim from the source GGUF.
compile_gguf_to_p64_q4_ffn_awq: AWQ-aware Q4_0 FFN compile (Path A) — FFN packed to 4-bit Q4_0 (AWQ’s design regime); all else verbatim from the source GGUF.
compile_gguf_to_p64_ternary_ffn: Task #12 / STELLAR §A — like compile_gguf_to_p64 but ternary-packs the FFN projections (gate/up/down) during the compile, producing a complete, runnable P64: hyperparameters + tokenizer are preserved (so the live loader boots it and builds the KV cache), while the FFN tensors are BitNet-1.58b ternary blobs (ternary::dequantize_blob / the 2-bit GPU kernel). Attention / norms / embeddings stay verbatim at their source precision. This is the loadable container the live FFN-ternary dispatch path will run + measure against.
compile_gguf_to_p64_ternary_ffn_awq: AWQ-aware ternary FFN compile.
compile_gguf_to_p64_with_layout: Like compile_gguf_to_p64 but selects a conversion-time layout policy.
compile_gguf_to_q42: Compatibility alias for the historical pre-P64 API name.
compile_gguf_to_q42_ffn_quant_awq: Compatibility alias for the historical pre-P64 API name.
compile_gguf_to_q42_q4_ffn_awq: Compatibility alias for the historical pre-P64 API name.
compile_gguf_to_q42_ternary_ffn: Compatibility alias for the historical pre-P64 API name.
compile_gguf_to_q42_ternary_ffn_awq: Compatibility alias for the historical pre-P64 API name.
has_p64_magic: Return true only for the canonical four-byte P64 container magic.
recommend_convert_layout: Recommend convert layout from source size (bytes on disk) and a VRAM headroom budget.
transcode_safetensor_to_p64
transcode_safetensor_to_p64_ffn_ternary
transcode_safetensor_to_p64_policy: Task #12 / STELLAR §A — policy transcode: ternary the FFN projections, keep everything else (attention, norms, embeddings) verbatim high-fidelity, in ONE P64. This is the real §A policy (tensor_roles::ternary_eligible): ternarising attention/norms wrecks coherence, so only ffn_gate/ffn_up/ffn_down are packed to 1.6 bits; the rest pass through unchanged.
transcode_safetensor_to_p64_ternary: Task #12 / STELLAR §A — streaming transcode with BitNet 1.58b ternary compression: safetensor (high-fidelity) → P64, each tensor quantized to {-1,0,+1} with a per-tensor absmean scale and packed at ≈ 1.6 bits/weight (ternary module) during transcode.
transcode_safetensor_to_q42_ffn_ternary
transcode_vision_tensors_to_p64: Transcode raw vision model tensors into a zero-copy P64 container format.

Type Aliases§

Q42TensorIndex