Expand description
Phase 4: AOT GGUF → .p64 LLM-weight container compiler.
A sibling of the semantic .q42 graph format — it carries an independent section magic b"p64\0"
so the two never collide. Per the architectural decision, the weights are stored as opaque,
cache-aligned (64-byte), contiguous quantized blobs.
The NQuin epistemic scaffold is removed from this probabilistic container. The 48-byte
declarative q42 system manages truth, while this 64-byte aligned p64 system manages
pure mathematical inference with zero-copy relative WASM pointers.
Output is little-endian on every host via explicit serialization.
Layout:
[ P64WeightHeader (64B) ] magic, version, flags, 32-bit relative offsets
[ P64TensorEntry[] (64B each) ] role, dtype, rank, dims, relative blob offsets
[ pad → 1<<page_log2 ]
[ tensor blob region ] quantized bytes; each tensor START page-aligned
(default 16KB) for single-fetch mmap.Structs§
- Gguf
Compile Budget - Gguf
Compile Plan - Immutable prepared metadata and borrowed source payloads. Vec-backed caller storage may be allocated once and written repeatedly without allocation. While this plan or retained descriptors borrow names, their backing order storage cannot be rebound. Dynamic scratch rebinding is not promised.
- Gguf
Compile Requirements - Capacity bounds, not an output-size prediction or proof of source validity. Three extra slots cover cached global bindings in a caller-modified index.
- Gguf
Compile Tensor - Caller storage for semantic ordering and bounded canonical names. Fields are private; every compile reconstructs them from the borrowed source index.
- Gguf
Compile Workspace - Descriptor storage can be inspected after compilation. Its retained names
still borrow order records: reinitialize the whole descriptor array before
rebuilding order. Calling
.fillalone does not end that Rust lifetime. - P64H
Params - P64Layer
Schedule Entry - One row of the optional layer schedule table (64 B, cache-line DOD).
Written when
P64_FLAG_LAYER_SCHEDULEis set; offset inrole_table_offset. - P64Round
Trip Report - Summary of a byte-exact P64 <-> GGUF round-trip verification.
- P64Tensor
Entry - P64Tensor
Index - Runtime reader: parses a P64 container’s header + manifest in microseconds. Tensor blobs
stay in the caller’s byte slice (zero-copy); only the small manifest is materialized. The
role/layer/blob_offsetfields map directly to the resident WebGPU weight arenas. - P64Weight
Header - RawVision
Tensor - A raw vision tensor description for P64 weight container packing.
- Safetensor
Transcode Budget - Bounds adapter workspace and output, excluding parsed source metadata and the sink.
- Safetensor
Transcode Requirements - Safetensor
Transcode Workspace - Every supplied capacity is charged; only required prefixes are modified.
- Transcode
Report
Enums§
- FfnQuant
- Target quantization for the FFN tensors in an AWQ P64 compile.
- Gguf
Compile Buffer - Gguf
Compile Error - Integrity
Mode - How thoroughly
P64TensorIndex::from_p64verifies integrity. - P64Convert
Layout - Conversion-time weight layout policy for
compile_gguf_to_p64_with_layout. - Safetensor
Transcode Error - Safetensor
Transcode Validation Error
Constants§
- FORMAT_
FLAG_ RAW_ TRANSCODE format_flagsbit: container produced by the raw streaming transcode (safetensor/MLX → P64) — tensors are verbatim high-fidelity blobs not yet mapped to engine GEMM roles, and the GGUF hyperparameter block is absent. (Distinguishes it from acompile_gguf_to_p64container.)- FORMAT_
FLAG_ TERNARY format_flagsbit: tensors were ternary-quantized (BitNet 1.58b) during transcode — each blob is[scale: f32][packed trits](ggml_type = ternary::GGML_TYPE_TERNARY_158); decode viaternary::dequantize_blob.- P64_
4K_ PAGE_ SIZE - P64_
CACHE_ LINE_ BYTES - P64_
DEFAULT_ PAGE_ LOG2 - 14 = 16 KB pages (default; minimizes page faults on large FFN blocks). 12 = 4 KB.
- P64_
DEFAULT_ PAGE_ SIZE - P64_
FLAG_ LAYER_ MAJOR - Tensor blob region is layer-major (known roles ordered by layer, then role). Decode residency / CUDA slab fill SHOULD walk entries in table order.
- P64_
FLAG_ LAYER_ PACK - Blobs use layer-pack alignment: page-align at layer boundaries only; 256 B within layer.
- P64_
FLAG_ LAYER_ SCHEDULE role_table_offsetpoints at a layer schedule table (P64LayerScheduleEntry× n_layer).- P64_
FLAG_ LITTLE_ ENDIAN - P64_
FLAG_ Q4K_ SOA - At least one 2-D weight matrix was converted to
GGML_TYPE_Q4_K_SOA(112). - P64_
FLAG_ RAW_ TRANSCODE - Alias of
FORMAT_FLAG_RAW_TRANSCODE(header-flag naming). - P64_
FLAG_ TERNARY - Alias of
FORMAT_FLAG_TERNARY(header-flag naming). - P64_
LAYER_ GLOBAL layersentinel for non-layer (global) tensors.- P64_
MAGIC - P64_
MANIFOLD_ ENTRY_ BYTES - Ten little-endian
f32values plus 24 bytes of zero padding. - P64_
MAX_ PAGE_ LOG2 - P64_
MIN_ PAGE_ LOG2 - P64_
MIN_ PAGE_ SIZE - P64_
ROLE_ ATTN_ K - P64_
ROLE_ ATTN_ NORM - P64_
ROLE_ ATTN_ OUTPUT - P64_
ROLE_ ATTN_ Q - P64_
ROLE_ ATTN_ SUBLN - P64_
ROLE_ ATTN_ V - P64_
ROLE_ FFN_ DOWN - P64_
ROLE_ FFN_ GATE - P64_
ROLE_ FFN_ NORM - P64_
ROLE_ FFN_ SUBLN - P64_
ROLE_ FFN_ UP - P64_
ROLE_ MOE_ DOWN_ EXPS - P64_
ROLE_ MOE_ GATE_ EXPS - P64_
ROLE_ MOE_ ROUTER - P64_
ROLE_ MOE_ SHARED_ DOWN - P64_
ROLE_ MOE_ SHARED_ GATE - P64_
ROLE_ MOE_ SHARED_ UP - P64_
ROLE_ MOE_ UP_ EXPS - P64_
ROLE_ OUTPUT - P64_
ROLE_ OUTPUT_ NORM - P64_
ROLE_ TOKEN_ EMBD - P64_
ROLE_ UNKNOWN - A source GGUF tensor preserved byte-for-byte but not consumed by a known engine role. Its source offset and name hash remain in the entry so a validator can still prove complete model preservation.
- P64_
ROLE_ VISION_ BN - P64_
ROLE_ VISION_ CONV2D - P64_
ROLE_ VISION_ FC - P64_
TENSOR_ ENTRY_ BYTES - P64_
VERSION - Container format version written by the canonical compiler.
Keep in lock-step with
docs/manuals/standards/p64-weight-container-standard.md. - P64_
VIEW_ FLAG_ BF16 - P64_
VIEW_ FLAG_ F16 - P64_
VIEW_ FLAG_ F32 - Precision view flags for multi-precision
.p64containers. - P64_
VIEW_ FLAG_ Q4_ K - P64_
VIEW_ FLAG_ Q8_ 0 - P64_
VIEW_ FLAG_ SOA - P64_
VIEW_ FLAG_ TERNARY_ 158 - P64_
WEIGHT_ HEADER_ BYTES
Functions§
- align_
up - Round
xup to the next multiple of power-of-two alignmenta. - checked_
align_ up - Checked variant of
align_upthat returnsNoneon overflow or non-power-of-twoa. - compile_
gguf_ index_ to_ p64_ into - Compile using caller-owned output and workspace, without adapter allocation. Parsed index, source and already serialized tokenizer storage are outside the budget. Every supplied workspace capacity is charged before mutation. Ordering and metadata preparation may change scratch on later source/planning refusal; output is unchanged on every refusal and its spare tail is left untouched.
- compile_
gguf_ to_ p64 - Compile all source tensors with the original semantic order and metadata. Verbatim payloads use Etalon’s checked layer-packed P64 writer. A zero page selector retains the 16 KiB default. Parsing retains its existing cold policy.
- compile_
gguf_ to_ p64_ ffn_ quant_ awq - AWQ-aware FFN-quantized P64 compile.
quantselects the FFN target (ternary or Q4_0). Whenawq_scalesisSome(per-layer per-input-channel salience fromcrate::llm_awq::snapshot) the gate/up input channeliis scaled bys_i^alphabefore packing andffn_normis divided bys_i^alpha— mathematically exact in f32 ((X·norm/s^a)·(W·s^a)=(X·norm)·W) — moving salient channels into a range the quant grid represents better.awq_scales = None/alpha == 0.0reproduces the plain (un-calibrated) compile. The down projection is left un-scaled (no clean fold site — a v2 item). Everything outside the FFN passes through verbatim from the source GGUF. - compile_
gguf_ to_ p64_ q4_ ffn_ awq - AWQ-aware Q4_0 FFN compile (Path A) — FFN packed to 4-bit Q4_0 (AWQ’s design regime); all else verbatim from the source GGUF.
- compile_
gguf_ to_ p64_ ternary_ ffn - Task #12 / STELLAR §A — like
compile_gguf_to_p64but ternary-packs the FFN projections (gate/up/down) during the compile, producing a complete, runnable P64: hyperparameters + tokenizer are preserved (so the live loader boots it and builds the KV cache), while the FFN tensors are BitNet-1.58b ternary blobs (ternary::dequantize_blob/ the 2-bit GPU kernel). Attention / norms / embeddings stay verbatim at their source precision. This is the loadable container the live FFN-ternary dispatch path will run + measure against. - compile_
gguf_ to_ p64_ ternary_ ffn_ awq - AWQ-aware ternary FFN compile.
- compile_
gguf_ to_ p64_ with_ layout - Select the original numerical conversion policy. Verbatim uses Etalon serialization.
- compile_
gguf_ to_ q42 - Compatibility alias for the historical pre-P64 API name.
- compile_
gguf_ to_ q42_ ffn_ quant_ awq - Compatibility alias for the historical pre-P64 API name.
- compile_
gguf_ to_ q42_ q4_ ffn_ awq - Compatibility alias for the historical pre-P64 API name.
- compile_
gguf_ to_ q42_ ternary_ ffn - Compatibility alias for the historical pre-P64 API name.
- compile_
gguf_ to_ q42_ ternary_ ffn_ awq - Compatibility alias for the historical pre-P64 API name.
- format_
p64_ tensor_ name - Format a canonical P64 tensor name (
token_embd.weight,blk.{layer}.{suffix}, ortensor.{hash:016x}) into anycore::fmt::Writesink without heap allocation. - gguf_
native_ model_ metadata - gguf_
verbatim_ requirements - has_
p64_ magic - Return
trueonly for the canonical four-byte P64 container magic. - is_
4k_ page_ aligned - True iff
offsetis aligned to a 4096-byte hardware page boundary. - is_
cache_ line_ aligned - True iff
offsetis aligned to a 64-byte P64 cache line. - is_
page_ aligned - True iff
page_sizeis a valid P64 page size (>= 256and power-of-two) andoffsetis aligned to it. - p64_
role_ suffix - GGUF tensor-name suffix bytes for a per-layer P64 role (
Nonefor global or unknown roles). - p64_
role_ suffix_ str - GGUF tensor-name suffix for a per-layer P64 role (
Nonefor global or unknown roles). - p64_
tensor_ name - Allocate and format a canonical P64 tensor name (
token_embd.weight,blk.{layer}.{suffix}, ortensor.{hash:016x}). - p64_
tensor_ name_ into - Zero-heap formatting of a canonical P64 tensor name into a caller-supplied byte buffer.
Returns
Some(bytes_written)orNoneifbytesis too small. - prepare_
gguf_ index_ to_ p64 - Freeze canonical metadata using admitted caller workspace. The parsed index is copied into immutable plan metadata; source/tokenizer/name bytes remain borrowed. Preparation may change scratch on refusal, never output.
- recommend_
convert_ layout - Recommend convert layout from source size (bytes on disk) and a VRAM headroom budget.
- resolve_
page_ log2 - Normalize
page_log2(0->P64_DEFAULT_PAGE_LOG2) and validate it lies in8..=30. - resolve_
page_ size - Resolve the byte page size
1usize << page_log2after normalizing0->P64_DEFAULT_PAGE_LOG2. - safetensor_
transcode_ requirements - Sizing excludes the caller’s existing parsed plan and source bytes.
- transcode_
safetensor_ plan_ to_ p64 - Allocates no adapter storage. Source/capacity/scratch refusals leave workspace unchanged; later planning refusals may leave it prepared but never write output. Sink failures may leave incomplete output. Parsing and sink storage are external.
- transcode_
safetensor_ to_ p64 - Transcodes a SafeTensor to the P64 weight stream.
- transcode_
safetensor_ to_ p64_ ffn_ ternary - Transcodes a SafeTensor FFN block to ternary P64.
- transcode_
safetensor_ to_ p64_ policy - Task #12 / STELLAR §A — policy transcode: ternary the FFN projections, keep everything else
(attention, norms, embeddings) verbatim high-fidelity, in ONE P64. This is the real §A policy
(
tensor_roles::ternary_eligible): ternarising attention/norms wrecks coherence, so onlyffn_gate/ffn_up/ffn_downare packed to 1.6 bits; the rest pass through unchanged. - transcode_
safetensor_ to_ p64_ ternary - Task #12 / STELLAR §A — streaming transcode with BitNet 1.58b ternary compression:
safetensor (high-fidelity) → P64, each tensor quantized to
{-1,0,+1}with a per-tensor absmean scale and packed at ≈ 1.6 bits/weight (ternarymodule) during transcode. - transcode_
safetensor_ to_ q42_ ffn_ ternary - Transcodes a SafeTensor FFN block to ternary in a Q42 volume.
- transcode_
vision_ tensors_ to_ p64