Skip to main content

Module ggml_quants

Module ggml_quants 

Source
Expand description

Canonical numerical encodings and descriptor helpers, with legacy Qualia paths.

Structs§

BlockQ6K
GGML block_q6_K — 210 bytes, 256 weights. Mirrors WGSL BlockQ6K layout.
GgmlBlockLayout
Elements per quantization block and packed byte size (from ggml).

Enums§

ExecutionError
Errors from zero-copy mmap tensor slicing.
GgmlDequantError

Constants§

BLOCK_Q4K_SOA_BYTES
Bytes per SoA superblock (256 weights).
BLOCK_Q4K_SOA_ELEMS
BLOCK_Q6K_BYTES
BLOCK_Q6K_ELEMS
GGML_TYPE_BF16
Brain float16 (1 sign / 8 exp / 7 mantissa) — used by Gemma-4 and other modern GGUFs for norms / residual scales alongside Q4_K weights (ggml_type enum value 30).
GGML_TYPE_F16
GGML_TYPE_F32
GGML element-type identifiers used in GGUF tensor-info headers.
GGML_TYPE_FP8_E4M3
FP8 E4M3 (1 sign / 4 exp / 3 mantissa) — 1 byte per element.
GGML_TYPE_IQ4_NL
GGML’s 32-value non-linear 4-bit block. Qwen3.8 Flash Next uses this specifically for its 160-wide PLE rows, even in the IQ4_XS model variant.
GGML_TYPE_IQ4_XS
Importance-aware, non-linear 4-bit quantization with per-32-value scales.
GGML_TYPE_MXFP4
GGML_TYPE_Q4_0
GGML_TYPE_Q4_K
GGML_TYPE_Q4_K_SOA
Qualia conversion-time SoA Q4_K (not a stock GGML type).
GGML_TYPE_Q5_0
GGML_TYPE_Q5_1
GGML_TYPE_Q5_K
GGML’s 256-value K-quant block with a fifth high-bit plane. Qwen3.8 Flash Next stores its fused GatedDeltaNet QKV projection in this layout.
GGML_TYPE_Q6_K
GGML_TYPE_Q8_0
GGML_TYPE_TERNARY_158
P64 BitNet 1.58-bit whole-tensor encoding.

Functions§

dequant_matrix_row_into
Dequantize one matrix row (row index along dims[1]) into out.
dequantize_row_into
Dequantize one embedding row from raw mmap bytes into out. Returns the number of f32 elements written (≤ out.len()).
expand_q4k_tensor_to_soa
Expand a full Q4_K tensor blob (row-major superblocks) into SoA layout. n_row_elems = dims[0] (weights per row). n_rows = dims[1].
fetch_tensor_bytes
Zero-copy slice of an entire tensor payload from the mmap.
fetch_tensor_row_range_bytes
Zero-copy slice covering vocabulary rows [row_start, row_start + row_count).
fetch_token_embedding
Return a zero-copy &[u8] slice of the packed embedding row for token_id.
ggml_block_layout
Return block layout for a GGML type, or None if unsupported.
ggml_row_bytes
Packed byte length of one logical row (n_elems weights) for the given GGML type.
q4k_block_to_soa
Convert one stock Q4_K superblock (144 B) → SoA superblock (160 B).
quantize_f32_to_q4_k_soa_tensor
Quantize a full f32 weight matrix to Q4_K_SOA layout.
tensor_byte_len
Total packed byte length of a GGUF tensor from its shape and ggml_type.
tensor_row_byte_len
Packed byte width of one logical matrix row (dims[0] elements).