Module ggml_quants 

GGML quantization block layout and zero-heap row dequantization.

Byte strides match ggml_row_size() in llama.cpp / ggml. Embedding lookup slices raw mmap bytes via fetch_token_embedding; this module dequantizes into caller-supplied &mut [f32] buffers (no Vec in the hot path).

Structs

BlockQ6K

GGML block_q6_K — 210 bytes, 256 weights. Mirrors WGSL BlockQ6K layout.

GgmlBlockLayout

Elements per quantization block and packed byte size (from ggml).

Enums

ExecutionError

Errors from zero-copy mmap tensor slicing.

GgmlDequantError

Constants

BLOCK_Q4K_SOA_BYTES

Bytes per SoA superblock (256 weights).

BLOCK_Q4K_SOA_ELEMS

BLOCK_Q6K_BYTES

BLOCK_Q6K_ELEMS

GGML_TYPE_BF16

Brain float16 (1 sign / 8 exp / 7 mantissa) — used by Gemma-4 and other modern GGUFs for norms / residual scales alongside Q4_K weights (ggml_type enum value 30).

GGML_TYPE_F16

GGML_TYPE_F32

GGML element-type identifiers used in GGUF tensor-info headers.

GGML_TYPE_Q4_0

GGML_TYPE_Q4_K

GGML_TYPE_Q4_K_SOA

Qualia conversion-time SoA Q4_K (not a stock GGML type).

GGML_TYPE_Q5_0

GGML_TYPE_Q6_K

GGML_TYPE_Q8_0

Functions

dequant_matrix_row_into

Dequantize one matrix row (row index along dims[1]) into out.

dequantize_row_into

Dequantize one embedding row from raw mmap bytes into out. Returns the number of f32 elements written (≤ out.len()).

expand_q4k_tensor_to_soa

Expand a full Q4_K tensor blob (row-major superblocks) into SoA layout. n_row_elems = dims[0] (weights per row). n_rows = dims[1].

fetch_tensor_bytes

Zero-copy slice of an entire tensor payload from the mmap.

fetch_tensor_row_range_bytes

Zero-copy slice covering vocabulary rows [row_start, row_start + row_count).

fetch_token_embedding

Return a zero-copy &[u8] slice of the packed embedding row for token_id.

ggml_block_layout

Return block layout for a GGML type, or None if unsupported.

ggml_row_bytes

Packed byte length of one logical row (n_elems weights) for the given GGML type.

q4k_block_to_soa

Convert one stock Q4_K superblock (144 B) → SoA superblock (160 B).

quantize_f32_to_q4_k_soa_tensor

Quantize a full f32 weight matrix to Q4_K_SOA layout.

tensor_byte_len

Total packed byte length of a GGUF tensor from its shape and ggml_type.

tensor_row_byte_len

Packed byte width of one logical matrix row (dims[0] elements).