Module inference_kernel_parity 

W3 — in-project GPU↔CPU kernel-parity oracle (no external LLM libs).

The “no external libraries” rule (Prime Directive #4) means the only trustworthy reference for a GPU kernel is an in-project CPU implementation of the identical math. This module supplies the comparison metrics (max / mean absolute error + ULP distance) and helpers to synthesize valid quantized weights, so a GPU shader can be checked against its CPU twin on random, controlled inputs — fast, deterministic, and library-free. The first consumer is the GEMM parity test (QTensorEngine::gemm_parity_probe: GPU dispatch_gemm_raw_into vs CPU stack_gemm_quant).

Pure CPU + zero-heap on the metric paths (slices in, scalars out); safe on every target.

Functions

f16_bytes

Bytes needed to hold n_elems weights as little-endian IEEE F16 (2 bytes each).

max_abs_err

Maximum absolute error between two equal-length slices. Returns +inf on length mismatch so a caller cannot silently pass a comparison of differently-shaped outputs.

max_ulp_diff

Maximum ULP (unit-in-the-last-place) distance over finite pairs; non-finite pairs are skipped. Returns u64::MAX on length mismatch.

mean_abs_err

Mean absolute error (f64 accumulation to avoid catastrophic cancellation over long vectors).

q4_0_bytes

Bytes for n_elems weights as ggml Q4_0 (18-byte blocks of 32: f16 scale + 16 nibble bytes).

q4_k_bytes

Bytes for n_elems weights as ggml Q4_K (144-byte super-blocks of 256).

q8_0_bytes

Bytes needed to hold n_elems weights in Q8_0 (ceil to whole 32-element blocks).

quantize_f16_from_f32

Encode weights (f32) as little-endian IEEE F16 into out (>= f16_bytes(weights.len())). The exact byte layout dequant_f16 / the GPU unpack2x16float path consume.

quantize_q4_0_from_f32

Quantize weights (f32) to ggml Q4_0 into out (>= q4_0_bytes). Matches ggml_quants::dequant_q4_0 exactly: per 32-block d = max_abs_signed / -8, nibble q = clamp(round(x/d)+8, 0..15), dequant x = (q-8)*d; interleaved layout — block index k < 16 is the low nibble of byte k, k >= 16 the high nibble of byte k-16.

quantize_q4_k_from_f32

Quantize weights (f32) to ggml Q4_K into out (>= q4_k_bytes). Matches ggml_quants::dequant_q4_k: super-block of 256 = 8 sub-blocks of 32, each with an asymmetric scale+min — 6-bit sub-scales (d*sc) and mins (dmin*m) packed via get_scale_min_k4, 4-bit quants (even sub-block = low nibble, odd = high nibble of the same qs byte). Dequant: x = d*sc[s]*q - dmin*m[s]. Simplified vs ggml’s iterative search (per-sub-block min clamped ≤ 0, which holds for zero-centred weights); round-trip tested. Q4_K’s 6-bit sub-scales make it markedly more accurate per bit than Q4_0 — AWQ’s intended 4-bit partner.

quantize_q8_0_from_f32

Quantize weights (f32) into Q8_0 blocks in out (must be >= q8_0_bytes(weights.len())). Standard ggml Q8_0: per 32-block, scale = absmax / 127, q = round(w / scale) clamped to i8, stored as little-endian f16 scale followed by 32 signed bytes. Returns false if out is too small.