Module gguf_bridge 

Model-inference runtime (honest module name: crate::inference_runtime).

This is the runtime that runs model inference — not an “AI engine”. It does two ordinary systems jobs: (1) reads the GGUF weight file format (tensors memory-mapped via memmap2, zero heap copy) and (2) dispatches the tensor program to a GPU backend (DirectML 1.15 on Windows x64; wgpu/WGSL — Vulkan/Metal/WebGPU — elsewhere).

The mathematics of inference is not here and is not proprietary: it lives in crate::solvers as named STEM — GEMM (linear_algebra::gemm), activations/softmax/normalization (activation), attention (attention), RoPE (rope), the SwiGLU FFN (feed_forward) — and each kernel in this crate is proven equal to that library definition (the *_stem_parity_tests and substrate_parity_tests). What remains here is plumbing: GPU command encoding, the KV cache, weight loading, the autoregressive loop, and the GGUF/dequant codec.

Re-exports

pub use crate::ggml_quants::fetch_token_embedding;

pub use crate::ggml_quants::ExecutionError;

Modules

wasm_cpu

Prepared CPU execution floor for browser WASM. This backend owns no wgpu objects and remains available when the browser exposes no WebGPU adapter. Qualia’s CPU-WASM LLM backend.

Structs

GgufLoadReport

KvCacheLayout

Static ring-buffer KV layout: [layer][slot][K | V] in f32, OR (W5a int8 mode) packed int8 + per-(slot,kv_head) f32 scale in the same 4-byte-element buffer. total_f32_elems counts 4-byte slots either way (u32/f32 share the size), so the allocation math is unchanged.

QTensor

Represents a Q4_K Quantized or standard float Tensor mapped from a monolithic GGUF file.

QTensorEngine

StreamingArgmaxResult

Streaming argmax result across chunked vocabulary projection.

Constants

KV_ATTENTION_MASK_WORDS

KV attention bitmask words uploaded to fused_attention.wgsl binding 5.

KV_CACHE_MAX_BYTES

Maximum bytes for the static KV arena (load-time allocation only).

MAX_CONTEXT_WINDOW

Hard context ceiling — sized to keep KV arena under the 512MB RAM floor (Gemma 42L).

MAX_PREFILL_BATCH_FLOATS

Max stacked embedding floats in a prefill chunk (MAX_STACK_GEMM_IN × 64).

PREFILL_CHUNK_SIZE

Prompt tokens per prefill GPU batch (stack + staging footprint = emb_dim × this).

PREFILL_CHUNK_STACK_FLOATS

llm_agent stack chunk buffer (Gemma 2560 × 64).

VOCAB_CHUNK_ROWS

Vocabulary projection rows per chunked logits sweep. 10240 is the native GEMM output-buffer ceiling and a 256-row multiple, so resident logits chunk offsets stay storage-binding aligned while the current 49k-vocab model drops from six output chunks/token to five.

Statics

GPU_WAIT_COUNT

Decode-profiler: count of GPU submit → poll(Maintain::Wait) round-trips. Incremented by QTensorEngine::poll_wait (every native blocking sync point routes through it); read/reset by the bench to derive per-token synchronization overhead.

Functions

dequantize_token_embedding_into

Dequantize a mmap embedding row into caller-supplied out (no heap allocation).

gpu_wait_count

Total native GPU blocking-wait round-trips since the last reset.

reset_gpu_wait_count

Reset the GPU blocking-wait counter before a measured run.