Expand description
Q-GGUF Hybrid Packaging
Parses monolithic .gguf files: vocabulary (KV section) and tensor names/offsets
(tensor-info section) are extracted into native Rust types; multi-gigabyte tensor
payloads are left on disk for direct VRAM mapping via gguf_bridge.rs.
Modules§
- hf_json
- Hugging Face
tokenizer.jsonparser.
Structs§
- GGuf
Sharder - Extracts the Ontological mapping and Lexicon from a raw GGUF file
- Gguf
Hyperparams - Architecture hyper-parameters parsed from the GGUF KV section.
- Gguf
Tensor Info - Shape + type + offset for one tensor parsed from the GGUF tensor-info section.
- Gguf
Tokenizer - Vocabulary and BOS/EOS metadata extracted from a GGUF KV section.
Used by
infer_local_model()to encode prompts and decode output token IDs. - Layer
Tensors - Per-layer transformer weight metadata (all
Option— absent tensors skipped). - Pretoken
Span - Qwen4
ExpPle Config - Fixed-capacity PLE n-gram metadata recorded by a Qwen4Exp GGUF.
Enums§
- Chat
Family - Chat-template family, detected from the special tokens a model’s vocab carries. Instruct models must have their prompt wrapped in this template (with an assistant-turn cue) or they degenerate (emit EOS immediately, or repeat) — a raw prompt gives the model no “your turn to answer” signal.
- Gguf
Header Error - Pretoken
Error
Constants§
- ARCH_
DEEPSEEK_ MOE - ARCH_
FLAG_ HAS_ PLE - Feature flags on [
ModelHyperparams::arch_flags]. - ARCH_
FLAG_ HAS_ QK_ NORM - ARCH_
FLAG_ HAS_ SHARED_ KV - ARCH_
FLAG_ HAS_ SOFTCAP - ARCH_
FLAG_ HAS_ SWA - ARCH_
GEMMA - ARCH_
GEMM A2 - ARCH_
GEMM A3 - ARCH_
GEMM A4 - Gemma 4 (E2B/E4B/…): dual head-dim SWA+global, PLE, shared KV — not standard Llama-shape.
- ARCH_
GLM4 - ARCH_
GRANITE_ HYBRID - Granite hybrid SSM + attention architecture; requires recurrent state execution.
- ARCH_
LLAMA - ARCH_
OTHER - ARCH_
PHI4 - ARCH_
QWEN2 - ARCH_
QWEN4EXP - Qwen3.8 Flash Next (
qwen4exp): hybrid GatedDeltaNet / QSA / MoE with a required hash-gathered PLE n-gram table. - ARCH_
QWEN35_ MOE - Qwen 3.5/3.6 MoE: hybrid SSM + periodic full attention, with fused QKV.
- ARCH_
UNKNOWN - Architecture id (stored in P64 hparams + used for support gating).
- DEFAULT_
ROPE_ FREQ_ BASE - Default RoPE base for Llama 3 / SmolLM2 when GGUF omits
llama.rope.freq_base. - MAX_
GGUF_ HEADER_ BYTES - Upper bound for the cold GGUF header read. This includes the vocabulary metadata and tensor-info directory, never any tensor payload. It keeps a native external-trunk activation from reserving/mapping the full model.
- MAX_
STOP_ TOKEN_ IDS - Max stop-token ids kept on the tokenizer (eos + chat-end family + extras).
Functions§
- parse_
architecture_ id - Map
general.architectureGGUF string →ARCH_*(). - write_
blk_ tensor_ name - Write
blk.{layer}.{suffix}intoout; returns total bytes written. - write_
hf_ layer_ tensor_ name - Write
model.layers.{layer}.{suffix}intoout; returns total bytes written.