Skip to main content

Module gguf_sharder

Module gguf_sharder 

Source
Expand description

Q-GGUF Hybrid Packaging Parses monolithic .gguf files: vocabulary (KV section) and tensor names/offsets (tensor-info section) are extracted into native Rust types; multi-gigabyte tensor payloads are left on disk for direct VRAM mapping via gguf_bridge.rs.

Modules§

hf_json
Hugging Face tokenizer.json parser.

Structs§

GGufSharder
Extracts the Ontological mapping and Lexicon from a raw GGUF file
GgufHyperparams
Architecture hyper-parameters parsed from the GGUF KV section.
GgufTensorInfo
Shape + type + offset for one tensor parsed from the GGUF tensor-info section.
GgufTokenizer
Vocabulary and BOS/EOS metadata extracted from a GGUF KV section. Used by infer_local_model() to encode prompts and decode output token IDs.
LayerTensors
Per-layer transformer weight metadata (all Option — absent tensors skipped).
PretokenSpan
Qwen4ExpPleConfig
Fixed-capacity PLE n-gram metadata recorded by a Qwen4Exp GGUF.

Enums§

ChatFamily
Chat-template family, detected from the special tokens a model’s vocab carries. Instruct models must have their prompt wrapped in this template (with an assistant-turn cue) or they degenerate (emit EOS immediately, or repeat) — a raw prompt gives the model no “your turn to answer” signal.
GgufHeaderError
PretokenError

Constants§

ARCH_DEEPSEEK_MOE
ARCH_FLAG_HAS_PLE
Feature flags on [ModelHyperparams::arch_flags].
ARCH_FLAG_HAS_QK_NORM
ARCH_FLAG_HAS_SHARED_KV
ARCH_FLAG_HAS_SOFTCAP
ARCH_FLAG_HAS_SWA
ARCH_GEMMA
ARCH_GEMMA2
ARCH_GEMMA3
ARCH_GEMMA4
Gemma 4 (E2B/E4B/…): dual head-dim SWA+global, PLE, shared KV — not standard Llama-shape.
ARCH_GLM4
ARCH_GRANITE_HYBRID
Granite hybrid SSM + attention architecture; requires recurrent state execution.
ARCH_LLAMA
ARCH_OTHER
ARCH_PHI4
ARCH_QWEN2
ARCH_QWEN4EXP
Qwen3.8 Flash Next (qwen4exp): hybrid GatedDeltaNet / QSA / MoE with a required hash-gathered PLE n-gram table.
ARCH_QWEN35_MOE
Qwen 3.5/3.6 MoE: hybrid SSM + periodic full attention, with fused QKV.
ARCH_UNKNOWN
Architecture id (stored in P64 hparams + used for support gating).
DEFAULT_ROPE_FREQ_BASE
Default RoPE base for Llama 3 / SmolLM2 when GGUF omits llama.rope.freq_base.
MAX_GGUF_HEADER_BYTES
Upper bound for the cold GGUF header read. This includes the vocabulary metadata and tensor-info directory, never any tensor payload. It keeps a native external-trunk activation from reserving/mapping the full model.
MAX_STOP_TOKEN_IDS
Max stop-token ids kept on the tokenizer (eos + chat-end family + extras).

Functions§

parse_architecture_id
Map general.architecture GGUF string → ARCH_*().
write_blk_tensor_name
Write blk.{layer}.{suffix} into out; returns total bytes written.
write_hf_layer_tensor_name
Write model.layers.{layer}.{suffix} into out; returns total bytes written.

Type Aliases§

GgufTensorIndex