Skip to main content

Module safetensor_loader

Module safetensor_loader 

Source
Expand description

Safetensor model loader — native zero-copy adoption of Safetensors models (including BF16).

Maps Hugging Face safetensors format (model.layers.N..., lm_head, model.embed_tokens) and GGUF-named safetensors into canonical GgufTensorIndex references without transcoding. Supports BF16 (GGML 30), F16 (GGML 1), and F32 (GGML 0).

Structs§

ModelJsonConfig
SafetensorNvfp4Matrix
Zero-copy view of one NVFP4-quantized matrix inside a Safetensors mmap (U8 packed nibbles + F8_E4M3 block scales + f32 global scale).

Constants§

GGML_TYPE_NVFP4_PACKED
GGML synthetic type tag for packed NVFP4 (U8 weight bytes paired with F8_E4M3 .weight_scale and F32 .weight_scale_2).

Functions§

parse_safetensor_to_index
Parse a safetensor mmap and build a canonical GgufTensorIndex.
resolve_nvfp4_matrix
Resolve an NVFP4 matrix triplet ({prefix}.weight, {prefix}.weight_scale, {prefix}.weight_scale_2) from a Safetensors mmap.