pub fn recommend_convert_layout(
source_bytes: u64,
vram_budget_bytes: u64,
) -> P64ConvertLayoutExpand description
Recommend convert layout from source size (bytes on disk) and a VRAM headroom budget.
Heuristic:
- F16Expand when Q4→f16 estimate (
source * 4) fits undervram * 0.55and the 4 GiB p64 offset cap — best decode path for small/mid models. - Else Q4kSoa for mid/large Q4 containers (SoA is ~11% larger than Q4_K; always under the f16 budget). Decode GEMV drops scale-unpack barriers.
- Else Verbatim.