Skip to main content

ffn_f16_enabled

qualia_core_db::inference::inference_bench

Function ffn_f16_enabled 

Source 
pub fn ffn_f16_enabled() -> bool
Expand description

Whether FFN weights should be promoted to f16 in VRAM for decode/prefill GEMV.