Skip to main content

kv_int8_enabled

qualia_core_db::inference::inference_bench

Function kv_int8_enabled 

Source 
pub fn kv_int8_enabled() -> bool
Expand description

Whether the KV cache should be int8-quantized.