Skip to main content

set_kv_int8

qualia_core_db::inference::inference_bench

Function set_kv_int8 

Source 
pub fn set_kv_int8(on: bool)
Expand description

Enable/disable the int8 KV cache (QUALIA_LLM_KV_INT8).