Skip to main content

resident_weights_enabled

Function resident_weights_enabled 

Source
pub fn resident_weights_enabled() -> bool
Expand description

Whether native GEMM should bind resident per-tensor weight buffers (upload-once) rather than re-uploading the weight every token. Env forces either direction; otherwise the atomic flag.