Expand description
Stub when CUDA feature is off — CUDA lane APIs return false / no-op.
Structs§
Constants§
Functions§
- cache_
dense_ weight - No-op — no device weight cache exists without CUDA.
- clear_
weight_ cache - No-op — no cache to clear without CUDA.
- dense_
weight_ cached - Returns
false— nothing is ever cached without CUDA. - device_
kv_ ready - Returns
false— device KV cache never exists without CUDA. - ensure_
device_ kv_ cache - Returns
false— no device KV cache can be allocated without CUDA. - preload_
q4k_ soa_ weights - Returns 0 — no device weight preloading without CUDA.
- q4k_
device_ weight_ count - Returns 0 — no Q4K weights are resident on-device without CUDA.
- try_
cuda_ batch_ gemv - Returns
false— CUDA GEMV unavailable; caller falls back to CPU. - try_
cuda_ batch_ gemv_ cached - Returns
false— CUDA GEMV unavailable; caller falls back to CPU. - try_
cuda_ batch_ gemv_ cached_ only - Returns
false— no cached device weights exist without CUDA. - try_
cuda_ mega_ pass - Returns
None— the fused mega-pass is unavailable without CUDA. - try_
q4k_ soa_ attention_ device - Returns
false— device-side attention unavailable without CUDA. - try_
q4k_ soa_ ffn_ block - Returns
false— fused FFN block unavailable without CUDA. - try_
q4k_ soa_ ffn_ block_ residual - Returns
false— fused residual FFN block unavailable without CUDA. - try_
q4k_ soa_ fused_ swiglu - Returns
false— fused SwiGLU unavailable without CUDA. - try_
q4k_ soa_ gemv - Returns
false— Q4K device GEMV unavailable without CUDA. - try_
q4k_ soa_ qkv - Returns
false— fused QKV projection unavailable without CUDA. - warm_
cuda_ context - Returns
false— no CUDA context to warm. - weight_
cache_ len - Returns 0 — the weight cache is empty by definition without CUDA.
- weight_
fingerprint - Returns 0 — no weight fingerprinting without CUDA (feature
cudaoff or wasm).