Skip to main content

Module pools

Module pools 

Source

Structsยง

BlockGeometry
Geometry of a physical KV cache block/page.
MemoryPoolBudget
Joint memory plan configuration across device VRAM and pools.