Expand description
Joint memory budget planning and checked arithmetic.
Provides explicit accounting for static model footprints, hardware runtime context, static graph capture headroom, physical page block geometry, and admission reservations (including input, reserved output, COW transients, and page boundary fragmentation).
Modules§
Structs§
- Block
Geometry - Geometry of a physical KV cache block/page.
- Memory
Pool Budget - Joint memory plan configuration across device VRAM and pools.
- Model
Memory Profile - Static memory profile of a loaded model on a device backend.
- Request
Reservation - An admitted request’s checked memory reservation.
Enums§
- Budget
Error - Errors returned during budget computation and admission reservation.