Skip to main content

Module budget

Module budget 

Source
Expand description

Joint memory budget planning and checked arithmetic.

Provides explicit accounting for static model footprints, hardware runtime context, static graph capture headroom, physical page block geometry, and admission reservations (including input, reserved output, COW transients, and page boundary fragmentation).

Modules§

model
pools
reservation

Structs§

BlockGeometry
Geometry of a physical KV cache block/page.
MemoryPoolBudget
Joint memory plan configuration across device VRAM and pools.
ModelMemoryProfile
Static memory profile of a loaded model on a device backend.
RequestReservation
An admitted request’s checked memory reservation.

Enums§

BudgetError
Errors returned during budget computation and admission reservation.