Skip to main content

Module queue

Module queue 

Source
Expand description

Bounded request queue and worker intake lifecycle.

Bridges asynchronous request intake to RequestScheduler and the sticky worker thread. Guarantees:

  1. Bounded intake queue (rejects with QueueFull rather than growing unbounded).
  2. Exact lifecycle states: Queued, Admitted, Prefill, Decode, Cancelled, Completed.
  3. Cancellation at every phase has exactly one terminal result and clean currency release.

Structs§

BoundedIntakeQueue
Bounded FIFO intake queue for the request scheduler.
IntakeRequest
Request descriptor within the bounded scheduler intake.

Enums§

IntakeError
IntakeState
Lifecycle state of a queued/running request.