Module residency_planner 

STELLAR §A AH-track H2 — residency + device-priority planner (decisions D24/D25/D30/D31).

Turns discovery (H0 HostTopology + H1(a) CapabilityMatrix) into an employment plan for a given model: which residency protocol, which device holds what, and the device priority order. It is a discovery-derived adaptive plan (D31) — not a fixed formula — and the core is a pure function of its inputs, so it is fully unit-testable with synthetic profiles (no GPU required).

Decision (D31), per machine, from measured inputs:

1. fits the highest-ranked circuit’s pool (minus a KV reserve) → Resident;
2. doesn’t fit → the overflow segment is placed by argmin(measured compute + measured transfer) over the candidate circuits (§ “Overflow cost model” below):
   • HeterogeneousOverflow when running the overflow in place on a large-pool secondary (iGPU/CPU reading system RAM, zero per-token transfer) is the cheapest estimate;
   • Streaming when double-buffering the overflow to the fast device over its bus (the A4 path — fast compute, but paying that device’s per-token transfer) is cheaper, or no in-place secondary big enough exists. Device priority order = the measured CapabilityMatrix order (D30), never a static hierarchy.

Overflow cost model (D31). Both axes are expressed as bytes over a measured bandwidth so they are directly comparable. Decode is memory-bound — the forward pass reads each weight once per token — so a segment’s compute time is estimated as overflow_bytes / compute_bytes_per_s, where compute_bytes_per_s is the circuit’s GEMV throughput (gemv_n² f32 elements / ms_per_gemv). A segment’s transfer time is overflow_bytes / (upload_gbps · 1e9); an in-pool circuit (upload_gbps = ∞, e.g. the CPU, or an iGPU running the overflow in its own system-RAM pool) pays zero transfer. The chosen protocol follows the per-segment argmin of compute + transfer — so a fast-but-far circuit (dGPU streaming overflow over PCIe) can lose to a slower-but-in-place circuit (iGPU), and vice-versa, purely on the numbers. This is an estimate from the measured throughput + bandwidth, not a profiled runtime (no attention/activation cost, no overlap of compute with transfer, memory-bound decode assumed); it is a principled ranking signal, honestly a first-order one. Native only.

Structs

DevicePlacement

One circuit’s assignment in the plan.

EmploymentPlan

The full employment plan — serializable for the cached passport / the progress record.

Enums

PlacementRole

What role a circuit plays in the plan.

ResidencyProtocol

The residency protocol chosen for a model on this host (D25).

Constants

DEFAULT_KV_RESERVE

Default KV-cache reserve on the primary compute device (matches the VRAM ledger cap).

ROUTE_ENV

Env flag gating H2 residency routing. Default OFF — when unset/false the route hooks do nothing behaviour-changing (a single cheap env read, no plan, no store).

Functions

last_employment_plan

Retrieve the last computed employment plan (H2 route), if any was recorded this process. This is the getter a future H3 execution stage consults; returns None when the flag was never enabled or no model has been loaded yet.

plan_employment

Pure planner: derive the employment plan from the discovered topology + capability matrix.

plan_for_model

Probe the real host (H0 + H1(a)) and plan for model_bytes. Heavy (runs the benchmark); for the fast path, plan against a cached matrix instead.

route_employment_for_model

H2 route (core): if enabled, compute the EmploymentPlan from the already-probed topology + capability matrix for a model of model_bytes, record it (log) and store it for retrieval — then return it. Gated by QUALIA_LLM_ROUTE; returns None (computing + storing nothing) when the flag is off.

route_employment_from_passport

H2 route (convenience): route using the cached hardware passport (topology + matrix already probed at boot — no re-probe, no benchmark, no GPU touched at load). Called from the model-load seam with the honest loaded weight-byte count. No-op when QUALIA_LLM_ROUTE is off or when there is no cached passport yet (logs a one-line note in the latter case).

route_enabled

Whether H2 residency routing is enabled (QUALIA_LLM_ROUTE). Default OFF.