Module inference_gpu_profiler 

Per-kernel GPU timing for the native LLM forward/decode path (W2 / D17).

Wraps each LLM compute pass with TIMESTAMP_QUERY writes, resolves the query set, and accumulates GPU-internal nanoseconds per Phase. Gated by a runtime flag (QUALIA_LLM_GPU_PROFILE=1 or set_enabled) so production decode pays nothing: when disabled, the pass_writes_* helpers return None and the pass is byte-identical to before (timestamp_writes: None).

Requires the shared device to have negotiated TIMESTAMP_QUERY (see crate::gpu_context::SharedGpuContext::timestamps_supported); degrades to a no-op otherwise, so it is safe to call unconditionally from every dispatch site.

Honesty note: the per-phase nanoseconds are GPU-internal (begin→end of the pass) and individually accurate. A profiling run serialises per-op readback, so the headline tok/s of a profiled run is perturbed — report the per-phase split, not the profiled tok/s.

Not re-entrant: it uses one shared 2-slot query set, which the single LLM engine thread’s per-op blocking readback serialises. That matches how decode actually runs.

Structs

PhaseTiming

Accumulated GPU time for one phase across all dispatches since the last reset.

Enums

Phase

Distinct kernel families on the LLM forward/decode path.

Functions

accumulate

Read the resolved timestamps and add end - begin ns to phase’s accumulator. Call after the kernel’s submit + device poll. No-op when profiling is off.

any_recorded

True if any phase recorded GPU time since the last reset (test/diagnostic helper).

enabled

True when profiling is requested AND the shared device negotiated TIMESTAMP_QUERY.

pass_writes_begin

timestamp_writes for the FIRST pass of a multi-pass kernel (begin only).

pass_writes_both

timestamp_writes for a single-pass kernel (writes both begin and end on this pass). Returns None (zero overhead) when profiling is off — drop straight into the descriptor.

pass_writes_end

timestamp_writes for the LAST pass of a multi-pass kernel (end only).

reset

Zero all phase accumulators (call before a measured run).

resolve

Encode the query-set resolve into encoder (call after the pass, before finish()). No-op when profiling is off.

set_enabled

Force GPU pass profiling on/off at runtime (bench-only; production leaves it off).

snapshot

Snapshot all phase accumulators (for the bench JSON).