Struct CudaComputeContext 

pub struct CudaComputeContext {
    pub ctx: Arc<CudaContext>,
    pub stream: Arc<CudaStream>,
    pub prefetch_stream: Option<Arc<CudaStream>>,
    pub module_cache: Mutex<HashMap<u64, (CudaFunction, Arc<CudaModule>, Arc<KernelSpec>)>>,
    pub adapter: AdapterIdentity,
    pub constraints: AdapterConstraints,
    pub allocator: QualiaSlabAllocator,
    pub slab: CudaSlice<u8>,
}

Fields

ctx: Arc<CudaContext>

stream: Arc<CudaStream>

prefetch_stream: Option<Arc<CudaStream>>

Secondary stream for overlapping H2D parameter writes with compute. Lazily created on first write_view_prefetch call to avoid overhead when double-buffering is not used.

module_cache: Mutex<HashMap<u64, (CudaFunction, Arc<CudaModule>, Arc<KernelSpec>)>>

Cache of loaded CUDA functions keyed by (source_hash, entry_point). Avoids redundant load_module JIT on every compile_pipe! call — the PTX text is already cached in NVRTC_PTX_CACHE, but the driver module load is a separate JIT step that was repeated per token.

adapter: AdapterIdentity

constraints: AdapterConstraints

allocator: QualiaSlabAllocator

slab: CudaSlice<u8>

Implementations

impl CudaComputeContext

pub fn new(capacity_bytes: usize) -> Result<Self, ForgeError>

pub fn begin_graph_capture(&self) -> Result<(), ForgeError>

Begin thread-local capture on the prepared compute stream.

pub fn end_graph_capture(&self) -> Result<CapturedCudaGraph, ForgeError>

Finish, instantiate and upload the current compute-stream capture.

pub fn launch_graph(&self, graph: &CapturedCudaGraph) -> Result<(), ForgeError>

Enqueue one replay on the graph’s retained compute stream.

pub fn allocate_and_write( &mut self, data: &[u8], binding: u32, group: u32, ) -> Result<BufferView, ForgeError>

pub fn write_view_prefetch( &mut self, view: &BufferView, data: &[u8], ) -> Result<(), ForgeError>

Overwrite a device view with host bytes on the prefetch stream, overlapping with compute on the primary stream. Caller must invoke [join_prefetch] before launching a kernel that reads this data.

pub fn join_prefetch(&self) -> Result<(), ForgeError>

Make the compute stream wait for all outstanding prefetch-stream work. Call this before launching a kernel that depends on prefetched data.

pub fn write_view( &mut self, view: &BufferView, data: &[u8], ) -> Result<(), ForgeError>

Overwrite an existing device view with host bytes (no new allocation). data.len() must be ≤ view.length_bytes.

pub fn allocate_transient( &mut self, size_bytes: usize, binding: u32, group: u32, ) -> Result<BufferView, ForgeError>

pub fn advance_read_head(&mut self, offset: usize)

pub fn clear_transient_allocations(&mut self)

pub fn write_checkpoint(&self) -> u64

See QualiaSlabAllocator::write_checkpoint.

pub fn restore_checkpoint(&mut self, write_count: u64)

See QualiaSlabAllocator::restore_checkpoint.

pub fn read_buffer_f32(&self, view: &BufferView) -> Result<Vec<f32>, ForgeError>

pub fn read_buffer_u32_into( &self, view: &BufferView, output: &mut [u32], ) -> Result<(), ForgeError>

Copy a device view into a caller-owned u32 slice.

Unlike Self::read_buffer_f32, this performs no host allocation. It is the decode token-readback boundary: the four-byte copy also synchronizes all preceding stream work.

pub fn read_buffer_f64(&self, view: &BufferView) -> Result<Vec<f64>, ForgeError>

Double-precision readback, the f64 mirror of Self::read_buffer_f32 (8 bytes/elem). Used by the native CUDA-f64 GEMM path — WGSL has no f64, so this is CUDA-only by construction.

Trait Implementations

impl OracleContext for CudaComputeContext

Available on crate feature cuda only.

fn run_kernel( &mut self, kernel: &KernelSpec, schedule: &Schedule, buffers: &[BufferView], element_count: usize, warmups: usize, samples: usize, ) -> Result<Vec<u64>, ForgeError>

Compile the kernel’s CUDA-C (NVRTC → PTX, emitted internally by CudaPipeline::compile_cuda_c) and run the warmup + timed-sample dispatch loop. Mirrors the wgpu loop shape so the generic oracle is backend-agnostic; the cross-backend CUDA oracle uses warmups = 0, samples = 1, reproducing the single dispatch the previous evaluate_*_cuda functions performed.

fn allocate_and_write( &mut self, data: &[u8], binding: u32, group: u32, _usage: BindingUsage, ) -> Result<BufferView, ForgeError>

Allocate a transient slab sub-range and upload data into it. usage selects the backing slab on wgpu (read-only/uniform vs read-write); the CUDA backend addresses one slab and ignores it.

fn allocate_transient( &mut self, size_bytes: usize, binding: u32, group: u32, _usage: BindingUsage, ) -> Result<BufferView, ForgeError>

Allocate a transient (uninitialised) slab sub-range. usage is honoured by the wgpu backend and ignored by CUDA, as for Self::allocate_and_write.

fn read_buffer_f32(&self, view: &BufferView) -> Result<Vec<f32>, ForgeError>

Read a device buffer back as f32s.

fn clear_transient_allocations(&mut self)

Release every transient allocation (resets the ring’s read head to the write head). Only valid once all device work on those ranges has completed.

fn adapter(&self) -> &AdapterIdentity

The adapter identity (vendor/device/driver) backing this context.

fn constraints(&self) -> &AdapterConstraints

The adapter’s intrinsic capability/limit constraints.

fn timestamp_supported(&self) -> bool

Whether GPU-timestamp timing is available (wgpu only; CUDA times on the host wall clock, so this is always false there).