Expand description
Deterministic WGSL generation, validation, certification, and tuning.
WGSL Forge treats shader semantics and hardware scheduling as separate typed inputs. Tuning is therefore allowed to change work distribution without changing the mathematical operation being certified.
Re-exports§
pub use runtime::ForgeRuntime;pub use dispatch::caps;pub use dispatch::coopmat_usable;pub use dispatch::fft_f32;pub use dispatch::gemm_cpu_f64;pub use dispatch::gemm_f32;pub use dispatch::gemm_f32_tc;pub use dispatch::gemm_f32_tc_coopmat;pub use dispatch::gemm_f32_tc_reduced;pub use dispatch::gemm_f64;pub use dispatch::gemm_f64_df64;pub use dispatch::gemv_cpu_f64;pub use dispatch::gemv_f32;pub use dispatch::gemv_f64;pub use dispatch::pairwise_sq_dist_cpu_f64;pub use dispatch::pairwise_sq_dist_f64;pub use observer_kernels::*;
Modules§
- attention
- audio
- Forge-native audio kernels (GPU compute with a CPU oracle + parity certify).
- backend
- cache
- calibration
- W10 — Forge calibration pipeline (the training-related upgrade).
- compute_
passes - Portable compute execution over caller-supplied data.
- device_
options - dispatch
- emit
- etalon_
bridge - execute
- graph_
ops - ir
- manifest
- observer_
kernels - Five Normative Observer Projection Kernels for Holographic & Continuum Rendering.
- oracle
- physics
- Certified physics forge kernels — complete, deterministic compute shaders with exact CPU oracles, naga validation, and GPU certification.
- profiling
- Caller-owned two-slot pass timer. Serialize use and request TIMESTAMP_QUERY first.
- roofline
- runtime
- schedule
- substrate_
gemm - The portable wgpu f32 GEMM — the execute half of the bridge. Until now
compute_bridgeonly probed and planned; this actually runs a kernel on the GPU and reads the result back, reusing the exact wgpu idioms the capability benchmark (crate::numerical::substrate_gemm) already uses on this hardware. - tune
- validate
Structs§
- Adapter
Constraints - Adapter
Identity - Affine
Params - Attention
Schedule - A hardware-safe attention schedule. The generated kernel uses one workgroup per attention head and an online tiled reduction over context.
- Attention
Shader - Generated attention kernel plus its Naga validation report.
- Attention
Shader Params - Uniform parameter block matching
AttentionParamsin the generated attention shader. - Attention
Shape - The attention workload bound into one Forge kernel.
- Buffer
Spec - Candidate
Evaluation - Candidate
Failure - Candidate
Result - Certification
Manifest - Comparison
Report - Compute
Caps - Probed compute capabilities of this machine. Every flag reflects what was
actually constructible at probe time, not what the build was compiled with — a
cuda-feature build on a machine with no NVIDIA device still reportscuda == false. - Etalon
Generated Shader - Etalon
Kernel Spec - Etalon
Schedule - Etalon
Validation Report - FfnParams
- 16-byte uniform block for the fused-FFN kernel.
- FftParams
- 16-byte uniform block for the radix-2 FFT kernel:
ncomplex elements,log2n = log2(n). The kernel runs one workgroup ofnthreads. - Generated
Shader - GpuEvaluation
- Hardware
Profile - Rich, queryable description of the local compute hardware (plan §9
profile-hardware). Acts as the topology fingerprint that keys the manifest cache (plan §8): tuning records are only reused when the topology hash matches. - Kernel
Spec - Manifest
Cache - Atomic, adapter-keyed storage for certification and tuning evidence.
- Oracle
Case - Oracle
Tolerance - P64Gpu
Words64 - 64-byte aligned GPU descriptor container for P64 weight records.
- Roofline
Estimate - Schedule
- Schedule
Space - Shared
Memory Spec - Timing
Summary - TopK
Params - 16-byte uniform block for the top-k kernel (
block_size== workgroup size). - Tuning
Config - Tuning
Manifest - Tuning
Result - Validation
Report
Enums§
- Buffer
Access - Buffer
Element - Builtin
Kernel - Etalon
Builtin Kernel - Etalon
Target Backend - Forge
Error - Op
- Roofline
Bound - Whether a kernel is dominated by memory traffic or by arithmetic.
- Scalar
Type - Shared
Len - Target
Backend - Timing
Source - Validation
Level
Constants§
- CUDARC_
API_ VERSION cudarccrate API version the cross-backend (CUDA) oracle is built against. Folded into the tuning/certification cache key (plan §8) so reuse is invalidated when the CUDA toolchain surface changes.- ETALON_
FORGE_ CACHE_ MAGIC - ETALON_
FORGE_ SCHEMA_ VERSION - FORGE_
SCHEMA_ VERSION - GEMM_
GPU_ THRESHOLD - Problem-size threshold (in
m * n * kmultiply-adds) below which GEMM stays on the CPU regardless of available accelerators. Small GEMMs are dominated by dispatch/transfer overhead, so the GPU path only earns its keep above this size.1 << 15(32768 FMAs, e.g. a 32×32×32 GEMM) is a conservative crossover that keeps the hand-checked unit tests (well below it) firmly on the CPU floor. - NAGA_
API_ VERSION - WGPU_
API_ VERSION
Functions§
- candidate_
evaluation - Evaluates a candidate WGSL expression.
- causal_
attention_ oracle - Caller-buffered stable scaled-dot-product attention oracle.
- certify_
builtin - Certifies a builtin oracle against expected outputs.
- certify_
etalon_ kernel - Certifies a WGSL kernel source string via Naga validation.
- compare_
f32 - Compares f32 results against tolerance. Nonfinite values fail. Invalid tolerances report a failure even for empty inputs.
- decode_
spirv_ words - Decode a
;-joined decimal SPIR-V word string back intoVec<u32>. - dft_cpu
- Naive
O(N²)forward Discrete Fourier Transform, the reference the GPU radix-2 FFT is differentially checked against. Complex data is interleaved f32: elementjis(input[2*j], input[2*j+1]) = (real, imag), so both theinputslice and the returned vector hold2*Nf32. - emit_
etalon_ shader - Emits a complete shader module from a
KernelSpecandSchedulefortarget. - emit_
shader - Emits
shader. - evaluate_
builtin - Evaluates a WGSL builtin on host for oracle checking.
- evaluate_
ffn - Cross-backend differential-oracle evaluation for the fused FFN against
ffn_cpu(plan §7). Generic overOracleContext— the same code runs on wgpu and CUDA; onlyOracleContext::run_kerneldiffers. One workgroup-thread per output element; the output buffer isoutput_size. Tensors, bindings, dispatch sizing and tolerance are identical to the prior wgpu/evaluate_ffn_cudapaths. - evaluate_
fft - Differential-oracle evaluation of the radix-2 FFT (
out = forward DFT(in)) againstdft_cpu. One workgroup ofn = schedule.workgroup_sizethreads (one complex element per thread;nmust be a power of two), mirroring the single-workgroup dispatch ofevaluate_topk():element_count = nwithworkgroup_size = nlaunches exactly one workgroup. The input/output buffers hold2*ninterleaved f32. - evaluate_
matmul_ tc - Differential-oracle evaluation of the cooperative-matrix (tensor-core) 8x8
GEMM tile
C = A * Bagainstmatmul_cpu. All-f32 — the only coopmat configuration wgpu/naga 29 implements (seecrate::numerical::emit::coopmat). One subgroup (32-lane NVIDIA warp) cooperatively computes the tile; the row-major loads/store reproduce the row-major CPU reference, so agreement is to f32 precision (a tiny tolerance covers tensor-core accumulation order). - evaluate_
p64 - Differential-oracle evaluation for the P64 projection against
p64_project_cpu. - evaluate_
topk - Cross-backend differential-oracle evaluation for the top-k kernel against
topk_cpu(plan §7). Generic overOracleContext— the same code runs on wgpu and CUDA; onlyOracleContext::run_kerneldiffers. - ffn_cpu
- CPU reference for the fused FFN, matching the emitted kernel’s op order
exactly (hidden outer, input inner) so GPU/CPU agree within tolerance:
out[o] = sum_h w2[o,h] * gelu(sum_i w1[h,i] * input[i]). - ffn_
tensors - Deterministic FFN test tensors. Weights are scaled by 1/sqrt(fan_in) so the pre-activations stay O(1) and GPU/CPU agree within a modest tolerance.
- fft_
inputs - Deterministic complex test signal as interleaved f32 (
2*nvalues), drawn from the same xorshift stream as every other oracle vector so it is reproducible. Both the real and imaginary parts land in[-1, 1]. - generate_
and_ certify_ via_ etalon - Compatibility wrapper that discards the report from
generate_with_validation_via_etalon. Successful emission alone does not establish device or numerical correctness. - generate_
builtin - Generates a WGSL builtin.
- generate_
with_ validation_ via_ etalon - Emits a shader and retains Etalon’s WGSL validation report unchanged.
- install_
device_ policy - matmul_
cpu - Row-major n×n matrix multiply reference:
c[i][j] = sum_k a[i][k] * b[k][j]. - matmul_
tc_ wgsl - Emits the cooperative-matrix 8x8 GEMM tile
C = A * B, all-f32. - p64_
project_ cpu - CPU reference for the P64 projection:
out[r] = sum_w weights[w] * f32(word_w), reading the 16 packed u32 words in the same lane order as the kernel. - p64_
records - Deterministic P64 descriptors with small (f32-exact) u32 words.
- resolve_
execution_ backend - Resolve which backend the execution/validation pipeline should actually use,
given a
preferredtarget and a predicate reporting whether a given backend’s native toolchain is available on this host. - roofline_
for - Roofline estimate for
kernelover a representative sizen(output elements / records / rays, depending on the kernel). - synthesize_
causal_ attention - Emit and Naga-validate a causal scaled-dot-product attention kernel.
- topk_
cpu - CPU reference for the per-block top-k: the
klargest values of eachblock_size-element block, in descending order. Blocks shorter thanblock_size(the tail) are padded with the sentinel, mirroring the GPU kernel’s out-of-range loads. - topk_
inputs - Deterministic xorshift test vector in
[-1, 1], matching the affine generator. - tune_
with - Tunes with the given config.
- validate_
native - Validate a non-WGSL shader by spawning the target’s offline compiler. When the
source was generated from a known
KernelSpec, pass it so the report carries that kernel’s real entry point and binding count; for an opaque external source (kernel = None) those fields are left empty/zero rather than guessed. - validate_
wgsl - Validates
wgsl.