Module metal_lane 

Metal mega-pass orchestrator: chain all transformer layers into one Metal command buffer with a single fence at the end. No per-layer readback.

This mirrors the CUDA cuda_lane/mega_pass.rs architecture but targets Apple Silicon via metal-rs or wgpu’s Metal backend. On non-Apple platforms, all functions return None/false — the wgpu WGSL path handles inference instead.

Architecture

1. Upload hidden state to device buffer (once).
2. For each layer: a. RMSNorm + QKV + RoPE (fused dispatch via fused-qkv-rope MSL kernel) b. KV cache write c. SDPA decode (via sdpa-decode MSL kernel) d. O-proj GEMV + residual add e. RMSNorm + SwiGLU (fused) f. Down GEMV + residual add
3. Output norm + logits GEMV + argmax
4. Single readback of the final token

All dispatches share one command buffer — the GPU never idles waiting for CPU between layers.

Structs

MetalPassLayerDims

Per-layer matmul dimensions (pre-computed by caller).

MetalPassLayerWeights

Per-layer weight references for the Metal mega-pass.

Functions

metal_double_buffered_prefetch

MSL double-buffered weight streaming: overlap H2D weight copies with compute using MTLBlitCommandEncoder on a separate Metal command buffer. This is the Metal equivalent of CUDA’s write_view_prefetch + join_prefetch on a secondary stream.

metal_mega_pass_available

Check if the Metal mega-pass is available on this platform.

try_metal_mega_pass

Metal mega-pass: chain all transformer layers into one Metal command buffer with a single fence at the end. No per-layer readback. The hidden state stays on-device for the entire forward pass; only the final logits/token come back to host.

warm_metal_context

Warm the Metal context (pre-compile kernels, pre-allocate arenas). On non-Apple platforms, this is a no-op.