Struct TernaryFfnResident 

pub struct TernaryFfnResident { /* private fields */ }

A1b (STELLAR §A): the resident 2-bit branchless ternary-FFN GEMM dispatcher — the perf core.

The D7 fix. The 2-bit kernel pipeline + bind-group layout are built once; every FFN ternary weight is rebaked base-3 → 2-bit (lossless; see rebake_ternary_blob_to_2bit) and uploaded once into a single resident VRAM buffer. (The prior 1.02× failure rebuilt the pipeline per call.) At decode each FFN GEMV binds its own weight sub-range — no re-upload — writes only the activation row + params, dispatches, and reads back n_out floats. Tensors are keyed by their P64 blob offset (GgufTensorInfo::byte_offset), the unique per-tensor handle.

On-disk the FFN weights stay base-3 (densest, 1.6 bit); 2-bit is the GPU-resident layout only (2.0 bit, shift/mask, divergence-free → 1.77× vs F16 on the A2000). Native-only; the wasm WebGPU ternary path reuses the same WGSL through the MC8 resident arena (a later step).

Implementations

impl TernaryFfnResident

pub fn build( device: &Device, queue: &Queue, tensors: &[(u64, usize, usize, &[u8])], ) -> Option<Self>

Build the resident set from (key, n_in, n_out, base3_blob) tuples. Each base-3 blob ([scale f32][5-trits/byte]) is rebaked to the 2-bit runtime layout here — load-time heap is the sanctioned path; the hot loop stays zero-heap. key is the tensor’s P64 blob offset. Returns None if tensors is empty or any blob is malformed.

pub fn contains(&self, key: u64) -> bool

Whether a tensor with key is resident (and its shape, for the dispatch guard).

pub fn resident_bytes(&self) -> u64

Total resident weight bytes (the VRAM footprint of the ternary FFN).

pub fn len(&self) -> usize

Number of resident ternary tensors.

pub fn is_empty(&self) -> bool

pub fn gemv( &self, device: &Device, queue: &Queue, key: u64, input: &[f32], out: &mut [f32], n_in: usize, n_out: usize, ) -> bool

Run out[..n_out] = scale · W·act for the resident tensor key (batch 1 = decode GEMV). Returns false (caller falls back) if the key is absent, the shape mismatches the resident metadata, the IO bounds are exceeded, or the GPU readback fails — fail-closed, never garbage.