Module dense_cpu
Expand description
Dense CPU transformer execution over caller-owned state and tensor kernels.
Tensor adapters supply real row decoding and GEMV, including target-specific quantization. This module executes the forward math; it supplies no tokenizer, architecture policy, backend fallback, sampler, publication or state rollback. A failed step may have changed KV and scratch. Positions remain caller-owned. Byte budgets cover the thirteen execution buffers, excluding borrowed model bindings, weights and callback-owned storage. Adapters must separately qualify their codec/target behavior and allocation contract; the trait cannot enforce it.
Structs§
Enums§
Traits§
- Tensor
Kernels - Implementations must decode the bound immutable tensors and allocate no hot storage. Calls receive the original used buffer lengths, excluding spare capacity. The trait is an execution boundary, not authentication of an implementation.
Functions§
- ingest_
token - Executes all dense layers and updates KV without a vocabulary projection.
- reset_
kv - Clears only used KV. This neither restores a checkpoint nor resets scratch/RNG.
- run_
token - Executes a complete dense token and finite greedy argmax; failure can leave partial KV/scratch updates. Caller-owned logits remain available for inspection.