Struct KvCacheLayout 

pub struct KvCacheLayout {
    pub max_context: u32,
    pub n_layer: u32,
    pub n_kv_head: u32,
    pub head_dim: u32,
    pub slot_kv_elems: u32,
    pub layer_stride: u32,
    pub total_f32_elems: usize,
    pub int8: bool,
    pub dict_k: u32,
    pub dict_n_atoms: u32,
}

Static ring-buffer KV layout: [layer][slot][K | V] in f32, OR (W5a int8 mode) packed int8 + per-(slot,kv_head) f32 scale in the same 4-byte-element buffer. total_f32_elems counts 4-byte slots either way (u32/f32 share the size), so the allocation math is unchanged.

Fields

max_context: u32

n_layer: u32

n_kv_head: u32

head_dim: u32

slot_kv_elems: u32

layer_stride: u32

total_f32_elems: usize

int8: bool

W5a: true ⇒ the arena is int8-quantized (K/V stored as packed i8 lanes + one f32 scale per (slot, kv_head), K then V). layer_stride is then the int8 slot layout, not the f32 one.

dict_k: u32

W5b Phase 4b: > 0 ⇒ the arena stores k-sparse dictionary codes (dict_k code-words per K/V vector — each word packs u16 atom-index | f16 coefficient), reconstructed in the attention shader. Mutually exclusive with int8 (dict mode wins). 0 ⇒ f32/int8 as above.

dict_n_atoms: u32

W5b Phase 4b: dictionary size (atoms) — the atoms live in each layer’s arena AFTER the code region ([K atoms n_atoms×head_dim][V atoms …]), so the per-layer binding covers them.

Implementations

impl KvCacheLayout

pub fn from_hyperparams(h: &GgufHyperparams) -> Option<Self>

pub fn ring_slot(&self, token_idx: u32) -> u32

pub fn k_index(&self, layer: u32, slot: u32, kv_head: u32, dim: u32) -> usize

pub fn v_index(&self, layer: u32, slot: u32, kv_head: u32, dim: u32) -> usize

pub fn code_index( &self, layer: u32, slot: u32, kv_head: u32, k_not_v: bool, i: u32, ) -> usize

W5b Phase 4b (dict mode): word offset of the i-th code word (0..dict_k) for the K (k_not_v = true) or V vector of (layer, slot, kv_head). Each code word packs u16 atom-index (high 16) | f16 coefficient (low 16). Slot layout mirrors f32 (K region then V region), but each vector is dict_k words instead of head_dim floats.

Trait Implementations

impl Clone for KvCacheLayout

fn clone(&self) -> KvCacheLayout

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for KvCacheLayout

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for KvCacheLayout

fn eq(&self, other: &KvCacheLayout) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Copy for KvCacheLayout

impl Eq for KvCacheLayout

impl StructuralPartialEq for KvCacheLayout