Skip to main content

paged_gqa_attention_into

Function paged_gqa_attention_into 

pub fn paged_gqa_attention_into(
    query: &[f32],
    arena: &[f32],
    block_table: &[u32],
    config: &PagedKvConfig,
    position: u32,
    n_head: u32,
    out: &mut [f32],
) -> Result<(), AttentionError>
Expand description

Scalar oracle for the production paged GQA online-softmax kernel.

block_table is one layer’s logical-to-physical page slice. The function is zero-allocation and writes one query head at a time into out.