Skip to main content

ingest_token

Function ingest_token 

pub fn ingest_token<H, K>(
    plan: &DensePlan<'_, H>,
    kernels: &mut K,
    workspace: DenseWorkspace<'_>,
    token: u32,
    position: usize,
    byte_budget: usize,
) -> Result<(), DenseError<<K as TensorKernels<H>>::Error>>
where H: Copy, K: TensorKernels<H>,
Expand description

Executes all dense layers and updates KV without a vocabulary projection.