Module prompt_lookup 

W6a — prompt-lookup (n-gram) speculative decoding: the proposer.

Standard “prompt lookup decoding” (a.k.a. LLMA): instead of a separate draft model, draft the next few tokens by finding where the current context suffix already occurred earlier in the same context (prompt + generated), and proposing the tokens that FOLLOWED that earlier occurrence. Those drafts are then VERIFIED by one batched forward and the longest agreeing prefix is kept — so the emitted text is bit-identical to greedy decode (the verify step never accepts a token the model would not have produced greedily). The win is pure latency on repetitive / quoting / structured text (code, JSON, lists, cited passages); on non-repetitive text it proposes little and costs ~nothing. This module is the PROPOSER only — pure, allocation-light, unit-tested, no GPU. The verify/accept wiring lives in the decode loop.

Structs

Draft

A drafted continuation: tokens[..len] are the proposed next-token ids.

Constants

MAX_DRAFT

Hard cap on draft length (also bounded by the caller’s scratch/batch width).

MAX_NGRAM

Longest n-gram suffix length to try when looking for a recurrence.

Functions

propose

Propose up to max_draft tokens by matching the longest available suffix of ctx against an earlier occurrence in ctx, and returning the tokens that followed that occurrence.