Trait PreparedDecodePlan 

pub trait PreparedDecodePlan {
    // Required methods
    fn backend(&self) -> PreparedBackend;
    fn state(&self) -> PreparedPlanState;
    fn counters(&self) -> ExecutionCounters;
    fn run_decode_step(
        &mut self,
        input: DecodeStepInput<'_>,
        output: &mut DecodeStepOutput,
    ) -> Result<(), DecodeStepError>;
}

Tier-1 execution contract for an already prepared model/backend pair.

Required Methods

fn backend(&self) -> PreparedBackend

fn state(&self) -> PreparedPlanState

fn counters(&self) -> ExecutionCounters

fn run_decode_step( &mut self, input: DecodeStepInput<'_>, output: &mut DecodeStepOutput, ) -> Result<(), DecodeStepError>

Execute one autoregressive step.

Implementations must not allocate, compile, discover tensors, upload immutable data, or select another backend. A failure is returned to the owner, which records any explicit fallback outside this call.

Implementors