Skip to main content

spec_decode_enabled

Function spec_decode_enabled 

Source
pub fn spec_decode_enabled() -> bool
Expand description

Whether the decode loop should run prompt-lookup speculative decode (the effective mode: env var wins if set, else the runtime flag). Read this to reflect the current mode in a UI.