Expand description
Prepared native inference runtime.
This library is the plan/run boundary for production inference. Model discovery, compilation, immutable upload, receipts, and artifact management are cold control-plane work. A prepared backend’s decode step is the Tier-1 zero-heap data plane.
Re-exports§
pub use artifacts::cleanup_stale_runs;pub use artifacts::ArtifactError;pub use artifacts::ArtifactFinish;pub use artifacts::ArtifactRetention;pub use artifacts::ArtifactStats;pub use artifacts::RunArtifactDir;pub use artifacts::StaleCleanupReport;pub use prepared::DecodeStepError;pub use prepared::DecodeStepInput;pub use prepared::DecodeStepOutput;pub use prepared::PreparedBackend;pub use prepared::PreparedDecodePlan;pub use prepared::PreparedPlanState;pub use receipt::capture_source_provenance;pub use receipt::sha256_file;pub use receipt::sha256_token_ids;pub use receipt::ArtifactCleanupCounters;pub use receipt::BackendKind;pub use receipt::BenchmarkManifest;pub use receipt::ExecutionCounters;pub use receipt::ExecutionReceipt;pub use receipt::SourceProvenance;pub use receipt::MANIFEST_SCHEMA_VERSION;pub use receipt::RAW_GREEDY_DECODE_POLICY;pub use receipt::RECEIPT_SCHEMA_VERSION;pub use scheduler::Admission;pub use scheduler::RequestScheduler;pub use scheduler::RequestState;pub use scheduler::RequestView;pub use scheduler::SchedulerError;
Modules§
- artifacts
- Bounded temporary artifacts and explicit evidence promotion.
- graph_
assist - Explicit graph-assisted inference ABI.
- kv
- KV-cache planning and ownership.
- prepared
- Backend-neutral prepared decode contract.
- receipt
- Serializable cold-path evidence and fixed-size hot-path counters.
- scheduler
- Bounded request scheduling for paged native inference.