Struct ForwardArena
pub struct ForwardArena {
pub max_batch: u32,
pub hidden_dim: u32,
pub num_layers: u32,
pub hidden_state_buf: Buffer,
pub residual_buf: Buffer,
pub norm_buf: Buffer,
pub readback_stage_buf: Buffer,
/* private fields */
}Expand description
Unified forward execution arena on an Etalon device.
Fields§
§max_batch: u32§num_layers: u32§residual_buf: Buffer§norm_buf: Buffer§readback_stage_buf: BufferImplementations§
§impl ForwardArena
impl ForwardArena
pub async fn try_read_token_async(
&self,
device: &Device,
) -> Result<u32, TokenReadbackError>
pub async fn try_read_token_async( &self, device: &Device, ) -> Result<u32, TokenReadbackError>
Submit the token copy first. Dropping the future cancels and unmaps staging. Native executors drive one nonblocking device poll per turn until completion or five seconds. No background worker survives cancellation. Browser callbacks use the event loop.
pub async fn read_token_async(&self, device: &Device) -> Option<u32>
pub fn try_read_token_blocking( &self, device: &Device, ) -> Result<u32, TokenReadbackError>
§impl ForwardArena
impl ForwardArena
pub fn new( device: &Device, max_batch: u32, hidden_dim: u32, num_layers: u32, ) -> ForwardArena
pub fn try_new_with_budget( device: &Device, max_batch: u32, hidden_dim: u32, num_layers: u32, byte_budget: u64, ) -> Result<ForwardArena, TensorArenaError>
pub fn readback_byte_size(&self, mode: ForwardMode) -> usize
pub fn readback_byte_size(&self, mode: ForwardMode) -> usize
Readback size in bytes for a given forward mode. Eliminates full logit tensor readbacks (reduces 9.72 MB -> 4-64 bytes).
pub fn try_readback_byte_size( &self, mode: ForwardMode, ) -> Result<u64, TensorArenaError>
pub fn record_readback_copy(
&self,
encoder: &mut CommandEncoder,
token_src: &Buffer,
mode: ForwardMode,
)
pub fn record_readback_copy( &self, encoder: &mut CommandEncoder, token_src: &Buffer, mode: ForwardMode, )
Record a copy of the final token index / top-k candidates into the readback staging buffer.
pub fn try_record_readback_copy( &self, encoder: &mut CommandEncoder, token_src: &Buffer, mode: ForwardMode, ) -> Result<(), TensorArenaError>
pub fn read_token_blocking(&self, device: &Device) -> Option<u32>
pub fn read_token_blocking(&self, device: &Device) -> Option<u32>
Blocking compatibility wrapper; WebGPU callers must await read_token_async.
Auto Trait Implementations§
impl !Freeze for ForwardArena
impl !RefUnwindSafe for ForwardArena
impl !UnwindSafe for ForwardArena
impl Send for ForwardArena
impl Sync for ForwardArena
impl Unpin for ForwardArena
impl UnsafeUnpin for ForwardArena
Blanket Implementations§
§impl<S, A> Aggregate<Result<S, Error>> for Awhere
A: Aggregate<S>,
impl<S, A> Aggregate<Result<S, Error>> for Awhere
A: Aggregate<S>,
Aggregate shares in an MPC protocol.
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more