Module feed_forward 

Feed-forward network (SwiGLU) — the STEM definition of the transformer FFN block as the composition it is:

FFN(x) = W_down · ( SiLU(W_gate · x) ⊙ (W_up · x) )

That is three matrix–vector products (super::linear_algebra::gemm::matvec), one activation (super::activation::silu), and one Hadamard product (super::linear_algebra::vector::hadamard_assign). Nothing proprietary — the LLM “FFN block” is exactly this. The runtime’s dispatch_ffn_block_pre_norm is a backend computing this same function over quantized weights on the GPU.

Caller-owned, zero internal allocation (gate/up scratch buffers are supplied).

Functions

swiglu_ffn

Compute out = W_down · ( SiLU(W_gate · x) ⊙ (W_up · x) ), row-major, caller-owned.