Function mix_streams_into
pub fn mix_streams_into(
normalized: &[f32],
mix_gate: &[f32],
stream_count: usize,
hidden: usize,
out: &mut [f32],
) -> Result<(), QwenNumericError>Expand description
Form the token-mixer or MoE input from the gated average of Hyper streams.
mix_gate is the sigmoid output of the streamed HC up-projection.