Skip to main content

gelu

Function gelu 

Source
pub fn gelu(x: &mut [f64])
Expand description

GELU (Gaussian Error Linear Unit), tanh approximation: 0.5·x·(1 + tanh(√(2/π)·(x + 0.044715·x³))). The standard GPT-2/transformer GELU.