Hermite Structure And Spectral Properties Of Symmetric Gated Activation Functions

Abstract

We study a family of self-gated functions where the gate satisfies a reflection symmetry, and this forces a rigid Hermite structure. We then specialize the abstract theory to the Swish and GELU families by proving strict monotonicity of the spectral ratio, and deriving both the small- and large-parameter asymptotic expansions. For GELU specifically, we also derive the exact closed form for the spectral ratio. We then enrich the symmetric family by adding odd perturbations and show that the leading Hermite mode becomes continuously controllable.