Neural Networks as Geometric Chaotic Maps

The use of artificial neural networks as models of chaotic dynamics has been rapidly expanding, but the theoretical understanding of how neural networks learn chaos remains lacking. Here, we employ a geometric perspective to show that neural networks can efficaciously model chaotic dynamics by themselves becoming structurally chaotic. First, we confirm the efficacy of neural networks in emulating chaos by showing that parsimonious neural networks trained only on few data points suffice to reconstruct strange attractors, extrapolate outside training data boundaries, and accurately predict local divergence rates. Second, we show that the trained network's map comprises a series of geometric stretching, rotation, and compression operations. These geometric operations indicate topological mixing and chaos, explaining why neural networks are naturally suitable to emulate chaotic dynamics.