Considerations about temporal rescaling, discretization, and linearization of RNNs

We explore the mathematical foundations of Recurrent Neural Networks (RNNs) and three fundamental procedures: temporal rescaling, discretization, and linearization. These techniques provide essential tools for characterizing RNN behaviour, enabling insights into temporal dynamics, practical computational implementation, and linear approximations for analysis. We discuss the flexible order of application of these procedures, emphasizing their significance in modelling and analyzing RNNs for computational neuroscience and machine learning applications. We explicitly describe here under what conditions these procedures can be interchangeable.