RNNs are what everyone used for time series data until the transformer and attention models were invented. They are useful to learn, but generally most modern deep learning models use some form of attention not the RNN architecture.

Recurrent Neural Networks (RNNs)

A Recurrent Neural Network (RNN) is a type of neural network that excels in handling sequential data. Unlike standard neural networks, RNNs have connections that loop back within the network, helping with tasks that require memory of prior inputs in the sequence. This makes RNNs useful for tasks like handwriting recognition and speech recognition. They are characterized by the direction of the flow of information between their layers, which can be bi-directional, allowing the output from some nodes to affect subsequent input to the same nodes.

Long Short-Term Memory (LSTM)

LSTMs are a special kind of RNN capable of learning long-term dependencies. They were introduced to combat the vanishing gradient problem of basic RNNs, which makes them ineffective at learning from earlier time steps in very long sequences. LSTMs have additional states called "cell states" that can maintain information in memory for long periods of time, making them more effective for tasks requiring memory of past data in the sequence.

Gated Recurrent Units (GRUs)

GRUs are another variant of RNNs that are computationally more efficient than LSTMs. They have fewer parameters and perform similarly on certain tasks. GRUs simplify the architecture of LSTMs by using a gating mechanism but without an output gate, making them quicker to compute.