Gated Memory: LSTMs and GRUs

A basic RNN compresses the entire past into one hidden vector updated at every step — and gradients over long unrolls vanish before they reach distant inputs. For credit-event sequences, order-book updates, or text, that bottleneck makes long-range dependencies unreliable. Long short-term memory (LSTM) and gated recurrent unit (GRU) cells add learned gates that decide what to remember, overwrite, and expose, giving the network a controllable memory path instead of an all-or-nothing state update.