Backpropagation supplies gradients, but a deep network still fails when those gradients vanish, explode, or overfit. This topic collects the practical interventions that stabilise training and make the feedforward machinery usable in real neural-network models.
By the end of this note you should be able to:
- diagnose vanishing and exploding gradients;
- choose Glorot or He initialisation, batch normalisation, dropout, and Adam for their distinct roles;
- distinguish training-time from inference-time behaviour for batch normalisation and dropout.
In this lesson
- Contents
- The big picture
The full lesson is for members.
Read the full lessonNew to SelfMadeQuant.com? See membership options or explore the curriculum.