Deep RL and Policy Gradients at the Survey Level

Fitted value iteration approximated the value function with a hand-chosen feature map $\phi(s)$ — workable for a low-dimensional, well-understood state, but not for raw pixels or high-dimensional state vectors where no one knows the "right" features in advance. Deep reinforcement learning replaces $\phi(s)$ with a neural network that learns its own features, exactly as the Neural Networks and AI unit did for supervised learning. This closing note surveys two families at the pointer level, matching the unit's length-flex note that RL is optional depth for a quant-focused reading: deep Q-learning (DQN), which scales the value-based methods of this topic to neural-network function approximators, and policy gradients (REINFORCE), a genuinely different, model-free family that optimises a policy directly without ever estimating a value function or a transition model.