Markov Decision Processes and Value/Policy Iteration

Every supervised algorithm in this unit assumed a dataset arrives already labelled. Reinforcement learning instead poses a sequential decision problem: an agent picks actions, the world responds with a new state and a reward, and the agent must learn — from that trial-and-error interaction alone — how to act so as to maximise its long-run cumulative reward. The Markov decision process (MDP) is the formal object this problem is posed in, and the Bellman equation is the single recursive identity that makes it tractable: it turns an infinite-horizon optimisation over entire action sequences into a fixed-point equation over one state at a time.