Skip to content

How Reinforcement Learning Frames Portfolio Rebalancing

Original question

How can reinforcement learning be applied to portfolio allocation, and what are the main limitations in real-world trading?

Build a portfolio environment in which an agent observes market features and current holdings, chooses target weights, and receives a reward based on net portfolio growth or a stated utility. The environment must enforce trading constraints, apply costs and update positions after returns. Validate the resulting policy chronologically against simple allocations and under changed execution assumptions.

Specify the state, action and reward as a trading problem

Let w~t\widetilde w_t be current weights after price drift and wtw_t the action's target weights. For a long-only portfolio with cash, use nonnegative weights that sum to one. Compute an illustrative turnover charge ct=κ∑i∣wt,i−w~t,i∣c_t=\kappa\sum_i |w_{t,i}-\widetilde w_{t,i}| over risky assets. If execution costs take a fraction ctc_t of equity and weights apply to the remaining capital, a growth-based reward is
Rt+1=log⁡ ⁣[(1−ct)(1+wt⊤rt+1)].R_{t+1}=\log\!\left[(1-c_t)(1+w_t^\top r_{t+1})\right].
This assumes positive wealth and a fill before the measured return. Update the weights using each asset's realised gross return. For an actual simulator, include settlement, cash and leverage limits, partial fills, borrow and costs appropriate to the instruments. Train on past periods, tune on subsequent validation periods, and assess the fixed policy on a final test. Compare with equal weight, buy-and-hold and a constrained rebalancing baseline. Vary costs and test a simpler strategy with the same information.

An agent starts at 60% stock and 40% cash. It targets 80% stock and 20% cash. It trades 20% of equity in the stock, so a 10-basis-point fee per dollar of risky-asset turnover costs 2 basis points of equity. If the stock then gains 1% and cash earns zero, portfolio value is multiplied by 0.9998×1.008=1.00779840.9998\times1.008=1.0077984. The new stock weight drifts to 0.8×1.01/1.008≈80.16%0.8\times1.01/1.008\approx80.16\%. The next decision must use this drifted weight.

If it instead stays at 60% stock, the next portfolio value is 1.006 times starting equity with no trading fee. One favourable episode does not settle whether the rebalance is sensible. A policy needs to make that decision repeatedly under the distribution of outcomes and costs it will face.

Include the existing holdings in the state because they affect the cost of the next action. Two agents can observe the same prices and forecasts yet face different trades to reach the same target weights. Apply prices and costs in the order they would occur after the decision, then carry post-move weights into the next state. If every episode starts from cash, check that the reset does not conveniently erase risky positions or avoid a liquidation cost. A simple periodic rebalance provides a baseline under the same execution model.

References and next steps