Why Small Return Forecast Errors Move Optimal Weights So Much
Original question
Why are portfolio optimisation results so sensitive to expected return estimates, and how can I make them more robust?
Mean-varianceoptimisation converts differences in forecast returns into trades. When two assets have similar exposures, trading one against the other can add very little estimated variance, so even a small forecast gap can drive a large reallocation. Shrinking return forecasts, imposing economically meaningful allocation and turnover limits, and accounting for forecast uncertainty can all reduce that sensitivity.
The optimiser trades the spread between similar assets
For two assets A and B, let w be the weight in A, 1−w the weight in B, c their covariance, and δ>0 the variance penalty. Maximising expected return minus δ/2 times variance gives the unrestricted interior solution
w=σA2+σB2−2cσB2−c+(μA−μB)/δ.
The denominator is the variance of the return spread RA−RB. The smaller that variance, the larger the weight response to a forecast change. If the weights must be long-only, clip this one-dimensional solution to [0,1]. With more assets, enforce the bounds inside the optimiser.
One way to incorporate uncertainty is to allow μ=μ^+V1/2z for ∥z∥2≤k, where V describes forecast error. The worst expected portfolio return in this set is w⊤μ^−kw⊤Vw by Cauchy–Schwarz. Maximising this worst-case return minus a variance penalty makes allocations pay for their exposure to forecast uncertainty.
Differentiate the optimal weight with respect to the expected-return gap. The slope is the reciprocal of risk aversion times the variance of RA−RB. When the assets move together, that denominator can be small. The optimiser sees a way to express the forecast gap with little incremental variance and scales up the trade. It has no way to tell whether the small gap is genuine signal or estimation noise. Bounds, a turnover penalty or shrinkage of the return forecast each address a different part of that problem.
Take two funds with annual volatility 20%, correlation 0.95 and risk aversion δ=4. Then c=0.038 and the spread variance is 0.004. With equal expected returns the first fund receives 50%. Raise its forecast by just 0.50 percentage points and its weight rises to 81.25%. Shrink the forecast gap by 75% towards equal expected returns and its weight becomes 57.81%. A turnover penalty would further temper the response if the starting portfolio held both funds equally. The appropriate shrinkage or penalty depends on forecast validation, trading costs and how much deviation from the current allocation is acceptable.
A constraint changes how that sensitivity reaches the actual portfolio. In the example the unconstrained solution with the half-point forecast advantage moves the first weight from 50% to 81.25%. If its maximum permitted weight is 60%, the same forecast moves it only to the cap. The return signal still enters the objective, but once the cap binds additional forecast changes no longer increase that position. Compare the objective and realised risk around the cap rather than assuming that the constrained optimum inherits the same sensitivity everywhere.
Measure how the allocation reacts to forecast error
import numpy as np
variance, covariance, risk_aversion = 0.04, 0.038, 4.0forecast_gap = 0.005defweight_in_first_fund(gap): spread_variance = 2 * (variance - covariance)
optimum = (variance - covariance + gap / risk_aversion) / spread_variance
return np.clip(optimum, 0.0, 1.0)
for gap in [0.0, forecast_gap, 0.25 * forecast_gap]:
print(f"{100 * weight_in_first_fund(gap):.2f}%")
Interactive example
See how correlation changes two-asset portfolio risk
Asset A has 15% annual volatility and asset B has 25%. Move the correlation or the long-only weight in A.