PCA truncation before regression

Why can discarding low-variance principal components improve numerical stability but lose predictive information?

Small singular values create unstable least-squares coefficients, so truncation can reduce estimation variance. PCA ranks directions by feature variance, not by their relationship with the target. A low-variance direction can therefore contain useful predictive information. Fit PCA inside each training fold and choose retained components by time-respecting validation; compare against ridge, which shrinks directions continuously rather than discarding them abruptly.