Least-squares regression via QR factorisation

Question

Explain how a QR decomposition is used to compute the least-squares coefficients $\beta$ minimising $\|y-X\beta\|^2$ .

Short answer

Factor $X=QR$ and solve $R\beta=Q^\top y$ . With $Q$ having orthonormal columns and $R$ upper triangular, the normal equations $X^\top X\beta=X^\top y$ reduce to $R^\top R\beta=R^\top Q^\top y$ , i.e. $R\beta=Q^\top y$ , solved by back-substitution. This avoids forming $X^\top X$ , whose condition number is the square of $X$ 's, so it is numerically more stable.

Adapted from Zhou, A Practical Guide to Quantitative Finance Interviews, sec. 3.6 Linear Algebra, 'QR decomposition and least squares', p.51; question and answer rewritten by SelfMadeQuant.com.

See the hint and the full worked solution in the question bank (free account).

Open the worked solution

Lessons behind this question

Start with a free lesson: Basic Statistics and Regression is part of a free topic. Create a free account, no card needed, to read it in full with worked solutions.

Read it free