The -test of the previous lesson answers "is significantly different from zero?" one coefficient at a time. This note repackages the same question in a form that survives the jump to many predictors: split the total variation in into a piece the regression explains and a piece it leaves over, and compare their sizes with an -statistic. For simple regression this is provably the identical test as the -test — a useful consistency check — but the sum-of-squares decomposition it is built from, , is exactly Theorem 2's projection picture read as a Pythagorean theorem, and it is this form of the argument — not the -test — that generalises cleanly to testing several predictors at once in Multiple Linear Regression. Along the way this note builds , the single most quoted (and most misquoted) number in applied regression, and its adjusted cousin.
Learning objectives. By the end of this note you should be able to:
- derive the sum-of-squares decomposition from the orthogonal-projection geometry of Lesson 1;
- construct the ANOVA -statistic for , state its exact null distribution, and prove it equals the square of the -statistic in simple regression;
- read and build a one-way regression ANOVA table;
- define , prove it equals the squared sample correlation in simple regression, and explain why it can never fall when a predictor is added;
- state what adjusted corrects for and why it is preferred for comparing models with different numbers of predictors.
In this lesson
- The sum-of-squares decomposition
- The -test
- : the coefficient of determination
Before you start
- Simple Linear Regression: OLS Estimators and Geometry
- Inference in Simple Linear Regression
- The Chi-Squared, t, and F Distributions
The full lesson, with worked examples and exercise solutions, is for members.
Read the full lessonNew to SelfMadeQuant.com? See membership options or browse the Probability, Statistics and Financial Data syllabus.