Every earlier topic in this unit answered a version of "what is my best guess for ?" — Topic 3 built the estimator, Topic 5 wrapped it in a range. This topic asks a sharper, binary question instead: given a specific claim about — a strategy's Sharpe ratio is zero, a model's default rate matches a benchmark, a coefficient is exactly zero — is the data surprising enough to reject that claim outright? And, crucially, once a decision rule for rejecting is on the table, is it the best possible rule, or merely a plausible one?
That second question is what makes this topic the deepest theoretical result in the unit. The Neyman-Pearson lemma proves, with essentially no assumptions on the underlying model, that for a simple hypothesis against a simple alternative, the test built from the likelihood ratio is not merely reasonable — it is provably unbeatable: no other test at the same significance level can ever have higher power. The Karlin-Rubin theorem then extends this exact optimality from the narrow simple-versus-simple case to the practically relevant case of one-sided composite hypotheses, using the structural property of a monotone likelihood ratio that nearly every exponential family — and hence nearly every named distribution used in quantitative finance — satisfies automatically. The topic closes by confronting where this exact machinery runs out: two-sided hypotheses admit no uniformly most powerful test, motivating the -value as the practical, continuous-valued substitute for a binary reject/accept decision, and multiple simultaneous tests demand their own error-rate discipline once a research pipeline screens more than a handful of hypotheses at once.
By the end of this topic you should be able to:
- state the hypothesis-testing framework precisely — null and alternative hypotheses, Type I and Type II error, the power function, and the rejection region — and compute error probabilities for a stated test;
- state and prove the Neyman-Pearson lemma, and use it to construct the most powerful test for a simple-versus-simple hypothesis;
- verify the monotone likelihood ratio property for an exponential family and apply the Karlin-Rubin theorem to construct a uniformly most powerful test for a one-sided hypothesis;
- explain precisely why no uniformly most powerful test exists for a two-sided alternative, and construct the standard two-sided compromise test;
- define the -value, prove it is -distributed under a simple null, and name the specific ways it gets misused in a research pipeline (the significance trap);
- apply the Bonferroni, Holm-Bonferroni, and Benjamini-Hochberg procedures to control, respectively, the family-wise error rate (flat and step-down) and the false discovery rate across many simultaneous tests.
In this lesson
- Contents
- The big picture
Before you start
- Parametric Statistical Models and the Likelihood
- Maximum Likelihood Estimation: Score and Log-Likelihood
- Pivotal Quantities and Exact Confidence Intervals
The full lesson is for members.
Read the full lessonNew to SelfMadeQuant.com? See membership options or browse the Probability, Statistics and Financial Data syllabus.