Skip to content
Probability, Statistics and Financial Data / Bayesian Estimation and Posterior Inference

Prior, Likelihood, and Posterior

Every estimator built in Point Estimation treats θ\theta as a fixed, unknown constant and asks the data to pin it down as tightly as possible. A portfolio manager updating a return forecast rarely starts from nothing, though — she has a house view, a macro thesis, or last quarter's calibration, and wants that view revised by the new data, not discarded. Bayesian inference formalises exactly this: treat θ\theta itself as a random variable with a prior distribution π(θ)\pi(\theta) encoding what is believed before the data arrive, combine it with the likelihood L(θ;x)L(\theta;x) the data supply, and read off a posterior distribution π(θ∣x)\pi(\theta\mid x) that is the mathematically correct fusion of the two. This is not a new likelihood — every quantity from Topic 3 (the score, the MLE, the Fisher information) survives unchanged inside a Bayesian calculation — it is a new use of the likelihood, governed by ordinary conditional-probability mechanics already met in Conditional Probability, Total Probability, and Bayes' Rule and its continuous extension, The Continuous Bayes' Rule.

This note states Bayes' theorem for a parameter, works the Poisson-Gamma updating example in full , distinguishes proper from improper priors, reads the prior as a regulariser on the likelihood, and closes with a Normal-Normal worked example that is exactly the calculation behind updating a return forecast with a prior view.

Learning objectives. By the end of this note you should be able to:
  • state Bayes' theorem for a parameter and derive it from the joint density of (θ,X)(\theta, X);
  • distinguish the prior, the likelihood, the marginal (evidence), and the posterior, and identify each in a worked calculation;
  • classify a prior as proper or improper, and check whether an improper prior still yields a proper posterior;
  • explain, at a formula level, why the log-posterior mode is a penalised log-likelihood, and name the regulariser two standard priors induce;
  • carry a Normal prior through a Normal likelihood update by hand, and read the resulting posterior mean as a precision-weighted average.

In this lesson

  1. Bayes' theorem for a parameter
  2. Worked example: updating a Poisson rate with a Gamma prior
  3. Proper and improper priors
  4. The prior as a regulariser
  5. Finance reading: updating a return forecast with a prior view

Before you start

The full lesson, with worked examples and exercise solutions, is for members.

Read the full lesson

New to SelfMadeQuant.com? See membership options or browse the Probability, Statistics and Financial Data syllabus.