Skip to content

Bayesian interpretation of ridge

Question

With Gaussian observation errors and an isotropic zero-mean Gaussian coefficient prior, explain the connection between MAP estimation and ridge.

Short answer

The negative log likelihood is proportional to squared residual error divided by the error variance. The negative log prior is proportional to the squared coefficient norm divided by the prior variance. Their sum is a ridge objective, with penalty strength equal to error variance divided by prior variance when the loss is an unnormalised residual sum of squares. Different averaging conventions change this numerical relationship.

Original question and answer written by SelfMadeQuant.com.

See the hint and the full worked solution in the question bank (free account).

Open the worked solution

Lessons behind this question

Start with a free lesson: Normal Distribution is part of a free topic. Create a free account, no card needed, to read it in full with worked solutions.

Read it free