Deriving a Conditional Gaussian Distribution

For jointly Gaussian

X

and

Y

with

\operatorname{Var}(Y)>0

, subtract the linear projection of

X

onto

Y

. The remaining Gaussian residual is independent of

Y

. Given

Y=y

, add the fitted conditional mean back to that residual. The resulting normal law has variance

\operatorname{Var}(X)(1-\rho^2)

when both marginal variances are positive.

Subtract the best linear prediction and identify the residual law

Let $\mu_X=E[X]$ , $\mu_Y=E[Y]$ , $\sigma_{XY}=\operatorname{Cov}(X,Y)$ , and $\sigma_Y^2=\operatorname{Var}(Y)$ . Define

a=\frac{\sigma_{XY}}{\sigma_Y^2}, \qquad R=X-\mu_X-a(Y-\mu_Y).

Then

\operatorname{Cov}(R,Y)=\sigma_{XY}-a\sigma_Y^2=0

. The pair

(R,Y)

is jointly Gaussian because it is an affine transformation of

(X,Y)

. The zero covariance implies independence. Expanding the variance of

R

gives

\sigma_X^2-\sigma_{XY}^2/\sigma_Y^2

, so the conditional law is

X\mid Y=y\sim N\left( \mu_X+a(y-\mu_Y),\ \sigma_X^2-\frac{\sigma_{XY}^2}{\sigma_Y^2} \right).

If the residual variance is zero, the law is a point mass at the conditional mean. The formula describes a regular conditional distribution. This is how conditioning on a particular real value is understood even though a continuous

Y

assigns probability zero to the event

Y=y

The choice of

a

makes the residual uncorrelated with

Y

. Because

(R,Y)

is jointly Gaussian, that orthogonality becomes independence. After observing

Y=y

, the mean part therefore becomes the number

\mu_X+a(y-\mu_Y)

while the residual has its original centred Gaussian distribution. This also explains why the conditional variance in the answer does not depend on

y

. The observed value changes the location. The correlation and the marginal variances determine how much uncertainty remains.

Using the correlation gives another way to read the calculation. Since $\sigma_{XY}=\rho\sigma_X\sigma_Y$ , the conditional mean adjustment is $\rho\sigma_X$ times the number of standard deviations by which $y$ exceeds $\mu_Y$ . In the example, observing 13 is one standard deviation above the mean of 10, so it adds $0.6(2)=1.2$ to the mean of $X$ . For the conditional standard deviation, take the square root of $\sigma_X^2(1-\rho^2)$ . These two quantities are what you need before standardising the threshold in the tail probability.

Turn one observed signal into a conditional probability

A model gives $\mu_X=1$ , $\sigma_X=2$ , $\mu_Y=10$ , $\sigma_Y=3$ and $\rho=0.6$ . What is the probability $X$ exceeds 2 when $Y$ is observed at 13?

The covariance is

0.6(2)(3)=3.6

, so

a=3.6/9=0.4

. The conditional mean is

1+0.4(13-10)=2.2

. The conditional variance is

4(1-0.6^2)=2.56

, with standard deviation 1.6. Therefore

P(X>2\mid Y=13)=1-\Phi\left(\frac{2-2.2}{1.6}\right)\approx0.550.

The unconditional probability is about 0.309. Observing a value of $Y$ one standard deviation above its mean shifts the conditional distribution enough to make the event substantially more likely in this example.

The same projection argument extends to several observed variables. Replace the scalar slope by a row vector of covariances multiplied by the inverse of the observations' covariance matrix. In a numerical implementation, solve the corresponding linear system to obtain the slope.

Compute the conditional mean and tail probability

from scipy.stats import norm

conditional_mean = 1 + (0.6 * 2 / 3) * (13 - 10)
conditional_sd = 2 * (1 - 0.6**2)**0.5
print(norm.sf(2, loc=conditional_mean, scale=conditional_sd))

Subtract the best linear prediction and identify the residual law

Turn one observed signal into a conditional probability

Compute the conditional mean and tail probability

References and next steps