Why a covariance matrix is positive semi-definite

Show that the covariance matrix of a random vector is positive semi-definite.

Because $w^\top\Sigma w=\operatorname{Var}(w^\top X)\ge0$ for every vector $w$. Expanding the variance of the linear combination $\sum_iw_iX_i$ gives $\sum_{i,j}w_iw_j\operatorname{Cov}(X_i,X_j)$. It is strictly positive definite unless some non-trivial combination of the components is almost surely constant.