12

One can easily prove that every covariance matrix is positive semi-definite. I come across many claims that the converse is also true; that is,

Every symmetric positive semi-definite matrix is a covariance marix of some multivariate distribution.

Is it true? If it is, how can we prove it?

Federico Magallanez
  • 1,878

3 Answers3

9

The answer is affirmative. Every positive semidefinite matrix $C$ can be orthogonally diagonalised as $QD^2Q^T$, where $Q$ is a real orthogonal matrix and $D$ is a nonnegative diagonal matrix. Let $\mathbf{Z}$ be a random vector following the standard multivariate normal distribution $N(0,I_n)$. It is straightforward to verify that $C$ is the covariance matrix of $\mathbf{X}=QD\mathbf{Z}$.

user1551
  • 139,064
4

The wikipedia article on covariance matrices answers that (the excerpt below is taken verbatim from that article):

From the identity just above, let $\mathbf{b}$ be a $(p \times 1)$ real-valued vector, then: $$\operatorname{var}(\mathbf{b}^{\rm T}\mathbf{X}) = \mathbf{b}^{\rm T} \operatorname{var}(\mathbf{X}) \mathbf{b},$$ which must always be nonnegative since it is the variance of a real-valued random variable and the symmetry of the covariance matrix's definition it follows that only a positive-semidefinite matrix can be a covariance matrix. The answer to the converse question, whether every symmetric positive semi-definite matrix is a covariance matrix, is "yes". To see this, suppose $\mathbf{M}$ is a $p\times p$ positive-semidefinite matrix. From the finite-dimensional case of the spectral theorem, it follows that $\mathbf{M}$ has a nonnegative symmetric square root, that can be denoted by $\mathbf{M}^{1/2}$. Let $\mathbf{X}$ be any $p\times 1$ column vector-valued random variable whose covariance matrix is the $p\times p$ identity matrix. Then: $$\operatorname{var}(\mathbf{M}^{1/2}\mathbf{X}) = \mathbf{M}^{1/2} (\operatorname{var}(\mathbf{X})) \mathbf{M}^{1/2} = \mathbf{M}.$$

user642796
  • 52,188
DCG
  • 51
  • 1
    While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. – Nick Peterson Sep 10 '13 at 17:11
  • I provided the article section. Sorry, its the first question that I answer on this website :-) – DCG Sep 10 '13 at 17:25
  • I made some minor formatting changes, as well as the major formatting change of putting the quoted part in blockquotes. Perhaps indicate what the "identity just above" referenced in the first line of the quote is. – user642796 Sep 10 '13 at 17:49
  • This is a great result. Do any books mention this? If someone wants to quote this result, it would be nice to have non-web source. – user3433489 Feb 23 '21 at 18:58
0

Q: Every Cov is a symmetric PSD. Is every symmetric positive definite matrix a Cov?

A: Yes.

Let $M$ be symmetric and PSD. Since it is symmetric PSD there is cholesky factorization $LL^T$. Given a random vector $Z$ with covariance $I$ and zero expectation we construct $X = LZ$. Then $$ V[X] = E[XX^T] - 0 = E[(LZ) (LZ)^T] = E[LZZ^TL^T] = LE[ZZ^T]L^T = LL^T = M. $$

So $M$ is the covariance matrix of $X$. Thus, every symmetric positive definite matrix is a covariance matrix.

Note that both the symmetric and the PSD property are required for the cholesky decomposition: If $A$ decomposes to $LL^T$ then $$ x A x = x LL^T x = (xL, xL) = \lVert xL \rVert^2 \geq 0 $$ so $A$ must be PSD. Further $A^T = (LL^T)^T = LL^T = A$ so $A$ must be symmetric.

This answer assumes real matrices.
Hunaphu
  • 371