3

By the law of total expectation, the conditional expected value of X given Y is given by $$E(X)=E(E(X|Y))$$ Can someone please point me to the theory or identity showing the relation in the case we have the weighted squared shown below $$E(X^TQX)$$ I can easily find the result for the non-conditional case but not for the conditional case.

Thanks

Darren DeBattista
  • 163
  • Not sure I understand the question. You can apply LIE as $E[X'QX]=E[X'E[Q|X]X],$ but is there something else you are specifically looking for? – Golden_Ratio Feb 02 '22 at 11:16
  • I'll try the explain the question further. X is a random vector and Q is a constant diagonal matrix. While there exist the identity showing that E(X'QX)=E(X)'QE(X)+tr(Qcov(X)), I cannot find a similar identity for the case where the random vector X is dependent on the random vector Y. – Darren DeBattista Feb 02 '22 at 11:23
  • Ok, so what exactly do you want? An identity for $E[X'QX|Y]$? – Golden_Ratio Feb 02 '22 at 11:36
  • Yes, though I'm not sure if the expression should be written as $$E[X'QX|Y]$$ or $$E[X'QX]|Y$$ – Darren DeBattista Feb 02 '22 at 11:43
  • The second one doesn't make sense. In any case, the identity should be the same, just conditioning on $Y$ wherever there is an expectation. – Golden_Ratio Feb 02 '22 at 11:44
  • I really don't see what this has to do with LIE though in terms of your original question. – Golden_Ratio Feb 02 '22 at 11:45
  • My case is that I have a random vector X conditioned on random vector Y. For the case of E(X) we can write E(E(X|Y)). If X was a variable instead of a vector, we can write E(X^2) conditioned on Y is E(Var[X|Y]+(E(X|Y))^2). I'm looking for a similar expression when X is a vector and we need to find E(X'QX) conditioned on Y given Q is a diagonal matrix. – Darren DeBattista Feb 02 '22 at 11:56
  • A random vector "conditioned" on another random vector doesn't make sense. A random variable is a function between two sample spaces; what would "$X|Y$" on its own mean? On the other hand, conditional expectation does make sense. And the identity you mention for $X$ scalar should be $E[X^2|Y]=\text{Var}(X|Y)+(E[X|Y])^2$ – Golden_Ratio Feb 02 '22 at 12:15
  • As mentioned, the identity you are looking for is exactly the same, just conditioning on $Y$, i.e. $E[X'QX|Y]=E[X|Y]'QE[X|Y]+tr(Qcov(X|Y))$ – Golden_Ratio Feb 02 '22 at 12:20
  • I got the nomenclature and definitions from the wiki page on Law of total expectations where they define both X and Y as being random. I'm still not sure if you understood what I'm looking for.... (thanks for the fast replys bdw...) – Darren DeBattista Feb 02 '22 at 12:33
  • Let us continue this discussion in chat. – Golden_Ratio Feb 02 '22 at 12:34

1 Answers1

3

Note the identity you mention in the comments,

$$E[X′QX]=E[X]′QE[X]+\text{tr}(Q\text{cov}(X)),$$

where $Q$ is constant and $\text{cov}(X)$ is the variance-covariance matrix of $X$ can be shown as follows:

First recall that the trace is linear and invariant to cyclic product permutations, i.e. $$\text{tr}(aA+bB)=a\text{tr}(A)+b\text{tr}(B),\quad a,b \text{ scalar}\\ \text{tr}(ABC)=\text{tr}(BCA)=\text{tr}(CAB).$$

Since $X′QX$ is a scalar and hence equal to its own trace, we have $$E[X′QX]=E[\text{tr}(X′QX)]=E[\text{tr}(QXX′)] =\text{tr}(QE[XX′])\\ =\text{tr}(Q(E[XX′]-E[X]E[X]'+E[X]E[X]'))\\ =\text{tr}(Q(\text{cov}(X)+E[X]E[X]'))\\ =\text{tr}(Q\text{cov}(X))+\text{tr}(QE[X]E[X]')\\ =\text{tr}(Q\text{cov}(X))+\text{tr}(E[X]'QE[X])\\ =\text{tr}(Q\text{cov}(X))+E[X]'QE[X].$$

Replacing $E[\cdot]$ with $E[\cdot|Y]$ in the above gives the analogous conditional identity

$$E[X′QX|Y]=\text{tr}(Q\text{cov}(X|Y))+E[X|Y]'QE[X|Y] \quad (1).$$

Finally, as discussed in comments, if you wish to write $E[X′QX]$ in terms of conditional expectations, we have by LIE $$E[X′QX]=E[E[X′QX|Y]],$$

where the inner expectation is as given in $(1)$.

Golden_Ratio
  • 12,591