0

For $P(x)=N(\mu,\sigma^2)$ and $Q(x)=N(0,1)$ I am supposed to calculate $KL(P(x)||Q(x))$, here is what I did

\begin{align*} KL(P(x)||Q(x)) & = \int P(x) \cdot \log\left(\frac{P(x)}{Q(x)}\right) dx \\ & = \int P(x) \cdot \log\left(\frac{\frac{1}{\sqrt{2\pi \sigma ^2}} \exp{-\frac{(x-\mu)^2}{2*\sigma ^2}}}{\frac{1}{\sqrt{2\pi}} \exp{-\frac{x^2}{2}}}\right) dx\\ & = \int P(x) \cdot \log\left(\frac{1}{\sigma^2}\right)^{\frac{1}{2}} dx + \int P(x) \left(-\frac{(x-\mu)^2}{2\sigma^2} + \frac{x^2}{2} \right) dx \\ & = \frac{1}{2} \log\left(\frac{1}{\sigma^2}\right) \int P(X) dx + \frac{1}{2\sigma^2} \left(-\int (x-\mu) ^2 P(x) dx \right) + \frac{1}{2} \left( \int x^2 P(x) dx \right) \end{align*}

now I use that $$\int x^2 P(x) dx = V(x)-E(x)^2 = \sigma^2 - \mu^2$$ $$\int P(X) dx = 1$$
$$-\int (x-\mu) ^2 P(x) dx = V(x) = -\sigma^2$$

so I get

$$KL(P(x)||Q(x)) = -\frac{1}{2}\log(\sigma^2) - \frac{1}{2} + \frac{\sigma^2}{2} - \frac{\mu^2}{2} = -\frac{1}{2}\left(log(\sigma^2)+1-\sigma^2 + \mu^2\right)$$

But I am supposed to get $$KL(P(x)||Q(x)) = -\frac{1}{2}\left(log(\sigma^2)+1-\sigma^2 - \mu^2\right) $$

I cannot see where the mistake with the $+\mu^2$ comes from? Is it not true that $\int x^2 P(x) dx = E(x^2)$ ??

LRS25
  • 11
  • 2

0 Answers0