0

A closed-form analytical solution for the entropy of the Normal distribution's pdf was derived here. But how about for the t-distribution? How is it derived?

\begin{align}h(X) &=-\int \frac{\mathrm{\Gamma}\left(\frac{\nu+1}{2}\right)}{\sqrt{\nu \pi} \mathrm{\Gamma}\left(\frac{\mathrm{v}}{2}\right)}\left(1+\frac{x^{2}}{\nu}\right)^{-\frac{\nu+1}{2}} \ln \frac{\mathrm{\Gamma}\left(\frac{\nu+1}{2}\right)}{\sqrt{\nu \pi} \mathrm{\Gamma}\left(\frac{\mathrm{v}}{2}\right)}\left(1+\frac{x^{2}}{\nu}\right)^{-\frac{\nu+1}{2}} dx\\ & = ? \end{align}

develarist
  • 1,514
  • 1
    Wikipedia gives $\frac{\nu+1}{2}\left[ \psi \left(\frac{1+\nu}{2} \right) - \psi \left(\frac{\nu}{2} \right) \right] + \ln{\left[\sqrt{\nu}B \left(\frac{\nu}{2},\frac{1}{2} \right)\right]},$ where $ψ$ is the digamma function and $B$ is the beta function – Henry Nov 13 '20 at 16:35
  • @Henry do you know of a source that gives the derivation of that formula? I cannot make sense of the current answer below – develarist Dec 23 '20 at 12:32

1 Answers1

1

When we plug in the t-distribution into the definition of the entropy we have to do an integral of the form

$$H=-C_{\nu}\int_{-\infty}^{\infty}\left(1+\frac{t^2}{\nu}\right)^{-\frac{1+\nu}{2}}\left[\ln C_{\nu}-\frac{\nu+1}{2}\ln\left(1+\frac{t^2}{\nu}\right)\right]=H_1+H_2$$

where the normalization constant is denoted by $C_{\nu}^{-1}=\sqrt{\nu}B\left(\frac{1}{2},\frac{\nu}{2}\right)$. The integrals $H_1$ and $H_2$ denote the integrals generated by the two terms in the brackets respectively. For $H_1$ it suffices to note that it is proportional to the normalization condition of the function so we easily get $$H_1=\ln C_{\nu}^{-1}$$ For the second term we perform a change of variables $x=\frac{t^2}{\nu}$: $$H_2=\frac{\nu+1}{2}\sqrt{\nu}C_{\nu}\int_{0}^{\infty}(1+x)^{-\frac{1+\nu}{2}}\ln(1+x)\frac{dx}{\sqrt{x}}$$ This integral can be written in terms of the Beta function $$\int_{0}^{\infty}(1+x)^{-\frac{1+\nu}{2}}\ln(1+x)\frac{dx}{\sqrt{x}}=-2\frac{d}{d\nu}\int_{0}^{\infty}dx\frac{x^{-1/2}}{(1+x)^{\frac{\nu+1}{2}}}=-2\frac{dB\left(\frac{1}{2},\frac{\nu}{2}\right)}{d\nu} $$ So the second integral is equal to $$H_2=-(\nu+1)\frac{d}{d\nu}\ln B\left(\frac{1}{2},\frac{\nu}{2}\right)=-(\nu+1)\frac{d}{d\nu}\left[\ln\Gamma\left(\frac{\nu}{2}\right)-\ln\Gamma\left(\frac{\nu+1}{2}\right)\right]=\frac{\nu+1}{2}\left[\psi\left(\frac{\nu+1}{2}\right)-\psi\left(\frac{\nu}{2}\right)\right]$$ Combining everything together we get the desired result $$H=\ln \left(\sqrt{\nu}B\left(\frac{1}{2},\frac{\nu}{2}\right)\right)+\frac{\nu+1}{2}\left[\psi\left(\frac{\nu+1}{2}\right)-\psi\left(\frac{\nu}{2}\right)\right]$$

develarist
  • 1,514
DinosaurEgg
  • 10,775