1

I have this very simple homework problem:

Let $X_1,\ldots,X_n$ be a random sample from the $\operatorname{Bernoulli}(p)$ distribution. Find the Fisher information $I(p)$.

I freely admit that my problem is I don't comprehend what a Fisher Information is. (Incidentally, is there a difference between Fisher Information and Fisher Information Matrix?) This is what I have thus far.

Since a Bernoulli Distribution is a Binomial Distribution, the samples are independent and individual. Since the likelihood function is $\mathbb{L} = \prod_{i=1}^n f(X_i\mid\theta)$ and the log likelihood is $\ell = \log(\mathbb{L}(\theta))$ this yields $I(\theta) = \mathbb{E}\{\mathbb{\ell}\,'(\theta)\mathbb{\ell}\,'(\theta)^T\} = -\mathbb{E}(\mathbb{\ell}\,''(\theta))$.

However, I do not know if I'm on the right path. I'd appreciate insight.

Thanks,
Andy

Andrew Falanga
  • 599
  • Information matrix is the generalization of Information function for multiple parameters (or a vector parameter). Here you only need the definition $I(p)=E_p\left[\frac{\partial}{\partial p}\ln f_p(X_1,\ldots,X_n)\right]^2$, which equals $nE_p\left[\frac{\partial}{\partial p}\ln f_p(X_1)\right]^2$ as $X_i$'s are i.i.d. – StubbornAtom Jan 27 '20 at 16:27
  • @StubbornAtom thank you. I have a couple of questions. First, I had thought I was needing the definition. What I have above is a copy from the textbook of, what I thought, was the definition. It is different than yours. How close am I? Second, you have the notation of $[ ]^2$. Is this indicating a matrix raised to a power, or second partial derivative? – Andrew Falanga Jan 27 '20 at 16:51
  • $I(p)$ here is not a matrix; it is simply a function of $p$. With the second derivative, an equivalent formula here is $I(p)=-nE_p\left[\frac{\partial^2}{\partial p^2}\ln f_p(X_1)\right]$. – StubbornAtom Jan 27 '20 at 17:03
  • Your question is answered at https://en.wikipedia.org/wiki/Fisher_information#Single-parameter_Bernoulli_experiment, https://math.stackexchange.com/q/2919044/321264 – StubbornAtom Jan 27 '20 at 17:38
  • @StubbornAtom thank you again. I believe I understand it better now. – Andrew Falanga Jan 28 '20 at 04:27

0 Answers0