4

Are there any grounds for the following claim?

For distributions for which the support in $x$ depends on $\theta$, the CRLB $=0$.

For a quick example (Casella-Berger, Example 7.3.13): Let $X_1,\dots,X_n$ be iid with pdf $f(x\mid\theta) = 1/\theta, 0 < x <\theta$. There, CB used the typical formula for Fisher information $I(\theta)$ to get CRLB $= \theta^2/n$ and subsequently showed that for the unbiased estimator $\hat{\theta} = \frac{n+1}{n}X_{(n)}$, $\mathrm{Var}\hat{\theta} = \frac{1}{n(n+2)}\theta^2$ which is uniformly smaller than the CRLB and thus Cramer-Rao Inequality is violated.

However, my lecturer disagrees with this approach as CB ignored the indicator function $\mathbf{1}_{(0<x<\theta)}$ when taking the derivative of the log-likelihood function. His approach was to note that because the likelihood function is not continuous in $\theta$, it is not differentiable in $\theta$. In situations like these, we should define $I(\theta) = +\infty$ and thus CRLB $= 0$, in which case the Cramer-Rao Inequality was not violated.

I Google'd a bit but did not find any references to validate this alternative approach although it does make some sense (although I can't seem to grasp the significance of having CRLB $= 0$). Has anyone come across something like this before?

NKC
  • 43

2 Answers2

5

The problem is not with the derivability of the indicator function $\mathbb{I}_{0\le x\le\theta}$ in $f_\theta(x)=\theta^{-1}\mathbb{I}_{0\le x\le\theta}$ at $\theta=x$ since the expectation $$\mathbb{E}^X_\theta\left[\left(\frac{\partial}{\partial\theta}\log\{f_\theta(X)\}\right)^2\right]$$ only requires the integrand to be defined almost everywhere, which is the case if only $X=\theta$ is excluded. The problem is that this expression is no longer the variance of the score function since the expectation of the score is no longer zero, that it no longer defines a lower bound on the variance, that it is no longer connected with the limiting distribution of the MLE, that it is no longer additive in the observations, &tc.

Xi'an ні війні
  • 766
4

The Cramer-Rao lower bound can never be violated if all of the following conditions are verified:

$(i) \quad\Theta$ is an open interval in $\mathbb{R}$

$(ii) \quad\{(x_1,...,x_n)\in \mathbb{X}| \ f_\theta(x_1,...,x_n)>0\} \ \ does \ not \ depend \ on \ \theta$

$(iii) \quad \frac{\partial}{\partial\theta}f_\theta(x_1,...,x_n) \ exists \ for \ each \ \ (x_1,...,x_n) \in \mathbb{X} \ \ and \ \ \theta \in \Theta$

$(iv) \quad \int_{\mathbb{X}} \frac{\partial}{\partial\theta}f_\theta(x_1,...x_n)dx_1...dx_n=0$

Of course, $(ii)$ will not be verified in those distribution families in which the support depends on $\theta$; like the uniform distribution in $(0,\theta)$ you just mentioned above.

Note that in practice the other conditions are straightforward and require no verification, as we deal with smooth functions: note for example that $(iv)$ is equivalent to say that the identity:

$$\int_{\mathbb{X}}f_\theta(x_1,...,x_n)dx_1...dx_n=1$$

can be differentiated under the integral operator

Akerbeltz
  • 2,638