Hint for MLE of $a$ for a random sample from a uniform distribution in $[-a, a]$

Question

Suppose $X_1, \dots, X_n$ are iid from a uniform distribution in $[-a, a]$. Their PDF is thus given by $$f(x) = \dfrac{1}{2a}\mathbb{I}(x \in [-a ,a])\text{.}$$ $\mathbb{I}$, above, denotes the indicator function. I wish to find a MLE for $a$.

The likelihood function is given by $$L(a \mid x_1, \dots, x_n) = \left(\dfrac{1}{2a}\right)^n\prod_{i=1}^{n}\mathbb{I}(x_i \in [-a, a])\text{.}$$ I am quite certain that one cannot do the traditional method of finding the loglikelihood, setting its derivative equal to $0$, and solving for $a$. I know there's some trick to doing this - perhaps involving an order statistic - but it has been a long time since I've seen the trick. Can I have a hint, not a complete solution on how to proceed with this problem?

Edit: Intuitively speaking, I think that if I wanted to choose $a$ to maximize $L$, I wouldn't choose $a$ to be a negative value. Actually, I would choose $a$ to be the smallest value of $x_i$ which is greater than $0$ - since it will be placed in the denominator of $L$ and as $x_i$ is larger, $L$ gets smaller.

Baily · Answer 1 · 2016-05-18T13:34:51.770

If there exists a $x_i\notin [-a,a]$, we have $L=0$. If all $x_i\in [-a,a]$, we have $L\left( {\left. a \right|{x_1},...,{x_n}} \right)=(1/2a)^n$, or $L\left( {\left. a \right|{x_1},...,{x_n}} \right)=(1/2a)^n$ when $0 \leqslant \mathop {\max }\limits_i \left| {{x_i}} \right| \leqslant a$. Then, $$\log L\left( {\left. a \right|{x_1},...,{x_n}} \right) = \left\{ \begin{gathered} n\log \left( {\frac{1} {{2a}}} \right),\,\,\,0 \leqslant \mathop {\max }\limits_i \left| {{x_i}} \right| \leqslant a, \hfill \\ 0,\,\,\,\,\,\,\,otherwi\operatorname{se} \\ \end{gathered} \right.$$ This gives $$\frac{{\partial \log L}}{{\partial a}} = \left\{ \begin{gathered} - n/a,\,\,\,0 \leqslant \mathop {\max }\limits_i \left| {{x_i}} \right| \leqslant a, \\ 0,\,\,\,\,\,\,\,otherwi\operatorname{se} \\ \end{gathered} \right.$$ Clearly, $\frac{{\partial \log L}}{{\partial a}} < 0$ for all $a \geqslant \mathop {\max }\limits_i \left| {{x_i}} \right|$. So, $\log L$ is a decreasing function on $[\mathop {\max }\limits_i \left| {{x_i}} \right|,\infty)$. Thus, $\log L$ attains maximum value on $[\mathop {\max }\limits_i \left| {{x_i}} \right|,\infty)$ at $a =\mathop {\max }\limits_i \left| {{x_i}} \right|$.

I think you should actually have $-n/a$ as the derivative of the log likelihood in the regime $a \geq \max |x_i|$. — Ian, May 18 '16 at 13:33

score 2 · Accepted Answer · answered May 18 '16 at 13:04

2

You still are maximizing the likelihood. In this case you can do it by inspection. The likelihood is zero if $|x_i|>a$ for any $i$. On the other hand, the product is just $1$ if $|x_i| \leq a$ for all $i$, so once that is the case, the likelihood function is $(2a)^{-n}$, which is a decreasing positive function of $a$. So the maximum occurs when...?

answered May 18 '16 at 13:04

Ian

101,645

See my edit. Is my hunch correct? – Clarinetist May 18 '16 at 13:05
@Clarinetist Not quite, what happens if your data is ${ -10,1 }$? – Ian May 18 '16 at 13:06
Hmm, still not quite seeing it. Would it be $\min_{i}|x_i|$? If I were to use the idea I had above, $L(\hat{a} = 1 \mid -10, 1)$ would be $\dfrac{1}{2^2} = \dfrac{1}{4}$. I'm not sure if I can maximize it any more than this. – Clarinetist May 18 '16 at 13:13
I would think the maximum would occur around $0$. – Clarinetist May 18 '16 at 13:14
@Clarinetist No, $L(1 | -10,1)$ is $0$ because $-10$ is not in $[-1,1]$. Then all of a sudden $L(10 | -10,1)$ is $\frac{1}{(2 \cdot 10)^2}$... – Ian May 18 '16 at 13:15
AH, I see... so we have to choose $\min_i |x_i|$ such that $|x_i| \leq a$? – Clarinetist May 18 '16 at 13:16
@Clarinetist I think I know what you meant: you want the smallest possible $a$ such that $|x_i| \leq a$ for all $i$. – Ian May 18 '16 at 13:17
So, essentially, $\hat{a} = \sup{|x_i|}$? – Clarinetist May 18 '16 at 13:19
@Clarinetist Yes. – Ian May 18 '16 at 13:20

score 0 · Answer 3 · answered May 18 '16 at 13:33

An attempt at a full solution:

We have the likelihood function $$L(a \mid x_1, \dots, x_n) = \left(\dfrac{1}{2a}\right)^n\prod_{i=1}^{n}\mathbb{I}(x_i \in [-a, a])\text{.}$$ This is $0$ if at least one of the $x_1, \dots, x_n \notin [-a, a]$, which will obviously minimize $L$, rather than maximize $L$. Suppose, then, that $x_1, \dots, x_n \in [-a, a]$. Then the product of indicators is $1$, and we are left with $$L(a \mid x_1, \dots, x_n) = \left(\dfrac{1}{2a}\right)^n\text{.}$$ We need to actually use the data $x_1, \dots, x_n$ in calculating the MLE of $a$. Notice that $x_1, \dots, x_n \in [-a, a]$. We probably shouldn't choose $a$ to be negative - as this definitely wouldn't maximize $L$. So instead, notice that $|x_1|, \dots, |x_n| \in [0, a]$.

Obviously, $L$ is decreasing with respect to $a$. Using the intuition I developed before, one might (mistakenly) think to take $\hat{a} = \min_{i}|x_i|$. But the problem with this is that if $\hat{a} = \min_i|x_i|$, assuming we have at least two distinct $|x_i|$ values, they will lie outside of $[-\hat{a}, \hat{a}]$ and will thus give an indicator of $0$ (and therefore $L = 0$). The indicator is particularly important, as we had mentioned earlier that all of the $x_i$ values must be in $[-a, a]$. So, to avoid $L = 0$, we choose $\hat{a} = \max_i|x_i|$. Unfortunately, we can't choose $\hat{a}$ to be any smaller, or $L$ will be $0$.

Hint for MLE of $a$ for a random sample from a uniform distribution in $[-a, a]$

3 Answers3

Linked