MLE of double exponential

Question

I am given the double exponential distribution under the form

$$f(x_i\mid\theta) = \frac{1}{2}e^{-\frac{1}{2}|x_i - \theta|}$$

and I need to find the MLE of $\theta$. I have two approaches until now. The first being

\begin{align} L(\theta \mid \mathbf{x}) &= \prod_{i=1}^n f(x_i\mid\theta) \\ & = \prod_{i=1}^{n} \frac{1}{2}e^{-\frac{1}{2}|x_i - \theta|} \\ & = \left(\frac{1}{2} \right)^ne^{-\frac{1}{2}\sum_{i=1}^n|x_i - \theta|} \\ \Rightarrow \log L(\theta \mid \mathbf{x}) = \ell(\theta \mid \mathbf{x}) & = n\log\frac{1}{2}-\frac{1}{2}\sum_{i=1}^{n}|x_i - \theta| \end{align}

So taking derivative w.r.t. $\theta$ of $\ell(\theta \mid \mathbf{x})$ and setting it to 0, we have

$$0 = \sum_{i=1}^n \operatorname{sgn}|x_i - \theta| $$

This where I am not sure how to proceed, my instinct and previous practice tells me that the answer is the median (because of that absolute value going around), but I am not sure how to justify it. My second approach was this, we had that

\begin{align} L(\theta \mid \mathbf{x}) &= \prod_{i=1}^{n} f(x_i\mid\theta) \\ & = \prod_{i=1}^{n} \frac{1}{2}e^{-\frac{1}{2}|x_i - \theta|} \\ & = \prod_{i=1}^{n} \frac{1}{2}e^{-\frac{1}{2}|x_{(i)} - \theta|} \\ & = \left(\frac{1}{2} \right)^ne^{-\frac{1}{2}\sum_{i=1}^n|x_{(i)} - \theta|} \\ \Rightarrow \log L(\theta \mid \mathbf{x}) = \ell(\theta \mid \mathbf{x}) & = n \log\frac{1}{2} - \frac{1}{2}\sum_{i=1}^n |x_{(i)} - \theta| \end{align}

Now, let $j$ be such that $x_{(j)}<\theta$ and so $x_{(j+1)}>\theta$, then

\begin{align} \ell(\theta \mid \mathbf{x}) & = n\log\frac{1}{2}-\frac{1}{2}\sum_{i=1}^n |x_{(i)} - \theta| = n\log\frac{1}{2}+\frac{1}{2}\sum_{i=1}^j(\theta-x_{(i)} ) - \frac{1}{2}\sum_{i=j+1}^n(x_{(i)} -\theta) \end{align}

and again I am stuck here. Any hints, help, or corrections to what I did will be appreciated thank you.

Answer 1

To see that the median is the point you seek, consider this: \begin{align} & x_{(1)} < x_{(2)} < x_{(3)} < \cdots < x_{(k)} \\[8pt] < {} & \Big(\text{you are here} \ ({}=\theta)\Big) \\[8pt] < {} & x_{(k+1)} < x_{(k+2)} < \cdots < x_{(n)} \end{align} If you're somewhere between some of the order statistics and some of the other order statistics, what happens to the sum of your distances from all of them if you move to the left or the right? (Recall that the sum of your distances from all of them is $\sum_{i=1}^n |x_{(i)} - \theta|$, where $\theta$ is of course where you are now.)

If you move one smidgen to the left, you decrease your distance from each of the order statistics to your left by one smidgen, thus you decrease the sum of those distances by $k$ smidgens. But you also increase your distance from each of the order statistics to your right by one smidgen, thus increasing the sum of those distances by $n-k$ smidgens. Does the sum of all the distance from the order statistics get bigger or smaller? That's just a question of which is bigger: $k$ or $n-k$? You're trying to make the sum of the distances as small as possible, so you move to the left if $k>n-k$ and to the right if $k<n-k$. When you reach the point where $k=n-k$, then moving either left or right keeps the sum of the distances the same, neither increasing nor decreasing. That happens if there are exactly as many order statistics to your right as to your left. If there's an odd number of order statistics, then there's only one point where that happens; if it's an even number, then there's an interval (and in that case you have a non-unique MLE).

Answer 2

Given the sample $\boldsymbol x = (x_1, \ldots, x_n)$, and its order statistics $x_{(1)}, \ldots, x_{(n)}$, the choice of $\theta$ that maximizes $$\ell(\theta \mid \boldsymbol x) = -\frac{1}{2} \sum_{i=1}^n |x_i - \theta|$$ is indeed the median if $n$ is odd; however, if $n$ is even, the MLE may not be unique, since any choice of $\hat \theta \in [x_{(n/2)}, x_{(n/2+1)}]$ will result in the same likelihood; and such a choice is unique only if these two order statistics are equal. Formally, $$\hat \theta \in \begin{cases} \{x_{(m)} \}, & n = 2m-1 \\ [x_{(m)}, x_{(m+1)}], & n = 2m. \end{cases}$$ I leave the proof of this result as an exercise; however, a hint is to consider how $\ell(\theta \mid \boldsymbol x)$ changes as a function of the membership of $\theta$ in each interval $\theta \in [x_{(k)}, x_{(k+1)})$, $k = 1, 2, \ldots, n-1$. You can also attempt to apply an induction argument on $n$. Finally, consider that you can also rewrite $$\ell(\theta \mid \boldsymbol x) = -\frac{1}{2}\sum_{k=1}^n |\theta - x_{(i)}|.$$