Minimize $\sum_{i=1}^n|x-a_i|$, $\max \{|x-a_i|, i=1,\cdots,n\}$, $\sum_{i=1}^n|x-a_i|^2$ and maximize $\Pi_{i=1}^n|x-a_i|$

Question

$a_1\le a_2\le \cdots \le a_n$ real numbers.

a) Minimize $\sum_{i=1}^n|x-a_i|$

b) Minimize $\max \{|x-a_i|, i=1,\cdots,n\}$

c) Minimize $\sum_{i=1}^n|x-a_i|^2$

d) Maximize $\Pi_{i=1}^n|x-a_i|$

a), if we take the derivative, we get

$$f(x) = \sum_{i=1}^n|x-a_i|\implies \frac{df}{dx} = \mbox{signal(x)}n$$

So the derivative is never $0$, but I can't say that a minimum for this funciton has to have derivative $0$ because this function is not continuously differentiable. However we can see that for $x>0$ the function is increasing and $x<0$ it is decreasing. I understand that the median is the point, but I need to prove analytically that $f$ indeed is smaller than $f$ in every other point. I've seen The Median Minimizes the Sum of Absolute Deviations (The $ {L}_{1} $ Norm) but I found no analytical proofs for the median being the minimizer, only explanations about how to find with some intuition that the median is smaller

b) I don't have an idea about this one

c) $$f(x) = |x-a_1|^2 + |x-a_2|^2 + \cdots + |a-x_n|^2$$

$$\frac{df}{dx} = 2|x-a_1| + \cdots + 2|x-a_n|$$

The derivative is always positive or 0, so the function is always increasing. This one is continuous so by making it equal to $0$ I should try to find points that satisfy the equality. I don't think there exists one, so I would say the derivative is always positive. How should I proceed?

d) The exercise gives no information about a bounary for $x$, so if I pick larger and larger $x$ shouldn't the funciton grow infinitely?

UPDATE:

b)

Let $x^* = \frac{a_1+a_n}{2}$. If $a_1<x<x^*$ then $\max\{|x-a_i|, i=1,\cdots,n\} = a_n-x$ which is decreasing. If $x>x^*$ then $\max\{|x-a_i|, i=1,\cdots,n\} = x-a_1$ which is increasing. Therefore the minimum must be at $x^*$. I think I need to argue better why, for example, $\max\{|x-a_i|, i=1,\cdots,n\} = a_n-x$

I need a more rigorous proof.

What exactly are you asking? Are you looking for closed-form solutions for these four problems? — David M., Aug 18 '18 at 02:43

score 2 · Accepted Answer · answered Aug 18 '18 at 17:40

2

a) The top answer to the linked question does provide the outline of a formal proof, it's just disguised a little. A more formal outline would go something like this: the function $\sum_i|x-a_i|$ is continuous as a function of $x$. It is decreasing on the domain $(-\infty,a_{n/2})$ and increasing on the domain $(a_{n/2},\infty)$. Hence, the minimum must occur at $a_{n/2}$ (of course I'm being quite sloppy when I say $a_{n/2}$. Read the conversation on the linked post for more precise definition of the "median"). This should lead to a perfectly rigorous proof (no derivatives/subgradients required).

b) The answer is the midpoint between $a_1$ and $a_n$ (i.e. $(a_1+a_n)/2$). See if you can work backwards from this answer.

c) Note that

\begin{equation} \sum_{i=1}^n|x-a_i|^2=\sum_{i=1}^n(x-a_i)^2. \end{equation} Taking the derivative w.r.t $x$ we obtain \begin{equation} 2\sum_{i=1}^n(x-a_i)=2nx-2\sum_{i=1}^na_i. \end{equation} Setting this equal to zero and solving for $x$, we obtain \begin{equation} x^*=\frac{1}{n}\sum_{i=1}^na_i, \end{equation} i.e. the arithmetic mean of the $a_i$. Since the function $\sum_i(x-a_i)^2$ is convex and continuously differentiable as a function of $x$, this is sufficient to conclude that $x^*$ is a global minimizer.

d) Your intuition is correct--as stated, the function is unbounded. A more interesting problem might be to consider maximizing the product over the domain $[a_1,a_n]$. This could be a good exercise.

answered Aug 18 '18 at 17:40

David M.

2,623

To add to Part (d), I do not think that the question can be answered in a closed form even if we are to maximize the product over the domain $[a_1,a_n]$. To see this, take the square of the required function to get $\prod\limits_{i=1}^n,(x-a_i)^2$. To maximize this, one needs to take the derivative and solve an $(2n-1)$-st degree polynomial equation. We already know that $\prod\limits_{i=1}^n,(x-a_i)$ is a factor of the derivative. – Batominovski Aug 18 '18 at 22:49
This leaves a polynomial of degree $n-1$ to be tackled (just realized that this is simply the derivative of $\prod\limits_{i=1}^n,(x-a_i)$). Unless the points $a_i$ are very special, it is unlikely that one can solve the polynomial equation and yield a close form for $n\geq 6$, since there does not exist a close form for a root of a generic quintic polynomial. – Batominovski Aug 18 '18 at 22:52
@Batominovski I think you’re right about this. Feel free to edit my post and add your thoughts – David M. Aug 19 '18 at 03:48
For 1), I've read the proofs on that post. The closer it gets on proving is sayign that since $f$ is decreasing before the median and increasing after the median, then the minimum is in the median. However I could have a median of value 10.000, as a completely crazy one point jump. To make it rigorous I think we need to suppose $f$ is continuous. However there's nothing about this there. Can you suggest something? – Guerlando OCs Aug 27 '18 at 01:03
There's another post in which it considers the derivative, and proves that at the median the derivative is $0$. But this only works to prove that the point is a minimum if the function is differentiable. Also, I should also prove things about the second derivative. – Guerlando OCs Aug 27 '18 at 01:04
@GuerlandoOCs The function is continuous as the sum of continuous functions – David M. Aug 29 '18 at 00:16
@DavidM. yes but I still can't use theorems about derivatives being $0$ because the function is not differentiable – Guerlando OCs Aug 29 '18 at 01:32
Right, but you don’t need to. The function is continuous, decreasing below the median, and increasing above the median. That is sufficient. – David M. Aug 29 '18 at 01:58
@DavidM. I can clearly picture this in my mind, however I've never seen a justification for this. Do you have an idea? For b), can you look at the update in my question? Thank you so much! – Guerlando OCs Aug 29 '18 at 03:08
Try using the definition of a decreasing/increasing function to show that the median is the minimum. You could also appeal to the convexity of the function as well. For part (b) it looks fine to me, basically. In your first sentence, you don't need $a_1<x<x^$, just $x<x^$. You're just using the fact that the reals are ordered...not sure how rigorous you're trying to be. I find your argument convincing. – David M. Aug 29 '18 at 03:21

Minimize $\sum_{i=1}^n|x-a_i|$, $\max \{|x-a_i|, i=1,\cdots,n\}$, $\sum_{i=1}^n|x-a_i|^2$ and maximize $\Pi_{i=1}^n|x-a_i|$

1 Answers1