10

When I was first introduced to the concept of average (mean), I was confused. What does average mean? How does one number $\sum_{i=1}^{n} a_{i}$ represent the "central tendency" of a set of data points $a_i$. Then I found a way to deal with this concept. I thought that the average (mean) is "the closest to all the data points at the same time".

Now, I want to prove this. Concisely:

Let $f$ $:$ $\Bbb{R}$$\to$$\Bbb{R}$ be defined by:

$$f(x) = \sum_{i=1}^{n}|x-a_i|$$

To prove: $f(x)$ hits a minimum at $x=\bar a$, where $\bar a$ is the mean of the discrete data points $a_i$. This would mean that the sum of the distances of the mean from the various data points is minimum as compared to any other number.

My attempt: Clearly $f(x)$ is continuous since it is a sum of continuous functions and piece-wise differentiable since it is a sum of such functions.

So, I find $f'(x)$. Before that, let's assume $a_1<a_2<\ldots<a_n$ [clearly, no loss of generality, here]:

$$ f'(x) = \begin{cases} -n, & \text {$x<a_1$} \\ -n+2, & \text{$a_1<x<a_2$} \\ -n+4, & \text{$a_2<x<a_3$} \\ . & . \\ . & . \\ . & . \\ -n+2n = n, & \text{$x>a_n$} \\ \end{cases} $$

Now, the problem arises: $f'(x)=0$ has no solutions for $n$ is odd. For n is even, it has the solution as an entire interval:

$$x \in (a_{n/2},a_{n/2+1})$$

This means i failed, my intuition was wrong from the very beginning. It can be proven [i think] that for $n$ is even, $\bar a$ lies in the above interval, but still: It means that there are more real numbers that are as much the "mean" of the data points as the mean itself [if my "definition" was right].

So, two questions:

  • Why was my intuition wrong?
  • Which intuition is right for averages (mean)?
Truth-seek
  • 1,427
  • 1
    Indeed, the quantity you are minimizing has an optimum solution at the median instead of the mean. The mean only minimizes the average squared error. – TMM Apr 22 '17 at 11:04
  • 2
    See e.g. https://math.stackexchange.com/questions/967138/formal-proof-that-mean-minimize-squared-error-function and https://math.stackexchange.com/questions/113270/the-median-minimizes-the-sum-of-absolute-deviations – TMM Apr 22 '17 at 11:06
  • 1
    This thread and that thread probably answer your question. – ccorn Apr 22 '17 at 11:07
  • So basically, the median is the closest to all data points at the same time and the mean is "as close to half of the data points as it is to the other half". – Truth-seek Apr 22 '17 at 11:20

1 Answers1

10

Your attempts is excellent, but will not lead you to success.

Actually, the minimum of

$$f_1(x) := \sum_{i=1}^{n}|x-a_i|$$

is known to be achieved by the median, another estimator of the central trend.

(The median is defined as the value of rank $n/2$ or the average of the two values of rank $(n\pm1)/2$; for even $n$, all values between the two middle ones, including the median, achieve the minimum.)

Instead, the average achieves the minimum of

$$f_2(x) := \sum_{i=1}^{n}(x-a_i)^2,$$ as one easily shows by canceling the derivative:

$$\frac12\dfrac{f_2(x)}{dx} = \sum_{i=1}^{n}(x-a_i)=nx-\sum_{i=1}^{n}a_i=0.$$

So the average is the solution of the equations $x=a_i$ in the least-squares sense. It is more sensitive than the median to far-away values, because of the squared weighting.

For the sake of the comparison, the plot shows $f_1(x)/4$ and $\sqrt{f_2(x)/4}$ for the points $1, 3, 4, 7$.

enter image description here

  • If we write the differences as $|x-a_i|^1$ and $|x-a_i|^2$ the similarity between the median and the mean becomes even clearer, and we can recover the mode as well via $|x-a_i|^0$ if we temporarily define 0^0=0. – DSM Apr 22 '17 at 15:52
  • @DSM: in continuous distributions, the frequencies exceed one with zero probability and this definition of the mode isn't very useful. –  Apr 24 '17 at 07:29