1

Deviation with respect of the Mode instead from the Mean Value

I am trying to figure out if the following calculations make sense. If I try to make a deviation measure from the Mode value "$\nu$" as it is traditionally done from the mean value "$\bar{x}$", I will have the following:

$$\begin{array}{r c l} \text{Var}\nu[x] & := & E[(x-\nu)^2]\\ & = & E[x^2-2\nu x+\nu^2] \\ & = & E[x^2]-2\nu \underbrace{E[x]}_{=\ \bar{x}}+\nu^2 \underbrace{E[1]}_{=\ 1}\\ & = & E[x^2]-2\nu \bar{x}+\nu^2\pm(E[x])^2 \\ & = & \underbrace{E[x^2]-(E[x])^2}_{=\ \text{Var}[x]}+\underbrace{(E[x])^2}_{=\ \bar{x}^2}-2\nu \bar{x}+\nu^2 \\ & = & \text{Var}[x]+\bar{x}^2-2\nu \bar{x}+\nu^2 \\ & = & \text{Var}[x]+(\bar{x}-\nu)^2 \end{array}$$

So far, if the algebra is right I found that the deviation is given by the traditional variance plus the squared of the difference among the Mode and the mean value, which it is just adding a constant value: $$\text{Var}\nu[x] = \text{Var}[x]+(\bar{x}-\nu)^2$$

Here is where I have doubts: I could have made the same metric for any arbitrary value instead of the Mode $\nu$, so I think that somehow this procedure could be mistaken: Maybe $\nu$ have to be treated as some $\nu[x]$ such it is not possible to treat it as a constant value $E[\nu[x]\cdot f(x)]\neq \nu[x]\cdot E[f(x)]$.

I hope you could help me to figure this out.

Joako
  • 1,380
  • 2
    $\mathbb E[(X-k)^2] = \mathbb E[(X-\mathbb E[X])^2] +(\mathbb E[X]-k)^2$ is correct for all values of $k$, assuming the expectations exist – Henry Dec 01 '23 at 08:05

1 Answers1

1

Your derivation is correct.

This is related to concept of the mean squared error of an estimate. If $\hat\theta$ is an estimate of $\theta$, then the mean squared error of $\hat\theta$ is defined as: $$MSE(\hat\theta) = E[(\hat\theta-\theta)^2]$$ and it's a well known result that this decomposes into: $$MSE(\hat\theta) = Var(\hat\theta) + (E(\hat\theta)-\theta)^2$$ where the second term is the square of the bias of $\hat\theta$ (i.e., the bias is defined as $E(\hat\theta)-\theta$).

In your example, you are using a single data point $x$ (whose expected value is $E(x)=\bar{x}$) to estimate the mode $\nu$, and your derivation shows that this estimate will have a mean square error ($\mathrm{Var}\nu[x]$) that's the sum of its variance ($\mathrm{Var}[x]$) and its (squared) bias with respect to $\nu$, the term $(x-\nu)^2$.

From a technical point of view, you could use the data point $x$ to estimate any $\theta$. The mean, mode, median, three-quarters the mode plus four-fifths of the median, or even the constant $\theta=42$. The data point $x$ might not be a good estimate of these $\theta$, but it would still be an estimate. Its MSE with respect to this estimate would decompose into the variance of the data point itself $\mathrm{Var}[x]$ plus the squared bias, which would depend on the actual $\theta$ being estimated.

For example, if you tried to use the data point $x$ to estimate the distribution's variance $\sigma^2$, the MSE would decompose the same way: $$MSE(\sigma^2) = \mathrm{Var}[x] + (\bar x - \sigma^2)^2$$ It wouldn't make a lot of practical sense in most situations, but the math works fine.

K. A. Buhr
  • 1,672