0

I can easily verify it algebraically and geometrically, but I definitely wouldn't have guessed that the Euclidean distance in e.g. $\mathbb R^3$ would be $\sqrt{x^2 + y^2+ z^2}$.

Why squared? Is there any deeper insight to be had here?

Willerie
  • 21
  • 3
    Um... Pythagoras' theorem? –  Jun 30 '23 at 15:26
  • Thanks, yeah, I know the Pythagorean theorem gives you the answer — in fact I originally wrote the words 'Pythagorean theorem' in the original post before editing it for brevity. The Pythagorean theorem is equivalently the thing I'm trying to more deeply grok.

    It feels like there should be something more substantive behind why $||x||_2$ gives the empirically correct answer, and not e.g. some other $||x||_p$.

     – Willerie Jun 30 '23 at 21:48

2 Answers2

2

It is well-known that there are other norms that equip $\mathbb R^n$ with the same topology. For example $$ \|x\|_p=(x_1^p+\dots+ x_n^p)^{\frac{1}{p}}\,,\quad\text{ or, }\quad \|x\|_\infty=\max_{i=1,\dots,n}|x_i|\,. $$ However, only the Euclidean norm $\|x\|_2$ satisfies the Pythagorean theorem. This is also the only norm under which balls and spheres are symmetric under arbitrary rotations. I believe that last property is it that singles out the Euclidean norm.

Another fact is that $\|x\|_p$ comes from an inner product $\langle\,.\,,\,.\,\rangle$ if and only if $p=2$. This is equivalent to the norm satisfying each of the following:

  • The parallelogram law: $$ 2\|x\|^2+2\|x\|^2=\|x+y\|^2+\|x-y\|^2\,. $$ holds for all $x,y\,.$

  • The possibility to define the inner product via one of the following equivalent polarization identities: \begin{align} \langle x,y\rangle&:=\frac{\|x+y\|^2-\|x-y\|^2}{4}\,,\\[2mm] \langle x,y\rangle&:=\frac{\|x+y\|^2-\|x\|^2-\|y\|^2}{2}\,,\\[2mm] \langle x,y\rangle&:=\frac{\|x\|^2+\|y\|^2-\|x-y\|^2}{2}\,. \end{align}

  • Ptolemy's inequality: $$ \|x-y\|\,\|z\|+\|y-z\|\,\|x\|\ge \|x-z\|\,\|y\|\,,\quad \forall x,y,z\,. $$

Kurt G.
  • 14,198
  • Thank you! This led me down a trail that caused me to understand a few things better, including having a better intuition for why the normal distribution has the functional form it does, given the knowledge that multivariate Gaussians are rotationally invariant (in short, $f(x)=e^{-x^2}$ is the normalizable functional form satisfying $f(x)f(y)=g(x^2+y^2)$). The answer "it's the only Lp norm with rotation invariance" feels like a noteworthy addition to my understanding. Cheers! – Willerie Jul 01 '23 at 06:14
  • I think the Gaussian distribution is special because of the central limit theorem. Anyway, there is an aspect of geometry in multivariate Gaussians because we can rotate the random vectors to make them independent (just diagonalize the covariance matrix). Another topic where the square features prominently is standard deviation and variance in statistics. – Kurt G. Jul 01 '23 at 08:30
1

There is no deeper insight. This is a double application of the Pythagorean theorem.

Imagine a point $(x,y,z)\in\mathbb R^3$. From $(x,y,z)$ to $(x,y,0)$, the distance is $\lvert z\rvert$.

From the origin to $(x,y,0)$ we get that the distance is $r=\sqrt{x^2+y^2}$ by the Pythagorean theorem.

Now we have a right triangle whose legs are $r$ and $\lvert z\rvert$ and whose hypotenuse is the distance from the origin to $(x,y,z)$. Another application of the Pythagorean theorem gives the distance $d=\sqrt{r^2+\lvert z\rvert^2}=\sqrt{x^2+y^2+z^2}$

John Douma
  • 11,426
  • 2
  • 23
  • 24