What is the Euclidean inner product intuition?

Question

The Euclidean inner product $<x,y>$ of the vectors $x , y \in \mathbb{R}^n$ is defined by:

$\langle x,y\rangle = x_1y_1 + x_2y_2 + x_3y_3 + ... + x_ny_n$

I am unable to find the intuition behind this? Why do we need this inner product? What happens to the vectors under equation and what does the scalar answer tell us?

P.S. I have already seareched it, most the articles take it into geometric terms and introduce $\cos\theta$. Can anybody help me visualize thing without angular stuff?

Edit: Question addressed here uses the angle to explain geometric dot product. I am wondering if there exist some explanation for dot product in Vector Space.

Answer 1

You are probably familiar with the following formula involving a vector $x = (x_1, \ldots, x_n) \in \mathbb R^n$ and $n$ constants $a_1, \ldots, a_n$ that are not all zero: $$ a_1 x_1 + a_2 x_2 + \cdots + a_n x_n = 0. $$

This formula describes an $(n-1)$-dimensional hyperplane in $\mathbb R^n.$ Let's call that hyperplane $A.$ That is, any vector $x$ that satisfies the formula lies in the hyperplane $A.$

If we define a vector $a = (a_1, \ldots, a_n),$ then another way to write the formula is $$ \langle x, a\rangle = 0. $$

Now if we choose an arbitrary vector $x \in \mathbb R^n,$ it may happen that $\langle x, a\rangle \neq 0.$ If we have two such vectors, let's say $x'$ and $x'',$ such that $\langle x', a\rangle > 0$ and $\langle x'', a\rangle > 0,$ then $x'$ and $x''$ will be on the same side of the hyperplane $A.$ But if $\langle x', a\rangle < 0 < \langle x'', a\rangle$ then the vectors are on opposite sides.

For a given vector $x,$ suppose you find a vector $y$ such that $x - \langle x, a\rangle y$ is in the hyperplane $A.$ Then $x' - \langle x', a\rangle y$ also is in the hyperplane $A$ for any other vector $x'.$ That is, $\langle x, a\rangle$ tells you how many times the length of $y$ you have to travel in the direction of $y$ to get from $x$ to the hyperplane, or in other words, $\langle x', a\rangle$ is a kind of measurement of the distance of $x$ from the hyperplane (measured in some particular units in some particular direction).

All of this works without any "angles" (unless you consider "parallel" to be a "zero angle").

You don't even need $(x_1, \ldots, x_n)$ to be coordinates over an orthonormal basis, although if the basis is orthonormal then other nice results follow. For example, $$\sqrt{\langle x, x\rangle} = \sqrt{x_1^2 + x_2^2 + \cdots + x_n^2},$$ which is the length of $x$ (according to the Pythagorean Theorem) if the basis of $(x_1, \ldots, x_n)$ is orthonormal. Admittedly, to speak of "normals" one must have a concept of things being perpendicular, which starts to sound like we're dealing with angles again. But they're right angles, which are especially simple to work with.

Answer 2

$<x,y>$ is positive if and only if the angle between $x$ and $y$ is less than 90 degrees. $<x,y>$ is sort of like a measure of how pointed in the same direction $x$ and $y$ are.

The geometry of least squares problems and separating hyperplanes is useful for gaining intuition about inner products.

Answer 3

The intuition is gathered from the following, given by the Law of Cosines

$$\| a -b\|^{2} = \| a\|^{2} + \| b\|^{2} - 2\| a\|^{2} \| b\|^{2}\cos(\theta) $$

$$ \| a-b\|^{2} = (a-b)^{2}\cdot(a-b)^{2} $$ $$ \| a-b\|^{2} = a\cdot a - a\cdot b - b\cdot a + b \cdot b $$

useful to note that $a \cdot a = \| a\| $

$$ \| a-b\|^{2} = \|a \| - a\cdot b - b\cdot a + \| b\|$$

further more we have $ a \cdot b = b\cdot a $ $$ \| a-b\|^{2} = \|a \| - 2 a\cdot b + \| b\|$$

so we have

$$\| a -b\|^{2} = \| a\|^{2} + \| b\|^{2} - 2\| a\|^{2} \| b\|^{2}\cos(\theta) $$

$$ \| a\|^{2} - 2 a \cdot b + \| b\|^{2} = \|a\|^{2} + \|b\|^{2} - 2\|a\| \|b\| \cos(\theta) - 2 a \cdot b = -2 \|a \| \|b\| \cos(\theta) $$

yielding finally $$a \cdot b = \| a \| \| b\| \cos(\theta) $$

Now suppose that

$$ \|a\| = \|b\| =1 $$ $$ a \cdot b = 1 \cdot 1 \cos(\theta) = \cos(\theta)$$ ok when is this negative. We have a unit circle here for vectors

$$ a \cdot b = \cos( \theta) < 0 \implies \frac{n\pi}{2} < \theta < \frac{3n\pi}{2} $$

that is we are in one part of the circle. However, it goes on forever.

Note from lamar