Intuition behind the component formula of the scalar product

Question

I have learned the component formula for the scalar product for quite a while, but really, it doesn't seem intuitive at all. I have 2 questions that confuse me while imagining what the scalar product really is, which are:

1- Why is the scalar product defined as $\vec{u} . \vec{v} = \sum u_{i}v_{i}$ (It doesn't make sense at all).

2- Does the concept of angles exist in dimensions higher than 3? In other words, can we always say that $\vec{u} . \vec{v} = ||\vec{u}|| \times ||\vec{v}|| \times \cos \theta$ ?

Answer 1

Let's start with question 2, because it's easier.

In one sense, an angle between two line segments in higher dimensions does make sense, because the two line segments (typically) form a two-dimensional plane. We can always embed a plane in $\Bbb{R}^n$ (or any real inner product space) into $\Bbb{R}^2$ isometrically, by choosing any orthonormal basis and mapping it to the standard basis of $\Bbb{R}^2$ linearly. The angle from the dot product formula will agree with the angle in the embedding, in the classic Euclidean sense (because linear isomorphisms preserve inner products, as well as norms).

In another sense, it makes more sense really to define an angle in terms of the inner product on the space. This way, we can define angles on more abstract inner products, e.g. on spaces of functions, and get an excellent concept of angles. The Cauchy-Schwarz inequality shows that $\cos^{-1} \frac{u \cdot v}{\|u\| \|v\|}$ is always well-defined, and given the previously mentioned isoometry property, we can even expect angle sums of triangles to be $\pi$, and other nice Euclidean properties.

---

The first question is the stickier one. I could respond with a simple "why not?"; we can define whatever we like, however we like. The dot product just so happens to be useful when we define it like that.

I could also choose to interpret your entire question as a request for an intuition about why a simple sum of products could say something so fundamental about angles. Intuition can always be a little tricky, because it is ultimately an individual response that others may not respond to, but here's where my intuition on the topic lies.

Fix a non-zero vector $u$ and an angle $\alpha \in (0, \pi)$. Consider two sets (picture them in $\Bbb{R}^3$): the unit sphere $S_X = \{x : \|x\| = 1\}$, a hollow cone consisting of all the vectors whose angle with $u$ is $\alpha$ (as well as $0$, the cone's vertex). The cone originates at the sphere's centre, so there's definitely going to be an intersection. What shape is this intersection?

In $\Bbb{R}^3$, it will be a circle laid on the sphere. On the cone, the circle will be an orthogonal cross-section. In higher dimensions, it will be a lower-dimensional sphere, a manifold of codimension $2$, rather than the usual $1$.

This same circle may also be formed by taking a hyperplane, whose normal direction is $u$, that is sufficiently offset from the origin, so as it slices either the sphere or the cone into that same circle.

The way I see it, this fact, that the same circle is the intersection of any two of the three geometric sets I've described, is the intuitive reason why angles are tied up with this product sum.

The hyperplane is a level set of a linear functional. That is, a hyperplane has the equation $f(x) = k$, where $f : \Bbb{R}^n \to \Bbb{R}$ is linear and $k$ is a constant. This hyperplane represents the sum product representation, as all linear functionals $f$ take the form: $$f(x_1, \ldots, x_n) = x_1 f(e_1) + \ldots + x_n f(e_n),$$ where $e_1, \ldots, e_n$ is the standard basis. In other words, they are a dot product between your argument $(x_1, \ldots, x_n)$ and a fixed vector depending only on $f$: $(f(e_1), \ldots, f(e_n))$.

The cone is as described: it's the set of all vectors with the fixed angle $\alpha$. This represents the angle formulation of the dot product.

If we begin with a particular value for $\sum_{i=1}^n a_i x_i$, i.e. we lie on a particular plane, but we restrict our norm of $(x_1, \ldots, x_n)$ to $1$, then we lie on a cone of fixed angles. Conversely, if we begin on a cone of fixed angles, and restrict our norm to $1$ once more, then we lie on a plane, and fix our sum $\sum_{i=1}^n a_i x_i$. In this way, we see, intuitively, the connection between these two seemingly disparate definitions.

Answer 2

In a nutshell, the dot/scalar product is an example of what we call inner product. The inner product is an operation that we use to make sense of the concept of angles and lengths in an abstract vector space, e.g. higher dimensional vector spaces. The point is, at beginning we define angles and then the cosine and literally see that $\cos\theta$ represents the projection of the vector $\vec{u}$ in the direction of $\vec{v}$ (where the angle between them is $\theta$). However, the opposite approach is valid as well: one defines an operation called inner product, which satisfies some of properties that we expect from the idea of "projection", which are

1. Fixing the vector $\vec{v}$, we expect linearity in the first component: $(\lambda \vec{u} + \vec{w})\cdot \vec{v} = \lambda(\vec{u}\cdot \vec{v}) + \vec{w}\cdot \vec{v}$
2. We also expect symmetry of the projection, i.e. the projection of $\vec{u}$ onto $\vec{v}$ is the same as the projection of $\vec{v}$ onto $\vec{u}$: $\vec{u}\cdot \vec{v} = \vec{v}\cdot\vec{u}$
3. If we imagine the vector $\vec{v}$ as an axis, we expect that the projection of $\vec{v}$ onto itself is always non negative; that is, $\vec{v}\cdot \vec{v} \geq 0$.

Having these properties we can define the norm by putting $\vert \vec{v}\vert^2 = \vec{v}\cdot\vec{v}$, and then we can prove the inequality $$ \vert \vec{u}\cdot\vec{v}\vert \leq \vert\vec{u}\vert\vert\vec{v}\vert \Leftrightarrow - 1 \leq \frac{\vec{u}\cdot\vec{v}}{\vert\vec{u}\vert\vert\vec{v}\vert} \leq 1 $$ which is known as Cauchy-Schwarz inequality. Now, if we think of $\vec{v}$ being a unit vector, i.e. $\vert\vec{v}\vert = 1$, we can think that the term above is some sort of "projective factor", since it says that $\vert \vec{u}\cdot\vec{v}\vert \leq \vert\vec{u}\vert$ or, in another words, the projection of $\vec{u}$ on $\vec{v}$'s direction is less than or equal its norm, which is what we expect, since projection often leads to a loss of information. To finish, we let $$ \theta = \cos^{-1}\left(\frac{\vec{u}\cdot\vec{v}}{\vert\vec{u}\vert\vert\vec{v}\vert}\right) \Rightarrow \cos\theta = \frac{\vec{u}\cdot\vec{v}}{\vert\vec{u}\vert\vert\vec{v}\vert} $$ since both of them reflect this "projective factor".

In the Euclidean case, each vector $\vec{v}$ can be written as $$ \vec{v} = v_1\hat{e}_1 + v_2\hat{e}_2 $$ where $\hat{e}_1 = (1,0)$ and $\hat{e}_2 = (0,1)$, and for our experience we think of them being orthogonal, i.e. $\hat{e}_1\cdot\hat{e}_2 = 0$. This naturally implies that \begin{align} \vec{u}\cdot\vec{v} &= (u_1\hat{e}_1 + u_2\hat{e}_2)\cdot\vec{v}\\ &= u_1\hat{e}_1\cdot\vec{v} + u_2\hat{e}_2\cdot\vec{v}\\ &= u_1\vec{v}\cdot\hat{e}_1 + u_2\vec{v}\cdot\hat{e}_2\\ &= u_1(v_1\hat{e}_1 + v_2\hat{e}_2)\cdot\hat{e}_1 + u_2(v_1\hat{e}_1 + v_2\hat{e}_2)\cdot\hat{e}_2\\ &= u_1v_1(\hat{e}_1\cdot\hat{e}_1) + u_1v_2(\hat{e}_2\cdot\hat{e}_1) + u_2v_1(\hat{e}_1\cdot\hat{e}_2) + u_2v_2(\hat{e}_2\cdot\hat{e}_2)\\ &= u_1v_1\vert\hat{e}_1\vert + u_2v_2\vert\hat{e}_2\vert\\ &= u_1v_1 + u_2v_2. \end{align} The same calculation can be done in higher dimensions.

Answer 3

For now, let's focus on two-dimensional vectors ($\mathbb{R}^2$). Suppose that we know the length ($r$ and $s$) and angles ($\alpha$ and $\beta$) for two vectors $\vec{u}$ and $\vec{v}$. In rectangular coordinates, these are:

$$\vec{u} = (r\cos\alpha, r\sin\alpha)$$ $$\vec{v} = (s\cos\beta, s\sin\beta)$$

Now, let's define $\vec{u} \cdot \vec{v} = \sum_{i=1}^2 u_iv_i$. For now, don't think about why we're defining it, just work through the math.

$$\vec{u} \cdot \vec{v} = (r\cos\alpha)(s\cos\beta) + (r\sin\alpha)(s \cos \beta)$$ $$=rs \cos\alpha\cos\beta + rs\sin\alpha\cos\beta$$ $$=rs(\cos\alpha\cos\beta + rs\sin\alpha\cos\beta)$$ $$=rs\cos(\alpha - \beta)$$

Which is the product of the lengths of the vectors and the cosine of the angle between them. In other words,

$$\vec{u} \cdot \vec{v} = \left|\vec{u}\right| \left|\vec{v}\right| \cos\theta$$ $$\theta = \arccos \frac{\vec{u} \cdot \vec{v}}{\left|\vec{u}\right| \left|\vec{v}\right|}$$

This gives us a simple formula for the angle between two vectors. Of course, if we're given the angles $\alpha$ and $\beta$ to begin with, we can just compute $\left| \alpha - \beta \right|$ directly. But the nice thing about the dot-product formula is that we don't need to express the individual vectors in polar coordinates in order to compute the angle between them. If we're just given the rectangular coordinates $\vec{u} = (x_1, y_1)$ and $\vec{v} = (x_2, y_2)$, then the formula becomes:

$$\theta = \arccos \frac{x_1y_1 + x_2y_2}{\sqrt{x_1^2 + y_1^2} \sqrt{x_2^2 + y_2^2}} $$

It can be shown that the formula $\vec{u} \cdot \vec{v} = \left|\vec{u}\right| \left|\vec{v}\right| \cos\theta$ is also valid for three-dimensional vectors. Actually showing this (perhaps using cylindrical or spherical coordinates) is left as an exercise for the reader.

And that's why the dot product exists: It gives us a simple operation that can be used to find a geometrically-useful thing (the angle between two vectors).

For dimensions 4 and higher, in which our conventional notions of geometry don't apply, we can even use $\arccos \frac{\vec{u} \cdot \vec{v}}{\left|\vec{u}\right| \left|\vec{v}\right|}$ to define the “angle” between two vectors.