24

I am wondering what exactly is the relationship between the three aforementioned spaced. All of them seem to show up many times in: Linear Algebra, Topology, and Analysis. However, I feel like I'm missing the bigger picture of how these spaces relate to each other. For example, in my course in multi-dimensional analysis, we started out talking about metric spaces, but later suddenly switched to normed vector spaces, without any explicit mention of this transition. In linear algebra we usually talked about inner product spaces, and in topology we talked about metric spaces and topological spaces.

The bigger picture of the relation between these three is still unclear to me. Which is used where, for what reason, and how do they relate?

I do know the definitions of all three of them:

A metric space is a pair $(S,d)$ with $S$ a set and $d: S \times S \to \mathbb{R}_{\geq 0}$ a metric:

  • $d(x,x) = 0$ for all $x \in S$ and $d(x,y) >0$ for $x \neq y$,
  • $d(x,y) = d(y,x)$,
  • $d(x,z) \leq d(x,y) + d(y,z)$.

A (real) inner product space is a pair $(V,\langle \cdot \rangle)$ where $V$ is a (real) vector space and $\langle \cdot \rangle: V \times V \to \mathbb{R}$ is an inner product:

  • $\langle v,w \rangle = \langle w,v \rangle$,
  • $\langle a_1 v_1 + a_2v_2,w \rangle = a_1\langle v_1,w \rangle + a_2\langle v_2,w \rangle$ for all $a_1,a_2 \in \mathbb{R}$,
  • $v \neq 0 \Longrightarrow \langle v,v \rangle > 0$.

A (real) normed vector space is a pair $(V,\|\cdot\|)$ where $V$ is a (real) vector space and $\|\cdot\|: V \to \mathbb{R}_{\geq 0}: v \mapsto \|v\|$ is a norm on $V$:

  • $\|v\| \geq 0$ and $\|v\| = 0 \ \Longleftrightarrow \ v = 0$.
  • For $t \in \mathbb{R}$ and $v \in V$ we have $\|tv\| = |t|\|v\|$
  • $\|v+w\| \leq \|v\| + \|w\|$.

I also know that an inner product gives rise to a norm by taking $\|v\| = \sqrt{\langle v,v \rangle}$, for example the Euclidean norm derives from the standard inner product on $\mathbb{R}^n$ in this way. And Cauchy-Schwarz: $|\langle x,y \rangle| \leq \|x\|\|y\|$.

I'm not interested in details about the definitions but in the intuition and bigger picture of these three spaces, and how they show up in Analysis.

Sigurd
  • 1,649
  • 10
  • 19
  • 4
    You may be interested in Jordan - Von Neumann theorem, stating a necessary and sufficient condition for a normed vector space to be a inner product space: here the keys are parallelogram law and polarization identity. – Bob Jul 05 '18 at 14:26
  • That's an interesting theorem I didn't know yet, thanks for mentioning! – Sigurd Jul 05 '18 at 14:34
  • 2
    In the same spirit: a metric defined onto a vector space comes from a norm (in the way explained in the answers) if and only if the metric is translation invariant and satisfies the scaling property. – Bob Jul 05 '18 at 14:41
  • Finally you may wonder if metric spaces are more general then normed vector spaces. In a sense, the answer is no: given a metric space X, there exists a normed vector space V and a subset A of V, such that X is isometric to A. – Bob Jul 05 '18 at 14:50
  • "In the same spirit: a metric defined onto a vector space comes from a norm (in the way explained in the answers) if and only if the metric is translation invariant and satisfies the scaling property". Sounds like a beautiful result! – Sigurd Jul 05 '18 at 14:52
  • I'm curious: I've never heard the term "inproduct space" before, only the term "inner product space". Is "inproduct space" a common descriptor outside of US English, or is this a typo/translation error that should be corrected? – Xander Henderson Jul 05 '18 at 16:12
  • Yeah I think you are right! They used this word in my (Dutch) textbook on linear algebra, so I got used to it and I thought it carries over to English right away, but I can't find it anywhere being used in an English text, so it should be inner product space then, thanks for the observation. – Sigurd Jul 05 '18 at 16:51
  • I've changed the terminology everywhere now – Sigurd Jul 05 '18 at 16:58
  • Thanks for the observation, it has been changed now. – Sigurd Jul 06 '18 at 00:56

3 Answers3

49

You have the following inclusions:

$$\{ \textrm{inner product vector spaces} \} \subsetneq \{ \textrm{normed vector spaces} \} \subsetneq \{ \textrm{metric spaces} \} \subsetneq \{ \textrm{topological spaces} \}.$$

Going from the left to the right in the above chain of inclusions, each "category of spaces" carries less structure. In inner product spaces, you can use the inner product to talk about both the length and the angle of vectors (because the inner product induces a norm). In a normed vector space, you can only talk about the length of vectors and use it to define a special metric on your space which will measure the distance between two vectors. In a metric space, the elements of the space don't even have to be vectors (and even if they are, the metric itself doesn't have to come from a norm) but you can still talk about the distance between two points in the space, open balls, etc. In a topological space, you can't talk about the distance between two points but you can talk about open neighborhoods.

Because of this inclusion, everything that works for general topological spaces will work in particular for all other spaces, but there are some things you can do in (say) normed vector spaces which don't make sense in a general topological space. For example, if you have a function $f \colon V \rightarrow \mathbb{R}$ on a normed vector space, you can define the directional derivative of $f$ at $p \in V$ in the direction $v \in V$ by the limit

$$ \lim_{t \to 0} \frac{f(p + tv) - f(p)}{t}. $$

In the definition, you are using the fact that you can add the vector $tv$ to the point $p$. If you try to mimick this definition in a topological space, then since the set itself doesn't have the structure of a vector space, you can't add two elements so this definition doesn't make sense. That's why during your studies you sometimes restrict your attention to a smaller category of spaces which has more structure so you can do more things in it.

You can discuss the notions of continuity, compactness only in the category (context) of topological spaces (but for reasons of simplicity it is often done in the beginning of one's studies in the category of metric spaces). However, once you want to discuss differentiability, then (in first approximation, before moving to manifolds) you need to restrict your category and work with normed vector spaces. If you also want to discuss the angle that two curves make, you will need to further restrict your category and work with inner product vector spaces in which the notion of angle makes sense, etc.

levap
  • 65,634
  • 5
  • 79
  • 122
  • Incredible answer, thanks! This total picture was exactly what I was looking for. To follow up: that means that every inner product vector space is in particular a topological space right? Interesting realisation, I always saw them as completely different structures. Another question: how exactly does an inner product allow you to measure angles between vectors? Though I've indeed heard of this before, it was never really clear to me what for example the standard inner product $\vec{x} \cdot \vec{y} = x_1y_1 + \ldots + x_ny_n$ has to do with the angle between these vectors. – Sigurd Jul 05 '18 at 14:49
  • A lot became clear while reading this answer, in combination with the others' input. – Sigurd Jul 05 '18 at 14:53
  • @Sigurd You are probably familiar with the concepts of Euclidian 'length' and 'angle', that can be seen or come from the dot product. The inner product is a generalisation of the concept of dot product, and so also of the concepts of length/distance and angle. – AnyAD Jul 05 '18 at 15:05
  • Specifically, by Cauchy-Schwartz, for unit vectors $x, y$, we have $|\langle x, y\rangle|\le|x|\cdot|y|=1$, so there's a unique angle $\alpha\in [0,\pi] $ satisfying $\cos\alpha=\langle x, y\rangle$. – Berci Jul 05 '18 at 15:39
  • Do you have any particular reason to write the inclusions with $\subseteq$? Clearly, all of those are actually proper subspaces. – leftaroundabout Jul 05 '18 at 16:14
  • @leftaroundabout: Not really, just a a habit. I've edited the answer to reflect it, thanks. – levap Jul 05 '18 at 16:16
  • 1
    @Sigurd: Let's say we work in $\mathbb{R}^2$ with the standard inner product and $\vec{x} = (1,0)$ while $\vec{y} = (a,b)$ with $a^2 + b^2 = 1$ (so that both vectors lie on the unit circle). Then $\vec{x} \cdot \vec{y} = a$. If you draw both vectors (eminating from the origin) and the angle $\theta$ between them, you'll see that $\cos \theta = a$. This might look like a special case, but given arbitrary vectors $\vec{x},\vec{y}$, the angle between doesn't depend on the length so you can always scale them to be of unit length and it also shouldn't change if we rotate them so you can always – levap Jul 05 '18 at 16:26
  • @Sigurd: rotate one of them so it becomes $(1,0)$ and then use the observation above. In general, in an inner product space (whose elements might not be "standard" vectors but functions, polynomials, etc), one defines the cosine of the angle between two unit vectors by the equation $\cos \theta = \left< \vec{x}, \vec{y} \right>$. One needs to show that the right hand side is between $-1$ and $1$ in absolute value (this follows from Cauchy-Schwartz) and then this definition generalizes the standard notion from $\mathbb{R}^2$ (and $\mathbb{R}^n$). – levap Jul 05 '18 at 16:29
  • 1
    Thanks, that's clarifying! And thanks to all for your help on this topic, the discussion was really useful for me to organize several concepts that were floating around in my knowledge somewhere. – Sigurd Jul 05 '18 at 16:49
  • It might be worth pointing out that metric spaces are not a subcategory of topological spaces - there can be multiple metrics which give the same topology. There is a fully faithful functor from metric spaces to topological spaces, though. And similarly for normed vector spaces to metric spaces (there could be different vector space structures with norms that give the same metric space, for example by choosing different origins in $\mathbb{R}^2$). – Daniel Schepler Jul 05 '18 at 17:23
  • Though inner product spaces to normed vector spaces does work as the subcategory of spaces with norm satisfying the parallelogram law, with the inner product recovered by the polarization identity. – Daniel Schepler Jul 05 '18 at 17:24
  • Just to add to this very good answer, in the vein of @DanielSchepler's comments: Building upon this, there are often interesting questions with non-trivial answers like: so which norms "come from"/"can be refined to" inner products (https://math.stackexchange.com/q/21792/96384), or which topologies "come from"/"can be refined to" a metric (google "metrizable space") etc.; and, which different inner products give the same metric or which different norms give the same topology? (Put abstractly, these questions ask about the functorial relationships between these categories.) – Torsten Schoeneberg Jul 05 '18 at 21:01
6

  1. Every inner product space is (can be naturally made) a normed space by defining $$\|x\|:=\sqrt{\langle x, x\rangle} $$ (following the leading example $\Bbb R^n$)

  2. Every normed space is, by definition, a linear space, and at the same time can be naturally equipped with a metric: $$d(x, y) :=\|y-x\|$$

You can check that the respective axioms are indeed satisfied.

Metric spaces provide a general framework for continuity and uniform continuity.
We can define differentiation on normed spaces.

Noticing that the class of (nice-in-a-way) real or complex valued functions themselves form a linear space, we can investigate several norms for them, even inner products, which is the study of functional analysis.

Berci
  • 90,745
2

An inner product space $\left(V,\langle\cdot,\cdot\rangle\right) $ is also a normed vector space $\left(V,\lVert\cdot\rVert\right) $, taking $$ \lVert v\rVert \colon= \sqrt{\langle v,v\rangle} \qquad\forall v\in V$$

A normed vector space $\left(V,\lVert\cdot\rVert\right) $ is also a metric vector space $(V,d) $, taking $$ d(x,y)\colon=\lVert x-y\rVert \qquad\forall x,y\in V$$

In short $$ \text{Inner product}\implies \text{Norm}\implies \text{Metric} $$

francescop21
  • 1,124