14

I completely understand the formal mathematical distinction between the direct sum and the tensor product of two vector spaces. I also understand that the direct sum has a nice visual interpretation (especially the direction sum of two 1D vector spaces, or of a 2D and a 1D vector space), where you simply think of attaching the vector spaces together at their respective origins in orthogonal directions inside some higher-dimensional ambient Euclidean space. Question 1: Is there any such simple geometrical picture for the tensor product? (Unfortunately, the simplest nontrivial tensor product - that of two 2D vector spaces - is four-dimensional, so perhaps not.)

Now assume that the vector spaces $V$ and $W$ have (finite) dimensions $N_V$ and $N_W$ respectively. Then there is a natural map from $V \oplus W$ to $V \otimes W$, where we map $(v \oplus w) \in (V \oplus W)$ to $(v \otimes w) \in (V \otimes W)$. The image of this map is a $(N_V + N_W - 1)$-dimensional set $M \subset (V \otimes W)$. (The "$-1$" is because this map is not one-to-one: the preimage of $v \otimes w \neq 0$ is the one-dimensional manifold $$\{ c v \oplus (1/c) w\ |\ c \in (\mathbb{F} \setminus \{0\}) \}.$$ ) Note that $M$ is not closed under vector addition, so it does not form a subspace of $V \otimes W$ (indeed, this fact is exactly the reason why the direct sum and the tensor product of vector spaces are different!). In terms of visual intuition, the fact that the manifold $M$ contains the origin but is not a vector space means that it is "curved" rather than "flat".

For example, if $V = \mathbb{R}^3$ and $W = \mathbb{R}^2$, and we choose bases for $V$ and $W$ in which $(v \in V) \to (v_1, v_2, v_3)$ and $(w \in W) \to (w_1, w_2)$, then we have that $$(v_1, v_2, v_3, w_1, w_2) \in V \oplus W$$ is naturally identified with $$(v_1 w_1, v_1 w_2, v_2 w_1, v_2 w_2, v_3 w_1, v_3 w_2) \in V \otimes W,$$ and that the expression on the LHS parameterizes the four-dimensional smooth manifold $M \subset \mathbb{R}^6$. Question 2: If there is a way to visualize the tensor product of vector spaces, is there an intuitive way to see within that visualization why the natural embedding of $V \oplus W$ into $V \otimes W$ is "curved"?

tparker
  • 6,219
  • 1
    I can't offer any way of visualizing the tensor product but you might find it interesting to know that if you identify the tensor product of $V = \mathbb{R}^n$ and $W = \mathbb{R}^m$ with $M_{n \times m}(\mathbb{R})$ then the decomposable tensors (the image of the embedding of $V \oplus W$ in the tensor product) are identified with matrices of rank $\leq 1$ which is a notion which might be more intuitive. When $n = m = 2$, this is a singular space homeomorphic to a cone over a torus - see https://math.stackexchange.com/questions/1563224/do-the-singular-matrices-form-a-topological-manifold. – levap Aug 06 '17 at 06:24
  • (and matrices of rank $\leq 1$ have no reason to be "straight" in $M_{n \times m}(\mathbb{R})$ as they are defined by the vanishing of quadratic polynomials). – levap Aug 06 '17 at 06:25
  • 1
    @levap Oops, that was a typo in the title. Fixed, thanks. Good point about thinking of the tensor product as a matrix rather than as a vector - indeed, the outer product $v \otimes w$ is always rank-$1$, unless either $v$ or $w$ equals zero. – tparker Aug 06 '17 at 07:00
  • @levap This seems relevant, but is a bit technical for me: https://en.wikipedia.org/wiki/Segre_embedding – tparker Aug 06 '17 at 08:36
  • @levap Apparently the set of $m \times n$ rank-1 matrices that you mentioned form the determinental variety $Y_1$. Haha, this variety possesses something called "syzygys". – tparker Aug 06 '17 at 08:46
  • 1
    The map $v\oplus w\mapsto v\otimes w$ is not injective. For example, $(2 v)\otimes w=v\otimes (2w)$, but $(2 v)\oplus w\ne v\oplus (2w)$. – Jack Lee Aug 06 '17 at 14:40
  • @JackLee Yes, for any $v \oplus w$ and nonzero real number $c$, the direct sum $cv \oplus w/c$ maps to the same tensor product, so the preimage of any tensor product under this map has the cardinality of $\mathbb{R}$. I guess this lack of injectivity reduce the dimensionality for the manifold $M$ to $N_V + N_W - 1$. – tparker Aug 06 '17 at 19:19
  • The dual numbers are a two dimensional algebra that behaves like $\mathbb R$ but with the addition of infinitesimals. Hence algebraically they are 2D but the geometric intuition is "1D". Taking the tensor product of an algebra like $\mathbb C$ with the dual numbers is like adding infinitesimals between the elements of $\mathbb C$. I think that idea can be generalised to arbitrary $V \otimes W$ by treating the elements of $W$ as 1d and infinitesimal – wlad Jan 03 '18 at 22:25
  • 1
    That the natural "embedding" (see comments above) is curved may seem surprising. What I think makes everything intuitively OK is the curved varieties you get are higher-dimensional analogs of doubly ruled surfaces of which the hyperbolic paraboloid is the most relevant example. So the image, although curved, contains two families of flats: one family is a set of copies of $V$, the other a set of copies of $W$. – Will Orrick May 18 '21 at 18:46
  • See also my comment to this question. – Will Orrick May 18 '21 at 18:46
  • ...and for further expansion on this, see my answer here. – Will Orrick Jun 30 '21 at 18:59
  • Sorry for being late, but the elements in the image of map $\varphi: V\oplus W \to V\otimes W$ defined as $\varphi: (v, w) \mapsto v\otimes w$ are called "pure tensors". Note this map is not a homomorphism. Because $(v, w) + (v', w')= (v+v', w+w')$ but $v\otimes w + v' \otimes w' \neq (v+v') \otimes (w + w')$. – user760 May 26 '23 at 19:14

1 Answers1

5

This is probably not what you are looking for, but my "geometric" picture for tensor product comes from function spaces. Consider $V$ to be the space of functions $f:A\to K$ and $W$ the space of functions $g:B\to K$ for two sets $A, B$ and a field $K$. Then the tensor product of $V$ and $W$ consists of the functions $A\times B \to K$.

If it comes to bases and such, the simplest case is the case of Hilbert spaces. If $(f_i)$ is an orthonormal basis of $V$ and $(g_j)$ is an orthonormal basis of $W$, then an orthonormal basis of $V\otimes W$ is given by the functions $(f_i\otimes g_j)(x,y) = f_i(x)g_j(y)$. If find this picture pretty intuitive and geometric. Even more concrete is the example of $A=B=[0,1]$ where the orthonormal basis in both spaces is the (real or complex) Fourier basis.

As a concrete example: The space of square integrable measurable functions $L^2(\Omega)$ it holds that $$ L^2(\Omega_1)\otimes L^2(\Omega_2) = L^2(\Omega_1\times\Omega_2) $$ and if $(\phi_k)$ and $(\psi_k)$ are orthonormal bases of $L^2(\Omega_1)$ and $L^2(\Omega_2)$, respectively, then the functions $$\Phi_{j,k}(x,y) = \phi_j(x)\psi_k(y)$$ form an orthonormal basis of $L^2(\Omega_1\times\Omega_2)$.

Dirk
  • 11,680