Definition of the Tensor Product

Question

Could anyone explain in basic language what the tensor product is? I am relatively new to matrix algebra and I am completely new to this specific concept. What exactly does it do for me and what properties of it should I know?

Answer 1

The tensor product is a gadget which allows you to turn bilinear maps into linear maps between vector spaces. I explain this below.

Let $V,W$ and $X$ be vector spaces. We say a function $f:V \times W \to X$ is bilinear if $f(\alpha v_1 + v_2, w) = \alpha f(v_1,w) + f(v_2,w)$ and $f(v, \beta w_1 + w_2) = \beta f(v, w_1) + f(v, w_2)$ for all $v_1, v_2, v \in V$, $w_1, w_2, w \in W$, $\alpha, \beta \in \mathbb{R}$ (let's work over $\mathbb{R}$). This is just saying that $f$ is linear in each "slot".

Now a tensor product of $V$ and $W$ is a vector space $T$, together with a bilinear map $\otimes$, such that if $f:V \times W \to X$ is bilinear, then there is a unique linear map $\varphi: T \to X$ such that $f = \varphi \circ \otimes$, i.e. $f(x,y) = \varphi(\otimes(x,y)) = \varphi(x \otimes y)$ ($\otimes(x,y)$ is denoted by $x \otimes y$).

Now given vector spaces $V$ and $W$, we can always construct a tensor product[1]. Here is one example:

$\otimes: \mathbb{R}^2 \times \mathbb{R}^2$ given by $x \otimes y = x y^t$ ($y^t$ is the transpose) is a bilinear map. One can check (maybe not so easily) that the vector space $T$, where $T = \operatorname{span}\{x \otimes y | x,y \in \mathbb{R}^2\}$, is a tensor product of $\mathbb{R}^2$ and $\mathbb{R}^2$.

I suggest you pick up a book on multilinear algebra, which will discuss these things. Or read other things on the internet. Dummit and Foote is also good.

[1] Multilinear Algebra, Werner Greub

[2] Abstract Algebra, Dummit and Foote