2

I am curious about tensors and tensor notation and how it translates to common linear algebra stuff that I already know. For instance, we can express an outer product $AA^\top$ as a sum of outer products like so, with $j$ representing the column indices.

$$ AA^\top = \sum_j a_ja_j^\top $$

How would this be expressed in tensor notation? It would seem that we would need to concatenate all of the tensor products along a new dimension and then multiply a vector of ones along the new dimension $i$ like so,

$$ B_{ijk} = [a_1 \otimes a_1, \dots, a_n \otimes a_n]^\top $$

and then take a vector of ones as a covector and multiply it so that it sums along the $i$ dimension.

$$ \mathbb{1}^iB_{ijk} = AA^\top $$

Questions

  1. Is this correct?
  2. Is there a better way to express this?
  3. Are there any good books or resources to recommend to go deeper into learning this sort of thing?

Thanks

Joff
  • 906
  • Your notation for $B_{ijk}$ doesn't make much sense. First of all, it looks like you're constructing a block matrix rather than concatenating along a new dimension, and second the right hand side makes no reference to the indices $j,k$ that are present on the left. – Ben Grossmann Sep 13 '22 at 15:17
  • Instead, I think the most sensible way to write this vector is as $$ B_{ijk} = [a_k \otimes a_k]{ij}, $$ or more simply as $B{ijk} = a_{ki}a_{ji}$. – Ben Grossmann Sep 13 '22 at 15:18
  • How can I read the $ij$ index in $B_{ijk} = [a_k \otimes a_k]_{ij}? I can see all the pieces are there but I can't make sense of it in my head. – Joff Sep 13 '22 at 15:31
  • In general, if $p,q$ are 1-dimensional tensors, then $[p\otimes q]_{ij} =p_iq_j$ – Ben Grossmann Sep 13 '22 at 15:35
  • 1
    By the way, I think it would be more natural to express $AA^\top$ as the contraction of the order-4 tensor product $B = A \otimes A^\top$, which is expanded in two new dimensions. – Ben Grossmann Sep 13 '22 at 15:47
  • Thanks, that is just hurting my brain right now, are you aware of any convenient proof or identity which shows why that is correct? If you can sum these things up into an answer I think it will have answered everything in my question. Any book recommendations for learning stuff like this? – Joff Sep 13 '22 at 16:09
  • 1
    You might find this post on the connection between tensor contraction and matrix multiplication to be useful. I don't have a recommendation, but you might find this list to be helpful – Ben Grossmann Sep 13 '22 at 16:41

1 Answers1

1

Preliminaries.

Matrix multiplication can be seen as a contraction of a tensor product: $$(AB)^i{}_j=A^i{}_kB^k{}_j=(A\otimes B)^{ik}{}_{kj}$$

If $\mathbf A$ is $(0,2)$ tensor its transpose has entries $$(A^\intercal )_{ij}=A_{ji}$$

Putting it together.

$$(AA^\intercal)^i{}_j=A^{ik}(A^\intercal)_{kj}=A^{ik}A_{jk}$$ Simple as that.

K.defaoite
  • 12,536