Dot product vs Matrix multiplication, is the later a special case of the first?

Question

I seemed to have thoroughly confused myself today...

Long story short, the question is simple. Is matrix multiplication just a special case of the dot product of two sets of vectors when the sets of vectors have the same cardinality and all vectors in both sets have the same length?

I assume the answer is yes from reviewing the computation of matrix multiplication and the dot product.

Answer 1

Dot product is defined between two vectors.

Matrix product is defined between two matrices.

They are different operations between different objects.

The connection between the two operations that comes to my mind is the following: To calculate the $c_{i,j}$ entry of the matrix $C:=AB$, one takes the dot product of the $i$'th row of the matrix $A$ with the $j$'th column of the matrix $B.$

Answer 2

$$ A^TB \ \equiv A \bullet B \iff A \ \text{and} \ B \ \text{are} \ n \times 1 \ \text{matrices}. $$

So you could think of a dot product as a special case of matrix multiplication.

Answer 3

What @Pawel said, additionally, though, I would like to add that there is a nice duality between $1\times 2$ matrices and 2d vectors.

3Blue1Brown covers this in the 9th episode of his series 'Essence of Linear Algebra'

Answer 4

There is a difference between vectors and matrices and also a connection

Vector

A vector is an object that has both a magnitude and a direction. Example Force and Velocity. Both have magnitude as well as direction. However, we need to specify also a context where this vector lives -Vector Space. For example, when we are thinking about something like a Force vector, the context is usually 2D or 3D Euclidean world.

The easiest way to understand the Vector is in a geometric context, say 2D or 3D cartesian coordinates, and then extrapolate it for other Vector spaces which we encounter but cannot really imagine.

Vectors are represented as matrices. But a matrix is just a rectangular array of numbers and nothing else.

This matrix below is an example of a Euclidean Vector in three-dimensional Euclidean space (or $R^3$). So a vector is represented as a column matrix or a row matrix.

$$ a = \begin{bmatrix} a_{1}\\a_{2}\\a_{3}\ \end{bmatrix} = \begin{bmatrix} a_{1} & a_{2} &a_{3}\end{bmatrix} $$

Dot product has a specific meaning. Matrix multiplication has no specific meaning, than may be a mathematical way to solve system of linear equations Why, historically, do we multiply matrices as we do?

Coming back to dot product - Algebraically, the dot product is the sum of the products of the corresponding entries of the two sequences of numbers.

$ \text{If } \vec a = \left\langle {a_1,a_2,a_3} \right\rangle \; \text{and } \vec b = \left\langle {b_1,b_2,b_3} \right\rangle \; \text{then } \; \vec a \cdot \vec b = {a_1}{b_1} + {a_2}{b_2} + {a_3}{b_3}$

Which if we write in matrix form, we need to mathematically take the transpose of a vector and do 'matrix' multiplication to get the above dot product.

So coming back full circle to the question - matrix multiplication is a tool to find vector dot product (assuming we are talking about matrices in the context of vectors)

$$ a = \begin{bmatrix} a_{1}\\a_{2}\\a_{3}\ \end{bmatrix} b = \begin{bmatrix} b_{1}\\b_{2}\\b_{3}\ \end{bmatrix} a.b = ab^T=\begin{bmatrix} a_{1}b_{1} & a_{2}b_{2} &a_{3}b_{3}\end{bmatrix} $$

https://mathinsight.org/dot_product_matrix_notation

Note

Geometrically dot product of two Euclidean vectors are the Euclidean magnitudes of the two vectors and the cosine of the angle between them

$\vec a \cdot \vec b = \left\| {\vec a} \right\|\,\,\left\| {\vec b} \right\|\cos \theta$

This is simple to visualize in Euclidean space but is true for all dimensions. If two vectors are in the same direction the dot product is positive and if they are in the opposite direction the dot product is negative. This can be visualized geometrically putting in the value of the Cosine angle.

So we could use the dot product as a way to find out if two vectors are aligned or not; which tends itself to more interesting uses

Answer 5

<Matrix multiplication(product)>

• Matrix multiplication is the multiplication of 2D or more D tensors(arrays). *Either of 2 operands can be a 1D array(tensor) but not both of them.
• The rule which you must follow to do matrix multiplication is the number of the columns of A tensor(array) must match the number of the rows of B tensor(array).

2D arrays:

    <A>            <B>
[[a, b, c], x [[g, h, i, j], = [[ag+bk+co, ah+bl+cp, ai+bm+cq, aj+bn+cr],
 [d, e, f]]    [k, l, m, n],    [dg+ek+fo, dh+el+fp, di+em+fq, dj+en+fr]]
               [o, p, q, r]]
2 rows         (3) rows
(3) columns    4 columns
[[2, 7, 4], x [[5, 0, 8, 6], = [[35, 58, 59, 69],
 [6, 3, 5]]    [3, 6, 1, 7],    [44, 38, 96, 67]]

               [1, 4, 9, 2]]   [[2x5+7x3+4x1, 2x0+7x6+4x4, 2x8+7x1+4x9, 2x6+7x7+4x2]
                                [6x5+3x3+5x1, 6x0+3x6+5x4, 6x8+3x1+5x9, 6x6+3x7+5x2]]

In PyTorch with @, matmul() or mm():

import torch
tensor1 = torch.tensor([[2, 7, 4], [6, 3, 5]])
tensor2 = torch.tensor([[5, 0, 8, 6], [3, 6, 1, 7], [1, 4, 9, 2]])
tensor1 @ tensor2  # tensor([[35, 58, 59, 69], [44, 38, 96, 67]])
torch.matmul(tensor1, tensor2) # tensor([[35, 58, 59, 69], [44, 38, 96, 67]])
torch.mm(tensor1, tensor2) # tensor([[35, 58, 59, 69], [44, 38, 96, 67]])

*Memos:

• @ or matmul() can multiply 1D or more D tensors(arrays) by dot or matrix multiplication.
• mm() can multiply 2D tensors(arrays) by matrix multiplication.

A 1D and 3D array. *B 3D tensor(array) has 3 2D tensors(arrays) which have 2 rows and 4 columns each:

  <A>             <B>
[a, b]   x   [[[c, d, e, f], = [[(ac+bg), (ad+bh), (ae+bi), (af+bj)],
               [g, h, i, j]],   [(ak+bo), (al+bp), (am+bq), (an+br)],
              [[k, l, m, n],    [(as+bw), (at+bx), (au+by), (av+bz)]]
               [o, p, q, r]],
              [[s, t, u, v],
               [w, x, y, z]]]
1 row         (2) rows
(2) columns   4 columns
[2, 7]   x   [[[6, 3, 5, 2], = [[47,  6, 66, 32],
               [5, 0, 8, 4]],   [20, 68, 65, 35],
              [[3, 6, 1, 0],    [42, 39, 21, 69]]
               [2, 8, 9, 5]],  [[2x6+7x5, 2x3+7x0, 2x5+7x8, 2x2+7x4],
              [[7, 2, 0, 3],    [2x3+7x2, 2x6+7x8, 2x1+7x9, 2x0+7x5],
               [4, 5, 3, 9]]]   [2x7+7x4, 2x2+7x5, 2x0+7x3, 2x3+7x9]]

In PyTorch with @ or matmul():

import torch
tensor1 = torch.tensor([2, 7])
tensor2 = torch.tensor([[[6, 3, 5, 2], [5, 0, 8, 4]],
                        [[3, 6, 1, 0], [2, 8, 9, 5]],
                        [[7, 2, 0, 3], [4, 5, 3, 9]]])
tensor1 @ tensor2
tensor([[47, 6, 66, 32], [20, 68, 65, 35], [42, 39, 21, 69]])
torch.matmul(tensor1, tensor2)
tensor([[47, 6, 66, 32], [20, 68, 65, 35], [42, 39, 21, 69]])

<Dot multiplication(product)>

• Dot multiplication is the multiplication of 1D tensors(arrays).
• The rule which you must follow to do dot multiplication is the number of the rows of A and B tensor(array) must be 1 and the number of the columns must be the same.

   <A>         <B>
[a, b, c] x [d, e, f] = ad+be+cf
1 row       1 row
3 columns   3 columns
[2, 7, 4] x [6, 3, 5] = 53
                 (2x6)+(7x3)+(4x5)
  [2, 7, 4]
   x  x  x
  [6, 3, 5]
      ||
 [12, 21, 20]
 12 + 21 + 20
      ||
      53

In PyTorch with @, dot() or matmul(). *dot() can multiply 1D tensors(arrays) by dot multiplication:

import torch
tensor1 = torch.tensor([2, 7, 4])
tensor2 = torch.tensor([6, 3, 5])
tensor1 @ tensor2 # tensor(53)
torch.dot(tensor1, tensor2) # tensor(53)
torch.matmul(tensor1, tensor2) # tensor(53)