2

In Deep Learning (page 44) it is stated that:

Specifically, every real symmetric matrix can be decomposed into an expression using only real-valued eigenvectors and eigenvalues: $$\mathbf A = \mathbf {Q Λ Q^T}$$ where $\mathbf Q$ is an orthogonal matrix composed of eigenvectors of $\mathbf A$, and $\mathbf Λ$ is a diagonal matrix. The eigenvalue $Λ_{i,i}$ is associated with the eigenvector in column $i$ of $\mathbf Q$, denoted as $\mathbf Q_{:,i}$. Because $\mathbf Q$ is an orthogonal matrix, we can think of $\mathbf A$ as scaling space by $λ_i$ in direction $\mathbf v^{(i)}$.

What does it mean that matrix $A$ is scaling space by $\lambda_i$ in direction $\mathbf v^{(i)}$?

gc5
  • 537
  • 2
    The matrix $A=\pmatrix{2&0\ 0&3}$, or the function $f(x,y)=(2x,3y)$, are scaling space in the direction of the $x$-axis by a factor of $2$, and scaling space in the direction of the $y$-axis by a factor of $3$. If you can understand this, you can understand the quoted passage. – user1551 Mar 28 '17 at 16:28

1 Answers1

2

One way to motivate the notion of eigenvectors and eigenvalues of a matrix $\mathbf A$ is to ask whether there are any lines that are left fixed by $\mathbf A$. If $\mathbf v$ is any vector on such a line, linearity requires that we have $\mathbf{Av}=\lambda\mathbf v$, where $\lambda$ is some fixed scalar. This equation is the defining equation for eigenvectors and eigenvalues: a non-zero vector that satisfies this equation is called an eigenvector (characteristic vector) of $\mathbf A$ and the corresponding scalar $\lambda$ is an eigenvalue (characteristic value). Geometrically, this equation can be interpreted as saying that in the direction defined by $\mathbf v$, the action of $\mathbf A$ is particularly simple: it’s just scaling by $\lambda$.

amd
  • 53,693