6

I'm given the matrix $X = xx^{T}\in\mathbb{R}^{d \ x \ d}, x\in\mathbb{R}^{d}$. Does somebody know how to compute $\lambda_{max}(X)$ or $\lambda_{min}(X)$? I only want to know these two eigenvalues, the others are not really important.

I seem to be stuck.

I'm thankful for any answer.

Michael W.
  • 75

3 Answers3

4

Assuming $x$ is non-zero, $X$ has rank $1$, so almost all the eigenvalues are $0$. As for the single non-zero eigenvalue, consider what $Xx$ becomes.

Arthur
  • 199,419
  • 2
    Another approach: the sum of the eigenvalues is $$ \operatorname{tr}(xx^T) = \operatorname{tr}(x^Tx) = x^Tx $$ – Ben Grossmann Aug 13 '19 at 15:50
  • 1
    I would say that $\lambda = x^{T}x$ is an eigenvalue of X, because $Xx = (xx^{T})x = x(x^{T}x) = (x^{T}x)x$. So this means $\lambda_{min}(X) = \lambda_{max}(X) = x^{T}x$. – Michael W. Aug 13 '19 at 16:04
  • @MichaelW. I would say that $\lambda_{\min}(X)=0$, but otherwise, yes, that is exactly what I'm hinting at. – Arthur Aug 13 '19 at 16:08
  • Yes sry. One follow up question. If I have a scalar before the matrix, meaning $\alpha{X} = \alpha{xx^{T}}$, then we would have $\lambda_{min}(\alpha{X}) = 0$ and $\lambda_{max}(\alpha{X}) = \alpha{x^{T}x}$? – Michael W. Aug 13 '19 at 16:16
  • @MichaelW. The eigenvector hasn't changed, the calculation stays the same, and you end up with $(\alpha X)x=(\alpha x^Tx)x$. So yes. – Arthur Aug 13 '19 at 16:26
3

We assume

$x \ne 0, \tag 1$

lest

$X = xx^T = 0, \tag 2$

and the problem is trivial. For

$x \ne 0, \tag 3$

we have

$Xx = (xx^T)x = x(x^Tx) = (x^Tx)x, \tag 4$

and we see that

$x^Tx > 0 \tag 5$

is an eigenvalue of $X = xx^T$ with associated eigenvector $x$.

Now if

$0 \ne y \in \Bbb R^d \tag 6$

is such that

$x^Ty = 0, \tag 7$

then

$Xy = (xx^T)y = x(x^Ty) = (0)y = 0, \tag 8$

i.e. $0$ is an eigenvalue of $X$ with eigenvector $y$. The mapping

$x^T(\cdot): \Bbb R^d \to \Bbb R, \; y \to x^Ty \tag 9$

is a linear functional on $\Bbb R^d$ and as such

$\dim \ker x^T(\cdot) = d - 1; \tag{10}$

thus the $0$-eigenspace of $X$, which is $\ker x^T(\cdot)$, is of dimension $d - 1$. Having exhausted the number of available dimensions of $\Bbb R^d$, we conclude that $x^Tx$ is an eigenvalue of multiplicity $1$, whilst the eigenvalue $0$ is of multiplicity $d - 1$; there are no other eigenvalues or eigenvectors of $X$.

Robert Lewis
  • 71,180
0

I can't add a comment to @Arthurs's answer because I don't have enough reputation, but I wanted to add my 2 cents anyway :)

Say you had another vector $\mathbf{y}$, with the same dimensions as $\mathbf{x}$. In that case $\mathbf{x^{T}y}=y_{proj_{x}}$, where $y_{proj_{x}}$ is the dot product between $\mathbf{x}$ and $\mathbf{y}$, or in other words the modulus of the projection of $\mathbf{y}$ on $\mathbf{x}$.

Similarly, the outer-product of $\mathbf{x}$ gives you your $\mathbf{X}$ matrix: $\mathbf{X}=\mathbf{xx^{T}}$. If you take a moment to look at it, $\mathbf{Xy}=\mathbf{xx^{T}y}=\mathbf{x(x^{T}y)}=\mathbf{x}y_{proj_{x}}$

Which is the projection of $\mathbf{y}$ on $\mathbf{x}$, so through this intuition you can say that the only eigenvector is the projection direction $\mathbf{x}$

Edit: Just realized that in order for it to be a projection, $\mathbf{x}$ should be a unit vector, otherwise it scales the projection by $\left\|\mathbf{x}\right\|^2$, so that would be the eigenvalue: $\left\|\mathbf{x}\right\|^2$

imochoa
  • 101