5

I want to show that $X^TX + \lambda \begin{pmatrix} 0 & 0 & 0 & 0 & \dots & 0 \\ 0 & 1 & 0 & 0 & \dots & 0\\ 0 & 0 & 1 & 0 & \dots & 0 \\ 0 & 0 & 0 & 1 & \dots & 0 \\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & 0 & 0 & \dots & 1\end{pmatrix} $ is invertible if $\lambda >0$

I saw it in an online course but the professor gave no explanation.

$X$ is an $m$ by $n+1$ matrix, and so the matrix $\lambda$ multiplies is $n+1$ by $n+1$, and I don't know if its relevant to the question, but we can safely assume that the first column of $X$ is all ones, but that may not be needed.

how can i prove this? there is no attribute of the determinant of the sum of matrices, this matrix is indeed symmetric so it might be positive definite, but I don't think we can say much about the eigenvalues...

Oria Gruber
  • 12,739

1 Answers1

7

Note that your matrix $M$ is the sum of (symmetric) positive semidefinite matrices, so it is itself positive semidefinite. Moreover, note that since $M$ is a positive semidefinite matrix, it is invertible if and only if it is positive definite. That is, it suffices to show that $v^TMv > 0$ whenever $v$ is a non-zero vector.

Now, note that for any vector $v$, we have $$ v^TMv = \\ v^T(X^TX + \lambda A)v = \\ v^TX^TXv + \lambda v^TAv =\\ \|Xv\|^2 + \lambda v^TAv $$ So, it suffices to show that if $\|Xv\| = 0$ and $v^TAv = 0$, then $v = 0$. However, $v^TAv = 0$ if and only if $v$ is a multiple of $e_1 = (1,0,\dots,0)^T$. And, if $v$ is a non-zero multiple of $e_1$ (that is, if $v = \alpha e_1$ for some $\alpha \neq 0$), then we know from the first column of $X$ that $Xv$ (which is $\alpha X e_1 = \alpha(1,\dots,1)^T$) can't be zero.

The conclusion follows.

Ben Grossmann
  • 225,327