7

Prove that, without using induction, A real symmetric matrix $A$ can be decomposed as $A = Q^T \Lambda Q$, where $Q$ is an orthogonal matrix and $\Lambda$ is a diagonal matrix with eigenvalues of $A$ as its diagonal elements.

I can see that all eigenvalues of $A$ are real, and the corresponding eigenvectors are orthogonal, but I failed to see that when putting all (interesting) eigenvectors together, they form a basis of $\mathbb{R}^n$.

Edit

The reason I asked this question is to show that a real symmetric matrix is diagonalizable, so let's not use that fact for a while. Other than that, any undergraduate level linear algebra can be used.

Edit 2

After reading Algebraic Pavel's answer, I feel like ruling out Schur Decomposition as well, but I can't keep ruling out theorems, so...if a proof is too obvious, that's probably not what I am looking for, though, it maybe a technically correct answer.

Thanks.

Not an ID
  • 877
  • 2
    If you note that all the eigenvectors are orthonormal, and that there exist $n$ eigenvectors, then their span must be $\mathbb{R}^{n}$ as they are linearly independent. – Thomas Russell Jul 23 '14 at 09:59
  • @Shaktal Suppose that ${\lambda}1, ..., {\lambda}_t$ are its eigenvalues, and that $X{11}...X_{1k_1}, ... , X_{t1}...X_{tk_t}$ are the corresponding eigenvectors, is it obvious that $k_1 + ... + k_t = n$? – Not an ID Jul 23 '14 at 10:05
  • what does it means "without using induction"? how do you prove that any tow basis of a vector space have the same cardinality? I mean, ok, you don't want to use explicitly induction, so: what are the therorems you are alloed to use? – user126154 Jul 23 '14 at 10:44
  • @user126154 Don't use induction on the order of the matrix. (Or better, don't explicitly use it anywhere in the proof) Any "undergraduate-level" linear algebra can be used. – Not an ID Jul 23 '14 at 10:50
  • well, can you use that any subspace $V$ of $\mathbb R^n$ has an orthonormal basis? – user126154 Jul 23 '14 at 11:14
  • @user126154 Of course – Not an ID Jul 23 '14 at 11:16
  • I mean, showing the existence of $Q$ is equivalent, by defintion, to show that there is an orthonormal basis of eigenvectors, eigenvectors realtive to different eigenvalues are orthogonals, so it remains to show that any eigenspace has an orthogonal basis. For that you have the orhtogonalization process, which basically uses induction. – user126154 Jul 23 '14 at 11:18
  • @Shaktal I guess that the existence of $n$ eigenvectors is what is supposed to be proved. Hence it is certainly not allowed to assume that. – Algebraic Pavel Jul 23 '14 at 11:45
  • @AlgebraicPavel basically what I want to prove, at the first glance of the proposition, is that the multiplicity of a eigenvalue is equal to the dimension of the corresponding eigenspace. In case that someone has another way to prove it, I didn't ask for this(the multiplicity thing). Your comment is technically a correct answer, but not something in my mind. – Not an ID Jul 23 '14 at 11:53
  • @NotanID It might be a bit unusual but there are very simple arguments for showing this using tridiagonal matrices :) Updated the answer. It is actually surprisingly simple and one needs only two ingredients to show the fact. I can provide more details if the explanation would not be clear. – Algebraic Pavel Jul 23 '14 at 12:11

2 Answers2

6

Provided that the Schur decomposition is an allowed tool:

Using the Schur decomposition, we have that there exists an orthogonal $Q$ and an upper triangular $R$ such that $A=QRQ^T$. Since $A$ is symmetric, $Q^TAQ=R$ is symmetric as well. Therefore $R$ is symmetric. A symmetric triangular matrix is necessarily diagonal.

There is also a neat theory behind tridiagonal matrices, which can help:

It is easy to show that for any real $A$ there is an orthogonal matrix $Q$ such that $Q^TAQ=H$, where $H$ is upper Hessenberg. If $A$ is symmetric it then follows that $H$ is symmetric as well and hence tridiagonal. Now if the tridiagonal matrix $H$ is unreduced (none of the upper and lower diagonal entries are zero), then the eigenvalues of $H$ (and therefore of $A$) are distinct. Equivalently, if $A$ has repeated eigenvalues then $H$ is reduced (some upper and lower (symmetrically) diagonal entries are zero and hence $H$ is a block diagonal matrix). Consequently, one must have that each repeated eigenvalue must be in different unreduced diagonal blocks of $H$. In each of these blocks we must find at least one eigenvector (actually, only one) and it is almost trivial to show that these eigenvectors must be linearly independent (therefore, we can orthogonalize them).

Algebraic Pavel
  • 22,928
4

A matrix $Q$ is orthogonal if and only if its columns forms a orthonormal basis, if and only if $Q^{-1}=Q^T$.

Therefore, if there exists an orthornomal basis of eigenvectors of $A$, we have that the matrix of change of basis if ortogonal. That is to say, there is $Q$ orthogonal so that

$Q^{-1}AQ=\Lambda$

But then $A=Q^{-1}\Lambda Q=Q^T\Lambda Q$

It remains to show that an orthonormal basis of eigenvectors exists.

Eigenspaces corresponding to different eigenvalues are orthogonal. As A is diagonalizable we have $\mathbb R^n=V_1\oplus V_2\dots\oplus V_k$ where $V_i$ is the eigenspace corresonding to the eignevalue $\lambda_i$.

Each $V_i$ has a orthonormal basis $v_{i,1},\dots,v_{i,n_i}$ where $n_i=\dim V_i$. (from the comments we know that we can use this fact.)

Thus, putting toghether these basis we got an orthonormal basis $v_{i,j}$ of $\mathbb R^n$ consisting of eigenvectors of $A$.

Edit If you cannot use that $A$ is diagonalizable, then use the following workaround to shot that in fact $A$ is diagonalizable.

Write $\mathbb R^n=V_1\oplus\dots\oplus V_k\oplus W$. Where the $V_i$ are the eigenspaces of $A$ and $W$ is the orthogonal complement of $V_1\oplus\dots\oplus V_k$.

Then, the resctriction of $A$ to $W$ is symmetric and has no eigenvectors. Now, consider the function

$\max \langle Aw,w\rangle$ when $w$ runs on the spaces of unitary vectors of $W$ (that is to say, $||w||=1$).

Any max point is an eigenvector, contradicting the fact that $A|_W$ has no eigen vectors. (See below.) Therefore $W$ contains no unitary vectors, hence $W=\{0\}$ and $V_1\oplus\dots\oplus V_k=\mathbb R^n$.

Let's see that max points are eigenvectore. We restrict to $W$ a and Let $F(w)=\langle Aw,w\rangle$ then the derivative of $F$ at point $w$, in the direction $v$ is $dF_w[v]=2\langle Aw,v\rangle$ (because $A$ is symmetric). The tangent space of $\{||w||=1\}$ at the point $w$ is $w^\perp$. Thus, $w$ is a critical value if and only if $dF_w[v]=0\forall v\in w^\perp$. That is to say, $\langle Aw,v\rangle=0$ for all $v$ such that $\langle w,v\rangle=0$. Therefore $Aw$ must be a multiple of $w$ as the orthogonal ov $w$ is contained in the orthogonal of $Aw$.

Edit another way to see that $A|_W$ has an eigenvector is to consider its characteristic polynomial. It has roots in $\mathbb C$ because of the fundamental theorem of algebra. They are reals because $A|_W$ is symmetric. So $A|_W$ has at least one eigenvalue, whence eigenvector.

user126154
  • 7,551
  • Thanks for the answer, but the reason I asked this question is to show that a real symmetric matrix is diagonalizable, sorry I didn't mention it in the question, so using the fact that it's diagonalizable is not acceptable. – Not an ID Jul 23 '14 at 11:32
  • I see... ha ha ha – user126154 Jul 23 '14 at 11:54
  • I edited the answer, but now I make uses of derivatives, may I? – user126154 Jul 23 '14 at 12:02
  • Why the max point of the function is an eigenvector? In the second edit, you mean proof by contradiction, right? I mean, if $W$ is not zero, $A$ would have a eigenvector in it, but $A$ doesn't, so $W$ must be zero, right? I like both of your edits. – Not an ID Jul 23 '14 at 12:39
  • sorry I confused $||Aw||$ with $\langle Aw,w\rangle$ I edited the answer – user126154 Jul 23 '14 at 12:49
  • by the way, even if you use $||Aw||$ you'll reach your scope ($\langle Aw,Aw\rangle=\langle A^2w,w\rangle$...) – user126154 Jul 23 '14 at 12:59
  • Your argument that $A|_W$ has an eigenvector (in the second edit) only works if you know that $W$ is $A$-invariant, or in other words that the restriction of $A$ to $W$ actually maps $W$ to $W$ itself. This is true, but is something you should explicitly mention. – Santiago Canez Jul 23 '14 at 13:11
  • $W$ is $A$ invariant because $W$ is defined as the orthogonal of the sum $V$ of eignespaces $V_i$. $A$ is symmetric so $\langle Aw,v\rangle=\langle w,Av\rangle$ so if $\langle w,v\rangle =0$ for all $v\in V$ then $\langle Aw,v\rangle=\langle w,Av\rangle=0$ for all $v\in V$. So if $w\in W$ then $Aw\in W$. That is to say $W$ is $A$-invariant – user126154 Jul 23 '14 at 13:40