What's the proof stategy for: Hermitian matrix has orthogonal eigenvectors for distinct eigenvalues?

Question

Linear Algebra (2015 5 ed) by Lay, p. 397. Theorem 7.1 involves only real numbers.

Let:

1. $A^* = \bar{A}^T $.

2. $v_i$ and $v_j$ be two eigenvectors of an Hermitian matrix H. Suppose that their respective eigenvalues i and j are different, i.e. $\lambda_i \neq \lambda_j$. This means $Hv_i = \lambda_iv_i$ and $Hv_j > = \lambda_jv_j \quad (3)$.

Take the Hermitian conjugate of $Hv_i = \lambda_iv_i$:

$\begin{align} (Hv_i)^* & = (\lambda_iv_i)^* \\ \implies v_i^* H^* & = > v_i^* \lambda_i^* \\ v_i^* H & = \qquad \qquad \text{ > because H* = H } \\ & = v_i^* \lambda_i \qquad \text{ because > http://math.stackexchange.com/q/462982/53259 } \end{align}$

Right-multiply the previous equation by $\color{green}{v_j}: \qquad > v_i^*H \color{green}{v_j} = v_i^* \lambda_i \color{green}{v_j} \qquad > (4)$

Left-multiply (3) by $\color{orangered}{v_i^*} : \quad > \color{orangered}{v_i^*}Hv_j = \color{orangered}{v_i^*} \lambda_jv_j > \qquad (5)$.

Equate the RHS of (4) and (5): $\quad v_i^* \lambda_i > \color{green}{v_j} = \color{orangered}{v_i^*} \lambda_jv_j \qquad > \blacksquare$.

I understand, and ask not about, the algebra. What's the proof strategy? For example, how can you divine when to take the Hermitian conjugate, what to multiply, and when to left- or right-multiply?

Answer 1

Let's write the problem in a better way. You have a Hermitian matrix $A$, two distinct eigenvalues $\lambda$ and $\mu$ and two eigenvectors $v\ne0$, $w\ne0$ such that $$ Av=\lambda v,\qquad Aw=\mu w. $$ In this first proof, the strategy is to work backwards. You want to prove that $v^*w=0$. So start with the assumption that $\lambda-\mu\ne0$; so $v^*w=0$ if and only if $(\lambda-\mu)v^*w=0$, that is $$ (\lambda v^*)w=\mu v^*w $$ On the LHS, but $Av=\lambda v$, so $\lambda v^*=v^*A^*$ (because $\lambda$ is real).
On the RHS, $\mu w=Aw$, so you're bound to prove that $$ (v^*A^*)w=v^*Aw $$ which immediately follows from $A^*=A$.

A different proof, but using the same techniques in a different order: start from $v^*Aw$ that you can rewrite as $$ v^*(Aw)=v^*(\mu w)=\mu (v^*w) $$ but also as $$ v^*A^*w=(Av)^*w=(\lambda v)^*w=\lambda(v^*w) $$ thus $$ \lambda(v^*w)=\mu (v^*w) $$ which can happen only if $\lambda=\mu$ or $v^*w=0$.

Remark on notation

With $v^*w$ I mean the product of the conjugate transpose (H-transpose, briefly) of $v$ by $w$, which is a $1\times 1$ matrix, which is the same as a number. The inner product we use here is exactly $v^*w$.

It's a question of conventions; $v^*w=0 \iff w^*v=0 \iff \color{orangered}{ ( } w^*v \color{orangered}{ )^T} =0 \iff v^T \bar{w}=0 $,
so orthogonality is the same with all three definitions.

Remark on the first proof

The assumption that $\lambda\ne\mu$ is clearly necessary, because there's no way to prove that two eigenvectors relative to the same eigenvalue are orthogonal.

Thus, if we want to prove $v^*w=0$, we can as well prove that $(\lambda-\mu)v^*w=0$, which allows to distribute over the subtraction.

Remark on the second proof

Here we use the other hypothesis, that is $A=A^*$. But this is not a necessary assumption, because the theorem is true as soon as $A$ is a normal matrix, that is $AA^*=A^*A$. We want a “skew-symmetric” expression and $v^*Aw=v^*A^*w$ is surely a first shot.