Here is for the proof of Independence of Generalized Eigenvectors in real world...which means, there may exist repeated Eigenvalues, such as multiple 0's, different Jordan block sizes associated with one Eigenvalue...
The proof comprises 3 parts:
- Basic definitions
- Reduce the problem into any one Eigenspace
- Further reduce the problem into the largest Jordan block(s)
Part I
To satisfy linear independence of Generalized EigenVectors(GEVs), we claim:
$$\sum\limits_{i=1}^{m}\sum\limits_{j=1}^{\gamma(\lambda_i)}\sum\limits_{k=1}^{\rho(\lambda_i)^j}a_{ijk}\,v_{ijk} = 0\tag{1}\label{1}$$
iff $a_{ijk}\equiv0$. The meaning of above symbols are as following:
- $m$ is the number of distinct Eigenvalues, $\lambda_1,\lambda_2,...\lambda_m$, of $n\times n$ matrix. There are $n$ GEVs by Jordan Normal Form. Here $m\le n$
- $\gamma(\lambda_i)$ is Geometric Multiplicity of EigenSpace $E_{\lambda_{i}}$, which is the number of Jordan Blocks for $\lambda_i$.
- $\rho(\lambda_i)^j$ is the sizes/dimensions for each Jordan Block of $\lambda_i$. The largest is $\rho(\lambda_i)^{max}$.
- $v_{ijk}$ is a GEV of $\lambda_i$ in bock $j$, $a_{ijk}$ is coefficient.
- $\mu(\lambda_i)=\sum\limits_{j=1}^{\gamma(\lambda_i)}\rho(\lambda_i)^j$, Algebraic Multiplicity of $\lambda_i$; and $\sum\limits_{i=1}^m\mu(\lambda_i)=n$
Now that dirty work done, begin proof!
Part II
As defined above $\rho(\lambda_i)^{max}$ is the largest Jordan Block associated with $\lambda_i$,
hence $(\lambda_{i}−)^{\rho(\lambda_i)^{max}}\;v_{ijk}= 0$, i.e., all GEVs $v_{ijk}\!\in \!ker((_{i}−)^{\rho(\lambda_i)^{max}})$.
We then construct an instrumental product $\prod\limits_{\ell = 1}^m(_{\ell}−)^{\rho_{\ell}^{max}\;\mathbb{1}\!(\ell\ne\zeta)}$, where $\mathbb{1}\!(\ell\ne\zeta)$ is an Indicator Function of $\ell\!\ne\!\zeta$, and multiply to both sides of above equation (1): $$\sum\limits_{i=1}^{m}\sum\limits_{j=1}^{\gamma(\lambda_i)}\sum\limits_{k=1}^{\rho(\lambda_i)^j}a_{ijk}\left(\prod\limits_{\ell = 1}^m(_{\ell}−)^{\rho_{\ell}^{max}\;\mathbb{1}\!(\ell\ne\zeta)}\right)v_{ijk} = 0 \tag{2}\label{2}$$ $$1 \le\zeta\le m$$ Observe lefthand side of equation (2), any summand $a_{i}\!\!\left(\prod\limits_{\ell = 1}^m(_{\ell}−)^{\rho_{\ell}^{max}\;\mathbb{1}\!(\ell\ne\zeta)}\;v_{i}\right)$ will vanish when $i\!\in\!\ell$, thus only one indexed $\zeta$ survives: $$\sum\limits_{j=1}^{\gamma(\lambda_{\zeta})}\sum\limits_{k=1}^{\rho(\lambda_{\zeta})^j}a_{\zeta jk}\left(\prod\limits_{\ell = 1}^m(_{\ell}−)^{\rho_{\ell}^{max}\;\mathbb{1}\!(\ell\ne\zeta)}\right)v_{\zeta jk} = 0 \tag{3}\label{3}$$ Considering $\color{blue}{the \;Lemma}$ proved in the question that GEV can not be shared with different Eigenspaces, which means:
$$\left(\prod\limits_{\ell = 1}^m(_{\ell}−)^{\rho_{\ell}^{max}\;\mathbb{1}\!(\ell\ne\zeta)}\right)v_{\zeta jk} \neq 0$$
and that $\zeta$ is an arbitrary index of Eigenvalues, thus our task is reduced to a possibily simple problem:
- prove GEVs associated with the same eigenvalue are independent
Turns out NOT so simple! There is one more thing!
Part III
For now we focus on Eigenspace $E_{\lambda_{\zeta}}$. We leave out the index $\zeta$, drop the instrumental product term, and rewrite the loop Sum (3) as below in a clear form:
$$\sum\limits_{j=1}^{\gamma(\lambda)}\sum\limits_{k=1}^{\rho(\lambda)^j}a_{jk}\,v_{jk} = 0\tag{4}\label{4}$$
With Jordan Normal Form of matrix $A$, we have:
$$(\lambda I - A)^{\rho(\lambda)^{max}} v = 0 $$
$v$ is any GEV of Eiegnspace $E(\lambda)$, and
$$(\lambda I -A) v_{jk} = v_{j(k-1)}$$
$$v_{j0} = 0$$
$(\lambda I - A)$ is acting like an annihilation operator on GEV.
- Let's break for a second and take a quiz to identify $\gamma(\lambda), \rho(\lambda)^{max}$ for Jordan blocks below, which are associated with the same eigenvalue $\lambda$ from a big matrix with Jordan Normal Form.
$$
\begin{pmatrix}
\lambda \\
& \lambda & 1\\
& & \lambda\\
& & &\lambda & 1\\
& & & &\lambda
\end{pmatrix}
$$
Here comes the last straw!
Multiply $(\lambda I -A)$ to both sides of the equation (4) above, and keep doing that to the reduced iteratively. At the end of day, we will either find only one term with an ordinary Eigenvector, which we can safely set its coefficient 0, or come across multiple terms of ordinary Eigenvectors in different blocks, one for each block with size $\rho(\lambda)^{max}$, which are built as linear independent (thinking about a special case, identity matrix $I$).
- Finally, the process show GEVs in different Jordan blocks are independent.
Q.E.D.
We have no idea about $\color{red}{Minimal\; Polynomial,\; invariant\; space}$ in this proof.
