INTUITION: Orthonormal columns implies orthonormal rows

Question

I was asked by a student and was stumped upon the very intuition of this statement. I searched on this site and found many proofs but little intuition. For example: Column Vectors orthogonal implies Row Vectors also orthogonal? and Intuition behind row vectors of orthonormal matrix being an orthonormal basis I am open to abstract linear algebra ideas but I don't think they help bring in much intuition. I am hoping for either a geometric intuition, a quick algebraic manipulation on vector components, or an intuitive explanation of the key step in the proof: $A^TA = AA^T = I$.

Edit: many comments went for the last option of the three. However, it was probably the hardest to gain any intuition with this option. I would personally prefer answers exploring the first two options, or something really really special about this last option.

Answer 1

Invert both sides of $I=AA^\top$, to $I=(A^\top)^{-1}A^{-1}$. Multiply both sides on the left by $A^\top$, and on the right by $A$, to obtain $A^\top A=I$.

Answer 2

Let $A$ have rows $\mathbf r_1^{\mathsf T}, \dots, \mathbf r_n^{\mathsf T}$ and columns $\mathbf c_1, \dots, \mathbf c_n$.

Suppose that $\mathbf r_1, \dots, \mathbf r_n$ are orthonormal. This means that $A \mathbf r_i = \mathbf e_i$, the $i^{\text{th}}$ standard basis vector. In particular, $A$ takes the orthonormal basis $\{\mathbf r_1, \dots, \mathbf r_n\}$ to the orthonormal basis $\{\mathbf e_1, \dots, \mathbf e_n\}$.

We can check that this means that $A$ preserves inner products in the sense that $\langle \mathbf x, \mathbf y\rangle = \langle A \mathbf x, A \mathbf y\rangle$; geometrically, this means that $A$ preserves angles and distances.

This, in turn, means that $A$ takes any orthonormal basis to another orthonormal basis, because the statement "$\{\mathbf q_1, \dots, \mathbf q_n\}$ is an orthonormal basis" is just a statement about the all inner products $\langle \mathbf q_i, \mathbf q_j\rangle$, and those are preserved by $A$.

In particular, $A$ will take the orthonormal basis $\{\mathbf e_1, \dots, \mathbf e_n\}$ to another orthonormal basis. But $A\mathbf e_i = \mathbf c_i$, so this tells us that the columns of $A$ are orthonormal.

(Technically, I just explained why orthonormal rows imply orthonormal columns, but you can go from columns to rows in the same way - you'd just have to either reason about row vectors the whole time, or talk about $A^{\mathsf T}$ instead of $A$.)

Answer 3

The left inverse of a square matrix $A$ is always a right inverse as well. This is a Linear Algebra fact that is not true for infinite-dimensional spaces.

For a finite-dimensional space $X$ and a linear $A: X \rightarrow X$, there is a minimal polynomial $m$ such that $m(A)=0$. The minimal polynomial can be normalized as a monomial $$ m(\lambda)=\lambda^m+a_{m-1}\lambda^{m-1}+\cdots+a_1\lambda+a_0. $$ The coefficient $a_0$ cannot be $0$ because that would give $$ A(A^{m-1}+a_{m-1}A^{m-2}+\cdots+a_1 I)=0, $$ which would either contradict $\mathcal{N}(A)=\{0\}$ or the minimality of the polynomial $m$. So, $a_0\ne 0$ for an invertible matrix $A$. So the minimal polynomial $m$ can always be normalized so that $a_0=1$. That gives an explicit left- and right- inverse for $A$ (and both are the same): \begin{align} I=(-A^{-m}-a_{m-1}A^{m-2}-\cdots-a_1I)A \\ =A(-A^{-m}-a_{m-1}A^{m-2}-\cdots-a_1I). \end{align}

A square matrix over a field $\mathrm{F}$ has a left inverse iff it has a right inverse and, in that case, the two inverses are the same. This is a consequence of working in a finite-dimensional setting.

An immediate consequence of this: If $A$ is an $n\times n$ matrix whose column vectors form an orthonormal basis, then $A^{T}A=I$, which then forces $AA^{T}=I$ and, hence, implies that the row vectors of $A$ also form an orthonormal basis.