7

I find it non-intuitive if I impose that all of a square matrix's columns are normalized and mutually orthogonal, then all its rows are also normalized and mutually orthogonal. Any intuitive explanation for this? Also if I relax the conditions to be only mutually orthogonal without being normalized, is this still true? And why so?

Sam
  • 461
  • 1
    What? I think there is the word "rows" missing a few times. – Jan May 26 '20 at 10:52
  • Sorry I have corrected the typo. – Sam May 29 '20 at 03:54
  • @ArturoMagidin I already corrected the title so perhaps u can uncomment already? – Sam Jun 30 '20 at 07:49
  • 5
    It appears unintuitive because it is unintuitive. The said property is a consequence of the fact that $AB=I$ iff $BA=I$ for square matrices $A$ and $B$. It relies on the finiteness of the dimension of the vector space and has little to do with inner products or geometry. In an infinite-dimensional inner product space, it can happen that $A^\ast A=I$ but $AA^\ast\ne I$ (e.g. when $A$ is the right-shift operator in $\ell^2$ and $A^\ast$ is the left shift). – user1551 Nov 17 '21 at 11:26
  • 2
    I'm doubling down on user1551's comment: It is critical that A have a finite size, and that A be square for the result to hold. – G. Fougeron Nov 18 '21 at 11:27

3 Answers3

2

I assume you mean orthonormal coulmn implies orthonormal rows. If so, than:

Let $Q$ be a square matrix with orthonormal columns. Therefore we know:

$$Q'Q=I \implies Q'=q^{-1}$$

$$(Q')'Q'=QQ'=I$$

Therefore, we have that the rows are also orthonormal.

Eminem
  • 651
  • 4
  • 12
  • Yes, when I write $Q'$ I mean transpose of matrix. – Eminem May 26 '20 at 11:12
  • Yea the math looks neat but is there some intuition? Also if the columns are not normalized is it still true? – Sam May 29 '20 at 03:52
  • The derivation above is incorrect. $Q'Q=I$ which follows $(Q'Q)'=I'$ $=>Q'Q=I$ which does not prove the point raised in question. To say that $Q'Q=I$ implies $(Q')'Q'=I$ is strictly incorrect. – Mohit Lamba Apr 06 '21 at 05:33
2

The derivation done by @eminem is mathematically incorrect. ′= does not imply (′)′′=. Below is a solution that could answer the question

$Q^\top Q=I => (Q^\top Q)^{-1}=I^{-1} => Q^{-1}Q^{\top -1}=I => QQ^{-1}Q^{\top -1}=Q => QQ^{-1}Q^{\top -1}Q^\top =QQ^\top => I=QQ^\top$

This means that orthonormal rows/columns imply orthonormal columns/rows. Of course, here I assume that $Q$ is a full rank square matrix. This conclusion is not always true for rectangle matrices.

Mohit Lamba
  • 121
  • 6
  • 1
    "mathematically incorrect" - I disagree. The logic in @eminem's proof is fine although it could have used a few words of explanation. $Q'Q = I$, therefore $Q'$ is a left inverse of $Q$. Since $Q$ is square, this implies that $Q'$ is the two-sided inverse of $Q$, and therefore $QQ' = I$. –  Apr 06 '21 at 05:55
  • @Bungo What you have written is absolutely correct. But (′)′′ written by eminem is misleading. – Mohit Lamba Apr 06 '21 at 06:00
  • I agree that it could have been stated more clearly. –  Apr 06 '21 at 06:01
2

For a matrix $Q$ with normalized orthogonal columns, $Q^TQ=I,$ then $Q^T=Q^{-1}$ and $QQ^T=Q Q^{-1}=I.$ $Q$ also has normalized orthogonal rows.

$A=\begin{bmatrix}1&0&2\\ 1&-1&-1\\ 1&1&-1\end{bmatrix}$ clearly is a matrix with orthogonal columns, but without orthogonal rows.

Assume $A^TA=D,$ $D$ is an invertible diagonal matrix. Let $AP=B,$ $P$ is a diagonal matrix which normalized columns. So $B$ is an orthogonal matrix. Let's see how $AP$'s rows are also normalized, i.e. $AP=B=QA,$ how $Q$ looks like?

$$PDP=P^TA^TAP=B^TB=I\Rightarrow P^2=D^{-1}\quad (\text{normalization, of course})$$ $$QAA^TQ^T=BB^T=I \Rightarrow Q^TQ=(AA^T)^{-1}$$

So $Q$ may not be a diagonal matrix. Otherwise $AA^T$ is also diagonal for sure, which contradicts our counter-example. This means when we normalize our column spaces, row basis may change during the operation.

I don't know what we can get further. I remember that when I study linear algebra, I don't find strong connection between row spaces and column spaces, especially geometrical meaning. I hope someone has learned matrix theory can answer this question deeply.

DreamAR
  • 986