41

If the column vectors of a matrix $A$ are all orthogonal and $A$ is a square matrix, can I say that the row vectors of matrix $A$ are also orthogonal to each other?

From the equation $Q \cdot Q^{T}=I$ if $Q$ is orthogonal and square matrix, it seems that this is true but I still find it hard to believe. I have a feeling that I may still be wrong because those column vectors that are perpendicular are vectors within the column space. Taking the rows vectors give a totally different direction from the column vectors in the row space and so how could they always happen to be perpendicular?

Thanks for any help.

xenon
  • 3,438
  • 3
    The equation $QQ^T = I$ only holds if the columns of $Q$ form an orthonormal set of vectors, not merely an orthogonal one. After all, some columns might be equal to $0$! – Arturo Magidin Jul 20 '11 at 18:38
  • Oh thanks for highlighting this! But still, if the columns are orthonormal and are unit vectors, why are its rows so coincidentally happen to be perpendicular too? If a column happens to be $0$, how it will no longer be an orthonormal set of vectors, will it? – xenon Jul 20 '11 at 18:43
  • 4
    If the columns form an orthonormal set, then $QQ^T=I$, so $Q^T = Q^{-1}$, so $Q^TQ=(Q^T)(Q^T)^T=I$, so the columns of $Q^T$ are also an orthonormal set, and these are the rows of $Q$; it's not "coincidental", it's forced on by the very strong condition that the columns must form an orthonormal basis. If a column is $0$, then the set of columns is no longer orthonormal because that column is not a unit vector. But it can be an orthogonal set, if all nonzero columns are pairwise orthogonal. See my full answer below for an example. – Arturo Magidin Jul 20 '11 at 18:49

5 Answers5

44

Recall that two vectors are orthogonal if and only if their inner product is zero. You are incorrect in asserting that if the columns of $Q$ are orthogonal to each other then $QQ^T = I$; this follows if the columns of $Q$ form an orthonormal set (basis for $\mathbb{R}^n$); orthogonality is not sufficient. Note that "$Q$ is an orthogonal matrix" is not equivalent to "the columns of $Q$ are pairwise orthogonal".

With that clarification, the answer is that if you only ask that the columns be pairwise orthogonal, then the rows need not be pairwise orthogonal. For example, take $$A = \left(\begin{array}{ccc}1& 0 & 0\\0& 0 & 1\\1 & 0 & 0\end{array}\right).$$ The columns are orthogonal to each other: the middle column is orthogonal to everything (being the zero vector), and the first and third columns are orthogonal. However, the rows are not orthogonal, since the first and third rows are equal and nonzero.

On the other hand, if you require that the columns of $Q$ be an orthonormal set (pairwise orthogonal, and the inner product of each column with itself equals $1$), then it does follow: precisely as you argue. That condition is equivalent to "the matrix is orthogonal", and since $I = Q^TQ = QQ^T$ and $(Q^T)^T = Q$, it follows that if $Q$ is orthogonal then so is $Q^T$, hence the columns of $Q^T$ (i.e., the rows of $Q$) form an orthonormal set as well.

Arturo Magidin
  • 398,050
  • Thank you so much Arturo for your very clear explanation! I finally understand now. Thanks!! :) – xenon Jul 20 '11 at 19:02
  • If the columns are pairwise orthogonal and have no zero entries, are the rows necessarily pairwise orthogonal? – Bohan Lu Dec 09 '14 at 19:22
  • 3
    $Q^{T}Q=I$ (1). How do we know that $QQ^{T}=I$ as well? Because we can transpose both sides, and knowing that $I^T=I$ it follows that $QQ^{T}=I$? – user216094 Mar 22 '15 at 18:10
  • Does it follow that all matrixes that are row orthogonal and column orthogonal are orthonormal sets? – Peter Oct 06 '17 at 09:29
  • 2
    Columns are orthogonal implies $Q^{T}Q$ is a diagonal matrix. How do you write $QQ^{T} = I$ even after you add the additional assumption that they have 2-norm 1. I dont follow your logic. It looks like the OP made the a mistake and you continued the same error in your answer. – seeker May 19 '18 at 22:48
  • 4
    @Hestenes: For square matrices $A$ and $B$, $AB=I$ if and only if $BA=I$. Thus, for a square matrix $Q$, $Q^TQ=I$ if and only if $QQ^T=I$. – Arturo Magidin May 20 '18 at 00:01
  • @ArturoMagidin Your statement is not correct. Left inverse, for example, is an inverse that need not be the actual inverse. – Mong H. Ng Jan 17 '19 at 08:27
  • @MongH.Ng: No, my statement is correct, and what you are saying is something else. If you don’t believe me, =go ahead and give me a counterexample with a square matrix over a field, then. You won’t find one, though, because they don’t exist. If $A$ is square, viewing it as a linear transformation over $F^n$, having a left inverse means that the transformation is one-to-one. Since a linear map from $F^n$ to itself is one-to-one if and only if it is onto, then $A$ has a right inverse as well, and then it is easy to verify they must be the same matrix on both sides. – Arturo Magidin Jan 17 '19 at 17:06
  • @MongH.Ng: For an arbitrary, non-square matrix (or in an arbitrary non-commutative ring), it is indeed possible to have a one-sided inverse that is not a two-sided inverse: an injective but not surjective linear transformation has a left inverse (in fact, many), but not a right inverse. But that’ is why I specified square matrices. – Arturo Magidin Jan 17 '19 at 17:08
  • @ArturoMagidin is right. See this post: https://math.stackexchange.com/questions/470159/matrices-left-inverse-is-also-right-inverse – Herman Jaramillo Jun 27 '19 at 19:02
14

Even if $A$ is non-singular, orthogonality of columns by itself does not guarantee orthogonality of rows. Here is a 3x3 example: $$ A = \left( \begin{matrix} 1 & \;2 & \;\;5 \\ 2 & \;2 & -4 \\ 3 & -2 & \;\;1 \end{matrix} \right) $$ Column vectors are orthogonal, but row vectors are not orthogonal.

On the other hand, orthonormality of columns guarantees orthonormality of rows, and vice versa.

As a footnote, one of the forms of Hadamard's inequality concerns the absolute value of the determinant of a matrix given the norms of the column vectors. That absolute value will be maximum when those vectors are orthogonal. The determinant, in absolute value, will be equal to the product of the norms. In the case of the above matrix, as the columns are orthogonal, 84 is the maximum possible absolute value of the determinant $-$ det(A) is -84 $-$ for column vectors with the given norms ($\sqrt {14}, 2\sqrt 3$ and $ \sqrt {42}$ respectively).

Although $det(A)=det(A^T)$, Hadamard's inequality does not imply neither orthogonality of the rows of A nor that the absolute value of the determinant is maximum for the given norms of the row vectors ($ \sqrt{30}, 2\sqrt 6$ and $ \sqrt{14}$ respectively; their product is $ 12 \sqrt{70} \cong 100.4 $).

goblin GONE
  • 67,744
Charles Valente
  • 141
  • 2
    I edited this answer to make it clearer. Imo it's by far the best answer here, and probably deserves to be the accepted answer. – goblin GONE Mar 29 '17 at 03:14
4

This condition says that $Q^{-1} = Q^t$. That means that you have $$Q^tQ = Q Q^t = I.$$
Yes, if the rows are orthonormal (basis -- oops my omission), so are the columns.

ncmathsadist
  • 49,383
  • Having orthonormal basis as columns implies $Q^TQ = I$. However, $Q^TQ = I$ does not implies that $Q^T$ is the inverse. See pseudoinverse: https://en.wikipedia.org/wiki/Moore%E2%80%93Penrose_inverse. – Mong H. Ng Jan 17 '19 at 08:46
  • @MongH.Ng The pseudoinverse is irrelevant here; the pseudoinverse is equal to the inverse when the latter exists, and if $A$ is a square matrix and there is no inverse at all, then it is not the case that $A^{\dagger}A=I$. You keep forgetting that $Q$ is a square matrix, and that has important implications. – Arturo Magidin Jan 17 '19 at 20:53
2

None of the existing answers quite worked for me, so...

If $A$ has orthonormal columns, $A^T A=I$. $A^T A$ is full of dot products of pairs of columns of $A$. By orthogonality, these are all zero off of the diagonal. By orthonormality, these are all one on the diagonal.

If $A$ is rectangular, $A^T$ cannot be a true inverse of $A$. If $A$ is square with orthonormal (or even independent non-zero) columns, we have $A^{-1} A=I$, suggesting that $A^T$ could be $A^{-1}$. But how to rule out that there may be more than one $B$ such that $BA=I$?

Well, if $XY=Z$, every column of $Z$ is a linear combination of the columns of $X$, and every row of $Z$ is a linear combination of the rows of $Y$. But if $X$ has independent columns, there's precisely one unique linear combination of those columns that gives any particular vector result (including each column of $Z$), so $Y$ is uniquely determined by $X$ and $Z$. Similarly if $Y$ has independent rows, each row of $X$ is uniquely determined by $Y$ and $Z$.

The columns of $A$ are orthonormal so clearly independent. If (and only if) $A$ is square, its rows are independent. And by those uniqueness properties, if $A^T A=I$, we can conclude that $A^T=A^{-1}$.

Since $A^T=A^{-1}$, $A A^T=I$ also - the rows of $A$ are orthonormal because $A A^T$ is made of dot products of pairs of rows which are all one on the diagonal, zero elsewhere (as earlier with $A^T A$, but for rows not columns).

I'm still assuming $B A=I$ implies $A B=I$ with the same $B=A^{-1}$ both ways, so...

If $B$ is the left inverse of $A$, and $C$ is the left inverse of $B$, then $BA=I$ and $CB=I$ by definition. Multiplying both sides of $BA=I$ gives $CBA=C$, so $A=C$ -- the left inverse of the left inverse is the original matrix. We knew that anyway because the left inverse is the transpose (good job - otherwise I'd have to prove the existence of the left inverse of the left inverse). But substituting $A=C$, $BA=I$ becomes $BC=I$ and $CB=I$ becomes $AB=I$, so left inverses are right inverses too.

This answer still isn't a full self-contained proof because I haven't proved the uniqueness properties for linear combinations of independent vectors, or the properties of dot products, or that matrix multiplication is associative.

-1

Proof that a square matrix with orthonormal columns must have orthonormal rows

Let a square matrix Q have Orthonormal columns qi, then QT Q = I (since qi qj= 1 when i=j else 0)

For Orthonormal rows we need Q QT = I which follows from Q QT = Q (I) QT = Q (QT Q) QT = (Q QT)(Q QT) which requires Q QT = I

Robert H Whigham
  • 7