Is the orthogonal polar factor of matrices of nullity 1 smooth?

Question

Let $M_n$ be the space of real $n \times n$ matrices, and let $\text{GL}_n^-=\{A \, | \, \det A < 0 \}$. Set $$S=\text{GL}_n^- \cup \{A \in M_n \, | \, \text{rank} (A) = n-1 \}.$$

Claim: There exist a unique continuous map $O:S \to \text{O}^{-}_n$, such that $A$ can be decomposed as $A=O(A)P(A)$, where $O(A) \in \text{O}^{-}_n$ and $P(A)$ is symmetric positive semi-definite. (here $\text{O}^{-}_n$ are the orthogonal matrices with determinant $-1$).

In the case where $A \in \text{GL}_n^-$, this decomposition is the well-known (unique) polar decomposition of $A$: We have $P(A)=\sqrt{A^TA}$ and $O(A)=A(\sqrt{A^TA})^{-1}$.

My question: Is the restriction of $O$ to $\{A \, | \, \text{rank} (A) = n-1 \}$ smooth, when considered as a map between the manifolds $\{A \in M_n \, | \, \text{rank} (A) = n-1 \} \to \text{O}^{-}_n$?

Here is an "explicit" description of $O(A)$ in terms of the SVD of $A$:

Let $A=U\Sigma V^T$ be the SVD of $A$, where $U,V \in \text{O}_n$, and suppose that one of $U,V$ is in $\text{SO}_n$ and the other one is in $\text{O}^{-}_n$. Then $O(A)=U V^T$. (That is $O(A)$ "forgets" the contractions and expansions of $A$).

The problem is that if $A$ has repeating singular values, then $U,V$ cannot always be chosen continuously around $A$. However, around any point with distinct singular values, $U,V$ can be chosen smoothly, so the only possible points of non-smoothness are those with repeating singular values.

Answer 1

Here are two arguments.

Inverse function theorem

Let $M'\subset S$ be the $n^2-1$ dimensional manifold of matrices of rank $n-1.$

Let $P'$ be the $n(n+1)/2-1$ dimensional manifold of positive semidefinite matrices of rank $n-1.$

Your claim implies that the multiplication map $\mu:O^-_n\times P'\to M'$ defined by $\mu(O,P)=OP$ is a homeomorphism - the inverse sends $A$ to $(O(A),O(A)^{-1}A).$ I claim that $\mu$ is a diffeomorphism. Thus your map $O:M'\to O^-_n$ is smooth - it's the inverse of $\mu$ composed with the projection to the $O^-_n$ component.

The dimensions match: $n(n-1)/2+n(n+1)/2-1=n^2-1.$ So for the inverse function theorem to apply we only need to check that $d\mu$ is injective.

Tangent vectors to $O^-_n\times P'$ at $(O,P)$ are of the form $(OA,S)$ where $A$ is antisymmetric and $S$ is symmetric. Assume that $d\mu_{(O,P)}(OA,S)$ is zero. So $OAP+OS=0.$ This means $AP$ is symmetric. Let $x_1,\dots,x_n$ be an orthonormal basis of eigenvalues of $P$ with corresponding eigenvalues $\lambda_1,\dots,\lambda_n.$ For indices $i\neq j,$ $$\lambda_jx_i^TAx_j=x_i^TAPx_j=x_i^T(AP)^Tx_j=-x_i^TPAx_j=-\lambda_ix_i^TAx_j,$$ but $\lambda_j$ can't equal $-\lambda_i$ for $i\neq j,$ so $x_i^TAx_j=0.$ And $x_i^TAx_i=0$ because $A$ is antisymmetric. This implies that $A$ is zero, and $S$ must therefore also be zero. So $d\mu$ is injective as required.

(If you wanted to show that $O$ is smooth on $S,$ in the manifold-with-boundary sense of all derivatives being continuous at each rank $n-1$ matrix, you could extend $P'$ to an open neighborhood such as the matrices with least eigenvalues $\lambda_1,\lambda_2$ satisfying $\lambda_1>-\lambda_2,$ use $\mathbb R^{n\times n}$ as the codomain for $\mu,$ and check it's still a local diffeo.)

Subtracting cofactor matrix

$\DeclareMathOperator{\cof}{cof}$ Let $\cof(A)$ denote the cofactor matrix of a matrix $A.$ The important properties are that $\cof$ is smooth (in fact algebraic) and $A \cdot \cof(A)^T=\det(A)I.$

If $U,V$ are orthogonal and $\Sigma$ is diagonal then $$\cof(U\Sigma V^T)=\det(UV^T)U\cdot \cof(\Sigma) \cdot V^T$$ and $\cof(\Sigma)$ is diagonal with $i$'th diagonal entry $\det(UV^T)\prod_{j\neq i}\Sigma_{jj}.$ (To verify that this is correct, it suffices to check $A \cdot \cof(A)^T=\det(A)I$ with $A=U\Sigma V^T,$ because that characterizes $\cof(A)$ for non-singular $A,$ and for singular $A$ we can use continuity in $\Sigma$ with $U,V$ fixed.)

The relevant situation is when $A=U\Sigma V^T$ is an SVD, the rank is $n-1,$ and $\det(UV^T)=-1.$ With the convention that singular values are in descending order, $\Sigma_{nn}=0.$ Then $\cof(\Sigma)$ is zero except for its $nn$ entry which is $\prod_{1\leq j<n}\Sigma_{jj}>0.$ So $\Sigma-\det(UV^T)\cof(\Sigma)=\Sigma+\cof(\Sigma)$ is strictly positive definite. This implies that $A-\cof(A)$ is a non-singular matrix with the correct orthogonal polar factor $O(A-\cof(A))=UV^T.$

$A\mapsto A-\cof(A)$ is smooth (in fact algebraic), and $O$ is smooth on non-singular matrices, so the composition $A\mapsto O(A-\cof(A))$ is smooth and well-defined when restricted to rank $n-1$ matrices.

(If you wanted to show that $O$ is smooth on $S,$ in the manifold-with-boundary sense of all derivatives being continuous at each rank $n-1$ matrix, you could use the fact that $A-\cof(A)$ must be non-singular in a ball around a given $A$ of rank $n-1.$)