3

I was trying to read Edelman et al.'s 1998 paper "The Geometry of Algorithms with Orthogonality Constraints" and since I don't have any differential geometry or much linear algebra background I am stuck at few places.

This is regarding the section 2.2.1 i.e tangent and normal spaces of the Steifel manifold. Here is the excerpt:

Let $Z$ be any n-by-p matrix. Let $sym(A)$ denote $(A + A^T)/2$ and $skew(A)$ denote $(A - A^T)/2$ , then at $Y$ , $\pi_N(Z) = Ysym(Y^TZ)$ defines a projection of Z onto the normal space. Similarly at $Y$ , $\pi_T(Z) = Yskew(Y^TZ) + (I - YY^T)Z$

I tried deriving $\pi_N(Z)$ as $\pi_N(Z) = tr(N^TZ) ; N = {YS}$ where $S$ is any p-by-p symmetric matrix but couldn't arrive at the result. But using that result I could easily derive $\pi_T(Z)$ because $Z = \pi_T(Z) + \pi_N(Z)$

Next, they go on to say that tangent directions $\Delta$ at $Y$ then have the general form : $\Delta = YA + Y_{\perp}B$ where $A$ is p-by-p skew symmetric , $B$ is any (n-p)-by-p matrix and $Y_{\perp}$ is any n-by-(n-p) matrix such that $YY^T + Y_{\perp}{Y_{\perp}}^T = 1$

I couldn't derive the general form of the tangents either. Can someone please help me out on this?

Thanks

Avitus
  • 14,018
tselvan
  • 173
  • I wrote a general answer which justifies the $\Delta$ matrices. Please, tell me if you need further steps/clarifications ok? – Avitus Aug 27 '13 at 10:22

2 Answers2

2

I will try with an informal approach.

  • Tangent space and curves on manifolds

Let $p$ be any point of a given manifold $S$ of dimension $n$. With $\gamma:[a,b]\rightarrow S$ we denote any path on $S$ passing through $p$, with $\gamma(0)=p$. To understand the tangent space $T_pS$ of $S$ at $p$, one needs to consider the tangent vectors

$$\frac{d(\phi\circ\gamma)}{dt}(0) $$

at $t=0$, denoting by $\phi:U\ni p\rightarrow \mathbb R^n$ any local chart around $p$. The derivative w.r.t. $t$ of $\phi\circ\gamma$ is computed using the standard techniques of Analysis. The tangent space at $p$ consists of the equivalence classes of such paths through $p$; two paths are equivalent if their first derivatives at $0$ coincide. One puts a linear space structure on $T_pS$ and checks that all constructions are chart-independent.

  • Tangent space on Stiefel manifolds

If we consider a Stiefel manifold $V_{n,p}$, some simplifications occurr. In fact, any point $Y$ on the Stiefel manifold $V_{n,p}$ is an $n\times p$ matrix which satisfies $$Y^T Y=I. $$

This implies that any path $\gamma:[a,b]\rightarrow V_{n,p}$ is s.t. $t\mapsto \gamma(t)=Y(t)$, i.e. the image of $\gamma$ is a matrix on the Stiefel manifold with elements which are functions of $t$, for all $t\in[a,b]$. In other words, we can use the standard calculus techniques to compute

$$\frac{d}{dt}\gamma $$

elements-wise in $\gamma(t)\in V_{n,p}$. In order to find the relations defining the tangent space at $Y\in V_{n,p}$ we follow the above scheme. We begin by considering a path $\gamma$ on the Stiefel manifold s.t. $\gamma(t)=Y(t)$ and $\gamma(0)=Y(0):=Y$; the relations $Y(t)^T Y(t)=I$ are equivalent to

$$\gamma^T(t)\gamma(t)=1(t), ~~(*)$$

denoting by $1(t)$ the image of the constant path on $V_{n,p}$, i.e. $1(t)=I$, for all $t$. We differentiate $(*)$, obtaining

$$\frac{d}{dt}\left(\gamma^T\gamma\right)=0,$$

and

$$\left(\frac{d\gamma}{dt}(0)\right)^T\gamma(0)+ \gamma(0)^T\frac{d\gamma}{dt}(0)=0$$

at $t=0$. Denoting by $\Delta:=\left(\frac{d\gamma}{dt}(0)\right)^T$ and recalling that, by definition, $\gamma(0)=Y$, we arrive at

$$\Delta^TY+ Y^T\Delta=0. $$

In the reference at pag. 307 you can find the above formula. $\Delta$, by definition, is any element of the tangent space at $Y$. I hope it helps.

  • On the projections onto the tangent, resp. normal space at $Y$

Let us consider the setting above. The normal space at $Y$ is the space

$$\mathcal N_Y=\{N\in V_{n,p}: tr(\Delta^TN)=0\}, $$

as the trace map $tr$ defines an inner product, as shown at pag. 308. The important step to show is the following: for any $Z\in V_{n,p}$, then

$$A=\pi_N(Z):=Y\operatorname{sym}(Y^TZ)\in \mathcal N_Y.$$

In fact

$$tr(\Delta^TA)=\frac{1}{2}tr(\Delta^T Y(Y^TZ+(Y^TZ)^T))=\frac{1}{2}\left(tr(\Delta^T Z)-tr(\Delta Z^T)\right)=0,$$

for the properties of trace. Similarly, one proves that $\pi_T(Z)\in T_Y V_{n,p}$, i.e.

$$\pi_T(Z)^T Y+Y^T\pi_T(Z)=0. $$

The final step towards the decomposition is to show that the equality

$$Z=\pi_T(Z)+\pi_N(Z), $$

holds. In other words, each $Z$ is determined by the orthogonal splitting in tangential and normal components.

Avitus
  • 14,018
  • Avitus : thanks for your clear exposition of the fundamentals but my doubt is still untouched. I want to know about the projection of some matrix Z on the normal space and tangent space at X. – tselvan Aug 27 '13 at 12:12
  • Ok @Tirumarai: let me work on it :-) I will edit the answer, as soon as I get some computations – Avitus Aug 27 '13 at 12:27
  • Thanks Avitus: Also please look into the followup question right here : http://math.stackexchange.com/questions/477945/horizontal-and-vertical-tangent-space-of-orthogonal-group – tselvan Aug 28 '13 at 06:42
  • @Avitus Your link for the references seems not to work. Could you quickly state the book or article you are referring to? - Thank you in advance. – Mathematics enthusiast May 11 '23 at 17:56
0

It is apparent that $\pi_{N}(Z) \neq tr(N^{T} Z)$ since $\pi_{N}(Z)$ should be an element in the ambient space $R^{n\times p}$, $tr(N^{T} Z)$ is only a real scalar, serving as the inner product between $N$ and $Z$.