2

I'm reviewing for my final exam and one of the practice problems given to me by my professor is this:

Consider the matrix $A = uv^T$, where $u \in \mathbb{R}^{m \times 1}$ and $v \in \mathbb{R}^{n \times 1}$. Find an orthogonal projector onto the range of $A$. Find an orthogonal projector onto the null space of $A$.

We discussed in class that, if $A = QR$ (the QR decomposition), then $QQ^T$ and $I - QQ^T$ are projectors where $QQ^T$ projects onto the range of $A$ and $I - QQ^T$ projects onto $\mathcal{R}(A)^{\perp}$. But I'm still confused about what it means for something to project onto the range of a matrix and don't really know how to come up with the QR decomposition of $A$ since I'm just given that $A = uv^T$.

Any help would be greatly appreciated. Thanks in advance!

M42
  • 71

1 Answers1

1

In a situation as elementary as this, you can really just go back to the basic definitions: $$ \mathcal{R}(A) := \{y \in \mathbb{R}^{n \times 1} : y = Ax \:\text{for some $x \in \mathbb{R}^n$}\},\\ \mathcal{N}(A) := \{x \in \mathbb{R}^{n \times 1} : Ax = 0\}. $$ Given the construction of $A$, it'll be easy to describe $\mathcal{R}(A)$ as the span of some orthonormal set and $\mathcal{N}(A)$ as the orthogonal complement of the span of some orthonormal set. Once you've done this, just remember that if $S = \operatorname{Span}\{v_1,\dotsc,v_k\}$ for some orthonormal set $\{v_1,\dotsc,v_k\}$ in $\mathbb{R}^{n \times 1}$, then the orthogonal projection onto $S$ is $$ P_S := v_1 v_1^T + \cdots + v_kv_k^T $$ and the orthogonal projection onto $S^\perp$ is $$ P_{S^\perp} = I_n - P_S; $$ the meaning of this is that for any $x \in \mathbb{R}^{n \times 1}$, the orthogonal projection of $x$ onto $S$ is $P_S x$ and the orthogonal projection of $x$ onto $S^\perp$ is $P_{S^\perp}x$.

Branimir Ćaćić
  • 13,684
  • P_s as defined here is a scalar or am I missing something? – Avedis Feb 08 '23 at 17:56
  • 1
    If $v$ is an $n \times 1$ matrix (i.e., a "$n$-dimensional" column vector), then $v^T v$ is a $1 \times 1$ matrix (i.e., the usual scalar product) while $vv^T$ is an $n \times n$ matrix. – Branimir Ćaćić Feb 08 '23 at 20:01
  • Pardon my ignorance, but wouldn't the projection simply be a matrix composed of the columns from the vectors of the set S? – Avedis Feb 09 '23 at 01:40
  • No. Please remember that if $A$ is an $m \times n$ matrix, then the $k$th column of $A$ is $Ae_k$, where ${e_1,\dotsc,e_n}$ is the standard orthonormal basis of $\mathbb{R}^n$. – Branimir Ćaćić Feb 09 '23 at 03:12
  • If you like, the $n \times k$ matrix you suggest would give a linear parametrization of $S$ in terms of $k$ independent parameters, not the $n \times n$ orthogonal projection matrix onto $S$. – Branimir Ćaćić Feb 09 '23 at 03:16
  • I still don't understand what you mean. If I have a matrix square matrix A and want to project a vector on to it, I do Ax. If i want the orthogonal projection, then I compute A=QR and do QQ^T*v. My understanding the difference between a projection and an orthogonal projection is simply which transformation preserves the length of the vector x. – Avedis Feb 09 '23 at 12:41
  • 1
    That is not the definition of projection or orthogonal projection. Suppose that $S$ is a subspace of $\mathbb{R}^n$. A projection onto $S$ is an $n \times n$ matrix $P$ such that $P^2 = P$ and $\operatorname{ran}(P) = S$, so that $\mathbb{R}^n = \operatorname{ran}(P) \oplus \ker(P)$; it is orthogonal in the case that $P = P^T$, which is equivalent to $\ker(P) = \operatorname{ran}(P)^\perp$. These are the standard definitions in linear algebra; these are the definitions referred to in the question and in my answer. – Branimir Ćaćić Feb 09 '23 at 13:29
  • 1
    Geometrically, orthogonal projection refers to something very specific: it's finding the best approximation (in a certain sense) to a given vector by a vector in $S$. It is not a term for the process of applying a linear transformation to a vector or multiplying a column vector on the left by a matrix. – Branimir Ćaćić Feb 09 '23 at 13:32
  • I get a little more of it (i think). One projection is parallel and the other is perpendicular. Does this make sense? So projecting onto A gives the vector that aligns best with A while projections on to perpendicular(A) gives the vector that is perpendular to A. – Avedis Feb 09 '23 at 13:55