1

I was reading this post about the fact that the rank of a matrix denotes how much information the matrix holds. The author defines a $m\times n$ matrix A and sais that if it has rank $p$ we can decompose it as $A=UV^T$, where $U$ is a $m\times p$ matrix and $C$ is a $n\times p$ matrix. But then to store the information we would need to store $mp+np$ entries no? Why is $2np-p^2$?

roi_saumon
  • 4,196
  • Looks like a typo. It certainly cannot not depend on $m$. Take, for instance, $n=p=1$. Perhaps they meant to say $mp+np-p^2$ (and for some reason were thinking of a square matrix $n\times n$ at the moment of writing)? –  Oct 19 '20 at 12:51
  • I see, thank you. But why do we subtract $p^2$? – roi_saumon Oct 19 '20 at 13:06

1 Answers1

1

You can in general specify $mp+np-p^2$ elements of an $m\times n$ matrix of rank $p$ over $\mathbb R$, the rest will be uniquely determined.

To (roughly) see that, let $A$ be an $m\times n$ matrix of rank $p$. WLOG assume the first $p$ rows are linearly independent. To specify those, you need $np$ (mostly) independent values. (I am saying "mostly" because there are constraints given by inequalities - all the minors of rank $p$ are nonzero.) The remaining $m-p$ rows are linear combinations of the first $p$ rows, so to specify those you need an $(m-p)\times p$ matrix of coefficients.

Altogether you need to specify $np+(m-p)p=mp+np-p^2$ parameters.

Staying for a moment with the field $\mathbb R$: this proof can be further formalised to state that the subset of all matrices of rank $p$ in $M_{mn}(\mathbb R)$ makes up for an $mp+np-p^2$-dimensional (smooth) manifold (informally - an $mp+np-p^2$-dimensional smooth "surface" embedded in $M_{mn}(\mathbb R)$). (Details omitted as probably too tedious, but the proof would proceed roughly the same as above.) I am saying that because the notion of "$mp+np-p^2$ independent variables..." is a bit vague, but one possible interpretation what it means is precisely that this set of matrices makes up an $mp+np-p^2$-dimensional manifold.

(Note I am not an expert in algebraic geometry, but I am sure within this discipline there will be another meaning of what "... independent variables ..." means. After all, the set of all matrices of rank $p$ is a set of solutions of a bunch of polynomial equations - all the minors of rank $p+1$ are zero, and a bunch of polynomial inequalities - all the minors of rank $p$ are nonzero... At least the former set produces what is called an "algebraic variety" which has a well-defined concept of dimension.)

The statement is false in general for finite fields. Take, for example, $\mathbb Z_2$ and take also $m=n=p=2$: You would expect to need $2^2=4$ independent parameters, but really you have only $3$ independent parameters. Namely, the matrix $$\begin{bmatrix}a&b\\c&d\end{bmatrix}\in M_{22}(\mathbb Z_2)$$ is of rank $2$ if and only if $ad-bc\ne 0$, but this means $ad-bc=1$ and so you can only pick three out of $a,b,c,d$ and the fourth (if exists) will be uniquely determined.

  • Thank you. I understand your reasoning. But when I have the decomposition $A=UV^T$ then if I follow the same reasonning, I need $p^2+(n-p)p$ values to store the first matrix and $p^2+(m-p)p$ values to store the second one so it seems like we kind of lost some information since $p^2+(n-p)p+p^2+(m-p)p = mp+np$. Did I do a mistake in the calculations? – roi_saumon Oct 20 '20 at 16:16
  • It is because some of these values are not independent. For example, take a rank-1 $n\times n$ matrix made by multiplying a $n\times 1$ column by a $1\times n$ row. Obviously you would get the same matrix if you multiply that column by a nonzero constant and you divide that row by the same constant. –  Oct 20 '20 at 16:49
  • I feel like I understand more or less what is going on but I cannot make it formal. Indeed for an orhogonal matrix $Q$, $UQ, VQ$ is another decomposition of $A=UV^T$ because $UQ(VQ)^T=UQQ^TV^T=UV^T$ and Q has $p^2$ elements. But does this help to get the term $-p^2$? – roi_saumon Oct 22 '20 at 12:11
  • @roi_saumon As I said in the answer, I expect there will be multiple ways to formalise this notion, in different mathematical disciplines. In real analysis/differential geometry it is going to be on the line of the hints I've given. In your comment, you don't need an orthogonal matrix, merely an invertible matrix: $UP, V(P^{-1})^T$ would do. –  Oct 22 '20 at 12:15