1

Suppose that we have a matrix $\mathbf{A}\in\mathbb{R}^{n\times p}$. If we want to find an approximation of $\mathbf{A}$ with maximum rank $k$, which is denoted as $\tilde{\mathbf{A}}$, is it possible to solve the following

$$\left\{\tilde{\mathbf{P}},\tilde{\mathbf{Q}}\right\} = \underset{\begin{aligned}\mathbf{P}\in\mathbb{R}^{n\times k}\\\mathbf{Q}\in\mathbb{R}^{p\times k}\end{aligned}}{\arg \min}~~~\left\|\mathbf{A}-\mathbf{P}\mathbf{Q}^\top\right\|_{\text{F}}^2$$ in an alternative way and simply let $\tilde{\mathbf{A}}=\tilde{\mathbf{P}}\cdot\tilde{\mathbf{Q}}^\top$?

Note that the constraint of a rank no more than $k$ is implied by the dimensions of $\mathbf{P}$ and $\mathbf{Q}$.

Any help is appreciated.

Rodrigo de Azevedo
  • 1
Mizera
  • 11
  • 1
  • 1
    Are you acquainted with the SVD? – Rodrigo de Azevedo Jun 07 '23 at 19:16
  • Yes, basically low rank approximation is done by SVD, while I am dealing with a problem reagarding a matrix $\mathbf{A}\in\mathbb{R}^{n\times p}$ whose rank is at most $k$, and in the problem settings $k\ll n\text{ and } p$. That's why I wonder is there any way to calculate the first $k$ components only. – Mizera Jun 08 '23 at 02:09
  • Are you acquainted with Eckart–Young–Mirsky? – Rodrigo de Azevedo Jun 08 '23 at 05:09
  • Thanks, now it may be clear for me: The approximation $\mathbf{A} \approx \mathbf{P}\mathbf{Q}^\top$ is NOT unique basically because, consider the equality $\mathbf{A} = \sum_{i=1}^k \sigma_i\cdot u_i v_i^\top$ where $\sigma_i$ is the singular value and $u_i, v_i$ are the left & right vectors, we can distribute the singular value $\sigma_i$ in any proportion as $\sigma_i=\alpha\cdot\beta,\text{ where } \alpha,\beta\in\mathbb{R}+$ and let $\mathbf{P}{(i, :)} = \alpha\cdot u_i, \mathbf{Q}_{(i, :)} = \beta\cdot v_i$ to form a vaild solution to the problem above. – Mizera Jun 08 '23 at 05:36
  • All in all, my conclusion now it that, the problem above is non-convex and may NOT converge at all, as my numerical tests shows. Thanks sincerely! Still, I wonder if there're mistakes remaining in my conclusion? – Mizera Jun 08 '23 at 05:44
  • On the non-uniqueness of the SVD, take a look at this – Rodrigo de Azevedo Jun 08 '23 at 05:46

0 Answers0