4

I know the definition of these two norms. Given a matrix ${\bf A} \in \Bbb R^{n\times n}$ or $\Bbb C^{n\times n}$, Frobinus norm $\|{\bf A}\|_F$ is simply the vector 2-norm applied on the vectorized $\bf A$, and $\|{\bf A}\|_2=\max_{{\bf x}:\|{\bf x}\|_2=1}\|{\bf Ax}\|_2$ can be interpreted as the maximum "stretching" power of linear operator $\bf A$ on the unit circle w.r.t. vector 2-norm, and it can be shown that $\|{\bf A}\|_2=\sqrt {\max (\sigma({\bf A}^*{\bf A}))}$ where ${\bf A}^*$ is the conjugate transpose of ${\bf A}$, and $\sigma (\cdot)$ denotes the set of all eigenvalues of the matrix.

I am curious that if there is any known bound on their difference $|\|{\bf A}\|_F-\|{\bf A}\|_2|$. Especially, if $\bf A$ is positive definite, $\|{\bf A}\|_2$ is the largest eigenvalue of $\bf A$, then in what situation we can use $\|{\bf A}\|_F$ to estimate $\|{\bf A}\|_2$, i.e. the largest eigenvalue?

Ralph B.
  • 2,067
  • 18
  • 29
  • 1
    I only know that $$|A|_2≤|A|$$ for any sub-multiplicative matrix norm $|\cdot|$, if $A$ is hermitian. And given $ε>0$ there exists a matrix norm $|\cdot|$ such that $$ |A|≤|A|_2 +ε.$$ (In the second case you don't need hermitian, if I remember correctly.) – P. Siehr Jul 26 '17 at 16:36
  • 1
    Also it holds: $$|A|_{Fr} = \sqrt{σ_1^2 + … + σ_n^2}$$, with the singular values of $A$. // I will formulate a full answer later ;) – P. Siehr Jul 26 '17 at 16:40
  • 1
    It is generally bad to estimate the maximum eigenvalue of an SPD matrix using its Frobenius norm. You see that this could work only if the remaining eigenvalues would be much smaller than the maximum one, that is, the matrix would be close to a rank-1 matrix for which this is exact. – Algebraic Pavel Jul 27 '17 at 09:13
  • 1
    You can use the bound from the first comment, e.g., like $|A|2\leq|A|\infty$ which is equivalent to using the Gershgorin theorem for this purpose. – Algebraic Pavel Jul 27 '17 at 09:15

1 Answers1

1

Using the two comments in the question, one can see the following two estimates: \begin{align*} \|A\|_2&≤\|A\|_{Fr}≤\sqrt{n}\|A\|_2 \end{align*} The right inequality is obvious, the left follows from:
Let $λ$ be any eigenvalue with corresponding normalized eigenvector $w$, then it is $$|λ|=|λ|\|w\|=\|Aw\|≤¹\|A\|\|w\|=\|A\|,$$ for any ¹submultiplicative norm $\|\cdot\|$.

For $n=1$ you get equality on the left.
And for $A=\text{diag}(λ)$, with $λ>0$, we get equality on the right, since \begin{align*} \|A\|_2 &= λ \\ \|A\|_{Fr}&=\sqrt{\sum_{i=1}^nλ^2} = \sqrt{n}λ=\sqrt{n}\|A\|_2 \end{align*} Therefore the estimate above is the best we can deduce.

Remark: Since that question was asked with the "numerical linear algebra" tag, one should keep in mind, that eigenvalue problems (in general) are numerically unstable. That means if the entries of the matrix are perturbed little, the eigenvalues might still see a huge change. Since the Frobenius-norm is somewhat a measure for the perturbation of the entries, that instability is of interest for this question.

We can see the effect of the perturbation with the following matrix $$\tilde{A}=\pmatrix{0&ε \\ 1 & 0}∈ℝ^{2×2}$$ Here the eigenvalues are $0$ for $ε=0$. For $ε\neq0$ they are $λ=\pm\sqrt{ε}$. So a small error $10^{-2}$ results in eigenvalues $10^{-1}$ - that is a factor of 10.
Now take the $n×n$-matrix that has $1$ on the left off-diagonal, and $ε$ in the top right corner: $$\tilde{A}=\pmatrix{0&&&ε \\ 1 & 0 \\ &\ddots & \ddots \\ &&1&0}∈ℝ^{n×n}$$ Again all eigenvalues $λ=0$, if $ε=0$. But if $ε=10^{-n}$, the result contains one eigenvalue $λ=10^{-1}$ and you get an error amplification factor of $10^{n-1}$.
And, as I stated above, the Frobenius norm only sees the change $10^{-n}$, while the $2$-norm sees the change of the eigenvalue.

In general the following theorem holds:
Let $A∈ℝ^{n×n}$ be a matrix with eigenvectors $w_i$. And let $\tilde{A}=A+δA$ be a perturbed matrix. Then it holds, with $W=(w_1,…,w_n)$: $$|λ(A)-λ(\tilde{A})|≤\text{cond}_{2}(W)\|δA\|.$$ So the conditioning of the eigenvalue problem depends on the condition number of $W$.
Sounds pretty bad, but for hermitian matrices there exists a orthonormal basis, which means that for these matrices the eigenvalue problem is stable. For arbitrary matrices it is arbitrary ill-conditioned.

P. Siehr
  • 3,672