Operator norm calculation for simple matrix

Question

Suppose

$$ A = \left( \begin{array}{cc} 1 & 4 \\ 5 & 6 \end{array}\right) $$

How do I calculate $\|A\|_{\text{OP}}$?

I know the definition of operator norm, but I am clueless on how to calculate it for real example like this. Can somebody please give me a step-by-step instruction on how to do this?

@TonyS.F. No. For example if $A=\begin{bmatrix}0&1\0&0\end{bmatrix}$ then the norm of $A$ is certainly non-zero, since $A\ne0$, but $0$ is the only eigenvalue. What you say is correct if $A$ is a symmetric matrix... — David C. Ullrich, Feb 28 '18 at 13:28

score 16 · Accepted Answer · edited Jun 20 '18 at 17:57

16

For a matrix $A$, $||A||_{OP}$ is the square root of the largest eigenvalue of $A^TA$, where $A^T$ is $A$'s transpose.

The transpose of $\left( \begin{array}{cc} 1 & 4 \\ 5 & 6 \end{array}\right)$ is $\left( \begin{array}{cc} 1 & 5 \\ 4 & 6 \end{array}\right)$, and hence:

$$A^TA=\left( \begin{array}{cc} 1 & 5 \\ 4 & 6 \end{array}\right)\left( \begin{array}{cc} 1 & 4 \\ 5 & 6 \end{array}\right)=\left( \begin{array}{cc} 26 & 34 \\ 34 & 52 \end{array}\right)$$

The eigenvalues of this matrix are $\{39 + 5\sqrt{53};\space 39-5\sqrt{53}\}$. Therefore, $$||A||_{OP}=\sqrt{39 + 5\sqrt{53}}$$

edited Jun 20 '18 at 17:57

amWhy

209,954

answered Feb 28 '18 at 10:04

Mr. Xcoder

1,163

Is the operator norm always defined as the square root of the largest eigen value? – abina shr Feb 28 '18 at 10:12
And one more stupid question, in matlab is norm(A,2), the operator norm? – abina shr Feb 28 '18 at 10:13
@abinashr 1) Yes, for matrices. More about the operator norm can be found at Wolfram MathWorld – Citing this article, The operator norm of a linear operator $T:V\rightarrow W$ is the largest value by which $T$ stretches an element of $V$ [...] When $T$ is given by a matrix, say $T(v)=Av$, then $||T||$ is the square root of the largest eigenvalue of the symmetric matrix $A^TA$, all of whose eigenvalues are nonnegative. 2) Sorry, I don't know MATLAB. But I am pretty sure that Mathematica's Norm gives you the answer too. – Mr. Xcoder Feb 28 '18 at 10:17
It is better to phrase the spectral norm as the largest singular value of $A$, as opposed to largest eigenvalue of $A^TA$ – information_interchange Apr 06 '20 at 02:13

Martin Argerami · Answer 2 · 2018-02-28T19:50:35.597

2

You need the square root of the largest eigenvalue of $A^TA $.

Or, if you want to do it by definition, it becomes a Lagrange multiplier problem. In fact, in this $2$-dimensional case, it can be reduced to a one-variable optimization.

Concretely, using a bit of first year calculus at the end, you have that \begin{align} \|A\|^2&=\max\{\|Ax\|^2:\ \|x\|^2=1\} =\max\{(x+4y)^2+(5x+6y)^2:\ x^2+y^2=1\}\\ \ \\ &=\max\{26x^2+52y^2+68xy:\ x^2+y^2=1\}\\ \ \\ &=\max\{52-26x^2+68x\sqrt{1-x^2}:\ 0\leq x\leq1\}\\ \ \\ &=39+5\sqrt{53}. \end{align} So $\|A\|=\sqrt{39+5\sqrt{53}}$.

edited Feb 28 '18 at 19:50

answered Feb 28 '18 at 09:53

Martin Argerami

205,756

I don't quite understand the second part. Could you please elaborate, or even better, if you show it by example. Sorry I am new to this and all the things I am reading are really confusing me. I hope an example will help me understand better. – abina shr Feb 28 '18 at 09:59

Rodrigo de Azevedo · Answer 3 · 2018-05-11T15:34:09.250

The $2$-norm of matrix $\mathrm A$ can also be computed numerically, say, by solving the following convex optimization problem in $t > 0$

$$\begin{array}{ll} \text{minimize} & t\\ \text{subject to} & \| \mathrm A \|_2 \leq t\end{array}$$

or, using the Schur complement, by solving the following semidefinite program (SDP)

$$\begin{array}{ll} \text{minimize} & t\\ \text{subject to} & \begin{bmatrix} t \, \mathrm I_2 & \mathrm A\\ \mathrm A^\top & t \,\mathrm I_2\end{bmatrix} \succeq \mathrm O_4\end{array}$$

Using CVXPY (with NumPy),

from cvxpy import *
import numpy as np

A = np.array([[1, 4],
              [5, 6]])

# create 2 x 2 identity matrix
I2 = np.identity(2)

# create optimization variable
t = Variable()

# create constraints
constraint1 = [ norm(A,2) <= t ]
constraint2 = [ bmat([[ t*I2,   A],
                      [  A.T,t*I2]]) >> 0 ]

# create optimization problem
optprob = Problem( Minimize(t), constraint1 )

# solve optimization problem
optprob.solve()
print t.value

Using

constraint1, the minimum is 8.68334897643.
constraint2, the minimum is 8.68262817347.

From the other answers, the exact value of the minimum is

$$\sqrt{39 + 5 \sqrt{53}} \approx 8.68334897642624$$

and, thus, using constraint1 produces more accurate results.

The optimization problem is trivial: it is minimized when $t = ||A||_{OP}=||A||_2$. Hence, in your code, the numerical computation of the norm is performed by the Python function numpy.linalg.norm (https://numpy.org/doc/stable/reference/generated/numpy.linalg.norm.html). The answer gives no clue about the implementation of this function. — jpvigneaux, Nov 28 '23 at 22:01

Operator norm calculation for simple matrix

3 Answers3

Linked