Optimize summation of L2 norm and infinity norm

Question

Consider the optimization problem: $$ \min_{x\in R^n} \lambda^2 \Vert x\Vert_2^2 + \Vert Ax - \xi \Vert_{\infty}^2$$ The first part of the objective function is L2 norm, and the second part is the infinity norm (i.e, $\Vert x \Vert_{\infty} = \max_{i} |x_i|$). $\lambda$ is a positive constant, $A$ is a diagonal matrix with positive diagonal entries, $\xi$ is a vector. Can this problem be formulated into a univariate optimization problem? Where the condition of matrix $A$ can be utilized? Thanks ahead for any advice!

Answer 1

I don't think you can transfer this into a univariate problem unless $ \boldsymbol{A} $ is a scalar matrix.

Yet you may solve it pretty efficiently using Alternating Direction Method of Multipliers (ADMM) if you change the formulation into:

$$ \arg \min_{\boldsymbol{x}} \frac{1}{2} {\left\| \boldsymbol{x} \right\|}_{2}^{2} + \lambda {\left\| \boldsymbol{A} \boldsymbol{x} - \boldsymbol{\xi} \right\|}_{\infty} $$

By setting $ \boldsymbol{z} = \boldsymbol{A} \boldsymbol{x} - \boldsymbol{\xi} $ the problem be easily solved as both terms have a closed form solution to the Proximal Operator.

In general the Scaled ADMM optimization will generate something along:

$$ \begin{align} \arg \min_{\boldsymbol{x}, \boldsymbol{z}} \quad & f \left( \boldsymbol{x} \right) + g \left( \boldsymbol{z} \right) \\ \text{subject to} \quad & \begin{aligned} P \boldsymbol{x} + Q \boldsymbol{z} - \boldsymbol{c} = \boldsymbol{0} \\ \end{aligned} \end{align} $$

Then the solution is given by iterating:

1. $ \boldsymbol{x}^{k + 1} = \arg \min_{\boldsymbol{x}} f \left( \boldsymbol{x} \right) + \frac{\rho}{2} {\left\| P \boldsymbol{x} + Q \boldsymbol{z}^{k} - \boldsymbol{c} + \boldsymbol{u}^{k} \right\|}_{2}^{2} $.
2. $ \boldsymbol{z}^{k + 1} = \arg \min_{\boldsymbol{z}} g \left( \boldsymbol{z} \right) + \frac{\rho}{2} {\left\| P \boldsymbol{x}^{k} + Q \boldsymbol{z} - \boldsymbol{c} + \boldsymbol{u}^{k} \right\|}_{2}^{2} $.
3. $ \boldsymbol{u}^{k + 1} = \boldsymbol{u}^{k} + P \boldsymbol{x}^{k + 1} + Q \boldsymbol{z}^{k + 1} - \boldsymbol{c} $.

So, for the case above, just set:

$$ \boldsymbol{A} \boldsymbol{x} - \boldsymbol{\xi} - \boldsymbol{z} = 0 $$

Then just iterate over as above.

I wrote MATLAB code which implements them both at Mathematics StackExchange Question 4804920 - GitHub. There is a test which compares the result to a reference calculated by CVX.

Remark: In case the matrix $ \boldsymbol{A} $ is indeed a scalar matrix you may solve the above problem in a closed form using calculus of the Proximal Operator.