3

I am looking for ways to compute the subderivative of $ ||Au||_{L^{\infty}} $, as I want to solve the minimization problem of \begin{equation} \min\limits_u \quad \lambda ||Au||_{L^{\infty}} + \frac{1}{2} || u - f ||^2_{L^{2}} \quad. \end{equation}

However I have no idea where to start. I tried looking at the Gateaux derivative to get an intuition on the subdifferential, but I am not even certain how the derivative interacts with the esssup of the norm.

After Boyd-Vanderbergh, there should exist a unique prox. operator for the min-problem. I am fammiliar with the procedure for subdiff/prox operator for $L^1$ and $L^2$, yet how do I deal with the $esssup$ ?

P.S: A similar question has already been asked here: proximal operator of infinity norm, but I am fairly certain that this is wrong as $L^1$ is not dual to $L^\infty$.

Rodrigo de Azevedo
  • 1
mirrormere
  • 55
  • The linked question is fine, though as a commenter points out it could use more development. But it will not help you here anyway, because the presence of $A$ changes things irretrievably. – Michael Grant Oct 13 '15 at 11:59

1 Answers1

6

Yes, the problem has a unique solution (by strong convexity of the objective), but you can't compute it closed-form...

Let's concentrate on your "real" problem: computing the subdifferential of that composite term. To this end, define $g = \|.\|_\infty$ and $f := g \circ A$. By basic properties of subdifferentials, it's clear that \begin{equation} \partial f(u) = A^T\partial g(Au) := \{A^Tv | v \in \partial \|Au\|_\infty\}. \end{equation} So it suffices to compute the subdifferential of the $\infty$-norm.

Now, for any $z$, $g(z) = \|z\|_\infty = \underset{\|w\|_1 \le 1}{\sup}z^Tw$, and so by the Danskin-Bertsekas Theorem for subdifferentials (Proposition A.22 of the PhD thesis of Bertsekas), it holds that \begin{equation}\partial \|z\|_\infty = \mathrm{conv}\{w\text{ s.t } \|w\|_1 \le 1, z^Tw = \|z\|_\infty\} = \{w\text{ s.t } \|w\|_1 \le 1, z^Tw = \|z\|_\infty\}. \end{equation} Putting things together, we have $\partial f(u) = \{A^Tw \text{ s.t } \|w\|_1 \le 1, w^TAu = \|Au\|_\infty\}$.

dohmatob
  • 9,535
  • thank you for the answer, although as it is an interesting result it seems like I just traded my minimization problem for a slightly different one as I now have to solve \begin{equation} w = argmax_{||w||1 =1} <w,Au> \quad , \end{equation} with the extra information that $max{||w||1 =1} <w,Au> = || Au||\infty $ – mirrormere Oct 14 '15 at 14:00
  • Also thank you for pointing out the Danskin-Bertsekas theorem :) Maybe the formulating the norm derivative by its predual supremum is too general ? I guess I will have to think about this for a while. – mirrormere Oct 14 '15 at 14:10
  • I'm afraid you'll have to solve your (original) problem iteratively. Lookup ADMM, primal-dual algorithm (of chambolle-pock), etc. These are all go-to methods for solving such problems with composite sums. Also, If you can post your complete problem here (not just the prox-computation part), then I can give you more explicit hints which will be more adapted to your total problem. – dohmatob Oct 15 '15 at 00:10
  • Yes, I ended up computing an ADMM algorithm. – mirrormere Oct 16 '15 at 00:46
  • In $\mathbb R^n$, the $\ell_\infty$ unit-ball is a cartesian product of $n$ identical pieces, namely $\mathbb B_\infty = [-1, 1]^n$. Thus $\text{proj}{\mathbb B\infty}(u) = (\text{proj}{[-1, 1]}(u_j)){j=1,\ldots, n}$. Exercise: Derive a formula for projection onto a compact 1D interval $[a, b]$. You're done. – dohmatob Jun 14 '16 at 13:45
  • I would add that specifically it is $ \operatorname{sign} \left( {z}{j} \right) \boldsymbol{e}{j} $ where $ j = \arg \min_{i} \left| {z}{i} \right| $ and $ \boldsymbol{e}{j} $ is the $ j $ -th standard basis. – Royi Mar 20 '20 at 14:28