4

I need to find the proximal operator for $f(x) = \lambda \sqrt{x^2 + a^2}$, where $x \in \mathbb{R}$ and $a, \lambda >0$ are some constants. Is there a closed form for it? I tried to derive the operator in a straightforward way, i.e., using its formal definition $${\rm prox}_f(y) = \inf_x \Big\{ \frac{1}{2} | x - y |^2 + \lambda \sqrt{x^2 + a^2} \Big\}$$ This approach, however, leads to a quartic equation, thus necessitating numerical root finding with subsequent deliberations as to which of the four roots to keep. I have also tried to take advantage of Moreau decomposition to express ${\rm prox}_f(y)$ in terms of ${\rm prox}_{f^\ast}(y)$, where $f^\ast$ is the convex conjugate of $f$ given by $$f^\ast(y) = \begin{cases}-a \sqrt{\lambda^2 - y^2}, \quad &|y| \le \lambda \\ \infty, \quad &\mbox{otherwise} \end{cases}$$ (if I am not mistaken). Unfortunately, this does not seem to simplify the original problem either. Finally, I have tried a number of numerical approaches (fixed point, localization, guarded Newton). However, every one of them exhibits (painfully) slow convergence when $a$ is relatively small (e.g., $a = 0.01$). So, I still hope there might be a way to find a closed form expression for ${\rm prox}_f(y)$. Any help in this regard would be greatly appreciated. Thanks!

Royi
  • 8,711
A. Coifman
  • 41
  • 2
    Resolution of 4th degree equations is a long resolved issue (and there are exact formulae to give you the answers). You're done! – dohmatob Aug 16 '16 at 15:15
  • I agree with @dohmatob. There's no magic here. I'd go with quartic, but if you're determined to use a numerical method, I would consider a starting point given by the soft thresholding prox, which is equivalent to your $a=0$ case. – Michael Grant Aug 16 '16 at 15:24
  • And, say, there is a double root and two simple roots. Which one am I supposed to retain then to be "done"? – A. Coifman Aug 16 '16 at 15:25
  • Pick the one with the smallest value of the prox. – Michael Grant Aug 16 '16 at 15:25

1 Answers1

2

The function is given by:

$$ f \left( x \right) = \sqrt{ {x}^{2} + {a}^{2} }, \; f : \mathbb{R} \to \mathbb{R} $$

The Prox is given by:

$$ \operatorname{Prox}_{\lambda f \left( x \right)} \left( y \right) = \arg \min_{x} \frac{1}{2} {\left( x - y \right)}^{2} + \lambda \sqrt{ {x}^{2} + {a}^{2} } $$

The Gradient is given by:

$$ x - y + \lambda \frac{ x }{ \sqrt{ {x}^{2} + {a}^{2} } } $$

The equality to zero yields Quartic Function with the form:

$$ {x}^{4} - 2 y {x}^{3} + \left( {a}^{2} + {y}^{2} - {\lambda}^{2} \right) {x}^{2} - 2 {a}^{2} y x + {a}^{2} {y}^{2} $$

It is easy to put it in a solver (Either analytical or numerical) and yield result. Since the solution should be real one should try all real solution into the objective function (The Prox) and choose the one which minimizes the value.

Another method which is valid for the cases $ \left| \frac{\lambda}{a} \right| < 1 $ is using Fixed Point Iteration on the function:

$$ g \left( x \right) = y - \lambda \frac{ x }{ \sqrt{ {x}^{2} + {a}^{2} } } $$

The convergence of the Fixed Point Iteration is guaranteed when $ \left| g' \left( x \right) \right| < 1 $. The derivative is given by:

$$ g' \left( x \right) = \lambda \frac{ {a}^{2} }{ {\left( {a}^{2} + {x}^{2} \right)}^{\frac{3}{2}} } $$. The absolute value of the maximum of the derivative is given by $ \left| \frac{\lambda}{a} \right| $ at $ x = 0 $ hence the condition $ \left| \frac{\lambda}{a} \right| < 1 $.

MATLAB code which implements the method is available at my Mathematics StackExchange Question 1894063 Repository.

Remark: I'm not sure about your dual. As I'm sure the solution should be some kind of thresholding. I tried using your Dual as I did in the Prox Operator of the $ {L}_{2} $ Norm yet it doesn't yield the expected result. If you do get the dual function you should just plug it into what I did.

Royi
  • 8,711