4

I would like the proximal operator of the following function: \begin{align} f(x) &= \lambda_1 \|x\|_1 + \lambda_2 \|x\|_{2,1} \\ &= \lambda_1 \|x\|_1 + \lambda_2 \sum_g \|x_g\|_2 \end{align} where $x$ is the concatenation of all $x_g$, $\|x\|_1=\sum_i|x_i|$, and $\|y\|_2=\sqrt{\sum_i y_i^2}$.

I appreciate any help that is offered.

Royi
  • 8,711
NicNic8
  • 6,951

1 Answers1

3

This is a special case of the more general class of summative proximable functions, i.e. functions of the form

$$ x \mapsto f(x) + g(x) $$ which satisfy $\mathrm{prox}_{f + g}(x) = \mathrm{prox}_{f} \left( \mathrm{prox}_g(x) \right)$. See Appendix D of A Primer on Coordinate Descent Algorithms for more details and a proof.

For your specific instance, notice that since $\| x \|_1 = \sum_{g \in G} \| x_g \|_1$ from your definition of $x$, you can decompose your proximity operator into $G$ separate instances of the form

$$ f_g(x) = \lambda_1 \| x_g \|_1 + \lambda_2 \| x_g \|_2. $$

Each of these is then solvable using the rule for summative functions, i.e.

$$ \mathrm{prox}_{f_g}(x) = \mathrm{prox}_{\lambda_1 \| \cdot \|_1} \big( \mathrm{prox}_{\lambda_2 \| \cdot \|_2}(x) \big). $$

Both of these proximal operators are known in closed form. See Chapter 6 in First Order Methods in Optimization.

Royi
  • 8,711
VHarisop
  • 3,840