3

Let $a_1,\ldots,a_k \in \mathbb R^n$ and consider the convex function $F:\mathbb R^n \to \mathbb R$ defined by $F(w) := \max_{i=1}^k a_i^\top w $.

Question. What is the proximal operator of $F$ ? That is, for $\lambda > 0$, find an a closed-form formual for at $z=z_\lambda(w) \in \mathbb R^n$ which minimizes $\lambda F(z) + (1/2)\|z-w\|^2$.

My attempt

$$ \min_z \lambda F(z) + (1/2)\|z-w\|^2 = \min_z \max_i \lambda a_i^\top z + (1/2)\|z-w\|^2\\ = \min_z \max_{q \in \Delta_{k-1}} \lambda q^\top Az + (1/2)\|z-w\|^2\\ = \max_{q \in \Delta_{k-1}} \min_z \lambda q^\top Az + (1/2)\|z-w\|^2. $$

The inner-most problem has explicity solution $z=w - \lambda A^\top q$. Thus, we would be done if we had a closed-form expression for $q$.

dohmatob
  • 9,535
  • You get $\max_{q \in \Delta_{k-1}} {\lambda q^TAw - 0.5 \lambda^2 q^T AA^T q }$. I cannot think of a closed-form solution. Also see this question. – LinAlg Nov 18 '20 at 23:05
  • Indeed, and that the objective in the above display is strongly convex if $A$ is full rank. Therefore sastifies a Kurdyka-Lojasiewicz inequality of exponent 1/2 around the optimal value of $q$. Therefore can be iteratively (like so https://math.stackexchange.com/a/1311138/168758) with linear convergence. This is perhaps one can get to closed-form solution. – dohmatob Nov 19 '20 at 02:01

0 Answers0