1

How would one find the dual of the following problem?

$min_x 1/2 ||y-x||_2^2 + \lambda||x||_1 $

Can someone please explain to me how to do this since there are no specific constraints?

John
  • 333

1 Answers1

1

You could reformulate your problem as \begin{align*} \operatorname*{minimize}_{x,u} &\quad \frac12 \|u\|_2^2 + \lambda \|x\|_1 \\ \text{subject to} &\quad u = y - x. \end{align*}

The Lagrangian is \begin{align*} L(x,u,z) &= \frac12 \|u\|_2^2 + \lambda \|x\|_1 + \langle z,y-x - u \rangle \\ &= \frac12 \|u\|_2^2 - \langle z,u \rangle + \lambda \|x\|_1 - \langle z,x \rangle + \langle z,y\rangle. \end{align*} The dual function is \begin{align*} g(z) &= \begin{cases} -\frac12 \| z \|_2^2 + \langle z,y \rangle \quad \text{if } \|z\|_{\infty} \leq \lambda \\ -\infty \quad \text{otherwise}. \end{cases} \end{align*}

The dual problem is \begin{align*} \operatorname*{maximize}_z & \quad -\frac12 \| z \|_2^2 + \langle z,y \rangle \\ \text{subject to} &\quad \|z\|_{\infty} \leq \lambda. \end{align*}

littleO
  • 51,938
  • I understand. Thanks. How would I find an explicit solution to this problem.

    If I take the derivative and set to $ 0 $ I get the following:

    $ ||z||_2 = -y $ Is that it?

     – John Oct 27 '13 at 22:47
  • The dual problem is equivalent to minimize $|z - y|2^2$ subject to $|z|{\infty} \leq \lambda$. So $z^$ is the projection of $y$ onto $\lambda B$, where $B$ is the infinity norm unit ball. From the KKT conditions we can obtain $u^ = z^$ and $x^ = y - \text{proj}_{\lambda B}(y)$, which is the standard result for the prox operator of the $1$-norm. A different (and simpler) way to solve the primal problem explicitly can be found here. – littleO Oct 28 '13 at 00:25
  • Since we are dealing with norms, do you think you can help with this one: http://math.stackexchange.com/questions/540020/duality-and-convex-functions ?

    It's not my question but I am quite interested in how to prove this.

     – John Oct 28 '13 at 02:02