0

The minimizing for $ \omega$ is equivalent to solving,for i=1,2,...,$n^2$, $$\min_{w_i}=\|w_i\|_2+\frac{\beta}{2}\|w_i-D_iu\|_2^2 \; \; \; \;\;\;\;(1)$$ Which we first introduce two auxiliary vectors $\omega_1,\omega_2\in \mathcal{R}^{n^2}$ to approximate $D^{(1)}u$ and $D^{(2)}u$,respectively,where we recall that $D^{(1)},D^{(2)}\in\mathcal{R}^{n^2\times n^2}$ and $u \in\mathcal{R}^{n\times n}$.We denote $\omega =(\omega_1;\omega_2) \in \mathcal{R}^{2n^2}$ and $D=(D^{(1)};D^{(2)})\in\mathcal{R}^{2n^2\times n^2}$,for i=1,2,...,$n^2$,we let $\omega_i=((\omega_1)_i;(\omega_2)_i) \in \mathcal{R}^2$,which is an approximation of $D_iu=[(D^{(1)}u)_i;(D^{(2)}u)_i]\in \mathcal{R}^2$.To keep $\omega_i$ close to $D_iu$,we apply a quadratic penalt term $\beta$ .In my opinion,to solve the (1): $$when\; w_i\neq 0$$ $$\min_{w_i}=\sqrt{(\omega_1)_i^2+(\omega_2)_i^2}+\frac{\beta}{2}([(\omega_1)_i-(D^{(1)}u)_i)^2+((\omega_2)_i-(D^{(2)}u)_i)^2)]$$ By derivation then we can get: $$\frac{(\omega_1)_i}{\left \|\omega_i \right \|}+\beta((\omega_1)_i-(D^{(1)}u)_i)=0$$ $$\frac{(\omega_2)_i}{\left \|\omega_i \right \|}+\beta((\omega_2)_i-(D^{(2)}u)_i)=0$$ So it means: $$\frac{\omega_i}{\left \|\omega_i \right \|}+\beta(\omega_i-D_iu)=0$$ $$\omega_i=D_iu-\frac{\omega_i}{\beta\left \|\omega_i \right \|}$$ What let me feel confuse is the paper said the solution for the (1):for which the unique minimizer is given by the following two-demensional(2D) shrinkage fomula: $$\omega_i=max\left \{{\left \|D_iu \right \|-\frac{1}{\beta}},0 \right \}\frac{D_iu}{\left \|D_iu \right \|},i=1,2,...,n^2\; \; \; \;\;\;\;(2)$$ So what is shrinkage formula doing for (1) to (2)?And why I get the different answer and where if I made mistakes?Thanks for your answers.

  • What do you mean by "how the formula worked"? – littleO Jun 22 '17 at 04:43
  • What is the shrinkage formula in this process? – Dajiang Lei Jun 23 '17 at 05:43
  • Check out the first few slides of chapter 8 of Vandenberghe's 236c notes. The shrinkage operator you are asking about in this question is also known as the "proximal operator of the $\ell_2$-norm." http://www.seas.ucla.edu/~vandenbe/236C/lectures/proxop.pdf – littleO Jun 23 '17 at 23:39
  • I edited my question,can you look again?Thank you. – Dajiang Lei Jun 24 '17 at 09:11
  • Are you sometimes writing $w_i$ when you mean $\omega_i$? Is $w_i$ a scalar? If so what does the expression $| w_i |_2$ mean? Are you attempting to solve for $\omega_i$? But you have $\omega_i$ on both the left and right sides of your equation, so you have not solved for $\omega_i$. In any case, there is an error in your approach because you must handle the nondifferentiabity of the $\ell_2$-norm carefully. – littleO Jun 24 '17 at 09:26
  • Sorry about the written error,all $w_i$ mean $\omega_i$ and $\omega_i$ is a vector $\in \mathcal{R}^2$.I think in (1) there should be "argmin",but the paper used the "min". – Dajiang Lei Jun 24 '17 at 09:38
  • I see. The main issue is that you can't simply set the gradient equal to $0$ because the $\ell_2$-norm is not differentiable at the origin. I think it's most clear to use the Moreau decomposition formula to evaluate this proximal operator. – littleO Jun 24 '17 at 09:48
  • The answer given here shows what I think is the best way to evaluate this proximal operator: https://math.stackexchange.com/questions/2167550/how-to-derive-the-proximal-operator-of-the-euclidian-norm/2168391#2168391 – littleO Jun 24 '17 at 09:51

0 Answers0