Minimize $\sum_{i=1}^p (y_i-x_i)^2 $ such that $\sum_{i=1}^{p'} y_i^2 - R^{2} \le 0$

Question

I'm solving the following optimization problem. Could you please verify if my proof is correct or contains logical mistake? Thank you so much!

Let $x = (x_1,\ldots,x_p) \in \mathbb R^p$, $p' \le p$, and $R>0$. Solve the optimization problem $$\begin{aligned} \text{min} &\quad \sum_{i=1}^p (y_i-x_i)^2 \\ \text{s.t} &\quad \sum_{i=1}^{p'} y_i^2 - R^{2} \le 0 \end{aligned}$$

---

My attempt:

Clearly, $y_i=x_i$ if $i > p'$. Moreover, if $\sum_{i=1}^{p'} x_i^2 \le R^{2}$, then $y_i = x_i$ for all $i \le p'$.

Next we consider the case $\sum_{i=1}^{p'} x_i^2 > R^{2}$. Let $f(y) = \sum_{i=1}^{p'} (y_i-x_i)^2$ and $g(y)=\sum_{i=1}^{p'} y_i^2 - R^{2}$. Our problem reduces to minimize $f(y)$ with constraint $g(y) \le 0$.

We have $\nabla f(y) =\big [ 2(y_{1}-x_{1}) \cdots 2(y_{p'}-x_{p'}) \big ]^{\intercal}$ and $\nabla g(y) = \big [ 2 y_{1} \cdots 2 y_{p'}\big ]^{\intercal}$. Because both $f,g$ are convex and $g(\mathbf{0})<0$, Slater's condition is qualified. By Karush-Kuhn-Tucker conditions, we have $$\begin{aligned} \begin{cases} \mu &\ge 0 \\ g(y) &\le 0\\ \mu g(y)&=0 \\ \nabla f (y)+\mu \nabla g (y) &=0 \end{cases} &\iff \begin{cases} \mu &\ge 0 \\ \sum_{i=1}^{p'} y_i^2 - R^{2} &\le 0\\ \mu (\sum_{i=1}^{p'} y_i^2 - R^{2})&=0 \\ \begin{bmatrix} 2(y_1-x_1) \\ \vdots \\ 2(y_{p'}-x_{p'})\end{bmatrix} + \mu \begin{bmatrix} 2y_1 \\ \vdots \\ 2 y_{p'}\end{bmatrix} &= \begin{bmatrix} 0 \\ \vdots\\0 \\ \end{bmatrix} \end{cases} \\ & \iff \begin{cases} \mu &\ge 0 \\ \sum_{i=1}^{p'} y_i^2 - R^{2} &\le 0\\ \mu (\sum_{i=1}^{p'} y_i^2 - R^{2})&=0 \\ \forall i \le p': (1+\mu)y_i &= x_i \end{cases} \end{aligned}$$

If $\mu=0$ then $y_i = x_i$ for all $i \le p'$. Then $\sum_{i=1}^{p'} y_i^2 = \sum_{i=1}^{p'} x_i^2 > R^2$. This is impossible. As such, $\mu >0$ and thus $\sum_{i=1}^{p'} y_i^2 = R^{2}$ Hence $\sum_{i=1}^{p'} \left ( \frac{x_i}{1+\mu} \right)^2 = R^2$. So $\mu+1 = \sqrt {\sum_{i=1}^{p'} x_i^2}/R$ and thus $y_i = R x_i/\sqrt {\sum_{i=1}^{p'} x_i^2}$.

To sum up, $y_i = x_i$ for all $i >p'$. For $i \le p'$: $y_i =x_i$ if $\sum_{i=1}^{p'} x_i^2 \le R^{2}$ and $y_i = R x_i/\sqrt {\sum_{i=1}^{p'} x_i^2}$ if $\sum_{i=1}^{p'} x_i^2 > R^{2}$.

Answer 1

As you pointed out the problem reduces to solving the case $p = p'$. By rescaling $x = (x_i)$ and $y = (y_i)$ we can also assume $R=1$. The problem is then $$ \hat x := \arg\min_{y:\; \|y\|_2 \le 1} \|y - x\|_2 $$ This is computing the projection onto the unit $\ell_2$ ball $B_2 = \{y: |y\|_2 \le 1\}$. Intuitively, if $x$ is in the ball, then $\hat x = x$. Otherwise, we scale it in the same direction to put it on the ball: $\hat x = \frac{x}{\|x\|_2}$. Thus, $$ \hat x = \frac{x}{\|x\|_2 \vee 1} = \begin{cases} x & \|x\|_2 \le 1 \\ x/ \|x\|_2 & \|x\|_2 > 1 \end{cases} $$ You can verify this by showing that $\langle y - \hat x, x- \hat x \rangle \le 0$ for any $y \in B_2$ which is a characterization of projection onto convex sets. (Geometrically, $x - \hat x$ is the normal for the tangent hyperplane to $B_2$ at $\hat x$ and the ball lies entirely to one side of the tangent hyperplane.)