Sufficient conditions for existence and uniqueness global minimum

Question

I have some questions on existence and uniqueness of global minimum in the following problems. Could you help me to understand which conditions are sufficient to guarantee existence and uniqueness?

Problem 1 Let $u$ be a $p\times 1$ vector taking value in $\mathcal{U}\subseteq \mathbb{R}^p$. Let $u^\star$ be another $p\times 1$ vector taking value in $\mathcal{U}$. Let $u_j$ be the $j$-th component of $u$. Let $C$ be a positive definite matrix. Let $\lambda_0\geq 0$.

Let $$ A(u):=(u-u^\star)' C (u-u^\star) + \lambda_0\sum_{j=1}^p|u_j| $$

Consider $$\min_{u\in \mathcal{U}}A(u)$$

Is $A(u)$ strictly convex? How do I show it? Moreover, which conditions on $\mathcal{U}$ are sufficient for existence of $argmin_{u\in \mathcal{U}}A(u)$?

Problem 2 Let $u$ be a $p\times 1$ vector taking value in $\mathcal{U}\subseteq \mathbb{R}^p$. Let $\lambda_n\geq 0$ $\forall n \in \mathbb{N}$. Let $Y$ be an $n\times 1$ vector with $i$-th component $Y_i$. Let $W_i$ be a $p\times 1 $ vector for $i=1,...,n$ with $j$-th component $W_{ij}$.

Let $$ B_n(u):=\sum_{i=1}^n (Z_i-\sum_{j=1}^pW_{ij}u_j)^2+\frac{\lambda_n}{n}\sum_{j=1}^p |u_j| $$

Consider $$\min_{u\in \mathcal{U}}B_n(u)$$

Is $B_n(u)$ strictly convex? Under which conditions? Moreover, which conditions on $\mathcal{U}$ are sufficient for existence of $argmin_{u\in \mathcal{U}}B_n(u)$?

Answer 1

$v \mapsto v^\top C v$ is a strictly convex function since its Hessian $2C$ is positive definite.

The map $u \mapsto (u-u^*)^\top C (u-u^*)$ is then strictly convex, because it is a composition of a linear map $u \mapsto u-u^*$ with the previous strictly convex map.

Since $u \mapsto \lambda_0 |u_j|$ is convex for each $j$ (it is the composition of a linear map $u \mapsto \lambda_0 u_j$ and a convex map $x \mapsto |x|$), your function $A$ is the sum of a strictly convex function with $p$ convex functions, so it is strictly convex.

---

Since $A$ is convex, it will have a minimizer if $\mathcal{U}$ is closed and convex. Uniqueness follows from strong convexity of $A_n$. See this question.

---

[I think you meant to write $Y_i$ instead of $Z_i$.] You can rewrite $B_n$ as $$B_n(u) = \| Y - Wu\|^2_2 + \frac{\lambda_n}{n} \|u\|_1.$$ As mentioned already, $u \mapsto \frac{\lambda_n}{n} \|u\|_1$ is convex since it is the sum of $p$ convex functions $u \mapsto \frac{\lambda_n}{n} |u_j|$.

$\|Y-Wu\|_2^2$ is the composition of a linear map $u \mapsto Y-Wu$ and a strictly convex map $v \mapsto \|v\|_2^2=\sum_{i=1}^n v_i^2$.

Therefore,

1. it is strictly convex on $\mathbb{R}^n$ as long $W$ is not zero. [This is to exclude the case where $u \mapsto Y-Wu$ maps everything to a single point, in which case $u \mapsto \|Y-Wu\|_2^2$ is constant and not strictly convex.]

2. More generally, if you are talking about strict convexity only on $\mathcal{U}$, then this condition generalizes to to "there exist $u,v \in \mathcal{U}$ such that $Wu\ne Wv$."

To conclude, under one of these conditions, the full map $B_n$ is strictly convex on either $\mathbb{R}^3$ or $\mathcal{U}$.

---

Again, there is a minimizer if $\mathcal{U}$ is closed and convex, and it is unique if strong convexity of $B_n$ holds.