1

I'm reading about optimization problems involving mixed norms. In particular I'm getting acquainted with the $\ell_{2,1}$ norm. For a matrix $\mathbf{X}$, the $\ell_{\alpha,\beta}$ norm, $\|\mathbf{A}\|_{\alpha,\beta}$ is defined as,

$$\|\mathbf{A}\|_{\alpha,\beta} = \big( \sum_i \|\mathbf{A}_i\|_\alpha^\beta \big)^{\frac{1}{\beta}}$$

where, $\mathbf{A}_i$ is the $i^{th}$ column of $\mathbf{A}$. According to this definition, $\mathbf{A}_{2,1}$ can be written as,

$$\|\mathbf{A}\|_{2,1} = \sum_i \|\mathbf{A}_i\|_2$$

Now my question is, why is the $\ell_{2,1}$ norm non-smooth? As far as I understand, smoothness of a function is related to continuous differentiability. Is there any reason that even if the $\ell_2$ norm is differentiable a sum of $\ell_2$ norms is non-differentiable. So in short, my question is, why $\ell_{2,1}$ norm is non-smooth? Can it be proved to be a non-smooth function? Will appreciate your help in understanding these concepts.

Royi
  • 8,711
A Das
  • 225

1 Answers1

1

The definition as given by Yale Chang actually states $$\|\mathbf{M}\|_{\alpha,\beta} = \left( \sum_i \|m_i\|_\alpha^\beta \right)^{\frac{1}{\beta}} = \left( \sum_i \left( \sum_j \|\mathbf{M}_{i,j}\|^\alpha \right)^\frac{\beta}{\alpha} \right)^\frac{1}{\beta}.$$

Accordingly, the $L_{1,2}$-norm goes as follows.

$$\|\mathbf{M}\|_{2,1} = \sum_i \left( \sum_j \mathbf{M}_{i,j}^2 \right)^{1/2},$$

$$\frac{\partial}{\partial m_{ij}} \|\mathbf{M}\|_{2,1} = \mathbf{M}_{i,j} \left( \sum_k \mathbf{M}_{i,k}^2 \right)^{-1/2}.$$ Which diverges if any row is zero, and thus $L_{1,2}$ is not differentiable. A similar question and great answer from the awesome Michael Grant can be found here.

Tobias Gurdan
  • 11