20

I am reading the book Convex Optimization, and I don't understand why a $\max$ function is convex.

The function is defined as:

$$f(x) = \max(x_1, x_2, \dots, x_n)$$

The book offers the proof shown below:

for $0 \leq \theta \leq 1$

$$\begin{aligned} f(\theta x + (1 - \theta)y) &= \max_i \left( \theta x_i + (1 - \theta)y_i \right)\\ & \leq \theta \max_i x_i + (1 - \theta)\max_i y_i\\ &= \theta f(x) + (1 - \theta)f(y) \end{aligned}$$

However, I don't understand why the following inequality holds.

$$\max_i (\theta x_i + (1 - \theta)y_i) \leq \theta \max_i x_i + (1 - \theta)\max_i y_i$$

Community
  • 1
hklel
  • 533

4 Answers4

30

Fix $k\in \{1,\ldots,n\}$. We have $$\theta x_k + (1-\theta)y_k \leq \theta \max_i x_i + (1-\theta)\max_i y_i$$ because $x_k \leq \max_i x_i$, $y_k \leq \max_i y_i$, $\theta \geq 0$ and $1-\theta \geq 0$.

Since the statement above is true for any $k$ we have:

$$\max_k [\theta x_k + (1-\theta)y_k]\leq \theta \max_i x_i + (1-\theta)\max_i y_i.$$

Hugo
  • 3,775
6

One line proof:

Since composition of convex functions is convex, we only need to show $\max{(x,y)}$ is convex. But $\max(x,y)=\frac{x+y}{2}+|\frac{x-y}{2}|$ and $|\cdot|$ is obviously convex.

Daniel Li
  • 3,200
0

A function $f: \mathbb R^n \to \mathbb R$ is convex if and only if its epigraph $\text{epi} f = \{(x,t) \in \mathbb R^n \times \mathbb R \mid f(x) \le t\}$ is a convex set. For $f(x) = \max(x_1,\dots,x_n)$, we have $$ f(x) \le t \iff x_i \le t, \; \text{for all}\; i=1,\dots,n. $$ The latter set is an intersection of $n$ hyperplanes, hence a convex set. (The intersection of convex sets is a convex set.)

passerby51
  • 4,140
0

$ \forall x_i, y_j \in \mathbb{R} , \forall \space i,j=1,2, $ $$ \begin{align} \max(x_1, x_2) + \max(y_1, y_2) &= \max(x_1 + y_1, x_1 + y_2, x_2 + y_1, x_2 + y_2) \\ &≥ \max(x_i + y_j). \end{align} $$

$ \forall x_i, y_j \in \mathbb{R} , \forall \space i,j=1,...,n, $ $$ \begin{align} \max(x_i) + \max(y_j) &= \max( \underbrace{x_1 + y_1, x_1 + y_2,...,x_n + y_n}_{\text{n×n terms}} ) \\ &≥ \max(x_i + y_j). \end{align} $$

Since $ \max(\theta x) = \theta \max(x) $, $$ \begin{align} \theta \max(x_i) + (1-\theta) \max(y_j) &= \max(\theta x_i) + \max((1-\theta)y_j) \\ &= \max( \underbrace{\theta x_1 + (1-\theta)y_1, \theta x_1 + (1-\theta)y_2,..., \theta x_n + (1-\theta)y_n}_{\text{n×n terms}} ) \\ &≥ \max(\theta x_i + (1-\theta)y_j). \end{align} $$

That is $$ \max \left( \theta x_i + \left( 1 - \theta \right) y_j \right) ≤ \theta \max(x_i) + (1-\theta) \max(y_j). $$

So $\max$ function is convex.

namewl
  • 1