Proving $\max\{f,g\}$, $\sum x_i^2$ and $e^{f(x)}$ are convex functions

Question

Verify if the functions below are convex.

a) $f(x) = \max \{g(x),h(x)\}$ where $h$ and $g$ are convex.

b) $t(x) = \sum_{i=1}^n x_i^2$

c) $s(x) = e^{f(x)}$, $f:\mathbb{R}^n\to\mathbb{R}$

a)

Suppose without loss of generality that $$\max\{g(\lambda x + (1-\lambda)y), h(\lambda x + (1-\lambda)y)\} = g(\lambda x + (1-\lambda)y)$$

then

$$f(\lambda x + (1-\lambda)y) = \max\{g(\lambda x + (1-\lambda)y), h(\lambda x + (1-\lambda)y)\}=\\g(\lambda x + (1-\lambda)y) \le \lambda g(x) + (1-\lambda) g(y) \le \lambda \max\{g(x),h(x)\} + (1-\lambda) \max\{g(y),h(y)\}=\\\lambda f(x) + (1-\lambda) f(y)$$

therefore $f$ is convex

b) $$t(\lambda x + (1-\lambda)y) = \sum_{i=1}^n(\lambda x_i + (1-\lambda) y_i)^2 \le \sum_{i=1}^n\lambda^2x_i^2 + (1-\lambda)^2 y_i^2\le^* \\\sum_{i=1}^n\lambda x_i^2 + (1-\lambda)y_i^2 = \lambda f(x) + (1-\lambda)f(y)$$

• because $\lambda\in[0,1]$ so $\lambda^2 \le \lambda$

so this function is also convex

c) there are no assumptions about $f$ so I'm gonna consider the cases where it is convex and where it is not.

If $f$ is convex:

$$s(\lambda x + (1-\lambda)y) = e^{f(\lambda x + (1-\lambda) y)}\le e^{\lambda f(x) + (1-\lambda)f(y)} = e^{\lambda f(x)}e^{(1-\lambda)f(y)}$$

I don't think it is less than $\lambda e^{f(x)} + (1-\lambda) e^{f(y)}$. So let's try to find an example of convex function such that

$$e^{\lambda f(x)}e^{(1-\lambda)f(y)}> \lambda e^{f(x)} + (1-\lambda) e^{f(y)}$$

for some $\lambda$. If we pick $\lambda=\frac{1}{2}$, $f(x) = |x|$, we must find $y$ such that

$$e^{\frac{1}{2} |x|}e^{\frac{1}{2}|y|}> \frac{1}{2} e^{|x|} + \frac{1}{2} e^{|y|}$$

for some $x$

I don't think there's a solution for this inequality, at least for this $\lambda$. I also tried $f(x)=x^2$ but coulnd't find anything.

For arbitrary $f$ (not necessairly convex), we should search for an inequality

$$e^{f(\lambda x + (1-\lambda) y)}> \lambda e^{f(x)} + (1-\lambda)e^{f(y)}$$

pick $\lambda = \frac{1}{2}$: $$e^{f(\frac{1}{2} x + \frac{1}{2} y)}> \frac{1}{2} e^{f(x)} + \frac{1}{2}e^{f(y)}$$

lets pick $x=0,y=1$ and try to find some $f$ such that

$$e^{f(\frac{1}{2})}> \frac{1}{2} e^{f(0)} + \frac{1}{2}e^{f(1)}$$

I could define a picewise function that would make this not be valid but I'm focusing on continuous $f$. I'm also having a hard time finding a funcion that satisfies this. Maybe the function $e^{f(x)}$ is indeed convex for continuous $f$ but I didn't notice.

Answer 1

(a) looks okay.

(b) You can not say $$\sum_{i=1}^n(\lambda x_i + (1-\lambda) y_i)^2 \le \sum_{i=1}^n\lambda^2x_i^2 + (1-\lambda)^2 y_i^2$$ unless you're sure that $x_iy_i >0$ for all $i$. For this problem I'll give you a hint.

HINT: Sum of convex functions is convex

(c) In general it doesn't hold, unless $f(x)$ is convex See here why. Counterexample let $n = 1$ and then the double derivative: $$g'(x)= f'(x)e^{f(x)}$$ $$g''(x) = (f''(x) + [f'(x)]^2)e^{f(x)}$$ Choose $f(x) = \sin x$ then $f'(x) = \cos x$ and $f''(x) = -\sin x$ hence $$f''(x) + [f'(x)]^2 = -\sin x + \cos^2 x$$ At $x = 0$, $g''(x) > 0$ and at $x = \frac{\pi}{2}$ we have $g''(x) <0$. Hence, the function could not be convex.

Answer 2

a) quite OK. b) Where do you have $2\lambda(1-\lambda)x_iy_i$? The convexity of a function in question follows directly by convexity of a quadratic function of one variable. c) for convex $f$ this is a composition of an increasing convex function (exponential) and the convex one. So it is convex. If $f$ is nonconvex it is false. For example the Gauss function $e^{-x^2}$ is of course non-convex.