What would be the formal proof that $ f(Ax + b) $ given $ f(x) $ is a convex function ? I got to the point where I expanded $$ f(\lambda(Ax+b) + (1- \lambda)(Ay+b)) = f(A(\lambda x + (1 - \lambda)y) + b) $$ Though I do not know how to proceed to incorporate the convexity of $f(x)$.
Asked
Active
Viewed 4,866 times
3
-
The convexity of $f$ gives you an upper bound of the left hand side. – Daniel Fischer Oct 16 '13 at 21:37
-
True. But how would I bound it to be less that $\lambda f(Ax+b) + (1-\lambda)f(Ay+b) $ – Ammar Husain Oct 16 '13 at 21:42
-
That is exactly what the convexity of $f$ gives you. – Daniel Fischer Oct 16 '13 at 21:44
1 Answers
9
Perhaps the concreteness of writing $f(Ax+b)$ shadows the important matter. Let us simply denote the affine map by $g$, and see how we can deduce that if $f$ is convex, then $f\circ g$ is also convex.
We need to show that for any $x,y$, and $\lambda \in [0,1]$, we have
$$(f\circ g)\left(\lambda x + (1-\lambda)y\right) \leqslant \lambda (f\circ g)(x) + (1-\lambda)(f\circ g)(y).$$
Now, the characteristic property of affine maps is that
$$g\left(tx + (1-t)y\right) = t\cdot g(x) + (1-t)\cdot g(y)$$
for all $x,y$ and $t\in \mathbb{R}$. So we have
$$\begin{align} (f\circ g)\left(\lambda x + (1-\lambda)y \right) &= f\left(g\left(\lambda x + (1-\lambda)y \right) \right)\\ &= f\left(\lambda g(x) + (1-\lambda)g(y) \right)\\ &\leqslant \lambda f\left(g(x)\right) + (1-\lambda)f\left( g(y)\right), \end{align}$$
which is exactly what we need.
Daniel Fischer
- 206,697