Logistic function as "difference of convex functions" (DC)

Question

is there a way to express the logistic function $$\frac{1}{1+\exp(-x)}$$ as the difference of two convex functions?

Thanks

Can you provide some background for this question? Although your question is a good question, there is an aversion to questions in which no effort is shown. If you provide some background or, if this is a homework question, your work so far, your question may not be closed, or if it is closed, it may be able to be reopened. — robjohn, Feb 24 '14 at 10:49
To those who have voted to close: please let the author know why you have voted to close and what they can do to improve their question. If they are new to the site, they may not know about the close votes. — robjohn, Feb 24 '14 at 10:52

score 2 · Answer 1 · answered Feb 23 '14 at 19:14

2

$$ \frac{d^2}{dx^2} (1+e^x)^{-1} = \frac{d}{dx} \left(-(1+e^x)^{-2}e^x\right) = e^x\left( -(1+e^x)^{-2}+2(1+e^x)^{-3}e^x \right) $$ $$ = \frac{e^x (-(1+e^x)+2e^x)}{(1+e^x)^3} = \frac{ e^x(e^x-1) }{(1+e^x)^3}. $$ This is a bounded function because it is everywhere continuous and goes to $0$ as $x\to\pm\infty$.

So let $f(x) = Ax^2 + \dfrac{1}{1+e^x}$ with $A$ big enough so that the second derivative $f''$ is always positive. Then the logistic function is the difference between the convex function $f$ and the convex function $x\mapsto Ax^2$.

answered Feb 23 '14 at 19:14

Michael Hardy

1

Thanks, but I meant if there was a "fundamental" way. Let me explain.
For instance, consider $$g(x) \triangleq \frac{1}{t} (x)^+$$, where $t>0$ and $(\cdot)^+$ denotes the positive part. We have something similar to the logistic function by $g(x+\frac{t}{2})-g(x-\frac{t}{2})$.

Is there a similar way to write the logistic (perhaps as difference of two exponential functions, just guessing)?

Thanks
– TheDon Feb 23 '14 at 19:29
This certainly answers the question. Did you compute the minimum of $A$? – robjohn Feb 24 '14 at 16:46

score 1 · Answer 2 · answered Feb 23 '14 at 19:24

A different approach.

If $f$ is any function with continuous second derivative let $$ f''_+=\max(f'',0)\quad f''_-=-\min(f'',0). $$ Then $f''_+$ and $f''_-$ are continuous, non-negative and $f''=f''_+-f''_-$. Now let $F_+$ and $F_-$ be such that $F_+''=f''_+$ and $F_-''=f''_-$. $F_+$ and $F_-$ are convex, and the constants of integration can be chosen so that $f=F_+-F_-$.

This is analogous to writing any $C^1$ function as the difference of two increasing functions.

score 0 · Accepted Answer · answered Feb 23 '14 at 20:04

A convex function has an increasing derivative and we can write any function of bounded variation as a difference of two increasing functions. So let us look at the derivative $$ \begin{align} \frac{\mathrm{d}}{\mathrm{d}x}\frac1{1+e^{-x}} &=\frac{e^{-x}}{(1+e^{-x})^2}\\ &=\frac1{(e^{x/2}+e^{-x/2})^2}\\ &=\tfrac14\,\mathrm{sech}^2(x/2) \end{align} $$ Thus, we can break this up into the difference of two increasing functions: $$ f'(x)=\left\{\begin{array}{l} \tfrac14&\text{when }x\ge0\\ \tfrac14\,\mathrm{sech}^2(x/2)\hphantom{-4}&\text{when }x\lt0 \end{array}\right. $$ and $$ g'(x)=\left\{\begin{array}{l} \tfrac14-\tfrac14\,\mathrm{sech}^2(x/2)&\text{when }x\ge0\\ 0&\text{when }x\lt0 \end{array}\right. $$ Then if $$ f(x)=\left\{\begin{array}{l} \tfrac14x+\tfrac12&\text{when }x\ge0\\ \tfrac12\tanh(x/2)+\tfrac12&\text{when }x\lt0 \end{array}\right. $$ and $$ g(x)=\left\{\begin{array}{l} \tfrac14x-\tfrac12\tanh(x/2)&\text{when }x\ge0\\ 0&\text{when }x\lt0 \end{array}\right. $$ $f$ and $g$ are convex and $f(x)-g(x)=\frac12+\frac12\tanh(x/2)=\dfrac1{1+e^{-x}}$

Logistic function as "difference of convex functions" (DC)

3 Answers3