16

I would like to know whether the following function $f: \mathbf{R}^4 \to \mathbf{R}$ is concave or not:

$$ f(x_1, x_2, x_3, x_4) = \log(e^{x_1} + e^{x_2}) - \log(e^{x_1} + e^{x_2} + e^{x_3} + e^{x_4}) $$

I tried to check whether the Hessian was negative semi-definite, but did not get anywhere. The Hessian can be written as

$$ \nabla^2 f(x) = \frac{1}{\tilde{Z}^2}(\tilde{Z} \cdot \text{diag}(\tilde{\mathbf{z}}) - \tilde{\mathbf{z}}\tilde{\mathbf{z}}^\intercal) - \frac{1}{Z^2}(Z \cdot \text{diag}(\mathbf{z}) - \mathbf{z} \mathbf{z}^\intercal), $$

where $$ \tilde{\mathbf{z}} = \begin{bmatrix} e^{x_1} & e^{x_2} & 0 & 0 \end{bmatrix}^\intercal \\ \mathbf{z} = \begin{bmatrix} e^{x_1} & e^{x_2} & e^{x_3} & e^{x_4} \end{bmatrix}^\intercal \\ \tilde{Z} = e^{x_1} + e^{x_2} \\ Z = e^{x_1} + e^{x_2} + e^{x_3} + e^{x_4} $$ but I did not get much further than that. Any help would be greatly appreciated!

This question discusses in detail the convexity of the log-sum-exp function, but does not apply to my case (difference of fcts).

Community
  • 1
lum
  • 376
  • 3
    Fixing $x_2=x_3=x_4=0$, we get the function $\log(e^x+1)-\log(e^x+3)$. Take the second derivative, simplify, see that it changes sign. –  Jun 28 '16 at 11:30
  • 2
    @Bookend, thanks, that's simple & clear! Would you like to write a proper answer for it? I'd be glad to accept it. – lum Jun 29 '16 at 09:22
  • 13
    @Bookend, I asked a question on meta about this: http://meta.math.stackexchange.com/questions/23554 - would love to get more explanations from you why you think this is a bad question. – lum Jun 29 '16 at 12:10
  • 12
    @Bookend This question is of perfect use to others. I, for one, found it very informative. – The Count Jun 29 '16 at 21:04

1 Answers1

7

(This answer builds up on the first comment to the question above).

Short answer: no, the function is not concave.

Instead of analyzing the full $4 \times 4$ hessian, we can start by restricting our attention to a subspace of the input, e.g., the line induced by setting $x_2 = x_3 = x_4 = 0$. Along this line, the original function can be rewritten as a univariate function $\tilde{f}(x) = \log(e^x + 1) - \log(e^x + 3)$.

the second derivative of $\tilde{f}$ is $$ \frac{d^2\tilde{f}}{dx^2} = \frac{e^x(6 - 2 e^{2x})}{(e^x + 1)^2 (e^x + 3)^2} $$ it is easy to see that the second derivative is positive for $x = 0$, hence $\tilde{f}$ is not concave (furthemore, it is negative for $x = 1$, so it is not convex either). We conclude that $f$ is not concave.

Note that the type of functions described here (difference of log-sum-exps) appears in the log-likelihood function of certain statistical models of paired comparisons, such as Elimination by Aspects and team comparisons.

lum
  • 376