0

I want to show that trying to find the minimum of the sum of two or more functions of two different groups is a not convex problem. For example: $ \min\limits_{Y,Z} f(X,Y,Z)=...$. Moreover the values $X,Y,Z$ are matrices. My idea is to show that the Hesse-Matrix of the sum of those added functions is not always positive semidefinit by finding a point x, where it is not psd?
$\operatorname{H}_f({Y},{Z})= \left(\frac{\partial^2f}{\partial c_i\partial c_j}({Y},{Z})\right) \begin{pmatrix} \frac{\partial^2 f}{\partial c_1\partial c_1}({X},{Y})&\frac{\partial^2 f}{\partial c_1\partial c_2}({X},{Y})&\cdots&\frac{\partial^2 f}{\partial c_1\partial c_n}({X},{Y})\\[0.5em] \frac{\partial^2 f}{\partial c_2\partial c_1}({X},{Y})&\frac{\partial^2 f}{\partial c_2\partial c_2}({X},{Y})&\cdots&\frac{\partial^2 f}{\partial c_2\partial c_n}({X},{Y})\\ \vdots&\vdots&\ddots&\vdots\\ \frac{\partial^2 f}{\partial c_n\partial c_1}({X},{Y})&\frac{\partial^2 f}{\partial c_n\partial c_2}({X},{Y})&\cdots&\frac{\partial^2 f}{\partial c_n\partial c_n}({X},{Y}) \end{pmatrix}$
with $c_i \in X,Y$, for example $c_i=x_{11}$ Then I pick one $X,Y,Z$ and show the the eigenvalues are $<0$. P.S. log is elementwise

N8_Coder
  • 61
  • 1
  • 7
  • Why should the sum of non-convex functions be convex in general? I don't understand your question. – user251257 Aug 21 '15 at 15:08
  • I just want to proof that there is no combination for the sum of my non convex function and some of my other functions so that the result is convex. (I think it can happen that the sum of a non convex and another non convex or convex function might be convex) – N8_Coder Aug 21 '15 at 15:30
  • if you only want to show that your example is not convex, please edit your question accordingly. – user251257 Aug 21 '15 at 15:33
  • No, it is not only for the example, my functions are much more difficults. I deleted it in the question ;) – N8_Coder Aug 21 '15 at 15:38
  • Well, but for your question, it is only about your example isn't it? Notice that there are some sums of non convex function which are convex, and some which are non convex. So in full generality your question makes little sense to me. – user251257 Aug 21 '15 at 15:44
  • @user251257 I don't know what you mean by my example. But is my way of prooving ok if I apply it on my functions? On of them is for example: $||X-YZ||^2+\sum (X_{i,j}-X_{i,j-1})^2$. – N8_Coder Aug 21 '15 at 19:13
  • yes. the method is okay, that is, showing that the hessian is not everywhere positive semidefinite. However, you could have it cheaper. In general product of variables is not convex, nor the square of a function that has a sign change. – user251257 Aug 21 '15 at 19:22
  • Thanks! I would give you a vote up, if you would answer/describeit in a seperate comment below. I think I will proove it elaborately, because the functions are quite complex. – N8_Coder Aug 22 '15 at 11:43
  • oh I am really sorry. I misunderstood your question. Sorry for the confusion. Usually the supremum of convex functions is convex. Infimum of non convex function can be convex or non convex. Also in general, the infimum of $C^2$ functions need not to be $C^2$. The hessian of the infimum agrees with the hessian of one of the functions only under very specific conditions. Sorry again. – user251257 Aug 22 '15 at 15:29
  • Confused. I have just some functions where I want to proof if they are convex or not. Therefore I want to divide them into their summands, build the hessian matrix, add all hessian matrices and try to find a point where it is not psd. Not a good way? Then, finding the minimum of a convex function is easy, if non convex it is hard. I do not understand what you want to tell me with your last post, sorry! – N8_Coder Aug 23 '15 at 07:40
  • hm. your post makes the impression that you want to know if $g(X) = \min_{Y,Z} f(X,Y,Z)$ is convex. – user251257 Aug 23 '15 at 12:30
  • that's right. But min_{Y,Z} f(X,Y,Z) is convex/non-convex, if f(X,Y,Z) is convex/non-convex, isn't it? This Problem is very similar: http://math.stackexchange.com/questions/393447/why-does-the-non-negative-matrix-factorization-problem-non-convex And there it is prooven if f(X,Y,Z) is non convex to show that the problem is non convex. – N8_Coder Aug 23 '15 at 18:08
  • The referred question was about the convexity of $f$. So the answer uses the Hessian characterization. – user251257 Aug 23 '15 at 21:18
  • no, the referred question from the link was about the convexity of $min_{Y,Z} f(X,Y,Z)$ – N8_Coder Aug 23 '15 at 22:03
  • the question was whether the minimization problem $\min_{Y,Z} f(X,Y,Z)$, for any $X$, is convex. By definition, the objective function $(Y,Z)\mapsto \phi_X(Y,Z) = f(X, Y,Z)$, for any $X$, needs to be convex. Thus, the answer shows that the hessian of $\phi_X$ is not positive semidefinite. However that says nothing about whether $g(X) = \min_{Y,Z} f(X,Y,Z)$ as function in $X$ is convex. What do you really want to know? – user251257 Aug 23 '15 at 22:17
  • Okay, it is becoming more clear to me. Thank you so much for your patience!! I am thinking about the first case you mentioned. The only difference is, that my function $f(X,Y,Z)$ is the sum of many other functions, for example: $f(X,Y,Z)=g(X,Y)+h(Z)+...$. If I calculate the Hessian of all those "subfunctions" $g,h,...$ add them up and find a pair of $X,Y,Z$, which creates negative Eigenvalues of the summed Hessian Matrix it is prooven, that the problem is not convex? – N8_Coder Aug 23 '15 at 22:47
  • yes. exactly. ${}$ – user251257 Aug 23 '15 at 23:02

0 Answers0