1

I'm reading about convex conjugate and its relation to subdifferential.

In order to characterise subgradients we will use the convex conjugate defined below. This is essentially a special case of the Legendre-Fenchel transform we defined in Section 4.2. we recall that the Legendre-Fenchel transform (a.k.a. the convex conjugate) is defined as $$ \varphi^{*}(y)=\sup _{x \in \mathbb{R}^{d}}(x \cdot y-\varphi(x)) . $$ The following proposition characterises the subdifferential.

Proposition 6.4. Let $\varphi$ be a proper, lower semi-continuous, convex function on $\mathbb{R}^{d}$. Then for all $x, y \in \mathbb{R}^{d}$ $$ x \cdot y=\varphi(x)+\varphi^{*}(y) \quad \Leftrightarrow \quad y \in \partial \varphi(x) . $$ Proof. Since $\varphi^{*}(y) \geq x \cdot y-\varphi(x)$ for all $x, y$ we have $$ \begin{aligned} x \cdot y=\varphi(x)+\varphi^{*}(y) & \Leftrightarrow x \cdot y \geq \varphi(x)+\varphi^{*}(y) \\ & \Leftrightarrow x \cdot y \geq \varphi(x)+y \cdot z-\varphi(z) \quad \forall z \in \mathbb{R}^{d} \\ & \Leftrightarrow \varphi(z) \geq \varphi(x)+y \cdot(z-x) \quad \forall z \in \mathbb{R}^{d} \\ & \Leftrightarrow y \in \partial \varphi(x) \end{aligned} $$ which proves the proposition. In fact if $\varphi$ is convex then $\varphi$ is differentiable almost everywhere, hence we have that $\partial \varphi(x)=$ $\{\nabla \varphi(x)\}$ for almost every $x$.

In the proof, the author does not use the lower semi-continuity of $f$ nor its convexity. As such, I feel that the proposition holds for arbitrary proper function. Could you confirm if my understanding is correct?

Analyst
  • 5,637

1 Answers1

0

Fenchel's inequality is TRUE for any function. Since we define $f^{\star }(p)=sup\left\{px-f(x) \right\}$ the inequality follows directly. Your proof is correct but there is no need for lower semicontinuity. However, in order to define a subdifferential we need a CONVEX function. I give an example. Let us define the subdifferential of the concave function $f(x)=-|x|$ at point zero. Then $-|x|-0\geq p(x-0)\Leftrightarrow -|x|\geq px$ for all x and hence for $x=p$. Therefore $-|p|\geq |p|^{2}$ which is true only for $ p=0$! Going back to the inequality defining the subdifferential, we get $-|x|\geq 0$ for all x, which is an obvious contradiction. So we conclude, Fenchel's inequality is true for any function, but the subdifferential is defined ONLY for convex functions! Of course there are other types of subdifferentials as eg Clarke's subgradient which is defined in a completely different way and for locally Lipschitz functions, not necessarily convex! The subdifferential can also be defined as $\partial f(x)=\left\{lim\triangledown f(y) ,\,y\to x \right\} $

  • It’d be nice if the downvoter would explain the reason for the downvote. – littleO Jun 22 '22 at 02:18
  • @LittleO I down-voted. The central assertion that the subdifferential requires convexity just to be defined is false. The subdifferential map has a perfectly sensible definition without convexity (and indeed, there sometimes is an occasion to define it independently of convexity, e.g. if you wish to show that $\partial f(x) \neq \emptyset$ for all $x \in \Bbb{R}^d$ implies convexity of $f$). The argument for this, to find a single non-convex function and a single point at which the subgradient is empty, does not support the thesis (and can even be done for certain convex functions). – Theo Bendit Jun 22 '22 at 03:15
  • @TheoBendit Thanks, enlightening to hear your explanation. – littleO Jun 22 '22 at 03:45
  • @littleO I don't normally like to down-vote without comment, but George and I got into a little bit of a fracas on an earlier question regarding the accuracy of our respective answers, and I didn't want to repeat history. – Theo Bendit Jun 22 '22 at 03:50
  • Oh gotcha. When I read this answer I had a similar thought — the subdifferential can be defined for nonconvex functions and it sometimes is, even if it’s mainly interesting for convex functions. – littleO Jun 22 '22 at 03:52
  • Mr littleO you seem to have a tendency to defy not Mathematics but Logic! If you can find one single counterexample of a nonconvex function where a subdifferential cannot be defined, that is all you need! As I said in my answer there exist other definitions of subgradients (for locally Lipschitz functions not necessarily convex) but these have nothing to do with our question and answer! To put it more clearly, if you claim that all cats are white and I find a grey cat your claim is FALSE! End of story! –  Jun 22 '22 at 06:01
  • @GeorgeTsoutsinos But, not every convex function is subdifferentiable at every point on its domain. Does this mean that convexity is not sufficient to define the subdifferential? Sometimes the subdifferential is just empty at a point. It can even be empty at every point, as is the case in the $-|x|$ example. As a set-valued map, it's still perfectly sensible (if not particularly interesting) when considered for non-convex functions. – Theo Bendit Jun 22 '22 at 16:53
  • I agree Theo. It can be empty at some points of its domain (see Rockafellar). But there is no concave analysis in literature! The reason for assuming convexity is that the epigraph of a convex function is a convex set. epif=$\left{(x,r):r\geq f(x) \right}$ and it has all the nice properties of convex sets. We can get separating hyperplanes from the Hahn-Banach and use them to prove many results. The effective domain is also a convex set. The support function of a convex set is convex. All these useful results are not valid if we abandon convexity! For a nonconvex approach see F.Clarke ! –  Jun 22 '22 at 17:39
  • @GeorgeTsoutsinos Absolutely. The subgradient is not nearly as useful in the non-convex case as it is the convex case, and the Clarke subgradient is a better tool for certain non-convex functions. I still disagree that convexity is necessary just to define the subgradient (as anemic and/or useless at it may be). The assertion that $y \in \partial f(x) \iff x \cdot y = \varphi(x) + \varphi^*(y)$ does hold true for non-convex functions, if you just keep the same definition of $\partial f$ that you use for convex functions. – Theo Bendit Jun 22 '22 at 17:49
  • OK but in this case taking $f(x)=-|x|$ we get $f^{\star}(x^{\star })=+\infty$ for all $x^{\star }$ so the equality is never true and we get an empty subdifferential for all x. What is the point of such consideration? The point of the subdifferential is to COINCIDE with the derivative where the function is differentiable. ie $\partial f(x)=\bigtriangledown f(x) $. This is not true for nonconvex functions. You can verify this by taking $f(x)=-|x|$ and $x_{0}=1$. Then $\partial f(1)$ is empty but $\bigtriangledown f(1)$={-1}. –  Jun 22 '22 at 22:52
  • In the case of that specific function? I don't know. I can't think of a reason why anyone would consider the subdifferential of this function other than as a pedagogical example, or as a creative way of proving that the function is not convex. But, generally speaking, I can see interest in subdifferentials of non-convex functions. For example, did you know that $\partial f(x) \neq \emptyset \implies f^{**}(x) = f(x)$, even when $f$ is not convex? It doesn't really matter if this was or was not what this concept was originally commonly used for. – Theo Bendit Jun 23 '22 at 07:11
  • Good morning Theo. I wonder how this last equality can be true. Since (in my example) $f^{\star \star }=-\infty $ for all x, whereas f is nowhere$ -\infty $??? –  Jun 23 '22 at 07:30
  • @GeorgeTsoutsinos Ah, but as you pointed out, the hypothesis that $\partial f(x) \neq \emptyset$ is not satisfied at any $x$ (for this particular function). It is vacuously true in this case. – Theo Bendit Jun 23 '22 at 08:10
  • Assume that $f$ is ANY concave function. Then -$f$ is convex. Consider the epi(-$f$ ) which is convex and take a point y$ \notin $ cl epi(-$f$ ). There is a separating line ax+b and a can be taken (wlog) as a$\neq$-x$^{\star }$. We clearly have $-f(x)\geq ax+b$ Then $f^{\star }(x^{\star }$)=sup$\left{x^{\star }x-f(x) \right}$ $\geqslant$sup{$ x^{\star }x+ax+b $}=$+\infty $ which implies that this is true for ALL concave functions and all $x^{\star }$. i.e. the subdifferential is empty for ALL concave functions , even differentiable ones! Do you find it useful to define such a thing?? –  Jun 23 '22 at 12:34
  • @GeorgeTsoutsinos Affine functions are also concave, but have non-empty subdifferentials. I don't see why I need to justify why subdifferentials are useful in the case of affine functions (as I said before, they're not). It would be like complaining that the usual derivative is defined for constant functions, even though their derivatives are really boring. We allow the derivative of constant functions because usually it's less tedious and artificial not to tack on exceptions based on aesthetics. So, I'd rather define $\partial f$ for any function, rather than functions that aren't concave. – Theo Bendit Jun 23 '22 at 18:24
  • Affine functions are also CONVEX!! You seem to forget this! The point is to consider non-convex functions (eg strictly concave) as I did and I proved their subdifferential empty ! –  Jun 24 '22 at 07:24
  • @GeorgeTsoutsinos I'm aware that affine functions are convex. I just thought I should point out the class of exceptions to your claim (recall, you said "Assume that $f$ is ANY concave function", not my emphasis). Some non-convex functions do have non-empty subgradients at certain points, e.g. $0 \in \partial f(1)$ where $f(x) = (x^2 - 1)^2$. If every non-convex function had empty subgradients at every point, then I'd be more inclined to take your point, but this is simply not true. – Theo Bendit Jun 24 '22 at 10:29
  • My proof is correct for any concave function which is not convex at the same time! Even if it is strictly concave on some small interval and affine out of this interval my proof is valid. I think that insisting on using convex analysis methods and definitions for non-convex functions is simply a mathematical eccentricity! The only reliable non-convex approach, (we both agree on that) is F.Clarke's subgradients! –  Jun 24 '22 at 13:23
  • "My proof is correct for any concave function which is not convex at the same time!" Yes, absolutely. A concave function with an affine minorant is affine. I'm not sure what you mean by "reliable" here. I gave some reasonable, interesting propositions that rely on subdifferentials being defined for non-convex functions (which, once again, they are), and which don't work when subdifferentials are replaced with Clarke subdifferentials. To artificially limit a definition based on aesthetic concerns or "reliability", ignoring interesting results, is what's genuinely eccentric. – Theo Bendit Jun 24 '22 at 18:21
  • There is no point to continue this minor argument! I have not seen in math literature anything like the definitions you use! (I mean for non-convex functions). And I explained why they don't appear in literature! I don't find the "results" interesting at all! And all the math literature agrees with me!! –  Jun 24 '22 at 20:16
  • @GeorgeTsoutsinos "all the math literature agrees with me!!" See, this is a sloppy claim. You haven't read all the literature, nor even close to it, and you know this. This is the kind of claim made for rhetoric, but has no substance, designed to aid a failed position. Now, the results I have mentioned are interesting; they're the "function" version of the dual representation of closed convex sets as intersections of half-spaces. But, they're also pretty trivial, so you won't find them in a paper. Your assertion that subgradients cannot be defined without convexity is demonstrably false. – Theo Bendit Jun 24 '22 at 21:15
  • 1
    I have most definitely discussed subgradients of non-convex functions with my supervisor, the late Prof. Jon Borwein, who wrote a book on convex functions (actually he wrote a few on convexity and adjacent topics). He would point out that minor points like, $x$ minimises a function if and only if $0 \in \partial f(x)$ works even if $f$ is non-convex. I honestly don't understand why you cannot admit you are wrong on any of the points on which I have proven you wrong. – Theo Bendit Jun 24 '22 at 21:19
  • Please give me a reference (preferably a book, because there are millions of articles), which contains your considerations You probably seem to believe that you see something important that Tyrell Rockafellar and Frank Clarke did'not see! OK any opinion is acceptable! –  Jun 24 '22 at 21:29
  • @GeorgeTsoutsinos "You probably seem to believe that you see something important that Tyrell Rockafellar and Frank Clarke did'not see!" No, I don't think so. I'm not contradicting any of their works. "OK any opinion is acceptable!" Mine is not an opinion. The definition of $\partial f$ involves no convexity whatsoever. This is objectively true. "Please give me a reference (preferably a book, because there are millions of articles), which contains your considerations" I can't, off the top of my head. Are you going to admit you are wrong about any of your errors, or just continue with strawmen? – Theo Bendit Jun 24 '22 at 22:34
  • My dear friend I don't know why you are doing this. Are you trying to provoke me? To give you an excuse to delete my answers or my comments?? OK you are right and everyone else is wrong! Please don't make any further comments because I am not going to answer them! –  Jun 24 '22 at 22:42