2

I'm stuck with proving following inequalities.

Let $X_i$ be independent r.v in [0,1]. Denote $S=X_1+\dots+X_n$.

Note that for any $a\in \mathbb{R}$ and $0\leq x\leq 1$, we have $$e^{ax}\leq 1-x+xe^a \tag{0}$$

  1. How to show the following inequalities, which provide upper bounds for moment-generating functions?

For any $a\in \mathbb{R}$,

$$M_{X_i}(a) := \mathbb{E}[e^{aX_i}]\leq e^{\mathbb{E}[X_i](e^a-1)} \tag{1.1}$$ and $$M_S(a) := \mathbb{E}[e^{aS}]\leq e^{\mathbb{E}[S](e^a-1)} \tag{1.2}$$

assuming each $M_{X_i}(a) := \mathbb{E}[e^{aX_i}]$ exists (and thus $M_S(a)$ exists)

  1. Also, how do I extend this to the following?

For any $0<t<1$,

$$\mathbb{P}(S\geq (1+t)\mathbb{E}[S])\leq(\frac{e^t}{(1+t)^{1+t}})^{\mathbb{E}[S]} \tag{2.1}$$

and

$$\mathbb{P}(S\geq (1-t)\mathbb{E}[S])\leq(\frac{e^{-t}}{(1-t)^{1-t}})^{\mathbb{E}[S]} \tag{2.2}$$

BCLC
  • 13,459
Ronald
  • 103
  • 6

1 Answers1

2

(formerly partial answer. now full answer, I hope).


To prove 1.1:

There's another inequality to use: We have for any $b \in \mathbb R$ that $$1+b \le e^b \tag{A}$$

By $(0)$, $M_X(a) \le 1+(e^a-1)E[X]$. By $(A)$ with $b=(e^a-1)E[X]$, we get $1+(e^a-1)E[X] \le e^{E[X](e^a-1)}$.

Note: I actually got the inequality $(A)$ from here: https://en.wikipedia.org/wiki/Moment-generating_function#Other_properties --> in the part with 'This follows from the simple inequality'.


To prove 1.2 from 1.1:

By independence (independence of $X_i$ implies independence of $e^{aX_i}$ because $f(x)=e^{ax}, f: \mathbb R \to \mathbb R$ is measurable or something. see here or here or my new question here...or use definition of independence in terms of joint density splitting up or something),

$$\mathbb{E}[e^{a(\sum_i X_i)} = \prod_i e^{aX_i}] = \prod_i E[e^{aX_i}]$$

Then 1.1 says $E[e^{aX_i}] \le e^{\mathbb{E}[X_i](e^a-1)}$. Hence, from non-negativity of each $E[e^{aX_i}]$,

$$\mathbb{E}[e^{a(\sum_i X_i)}] \le \prod_i e^{\mathbb{E}[X_i](e^a-1)} = e^{\sum_i \mathbb{E}[X_i](e^a-1)} = e^{E[S](e^a-1)}$$


For 2 (attempt 1):

Not sure Markov's inequality is helpful, but...

Markov's inequality, assuming applicable, gives

$$\mathbb{P}(S\geq (1 \pm t)\mathbb{E}[S])\le \frac{E[S]}{(1 \pm t)\mathbb{E}[S]} = \frac{1}{1 \pm t}$$

Here, Markov's inequality is applicable because $S$, $1 \pm t$ and $\mathbb{E}[S]$ are non-negative, BUT IF $\mathbb{E}[S]$ is nonzero. At this point, I guess let's use that non-negative random variables have zero mean if and only if they are almost surely zero.


For 2 (attempt 2):

I think Markov's inequality is helpful but not for $P(S \ge (1 \pm t)\mathbb{E}[S])$ directly but rather for $P(S \ge (1 \pm t)\mathbb{E}[S]) = P(e^{mS} \ge e^{m[(1 \pm t)\mathbb{E}[S])]})$, for any $m > 0$. We get

$$P(S \ge (1 \pm t)\mathbb{E}[S]) \le e^{-m((1 \pm t)\mathbb{E}[S])} M_S(m) \tag{B}$$

based on (see here) that for any random variable $Z$ (doesn't have to be non-negative) and for any $z$ (doesn't have to be positive) and for any $m > 0$

$$P(Z \ge z) \le e^{-zm} M_Z(m) := e^{-zm} E[e^{mZ}]$$


I think I got it for (2.1):

In $(B)$, choose $m=\ln(1+t)$, which is indeed positive for $0 < t$ (not sure where $t < 1$ is used. maybe this is used more in (2.2)) and then use $(0)$ on $E[e^{mS}]$ with $a=m$:

$$P(S \ge (1 + t)\mathbb{E}[S]) \le e^{(e^m-1-m-mt)E[S]}$$

and then $e^{(e^m-1-m-mt)E[S]}$ is indeed $\le (\frac{e^t}{(1+t)^{1+t}})^{\mathbb{E}[S]}$, which (I hope!) is true if and only if $(e^m-1-m-mt)E[S] \le \mathbb{E}[S] \ln(\frac{e^t}{(1+t)^{1+t}})$, which is true if and only if $e^m-1-m+mt \le \ln(\frac{e^t}{(1+t)^{1+t}} = t - (1+t)\ln(1+t)$


Similarly for 2.2: (maybe related: Borel-Cantelli-related exercise: Show that $\sum_{n=1}^{\infty} p_n < 1 \implies \prod_{n=1}^{\infty} (1-p_n) \geq 1- S$.)

Do $m=ln(1-t)$. Now this uses $t < 1$ (not sure if $0 < t$ is used). Instead of having to prove '$e^m-1-m-mt \le t - (1+t)\ln(1+t)$', we have to prove...either

$e^m-1-m+mt \le - t \pm (1-t)\ln(1-t)$. You double check which of the $\pm$ but regardless I believe we have

$e^m-1-m+mt \le - t - (1-t)\ln(1-t)$. Finally, in case it's the $+$, just use that $- t - (1-t)\ln(1-t) \le - t + (1-t)\ln(1-t)$.


Remark: Remember to prove Markov's inequality, if you haven't in class!

BCLC
  • 13,459
  • 1
    Thanks for your help. I really appreciate it. – Ronald Dec 11 '20 at 11:03
  • @Ronald You're welcome but don't accept my answer (yet) because it's (currently) incomplete! – BCLC Dec 11 '20 at 11:11
  • @Ronald Unless you figured out how to do (2)? – BCLC Dec 11 '20 at 11:11
  • 1
    I'm trying to do.. But I'm not sure – Ronald Dec 11 '20 at 11:13
  • ayt. keep at it. i'm about to edit answer. you can upvote me in the mean time if you want @Ronald – BCLC Dec 11 '20 at 11:13
  • @Ronald edited. full answer now i hope. – BCLC Dec 11 '20 at 11:58
  • @Ronald edited further. – BCLC Dec 11 '20 at 12:26
  • 1
    I think it is right approach. Thank you so much. – Ronald Dec 11 '20 at 12:36
  • you're welcome. you don't have to accept my answer until you're done analysing. it's up to you. btw, stay tuned for any answers I get here: https://math.stackexchange.com/questions/3944284/prove-that-for-independent-random-variables-x-i-we-have-f-ix-i-are-indepe @Ronald – BCLC Dec 11 '20 at 12:40
  • or in your class @Ronald was it proven that if $X_i$ are independent then $e^{aX_i}$ are independent or something? – BCLC Dec 11 '20 at 12:40
  • @Ronald maybe uncheck my answer so you can get more attention for this question. i'm interested also to see what other users have to say about my answer. – BCLC Dec 11 '20 at 12:41
  • @Ronald got update for 1.2 with accepted answer in my other question – BCLC Dec 28 '20 at 05:39