Let $X$ be a random variable, $\frac{\mathbb{E}[e^{sX}-e^{tX}]}{s-t}$ for $s \approx t$. As

Question

Question

Let $X$ be a random variable for which we only have the value of its Moment Generating Function $M_X$ on a discrete set of points, I am looking for a stable method to compute: $$\frac{M_X(s) - M_X(t)}{s-t}=\frac{\mathbb{E}[e^{sX}-e^{tX}]}{s-t}, \qquad s \approx t \approx 0.$$

Thoughts/ Attempts

The problem is, clearly, that both the numerator as the denumerator becomes small as $|s-t| \rightarrow 0$. We can however use the Taylor expansion of $e^{sX}$ to overcome this problem, indeed we have (with $ \gamma_n(x,y) = \sum_{m=0}^{n-1} x^m y^{n-1-m} $): \begin{align*} \frac{\mathbb{E}[e^{sX}-e^{tX}]}{s-t} &= - \sum_{n=1}^\infty \frac{\gamma_{n-1}(s, t)}{n!} \left( \frac{d^n M_X}{ds^n} \right)\bigg|_{s=0}. \end{align*} This resolves our original problem. However it introduces another problem, namely the computation of: $\left( \frac{d^n M_X}{ds^n} \right)\bigg|_{s=0}$, which again causes numerical instability. I have tried to stabilize this differentiation. I see no method to do this (except by applying Cauchy's integral formula, which can not be applied here as we only know the value of $M_X(s)$ for real values $s$). Maybe we can rewrite this formula again in function of $M_X(s)$ but I am not sure how.

I have also implemented the suggestions found here but high order derivatives unfortunately can not be computed this way.

Background/Reason For Question

I am using this quantity in a recursion and numerical errors introduced by it make the recurrence fail. This happens after $10-20$ steps, some precision is lost in each subsequent step and this inaccuracy explodes..

Answer 1

The fact that this function is the moment generating function of a random variable is really irrelevant here. Ultimately, you have a function $M_X$ and you are trying to find its slope between the points $s$ and $t$:

$$\text{Slope of } M_X \text{ between } s \text{ and } t = \frac{M_X(s)-M_X(t)}{s-t}.$$

You say that you only have the values of this function $M_X$ at a discrete set of points, and it is unclear from your question whether there is any impediment to calculating the function, or its derivatives, at other points. If the points $|s-t|$ is small, and you are having trouble computing the function at these points, then the most obvious thing to do is to approximate the slope of the function between these points by its derivative at the closest point you have (ideally some point between $s$ and $t$). That is, take some point $s < p < t$ and use the approximation:

$$\frac{M_X(s)-M_X(t)}{s-t} \approx M_X'(p) \quad \quad \quad \text{for } s < p < t.$$

(Of course, this depends on the function $M_X$ being differentiable, and assumes that you can calculate the derivative of this function and compute it at a desired point.) As $|s-t| \rightarrow 0$ this approximation converges to exactness, since the limit of the slope as the points move together defines the derivative of the function. You could of course use higher-order derivatives to improve the approximation (via Taylor series) but if your points are close together then the first-derivative should already give you a good approximation to the actual slope (assuming the function is continuously differentiable, etc.).