A demonstration of Lagrange's Form for the Remainder of a Taylor Series

Question

The following argument for Lagrange's Form for the Remainder of a Taylor polynomial is a typical one in analysis books. Even in the case of finding the remainder when the Taylor polynomial is a linear polynomial, deciding on the functions $g(x)$ and $h(x)$ is not apparent. (See the following argument.) Can someone provide the motivation for these functions? Also, does anyone know who concocted this argument?

Lagrange's Form for the Remainder

$f$ is a twice differentiable function defined on an interval $I$, and $a$ is an element in $I$ distinct from any endpoints of $I$. For every real number $x \in I$ distinct from $a$, there is a real number $c$ between $a$ and $x$ such that \begin{equation*} R_{1}(x) = \frac{f^{(2)}(c)}{2!} \, (x - a)^{2} . \end{equation*} So, if $T_{1}$ denotes the linear Taylor polynomial of $f$ at $a$, \begin{equation*} f(x) = T_{1}(x) + \frac{f^{(2)}(c)}{2!} \, (x - a)^{2} . \end{equation*}

Demonstration \begin{equation*} g(u) = f(x) - \Bigl[f(u) + f^{\prime}(u)(x - u)\Bigr] \end{equation*} is a differentiable, and \begin{equation*} g^{\prime}(u) = - f^{\prime\prime}(u)(x - u) . \end{equation*} The function \begin{equation*} h(u) = g(u) - g(a) \left(\frac{1}{x - a}\right)^{2} (x - u)^{2} \end{equation*} is a also differentiable, and \begin{equation*} h^{\prime}(u) = - f^{\prime\prime}(u)(x - u) + 2g(a) \left(\frac{1}{x - a}\right)^{2} (x - u) . \end{equation*} Moreover, $h(x) = g(x) = 0$, and $h(a) = 0$. According to Rolle's Theorem, there is a real number $a < c < x$ such that $h^{\prime}(c) = 0$. \begin{equation*} 0 = h^{\prime}(c) = - f^{\prime\prime}(c)(x - c) + 2g(a) \left(\frac{1}{x - a}\right)^{2} (x - c) , \end{equation*} or equivalently, since $x \neq c$ and since \begin{equation*} g(a) = f(x) - \Bigl[f(a) + f^{\prime}(a)(x - a)\Bigr] , \end{equation*} \begin{equation*} f(x) - \Bigl[f(a) + f^{\prime}(a)(x - a)\Bigr] = \frac{f^{\prime\prime}(c)}{2!} \, (x - a)^{2} . \end{equation*}

Answer 1

I don't have the historical perspective on Taylor's series so I can't say "who concocted this argument". But I can provide an intuitive justification for the technique used in the proof of Taylor's Theorem.

First of all note that the Taylor's theorem main goal is to express the function $f(x)$ as a power series in powers of $(x - a)$ where $a$ is a suitable point where $f$ behaves in a very good manner. By good manner we mean here that the the function $f$ has derivatives upto a certain order in the neighborhood of the point $a$. Thus if we assume $$f(x) = a_{0} + a_{1}(x - a) + a_{2}(x - a)^{2} + \cdots$$ and assume that the series above can be differentiated repeatedly then we get the coefficients as $$a_{n} = \frac{f^{(n)}(a)}{n!}$$ Thus if $f$ is differentiable $n$ times in the neighborhood of $a$ we can consider the Taylor polynomial $$T_{n}(x, a) = f(a) + f'(a)(x - a) + \cdots + \frac{f^{(n - 1)}(a)}{(n - 1)!}(x - a)^{n - 1}$$ and our hope is that this Taylor polynomial is a good approximation of $f(x)$ in the neighborhood of $a$. And the error in approximation is $$R_{n}(x, a) = f(x) - T_{n}(x, a)$$ Finding an explicit expression of $R_{n}(x, a)$ in terms of $f, n, a, x$ is the real crux of Taylor's Theorem.

And yes this part is tricky. Suppose we used the Taylor polynomial $T_{n + 1}(x)$ instead of $T_{n}(x)$. The difference between $T_{n}$ and $T_{n + 1}$ is the extra term of type $a_{n}(x - a)^{n}$. Since we actually don't use $T_{n + 1}$ but rather $T_{n}$ so the expectation is that the Remainder should behave almost like the extra term $a_{n}(x - a)^{n}$ and hence it makes sense to analyze the express $$\frac{R_{n}(x, a)}{(x - a)^{n}}$$ The tricky part now is to fix the variable $x$ in the above expression and consider it as a function $a$. And then we change the notation slightly to $$\frac{R_{n}(x, u)}{(x - u)^{n}}$$ where $u$ lies between $a$ and $x$ and compare this error with maximum error $$\frac{R_{n}(x, a)}{(x - a)^{n}}$$ and thus our final function is $$g(u) = \frac{R_{n}(x, u)}{(x - u)^{n}} - \frac{R_{n}(x, a)}{(x - a)^{n}}$$ and we want to get rid of $(x - u)^{n}$ in denominator (to simplify calculation) so we instead use the function $$F(u) = (x - u)^{n}g(u) = R_{n}(x, u) - \left(\frac{x - u}{x - a}\right)^{n}R_{n}(x, a)$$ Doing this achieves the important goal $F(a) = F(x) = 0$ so that we can apply Rolle's Theorem to get $F'(c) = 0$ for some $c$ between $a, x$. The expression for $F'(u)$ contains the expression $R_{n}(x, a)$ and the equation $F'(c) = 0$ allows us to expression $R_{n}(x, a)$ in terms of $x, n, a$ (although this is somewhat indeterminate because the exact value of $c$ is not known).

---

Above technique is based on differential calculus. There is another simpler way to look at it if we use integration. The expression $R_{n}(x, a)$ seen as a function of $x$ is such that all its derivatives upto order $(n - 1)$ vanish at $x = a$ and $R_{n}^{(n)}(x, a) = f^{(n)}(x)$. Thus $R_{n}(x, a)$ is the $n^{\text{th}}$ anti-derivative of $f^{(n)}(x)$ and moreover its first $(n - 1)$ derivatives vanish at $x = a$. Under these circumstances it is easy to prove via integration by parts (and induction on $n$) that $$R_{n}(x, a) = \frac{1}{(n - 1)!}\int_{a}^{x}(x - t)^{n - 1}f^{(n)}(t)\,dt$$ Putting $t = x + (a - x)u$ we get $$R_{n}(x, a) = \frac{(x - a)^{n}}{(n - 1)!}\int_{0}^{1}u^{n - 1}f^{(n)}(x + (a - x)u)\,du$$ The integral can be approximated using mean value theorems for integrals to get $$R_{n}(x, a) = \frac{(x - a)^{n}}{(n - 1)!}f^{(n)}(c)\int_{0}^{1}u^{n - 1}\,du$$ i.e. $$R_{n}(x, a) = \frac{(x - a)^{n}}{n!}f^{(n)}(c)$$ where $c$ is some point between $a$ and $x$.

Both the approaches can be modified in a simple manner to give rise to Cauchy's form of remainder. Also note that the approach based on differential calculus assumes that $f^{(n)}(x)$ exists in some neighborhood of $a$ whereas the approach based on integration requires that $f^{(n)}(x)$ is continuous on some neighborhood of $a$. There is another form of remainder for Taylor series which goes by the name of Peano's form of remainder and this requires a totally different approach for its proof.

Answer 2

$\newcommand{\angles}[1]{\left\langle\,{#1}\,\right\rangle} \newcommand{\braces}[1]{\left\lbrace\,{#1}\,\right\rbrace} \newcommand{\bracks}[1]{\left\lbrack\,{#1}\,\right\rbrack} \newcommand{\dd}{\mathrm{d}} \newcommand{\ds}[1]{\displaystyle{#1}} \newcommand{\expo}[1]{\,\mathrm{e}^{#1}\,} \newcommand{\half}{{1 \over 2}} \newcommand{\ic}{\mathrm{i}} \newcommand{\iff}{\Longleftrightarrow} \newcommand{\imp}{\Longrightarrow} \newcommand{\Li}[1]{\,\mathrm{Li}_{#1}} \newcommand{\mc}[1]{\mathcal{#1}} \newcommand{\mrm}[1]{\mathrm{#1}} \newcommand{\ol}[1]{\overline{#1}} \newcommand{\pars}[1]{\left(\,{#1}\,\right)} \newcommand{\partiald}[3][]{\frac{\partial^{#1} #2}{\partial #3^{#1}}} \newcommand{\root}[2][]{\,\sqrt[#1]{\,{#2}\,}\,} \newcommand{\totald}[3][]{\frac{\mathrm{d}^{#1} #2}{\mathrm{d} #3^{#1}}} \newcommand{\ul}[1]{\underline{#1}} \newcommand{\verts}[1]{\left\vert\,{#1}\,\right\vert}$

The $\underline{motivation}$ comes from repeatedly integration by parts of the following identity: $$ \mrm{f}\pars{x} = \mrm{f}\pars{0} + \int_{0}^{x}\mrm{f}'\pars{x - t}\,\dd t $$

\begin{align} \mrm{f}\pars{x} & = \mrm{f}\pars{0} + \int_{0}^{x}\mrm{f}'\pars{t}\,\dd t = \mrm{f}\pars{0} + \int_{0}^{x}\mrm{f}'\pars{x - t}\,\dd t \\[5mm] & = \mrm{f}\pars{0} + \mrm{f}'\pars{0}x + \int_{0}^{x}t\,\mrm{f}''\pars{x - t} \,\dd t \\[5mm] & = \mrm{f}\pars{0} + \mrm{f}'\pars{0}x + \half\,\mrm{f}''\pars{0}x^{2} + \half\int_{0}^{x}t^{2}\,\mrm{f}'''\pars{x - t}\,\dd t \\[5mm] & = \mrm{f}\pars{0} + \mrm{f}'\pars{0}x + \half\,\mrm{f}''\pars{0}x^{2} + {1 \over 3 \times 2}\,\mrm{f}'''\pars{0}x^{3} + {1 \over 3 \times 2} \int_{0}^{x}t^{3}\,\mrm{f}^{\pars{\texttt{IV}}}\pars{x - t}\,\dd t \\[5mm] & = \cdots = \sum_{k = 0}^{n}{\mrm{f}^{\pars{n}}\pars{0} \over n!}\,x^{n} + {1 \over n!}\int_{0}^{x}t^{n}\,\mrm{f}^{\pars{n + 1}}\pars{x - t}\,\dd t \end{align}