7

Consider an $n$-times differentiable map $f$ between finite dimensional real vector spaces. Specifically, let $f$ be defined on an open subset $U$.

Suppose the closed interval $[x,x+h]\subset U$ is entirely contained in $U$ and that the $(n+1)^\text{th}$ derivative exists on the interior $(x,x+h)$ of the interval. Write $M=\sup_{(x,x+h)}\|f^{(n+1)}(c)\|$.

I would like to prove the remainder of order $n$ satisfies the inequality $$\|R_nf(x)\|\leq\frac{M}{(n+1)!}\|h\|^{n+1}.$$

The only thing I can think of is precomposing with the straight path from $x$ to $x+h$ and then choosing coordinates on the target space to apply the mean value theorem in each coordinate. However, this gives an estimate in terms of the operator norms of the coordinates of the $(n+1)^\text{th}$ derivative which is looser than the "total" operator norm.


I probed a bit online and found the following "guide".

Consider the comopsite $f(x+th)$ with $t\in [0,1]$. Write $g_n(t)$ for the $n^\text{th}$ order remainder for $f(x+th)$. Prove the estimate for every $t\in (0,1)$ $$\|g_n^\prime(t)\|\leq \frac{1}{n!}M\|th\|^n\|h\|.$$ Use it to deduce $\|g_n(1)-g_n(0)\|\leq \frac{1}{(n+1)!}M\|h\|^{n+1}$ as desired.

Given the estimate on $g^\prime _n$ I know how to get the second inequality. However, I don't see how to prove this first estimate. Induction does not seem to help.

Arrow
  • 13,810

2 Answers2

3

In these situations, the Mean-value Theorem/inequality (whatever you want to call it) is your best friend. What I'm about to write, I learnt from Henri Cartan's book on Differential Calculus. The relevant results are Theorem $3.11$ (Mean value theorem) and Corollary $5.53$, and Theorem $5.62$ (Taylor's theorem with Lagrange Remainder).

Theorem $3.11$ (Mean-Value Theorem):

Let $E$ be any real Banach space, $[a,b]$ an interval in $\Bbb{R}$ with $a< b$. Let $u:[a,b] \to E$ and $g:[a,b] \to \Bbb{R}$ be continuous functions on $[a,b]$, and differentiable on the open interval $(a,b)$. Suppose that for all $t \in (a,b)$, it is true that \begin{align} \lVert u'(t)\rVert & \leq g'(t) \end{align} Then, $\lVert u(b) - u(a)\rVert \leq g(b) - g(a)$.

The proof of this theorem, while elementary in methods, is slightly technical. But, you said you know how to deduce the second inequality from the first, so I guess you must know of the Mean value theorem either in this form or something similar, so I'll omit the proof.


Next, we have the following result (Proposition $5.51$ and Corollary $5.53$ combined)

Theorem:

Let $E$ be a real Banach space, $I$ an open subset of $\Bbb{R}$ containing the closed interval $[0,1]$, and let $v: I \to E$ be $(n+1)$-times differentiable on $I$, and suppose that $M:= \sup\limits_{t \in [0,1]} \lVert v^{(n+1)}(t)\rVert < \infty$. (Note that boundedness of derivative is obviously a weaker assumption than the $(n+1)^{th}$ derivative being continuous on $I$). Then the $n^{th}$ order remainder of $v$ at $0$ satisfies: \begin{align} \left \lVert v(t) - \sum_{k=0}^n \dfrac{v^{(k)}(0)}{k!} t^k\right \rVert &\leq \dfrac{M}{(n+1)!}. \tag{$\ddot{\smile}$} \end{align}

This will be the crucial link to help you deduce the theorem you want, because, we'll apply it to $v(t) = f(x + th)$, for $t$ in an open interval containing $[0,1]$. Here's the proof of this theorem:

Define the functions $u: I \to E$, and $g: I \to \Bbb{R}$ by \begin{align} u(t) &= \sum_{k=0}^n \dfrac{(1-t)^k}{k!} v^{(k)}(t) \quad \text{and} \quad g(t) = -M \dfrac{(1-t)^{n+1}}{(n+1)!} \end{align}

Then a simple calculation using the product rule (there will be a telescoping sum) shows that \begin{align} u'(t) &= \dfrac{(1-t)^n}{n!} v^{(n+1)}(t) \end{align} Hence, for all $t \in [0,1]$, we have \begin{align} \lVert u'(t)\rVert & \leq M \dfrac{(1-t)^n}{n!} = g'(t) \end{align} This is exactly the situation of the Mean-value theorem, so, we have $\lVert u(1) - u(0)\rVert \leq g(1) - g(0)$. If you plug in what $u$ and $g$ are, you'll find exactly $(\ddot{\smile})$. (The $u$ was constructed to ensure exactly this).


Lastly, we let $v(t) = f(x+th)$, where the $f,x,h$ are as in your question. Then, the $(n+1)^{th}$ derivative of $v$ at $t$ is given by (chain rule, and induction) \begin{align} v^{n+1}(t) &= (D^{n+1}f)_{x+th} [h]^{n+1} \in E \end{align}

Btw here, the RHS means the $(n+1)^{th}$ Frechet derivative of $f$ at $x+th$ evaluated on the $n+1$ tuple $(h, \dots, h) \in E^{n+1}$. Now, we can bound this using the operator norm, and properties of continuous multi-linear maps: \begin{align} \lVert v^{n+1}(t)\rVert & \leq \lVert D^{n+1}f{(x+th)}\rVert \cdot \lVert h\rVert^{n+1} \\ & \leq M \cdot \lVert h \rVert^{n+1} \end{align}

So, if in $(\ddot{\smile})$ you now substitute what $v$ is, and replace $M$ by $M \cdot \lVert h\rVert^{n+1}$, then you find that \begin{align} \left \lVert f(x+th) - \sum_{k=0}^n \dfrac{D^kf_x(th)^k}{k!} \right \rVert & \leq \dfrac{M}{(n+1)!}\lVert h \rVert^{n+1} \end{align} which is exactly the bound on the remainder term.

peek-a-boo
  • 55,725
  • 2
  • 45
  • 89
  • Dear peek-a-boo, with your refined mean value inequality I see how things work. However, what if I only know the version described at the beginning of this question i.e $ \lVert u(b)-u(a)\rVert \leq \sup \lVert u'(t) \rVert |b-a|$? – Arrow Dec 22 '19 at 11:53
  • @arrow I initially tried using that version, but unfortunately with that I only got a weaker bound with a $1/n!$ instead of $1/(n+1)!$. perhaps we might have to apply the mean value inequality to a differently chosen $u$, but at the moment I can't see how – peek-a-boo Dec 22 '19 at 13:37
  • 1
    Yes, I got the same weak bound. I think I might have an alternative proof using your mean value inequality. I'll post it soon, and would be happy to hear if you think it's correct. – Arrow Dec 22 '19 at 13:40
  • I've seen this in Cartan's book. How does one make sense of it all, I wonder. $v^{(k)}(t)$ is a $k-$linear map from the reals in $E$, so $u(t)$ cannot be in $E$. I cannot think of a meaning for $5.51$ - $5.53$. It is possible to identify a linear map from the reals into $E$ with just one value in $E$, it is not however possible to do that in higher dimensions (you need $k$ values in $E$ for a $k-$linear map). – user643717 Jan 04 '21 at 17:30
  • @user643717 If $E$ is any Banach space over the field $\Bbb{F}\in{\Bbb{R},\Bbb{C}}$, then $L(\Bbb{F},E)$ and $E$ are isometrically isomorphic Banach spaces (the "evaluation at $1$ map "$T\mapsto T(1)$ is an isomorphism); see his remarks in page 26. Also for Banach spaces $E,F,G$ we have a natural isometric isomorphism of the space of linear maps $L(E,L(F,G))$ with $L^2(E,F;G)$, the space of bilinear maps (see section 1.9). So, if you continue inductively, what is the isomorphism between the $k$-linear mappings $\Bbb{F}\to E$ and $E$? – peek-a-boo Jan 05 '21 at 08:47
  • Also, just a notational difference: when Cartan writes $f'(a)\in L(E,F)$ for the derivative of a mapping at the point $a$, I write $Df_a$. I reserve the prime notation for when the domain $E$ is actually the field $\Bbb{F}$ and I refer to the limit of the difference quotient $\lim\limits_{h\to 0}\frac{f(a+h)-f(a)}{h}$, so $f'(a)= Df_a[1]$ (you can show that when the domain is $\Bbb{F}$, we have $f'(a)$ exists if and only if $Df_a$ exists in which case $f'(a)=Df_a[1]$). So in this sense I reserve the prime notation for the "usual single variable definition", so indeed $u(t)\in E$. – peek-a-boo Jan 05 '21 at 08:50
0

Let $j^n_a(x)$ be the $n^\text{th}$ order Taylor expansion of $f$ about $a$. Let $R_a^nf(x)=f(x)-j_a^n(x)$ be the remainder. The answer below makes crucial use of the mean value inequality of peek-a-boo's answer.

Theorem. Let $V$ be a finite-dimensional real vector space. Let $I\supset(a,a+h)$ be an open subset of $\mathbb{R}$. Let $f:I\to V $ be $(n+1)$-times differentiable in $(a,a+h)$. Suppose for any $t\in(a,a+h)$ we have $$ m\leq\|D^{n+1}f(t)\|\leq M. $$ Then for any $t\in(a,a+h)$ we have $$ \tfrac{m}{(n+1)!}(t-a)^{n+1}\leq\|R_{a}^{n}f(t)\|\leq\tfrac{M}{(n+1)!}(t-a)^{n+1}. $$

Proof. By induction on $n$. The case $n=-1$ is trivial. Assume the assertion holds for $n-1$. We shall apply this to the function $Df$ which is by assumption $n$-times differentiable on $(a,b)$. The function $Df$ satisfies the hypothesis because $D^{n}Df=D^{n+1}f$ whence $$ m\leq\|D^{n}Df(t)\|\leq M. $$ By the induction hypothesis $$ \tfrac{m}{n!}(t-a)^{n}\leq\|R_{a}^{n-1}Df(t)\|\leq\tfrac{M}{n!}(t-a)^{n}. $$ Observe the LHS and RHS are respectively the derivatives at $t$ of the functions $$ \tfrac{m}{(n+1)!}(t-a)^{n+1},\tfrac{M}{(n+1)!}(t-a)^{n+1}. $$ Moreover, $R_{a}^{n-1}Df=DR_{a}^{n}f$. The mean value inequality along with the fact $R_{a}^{n}f(a)=0$ therefore gives the desired inequality.

Theorem. (Vector-valued Taylor theorem.) Let $V,W$ be finite-dimensional real vector spaces and $U\subset V$ open. Suppose $[a,a+h]\subset U$ and let $f:U\to W $ be $(n+1)$-times differentiable at the points of the open interval $(a,a+h)$. If for every $x\in(a,a+h)$ we have $$ \|D^{n+1}f(x)\|\leq M. $$ Then for any $x\in(a,a+h)$ we have $$ \|R_{a}^{n}f(x)\|\leq\tfrac{M}{(n+1)!}\|x-a\|^{n+1}. $$

Proof. Define $\gamma(t)=a+th$ and consider the composite $f\circ\gamma$ on the open unit interval $(0,1)$. By the chain rule and induction $$ (f\circ\gamma)^{(k)}(t)=D^{k}f(a+th)(h^{\otimes k})\in{\bf V}. $$ This has two consequences. First, $$ \sup_{(0,1)}\|D^{n+1}(f\circ\gamma)(t)\|\leq\|D^{n+1}f(a+th)\|\|h\|^{n+1}\leq M\|h\|^{n+1} $$ whence $$ \|R_{0}^{n}(f\circ\gamma)(t)\|\leq\tfrac{M\|h\|^{n+1}}{(n+1)!}t^{n+1}=\tfrac{M}{(n+1)!}\|th\|^{n+1} $$ Second, taking $t=0$ shows $$ (f\circ\gamma)^{(k)}(0)=D^{k}f(a)(h^{\otimes k}) $$ whence $$ (f\circ\gamma)^{(k)}(0)t^{k}=D^{k}f(a)(h^{\otimes k})t^{k}=D^{k}f(a)(th)^{\otimes k}. $$ Combining these equalities for all $0\leq k\leq n$ gives by definition $$ R_{0}^{n}(f\circ\gamma)(t)=R_{a}^{n}f(\gamma t). $$ Now, the above theorem and the above estimate combine to give $$ \|R_{a}^{n}f(\gamma t)\|=\|R_{0}^{n}(f\circ\gamma)(t)\|\leq\tfrac{M}{(n+1)!}\|th\|^{n+1}. $$ Finally, $x\in(a,a+h)$ means $x=a+th$ so that $th=x-a$. Plugging this in gives the desired inequality.

Arrow
  • 13,810
  • I didn't scrutinize the second theorem's proof, because it seems similar in spirit to what I wrote, and really it't a minor application of the first theorem. However, for the first theorem there seem to be some issues. Firstly, why do you start the base case at $n=-1$? That doesn't make sense because what do you mean by the $-1^{th}$ Taylor approximation of a function? So, I think you should start with the base case of $n=0$ (as such, the base case will be a simple application of even the simple version of the mean value inequality). – peek-a-boo Dec 23 '19 at 16:54
  • The next issue I would like to raise is that the mean value inequality only holds for upper bounds, not lower bounds; i.e the following is NOT true: if $f: [a,b] \to V$ is continuous on $[a,b]$ and differentiable on $(a,b)$, and there exists an $m \geq 0$ such that for all $t \in (a,b)$, $m \leq \lVert f'(t)\rVert$, then $m(b-a) \leq \lVert f(b) - f(a)\rVert$. A simple counter example is $f: [0, 2 \pi] \to \Bbb{R}^2$, defined by $f(t) = (\cos t, \sin t)$. Then, for all $t$, $\lVert f'(t)\rVert = 1$, but due to periodicity of trig functions, this function doesn't satisfy the inequality above. – peek-a-boo Dec 23 '19 at 16:59
  • But if you start the proof of first theorem from $n=0$, then I think the proof of the upper bound inequality is fine (and in fact nicely generalizes one of the first remainder proofs I learnt in single variable calc... so thanks for writing this up) – peek-a-boo Dec 23 '19 at 17:07
  • I thought to define the remainder to equal $f$ for $n=-1$. Thanks for your elegant counterexample - I should have considered it! – Arrow Dec 25 '19 at 15:35
  • Dear @peek-a-boo, I'd like to check I understand the relation to the integral form of Taylor theorem. In the Lagrange form here we require the existence of bounded $(n+1)^{\text{th}}$ derivative to obtain a bound. It seems the integral form requires $f$ to be merely $C^n$, with the $n^\text{th}$ order remainder given by integration of increments of the $n^\text{th}$ derivative itself. So it seems the integral form strictly strengthens the Lagrange form using integration theory. Am I missing something? – Arrow Jan 24 '20 at 11:07
  • no, the integral form of remainder (even in single variable calculus) requires one to integrate the $(n+1)^{th}$ derivative. In single variable, the Lagrange form merely requires the existence of $f^{(n+1)}$ on a certain interval, whereas the integral form requires $f^{(n+1)}$ to be integrable. In higher dimensions, the Lagrange remainder inequality as we discussed above requires boundedness of $f^{(n+1)}$, but of course boundedness is still a weaker assumption than integrability. – peek-a-boo Jan 24 '20 at 22:01
  • Dear @peek-a-boo, I am looking at theorem 5.1 in this document by Brian Conrad, where he gives an integral formula for the $p^\text{th}$ remainder in terms of increments of the $p^\text{th}$ derivative (not the $(p+1)^\text{th}$). Indeed he explicitly assumes his map to only be of class $C^p$. What am I missing? – Arrow Jan 31 '20 at 10:53
  • yes so the remainder given there is stated/formulated slightly differently to what (I and I suspect others) usually refer to as "the integral form of remainder". In particular, refer to the paragraph in Section 4, before Definition 4.1. Here the author comments on the difference between his $R_{p,a}(h)$ and the form of the remainder $R_{p,a}(h)$, as given in Spivak's Calculus textbook (which is where I learnt everything, and hence what I was using as my reference). – peek-a-boo Jan 31 '20 at 23:12
  • The point is the author has: for a given $C^p$ function ($p \geq 1$) given a Taylor expansion $f(a+h) = \sum_{j=0}^p \dfrac{(D^jf)a[h^{(j)}]}{j!} + R{p,a}(h)$ up to order $p$... i.e it contains $\beta := (D^pf)a(h^p)$. But if you inspect his remainder $R{p,a}(h)$ carefully, there is a subtraction term which cancels out with $\beta$. So really, there is only a $(p-1)^{th}$ order Taylor approximation, and the integral remainder uses a $(D^pf)_a$. Long story short: the author simply added and subtracted the same term, and his choice of notation... – peek-a-boo Jan 31 '20 at 23:17
  • ... for $R_{p,a}$ without the additional subtracted term is what others might write as $R_{p-1,a}$. so really, its the same math, with different presentation, and no more/less generality than is usual. So, this doesn't contradict the statement I made previously about the integral form of remainder requiring more regularity assumptions than the Lagrange form. – peek-a-boo Jan 31 '20 at 23:18