How Does Integrating the Derivative of this Approximation for Tangent Increase its Accuracy?

Question

In this answer, Raymond Manzoni uses a beautiful method to find the Taylor Series of $\tan x$, which is recursively defined as $x_{n+1}=\int1+x_n^2dx$, which comes from the fact that $\frac d {dx}\tan x=\sec^2x=1+\tan^2x$. Why does this work and are there any other series that can be derived recursively like this?

Answer 1

This is just the Picard iteration for the initial value problem $y'=f(x,y)=1+y^2$, $y(0)=y_0=0$. In general, $$ y_{n+1}=y_0+\int_0^x f(s,y_n(s))\,ds $$ and the theorem of Picard-Lindelöf tells us that under rather general assumptions this converges locally.

If $f$ is polynomial or analytical, then if one replaces $y_n$ with its truncated power series, then the evaluation of $f(s,y_n(s))$ will also be a truncated power series that can then be integrated. When $y_n=y(x)+O(x^{n+1})$, then $$f(x,y_n(x))=f(x,y(x))+f_y(x,y_n)O(x^{n+1})=y'(x)+O(x^{n+1})$$ and integration of this will give $$ y_{n+1}(x)=y_0+\int_0^x (y'(s)+O(s^{n+1}))\,ds=y(x)+O(x^{n+2}), $$ that is, the truncation degree increases and the coefficient of $x^{n+1}$ will be known and correct.

For instance, the most trivial non-trivial polynomial is $f(x,y)=y$ which in its integration gives term-for-term the exponential series.