Relation to Smooth Infinitesimal Analysis
Stanford Encyclopedia of Philosophy discusses Smooth Infinitesimal Analysis (using nilpotent $\epsilon^n = 0$, not just $\epsilon^2 = 0$ as in the above discussion) in contrast to non-standard analysis (using hyperreals). In fact, SIA is very close to how Theoretical Physicists manipulate infinitesimal quantities, to the accuracy they desire.
Depending on how much details you want, the construction of these infinitesimals involves sheaves and toposes, and the principle of microaffineness.
Stanford Enclycopedia includes this startling comment:
We observe that the postulates of smooth infinitesimal analysis are incompatible with the law of excluded middle of classical logic.
This [blog] uses infinitesimals but they only go to second order $dx^2 = 0$. Introductory books on Calculus have been written using both approaches:
For more advanced reading, some students have wrtten Master's Theses ont he topic:
Taylor's Theorem
Using SIA a form of Taylor's theorem is exact since we choose $\epsilon^n = 0$
$$ f(x+\epsilon) = \sum_{k=0}^n f^{(k)}(x) \frac{\epsilon^k}{k!} $$
Therefore we can show the derivative of the exponential function is itself in SIA and expand:
$$ (e^x)'= e^x \hspace{0.125in}\text{ and }\hspace{0.125in} e^\epsilon = \sum_{k=0}^n \frac{\epsilon^k}{k!} \hspace{0.125in}\text{ or }\hspace{0.125in} e = \lim_{n \to \infty} (1 + \epsilon)^n $$
In general, this seems to be a way of obtaining infinite series as the Taylor series expansion and there is duality between $\epsilon \leftrightarrow n$.
COMMENT Upon further investigation, there doesn't seem to be any analogue of the Mean Value Theorem or Intermediate Value Theorem (which is provable in nonstandard-analysis). So in theory we can't prove:
Intermediate Value Let $a < b$ and $f(a) > 0$ and $f(b) < 0$, then there exists $a < c < b$ such that $f(c) = 0$ (False in $\mathbb{S}$)
This result breaks down in SIA - related to the Law of Excluded Middle. The Mean Value Theorem and (global) Taylor's Theorem seems to enjoy similar fates.
While we can take deritives (and integrals!) there doesn't seem to be any way to connect local and global information in the smooth world $\mathbb{S}$.
Divergence of Harmonic Series $\sum \frac{1}{k} = \infty$
The standard way to prove the divergence of Harmonic series is to use the integral test which states that if $f$ is monotone decreasing then
$$ \int_0^\infty f(x) \, dx < \sum_{k=0}^\infty f(k) < f(0) + \int_0^\infty f(x) \, dx$$
In our cases $f(x) = \frac{1}{x}$ but how to we know the integral test works in SIA? What is an integral? Bell's Primer of Infinitesimal Calculus defines $\mathbf{\color{blue}{\int}}$ rather obliquely as the "area under the curve" but now you're done.
