How to prove that the exponential function and the logarithm function are the inverses of each other? I want it the following way. We must use the definition as power series, and must verify that all the terms of the composition except the coefficent of $z$ vanish, and that the first degree term is $1$.
I can write down the proof for the coefficient of $x^n$ for arbitrary but fixed $n$ by explicit verification. But how to settle this for all $n$ at one go?
There is a combinatorial proof. If $A(x) = \sum_{n \ge 0} a_n \frac{x^n}{n!}$ is an exponential generating function for the number of ways $a_n$ to put a certain structure on a set of size $n$, where $a_0 = 0$, then $\exp A(x) = \sum_{n \ge 0} b_n \frac{x^n}{n!}$ has coefficients $b_n$ which count the number of ways to split a set of size $n$ into subsets, each of which is given an $A$-structure. For example, $A(x) = e^x - 1$ is the structure of "being a nonempty set," and $\exp A(x)$ counts the number of ways to partition a set into nonempty subsets.
Now $\log \frac{1}{1 - x} = \sum_{n \ge 1} \frac{x^n}{n}$ can be thought of as the structure of "being a cycle," which is to say there are $(n-1)!$ ways to arrange $n$ objects into a cycle $s_1 \to s_2 \to ... \to s_n \to s_1$. So $\exp \log \frac{1}{1 - x}$ counts the number of ways to split $n$ objects up into disjoint cycles, which is the same as the number of permutations of $n$ by cycle decomposition. So
$$\exp \log \frac{1}{1 - x} = \frac{1}{1 - x}.$$
A generalization of this argument lets you compute the cycle index polynomials of the symmetric groups. Note that all of the above manipulations take place in the ring of formal power series over $\mathbb{Z}$.
Qiaochu Yuan's proof is without doubt the most elegant. However, here is a purely formal argument using the derivative:
Let $L(x) = \log \frac 1{1-x} = \sum_{n\geq 1} \frac{x^n}n$. I will show that $\exp(L(x))=\frac 1{1-x}$.
Let $u(x)=\exp(L(x))$. By the chain rule for differentiation, we have $$ u'(x) = L'(x)u(x). $$ Noting that $L'(x)=(1-x)^{-1}$, the above can be rewritten as $$ u(x)=(1-x)u'(x). $$ Suppose $u(x)=\sum_{n\geq 0} a_nx^n$, then comparing the coefficients of each power of $x$ in the preceding identity gives $$ a_n = (n+1)a_{n+1} - na_n $$ for each $n\geq 0$. Also, $a_0=1$. It follows that $a_n=1$ for all $n$, so $u(x)=\frac 1{1-x}$, as desired.
The reverse identity, $\log(\exp(x))=x$ is simpler; the chain rule tells us that the derivative of $\log(\exp(x))=1$.
