Are there any interesting theorems outside of set theory that use ordinals in their proofs? The only example I know of is Goodstein's theorem, and I haven't been able to find anything else.
In other (more vague) words, what is the use of ordinals? (Other than Goodstein.)
Theorems that use the word "ordinals" in their statement don't count; Goodstein is a good example.
The first uncountable ordinal $\omega_1$ is important in creating counterexamples in general topology. $\omega_1$ provides an example of a topological space that is sequentially compact (and countably-compact) but not compact. That $\omega_1$ is sequentially compact (and countably-compact) follows from the fact that sequences are indexed by $\omega$, and because $\omega_1$ is an uncountable ordinal, every sequence of elements in $\omega_1$ indexed by $\omega$ has an upper bound (and hence least upper bound). However, $\omega_1$ is not compact because it is not a closed interval (i.e., it doesn't contain $\omega_1$ itself).
Additionally, $\omega_1$ is used in the construction of the long line, $L=(\omega_1\times [0,1))\setminus (\emptyset,0)$ with the dictionary order. This has the property that is a path-connected linear continuum for which every point has a neighborhood homeomorphic to an open interval of $\mathbb{R}$, but is not metrizable. Thus, $L$ locally looks like the real line $\mathbb{R}$, but in certain senses in longer than $\mathbb{R}$.
Another important counter-example is that of the Tychonoff plank, $\omega_1^+\times \omega^+$ (where $\omega_1^+$ and $\omega^+$ are the successors of $\omega_1$ and $\omega$, respectively) in the product topology. This has the property that it is normal, but has a non-normal subspace (e.g. the deleted Tychonoff plank, $(\omega_1^+\times \omega^+)\setminus (\omega_1,\omega)$).
There are quite a few examples in analysis, typically through the use of transfinite induction or transfinite recursion of length $\omega_1$.
The Baire-Cantor stationary principle states that if $C_0\supseteq C_1\supseteq C_2\supseteq\dots\supseteq C_\alpha\supseteq\dots$, $\alpha<\omega_1$, is a decreasing sequence of closed sets of reals, then there is some countable $\alpha$ such that $C_\alpha=C_{\alpha+1}=C_{\alpha+2}=\dots$
A common application of the above is obtained through the Cantor-Bendixson derivative, by starting with a closed set $C=C_0$ and setting $C_{\alpha+1}=C_\alpha'$ and $C_\lambda= \bigcap_{\alpha<\lambda}C_\alpha$ for $\lambda$ limit.
Of course, this mentions ordinals explicitly, but it has many applications (to statements where ordinals are not relevant).
For instance, Baire proved that a function $f:\mathbb R\to\mathbb R$ is the pointwise limit of a sequence of continuous functions (that is, $f$ is Baire one) iff whenever $C$ is a nonempty closed subset of $\mathbb R$, the restriction $f\upharpoonright C$ has a continuity point.
Here is a sketch of the proof.
The forward direction is easy: If $C$ is countable, it has an isolated point. If $C$ is uncountable, an easy computation shows that in fact $f\upharpoonright C$ is continuous on a dense subset of $C$.
The interesting direction goes by showing that if $f\upharpoonright C$ has a continuity point for each nonempty closed set $C$, then for any $a<b$ there is a set $F$ that is simultaneously $F_\sigma$ and $G_\delta$ and such that $f(x)>a$ for all $x\in F$ and $f(x)<b$ for all $x\notin F$. From this it is standard to check that $f$ is the uniform limit of a sequence of Baire one functions, and therefore it is Baire one itself.
Now, to verify that there is a such a set $F$, one proceeds by transfinite recursion: Given $C_0=\mathbb R$, we define a decreasing sequence of closed sets $C_\alpha$, $\alpha<\omega_1$. If $C_\alpha=\emptyset$ so is $C_\beta$ for all further values of $\beta$. At limit stages we simply set $C_\alpha=\bigcap_{\beta<\alpha}C_\beta$. Finally, if $C_\alpha\ne\emptyset$, we pick a continuity point $x_0$ of $f\upharpoonright C_\alpha$. If $f(x_0)>a$, there is an open interval $I$ such that $f(x)>a$ for all $x\in C\cap I$, and similarly if $f(x_0)<b$. We then set $C_{\alpha+1}=C_\alpha\setminus I$. By the Cantor-Baire stationary principle, $C_\alpha=\emptyset$ for all sufficiently large countable ordinals $\alpha$, say for $\alpha\ge\alpha_0$.
The sets $C_\alpha\setminus C_{\alpha+1}$ for $\alpha<\alpha_0$ are pairwise disjoint, and their union is $\mathbb R$. By design, for any $\alpha<\alpha_0$, either $f(x)>a$ for all $x\in C_\alpha\setminus C_{\alpha+1}$, or else $f(x)<b$ for all such $x$. Since $\alpha_0$ is countable, obtaining the required set $F$ is now easy.
For another example, Denjoy and Khintchine developed a method for "reconstructing" the primitive of nonsummable derivatives (in a countable number of steps).
More precisely, by a thinning out process that uses the Cantor-Baire stationary principle, intervals where the given derivative $f$ is not integrable can be reduced while arranging that certain series converge, from which an antiderivative of $f$ can be computed. The process is a bit cumbersome to describe here in a precise manner, but it is explained in detail in $\S\,5.2$ of Andrew Bruckner's monograph, Differentiation of real functions.
To give an idea, Bruckner first proves (as theorem 5.2.1) the following: Suppose $E$ is a closed subset of the interval $[a,b]$, and that the complement of $E$ consists of the sequence of open intervals $(a_i,b_i)$. Suppose $F:[a,b]\to\mathbb R$ is continuous, $F'$ exists (and is finite) on a cocountable subset of $E$, and $F'$ is (Lebesgue) integrable on $E$. If $\sum_i|F(b_i)-F(a_i)|$ converges, then $$ F(b) - F(a) = \int_E F'(x) dx +\sum_i (F(b_i)-F(a_i)). $$ The Denjoy-Khintchine analysis then proceeds to explain how to identify $F$ when the assumptions that the series converges or that $F'$ is integrable on $E$ fail.
(Nowadays, this "reconstruction" of the primitive is discussed not through a transfinite process, but via the Henstock–Kurzweil integral.)
As a last example, let me mention a result of He and Schramm in complex analysis, from 1993. Work in $\bar {\mathbb C}$, the Riemann sphere. A domain is a connected open set. A circle domain is a domain such that each connected component of the boundary is either a circle or a point. In 1908, Koebe conjectured a significant generalization of the Riemann mapping theorem, namely that any plane domain is conformally homeomorphic to a circle domain in $\bar {\mathbb C}$. The Riemann mapping theorem is the case where the plane domain is simply connected. Koebe proved the conjecture when the domain is finitely connected. The general case remains open. What He and Schramm proved is the case where the boundary of the domain has at most countably many components.
A key ingredient of their (highly nontrivial) argument is an analysis of the Cantor-Bendixson rank of the boundary of the domain (that is, the set of components of the boundary is provided with an appropriate compact Hausdorff topology. Since it is a countable set, its rank is a countable ordinal).
Since then, He and Schramm have extended this result to also include some domains whose boundary has uncountably many (but "well-behaved") components.
Many other uses of ordinals in analysis come via descriptive set theory, through the use of ranks (the Cantor-Bendixson rank of a closed set is but one example). Alexander Kechris's book Classical descriptive set theory contains several examples of such ranks (on collections of compact sets, classes of differentiable functions, etc). Further examples have been investigated in the theory of Baire one functions (by Bourgain, Argyros, Kechris-Louveau, etc).
Gentzen's proof of the consistency of first-order arithmetic seems a good illustration. The notion of ordinal does not appear in the statement of the theorem, but in the proof transfinite induction is required up to ordinal $\varepsilon_0$.
Judging from the question, there seems to be a shortage of mathematical proofs using ordinal numbers. At one time, it seems, there was an overabundance of such proofs, because Zorn's lemma was invented for the purpose of eliminating ordinals in mathematical proofs. Look at the title of Kuratowski's paper, "Une méthode d'élimination des nombres transfinis des raisonnements mathématiques" ("A method of eliminating transfinite numbers from mathematical reasoning"), about an early version of Zorn's lemma.
For a specific example of a proof using ordinals, take any theorem that can be proved by transfinite induction. The existence of a Hamel basis. The Cantor-Bendixson decomposition of a closed set.
