Here's one approach. By the spectral theorem, every orthogonal matrix $X$ has an orthonormal basis of eigenvectors $v_i$ over $\mathbb{C}$. By considering the action of $X$ on pairs $\{ v_i, \overline{v_i} \}$ you can show that this implies that $X$ is conjugate, in $O(n)$, to a block matrix whose blocks are $2 \times 2$ matrices in $O(2)$ or, when $n$ is odd, a $1 \times 1$ block with entries $\pm 1 \in O(1)$.
Now $O(1) = \{ \pm 1 \}$ certainly two connected components. It's also not hard to see that $O(2)$ has two connected components given by the rotations and the reflections. Now we have a natural invariant of a block matrix as above to consider, which is the number of reflections that appear in it. You can show that a direct sum of two reflections is conjugate to a direct sum of two rotations, and consequently when $n \ge 3$ there are two cases:
- An even number of reflections. In this case $X$ is conjugate to a direct sum of rotations and hence is itself a rotation, so lies in the connected component of the identity: said another way, it lies in the image of the exponential map $\mathfrak{o}(n) \to O(n)$.
- An odd number of reflections. In this case $X$ is conjugate to a direct sum of rotations and a single reflection, so is connected to every other reflection (because we can multiply $X$ by a suitable rotation to connect it to a diagonal matrix with an odd number of $-1$s, and all of these can be connected to each other). Hence the connected component of the identity has exactly one other coset as desired.
Edit (9/11/19): Mea culpa; this argument is incomplete. It doesn't show that the reflections are actually disconnected from the rotations, just that there are at most two connected components. We can argue that the image of the exponential map generates the connected component of the identity, then try to show that the product of two rotations is another rotation, but I don't see a clean way to do this off the top of my head.
Here's another approach. The idea is to use inductively the action of $O(n)$ on the sphere $S^{n-1}$, which produces a sequence of fiber bundles
$$O(n-1) \to O(n) \to S^{n-1}.$$
Associated to each of these is a long exact sequence in homotopy
$$\dots \to \pi_1(S^{n-1}) \to \pi_0(O(n-1)) \to \pi_0(O(n)) \to \pi_0(S^{n-1})$$
which, using the fact that $S^{n-1}$ is simply connected for $n \ge 3$, implies that the map $\pi_0(O(n-1)) \to \pi_0(O(n))$ is an isomorphism for $n \ge 3$. Thus, as above, it suffices to check the number of connected components of $O(1)$ and $O(2)$ to get the result in general. (Usually these long exact sequences are used to compute the fundamental group and higher homotopy groups of the $O(n)$.)