Let's let $z=e^{i\theta}$ so that $dz=i\,e^{i\theta}d\theta$. This complex plane contour $C$ is the unit circle. Thus,
$$\begin{align}
\int_{-\pi}^{\pi}\frac{\cos (n\theta)}{1-2a\cos \theta+a^2}d\theta&=\frac12\,\oint_C \frac{z^n+z^{-n}}{1-a(z+z^{-1})+a^2}\frac{dz}{iz}\\\\
&=\frac{i}{2a}\,\oint_C \frac{z^n+z^{-n}}{(z-a)(z-1/a)}dz\\\\
&=\frac{i}{2a}\,\oint_C \frac{z^n}{(z-a)(z-1/a)}dz+\frac{i}{2a}\,\oint_C \frac{z^{-n}}{(z-a)(z-1/a)}dz
\end{align}$$
Note that the first integral has a singularity at $z=a$ only (since $|a|<1$; we examine the case $|a|>1$ in Note 1), while integrand has singularities at both $z=0$ (a pole of order $n$) and $z=a$. To evaluate the first integral one needs to determine the residue at $a$. This is simply, $\frac{a^n}{a-1/a}=\frac{a^{n+1}}{a^2-1}$.
In Note 2, we will exploit symmetry to simplify the second integral. This development is different from those shown in other posts inasmuch as the symmetry is exploited after transforming the original integral to a contour integral in the complex plane.
Here, we proceed "brute force." We need to determine the residues at both $0$ and $a$. The residue at $a$ is straightforward and is $a^{-n}/(a-1/a)=a^{-n+1}/(a^2-1)$. For the residue at $0$, we use the formula for an $n$th order pole
$$\begin{align}
\text{Res}_{z=0}&=\frac{1}{(n-1)!}\lim_{z\to 0} \frac{d^{n-1}}{dz^{n-1}}\left(\frac{1}{(z-a)(z-1/a)}\right)\\\\
&=\frac{1}{(n-1)!}\lim_{z\to 0} \sum_{k=0}^{n-1} \binom{n-1}{k} \frac{d^{k}}{dz^{k}}\left(\frac{1}{(z-a)}\right)\,\frac{d^{n-1-k}}{dz^{n-1-k}}\left(\frac{1}{(z-1/a)}\right)\\\\
&=a^{-n+1}\sum_{k=0}^{n-1} a^{2k}\\\\
&=\frac{a^{n+1}-a^{-n+1}}{a^2-1}
\end{align}$$
Adding the two residues reveals that the second integral is $a^{n+1}/(a^2-1)$, which is identical to the value of the first integral.
Finally, adding the results and multiplying by $(2\pi i) (i/2a)$ yields
$$\int_{-\pi}^{\pi}\frac{\cos (n\theta)}{1-2a\cos \theta+a^2}d\theta=2\pi \frac{a^n}{1-a^2}$$
which agrees with previously obtained results.
Note 1: Integral for $|a|>1$
Note that the entire development for the case $|a|>1$ is trivial given the symmetry of the integrand in $a$ and $1/a$. Thus we can immediately write
$$\begin{align}
\int_{-\pi}^{\pi}\frac{\cos (n\theta)}{1-2a\cos \theta+a^2}d\theta&=
2\pi \frac{a^{-n}}{1-a^{-2}}\\\\
&=2\pi \frac{a^{-n+2}}{a^2-1}
\end{align}$$
Note 2: Exploiting Symmetry
As was shown in other posts, symmetry can be exploited up front in the original integral representation. Here, we show that this tactic can be delayed until after recasting the problem as a contour integral. To that end, we write
$$\begin{align}
\int_{-\pi}^{\pi}\frac{\cos (n\theta)}{1-2a\cos \theta+a^2}d\theta&=\frac{i}{2a}\,\oint_C \frac{z^n+z^{-n}}{(z-a)(z-1/a)}dz\\\\
&=\frac{i}{2a}\,\left(\oint_C \frac{z^n}{(z-a)(z-1/a)}dz+\oint_C \frac{z^{-n}}{(z-a)(z-1/a)}dz\right)
\end{align}$$
and focus on the second integral on the right-hand side of the last expression. Here, we make the substitution $z=w^{-1}$. Note that (1) $dz=-w^{-2}dw$, (2) the contour's orientation transforms from counter-clockwise to clockwise, (3) $z^{-n}=w^{n}$, and (4) $(z-a)(z-1/a)=w^{-2}(w-a)(w-1/a)$. Implementing the substitution shows that
$$\begin{align}
\oint_C \frac{z^{-n}}{(z-a)(z-1/a)}dz=\oint_C \frac{z^{n}}{(z-a)(z-1/a)}dw
\end{align}$$
where the contour $C$ is identical (i.e., counter-clockwise) as the minus sign in the differential has been absorbed. Thus, the integral is an even function of $n$ and we need only evaluate the integral with $z^n$ in the numerator.
HINT Let $|z|=1$ ($z=e^{i\theta}$) and $\cos(\theta)=(1/2)(z+\overline{z})$, then since $dz=ie^{i \theta} d\theta$ implies $d\theta=-i/z$ obtain you integral $\int_{-\pi}^{\pi}d\theta$ in the form $\int_{|z|=1}dz$ and now use Residues
– Alonso Delfín Apr 25 '15 at 21:35