Evaluate by complex methods
$$\int_0^{\pi} \frac{\sin^2 \theta}{(1-2a\cos\theta+a^2)(1-2b\cos\theta+b^2)}\mathrm{d\theta}, \space 0<a<b<1$$
Sis.
Evaluate by complex methods
$$\int_0^{\pi} \frac{\sin^2 \theta}{(1-2a\cos\theta+a^2)(1-2b\cos\theta+b^2)}\mathrm{d\theta}, \space 0<a<b<1$$
Sis.
This integral is $1/2$ the integral over $[0,2 \pi)$. Let $z=e^{i \theta}$, $d\theta = dz/(i z)$; the result is
$$\frac{1}{2 i}\oint_{|z|=1} \frac{dz}{z} \frac{-\frac{1}{4} (z^2-1)^2}{(a z^2-(1+a^2)z+a)(b z^2-(1+b^2)z+b)}$$
which can be rewritten as
$$\frac{i}{8} \oint_{|z|=1} \frac{dz}{z} \frac{(z^2-1)^2}{(a z-1)(z-a)(b z-1)(z-b)}$$
There are 5 poles, although because $0<a<b<1$, only 3 of them fall within the contour. This integral is then $i 2 \pi$ times the sum of the residues of these poles. The residues of these poles are actually straightforward:
$$\mathrm{Res}_{z=0}\frac{i}{8} \frac{(z^2-1)^2}{z(a z-1)(z-a)(b z-1)(z-b)} = \frac{i}{8 a b}$$ $$\mathrm{Res}_{z=a}\frac{i}{8} \frac{(z^2-1)^2}{z(a z-1)(z-a)(b z-1)(z-b)} = \frac{i}{8 a} \frac{a^2-1}{(a b-1)(a-b)}$$ $$\mathrm{Res}_{z=b}\frac{i}{8} \frac{(z^2-1)^2}{z(a z-1)(z-a)(b z-1)(z-b)} = -\frac{i}{8 b} \frac{b^2-1}{(a b-1)(a-b)}$$
There is vast simplification from adding these pieces together, which I leave to the reader. The result is
$$\int_0^{\pi} d\theta \frac{\sin^2 \theta}{(1-2a\cos\theta+a^2)(1-2b\cos\theta+b^2)} = \frac{\pi}{2} \frac{1}{1-a b}$$
It can be done easily without complex, if we note that $$ \frac{\sin x}{1-2a\cos x+a^2}=\sum_{n=0}^{+\infty}a^n\sin[(n+1)x]$$
Just saying.
EDIT: for proving this formula, we actually use complex method
Using the series presented by Cortizol, the integral can be written as: $$\int_0^{\pi} dx\left(\sum_{n=1}^{\infty} a^{n-1}\sin (nx)\right)\left(\sum_{m=1}^{\infty} b^{m-1}\sin (mx)\right)=\frac{1}{ab}\sum_{n=1}^{\infty} \sum_{m=1}^{\infty} a^nb^m\int_0^{\pi} \sin(nx)\sin(mx)\,dx $$ Notice that for $n\ne m$, the integral is always zero, hence we look at only those cases where $n=m$ i.e $$\frac{1}{ab}\sum_{n=1}^{\infty} (ab)^n\int_0^{\pi} \sin^2(nx)\,dx=\frac{\pi}{2ab}\sum_{n=1}^{\infty} (ab)^n=\boxed{\dfrac{\pi}{2}\dfrac{1}{1-ab}}$$ $\blacksquare$
$\newcommand{\bbx}[1]{\,\bbox[15px,border:1px groove navy]{\displaystyle{#1}}\,} \newcommand{\braces}[1]{\left\lbrace\,{#1}\,\right\rbrace} \newcommand{\bracks}[1]{\left\lbrack\,{#1}\,\right\rbrack} \newcommand{\dd}{\mathrm{d}} \newcommand{\ds}[1]{\displaystyle{#1}} \newcommand{\expo}[1]{\,\mathrm{e}^{#1}\,} \newcommand{\ic}{\mathrm{i}} \newcommand{\mc}[1]{\mathcal{#1}} \newcommand{\mrm}[1]{\mathrm{#1}} \newcommand{\on}[1]{\operatorname{#1}} \newcommand{\pars}[1]{\left(\,{#1}\,\right)} \newcommand{\partiald}[3][]{\frac{\partial^{#1} #2}{\partial #3^{#1}}} \newcommand{\root}[2][]{\,\sqrt[#1]{\,{#2}\,}\,} \newcommand{\totald}[3][]{\frac{\mathrm{d}^{#1} #2}{\mathrm{d} #3^{#1}}} \newcommand{\verts}[1]{\left\vert\,{#1}\,\right\vert}$ \begin{align} &\bbox[5px,#ffd]{\left.\int_{0}^{\pi}{\sin^{2}\pars{\theta} \over \bracks{1 - 2a\cos\pars{\theta} + a^{2}} \bracks{1 - 2b\cos\pars{\theta} + b^{2}}}\,\dd\theta \,\right\vert_{\, 0\ <\ a\ <\ b\ <\ 1}} \\[5mm] = &\ {1 \over 2} \int_{-\pi}^{\pi}{\pars{\expo{\ic\theta} - \expo{-\ic\theta}}^{2}/\pars{-4} \over \pars{a - \expo{\ic\theta}}\pars{a - \expo{-\ic\theta}} \pars{b - \expo{\ic\theta}} \pars{b - \expo{-\ic\theta}}}\,\dd\theta \\[5mm] = &\ -{1 \over 8}\,\Re \int_{-\pi}^{\pi}{\pars{z - z^{-1}}^{2} \over \pars{a - z}\pars{a - z^{-1}} \pars{b - z} \pars{b - z^{-1}}}\,{\dd z \over \ic z} \\[5mm] = &\ -{1 \over 8ab}\,\Im \int_{-\pi}^{\pi}{\pars{z^{2} - 1}^{2} \over z\pars{z - a}\pars{z - a^{-1}} \pars{z - b} \pars{z - b^{-1}}}\,\dd z \end{align} The integrand has simple poles at $\ds{z \in \braces{0,a,b}}$ with $$ \left\{\begin{array}{rcl} \ds{\on{Res}_{\, z\ =\ 0}} & \ds{=} & \ds{\phantom{-}1} \\[1mm] \ds{\on{Res}_{\, z\ =\ a}} & \ds{=} & \ds{-{\pars{1 - a^{2}}b \over \pars{b - a}\pars{1 - ab}}} \\[1mm] \ds{\on{Res}_{\, z\ =\ b}} & \ds{=} & \ds{\phantom{-} {a\pars{1 - b^{2}} \over \pars{b - a}\pars{1 - ab}}} \end{array}\right. $$ Then, \begin{align} &\bbox[5px,#ffd]{\left.\int_{0}^{\pi}{\sin^{2}\pars{\theta} \over \bracks{1 - 2a\cos\pars{\theta} + a^{2}} \bracks{1 - 2b\cos\pars{\theta} + b^{2}}}\,\dd\theta \,\right\vert_{\, 0\ <\ a\ <\ b\ <\ 1}} \\[5mm] = &\ -{1 \over 8ab}\Im\bracks{2\pi\ic\pars{\on{Res}_{\, z\ =\ 0}\ +\ \on{Res}_{\, z\ =\ a}\ +\ \on{Res}_{\, z\ =\ b}}} \\[5mm] = &\ \bbx{\pi/2 \over 1 - ab} \\ & \end{align}