I'm dealing with these two integrals:
$$I_1=\int_{-\pi}^{\pi} \frac{\cos(x)\cos(nx)}{(1+e \cos(x))^3} \mathrm{d}x, \quad I_2=\int_{-\pi}^{\pi} \frac{\sin(x)\sin(nx)}{(1+e \cos(x))^3}\mathrm{d}x$$
is there a way to reduce them to a Hypergeometric form or to solve them analytically?
From this answer we know that
$$ \int_0^\pi\frac{\cos(mx)}{p-q\cos (x)}\ \mathrm{d}x=\frac{\pi}{\sqrt{p^2-q^2}}\left(\frac{p-\sqrt{p^2-q^2}}{q}\right)^m\quad\hbox{for}\quad |q|<p \tag{1} $$
We'll come back to this identity to solve both integrals.
For $I_1$ first notice that $$ \frac{\partial}{\partial p}\left(\frac{\partial}{\partial q}\frac{\cos(mx)}{p-q\cos (x)} \right) = \frac{\partial}{\partial p}\left(\frac{\cos(x)\cos(mx)}{\left(p-q\cos (x)\right)^2 }\right) = -2 \frac{\cos(x)\cos(mx)}{\left(p-q\cos (x)\right)^3} $$ So using Feynman's trick, differentiating both sides of $(1)$ first with respect to $q$ and then $p$, we get \begin{align} -2 \int_0^\pi\frac{\cos(x)\cos(mx)}{\left(p-q\cos (x)\right)^3} \ \mathrm{d}x & =-\frac{\pi \left(p - \sqrt{p^2 -q^2}\right)^m\left(m^2 p\left(p^2 -q^2\right) + m\left(p^2 + 2q^2\right) \sqrt{p^2 - q^2}+ 3pq^2 \right) }{q^{m+1}\left(p^2 - q^2 \right)^{\frac{5}{2}}} \end{align} Now, since $I_1$ has an even integrand, we know that $\int_{-\pi}^{\pi} = 2 \int_{0}^{\pi}$. Substituting $p=1$ and $q = - \xi$ (which holds since you established that the parameter $0<\xi<1$) we can combine all the above to get \begin{align} \boxed{\int_{-\pi}^\pi\frac{\cos(x)\cos(nx)}{\left(1+ \xi\cos (x)\right)^3} \ \mathrm{d}x = - \frac{\pi \left( \sqrt{1 -\xi^2}-1\right)^n\left(n^2 \left(1 -\xi^2\right) + n\left(1 + 2\xi^2\right) \sqrt{1 - \xi^2}+ 3\xi^2 \right) }{\xi^{n+1}\left(1 - \xi^2 \right)^{\frac{5}{2}}}} \end{align} valid for $0<|\xi|<1$.
For $I_2$, integrating by parts gives $$ I_2 = \int_{-\pi}^{\pi} \sin(nx)\left[\frac{\partial}{\partial x} \ \frac{1}{\color{blue}{2}\xi(1+\xi \cos(x))^2} \right] \ \mathrm{d}x = - \frac{n}{\xi} \int_{\color{blue}{0}}^{\pi} \frac{\cos(nx)}{(1+\xi \cos(x))^2} \ \mathrm{d}x $$ since $\sin(nx)=0$ at $x = \pm \pi$. Differentiating $(1)$ with respect to $p$ gives $$ -\int_0^\pi\frac{\cos(mx)}{(p-q\cos (x))^2}\ \mathrm{d}x = -\frac{\pi \left(\frac{p - \sqrt{p^2 - q^2}}{q}\right)^m \left(p + m \sqrt{p^2 - q^2}\right)}{\left(p^2 - q^2\right)^{\frac{3}{2}}} $$ And again substituting $p=1$ and $q = - \xi$, we can conclude that $$\boxed{ \int_{-\pi}^{\pi} \frac{\sin(x)\sin(nx)}{(1+\xi \cos(x))^3}\ \mathrm{d}x = -\frac{n \pi \left( \sqrt{1 - \xi^2}-1\right)^n \left(1 + n \sqrt{1 - \xi^2}\right)}{\xi^{n+1} \left(1 - \xi^2\right)^{\frac{3}{2}}} } $$ also valid for $0<|\xi|<1$.
Partial answer.
If $n$ is an integer, there is no doubt that @Robert Lee's elegant solution is the answer.
In terms of regularized generalized hypergeometric functions $$I_1=\frac{2 \pi }{(1-e)^3}\Bigg[\, _3\tilde{F}_2\left(\frac{3}{2},2,3;2-n,n+2;-\frac{2 e}{1-e}\right)-\, _3\tilde{F}_2\left(\frac{1}{2},1,3;1-n,n+1;-\frac{2 e}{1-e}\right) \Bigg]$$, provided that $\Im(e)\neq 0\lor 0\leq \Re(e)<1$.
There is no restriction about $n$.
For $I_2$, I am still stuck.
