Find $x^5+\dfrac1{x^5}$ in its simplest form given that $x^2+\dfrac1{x^2}=a$ for $a,x>0$.
Attempt:
We write $$x^2+\frac1{x^2}=a\implies x^4-ax^2+1=0\implies x^5=ax^3-x$$ and $$\frac1{x^2}=a-x^2\implies \frac1{x^4}=a^2-2ax^2+x^4\implies \frac1{x^5}=\frac{a^2}x-2ax+x^3$$ so $$\begin{align}x^5+\frac1{x^5}&=(1+a)x^3-(1+2a)x+a^2\cdot\frac1x\\&=(1+a)x^3-(1+2a)x+a^2(ax-x^3)\\&=(1+a-a^2)x^3-(1+2a-a^3)x\\\implies x^5+\frac1{x^5}&=(a^2-a-1)x(a+1-x^2)\end{align}$$ But is this in its simplest form?
Hint: Find the value of $x+{1\over x}$ and from that find $x^3+{1\over x^3}$
After that multiply $x^3+{1\over x^3}$ and $x^2+{1\over x^2}$
Following @LoveInvariants's hint, we get the following. $$\left(x+\frac1x\right)^2=x^2+\frac1{x^2}+2=a+2\implies x+\frac1x=\sqrt{a+2}$$ since $x>0$.
Next, we have $$\left(x^2+\frac1{x^2}\right)\left(x+\frac1x\right)=x^3+\frac1{x^3}+x+\frac1x=a\sqrt{a+2}$$ so $$x^3+\frac1{x^3}=(a-1)\sqrt{a+2}$$
Now do $$\left(x^3+\frac1{x^3}\right)\left(x^2+\frac1{x^2}\right)=x^5+\frac1{x^5}+x+\frac1x=a(a-1)\sqrt{a+2}$$ and therefore we have $$x^5+\frac1{x^5}=[a(a-1)-1]\sqrt{a+2}$$
Since $x+1/x=\sqrt{a+2}$ while $x^4+1/x^4=a^2-2$, $$x^3+1/x^3=(x+1/x)(x^2+1/x^2-1)=(a-1)\sqrt{a+2}$$ while $$x^5+1/x^5=(x^2+1/x^2)(x^3+1/x^3)-(x+1/x)= (a^2-a-1)\sqrt{a+2}.$$
Just to give a slightly different approach, note that
$$a^2=\left(x^2+{1\over x^2}\right)^2=x^4+{1\over x^4}+2\implies x^4+{1\over x^4}=a^2-2$$
and
$$a^3=\left(x^2+{1\over x^2}\right)^3=x^6+{1\over x^6}+3\left(x^2+{1\over x^2}\right)\implies x^6+{1\over x^6}=a^3-3a$$
From these we find that
$$\left(x^5+{1\over x^5}\right)\left(x+{1\over x}\right)=x^6+{1\over x^6}+x^4+{1\over x^4}=a^3-3a+a^2-2=(a^2-a-1)(a+2)$$
Finally,
$$\left(x+{1\over x}\right)^2=x^2+{1\over x^2}+2=a+2\implies x+{1\over x}=\sqrt{a+2}$$
(using the assumption $x,a\gt0$), so we have
$$x^5+{1\over x^5}=(a^2-a-1)\sqrt{a+2}$$
Observe that $$(X+1/X)^2=a+2$$. This gives you $X+1/X$. Now note that $$(X+1/X)^3=X^3+1/X^3+3(X+1/X)$$ and obtain $X^3+1/X^3$. Finally $$(X^2+1/X^2)(X^3+1/X^3)=X^5+1/X^5+(X+1/X)$$.
$$ x+\frac{1}{x} = b\Rightarrow \left(x+\frac{1}{x}\right)^2=b^2 $$
then
$$ x^2+\frac{1}{x^2}= b^2-2 = a\Rightarrow b = \pm\sqrt{a+2} $$
$$ \left(x^2+\frac{1}{x^2}\right)^2=a^2\Rightarrow x^4+\frac{1}{x^4} = a^2-2 $$
$$ \left(x^2+\frac{1}{x^2}\right)^2\left(x+\frac{1}{x}\right) = x^3+\frac{1}{x^3}+x+\frac{1}{x} $$
$$ \left(x^4+\frac{1}{x^4}\right)^2\left(x+\frac{1}{x}\right) =x^5+\frac{1}{x^5}+x^3+\frac{1}{x^3} $$
Note that $$\bigg(x+\frac 1x\bigg)^2=x^2+\frac{1}{x^2}+2\to x+\frac 1x=\sqrt{a+2}$$ $$\bigg(x+\frac{1}{x}\bigg)\bigg(x^2+\frac{1}{x^2}\bigg)=x^3+\frac{1}{x^3}+x+\frac 1x\to x^3+\frac{1}{x^3}=(a-1)\sqrt{a+2}$$ $$\bigg(x^2+\frac{1}{x^2}\bigg)^2=x^4+\frac{1}{x^4}+2\to x^4+\frac{1}{x^4}=a^2-2$$ $$\bigg(x^4+\frac{1}{x^4}\bigg)\bigg(x+\frac 1x\bigg)=x^5+\frac{1}{x^5}+x^3+\frac{1}{x^3}$$ $$\to (a^2-2)(\sqrt{a+2})=x^5+\frac{1}{x^5}+(a-1)(\sqrt{a+2})$$ $$\to x^5+\frac{1}{x^5}=(a^2-a-1)(\sqrt{a+2})$$
With $S_n:=x^n+x^{-n}$, you can establish the recurrence
$$S_{n+1}=S_nS_1-S_{n-1}.$$
From this, $$S_2=S_1^2-2, \\S_3=S_1(S_2-1), \\S_4=S_2^2-2, \\S_5=S_1(S_2^2-S_2-1). $$
You draw $S_1=\sqrt{S_2+2}$ from the first identity.