Question: How would you prove the equation$$\small\cos\dfrac {2\pi}{17}=\dfrac {-1+\sqrt{17}+\sqrt{34-2\sqrt{17}}+2\sqrt{17+3\sqrt{17}-\sqrt{34-2\sqrt{17}}-2\sqrt{34+2\sqrt{17}}}}{16}\tag1$$
I'm not too sure how to prove it and I'm not sure where to begin. I started with $\exp(2\pi i)=\cos(2\pi)+i\sin(2\pi)=1\implies \cos(2k\pi)+i\sin(2k\pi)=\exp(2k\pi)$. But I'm not sure what to do next.
I will show this by using Galois-theory. I will be using some properties of the fields $\mathbf{Q}(\zeta_p)$ with $p$ prime, which you can find here on page 51.
I start by looking at the extension $\mathbf{Q} \subset \mathbf{Q}(\zeta_{17})$, where I define $\zeta:=\zeta_{17}:= \text{exp}( \frac{2 \pi i}{17}) = \cos\left(\frac{2\pi}{17}\right)+i\sin\left(\frac{2\pi}{17}\right)$. The galois-group $\text{Gal}(\mathbf{Q}(\zeta)/\mathbf{Q})$ is isomorphic to $\mathbf{F}_{17}^*$. Note that the order of this group is $16$, which is a power of two, therefore $\zeta$ is constructible using ruler and compass. In other words, it is possible to express $\zeta$ using square roots (square roots of square roots ..).
Consider the Gauss periods: \begin{align} \eta_2&:=\zeta +\zeta^{-1} +\zeta^2 +\zeta^{-2} + \zeta^4 + \zeta^{-4} + \zeta^8 +\zeta^{-8} \\ \eta_4&:= \zeta +\zeta^{-1} + \zeta^4 + \zeta^{-4}\\ \eta_8&:= \zeta^1 + \zeta^{-1}. \end{align} Notice furthermore $\zeta^{-1}$ is the complex conjugated of $\zeta$, thus $\eta_8 := \zeta+\zeta^{-1} = 2\cos\left(\frac{2\pi}{17}\right)$. Therefore, I will express $\eta_8$ using radicals.
Let's look at the tower of fields: $\mathbf{Q} \subset \mathbf{Q}(\eta_2) \subset \mathbf{Q}(\eta_4) \subset \mathbf{Q}(\eta_8) \subset \mathbf{Q}(\zeta)$. Note that $\mathbf{Q}(\eta_8)$ corresponds to the unique subgroup $H_8:=\langle -1 \rangle \subset \mathbf{F}_{17}^*$ of index $8$. Where $\mathbf{Q}(\eta_4)$ resp. $\mathbf{Q}(\eta_2)$ correspond to the unique subgroups $H_4 := \langle 4 \rangle$ resp. $H_2:=\langle -8 \rangle$ of index $4$ resp. $2$. So, each step in the tower is quadratic, and we have the corresponding chain of subgroups: $\mathbf{F}_{17}^* \supset H_2 \supset H_4 \supset H_8 \supset \{1\}$.
What I will do now is express $\eta_4$ in terms of $\eta_2$ (and square roots of it), and $\eta_8$ in terms of $\eta_4$ and $\eta_2$ (and square roots of them), by determining minimum polynomials of each $\eta_j$ over $\mathbf{Q}(\eta_k) \subset \mathbf{Q}(\eta_j)$ and then using the abc-formula for quadratics.
The minimum polynomial of $\eta_2$ over $\mathbf{Q}$ denoted by $f^{\eta_2}_{\mathbf{Q}}$ is given by: \begin{align} f^{\eta_2}_{\mathbf{Q}} = (x-\eta_2)\cdot \left(x-(\zeta^3+\zeta^{-3}+\zeta^5+\zeta^{-5}+\zeta^6+\zeta^{-6}+\zeta^7+\zeta^{-7} \right)=x^2+x-4. \end{align} This fact follows from the fact that $\mathbf{F}_{17}^* / H_2 = \{ H_2, 3H_2\}$ ($\eta_2$ has one different conjugate). I can see that $\eta_2 = \frac{1}{2}(-1\pm \sqrt{17})$ using the abc-formula. Note that you can verify with a calculator (check that the sum$\sum_{k \in H_2}\cos(2k\frac{\pi}{17})$ is positive), by using the definition of $\eta_2$, that in fact: \begin{align} \eta_2 = \frac{1}{2}(-1+ \sqrt{17}). \end{align}
I turn to $\eta_4$, where I have that $\mathbf{F}_{17}^* / H_4 = \{ H_4,3H_4,6H_4,8H_4\}$, which corresponds to the extension $\mathbf{Q} \subset \mathbf{Q}(\eta_4)$. However, I restrict to $\mathbf{Q}(\eta_2) \subset \mathbf{Q}(\eta_4)$ which corresponds to $H_2/H_4 = \{ H_4,8H_4\}$ ($\eta_4$ has one other conjugate over $\mathbf{Q}(\eta_2)$). Therfore the minimum polynomial $f^{\eta_4}_{\mathbf{Q}(\eta_2)}$ of $\eta_4$ over $\mathbf{Q}(\eta_2)$ is given by: \begin{align} f^{\eta_4}_{\mathbf{Q}(\eta_2)} &= (x-\eta_4)\cdot \left(x- (\zeta^2+\zeta^{-2}+\zeta^8+\zeta^{-8} )\right) \\ &= x^2 -(\eta_4 +\zeta^2+\zeta^{-2}+\zeta^8+\zeta^{-8})x +\eta_4 \cdot (\zeta^2+\zeta^{-2}+\zeta^8+\zeta^{-8})\\ &= x^2-\eta_2x-1. \end{align} Where you can verify that $\eta_4 \cdot (\zeta^2+\zeta^{-2}+\zeta^8+\zeta^{-8}) = -1$ using that the minimum polynomial of $\zeta$ is $\Phi_{17} = \sum^{16}_{k=0}x^k$ ( in other words $\sum^{16}_{k=0}\zeta^k =0$). Now using the abc-formula you can see that: $\eta_4 = \frac{1}{2}(\eta_2 \pm \sqrt{\eta_2^2+4})$. Where you can check again that in fact: \begin{align} \eta_4 = \frac{1}{2}(\eta_2 + \sqrt{\eta_2^2+4}), \end{align} by simply noting that $\sum_{k \in H_4} \cos(2k\frac{\pi}{17})$ is positive.
Last but not least, I consider $\eta_8$. I'm interested in the extension $\mathbf{Q}(\eta_4) \subset \mathbf{Q}(\eta_8)$ with corresponding group $H_4/H_8 = \{ H_8,4H_8\}$. So, the minimum polynomial of $\eta_8$ over $\mathbf{Q}(\eta_4)$ is given by: \begin{align} f^{\eta_8}_{\mathbf{Q}(\eta_4)} &= (x-\eta_8) \cdot \left(x- (\zeta^4+\zeta^{-4})\right) \\ &= x^2-(\eta_8+\zeta^4+\zeta^{-4})x+\eta_8(\zeta^4+\zeta^{-4})\\ &= x^2-\eta_4x+\beta. \end{align} Now $\beta := \eta_8(\zeta^4+\zeta^{-4}) = (\zeta+\zeta^{-1})(\zeta^4+\zeta^{-4})$ can be expressed using $\eta_4$ and $\eta_2$ by noticing that: \begin{align} \eta_4^2 &:= (\zeta+\zeta^{-1} + \zeta^4+\zeta^{-4})^2 = (\zeta+\zeta^{-1})^2 + 2(\zeta+\zeta^{-1})(\zeta^4+\zeta^{-4}) + (\zeta^4+\zeta^{-4})^2\\ &= \zeta^2+\zeta^{-2} + \zeta^8+\zeta^{-8} + 4 + 2 \beta \\ &= \eta_2 -\eta_4 + 4 + 2\beta. \end{align} Thus $\beta := \eta_8(\zeta^4+\zeta^{-4}) = \frac{1}{2} (\eta_4^2+\eta_4-\eta_2-4)$. So I can see that: \begin{align} f^{\eta_8}_{\mathbf{Q}(\eta_4)} = x^2 - \eta_4x + \frac{1}{2} (\eta_4^2+\eta_4-\eta_2-4). \end{align} Using the abc-formula (and checking whether it has to plus or minus) I find that: \begin{align} \eta_8 &= \frac{1}{2} \left(\eta_4 + \sqrt{\eta_4^2-2(\eta_4^2+\eta_4-\eta_2-4)}\right)\\ &= \frac{1}{2} \left(\eta_4 + \sqrt{-\eta_4^2-2(\eta_4-\eta_2-4)}\right). \end{align} Working everything out (not very long) you end up with: \begin{align} \eta_8 = \frac{1}{8} (-1 + \sqrt{17}) + \frac{1}{8} \sqrt{34-2\sqrt{17}} + \frac{1}{4} \sqrt{17+3\sqrt{17} - \frac{1}{2}(3+\sqrt{17})\sqrt{34-2\sqrt{17}}}. \end{align} Now I conclude that: \begin{align} \cos\left({\frac{2\pi}{17}}\right) = \frac{1}{2}\eta_8 = \frac{1}{16} (-1 + \sqrt{17}) + \frac{1}{16} \sqrt{34-2\sqrt{17}} + \frac{1}{8} \sqrt{17+3\sqrt{17} - \frac{1}{2}(3+\sqrt{17})\sqrt{34-2\sqrt{17}}}. \end{align} Note that in the big square root the term $ \frac{1}{2}(3+\sqrt{17})\sqrt{34-2\sqrt{17}}$ can be seen, while Gauss actually gave the term $-\sqrt{34-2\sqrt{17}}-2\sqrt{34+2\sqrt{17}}$. You can show by squaring both expressions that you get $170+38\sqrt{17}$, thus both are roots of $170+38\sqrt{17}$, and both are negative, therefore they represent the same number.
You can try the same process with $\zeta_{257}$ (which will take 7 steps, this took 3 steps).
The more natural item is $$ 2 \cos \left( \frac{2 \pi}{17} \right), $$ which is one root of $$ x^8 + x^7 - 7 x^6 - 6 x^5 + 15 x^4 + 10 x^3 - 10 x^2 - 4 x + 1, $$ from page 18 of Reuschle (1875) using a method due to Gauss, having to do with cyclotomy, and predating Galois. If $\omega \neq 1$ but $\omega^{17} = 1,$ it is not hard to show that $\omega + \frac{1}{\omega}$ is a root of the given polynomial, using $$ \small 1 + \omega + \omega^2 + \omega^3 + \omega^4 + \omega^5 + \omega^6 + \omega^7 + \omega^8 + \omega^9 + \omega^{10} + \omega^{11} + \omega^{12} + \omega^{13} + \omega^{14} + \omega^{15} + \omega^{16} =0 $$
