Let $A=\begin{pmatrix}1 & 2 & 3\\ 2 & 3 & 4\\ 3 & 4 & 5 \end{pmatrix}$ be a real matrix. Find an invertible matrix $P\in M_{3}(\mathbb{R})$ such that $P^TAP$ is diagonal whose elements on the diagonal are all in the set $\{-1, 0, 1\}$
I've been trying to solve this with Lagrange's squares method, but I'm stuck at the end where I seem to get "less" squares then required. I assume that this is because $A$ is degenerate, but I'm unable to complete my form to a basis that would satisfy the requirements as a result, and would love a general method for this.
These are my calculations so far:
Let $\begin{pmatrix}x_{1}\\ x_{2}\\ x_{3} \end{pmatrix}\in\mathbb{R}^{3}$. Then
$$\begin{aligned}\begin{pmatrix}x_{1}\\ x_{2}\\ x_{3} \end{pmatrix}^{T}\begin{pmatrix}1 & 2 & 3\\ 2 & 3 & 4\\ 3 & 4 & 5 \end{pmatrix}\begin{pmatrix}x_{1}\\ x_{2}\\ x_{3} \end{pmatrix}= & x_{1}^{2}+4x_{1}x_{2}+6x_{1}x_{3}+3x_{2}^{2}+5x_{3}^{2}+8x_{2}x_{3}\\ = & x_{1}^{2}+2x_{1}\left(2x_{2}+3x_{3}\right)+\left(2x_{2}+3x_{3}\right)^{2}\\ & -\left(2x_{2}+3x_{3}\right)^{2}+3x_{2}^{2}+5x_{3}^{2}+8x_{2}x_{3}\\ = & \left(x_{1}+\left(2x_{2}+3x_{3}\right)\right)^{2}-4x_{2}^{2}-12x_{2}x_{3}-9x_{3}^{2}+3x_{2}^{2}+5x_{3}^{2}+8x_{2}x_{3}\\ = & \left(x_{1}+2x_{2}+3x_{3}\right)^{2}-x_{2}^{2}-4x_{2}x_{3}-4x_{3}^{2}\\ = & \left(x_{1}+2x_{2}+3x_{3}\right)^{2}-\left(x_{2}+2x_{3}\right)^{2} \end{aligned} $$
and you can see I have only two "elements" for the basis, and I'm stuck..