This trick is due to Hermite. It is especially useful when you have a symmetric matrix of integers. First, we write a certain function in three variables,
$$ f(x,y,z) = x^2 + 3 y^2 + 5 z^2 + 8 yz+ 6 zx +4xy, $$ because this is exactly the result of calculating $v^t A v,$ with
$$
v =
\left(
\begin{array}{c}
x \\
y \\
z
\end{array}
\right)
$$
In order to clear the terms with $x,$ we write
$$ (x + 2 y + 3 z)^2 = x^2 + 4 y^2 + 9 z^2 + 12 yz+ 6 zx +4xy. $$
So far,
$$ f(x,y,z) - (x + 2 y + 3 z)^2 = -y^2 - 4 z^2 - 4 y z. $$
Next we clear all $y,$
$$ (y + 2 z)^2 = y^2 + 4 z^2 + 4 y z, $$ and
$$ f(x,y,z) - (x + 2 y + 3 z)^2 + (y + 2 z)^2 = 0, $$
$$ \color{red}{ f(x,y,z) = (x + 2 y + 3 z)^2 - (y + 2 z)^2 }. $$
The matrix multiplication that this shows is
$$
\left(
\begin{array}{ccc}
1 & 0 & 0 \\
2 & 1 & 0 \\
3 & 2 & 0
\end{array}
\right)
\left(
\begin{array}{ccc}
1 & 0 & 0 \\
0 & -1 & 0 \\
0 & 0 & 0
\end{array}
\right)
\left(
\begin{array}{ccc}
1 & 2 & 3 \\
0 & 1 & 2 \\
0 & 0 & 0
\end{array}
\right) =
\left(
\begin{array}{ccc}
1 & 2 & 3 \\
2 & 3 & 4 \\
3 & 4 & 5
\end{array}
\right)
$$
That is actually the right way to do it.
However, I see that someone asked for invertible $P,$ despite non-full rank. Also can be done, and easily:
$$
\left(
\begin{array}{ccc}
1 & 0 & 0 \\
2 & 1 & 0 \\
3 & 2 & 1
\end{array}
\right)
\left(
\begin{array}{ccc}
1 & 0 & 0 \\
0 & -1 & 0 \\
0 & 0 & 0
\end{array}
\right)
\left(
\begin{array}{ccc}
1 & 2 & 3 \\
0 & 1 & 2 \\
0 & 0 & 1
\end{array}
\right) =
\left(
\begin{array}{ccc}
1 & 2 & 3 \\
2 & 3 & 4 \\
3 & 4 & 5
\end{array}
\right)
$$
The effect of this is to add $0$ to the function in the shape of $0z^2.$
ADDED: looking at the question again, we do need the extra $1$ in the lower right, because the matrix $P$ requested is actually the inverse of the on I display above. Life goes on.