Setting $I = \int^{\pi/2}_0 \ln^2 (\tan^2 x) \, dx$, after letting $u = \tan x$ the integral becomes
$$I = 4 \int^\infty_0 \frac{\ln^2 u}{1 + u^2} \, du.$$
Writing this integral as
$$I = 4 \int^1_0 \frac{\ln^2 u}{1 + u^2} \, du + 4 \int^\infty_1 \frac{\ln^2 u}{1 + u^2} \, du,$$
if, in the second of these integrals we set $u = 1/x$, one has
$$I = 4 \int^1_0 \frac{\ln^2 u}{1 + u^2} \, du + 4 \int^1_0 \frac{\ln^2 u}{1 + u^2} \, du = 8 \int^1_0 \frac{\ln^2 u}{1 + u^2} \, du.$$
Writing the term $1/(1 + x^2)$ that appears in the integrand as a geometric series, namely
$$\frac{1}{1 + x^2} = \sum^\infty_{n = 0} (-1)^n x^{2n}, \quad |x| < 1,$$
after interchanging the summation with the integration (by Tonelli's theorem) we have
$$I = 8 \sum^\infty_{n = 0} (-1)^n \int^1_0 x^{2n} \ln^2 x \, dx.$$
Integrating by parts twice, we find
$$I = 16 \sum^\infty_{n = 0} \frac{(-1)^n}{(2n + 1)^3}.$$
To evaluate the sum we write
\begin{align*}
I &= 16 \sum^\infty_{n = 0} \frac{(-1)^n}{(2n + 1)^3} = 2 \sum^{\infty}_{n = 0} \frac{(-1)^n}{(n + \frac{1}{2})^3}\\
&= 2 \sum^{\infty}_{n = 0,n \in \text{even}} \frac{1}{(n + \frac{1}{2})^3} - 2 \sum^{\infty}_{n = 0,n \in \text{odd}} \frac{1}{(n + \frac{1}{2})^3}.
\end{align*}
Reindexing, in the first sum let $n \mapsto 2n$ while in the second sum let $n \mapsto 2n + 1$. Thus
\begin{align*}
I &= 2 \sum^\infty_{n = 0} \frac{1}{(2n + \frac{1}{2})^3} - 2 \sum^\infty_{n = 0} \frac{1}{(2n + \frac{3}{2})^3}\\
&= \frac{1}{4} \sum^\infty_{n = 0} \frac{1}{(n + \frac{1}{4})^3} - \frac{1}{4} \sum^\infty_{n = 0} \frac{1}{(n + \frac{3}{4})^3}\\
&= -\frac{1}{4} \left [\zeta(3,\frac{3}{4}) - \zeta(3, \frac{1}{4}) \right ],
\end{align*}
where $\zeta (s,z)$ is the Hurwitz zeta function.
Now the polygamma function $\psi^{(m)} (z)$ is related to the Hurwitz zeta function by
$$\zeta (1 + m, z) = \frac{(-1)^{m + 1}}{m!} \psi^{(m)}(z).$$
Using this result to rewrite the Hurwitz zeta functions in terms of polygamma functions we have
\begin{align*}
\zeta (3, \frac{1}{4}) &= \zeta (1 + 2, \frac{1}{4}) = -\frac{1}{2} \psi^{(2)} (\frac{1}{4})\\
\zeta (3, \frac{3}{4}) &= \zeta (1 + 2, \frac{3}{4}) = -\frac{1}{2} \psi^{(2)} (\frac{3}{4})
\end{align*}
So our integral becomes
$$I = \frac{1}{8} \left [\psi^{(2)}(\frac{3}{4}) - \psi^{(2)} (\frac{1}{4}) \right ].$$
Now making use of the reflection formula for the polygamma function, namely
$$\psi^{(m)}(1 - z) + (-1)^{m + 1} \psi^{(m)}(z) = (-1)^m \pi \frac{d^m}{dz^m} \cot (\pi z),$$
setting $m = 2$ we have
\begin{align*}
\psi^{(2)}(\frac{3}{4}) - \psi^{(2)} (\frac{1}{4}) &= \psi^{(2)}(1 - \frac{1}{4}) - \psi^{(2)} (\frac{1}{4})\\
&= \pi \left. \frac{d^2}{dz^2} \cot (z \pi) \right |_{z = 1/4}\\
&= \pi [2 \pi^2 \cot (z\pi) \text{cosec}^2 (z \pi)]_{z = 1/4}\\
&= \pi \cdot 4\pi^2 = 4 \pi^3.
\end{align*}
So finally
$$I = \frac{1}{8} \cdot 4 \pi^3 = \frac{\pi^3}{2},$$
as required.