I saw in a paper by @Jack D'aurizio the following integral $$I=\int_0^{\pi/2}\log^2(\cos^2x)\mathrm{d}x=\frac{\pi^3}6+2\pi\log^2(2)$$ Below is my attempt.
$$I=4\int_0^{\pi/2}\log^2(\cos x)\mathrm{d}x$$ Then we define $$F(a)=\int_0^{\pi/2}\log^2(a\cos x)\mathrm{d}x$$ So we have $$F'(a)=\frac2a\int_0^{\pi/2}\log(a\cos x)\mathrm{d}x$$ Which I do not know how to compute. How do I proceed? Thanks.
Let $$I(a)=\int_0^{\frac {\pi}{2}} (\cos^2 x)^a dx$$
Hence we need $I''(0)$.
Now recalling the definition of Beta function we get $$I(a)=\frac 12 B\left(a+\frac 12 ,\frac 12\right)=\frac {\sqrt {\pi}}{2}\frac {\Gamma\left(a+\frac 12\right)}{\Gamma(a+1)}$$
Hence we have $$I''(a) =\frac {\sqrt {\pi}}{2}\frac {\Gamma\left(a+\frac 12\right)}{\Gamma(a+1)}\left(\left[\psi^{(0)}\left(a+\frac 12 \right)-\psi^{(0)}(a+1)\right]^2 +\psi^{(1)}\left(a+\frac 12 \right)-\psi^{(1)}(a+1)\right) $$
Substituting $a=0$ in above formula yields the answer.
Differentiation of the Beta function is a viable approach, but since we are dealing with a squared logarithm and not higher powers, it is faster to just exploit the Fourier (cosine) series of $\log(\cos^2\theta)$ and Parseval's theorem. Given
$$-\log(\cos^2\theta)=2\log 2+2\sum_{k\geq 1}\frac{(-1)^k}{k}\cos(2k\theta),\tag{1} $$ since $\int_{0}^{\pi/2}\cos(2jx)\cos(2kx)\,dx =\frac{\pi}{4}\delta(j,k)$, we immediately have $$ \int_{0}^{\pi/2}\log^2(\cos^2\theta)\,d\theta = 2\pi\log^2 2+\pi\zeta(2).\tag{2}$$
Since the LHS equals $\int_{0}^{1}\frac{4\log^2 x}{\sqrt{1-x^2}}\,dx$, we have just found the value of the hypergeometric series $$ 8\sum_{n\geq 0}\frac{\binom{2n}{n}}{4^n(2n+1)^3}=8\cdot\phantom{}_4 F_3\left(\tfrac{1}{2},\tfrac{1}{2},\tfrac{1}{2},\tfrac{1}{2};\tfrac{3}{2},\tfrac{3}{2},\tfrac{3}{2};1\right)\tag{3} $$ which also equals $\int_{0}^{+\infty}\frac{\log^2(1+t^2)}{1+t^2}\,dt$ or $\frac{1}{2}\int_{0}^{1}\frac{\log^2(x)}{\sqrt{x(1-x)}}\,dx$.
\begin{align}J=\int_0^{\frac{\pi}{2}} \ln^2\left(\cos x\right)\,dx\end{align}
Observe that,
\begin{align}I&=4J\\ J&=\int_0^{\frac{\pi}{2}} \ln^2\left(\sin x\right)\,dx\\ \int_0^{\frac{\pi}{2}} \ln\left(\sin x\right)\,dx&=\int_0^{\frac{\pi}{2}} \ln\left(\cos x\right)\,dx \end{align}
(change of variable $y=\dfrac{\pi}{2}-x$ )
\begin{align} K&=\int_0^{\frac{\pi}{2}} \ln^2 \left(2\sin x\cos x\right)\,dx\\ &=\int_0^{\frac{\pi}{2}} \ln^2 \left(\sin\left(2x\right)\right)\,dx\\ \end{align}
Perform the change of variable $y=2x$,
\begin{align} K&=\frac{1}{2}\int_0^{\pi} \ln^2 \left(\sin x\right)\,dx\\ &=\frac{1}{2}\int_0^{\frac{\pi}{2}} \ln^2 \left(\sin x\right)\,dx+\frac{1}{2}\int_{\frac{\pi}{2}}^\pi \ln^2 \left(\sin x\right)\,dx\\ \end{align}
In the latter integral perform the change of variable $y=\dfrac{\pi}{2}-x$ and recall $\sin\left(\pi-x\right)=\sin x$ for $x$ real,
\begin{align} K&=\int_0^{\frac{\pi}{2}} \ln^2 \left(\sin x\right)\,dx\\ &=\int_0^{\frac{\pi}{2}} \ln^2 \left(\cos x\right)\,dx\\ &=J \end{align}
On the other hand,
\begin{align} K&=\int_0^{\frac{\pi}{2}}\left(\ln 2+\ln(\sin x)+\ln(\cos x)\right)^2 \,dx\\ &=\frac{\pi}{2}\ln^2 2+\int_0^{\frac{\pi}{2}}\ln^2(\sin x)\,dx+\int_0^{\frac{\pi}{2}}\ln^2(\cos x)\,dx+2\ln 2\int_0^{\frac{\pi}{2}}\ln(\cos x)\,dx+\\ &2\ln 2\int_0^{\frac{\pi}{2}}\ln(\sin x)\,dx+2\int_0^{\frac{\pi}{2}}\ln(\sin x)\ln(\cos x)\,dx\\ &=\frac{\pi}{2}\ln^2 2+2J+4\ln 2\int_0^{\frac{\pi}{2}}\ln(\cos x)\,dx+2\int_0^{\frac{\pi}{2}}\ln(\sin x)\ln(\cos x)\,dx\\ \end{align}
\begin{align}L&=\int_0^\infty \frac{\ln^2 x}{1+x^2}\,dx\end{align}
Perform the change of variable $x=\tan y$,
\begin{align}L&=\int_0^{\frac{\pi}{2}} \ln^2\left(\tan x\right)\,dx\\ &=\int_0^{\frac{\pi}{2}}\left(\ln\left(\sin x\right)-\ln\left(\cos x\right)\right)^2\,dx\\ &=2J-2\int_0^{\frac{\pi}{2}}\ln(\sin x)\ln(\cos x)\,dx\\ \end{align}
Therefore,
\begin{align}K+L&=\frac{\pi}{2}\ln^2 2+4J+4\ln 2\int_0^{\frac{\pi}{2}}\ln(\cos x)\,dx \end{align}
Therefore (recall $K=J$),
\begin{align}J&=\frac{1}{3}L-\frac{\pi}{6}\ln^2 2-\frac{4}{3}\ln 2\int_0^{\frac{\pi}{2}}\ln(\cos x)\,dx\end{align}
On the other hand,
\begin{align}L&=\int_0^1 \frac{\ln^2 x}{1+x}\,dx+\int_1^\infty \frac{\ln^2 x}{1+x}\,dx\end{align}
In the latter integral perform the change of variable $y=\dfrac{1}{x}$,
\begin{align}L&=2\int_0^1 \frac{\ln^2 x}{1+x}\,dx\end{align}
But it is well known that,
\begin{align}\int_0^1 \frac{\ln^2 x}{1+x^2}\,dx&=\frac{\pi^3}{16}\\ \int_0^{\frac{\pi}{2}}\ln(\cos x)\,dx&=-\frac{1}{2}\pi\ln 2 \end{align}
Therefore,
\begin{align}J&=\frac{\pi^3}{24}-\frac{\pi}{6}\ln^2 2-\frac{4}{3}\ln 2\times -\frac{1}{2}\pi\ln 2\\ &=\boxed{\frac{\pi^3}{24}+\frac{1}{2}\pi\ln^2 2}\\ \end{align}
PS: See: https://math.stackexchange.com/a/2942594/186817
(in this post i assume only the value of $\zeta(4)$ )
Since $$\sum_{n=0}^{\infty}\frac{H_n}{n+1} x^{n+1} =\frac{\log^2(1-x)}{2}$$ with $x \mapsto \sin^2 x$. and hence on integrating we obtain $$\int_0^{\frac{\pi}{2}}\log^2(\cos^2 x) dx = \pi\sum_{n=1}^{\infty}\frac{H_{n-1}}{n4^n}{2n\choose n}=\pi\sum_{n=1}^{\infty}\frac{H_n}{n4^n}{2n\choose n} -\pi\sum_{n=1}^{\infty}\frac{1}{n^24^n}{2n\choose n}$$ since the latter series $\mathcal{LS}$ is known whose value is $\displaystyle \zeta(2)-2\log^2(2)$ which is easily do able by computing the following integral $$2\int_0^1 \log\left(\frac{2}{\sqrt {1-x}+1}\right)\frac{dx}{x}=\zeta(2)-2\log^2(2)$$. Below I have derived the generating function for $\displaystyle \sum_{n\geq 1}{2n\choose n}\frac{y^n}{n^24^n}$
and to evaluate the former series $\mathcal{FS}$, we use the elementary integral $\displaystyle \int_0^1 x^{n-1} \log(1-x)dx =-\frac{H_n}{n}$ and generating function of central binomial coefficients $\displaystyle \sum_{n\geq 1} \frac{x^n}{4^n}{2n\choose n}=\frac{1}{\sqrt{1-x}}-1$ which we too use to calculate the latter series. Now, we just need to divide both sides by x, multiply $\log(1-x)$ and integrating from 0 to 1 gives $$-\mathcal{FS} =\int_0^1\frac{\log(1-x)}{x\sqrt{1-x}}dx-\int_0^1\frac{\log(1-x)}{x}dx=4\int_0^{\frac{\pi}{2}}\frac{\log(x)}{1-x^2} dx+\zeta(2)$$ since $$\int_0^1\frac{\log(1-x)}{x\sqrt {1-x}} dx=2\int_0^1\frac{\log\left(\sqrt{1-x}\right)}{x\sqrt{1-x}}dx\stackrel{\rm \sqrt{1-x}\mapsto x}{=}4\int_0^1\frac{\log(x)}{1-x^2}dx=-\frac{\pi^2}{2}$$ so $$\displaystyle -\pi\mathcal{FS}=\frac{\pi^3}{3},\; \pi\mathcal{LS} =\frac{\pi^3}{6}-2\pi\log^2(2)$$ and hence $-\pi\mathcal{FS}-\pi\mathcal{LS} =\frac{\pi^3}{6}+2\pi\log^2(2)$.
Getting rid of the first roadblock:
Use the fact that $$\log(a\cos x) = \log a + \log\cos x$$ along with the result that $$ \int_0^{\pi/2} \log\cos x\,dx = -\frac{\pi}{2}\log 2 $$ to get $$ F'(a) = \frac{\pi\log a}{a} -\frac{\pi}{2a}\log 2 $$
$\newcommand{\bbx}[1]{\,\bbox[15px,border:1px groove navy]{\displaystyle{#1}}\,} \newcommand{\braces}[1]{\left\lbrace\,{#1}\,\right\rbrace} \newcommand{\bracks}[1]{\left\lbrack\,{#1}\,\right\rbrack} \newcommand{\dd}{\mathrm{d}} \newcommand{\ds}[1]{\displaystyle{#1}} \newcommand{\expo}[1]{\,\mathrm{e}^{#1}\,} \newcommand{\ic}{\mathrm{i}} \newcommand{\mc}[1]{\mathcal{#1}} \newcommand{\mrm}[1]{\mathrm{#1}} \newcommand{\pars}[1]{\left(\,{#1}\,\right)} \newcommand{\partiald}[3][]{\frac{\partial^{#1} #2}{\partial #3^{#1}}} \newcommand{\root}[2][]{\,\sqrt[#1]{\,{#2}\,}\,} \newcommand{\totald}[3][]{\frac{\mathrm{d}^{#1} #2}{\mathrm{d} #3^{#1}}} \newcommand{\verts}[1]{\left\vert\,{#1}\,\right\vert}$
$\ds{I \equiv \int_{0}^{\pi/2}\ln^{2}\pars{\cos^{2}\pars{x}}\,\dd x = {\pi^{3} \over 6} + 2\pi\ln^{2}\pars{2}:\ {\LARGE ?}}$.
\begin{align} I & \equiv \bbox[10px,#ffd]{\int_{0}^{\pi/2}\ln^{2}\pars{\cos^{2}\pars{x}}\,\dd x} \,\,\,\stackrel{x\ \mapsto\ \pi/2\ -\ x}{=}\,\,\, 4\int_{0}^{\pi/2}\ln^{2}\pars{\sin\pars{x}}\,\dd x \\[5mm] & = \left.4\,\Re\int_{x\ =\ 0}^{x\ =\ \pi/2}\ln^{2}\pars{{1 - z^{2} \over 2z}\,\ic}\,{\dd z \over \ic z} \,\right\vert_{\ z\ =\ \exp\pars{\ic x}} \\[5mm] & = \left.4\,\Im\int_{x\ =\ 0}^{x\ =\ \pi/2}\ln^{2}\pars{{1 - z^{2} \over 2z}\,\ic}\,{\dd z \over z} \,\right\vert_{\ z\ =\ \exp\pars{\ic x}} \\[1cm] & \stackrel{\mrm{as}\ \epsilon\ \to\ 0^{+}}{\sim}\,\,\, -\, 4\,\ \overbrace{\Im\int_{1}^{\epsilon}\ln^{2} \pars{1 + y^{2} \over 2y}\,{\dd y \over y}}^{\ds{=\ \large 0}} \\[2mm] & \phantom{\stackrel{\mrm{as}\ \epsilon\ \to\ 0^{+}}{\sim}\,\,\,\,\,\,\,} - 4\,\Im\int_{\pi/2}^{0}\bracks{\ln\pars{1 \over 2\epsilon} + \pars{{\pi \over 2} - \theta}\ic}^{2}\,\ic\,\dd\theta \\[2mm] & \phantom{\stackrel{\mrm{as}\ \epsilon\ \to\ 0^{+}}{\sim}\,\,\,\,\,\,\,\,} -4\,\Im\int_{\epsilon}^{1}\bracks{\ln\pars{1 - x^{2} \over 2x} + {\pi \over 2}\,\ic}^{2}{\dd x \over x} \\[1cm] & = 4\int_{0}^{\pi/2}\bracks{\ln^{2}\pars{2\epsilon} - \pars{{\pi \over 2} - \theta}^{2}}\,\dd\theta -4\pi\int_{\epsilon}^{1}\ln\pars{1 - x^{2} \over 2x} {\dd x \over x} \\[5mm] & \stackrel{\mrm{as}\ \epsilon\ \to\ 0^{+}}{\sim} \bracks{2\pi\ln^{2}\pars{2\epsilon} - {\pi^{3} \over 6}} - \bracks{4\pi\int_{0}^{1}{\ln\pars{1 - x^{2}} \over x}\,\dd x - 4\pi\int_{\epsilon}^{1}{\ln\pars{2x} \over x}\,\dd x} \\[5mm] & = \bracks{2\pi\ln^{2}\pars{2\epsilon} - {\pi^{3} \over 6}} - \bracks{2\pi\int_{0}^{1}{\ln\pars{1 - x} \over x}\,\dd x + 2\pi\ln\pars{\epsilon}\ln\pars{4\epsilon}} \\[5mm] & \stackrel{\mrm{as}\ \epsilon\ \to\ 0}{\to} \bbx{{\pi^{3} \over 6} + 2\pi\ln^{2}\pars{2}} \approx 8.1865 \end{align}
Note that $\ds{\int_{0}^{1}{\ln\pars{1 - x} \over x}\,\dd x = -\,\mrm{Li}\pars{1} = -\,{\pi^{2} \over 6}}$.
Noting that $$ \int_{0}^{\frac{\pi}{2}} \ln ^{2}\left(\cos ^{2} x\right) d x=4 \int_{0}^{\frac{\pi}{2}} \ln ^{2}(\cos x) dx $$
Let $$ I:=\int_{0}^{\frac{\pi}{2}} \ln ^{2}(\cos x) d x \stackrel{x\mapsto\frac{\pi}{2}-x}{=} \int_{0}^{\frac{\pi}{2}} \ln ^{2}(\sin x) d x $$
Using the identity $$[\ln (\sin x)+\ln (\cos x)]^{2}+[\ln (\sin x)-\ln (\cos x)]^{2} =2\left[\ln ^{2}(\sin x)+\ln ^{2}(\cos x)\right],$$
we have $$ 4 I=\underbrace{\int_{0}^{\frac{\pi}{2}} \ln ^{2}\left(\frac{\sin 2 x}{2}\right)}_{J} d x+\underbrace{\int_{0}^{\frac{\pi}{2}} \ln ^{2}(\tan x) d x}_{K} $$ For the first integral, using $\int_{0}^{\frac{\pi}{2}} \ln (\sin x) d x=-\dfrac{\pi}{2} \ln 2$ yields $$ \begin{aligned} J &=\int_{0}^{\frac{\pi}{2}}[\ln (\sin 2 x)-\ln 2]^{2} d x \\ &=\int_{0}^{\frac{\pi}{2}} \ln ^{2}(\sin 2 x) d x-2 \ln 2 \int_{0}^{\frac{\pi}{2}} \ln (\sin 2 x) d x +\frac{\pi \ln ^{2} 2}{2} \\ & \stackrel{x\mapsto 2x}{=} \frac{1}{2} \int_{0}^{\pi} \ln ^{2}(\sin x) d x-\ln 2 \int_{0}^{\pi} \ln (\sin x) d x+\frac{\pi \ln ^{2} 2}{2} \\ & \stackrel{symmetry}{=} I-\ln 2(-\pi \ln 2)+\frac{\pi \ln ^{2} 2}{2} \\ &=I+\frac{3 \pi \ln ^{2} 2}{2} \end{aligned} $$ For the second integral, letting $y=\tan x $ and using the post, yields $$ \int_{0}^{\infty} \frac{\ln ^{2} y}{1+y^{2}} d y=\frac{\pi^{3}}{8}, $$
then $$ 4I=I+\frac{3 \pi \ln ^{2} 2}{2}+\frac{\pi^{3}}{8} \Rightarrow I=\frac{\pi^{3}}{24}+\frac{\pi \ln ^{2} 2}{2} $$
Hence $$\boxed{\int_{0}^{\frac{\pi}{2}} \ln ^{2}\left(\cos ^{2} x\right) d x= \frac{\pi^{3}}{6}+2 \pi \ln ^{2} 2}$$
