Empirically, i have obtained the following value: \begin{align}K&=\int_0^1 \frac{\arctan x\ln^2 x}{1+x^2}\,dx\\ &=\frac{151}{11520}\pi^4-\frac{1}{24}\ln^4 2-\text{Li}_4\left(\frac{1}{2}\right)+\frac{1}{24}\pi^2\ln^2 2-\frac{7}{8}\zeta(3)\ln 2\end{align}
How to prove this?
My attempt:
Observe:
\begin{align}K&=\int_0^1 \int_0^1\frac{x\ln^2 x}{(1+x^2)(1+t^2x^2)}\,dt\,dx\\
\end{align}
On the other hand,
\begin{align}K&\overset{\text{IBP}}=\left[\left(\int_0^x \frac{\ln^2 t}{1+t^2}\,dt\right)\arctan x\right]_0^1-\int_0^1 \int_0^1\frac{x\ln(tx)^2}{(1+x^2)(1+t^2x^2)}\,dt\,dx\\ &=\frac{\pi^4}{64}-K-\int_0^1\int_0^1 \frac{x\ln^2 t}{(1+x^2)(1+t^2x^2)}\,dt\,dx-2\int_0^1\int_0^1 \frac{x\ln t\ln x}{(1+x^2)(1+t^2x^2)}\,dt\,dx\\ \end{align} Moreover, one can prove: \begin{align}\int_0^1 \int_0^1\frac{x\ln^2 t}{(1+x^2)(1+t^2x^2)}\,dt\,dx&=\frac{1}{64}\pi^4-\text{G}^2\end{align}
Unfortunately, $\displaystyle U= \int_0^1\int_0^1 \frac{x\ln t\ln x}{(1+x^2)(1+t^2x^2)}\,dt\,dx$ seems not easier to compute than $K$
Edit: \begin{align}U&=\int_0^1\int_0^1 \frac{x\ln t\ln x}{(1-t^2)(1+x^2)}\,dt\,dx -\int_0^1\int_0^1 \frac{xt^2\ln t\ln x}{(1-t^2)(1+t^2x^2)}\,dt\,dx\\ &=\frac{1}{384}\pi^4-\int_0^1\int_0^1 \frac{xt^2\ln t\ln(tx)}{(1-t^2)(1+t^2x^2)}\,dt\,dx+\int_0^1\int_0^1 \frac{xt^2\ln^2 t}{(1-t^2)(1+t^2x^2)}\,dt\,dx\\ \end{align} The last one is doable and, \begin{align}V&=\int_0^1\int_0^1 \frac{xt^2\ln t\ln(tx)}{(1-t^2)(1+t^2x^2)}\,dt\,dx\\ &=\int_0^1 \frac{\ln t}{1-t^2}\left(\int_0^t \frac{u\ln u}{1+u^2}\,du\right)\,dt\\ &=\frac{1}{4}\int_0^1 \frac{\ln t}{1-t^2}\left(\int_0^{t^2} \frac{\ln u}{1+u}\,du\right)\,dt\\ \end{align}
Edit2:
Since for $t\neq 1$, $\displaystyle \frac{1}{1-t^2}=\frac{1}{2}\times \frac{2t}{1-t^2}+\frac{1}{1+t}$ then,
\begin{align}V&=\frac{1}{4}\int_0^1 \left(\frac{1}{2}\times \frac{2t}{1-t^2}+\frac{1}{1+t}\right)\ln t\left(\int_0^{t^2} \frac{\ln u}{1+u} \,du\right)\,dt\\ &=\frac{1}{4}\int_0^1 \frac{\ln t}{1+t}\left(\int_0^{t^2} \frac{\ln u}{1+u}\,du\right)\,dt+\frac{1}{16}\int_0^1 \frac{\ln t}{1-t}\left(\int_0^t \frac{\ln u}{1+u}\,du\right)\,dt \end{align}
$$\int_0^1 \frac{\arctan x\ln^2 x}{1+x^2}dx=\frac{1}{4} \sum_{n = 1}^\infty \frac{(-1)^n H_{2n}}{n^3} + \frac{1}{8} \sum_{n = 1}^\infty \frac{(-1)^n H_n}{n^3}$$ Both series appears here at $(650)$ and $(663)$ and most likely both were posted on MSE too.
– Zacky Dec 30 '19 at 22:03