To use a keyhole contour $C$, consider
$$\oint_C dz \frac{\log^3{z}}{1+z^2}$$
You can show that the integrals about the circular arcs about the origin, both large and small, vanish as theire respective radii go to $\infty$ and $0$. The contour integral is therefore equal to
$$\int_0^{\infty} dx \frac{\log^3{x} - (\log{x}+i 2 \pi)^3}{1+x^2} $$
which simplifies to
$$-i 6 \pi \int_0^{\infty} dx \frac{\log^2{x}}{1+x^2} + 12 \pi^2 \int_0^{\infty} dx \frac{\log{x}}{1+x^2} + i 8 \pi^3 \int_0^{\infty} \frac{dx}{1+x^2} $$
The contour integral is equal to $i 2 \pi$ times the sum of the residues of the poles of the integrand, namely $z=e^{i \pi/2}$ and $z=e^{i 3 \pi/2}$. (Note that it is crucial that $\arg{z} \in [0,2 \pi)$.) This sum is
$$\frac{-i \pi^3/8}{2 i} + \frac{-i 27 \pi^3/8}{-2 i} = \frac{13 \pi^3}{8} $$
Multiplying by $i 2 \pi$, we have
$$i \left [- 6 \pi \int_0^{\infty} dx \frac{\log^2{x}}{1+x^2} + 8 \pi^3 \int_0^{\infty} \frac{dx}{1+x^2} \right ] + 12 \pi^2 \int_0^{\infty} dx \frac{\log{x}}{1+x^2} = i \frac{13 \pi^4}{4}$$
Equating imaginary parts and noting that the second integral in the brackets is simply $\pi/2$ (you should have permission to evaluate that without residues), we have
$$- 6 \pi \int_0^{\infty} dx \frac{\log^2{x}}{1+x^2} = \frac{13 \pi^4}{4} - 4 \pi^4 = -\frac{3 \pi^4}{4} $$
or
$$\int_0^{\infty} dx \frac{\log^2{x}}{1+x^2} = \frac{\pi^3}{8}$$