I realize this is an ancient post, but I thought it may be fun to try and evaluate this one using contours. It is not the most efficient method, but what the heck. It's about the challenge. I have not seen this one on the site done using contours, so I have been picking at it. Ron G?. where were ya', buddy :)
First, let's try splitting it up.
$$\int_{0}^{\infty}\frac{\log(x+1)}{x^{2}+1}dx=\int_{0}^{1}\frac{\log(x+1)}{x^{2}+1}dx+\int_{1}^{\infty}\frac{\log(x)}{x^{2}+1}dx+\int_{1}^{\infty}\frac{\log(1+1/x)}{x^{2}+1}dx$$
Note that the second one on the right will be used as a lemma since it evaluates to $$G, \;\ \text{the Catalan}$$.
The last integral can be shown to equal the first by a simple sub $$u=1/x$$, so that:
$$\int_{0}^{\infty}\frac{\log(x+1)}{x^{2}+1}dx=\int_{0}^{1}\frac{\log(x)}{x^{2}+1}dx+2\int_{0}^{1}\frac{\log(x+1)}{x^{2}+1}dx...........(1)$$
The idea is to evaluate $$\int_{0}^{1}\frac{\log(x+1)}{x^{2}+1}dx$$ via residues and use the catalan integral as a lemma, thus obtaining the result. Now, to show that
$$\int_{0}^{1}\frac{\log(x+1)}{x^{2}+1}dx=\frac{\pi}{8}\log(2)$$.
Use a quarter-circle in the first quadrant with a quarter circle indent around the pole at $i$. Thus avoiding the singularity at $x=-1$.
The first portion along the x-axis returns the integral in question by using $z=x$.
$1:$ $$\int_{0}^{1}\frac{\log(x+1)}{x^{2}+1}dx$$
$2$: Up along the arc, parameterize with $z=e^{it}, \;\ dz=ie^{it}dt$:
$$\int_{0}^{\frac{\pi}{2}}\frac{\log(1+e^{it})}{1+e^{2it}}ie^{it}dt$$
$3$: around the indent at $i$, parameterize with $z=i+\epsilon e^{i\phi}, \;\ dz=i\epsilon e^{i\phi}d\phi$
$$-\int_{-\frac{\pi}{2}}^{0}\frac{\log(1+i+\epsilon e^{i\phi})}{(i+\epsilon e^{i\phi})^{2}+1}i\epsilon e^{i\phi}d\phi$$
$4$: down the y-axis, parameterize with $z=iy, \;\ dz=idy$
$$-\int_{0}^{1}\frac{\log(1+iy)}{(iy)^{2}+1}idy$$
Now, some gymnastics are in order:
1 is OK the way it is. Maybe call it $I$.
2 can be whittled away at by using some identities such as:
$\log(1+e^{it})=1/2\log(2+2\cos(t))+\frac{it}{2}$ and $\frac{e^{it}}{1+e^{2it}}=1/2\sec(t)$
make these subs into 2:
$$i\int_{0}^{\infty}\frac{(1/2\log(2+2\cos(t))+\frac{it}{2})}{2\cos(t)}dt$$
$$=\frac{i}{4}\int_{0}^{\frac{\pi}{2}}\frac{\log(2+2\cos(t))}{\cos(t)}dt-1/4\int_{0}^{\frac{\pi}{2}}t\sec(t)dt$$
part 3 can be whittled at in order to get a contribution:
$$-\int_{\frac{-\pi}{2}}^{0}\frac{\log(1+i+\epsilon e^{i\phi})}{\epsilon^{2}e^{2i\phi}+2\epsilon ie^{i\phi}}i\epsilon e^{i\phi}d\phi$$
factor out $i\epsilon e^{i\phi}$, then let $\epsilon \to 0$:
$$-\int_{-\frac{\pi}{2}}^{0}\frac{\log(1+i+\epsilon e^{i\phi})}{i\epsilon e^{i\phi}(-\epsilon ie^{i\phi}+2)}i\epsilon e^{i\phi}d\phi$$
$$-\int_{-\frac{\pi}{2}}^{0}\frac{\log(1+i)}{2}d\phi=\frac{-\pi}{8}\log(2)-\frac{\pi^{2}}{16}i$$
part 4: $-ilog(1+iy)=\tan^{-1}(y)-\frac{i}{2}\log(y^{2}+1)$
$$-i/2\int_{0}^{1}\frac{\log(1+y^{2})}{1-y^{2}}dy+\int_{0}^{1}\frac{\tan^{-1}(y)}{1-y^{2}}dy$$
Now, finally, take the real parts of all of those portions.
The real parts are part 1, the real part of the integral evaluated in part 3, and the arctan integral from part 4:
since the pole was avoided, there are none inside the contour. Set all this to 0.
$$\int_{0}^{1}\frac{\log(x+1)}{x^{2}+1}dx-\int_{0}^{\frac{\pi}{2}}\frac{t}{4\cos(t)}dt-\frac{\pi}{8}\log(2)+\int_{0}^{1}\frac{\tan^{-1}(y)}{1-y^{2}}dy=0$$
The trick now is to cancel those two integrals. This will happen if they are the same. Yet, they are divergent. This may seem like one of Ron's "deus ex machina" tricks.
This can be shown by making the Weierstrass sub into $\int_{0}^{\frac{\pi}{2}}t\sec(t)dt$:
$y=\tan(t/2), \;\ \cos(t)=\frac{1-y^{2}}{1+y^{2}}, \;\ dt=\frac{2}{y^{2}+1}dy$
$$\int_{0}^{1}\frac{2\tan^{-1}(y)}{4\left(\frac{1-y^{2}}{1+y^{2}}\right)}\cdot \frac{2}{y^{2}+1}dy=\int_{0}^{1}\frac{\tan^{-1}(y)}{1-y^{2}}dy$$
Thus, they are equivalent and can be cancelled.
All that remains is the $$\int_{0}^{1}\frac{\log(x+1)}{x^{2}+1}dx-\frac{\pi}{8}\log(2)=0$$
Combining this with the Catalan/log integral in $(1)$ up top, and the other integral of the form just done, the final result is:
$$\int_{0}^{\infty}\frac{\log(x+1)}{x^{2}+1}dx=\int_{0}^{1}\frac{\log(x)}{x^{2}+1}dx+2\int_{0}^{1}\frac{\log(x+1)}{x^{2}+1}dx$$
$$\int_{0}^{\infty}\frac{\log(x+1)}{x^{2}+1}dx=G+2\left(\frac{\pi}{8}\log(2)\right)$$
$$\boxed{\frac{\pi}{4}\log(2)+G}$$