I managed to find it using the good ol' Feynman's trick but it was an extremely long process, so i'd like to know if theres more efficient/faster methods out there that you guys could provide. thanks.
Ill share my attempt, but it is ridiculously long.
My attempt.
$$I\left(a\right)=\int _0^1\frac{\ln \left(ax^2+b\right)}{x+1}\:dx$$ $$I'\left(a\right)=\int _0^1\frac{x^2}{\left(x+1\right)\left(ax^2+b\right)}\:dx=\frac{1}{a+b}\int _0^1\frac{b\left(x-1\right)}{ax^2+b}+\frac{1}{x+1}\:dx$$ $$I'\left(a\right)=\frac{1}{a+b}\left(\frac{b}{2a}\ln \left(a+b\right)-\frac{b}{2a}\ln \left(b\right)-\frac{\sqrt{b}}{\sqrt{a}}\arctan \left(\frac{\sqrt{a}}{\sqrt{b}}\right)+\ln \left(2\right)\right)$$ $$\int _b^aI'\left(a\right)\:da=\frac{b}{2}\int _b^a\frac{\ln \left(a+b\right)}{a\left(a+b\right)}\:da-\frac{b}{2}\ln \left(b\right)\int _b^a\frac{1}{a\left(a+b\right)}\:da-\sqrt{b}\underbrace{\int _b^a\frac{\arctan \left(\sqrt{\frac{a}{b}}\right)}{\sqrt{a}\left(a+b\right)}\:da}_{u=\sqrt{\frac{a}{b}}}+\ln \left(2\right)\int _b^a\frac{1}{a+b}\:da$$ Now lets calculate $I\left(b\right)$. $$I\left(b\right)=\ln \left(b\right)\int _0^1\frac{1}{x+1}\:dx+\int _0^1\frac{\ln \left(x^2+1\right)}{x+1}\:dx=\ln \left(b\right)\ln \left(2\right)+\frac{3\ln ^2\left(2\right)}{4}-\frac{\pi ^2}{48}$$ Now resuming on the original expression: $$I\left(a\right)-\ln \left(b\right)\ln \left(2\right)-\frac{3\ln ^2\left(2\right)}{4}+\frac{\pi ^2}{48}=\frac{1}{2}\underbrace{\int _b^a\frac{\ln \left(a+b\right)}{a}\:da}_{a=bt}-\frac{1}{2}\int _b^a\frac{\ln \left(a+b\right)}{a+b}\:da-\frac{\ln \left(b\right)}{2}\int _b^a\frac{1}{a}\:da+\frac{\ln \left(b\right)}{2}\int _b^a\frac{1}{a+b}\:da-2\int _1^{\sqrt{\frac{a}{b}}}\frac{\arctan \left(u\right)}{u^2+1}\:du+\ln \left(2\right)\ln \left(a+b\right)-\ln \left(2\right)\ln \left(2b\right)$$ $$I\left(a\right)=\frac{\ln \left(b\right)}{2}\int _1^{\frac{a}{b}}\frac{1}{t}\:dt+\frac{1}{2}\int _1^{\frac{a}{b}}\frac{\ln \left(t+1\right)}{t}\:dt-\frac{\ln ^2\left(a+b\right)}{4}+\frac{\ln ^2\left(2b\right)}{4}-\frac{\ln \left(b\right)\ln \left(a\right)}{2}+\frac{\ln ^2\left(b\right)}{2}+\frac{\ln \left(b\right)\ln \left(a+b\right)}{2}-\frac{\ln \left(b\right)\ln \left(2b\right)}{2}-\arctan ^2\left(\sqrt{\frac{a}{b}}\right)+\frac{\pi ^2}{16}+\ln \left(2\right)\ln \left(a+b\right)-\ln \left(2\right)\ln \left(2b\right)+\ln \left(b\right)\ln \left(2\right)+\frac{3\ln ^2\left(2\right)}{4}-\frac{\pi ^2}{48}$$ $$I\left(a\right)=\frac{1}{2}\ln \left(b\right)\ln \left(\frac{a}{b}\right)+\frac{1}{2}\underbrace{\int _0^{\frac{a}{b}}\frac{\ln \left(t+1\right)}{t}\:dt}_{t=-t}-\frac{1}{2}\int _0^1\frac{\ln \left(t+1\right)}{t}\:dt-\frac{\ln ^2\left(a+b\right)}{4}+\frac{\ln ^2\left(2\right)}{4}+\frac{\ln \left(2\right)\ln \left(b\right)}{2}+\frac{\ln ^2\left(b\right)}{4}-\frac{\ln \left(b\right)\ln \left(a\right)}{2}+\frac{\ln ^2\left(b\right)}{2}+\frac{\ln \left(b\right)\ln \left(a+b\right)}{2}-\frac{\ln \left(2\right)\ln \left(b\right)}{2}-\frac{\ln ^2\left(b\right)}{2}-\arctan ^2\left(\sqrt{\frac{a}{b}}\right)+\frac{\pi ^2}{24}+\ln \left(2\right)\ln \left(a+b\right)-\ln ^2\left(2\right)-\ln \left(2\right)\ln \left(b\right)+\ln \left(b\right)\ln \left(2\right)+\frac{3\ln ^2\left(2\right)}{4}$$ $$I\left(a\right)=\frac{\ln \left(b\right)\ln \left(a\right)}{2}-\frac{\ln ^2\left(b\right)}{2}+\frac{1}{2}\int _0^{-\frac{a}{b}}\frac{\ln \left(1-t\right)}{t}dt-\frac{\pi ^2}{24}-\frac{\ln ^2\left(a+b\right)}{4}+\frac{\ln ^2\left(b\right)}{4}-\frac{\ln \left(b\right)\ln \left(a\right)}{2}+\frac{\ln \left(b\right)\ln \left(a+b\right)}{2}-\arctan ^2\left(\sqrt{\frac{a}{b}}\right)+\frac{\pi ^2}{24}+\ln \left(2\right)\ln \left(a+b\right)$$ To solve the integral remaning we can use the following identity: $$\text{Li}_2\left(z\right)=-\int _0^z\frac{\ln \left(1-t\right)}{t}\:dt$$ After using it and simplifying a bit more we finally arrive at the solution being: $$\boxed{I\left(a\right)=-\frac{\ln ^2\left(b\right)}{4}-\frac{\text{Li}_2\left(-\frac{a}{b}\right)}{2}-\frac{\ln ^2\left(a+b\right)}{4}+\frac{\ln \left(b\right)\ln \left(a+b\right)}{2}-\arctan ^2\left(\sqrt{\frac{a}{b}}\right)+\ln \left(2\right)\ln \left(a+b\right)}$$ One of the integrals i used can be proved with this: $$\int _0^1\frac{\ln \left(x^2+1\right)}{x+1}\:dx=-\frac{\text{Li}_2\left(-1\right)}{2}-\frac{\ln ^2\left(2\right)}{4}-\arctan ^2\left(1\right)+\ln ^2\left(2\right)=\frac{\pi ^2}{24}+\frac{3\ln ^2\left(2\right)}{4}-\frac{\pi ^2}{16}=\frac{3\ln ^2\left(2\right)}{4}-\frac{\pi ^2}{48}$$