Too long for a comment. This is only a partial answer, but I believe it may help in some manner.
As pointed out by @Po1ynomial, the integral is equivalent to
$$ I = \int_0^{\frac{1}{\sqrt{3}}} \arctan \left(x \frac{\sqrt{x^2+5}}{\sqrt{1-3x^2}}\right) \frac{dx}{x^2+1}$$
Then we let
$$ I(a) = \int_0^{\frac{1}{\sqrt{3}}} \arctan \left(ax \frac{\sqrt{x^2+5}}{\sqrt{1-3x^2}}\right) \frac{dx}{x^2+1}$$
Differentiating with respect to $a$, we get:
$$I'(a) = \int_0^{\frac{1}{\sqrt{3}}} \frac{x\sqrt{x^2+5}\sqrt{1-3x^2}}{(1 + (5a^2-3)x^2 + a^2x^4)(x^2+1)} dx$$
Let
$$ \sqrt{1-3x^2} = u$$
Then
$$ x = \frac{1}{\sqrt{3}}\cdot \sqrt{1-u^2}$$
$$dx = \frac{-1}{\sqrt{3}}\cdot \frac{u}{\sqrt{1-u^2}} du$$
The integral becomes
$$I'(a) = \int_0^1 \frac{\frac{1}{\sqrt{3}}\cdot \sqrt{1-u^2} \cdot \frac{\sqrt{16-u^2}}{\sqrt{3}} \cdot \frac{1}{\sqrt{3}}\cdot \frac{u}{\sqrt{1-u^2}}}{\left(1 + (5a^2-3)\cdot \frac{1-u^2}{3} + a^2\cdot \frac{(1-u^2)^2}{9}\right) \cdot (\frac{4-u^2}{3})}$$
$$I'(a) = 3\sqrt3 \cdot \int_0^1 \frac{u^2 \sqrt{(16-u^2)}}{(16a^2-(17a^2-9)u^2 + a^2u^4)(4-u^2)} du$$
Now substitute $u = 4\mathrm{sin}t$. The integral then becomes
$$I'(a) = 3\sqrt3 \cdot \int_0^{\alpha} \frac{ 16\sin^2t \cdot 16\cos^2t}{(16a^2-(17a^2-9)\cdot 16\sin^2t + a^2 \cdot 256\sin^4t)(4-16\sin^2t)} dt$$
Where $\sin{\alpha} = \frac{1}{4}$
Dividing by $\sec^6t$ throughout and substituting $\operatorname{tan}t = v$, we get:
$$I'(a) = \frac{3\sqrt3}{4} \cdot \int_0^{\tan{\alpha}} \frac{v^2}{[16a^2(v^2+1)^2 - (17a^2-9)\cdot v^2(v^2+1) + 16a^2 \cdot v^4](1-3v^2)}dv$$
$$ = \frac{3\sqrt3}{4}\cdot \int_0^{\frac{1}{\sqrt{15}}} \frac{v^2}{\left[(15a^2+9)v^4 + (15a^2+9)v^2 + 16a^2)\right]\cdot (1-3v^2)}dv$$
I am stuck here. Maybe someone else can find this useful.