Performing the change of variables $2u = x^2$ together with two integration by parts, we get, $$ \int_0^\infty \cos(x^2)dx = \frac{1}{\sqrt{2}}\int^\infty_0\frac{\cos(2x)}{\sqrt{x}}\,dx\\=\frac{1}{2\sqrt{2}}\underbrace{\left[\frac{\sin 2 x}{x^{1/2}}\right]_0^\infty}_{=0} +\frac{1}{4\sqrt{2}} \int^\infty_0\frac{\sin 2 x}{x^{3/2}}\,dx\\=
\frac{1}{4\sqrt{2}}\underbrace{\left[\frac{\sin^2 x}{x^{1/2}}\right]_0^\infty}_{=0} +\frac{3}{8\sqrt{2}} \int^\infty_0\frac{\sin^2 x}{x^{5/2}}\,dx$$
Hence $$\int^\infty_0\frac{\sin^2 x}{x^{5/2}}\,dx = \frac{8\sqrt2}{3}\int_0^\infty \cos(x^2)dx = \frac{4\sqrt \pi}{3}$$
Since See Here, $$\int_0^\infty \cos(x^2)dx= \sqrt\frac\pi8$$
or
How to prove only by Transformation that: $ \int_0^\infty \cos(x^2) dx = \int_0^\infty \sin(x^2) dx $