I am posting an answer on the basis of hints given in comments, hope it helps if someone has the same question. Please tell if this is correct.
\begin{align*}
\lim\limits_{x \to 0}\frac{1}{x} \int_{0}^{x} \sin^2\left( \frac{1}{u}\right) \mathrm du & = \lim\limits_{x \to 0}\frac{1}{x} \int_{0}^{x} \frac{\left(1-\cos\left( \frac{2}{u}\right)\right)}{2}\ \mathrm du\\
& = \lim\limits_{x \to 0}\left(\frac{1}{x}\cdot \frac{1}{2}\int_{0}^{x} du - \frac{1}{x}\cdot \frac{1}{2} \int_{0}^{x} \cos\left(\frac{2}{u} \right)\mathrm du \right)\\
& = \lim\limits_{x \to 0} \left(\frac{1}{2x}\cdot x \right) - \lim\limits_{x \to 0}\left( \frac{1}{2x} \int_{0}^{x} \cos \left( \frac{2}{u}\right)\mathrm du \right)\\
& = \frac{1}{2} - \frac{1}{2}\ \lim\limits_{x \to 0} \frac{1}{x} \int_{0}^{x} \cos \left( \frac{2}{u}\right) \mathrm du.\\
\end{align*}
{Claim} : $$ \lim\limits_{x \to 0} \frac{1}{x} \int_{0}^{x} \cos \left( \frac{2}{u} \right)\mathrm du = 0$$
{Proof} : Consider the function $F(x) = x^2\sin \left( \frac{2}{x} \right).$ Then,
$F(0) = 0$ as $\lim\limits_{x \to 0} x^2\sin \left(\frac{2}{x} \right) = 0\ (\because -1 \le \sin \left( \frac{2}{x} \right)\le 1 \Rightarrow -x^2 \le x^2 \sin \left( \frac{2}{x}\right) \le x^2.) $ Hence, we define function value to be $0$ at $x =0$ making $F(x)$ continuous at $x= 0$.
Also, we can calculate using limit definition that $F'(0) = 0 $ as follows :
\begin{align*}
F'(0) & = \lim\limits_{x \to 0} \frac{F(x) - F(0)}{x-0}\\
& = \lim\limits_{x\to 0} \frac{x^2 \sin \left( \frac{2}{x}\right)}{x}\\
& = \lim\limits_{x \to 0} x \sin \left( \frac{2}{x}\right)\\
& = 0 \ \ \ \ \ \ \ \ \ \text{(using sandwitch theorem as done above).}\\
\end{align*}
We also know that $F'(x) = 2x \sin \left( \frac{2}{x} \right)- 2 \cos \left( \frac{2}{x}\right)$ whenever $x \ne 0$. To prove our claim, assume $$G(x) = \int_{0}^{x} \cos \left( \frac{2}{u} \right) \mathrm du$$
Now, by Fundamental theorem of calculus (FTOC), $$F(x) = \int_{0}^{x} F'(u) \mathrm du = 2 \int_{0}^{x} u\sin \left(\frac{2}{u}\right) \mathrm du - 2 G(x)$$
Next, dividing by $x$ and taking $\lim\limits_{x \to 0}$ we get,
\begin{align*}
\lim\limits_{x \to 0} \frac{F(x)}{x} & = \lim\limits_{x \to 0}\frac{2}{x} \int_{0}^{x} u \sin \left(\frac{2}{u}\right) \mathrm du - \lim\limits_{x \to 0} \frac{2}{x} G(x)\\
\Rightarrow F'(0) & = \lim\limits_{x \to 0}\frac{2}{x} \int_{0}^{x} u \sin \left(\frac{2}{u}\right) \mathrm du - \lim\limits_{x \to 0} \frac{2}{x} G(x)\\
\end{align*}
We see that $$\lim\limits_{x \to 0}\frac{2}{x} \int_{0}^{x} u \sin \left(\frac{2}{u}\right) \mathrm du = 0$$
To see this, we apply similar method as done above. Let $h(x) = x\sin \left(\frac{2}{x}\right).$ Since, $\lim\limits_{x \to 0}x\sin \left(\frac{2}{x} \right) = 0$, we define function value to be $0$ at $x =0$, making it continuous there. Next, let
\begin{align*}
H(x) &= \int_{0}^{x} h(u)\mathrm du\\
\Rightarrow \lim\limits_{x \to 0}\frac{H(x)}{x} &= \lim\limits_{x \to 0} \frac{1}{x}\int_{0}^{x} u\sin\left(\frac{2}{u} \right) \mathrm du\\
\Rightarrow H'(0) &= \frac{d}{dx} \int_{0}^{x} u\sin \left(\frac{2}{u} \right) \mathrm du\\
\Rightarrow H'(0) &= h(0) \ \ \ \ \ \text{(using FTOC)}\\
\Rightarrow H'(0) &= 0\\
\end{align*}
Therefore we have that,
\begin{align*}
F'(0) &= 0- \lim\limits_{x \to 0} \frac{2}{x}G(x)\\
&\Rightarrow \lim\limits_{x \to 0} \frac{2}{x}G(x) = 0\\
&\Rightarrow \lim\limits_{x \to 0} \frac{1}{x} \int_{0}^{x} \cos \left( \frac{2}{u} \right)\mathrm du = 0.\\
\end{align*}
Hence, the claim is proved. Next, using the claim we obtain finally,
$$ \lim\limits_{x \to 0}\frac{1}{x} \int_{0}^{x} \sin^2\left( \frac{1}{u}\right) \mathrm du = \frac{1}{2}\ .$$