The question itself is a standard theorem to be found in any elementary textbook. Even calculus books will attempt some kind of "proof." Here is one I will steal from an elementary real analysis text (see reference [1] below if you need the full treatment of this theory).
Theorem 8.26: (Differentiation of the Indefinite Integral) If $f$ is Riemann integrable on $[a, b]$ then the function
$$F(x) = \int_a^x f(t) \,dt$$
is continuous on $[a, b]$ and $F′(x) = f(x)$ at each point $x$ at which the function $f$ is continuous.
[Notes (i) there might be points where $f$ is discontinuous and yet $F'(x)=f(x)$. This theorem guarantees that at points of continuity this must be true. (ii) Note that the theorem calls $F$ an "indefinite integral." This is different from the other meaning that one learns for the notation $\int f(x)\,dx$. Make sure you understand the difference.]
The standard proof [see @copper.hat comment] uses these computations, easy to see when $h>0$, but true otherwise:
$$\left| \frac{1}{h} [F(x+h)-F(x)] - f(x)\right| =\left| \frac{1}{h}\left\{ \int_x^{x+h}f(t)\,dt - hf(x) \right\}\right|$$
$$
= \left| \frac{1}{h}\left\{ \int_x^{x+h}f(t)\,dt - f(x)\int_x^{x+h}\,dt \right\}\right|$$
$$
= \left| \frac{1}{h}\left\{ \int_x^{x+h}[f(t)-f(x)]\,dt \right\}\right|$$
$$
\leq \frac{1}{h} \int_x^{x+h} \left| f(t)-f(x)\right| \,dt .$$
There is a lot going on here for a beginning student of integration theory. So it is easy to foget when you know this stuff just how detailed and intimidating these things can be. With this inequality and assuming that $f$ is continuous at $x$ then take $\epsilon>0$, choose $\delta>0$
so that $|f(x)-f(t)|<\epsilon$ if $|x-t|<\delta$. So if $0<h<\delta$ deduce
that
$$\left| \frac{1}{h} [F(x+h)-F(x)] - f(x)\right|
\leq \frac{1}{h} \int_x^{x+h} \left| f(t)-f(x)\right| \,dt < \frac{1}{h} \int_x^{x+h} \epsilon \,dt =\epsilon.$$
Etc. [I mean "etc. if you remember what $F'(x)$ is defined to be."]
But that is not what the OP really wants. So let us address that.
The OP wanted a justification for swapping limits in his suggested proof. This is a big, important topic. When can we justify the interchange of limit operations? Maybe one of the most important issues in analysis.
For a double sequence $\{a_{mn}\}$ is it true that $$ \lim_{n\to\infty}
\lim_{m\to\infty} a_{mn} = \lim_{m\to\infty} \lim_{n\to\infty} a_{mn}\tag{*}$$ if
the separate limits exist?
The answer is "of course not!" There are lots of simple examples when this fails. The most frequent suggestion to get around this is to assume that there is some kind of uniformity needed in these limits. That is not always necessary, but it should be the first thought that occurs to you.
So shall we see if the idea the OP has is workable?
Take a decreasing sequence $h_m\to 0$. On the interval $[0,h_m]$ use the fact that
$$\int_x^{x+h_m}f(t)\,dt = \lim_{n \to \infty} \sum_{i=1}^n f\left(x + \frac{ih_m}{n} \right)\frac{h_m}{n} $$
Write $$a_{mn} = \sum_{i=1}^n f\left(x + \frac{ih_m}{n} \right)\frac{1}{n} =
\frac{f(x) + f(x+ \frac{ h_m}{n}) + f(x+\frac{ 2h_m}{n})+ + \dots + f(x+h_m)}{n}.$$
Thus it must be true, since $h_m\to 0$ and $f$ is continuous at $x$, that
$$ \lim_{m \to \infty}a_{mn} = f(x).$$
Remember that we also have
$$ \lim_{n \to \infty}a_{mn} = \frac{1}{h_m}\int_x^{x+h_m}f(t)\,dt.$$
CAN WE DO THE BIG SWAP NOW IN (*)? If so then
$$ \frac{1}{h_m}\int_x^{x+h_m}f(t)\,dt \to f(x)$$
as $m\to\infty$. As this is available for any such sequence we have our proof. Well ...if the swap is available?
Is it? No spoilers here. Sorry. This is a research question for you. It is far more important you pursue this than get a smug answer here. Check the StackExchange post in reference [2].
REFERENCES:
[1] https://classicalrealanalysis.info/documents/TBB-AllChapters-Landscape.pdf
[2] When can you switch the order of limits?
