Consider the standard calculus definition of continuity: The function $f$ is continuous at $x$ iff
$$f(x+h) \xrightarrow[h\to0\,]{} f(x)\,.\vphantom{\int}$$
Now apply this to our function $g$ and see what happens. The Riemann integrable $f$ should be defined on some closed interval. Let $x$ be an inner point of this interval and consider $g(x+h)$ for some small $h$. (If $x$ were a boundary point we should only check right or left continuity.) If we want to be careful we can split $g(x+h)$ into two parts:
$$g(x+h) = \int_0^{x+h}f(t)dt = \int_0^{x}f(t)dt + \int_x^{x+h}f(t)dt \,.$$
If $h$ is negative the last term is defined as $\left(-\int_{x+h}^x f(t)dt\right)$.
Now,
$$g(x+h) = g(x) + \int_x^{x+h}f(t)dt$$
and the only thing we need, to get what we want, is
$$\int_x^{x+h}f(t)dt\ \xrightarrow[h\to 0\,\,]{} \int_x^{x}f(t)dt\ = 0 \,,$$
which looks very plausible. To be formal, we can choose $a<x<b$ such that $f$ is defined on $[a,b]$ and $M>0$ such that $M\geq |f(t)| \;\forall t\in [a,b].$ Then
$$\left|\int_x^{h}f(t)dt\right|\leq \int_x^h |f(t)|dt \leq M|h| \xrightarrow[h\to 0\,]{} 0\,.$$
Thus
$$g(x+h) = g(x) + \int_x^{x+h}f(t)dt \xrightarrow[h\to 0\,]{} g(x)\,,$$
which means that $g(x)$ is continuous at $x$.