The answer by nullUser is good, but uses some sophisticated ideas like an understanding of the relationship between Lebesgue integrability and Riemann integrability. I thought it might be nice to give a direct, straightforward answer as well.
We will show that $f$ is integrable on $[a,b]$ by constructing partitions for the definition of Riemann integration.
So let $\epsilon > 0$ be an allowed error. Denote by $M$ the maximum value of $|f|$ on the interval $[a,b]$.
(In your post, you imply that $f$ needs only an upper bound, but this is insufficient. Consider $-1/x$ on the interval $[0,1]$, defined to be $0$ at $0$, for an instance of a function integrable on any $[\delta, 1]$ for $\delta > 0$, but not integrable on $[0,1]$.)
Now choose $\delta > 0$ such that $M\delta < \frac{\epsilon}{4}$. Fix a partition $P'$ of $[a + \delta, b]$ such that the difference between the upper Riemann bound and lower Riemann bound of $f$ on $P'$ is at most $\frac{\epsilon}{2}$, and which we know exists as $f$ is Riemann integrable.
We construct a partition $P$ for the interval $[a,b]$ from the partition $P'$ for $[a + \delta, b]$ by adding in the single point $a$. We see that the difference between the upper bound Riemann bound and lower Riemann bound for $P$ is at most the greatest difference on $[a, a + \delta]$ added to the greatest difference on $[a + \delta, b]$, which is
$$ 2 \frac{\epsilon}{4} + \frac{\epsilon}{2} = \epsilon.$$
So for any $\epsilon$< we have explicitly constructed a partition satisfying the requirements for Riemann integrability.
For the second part: now that we know that $f$ is integrable on $[a,b]$, we know that
$$ \int_a^b f = \int_a^c f + \int_c^b f.$$
Since
$$ \left \lvert \int_a^c f(x)dx \right \rvert \leq M \lvert c-a\rvert \to 0$$
as $c \to a$, we have that $\int_c^b f \to \int_a^b f$. $\spadesuit$