There are several answers to this.
As a first example, consider the introductin of the complex numbers. Even if you are only interested in calculating for example real (Riemann) integrals, then theorems from complex analysis like the Residue theorem can help you calculate certain real integrals which are very difficult to calculate directly (see http://en.wikipedia.org/wiki/Residue_theorem#Example).
Hence, introducing a further structure containing a lot of new elements in which we are not really interested (imaginary numbers, oh my), can suddenly lead to new insights for the old structure.
Also, we are not necessarily only interested in Riemann integrals. What one often wants is some form of an integral with good properties.
For example, one thing one wants to introduce/consider/study/use is the scale of spaces $L^p(U)$ for $U \subset \Bbb{R}^n$ open (take $n=1$ if you wish). For certain problems, these spaces are far superior to the "classical" spaces $C_b (U)$ (bounded continuous functions) or $C_b^k (U)$ and the like (one can also define the Sobolev spaces $W^{k,p}(U)$ which will give you something similar to $C_b^k$), because they are reflexive (at least for $1 < p < \infty$), which allows a certain amount of compactness arguments, for example every norm-bounded sequence has a weakly convergent subsequence. This is used often in PDE-theory or in the calculus of variations.
If you define these spaces using the Riemann-integral as
$$
L^p (U) = \{f : U \to \Bbb{R} \mid f \text{ Riemann integrable and } \int |f|^p \, dx < \infty\},
$$
the resulting space will not be complete, but if you define (essentially) the same thing using Lebesgue-measurable functions for which $\int|f|^p \, dx < \infty$, you will get a complete space and you can get your theory going.
As a further example, you might be interested in the fundamental theorem of calculus, i.e.
$$
f(b) - f(a) = \int_a^b f'(t) \, dt.
$$
Using the Riemann integral, you will only be able to show this as long as (at least) the derivative is Riemann integrable. Hence, you are somewhat artificially restricting your theorem, because you insist on using the Riemann integral. Using for example the Lebesgue-integral, one can show that it suffices if $f ' $ is Lebesgue integrable and $f'(t)$ exists at every $t$. One can even exclude a countable set of $t$, as long as $f$ is continuous, see here (A Fundamental Theorem of Calculus) if you are interested.
Also, you can get many properties of the Riemann integral through having the Lebesgue integral.
This is somewhat similar to the introduction of the real numbers. Often/sometimes, you are only interested in the fact whether a certain equation has a rational/algebraic solution.
In this case, it can be much easier to show that the equation has a real solution (because you have certain devices like the intermediate value theorem or Banach's fixed point theorem which fail on the rationals) and once you have shown that, you show that the solution is actually rational/algebraic/...
As above, you will sometimes show that a certain (partial differential) equation or a certain minimization problem has a solution in the Sobolev space $W^{k,p}$, but once you know that such a solution exists, you can afterwards show that the solution is even $C^k$, hence solving your original problem.
Also, consider the following:
You have a sequence $f_n \to f$ (pointwise) of Riemann integrable functions on $[0,1]$ with $f$ also Riemann integrable and $|f_n| \leq K$ for all $n$. Then you want to know that
$$
\int_0^1 f_n \,dx \to \int_0^1 f \,dx.
$$
You can show this easily using the theory of the Lebesgue integral (dominated convergence), but try proving it directly for the Riemann integral! (I don't know how to do that!)
You know something like: If $f : [0,1] \to \Bbb{R}$ is bounded, it is Riemann integrable iff the set of discontinuities is a Lebesgue null-set. This allows for (easily) showing things like "if $g : \Bbb{R} \to \Bbb{R}$ is continuous and $f$ Riemann integrable, then $g \circ f $ is Riemann integrable." You can certainly do this without Lebesgue theory, but it is very convenient.
