What Martin said is all good. The exponentials (and logs) are deeply embedded in calculus.
I suspect the expectation of elementary function solutions to integrals comes from the fact that it seems easy to do derivatives and get elementary functions. The easy part is largely because your textbook or prof pick problems which are suitable for students; they are not all so easy.
Regardless, remember that inverse functions are never as easy to calculate as the original function (the integral being the inverse of derivative). arcsin is harder calculate than sin; logx is harder to calculate than e$^x$; even division is harder than multiplication and was an intractable problem until the use of the place decimal system. Just try dividing in Roman numerals, you will see. People had tables of division, just as we now have tables of logs (or computerized versions thereof).
With all that said, the elementary functions are indeed those of convenience. There are many, many other functions in constant use out there: gamma functions, zeta functions, Bessel functions, and thousands more special functions which have had reason to be named. (And what is "log" anyway except a name?)
Some of the integrals you can't "solve" can be expressed in terms of one or more of these special functions, and some just cannot (unless we invent a new special function and give it a name -- although that is kind of running around in circles).
For practical purposes it can all be dealt with. If you have a definite integral it can be calculated to any desired accuracy with numeric methods. If in the middle of a proof you have denoted something as a indefinite integral, you can always replace the integrand with some kind of approximation that does have a nice solution, or known behavior.
Many students ask your question. I guess in general one shouldn't hope anything will be easy, mathematically or otherwise. Then if it is you get a nice surprise.