Here is my two cents on the topic:
There are three distinct types of "known" functions. I will evaluate each case separately, and show how we build up our "set of functions"
1. Elementary Functions
The elementary functions are given a special category because they share special properties (*cough* mumbles something about differential field *cough*) Fundamentally, an elementary functions is anything involving addition, multiplication, and exponentiation, along with the inverses (when working in $\mathbb{C}$). We can (finitely) compose these operations to get new elementary functions - for example, $\sin(2x) - 5$ can be broken up into
- $\sin(x)$, which is defined as the imaginary part of a complex exponential involving multiplication
- $2x$, which is a mulitiplication composed into $\sin(x)$
- $-5$, the addition of the additive inverse of $5$ the expression above.
Once we start messing with compositions or operators outside of this list we encounter issues. A simple is example is $\int \frac{e^x}{x}\mathrm{d}x$, which is not in our list. Finding a way to include this into our list of functions, we get:
2. Special Functions
The special functions are a way to expand our set of elementary functions to include other commonly needed functions. One simple example is the integration above; we often term this function $\operatorname{Ei}(x)$, known as the Exponential Integral. We thus clearly see that integration and differentiation operators are not in our set of elementary operations; for most intents and purposes we really want these operations, and so extending our set of function seems natural.
- Addendum: We allow the functions created by the integration to be "Special Functions", but not the integration or differentiation operators themselves. See this page for more details
Another common example is the Gamma Function. We all know that the factorial function $n!$ is elementary; it is defined purely by multiplication (or division if you want to easily justify $0!$). Special Functions are thus inherently tied to the desire for "Analytic Continuation", i.e. extending the domains of the functions inside of our set.
It now seems like we can just keep expanding our set of functions to be larger and larger, and this is true; however, for there are a number of reasons for having finitely many special functions.
- One largely philosophical reason is simply that humans live for finite time and can accomplish only finitely many tasks, while there are infinitely many functions.
- A more grounded reason is that many common tasks involve the same functions, and so we study these functions more in depth and know a lot of properties about them. This allows researchers to share knowledge of the functions more easily; for example, it is a lot harder to share information on the set of symbols "$\int \frac{e^x}{x} \mathrm{d}x$" than to share information on "$\operatorname{Ei(x)}$" or "The Exponential Integral". If you don't believe me, you haven't ever tried to find information online, e.g. at Wolfram MathWorld
Nevertheless, there are simply going to be problems we desire to solve that involve functions we haven't included yet in our set. One concrete example is the integral of $x^x$ - What do we do with this? We can expand our list of functions to include (some):
3. Other Functions
What falls into this list is really subjective. In my book though, I generally include (generalized) power series, differintegrals, and so forth.
Also included in this category is what you need in the moment. For example, at one point in my research I needed a way to compactly express the integral of $x^x$. After a little research I found this paper that had a bit on the subject, creating a function $\operatorname{Sphd}(x)$ to generalize the notion of the Sophomore's Dream. This helped me a bit, but what I actually did was just invent "Brevan's Function", which I defined to be $\operatorname{B}(x)=\int x^x \mathrm{d}x$.
Now comes the important distinction here: If I study $\operatorname{B}(x)$ enough it could become a special function. Creating a new special function generally entails having discovered and published a lot of details on the function, as well as having other people use the function.
Note: I feel that this is currently a fairly week analysis, especially the last section. I will update, expand, and correct this post when I have time.