This is precisely the definition of the error function. The integral is one of the countless integrals that simply isn't expressible as a finite expression involving radicals, exp and trig functions. Most of the integrals are that way (only a very small number of integrals can be written in terms of elementary functions). However, this particular integral (integral of the Gauss distribution) is so commonly used (statistics, physics, who knows what else), that someone named it the error function and put it in tables and computer software.
If you think about it, every function has to be defined and calculated in some way. Computing ${\rm erf}(x)$ numerically in the background when you push a button is not much different than what happens when you push a button for $\sin (x)$ or any other transcendental function. It's just our choice what functions are important (although one could argue that exp+trig are elementary because of their complex analytical properties).
Your question somehow mentions the Taylor expansion. There's nothing special about it and has no connection with the definition of ${\rm erf}(x)$.