I presume you mean the original iPhone, and the 2G is referring to the antenna capabilities, rather than the 2nd Generation iPhone (which was the 3G.... yeah, confusing)?
Either way, all iPhones have a built in battery. The wording is there to suggest that it is not user replaceable via a simple swap in/out method by removing a panel or similar user appropriate action that does not require disassembly. The iPhone 5 still uses broadly similar language "Built-in rechargeable lithium-ion battery".
You can swap it out, but it's not a simple job. Just like cars have a built in engine, you can get that sucker out, but you need the tools, parts and skills for the job. If I recall, there is a lot og glue involved in earlier iPhone models. And sticky tape. It's not just a case of unscrewing some stuff and swapping. But even then, the glue can be removed, the tape replaced etc etc. The phrasing user-replaceable or more commonly user-serviceable often means it's a supported solution, and there is no way that taking any iPhone to bits is covered under warranty.
So, in short, built in means it's not meant to be removed, but that doesn't mean you cannot do it if you tried hard enough.