Here's something I wrote about this viewpoint earlier:
A better answer to your misgivings (in the sense of being closer to the mainstream presentation) is probably simply to jump in with both feet and declare that a line is not really made of points.
Decide to think of a line as something that is in principle a different kind of thing from a bunch of points glued together. You can do this and still acknowledge that points exist and some of them are on the line while others are not.
It then turns out that all of the properties of a line segment (or a smooth curve in general) can be recovered from knowing which points lie on it and which don't. This doesn't necessarily mean that the points make up the line, but merely that the points tell us enough about the line.
It is technically convenient, then, to speak about the set of points on the line as a placeholder for the line itself, when we're formalizing our reasoning -- for the pragmatic reason that we have a well-developed common machinery and notation for speaking of sets of things, which means that we don't need to introduce a new formalism for an entirely different kind of things.
Some people are so comfortable with this representation that they happily declare that the line IS its set of points -- but nobody says you have to think of it that way. As long as you agree that the set of points determines the line, you can still communicate with people who prefer the other idea.
Why would one want to do that, you ask?
One reason is if it happens to be more comfortable philosophically for you to think that way. It's up to you whether it is or not, of course -- but, for example, the asker I wrote the above text for had gotten himself into conceptual trouble trying to imagine how points can have zero length and width, and yet combine to form a line of nonzero length.
Another reason is that you can avoid talking about sets at all for some purposes. Mathematicians use sets all the time, of course, but set theory does come at a foundational cost. For example, you can't have a formal theory that admits sets in a useful way without Gödel's incompleteness theorem applying, so the theory would necessarily be incomplete.
In particular, plain old Euclidean geometry can be formalized without having any concept of a set of points. That's what Euclid was doing, of course, though he missed a number of "intuitive" continuity properties. Hilbert tried to repair that, now explicitly considering "line" to be a primitive concept, but he failed to make it completely formal because formal logic was still being invented. (Later Tarski constructed an actual complete first-order theory of geometry, but he did it by omitting any concept of "line" at all, speaking instead of "collinear" as a relation between three points).