This isn't a question with a correct answer. I think most people who've been around math departments for a while have seen students go way too far to one side of this or another: students who think they know the first week of their Ph.D. the only field of math in which they want to work, and students who seem to constantly be learning new things, but never seem to settle down and concentrate on something. Lots of different strategies can work, and there's a lot of luck in how these things work out.
If you plan to do a Ph.D. in the US, then I would think about things this way: while you're doing your masters, you should be figuring out where you want to do your Ph.D. and preparing yourself for that. If you plan on going to another school, there's not too much point in specializing before you get there, since don't know what advisors will be available there. Taking some advanced courses, and choosing a brand area is fine, but you can feel comfortable trying to get a broad base.
You can reasonably expect to have 5 years to finish your Ph.D. after you arrive at the school where you will do so (so after you finish your master's degree, if you plan on moving). If this is the case, then, I would worry if:
- at the end of your first year, you don't know what broad area of math you'd like to work in (number theory, PDE's, etc.).
- at the end of your second year, you don't know who your advisor will be.
- at the end of your third year, you don't know what the underlying objects in your thesis will be and how to work with them
- at the end of your fourth year, you don't know the statement of the main theorem of your thesis
You shouldn't take this as gospel (at some schools expect students to finish in 6 year, so you can sneak an extra year in there somewhere), and of course, being worried doesn't mean that things can't be fixed, but what you can relax if you haven't done yet.