I have taken a couple of math courses other than those required for my physics degree. Also, with leftover 1.5 years at college, I can definitely pursue a minor in math but I don't think I can double major in math.
I am currently working in an applied math lab, and I can safely say that the math courses that only serve to rigorize what I already learned application-based like abstract linear algebra and analysis wouldn't be of help anyways, so I do not really feel behind in that aspect.
There is no generic answer. It depends very much on the programme and university: check their admission criteria and maybe contact responsible people attaching your documents.
For example in Europe we use the ECTS system where you need a certain amount of ECTS points from your bachelor in a specific area (e.g. at least 30 ECTS points in mathematics and 20 ECTS in programming) to be able to continue with the specific master. It will tell you on the webpage of the degree how many ECTS you need from which field. Compare this with your bachelor curriculum (if you are short by just a few points they might still allow you to start the master but you might have to take additional courses during the frist term).
