Anything you can do with multiple-valued "functions", you can do with single-valued functions. This is yet another instance where maths and computer coding really help each other out in terms of understanding the other - it's basically a question of "implementing" the desired functionality using a simpler set of basic tools, like in coding, implementing a binary tree using an array with suitable addressing. And one thing mathematicians like to do is to try and keep the set of basic tools for "implementational" use as simple as possible, hence single-valued functions would be preferred to multi-valued functions as the more basic concept.
One way to implement multi-valued functions using single-valued ones is to make the codomain of what would be the multi-valued function something other than a number, such that it packages the multiple values together. A common way to do this is to use sets: a "multi-valued function" with domain $D$ and codomain $C$ becomes instead a single-valued function with domain $D$ and codomain $2^C$, so that the multiple output values become bundled together into a single set value. When this definition is in play, the statement $\sqrt{4} = \{ +2, -2 \}$ is true.
Another way to do it, is to keep the codomain as-is, change what the domain of the multi-valued function is. The logic here is that multi-valued functions like $\sqrt{\cdot}$ can be understood as arising from the fact that they are trying to invert "lossy" operations, that is to say, operations (here, squaring) that lose information. Because the information is missing, we cannot unambiguously select a single value with which to define a function. The "codomain" approach, then, basically consists of asking the user of the multi-valued function for more - that is to say, to supply that missing information. A simple way for the square root would be to include along with the number we want to take that square root of, a second variable indicating the sign of the number we presume was squared to create it. Note this is not the same thing as a square root operation taking a signed number as input - rather, it must take a sign and a number separately. There is no standard notation for this - we might be tempted to, say, write $\sqrt{(+, 4)}$, which then is the same thing as "$2$", and $\sqrt{(-, 4)}$, which then is the same as $-2$.
Fortuitously, however, it turns out the notations $\pm \sqrt{n}$ also essentially have an identical effect; and this is likely a reason why we just say "make $\sqrt{n}$ the positive root of $n$ and get it over with". But that isn't the case in more complex cases, e.g. the inverse trigonometric functions, or the complex logarithmic function. In those cases, one may imagine the second argument to be, say, a range in which we are presuming the output angle came from. We might, say, define $\mathrm{arcsin}(a, b; x)$ - again, not a "standard" notation, but one we are introducing to demonstrate the principle - to be the value of the inverse sine of $x$ under the assumption the angle came from the interval $[a, b)$, where we require that $(a, b)$ are subject to the constraints (i.e. limiting the scope of our extended domain), to only those pairs of the form $\left(\left(n - \frac{1}{2}\right) \frac{\tau}{2}, \left(n + \frac{1}{2}\right) \frac{\tau}{2}\right)$, for integers $n$. I.e. $\mathrm{arcsin}(0, 0.3; 1)$, is a domain error.
Of course, there is one more caveat here, and that's that in either case, we often deal with expressions like $2 + \sqrt{4}$ and the like, yet we have a "data type incompatibility" here: if we defined $\sqrt{\cdot}$ in terms of taking a pair with a sign, it is not allowed to take a "$2$" as input, and if we defined it take numbers but produces a set as output, now "$+$" chokes, because it "expected" to take two real numbers, not a real number and a set. Hence, we have to either make some definitions for how we are to, say, promote "$4$" to a pair like $(+, 4)$ - which is, in effect then doing the same thing as simply arbitrarily choosing a single-valued branch, or we have to define how that $+$ is supposed to operate when one of its arguments is not another real number, but a set of real numbers. The latter isn't unheard of at all - typically, the way we do it is just elementwise:
$$a + S := \{ a + s : s \in S \}$$
for adding the real number $a$ and set $S$. Then $2 + \sqrt{4} = \{0, 4\}$ - but the point is, you still have to make this definition and be clear you are doing that, or else, you might run into some form of incorrect mathematical reasoning.
Regardless of the method chosen, however, the point stands: anything you can do with multiple-valued functions you can do with single-valued functions, so that multiple-valued functions provide no greater expressive power to your language. (They also provide no less, since a single-valued function is in effect a degenerate case of a multiple-valued function.) Thus, as a matter of mathematical economy, it's useful to simply not use them, and instead employ single-valued functions more artfully ...
... except that, unfortunately, unlike in computer coding where the compiler ensures our code must conform exactly, maths in practice kind of blurs some of the idealism of the unambiguous formal language it tries to represent into messy notation and usage that is formally nonsense, and thus you will see things where a "multi-valued function symbol" gets tossed around carelessly as though it were a single-valued function. Which can lead to trouble unless you're actually aware of what is going on, which a skilled mathematician will be, but which will cause a learner many hours of frustration with "well what's wrong with it?!" in things like
$$1 = \sqrt{1} = \sqrt{(-1)(-1)} = \sqrt{-1} \sqrt{-1} = i \cdot i = -1.$$