The symbol refers to one of the square roots. The fact that we call it "the" square root" instead of "the positive square root when one exists" is inaccurate laziness but coloquially understood.
So the solutionS to $x^2 = 16$ are both 4 (which is $\sqrt {16}$) AND -4 (which is $-\sqrt{16}$) and we frequently write it as $\pm\sqrt{16}$)
=====
Note: $x^2 = 16$ and $x = \sqrt{16}$ are two different statements altogether. They are related statements but still different. In getting from $x^2 = 16$ to $x = \sqrt{16}$, we could just as easily have gone to the statement $-x = \sqrt{16}$.
If we want to be pedantic about it, the proper statement should have been $\pm x = \pm \sqrt{16}$ however these four statements are redundant, so simply $x = \pm \sqrt{16}$ suffices. (Although techinally $\pm x = \sqrt{16}$ is equally valid.)