I vaguely recall a teacher telling me that he dislikes introducing the imaginary unit $i$ as "the square root of $-1$", but I can't remember why. Is there a lack of rigour in the statement, or is it a misleading statement in any other way?
I suppose it was not something like "the square root is only defined for non-negative numbers".
There are two square roots of $-1$. Using the definite article "the" implies uniqueness.
See a related question, which might actually be a duplicate...
The answers above are fine for someone who already knows about complex numbers, but I understand the question differently.
The problem with introducing $i$ as "the square root of $-1$" (and using this to define complex numbers) is that, of course, for someone who only knows the reals, "the square root of $-1$" makes no sense. As is clear from the other answers, you have to change it to "a square root of $-1$".
But still: what is this supposed to mean? We know that there exists no real square root of $-1$, so what kind of objects (and what kind of multiplication) are we talking about? These are the questions a student should ask when confronted with such a "definition".
The answer is that we need an entirely new set, most commonly pairs of real numbers with $(a,b)\cdot(c,d)=(ac-bd,ad+bc)$ as multiplication. Only then we can define $i=(0,1)$ and discover that $i^2=-1$. The more experienced students get, the more they can safely forget about these technicalities, but they have to see them once, if only to answer the question in what way complex numbers "exist".
I remember that, when I read about complex numbers at school, I thought the whole theory was based on the assumption that there is a real square root of $-1$, and I asked myself how such a theory could be of any use relying on an obviously false assumption. The point is that we don't make any assumption at all, we just define a whole new set of numbers, and beginning with "the square root of $-1$" leads to confusion in that regard.
The problem is there are two possible values for "the square root of -1", since the equation $x^2 = -1$ has two solutions. In the real case (i.e., $x^2 = a$ for $a \ge 0$), the square root is taken to be the positive solution, but this distinction breaks down in the imaginary case, since there is no greater-than relation that is compatible with the field structure.
Both $-i$ and $i$ satisfy the equation $x^2=-1$ but you cannot decide for a positive one because you do not have this concept in $\mathbb{C}$.
The first reason that comes to mind is that if you think of $i=\sqrt{-1}$, then you may be tempted to argue that
$$\begin{align*} i^2&=\sqrt{-1}\sqrt{-1}\\ &=\sqrt{(-1)(-1)}\\ &=\sqrt{1}\\ &=1, \end{align*}$$ which is clearly false, as $i^2=-1$. I've always thought of $i$ as something you square to get $-1$, rather than thinking of it as a square root.
There is a bit more complicated, but more thorough explanation, however, involving complex analysis. The problem lies in trying to take fractional exponents of negative numbers, e.g. $(-1)^{1/2}$. In high school you are told that square roots and logarithms are not defined for negative numbers, but the truth is that they do exist, but only as complex numbers.
For a complex number $z$ (which would include all negative real numbers) we define the natural logarithm of $z$ as $$\mathcal L_\tau(z)=\ln|z|+i\;\arg_\tau(z),$$ where $\ln{|z|}$ is the real natural logarithm of the modulus of $z$ (which is always positive and real), and $\arg_\tau(z)$ is the argument of the complex number $z$ (the angle it makes with the positive real axis in the complex plane) with values in the range $[\tau,\tau+2\pi)$. We then have that $$\begin{align*} \exp{(\mathcal L_\tau(z))}&=\exp(\ln|z|+i\;\arg_\tau(z))\\ &=e^{\ln|z|}e^{i\;\arg_\tau(z)}\\ &=|z|(\cos(\arg_\tau(z))+i\;\sin(\arg_\tau(z)))\\ &=r(\cos(\theta)+i\;\sin(\theta))\\ &=z, \end{align*}$$ regardless of our choice of of $\tau$.
Therefore, when can use this definition to define $z^{\alpha}$, where $\alpha$ is any complex number. We simply define it as follows: $$z^{\alpha}:=\exp(\alpha \mathcal L_\tau(z)),$$ for some choice of $\tau$. This makes sense since $\mathcal L_\tau(z^n)=n\mathcal L_\tau(z)$, for $n$ an integer, and since $\exp$ is the inverse of $\mathcal L_\tau$.
However, since the choice of $\tau$ is not unique, then we can have different values of $z^{\alpha}$ for different choices of $\tau$. This makes complex (and fractional) powers of complex (and negative) numbers trickier than we'd like.
