The language is regular.
Denote
$ L^k = \{ x \ | \ \forall \alpha \in Prefix(x): |\#_0(\alpha)-\#_1(\alpha)| \leq k \} $
Your language is $L = L^1$.
We shall prove, using Myhill-Nerode theorem that each $L^1$ is regular (actually for any $k,\ L^k$ is regular).
Define equivalence classes of $R_L$ as follow:
- for $-k \leq i \leq k: \ S^i = \{ x \ | \ \#_0(\alpha)-\#_1(\alpha) = i, \forall \alpha \in Prefix(x): |\#_0(\alpha)-\#_1(\alpha)| \leq k \}$
- $S_{out} = \{ w | \exists \alpha\in Prefix(x): |\#_0(\alpha)-\#_1(\alpha)| \gt k\}$
We can see that $L^k = \bigcup_{-k \leq i \leq k}S^i $
Clearly, the equivalence classes are not empty, they don't intersect and the union is $\Sigma^*$
Assume $x,y \in S_{out}$, we show that $x R_L y$: clearly from every $z \in \Sigma^*: \ xz,yz \notin L$ (have "bad" prefixes")
Assume $x,y \in S^i$, and assume there is a $z \in \Sigma^*$ such that $xz \in L$ but $yz \notin L$. Then $yz$ has a prefix $y \beta$ such that the difference between 0's and 1's is larger than k ($\beta$ is prefix for z). By counting the number of 0's and 1's of $x \beta$ we get that it's larger then k, in contradiction that $xz \in L$
Now we show that if $x,y$ in different classes, then $(x,y) \notin R_L$.
Assume $ x \in S^i, \ y \in S_{out}$: then for $z = \epsilon: \ xz = x \in L, yz = y \notin L$. Therefore $(x,y) \notin R_L$
Assume $x \in S^i, \ y \in S^j$, and $j \neq i$. w.l.o.g, $i > j$. Examine $z = 0^{k-i+1}$. Counting 0's and 1's, we get that $xz \notin L$, and $yz \in L$. Therefore $(x,y) \notin R_L$
By that We've proved that $L = L^1$ is regular