I agree with you that $4$ is the limit.
Informally, the definition of $x \to a \implies f(x) \to L$ says that for any distance we pick around $L$, we should be able to find a distance around $a$ such that for all $x$ within that distance around $a$ (i.e., for all $x$ *really close to $a$), we will have all of the $f(x)$ within the first distance of $L$ (i.e., we will have all of the $f(x)$ really close to $L$). In math, this is:
$f(x) \to L$ as $x \to a$ if for every $\epsilon > 0$, $\exists \delta > 0$ such that $|x - a| < \delta$ implies $|f(x) - f(a)|< \epsilon$. Notice that we are saying for every $\epsilon > 0$, so it should be true as $\epsilon$ gets smaller and smaller, that we will always be able to find some distance around $a$ such that all of the elements $x$ within that distance will have $f(x)$ within $\epsilon$ of $L$.
Now, the work you did was kind of working "backwards" to find the $\delta$, and this is very common in $\epsilon$-$\delta$ proofs. You said: suppose we have that $|\dfrac{x^{2} + 2x - 3}{x - 1} - 4| < \epsilon$, i.e., the distance between $f(x)$ and $4$ is less than $\epsilon$. Well, if that's true, then your next line should be true, and if that is true then the next line after that should be true, etc. You got all the way down to $|x - 1| < \epsilon$. So that means if it is true that $|x - 1|< \epsilon$ (i.e., the distance between $x$ and $1$ is less than $\epsilon$), then it's true that our function is less than $\epsilon$ away from $4$. So in your proof, the $\delta$ you found was actually $\delta = \epsilon$.