Doesn't this make the anti-derivative of $\frac 1 z$ definable? Although this antiderivative will be discontinuous, but still valid. So, why is this not the case?
An antiderivative of a function $f$ on a domain $G$ is a function $F$ on $G$ such that $F'=f$ on $G$, and in particular $F$ must be differentiable at each point in $G$. If a function is differentiable at a point, it is also continuous at that point. So your candidate for an antiderivative is not one, being nondifferentiable at each point of discontinuity.
Other answers have already pointed out a slicker way to see that such an antiderivative is impossible, but one could also use your approach of working with a specific logarithm to see it is impossible to have a global antiderivative. Suppose there is one, and that $F'(z) = \dfrac1z$ on $\mathbb C\setminus\{0\}$. Let $\operatorname{Log}$ be defined on $\mathbb C\setminus\{0\}$ by $\operatorname{Log}(z)=\log|z|+i\theta$, with $z=|z|e^{i\theta}$ and $-\pi<\theta\leq \pi$. Then on $\mathbb C\setminus(-\infty,0]$, $(F-\operatorname{Log})'=0$, from which it follows that $F=\operatorname{Log}+C$ for some constant $C$. By continuity of $F$ (following from differentiability), $$F(-1)=\lim\limits_{y\searrow 0}F(-1+yi)=\operatorname{Log(-1)}+C = \pi i+C,$$ and $$F(-1)=\lim\limits_{y\searrow 0}F(-1-yi)=\lim\limits_{y\searrow 0}\operatorname{Log}(-1-yi)+C=-\pi i+C.$$ This contradiction shows such an $F$ can't exist.
(I am taking for granted that $\operatorname{Log}'(z)=\frac1z$ on $\mathbb C\setminus(-\infty,0]$, which follows from the appropriate version of the inverse function theorem. I am also taking for granted that $g'=0$ on a connected open set in $\mathbb C$ implies $g$ is constant.)
