Since you're deliberately ignoring the conditions for exchanging the order of integration, you can't hope for anything that actually works here, of course.
As you have discovered, evaluating the inner integral symbolically leads to nonsense -- indeed, the improper integral never converges. So you'll have to lower your standards of rigor considerably in order to extract any intuition from this rewriting. One way which you may or may not find illuminating is this:
In the case $x=z$ the integral is just $\int_{-\infty}^\infty 1dy$ which clearly diverges to infinity in a nice restrained way.
When $x\ne z$ the integrand $e^{i(x-z)y}$ keeps circling around the origin at constant speed, and half of each round will cancel out the other half. So if one is to pretend that the integral has any value at all, it will have to be $0$ -- another way to argue this that the integrand becomes minus itself by substituting $y\mapsto y+\frac{\pi}{x-z}$, but such a translation should not change the result when we integrate over all of $\mathbb R$. Thus the integral ought to be minus itself, or in other words 0.
So the outer integral has an integrand that is zero everywhere except at $z=x$ in which case it is infinity times $f(x)$. It makes some intuitive sense that this "ought to" integrate to something proportional to $f(x)$, but the particular factor $2\pi$ needs a more careful analysis (that doesn't swap integration orders willy-nilly) to derive.
By the way, essentially these considerations, dating straight back to Fourier himself, originally led to the classical integral form for the Dirac delta function:
$$ \delta(x) = \frac{1}{2\pi}\int_{-\infty}^{\infty} e^{itx}\, dt $$
Finding a way in which this would make rigorous sense was part of the impetus behind the theory of distributions.