Your last question/surmise hits the real point: it is possible to contrive functions in $L^1\cap L^2$ whose Fourier transforms are not in $L^1\cap L^2$. And so on... Nevertheless, indeed, it is elementary-traditional to discuss Fourier transforms for such functions. A moderately clever person can find many such functions, so that one would feel vaguely confident that this is ok... But, no, this is making it more complicated than it need be, for non-elementary reasons.
Although questions such as your first two may indeed be critical in some situations, they are misleading in subtle and insidious ways, as the $L^2\cap L^1$ issue highlights. Thus, despite many decades of progress on Fourier transforms, Laurent Schwartz' late-1940s creation of the theory of distributions, and of tempered distributions, included many ideas which (in my opinion) lent sanity and simplicity to these questions. (Without slandering/libeling any deceased very-good mathematicians by naming them, I must note that at the time some people thought it was outrageous that Schwartz won a Fields Medal for "distributions", because, in effect, "the experts" could already do all those things. My observation is that these mechanisms were potentially simple enough that "any idiot could do it", and Schwartz found/manifested that simplicity.)
Specifically, Schwartz observed that the (Frechet) space of infinitely-differentiable functions all of whose derivatives decay faster than any power of $1/(1+x^2)$ _maps_to_itself_ under Fourier transform.
The major "objection" was/is that the Schwartz space is not Hilbert, nor Banach. Well, ok, but it is an excellent answer to an important question, so to the extent that these "simpler" notions about function spaces are insufficient, this story is evidence that we should enlarge our repertoire of function-spaces right away!
The non-triviality of finding this sort of balance is meaningfully illustrated in many other "eigenfunction expansion" scenarios, wherein we want to express "general" functions as "superpositions" (sums and integrals...) of special functions, maybe eigenfunctions ($L^2$?) for some naturally-occurring operator.
Returning to the original question: Fourier inversion is literally an integral only when both $f$ and $\hat{f}$ are in $L^2\cap L^1$, as you say... but, even then, there is a pitfall: in what sense do we mean the equality of functions, the original, and the integral of exponentials??? This is a very serious question. The convergence need not be pointwise at all, certainly not uniformly pointwise. Convergence in an $L^2$ sense is assured (Plancherel), but couldn't make sense unless both $f$ and $\hat{f}$ were in $L^2$. :)
So, if we are trying to be clever, we can define $\hat{f}$ by the Plancherel-isometric extension of the integral giving the "forward" Fourier transform, and only worry about the integral for inversion, soooo with $f\in L^2$ and $\hat{f}\in L^2\cap L^1$, the integral form of Fourier inversion is literally correct. :)
But, seriously, this kind of analysis is a bit frivolous, all the more so since (I think) understanding what's going on is only possible after taking up a more sophisticated viewpoint (Schwartz...). But I hope the discussion does respond helpfully.