I know this question is a cliché but this is something I just cannot understand. This is my context: We define logic so we can define a formal language and then set theory, but to define logic we need set theory. So, which came first, the chicken or the egg?
This is my attempt to understand this dilemma.
We take a system of axioms (must probably ZF) that we might call meta-set theory and using the basic rules of logic and meta-mathematics (natural numbers, recursion, etc.) we develop a formal language and then, under such rules of language we construct the theory of logic, then we define set theory formally (again ZF but formally) and finally all the mathematical structures. (In this case I'd say ZF is both a system of axioms(a meta-theory) and also a formal theory).
This would be something tricky that avoids answering the question because we take as given both meta-logic and meta-set theory at the same time (or at least the question of asking which was first loses all sense and interest).