(In case someone thinks this is another question about the seeming circularity in formal languages and is going to downvote because of this, it's really not; don't downvote yet, keep reading)
Perhaps the most widely known example of a formal language is the language of first-order logic, whose alphabet consists, among others, of symbols (defined as "objects meaning other objects" like ${\forall}$ or ${\implies}$. As alphabet is a set of symbols used in the language, then in the ZFC set theory$^1$, where every element of a set is another set, these symbols, since they belong to a set, should be sets too. I haven't seen any way to define them as sets (like I did in the case of natural numbers, where for example $0$ is often defined as ${\emptyset}$). Did anyone propose a way how to do such a thing?
$^1$I feel like there is a way of using ZFC even before defining a formal language: list the axioms in a natural language$^2$, define set as every object which can be built from these axioms$^3$, build a first-order logic language using this notion of a set, write down the ZFC using this first-order logic language and use this new list of axioms. Is this correct?
$^2$The meaning which ZFC tries to convey is possible to be said in a natural language.
$^3$In case somebody says that at this level the concept of a set doesn't need to be formalised and is best left to be intuitively understood, one of the reasons ZFC is so popular is that it reproduces intuitive properties of sets - so there seems to be no harm in using ZFC this way.
