In its most formal description, a theory knows only the basic logical symbols $\land, \lor, \neg,\to,\leftrightarrow,\forall,\exists, =$ and an infinite supply of variables $a,b,c,\ldots, a_1,a_2,,\ldots, a',a'',\ldots$ which can be combined in certain manners forming well-formed formulas. Depending on the topic at hand, some additional symbols may be introduced. In particular, for a set theory (not a sets-and-classes theory), one single additional symbol $\in$ is added to the language. You might say that this additional symbol $\in$ carries no a priori meaning and it is the axioms that provide it with meaning or force us to interpret the (a priori meaningless) variables as our mental idea of sets as some kind of collection and the $\in $ relation as element containment.
But to make our lives easier, we tend to enhance the language by additional symbols, each time specifying a way to get rid of that symbol in order to arrive at the original language of the theory. For example we may introduce a $\ne$ symbol that can appear anywhere the $=$ symbol can appear and define $v_1\ne v_2$ as $\neg(v_1=v_2)$. And we certainly introduce $x\subseteq y$ as a shorthand for $\forall z\,(z\in x\to z\in y)$. Or we might introduce $\exists !$ as a new symbol that can appear anywhere a quantor can appear and define $\exists!v\, \phi(v)$ as $\exists v\,(\phi(v)\land\forall u\,\phi(u)\to u=v)$ (with some elaboration on $\phi$ and $u$ needed here).
Or we might even have started with less symbols than listed above and introduced some of the this way, e.g., $\phi\leftrightarrow \psi$ as $(\phi\to \psi)\land (\psi\to \phi)$.
Furthermore, we introduce more of the above when motivated by the axioms or theorems of our theory. Actually, the quoted formulation of the axioms already does that by introducing notations such as $\{x,y\}$. More formally, one could write the Axiom of Pairing as
$$ \forall a\,\forall b\,\exists z\,\forall t\,(t\in z\leftrightarrow (t=a\lor t=b)).$$
Then, using the Axiom of Extensionality, one can show as a theorem that $\exists z$ can be replaced with $\exists!z$, which motivates the new notation $\{a,b\}$ that can appear anywhere a variable can appear - except after a quantor, and the required elimination rule is that each occurance of $\{a,b\}$ (i.e., on either side of a $\in $ or $=$ symbol) is replaced with a new variable $z$, which is introduced suitably nearby, so e.g., $\{a,b\}\in c$ becomes $\exists z\,(\forall t\,(t\in z\leftrightarrow (t=a\lor t=b)\land z\in c)$.
The same can be done with the other notations introduced (i.e., for all those axioms that postulate the existence of a set with specific properties where we can prove that this set is in fact unique), i.e., Separation, Power, Union, and Replacement. And the same applies to some set constants that we can define, such as introducing the symbol $\emptyset$ for the (provably existing and unique) set without element or the symbol $\omega$ for the (provably existing and unique) minimal inductive set.
Note that also the Axioms of Extensionality, Infinity, Regularity, and Choice are (or can be) formulated as existence claims (perhaps surprisingly in case of Extensionality: $ \forall a\,\forall b\,\exists z\,(a=b\leftrightarrow(z\in a\leftrightarrow z\in b))$). But in these cases we do not have uniqueness, and that is why these do not motivate us to introduce new notation.
Of course, we can continue to introduce new notation as long as this happens by definitions that can be properly eliminated just as with the examples so far. For example, we can define singleton set notation $\{a\}$ by defining this as $\{a,a\}$, or define ordered pairs $(a,b)$ as $\{\{a\},\{a,b\}\}$ and then derive the theorem(!) that $(a,b)=(c,d)\leftrightarrow (a=c\land b=d)$.
It is perhaps about time I mention classes.
We can avoid viewing classes as objects of their own in our set theory and introduce them as mere matters of speech. Given any predicate, we may want to introduce new notation that suggestively looks suspiciously like an element-of-a-set relation, but truely isn't. For example, if we are interested in the predicate "is an ordered pair", i.e., $\text{is-a-pair}(x)\iff \exists a\,\exists b\,x=\{\{a\},\{a,b\}\}$, it is desirable to introduce a less clumsy notation such as $x\in\text{Pairs}$. This is formally confusing because on the right of $\in$, only sets (more specifically, variables - but these always stand for sets) are "allowed" so far. So one should not view this as a symbol $\text{Pairs}$ introduced that can occur only (in a "forbidden" way) to the right of $\in$, but rather as a single symbol $\in\text{Pairs}$ that stands for a predicate in postfix notation and happens to use the symbol $\in$ as its first letter. This way, the class $\text{Pairs}$ does not exist as any new kind of object (and it is guaranteed that we cannot even express syntactically that the class is an element of something) and thus there is no way of saying that classes obey any specific axioms that would need to be added. We do have some theorem schemes though that look as if we can perform a few basic operations of set theory on classe, e.g., if $A,B$ are classes, we can speak of $A$ being a subclass of $B$ if $\forall t\,(t\in A\to t\in B)$, or we can define a class $A\cup B$ such that $$x\in A\cup B\iff x\in A\lor x\in B$$ or a class $A\times B$ such that $$x\in A\times B\iff \exists a\,\exists b\,(x=(a,b)\land a\in A\land b\in B).$$
Just as the Axiom Schema of Separation allowed us to introduce the set builder notation $\{\,u\in X:P(u,p)\,\}$, we can introduce a class builder notation $\{\,u:P(u,p)\,\}$ (i.e., unrestricted comprehension) with the same caveat as above, i.e., we actually only introduce the postfix predicate $\in\{\,u:P(u,p)\,\}$ with the property
$$ x\in \{\,u:P(u,p)\,\}\iff P(x,p).$$
As a final remark: Sometimes we want to express that a class $A$ in fact equals a set $a$. Since only $\in A$ and not $=A$ is allowed yet, this should be introduced by defining
$$a=A\iff \forall t\,(t\in a\leftrightarrow t\in A) $$
