temo:
The idea of symbols is easy to formalize, though whichever route you follow, the formalization may look a bit awkward while you get used to it. One way is as follows: A set of symbols consists of 3 disjoint sets $A,B,C$. We call the elements of $A$ "constant symbols". We call the elements of $B$ "function symbols", and the elements of $C$ "relation(al) symbols". Moreover, $B$ is the disjoint union of sets $B_n$, $0<n\in{\mathbb N}$, and the elements of $B_n$ are the symbols for $n$-ary functions. Similarly, $C$ is the union of sets $C_n$, and the elements of $C_n$ are the symbols for $n$-ary relations.
One can change this in several ways. For example, we can think of constant symbols as $0$-ary function symbols, and we could even make do with only relation symbols at the cost of adding some additional axioms later (indicating that some symbols represent the graph of a function). Also, one could add infinitary relation or function symbols, if one wanted, but the setting above is the most classical.
One can also be even more precise, if one so wanted. For example, you could state that a symbol is a triple $(\alpha,\beta,\gamma)$ where $\alpha=0$ or $1$, $\beta\in{\mathbb N}$, and $\gamma$ is an ordinal. We call triples of the form $(0,0,\gamma)$ constant symbols, of the form $(0,n,\gamma)$ with $n>0$ function symbols, and of the form $(1,n,\gamma)$ relation symbols, and we require that $n>0$ in the last case. A set of symbols is then just a subset of the class of triples just described.
The above is simply a pedantic way of emphasizing that there is nothing "non-mathematical" in the concept of "symbol".
Now, given a set $S$ of symbols, a structure of type $S$ (or "in language $S$") is a tuple $M=(A,F)$ where $A$ is a non-empty set, and $F$ is a function with domain $S$. We have that
- $F(0,0,\alpha)\in A$ for all $(0,0,\alpha)\in S$,
- $F(0,n,\alpha)$ is a function $A^n\to A$ for all $(0,n,\alpha)\in S$ with $n>0$, and
- $F(1,n,\alpha)\subseteq A^n$ for all $(1,n,\alpha)\in S$.
Of course, one never writes things this way. Given a symbol $c$, we write $c^M$ for $F(c)$, so $c^M$ is a constant (an element of $A$) if $c$ is a constant symbol, and similarly, $f^M$ is a function (an $n$-ary function, in fact, if $f$ is of the form $(0,n,\alpha)$), and $R^M$ is an $n$-ary relation if $R$ is a relation symbol of the form $(1,n,\alpha)$.
Still the above looks ridiculously convoluted. Note that different symbols $c$ and $d$ may have the same interpretation $c^M=d^M$, for example.
The question is then, independently of how one ends up formalizing the concept of "symbol", why do we use it at all instead of just using the functions, etc, directly.
One of the points here is that we may want, for example, to talk about certain structures defined by axioms. Say, groups. We need a language to write down these axioms. The language consists of some logical symbols and some non-logical ones. The non-logical ones typically consist of a constant symbol $e$, a unary function symbol $i$, and a binary function symbol $f$. We could set $e=(0,0,0)$, $i=(0,1,0)$, and $f=(0,2,0)$ if we wanted.
We write the axioms saying that $e$ is to be interpreted as the identity of $f$: For all $a$, $f(a,e)=f(e,a)=a$; that $f$ is associative: For all $a,b,c$, $f(a,f(b,c))=f(f(a,b),c)$; and $i$ is an inverse for $f$: For all $a$, $f(a,i(a))=f(i(a),a)=e$.
Again, we may write this in more palatable ways, $a*(b*c)=(a*b)*c$, etc.
A group is then a structure $M$ in the language of groups, that satisfies the three axioms above when $e,i,f$ are understood to mean $e^M,i^M,f^M$, respectively.
Note that the same language allows us to talk about many different classes of structures, not just groups. Using symbols allows us to be able, in a uniform way, to refer to these structures, to define what a homomorphism between structures is, to explain how a structure of a certain kind can be transformed into a structure of another kind, etc, without having to go into the details of what the structures actually are or what properties they satisfy.
Of course, one may very well argue directy in terms of structures, and never mention symbols. Neither approach is better than the other. We use the formal format above because it makes certain details easier, but at least at an introductory level, not much would be lost if we never used symbols directly. But it is a good idea to invest some time in working with them.
Here is an example: Say we have a binary relation symbol $R$. A typical example would be a structure $M$ where the interpretation $R^M$ is a linear order $<$. But there are many different possibilities: $R$ could be a partial order, the edge relation in a directed graph, or something else entirely. The statement "For all $a$ there is a $b$ with $aRb$" is true in some of these structures and false in others. But to even say this would be awkward without symbols, because you need to talk about what $R$ means in each specific structure otherwise, and you would end up referring to the interpretation of $R$ in each case anyway.
A word about your final comment: A group $(G,\cdot)$ is a tuple, where $\cdot$ is a mapping. This mapping is the interpretation of a symbol for a mapping. When you discuss the axioms of groups or specific properties you want to study, you use a symbol for a mapping. When discussing groups that may or may not have the properties you are looking for, you use mappings that interpret these symbols.
Let me close with a comment that this idea of distinguishing between symbols and their interpretations actually becomes very important in some contexts. For example, in logic, we have Gödel's famous incompleteness theorem that tells you, in particular, that the basic axioms for (first order) number theory cannot prove all truths about numbers. The first step in the proof of this result consists in coding the statements of number theory using number theory. Note this is a bit of a problem: Number theory is about numbers, not about logical formulas that talk about numbers. What we do first is to code the symbols of number theory using numbers, so "2+3=5" could be coded by, say, a number of the form $2^a3^b5^c7^d11^e$ where $a$ is a number that codes the symbol 2, $b$ is a number that codes the symbol $+$, $c$ is a number that codes the symbol $3$, etc, and here it becomes clear that it is important to distinguish, for example, between the actual number 2 and the code for it.
