Broadly taken, a counterpart of this issue is the question of semantic ascent after W. V. Quine in philosophy. I believe it will be beneficial to expand on the issue in that perspective as well.
Let us set down the formal terminology (mostly because of shortcut usages, there are variations, even incoherences among authors):
- $A\rightarrow B$ is a proposition in the form of material implication. It is a well-formed formula of propositional calculus. The language of propositional calculus is the object language: Its objects are propositions, we express propositions and their relations through it.
- $\Gamma\vdash A\rightarrow B$ is a sentence in the metalanguage associated with propositional calculus. It states a judgement, that $A\rightarrow B$ is deducible from $\Gamma$. We express judgements about propositions and their relations through the metalanguage.
- $\Gamma\vDash A\rightarrow B$ is, likewise, a sentence in the metalanguage associated with propositional calculus. It states a judgement, that $A\rightarrow B$ is logical consequence of $\Gamma$.
- $\vDash A\rightarrow B\;\Rightarrow\;\vdash A\rightarrow B$ is also a sentence in the metalanguage associated with propositional calculus. It states a judgement, that if $A\rightarrow B$ is a tautology then it is a theorem of propositional calculus, which is a metatheorem.
Ascending to a higher level language (metalanguage) is an almost indispensable method; thereby we are able to state generalisations that otherwise (unless an alternative method is found) would not be possible. Examples are abundant, any general thesis about a system that contains infinitely many axioms (Peano Arithmetic, ZFC, . . .) cannot be stated by any finite sentence.
In Logical Syntax of Language, Rudolf Carnap makes a distinction between material and formal modes of speech. In the material mode, we talk mainly about the world, extra-linguistic reality; in the formal mode, we talk about the language that describes the world. Roughly speaking, Carnap's material mode of speech corresponds to the object language and formal mode to the metalanguage in the present context.
Thus, "$\omega$ is the first transfinite ordinal number" is a description in material mode; and "'$\omega$' is a name that belongs to the domain of numbers" is a formal specification.
Quine takes on Carnap's distinction and adds an ontological concern to it: Moving up the hierarchy of languages, we also move away from the talk about factuality, the objects and their relations we are empirically familiar to the talk that purports to refer to them in such a way as Bertrand Russell has said in Mysticism and Logic and Other Essays about mathematics: ". . . in which we never know what we are talking about, nor whether what we are saying is true."
Beside the usefulness of metalinguistic discourse mentioned above, Quine points out in Word and Object that semantic ascent is widespread device we come across under various appearances:
The strategy [of semantic ascent] is one of ascending to a common part
of two fundamentally disparate conceptual schemes [compare to
proof-theoretic and model-theoretic views of logical consequence], the
better to discuss the disparate foundations. No wonder it helps in
philosophy.
But it also figures in the natural sciences. Einstein's theory of
relativity was accepted in consequence not just of reflections on
time, light, headlong bodies, and the perturbations of Mercury, but of
reflections also on the theory itself, as discourse, and its
simplicity in comparison with alternative theories. Its departure from
classical conceptions of absolute time and length is too radical to be
efficiently debated at the level of object talk unaided by semantic
ascent.
Formally, we can multiply the linguistic/theoretic meta-levels just as we can multiply the dimensions of a vector space. This poses a dilemma to us: While the ascent is a useful method, strategy, tool, it distances us from the realm where our primary concerns, in fact, reside. On the other hand, without it, we cannot get off the ground. Given the stage of the development of matters, a reconciliatory principle, if not its unfolding ways, can be that we may push on till the level up to which our generalisations keep meaningful and our abstractions keep truthful.