First, let's consider the structure of the two expressions jemandem (etwas) geben and zu jemandem kommen.
It depends on the verb which object it takes (e.g. a prepositonal object, a dative object, an accusative object):
The verb geben requires two objects: a dative object (receiver) and an accusative object (given thing):
jemandem(=Dativ) etwas(=Akkusativ) geben.
Therefore, your second example uses solely the dative pronoun mir without any preposition.
English has lost most of its cases, so there isn't anything that can be really compared to the German dative case; English has no cases to distinguish two objects. It uses a different means to mark the receiver and the given thing: In the sentence give me the book this is done by word order; in give it to me by the preposition to.
The verb kommen needs a destination, where the person is going to. That destination is linked to kommen by a preposition, i.e., kommen takes a prepositional object. Which prepositions are acceptable has to be learnt by heart or deduced from context via logic. The case of the noun following the preposition is determined by the preposition itself. For example, you could say:
er kommt in das Haus (in + Akkusativ; into the house)
er kommt auf das Dach (auf + Akkusativ; onto the roof)
er kommt zu der Straße (zu + Dativ; to, towards the street)
Note, that the so called Wechselpräpositionen use accusative case when refering to a direction rather than a location.
The preposition zu is an appropriate one to go with "kommen". It isn't a Wechselpräposition and it always requires dative case. So you can see that the preposition zu is not really "added" in the phrase komm zu mir as if there were a hole where you can just put the zu in. Don't think of it as if the mir was there all the time and you only had to decide whether or not to put the zu there. The reasoning is the other way round: kommen can go with the preposition zu which requires the dative pronoun mir.
Now, let's think about how to me can be translated to German. By equating mir = to me or zu mir = to me things are being oversimplified.
There are several possibilities for translating to me to German, depending on context:
- translate it as a pure dative mir (as in give it to me = gib es mir)
- translate it as an accusative mich (as in don't lie to me = lüg mich nicht an)
- translate it as prepositional object zu mir (as in come to me = komm zu mir)
- translate it as a different prepositional object, e.g. für mich (as in sounds good to me = hört sich für mich gut an)
To choose the correct one, you have to know whether the verb in question needs a dative object or an accusative object or a prepositional object. And if the verb requires a prepositional object, then you must know/deduce which prepositions are possible and which case each of these prepositions takes.
Finally, let's come back to your question:
Am I right in assuming that when you must express "to me" literally, it is okay to say "zu mir," and when it should be implied, it is simply "mir?"
No. I hope it became clear that you cannot simply say that whenever an English sentence contains to me it has to be translated by zu mir or that an "implied" to me must be translated as mir. English and German need not use the same expressions/structures:
- Take for example the phrase I ask him. Somehow it is "implied" that the question points to him; he is the receiver of what is asked. Nonetheless, you don't say I ask to him nor do you say ich frage ihm(=Dativ).
- Another counter example would be something happened to me. There is a literal to me in this sentence. However, the German translation would be mir ist etwas passiert, not zu mir ist etwas passiert.