1+1=2. My question is : is this a fact that can be proved, or is it a just a postulate as those in geometry ?
For the question even to make sense, you'd have to precisely specify what you mean by "1", "2", "+" and "=". Otherwise no go. Usually "2" is defined as "1+1", and "x = y" is defined to mean "x is the same as y", so $1+1=2$ simply by definition. This has nothing to do with babies, unless you interpret it as a statement about the real world, namely that "1" is used to denote one entity, while "2" is used to denote two entities, and we ignore the individual natures of entities, and "+" is used to denote putting together. Then clearly "1+1" means "putting together one entity and one entity", which clearly is the same as "2" which means "two entities". So if anyone says babies understand this, then they are simply saying that babies understand that putting one entity together with another gives two entities (not one, not zero, not three, ...).
Also what about the saying "A given statement can be either true or false." ? Why are there two and only two cases for a statement ?
Again, you'd have to specify what is meant by "statement" and "true" and "false" and "or". Usually we only consider factual statements, which by definition are statements that are either true or false about the (one) real world in the current context. If you don't like this definition, then you'd have to specify your own. The same factual statement might be true in one context but false in another. Also, English has other kinds of sentences that aren't factual, such as "Come here!" (imperative) and "What is this?" (question), and also fragments, such as "Yes." and "I agree.". All these have to be excluded.
Furthermore, not every declarative English sentence is a factual statement (can be assigned a truth value), even if it is of the form "X is Y". For example consider Quine's paradox:
" preceded by the quotation of itself is a false sentence." preceded by the quotation of itself is a false sentence.
This is a sentence that is unambiguously understood but cannot have a truth value. It is different from Russell's paradox because it has absolutely no self-reference. All it says is that if you take the phrase in quotes and precede it by the quotation of itself, then the result is a false sentence. It seems like this should be either true or false, but it is neither, unless you like contradictions.
Since there is a priori no reason that every declarative sentence in English (or any other natural language) must have a truth value, we have no reason to expect to be able to say that the above sentence is either true or false.
Now the situation is slightly different in mathematics. In classical first-order logic, we first define syntactically what mathematical statements are valid sentences, and then we choose some valid sentences as axioms. Then we say that we have a model of the axioms if there is a mathematical structure for which all the axioms are true. Note that there may be models of the same axioms that disagree on the truth value of some sentence, even they agree that all the axioms are true!
Just for example, consider the following axiom:
There are at least two different things. / $\exists x,y\ ( x \ne y )$.
Here are two different models of this axiom:
This structure has exactly two different things.
This structure has exactly three different things.
Now notice that the following sentence is false for the first structure but true for the second structure:
There are at least three different things. / $\exists x,y,z\ ( x \ne y \land y \ne z \land z \ne x )$.
The bottom line is that truth in mathematics is never absolute, but always with respect to a given structure. Truth in the real world can be considered absolute only in the sense that it is with respect to the (one) real world, but that is if you can define truth in the first place. As Quine's paradox shows, this is not going to be easy if at all possible.