Partial answer
How can $F$ be differentiable and not continuous at $c$?
It can’t. But this isn’t what Spivak claims; you’re mixing up $F$ and $F’$.
Spivak claims that $F’$, not $F$, need not be continuous at $c$. And in the solution manual he gives an explicit counterexample: $f$ can be Riemann integrable on $[a,b]$ and differentiable at an interior point $c$ without $F’$ being continuous at $c$. But as I discuss here, this counterexample only works because of Spivak’s very stringent definition of a limit at a point, which requires the function to be defined on an interval around the point (except possibly at the point itself). In his counterexample, the reason $F’$ isn’t continuous at $c$ is that $F’$ isn’t even defined at a series of points that converges to $c$.
Because of this, I find Spivak’s example “morally” disappointing: it works only because of a technicality. It’s more a consequence of a bad definition than a statement about the regularity of the function $F’$.
Indeed there is a more general definition of the limit of a function at a point that doesn’t require the function to be defined on a deleted interval around the point, and according to that definition, Spivak’s counterexample no longer works: his $F’$ is indeed continuous at $c$ using that other definition. In other words, near $c$ his $F’$ is pretty well-behaved wherever it’s actually defined.
I don’t know yet whether the claim “$f$ differentiable at $c$ implies $F’$ continuous at $c$” is true using this more general definition of a limit.
EDIT
User zhw proves here that the claim becomes true if we use the more general definition of continuity.