@Wolfy, your intuition is correct. Here is a detailed proof following your intuition.
Let $F:\mathbb{R}\rightarrow \mathbb{R}$ be increasing. The set of points at which $F$ is discontinuous is countable.
Proof:
Since $F$ is increasing, we have that, for any $x\in\mathbb{R}$, the lateral limits $\lim_{y\rightarrow x^-}F(y)$ and $\lim_{y\rightarrow x^+} F(y)$ actually exit.
So, for any $x\in\mathbb{R}$ where $F$ is discontinuous, we have
$$F(x^-)=\lim_{y\rightarrow x^-}F(y) < \lim_{y\rightarrow x^+} F(y)=F(x^+) $$
So open interval $(F(x^-), F(x^+))$ is non-empty.
Moreover , if $x_1,x_2\in\mathbb{R}$, $x_1<x_2$, we have
$$F(x_1^+)=\lim_{y\rightarrow x_1^+}F(y) < \lim_{y\rightarrow x_2^-} F(y)=F(x_2^-) $$
So, if $x_1,x_2\in\mathbb{R}$, $x_1 \neq x_2$ and $F$ is discontinuous at $x_1$ and $x_2$, we have that $(F(x_1^-), F(x_1^+))$ and $(F(x_2^-), F(x_2^+))$ are disjoint.
Let $D=\{ x\in\mathbb{R} : F \textrm{ is discontinuous at } x \}$.
Let us define $\psi: D \rightarrow \mathbb{Q}$, where for each $x \in D$, we choose a rational in $(F(x^-), F(x^+))$. Its is possible because, for each $x\in D$, $(F(x^-), F(x^+))$ is a non-empty open interval.
Moreover, $\psi$ is injetive. In fact, if $x_1,x_2\in D$ and $x_1 \neq x_2$, $(F(x_1^-), F(x_1^+))$ and $(F(x_2^-), F(x_2^+))$ are disjoint, so $\psi(x_1)\neq \psi(x_2)$.
So $\psi$ is an injective map from $D$ into $\mathbb{Q}$.