I decided to make my comment an answer since it appears to be long.
Observe that $\left\lvert\left\lvert F\right\rvert\right\rvert_{TV([a+\epsilon,b])}\le\left\lvert\left\lvert F\right\rvert\right\rvert_{TV([a,b])}$ since if $P$ is a partition of $[a+\epsilon,b]$
then
$$\sum_{i=1}^{n}\left\lvert F(x_{i})-F(x_{i-1})\right\rvert \le\left\lvert F(a+\epsilon)-F(a)\right\rvert+\sum_{i=1}^{n}\left\lvert F(x_{i})-F(x_{i-1})\right\rvert\le\left\lvert\left\lvert F\right\rvert\right\rvert_{TV([a,b])}$$
Taking a supremum over partitions gives the described result. Thus, taking a limit in $\epsilon$ we obtain
$$\limsup_{\epsilon\to0^{+}}\left\lvert\left\lvert F\right\rvert\right\rvert_{TV([a+\epsilon,b])}\le\left\lvert\left\lvert F\right\rvert\right\rvert_{TV([a,b])}$$
Observe that through a similar proof we may obtain:
$$\left\lvert\left\lvert F\right\rvert\right\rvert_{TV([a+\epsilon_{1},b])}\le\left\lvert\left\lvert F\right\rvert\right\rvert_{TV([a+\epsilon_{2},b])}$$
when $\epsilon_{1}>\epsilon_{2}$. Now take a partition $P=\left\{a=x_{0}<\ldots<x_{n}=b\right\}$. By continuity of $F$ at $x=a$ we can find $\delta>0$ so that $\left\lvert F(x)-F(a)\right\rvert<\eta$ if $\left\lvert x-a\right\rvert<\delta$. By perhaps refining our partition and increasing the variation we may assume $\left\lvert x_{1}-a\right\rvert<\delta$. So:
$$\sum_{i=1}^{n}\left\lvert F(x_{i})-F(x_{i-1})\right\rvert\le\eta+\left\lvert\left\lvert F\right\rvert\right\rvert_{TV([x_{1},b])}\le\eta+\liminf_{\epsilon\to0^{+}}\left\lvert\left\lvert F\right\rvert\right\rvert_{TV([a+\epsilon,b])}$$
So $$\left\lvert\left\lvert F\right\rvert\right\rvert_{TV([a,b])}\le\eta+\liminf_{\epsilon\to0^{+}}\left\lvert\left\lvert F\right\rvert\right\rvert_{TV([a+\epsilon,b])}$$
So $$\left\lvert\left\lvert F\right\rvert\right\rvert_{TV([a,b])}\le\liminf_{\epsilon\to0^{+}}\left\lvert\left\lvert F\right\rvert\right\rvert_{TV([a+\epsilon,b])}$$
Note that if you remove continuity of the function at $a$ then the limit is not true. For example, consider $f:[0,1]\to\mathbb{R}$ defined by $f(x)=\chi_{\left\{0\right\}}(x)$. Then $\left\lvert\left\lvert f\right\rvert\right\rvert_{TV([\epsilon,1])}=0$ for $\epsilon>0$ but $\left\lvert\left\lvert f\right\rvert\right\vert_{TV([0,1])}=1$.