I am trying to use this property in the context of answering Rudin Exercise 6.1. I will repeat the problem statement and my proof below, and I will embolden the part where I use the statement in the title.
Suppose $\alpha$ increases on $[a,b]$, and $\alpha$ is continuous at some $c \in [a,b]$. If $f(c) = 1$ and $f(x) = 0$ for $x \neq c$, then $f \in \mathcal{R}(\alpha)$ and $\int_a^b f(x) d\alpha(x) = 0$.
We verify the existence and value of the limit directly. Observe that \begin{align*} \underline{\int_a^b}f dx &= \sup L(P,f,\alpha)\\ &= \sup\left\{\sum_{i=1}^n m_i\Delta \alpha_i\right\} =0 \end{align*} The above follows from the fact that the infimum of the function on any interval will be $0$ since the function is $0$ everywhere except at $c$.
Meanwhile, for the upper Riemann-Stieltjes integral, we claim that \begin{align*} \overline{\int_a^b} f dx &= \inf U(P,f,\alpha) \\ &=\inf\left\{\sum_{i=1}^n M_i\Delta \alpha_i\right\} =0 \end{align*} To prove the equality above, let $\varepsilon>0$ be arbitrary. Since $c \in [a,b]$ it is not an isolated point. Since $\alpha$ is continuous and monotonic, we can find some interval $[x_k, x_{k+1}]$ around $c$ for which $\alpha$ is continuous. Choose points $x_k', x_{k+1}'$ in this interval so that $c \in [x_k', x_{k+1}']$ and so that for any positive integer $n$ $$\alpha(x_{k+1}') - \alpha(x_{k}') = \frac{\alpha(x_{k+1}) - \alpha(x_k)}{n}$$ we can do this by the Intermediate Value Theorem in a similar way to Theorem 6.9 in Rudin. Let us include $x_k', x_{k+1}'$ in our partition. Then we have $$\sum_{i=1}^n M_i\Delta\alpha_i = \alpha(x_{k+1}') - \alpha(x_{k}')= \frac{\alpha(x_{k+1}) - \alpha(x_k)}{n} < \varepsilon$$ for large enough $n \in \mathbb{N}$. Thus, for every $\varepsilon>0$ we can find a partition such that $\sum_{i=1}^n M_i\Delta\alpha_i < \varepsilon$. Therefore, $$\underline{\int_a^b}f dx = \overline{\int_a^b} f dx = 0$$