An equivalent integral
Plugging a relationship similar to the one used by Jack D'Aurizio, namely
$$\frac{1}{k!(k-1)k}=\int_0^1 \frac{x^{k-2}(1-x)}{k!} dx$$
into this integral
$$\int_0^1 \frac{(1-x)(e^x-1-x)}{x^2}dx = 3-e$$
with non-negative integrand in $(0,1)$ that proves $e<3$,
yields the series in the question.
$$\sum_{k=2}^\infty \frac{1}{k!(k-1)k}=3-e$$
This relates the series with the inequality $1+x \leq e^x$.
Similar approximations
This integral is similar to the one used to explain why $e$ is close to the eighth harmonic number.
$$\frac{1}{14} \int_0^1 x^2(1-x)^2(e^x-1-x)dx = e-\frac{761}{280}=e-H_8\approx 0$$
with corresponding series
$$e=\frac{761}{280}+\frac{1}{7}\sum_{k=2}^\infty \frac{1}{k!(k+3)(k+4)(k+5)}$$
and
$$\frac{1}{2}\int_0^1 (1-x)^2\left(e^x-1-x-\frac{x^2}{2}\right)dx = e-\frac{163}{60}$$
used to explain the observation by Lucian that $2\pi+e$ is close to $9$.
Another series for $3-e$
Yet another series to prove $e<3$ is related to the integer sequence http://oeis.org/A165457.
$$\frac{1}{e}=\frac{1}{3}+\sum_{k=1}^\infty \frac{1}{(2k+1)!(2k+3)}$$