Your main idea is correct more formally you can proceed like this :
Let $\epsilon>0$ since $\alpha_n \to \alpha $ there is an $N_1$ such that
$|\alpha_{n} - \alpha| < \frac{\epsilon}{2}$ for every $n \geq N_1$.
Now for every $n > N_1$ you estimate the difference
$\biggl|\dfrac{\alpha_1+...+\alpha_n}{n}-\alpha \biggr|=\biggl|\dfrac{(\alpha_1-\alpha)+...+(\alpha_{N_1}-\alpha)+(\alpha_{N_{1}+1}+\alpha)...+(\alpha_n-\alpha)}{n}\biggr| \leq$
$\biggl|\dfrac{(\alpha_1-\alpha)+...+(\alpha_{N_1}-\alpha)}{n}\biggr|+\biggl|\dfrac{(\alpha_{N_{1}+1}-\alpha)+...+(\alpha_n-\alpha)}{n}\biggr|\leq$
$\biggl|\dfrac{(\alpha_1-\alpha)+...+(\alpha_{N_1}-\alpha)}{n}\biggr| + \dfrac{n-N_1}{n}\dfrac{\epsilon}{2} \leq \ \biggr|\dfrac{(\alpha_1-\alpha)+...(\alpha_{N_1}-\alpha)}{n}\biggr| + \dfrac{\epsilon}{2}.$ $\ $ $(*)$
By using again the fact that $\alpha_n -\alpha \to 0 $ we can find an $M>0$ such that $|\alpha_n-\alpha| \leq M.$
So we can write the $(*)$ equation like this :
$\biggl|\dfrac{\alpha_1+...+\alpha_n}{n}-\alpha\biggr| \leq \dfrac{N_1M}{n} +\dfrac{\epsilon}{2}$. $\ $ $(**)$
Now we pick an $n_0>N_1$ such that $\dfrac{1}{n_0} <\dfrac{\epsilon}{2}\dfrac{1}{N_1M}.$ $(1)$
And now for every $n \geq n_0$ from $(**)$ and $(1)$ we finally have
$\biggl|\dfrac{\alpha_1+...+\alpha_n}{n}-\alpha\biggr| < \dfrac{\epsilon}{2}+\dfrac{\epsilon}{2} = \epsilon.$
Which means that $\dfrac{\alpha_1+...\alpha_n}{n} \to \alpha.$