Both of your rings $\,\mathbb R\,$ and $\,\mathbb R[x]\,$ are in fact instances of an algebra over a field or, more generally, an algebra over ring. Recall that an algebra over a ring $\,R\,$ is simply a ring $\,A\,$ containing a central image of $\,R,\,$ i.e. $\,A\,$ contains a subring $\,R' \cong R/I\,$ such that the elements of $\,R'$ commute with all elements of $\,A.$
For example $\,R/I\,$ is an $\,R$-algebra for all ideals $\,I\subset R,\,$ including $\,R/0\,\cong R.\,$ Another well known example are polynomial rings over $\,R\,$ or $\,R/I,\,$ where the central image is simply the coefficient subring (the coefficients being central is precisely the condition required to ensure that the evaluation map from polynomial rings is a ring homomorphism and, hence, the fundamental ubiquitous universal property of polynomial rings).
Your examples are also instances of the more general notion of algebraic structures studied in Universal Algebra or General Algebra, i.e. classes of structures with a fixed set of finitary operations satisfying specified (equational) axioms, e.g. monoids, groups, rings, fields.
The latter more general notion of "algebra" encompassess most of the algebraic structures that are studied before abstract algebra, so it is certainly both consistent and acceptable to use the term "algebra" in those elementary contexts.