Bill Dubuque gave an answer in the commutative case. For the general case, we can just say, well, "closed under taking inverses", but in functional analysis the term inverse closed (also inverse-closed) is common to denote Banach subalgebras with that property. See for example:
Bruce A. Barnes, Inverse Closed Subalgebras and Fredholm Theory, Proceedings of the Royal Irish Academy. Section A: Mathematical and Physical Sciences, Vol. 83A, No. 2 (1983), pp. 217-224 (8 pages)
Karlheinz Gröchenig, Michael Leinert, Inverse-Closed Subalgebras of Noncommutative Tori, arXiv:1208.6229
Since the property just refers to the underlying rings (actually, just the underlying multiplicative monoids) of the Banach algebras in consideration, it does make sense to use the same terminology for general rings. And indeed, the terminology inverse-closed subrings is used in a few papers, such as:
Dragana Cvetkovic-Ilic, Robin Harte, The spectral topology in rings, Studia Mathematica 200 (3) (2010)
Beren Sanders, Higher comparison maps for the spectrum of a tensor triangulated category, Advances in Mathematics 247, pp. 71-102
The property is also studied in the paper
Jeno Szigeti, Leon van Wyk, Subrings which are closed with respect to taking the inverse, Journal of Algebra 318(2)
It is remarkable that they do not use any terminology (well, except for the one in the title). They just repeat the condition $S \cap U(R) = U(S)$ over and over again (sometimes with elements).