You should think of $|R:F|$ as size of the basis needed so that the F-span of them is R (i.e. coefficients in F).
I think the comments answered some of your question so I will just answer the main part. (I assume that an integral domain is commutative in this context)
I will use some Galois theory (and so I hope you have been introduced to what I will be using).
Since $|R:F|< \infty$ (say $|R:F|=n$) you have for every $r\in R$ the set $\{1,r,r^2,\dots,r^n\}$ is linearly dependent and none of which are zero as we are in an integral domain. In particular every element in R is algebraic over F.
(note algebraic means there exists a polynomial such that r is a root)
So take the algebraic closure $\bar{F}$ of F (which is a field and it is the collection of all roots for all polynomials).
We get $ F \subseteq R \subseteq \bar{F}$ (note that being an integral domain means that the only thing that is not assured is the existence of inverses so the operation of R will not violate operations of $\bar{F}$. so we can have this embedding).
So the following claim should be enough
Let L be an algebraic extension of F. If $ F \subseteq R \subseteq L$ for some ring R. Then R is a field.
proof:
All properties of being a field hold by virtue of being imbedded in L which is a field. The only nontrivial part is the existence of an inverse.
Let r be an element of R. r is algebraic so:
$\sum_{m=0}^{n}{a_mr^m}=0$ where $a_m \in F$ not all zero. Say j is the smallest integer such that $a_j \ne 0$
So we have $\sum_{m=j}^{n}{a_mr^m}=0$
$r\in L$ a field, so $r^{-1} \in L$. so by multiplying by $r^{-j-1}$ we get
$r^{-1}= -a_j^{-1}\sum_{m=0}^{n-j}{a_mr^m}$. However all powers of r are in R as R is a ring so we get the left hand side must be in R too.