There does indeed exist a group $G$ with precisely one proper subgroup $H$ of infinite order.
Preamble:
Before we proceed, let's discuss the Prüfer $p$-group. Let $p$ be a prime.
We know that for every $n\geq 1$, the cyclic group $C_{p^n}$ has precisely one subgroup isomorphic to $C_{p^{n-1}}$. Therefore, we may form an infinite chain of groups $S_0<S_1<S_2<S_3<\cdots$ where each $S_i\cong C_{p^i}$. We define the Prüfer $p$-group as:
\begin{equation}
C_{p^\infty}\equiv\bigcup_{i\in\mathbb{N}}S_i
\end{equation}
The group is also isomorphic to the group given by $\{e^{2ki\pi/p^n}:n\in\mathbb{N},0\leq k<p^n\}$ under standard complex multiplication.
It should be somewhat clear from these definitions that $C_{p^\infty}$ has no infinite proper subgroups, since any infinite subgroup $K$ must contain elements of arbitrarily high order $p^i$, and thus contain $S_i$ as a subgroup for arbitrarily high index $i$, meaning that $K=\bigcup_{i\in\mathbb{N}}S_i=C_{p^\infty}$.
Explicit construction:
Now, to solve your problem, we can use the groups $G=C_q\times C_{p^\infty}$ and $H=\{e\}\times C_{p^\infty}$ where $p,q$ are distinct primes. To see this, suppose that $K<G$ is a subgroup of infinite order. There are two cases:
Case $1$: $K$ has no element of the form $(g,h)$ where $g$ is a generator of $C_q$, then it is clear that $K\leq H$, but since $H$ cannot have infinite proper subgroups, then $K=H$.
Case $2$: $K$ contains an element $(g,h)$ where $g$ is a generator of $C_q$. Suppose that $h$ has order $m=p^n$, then $(g,h)^m=(g^m,e)\in K$, (note that $g^m$ is a generator of $C_q$, since $q,m$ are coprime) and thus $K$ contains as a subgroup $C_q\times \{e\}$. Now, Since $K$ is infinite, Abelian, has no elements of infinite order, and is not the product of infinitely many cyclic groups, then $K$ must have elements of arbitrarily high order; this means that for $N\geq 1$, $K$ must contain an element $(g',h')$ of order $o(g',h')\geq qN$. In fact $N\leq\frac{1}{q}o(g',h')=\frac{1}{q}o(g')o(h')\leq o(h')=o({h'}^q)$(since $p,q$ are coprime). We have that $(g',h')^q=(e,{h'}^q)\in K$. In other words, $K$ contains an element $(e,{h'}^q)\in\{e\}\times C_{p^\infty}$ of order $o(e,{h'}^q)=o({h'}^q)\geq N$ for every $N\geq 1$. This means that $K$ contains a subgroup of $\{e\}\times C_{p^\infty}$ of infinite order, but we know that the only subgroup of $\{e\}\times C_{p^\infty}$ of infinite order is itself, so $K$ must contain $\{e\}\times C_{p^\infty}$. Finally, since $K$ contains $\{e\}\times C_{p^\infty}$ and $C_q\times \{e\}$ as subgroups, then it must contain $G=C_q\times C_{p^\infty}$ as a subgroup, which means that $K=G$.
We have thus proven that $G$ has only one proper subgroup of infinite order, which is $H$, as desired.
