The conjecture is false. Here is a counterexample.
Suppose $R$ is a ring with the property that every $r \in R^\times$ satisfies $r^2 = 1$.
Then both $(R,+)$ and $(R^\times,\cdot)$ are abelian groups with the property that every element has exponent $2$ — that is, they are vector spaces over $\mathbf{F}_2$.
If $B$ is a basis for a vector space $V$ over $\mathbf{F}_2$, then the elements of $V$ can be identified with finite subsets of $B$. If $B$ is infinite, it has the same cardinality as its set of finite subsets. Consequently, $(R,+)$ and $(R^\times, \cdot)$ are isomorphic if and only $R$ and $R^\times$ have the same cardinality.
Let $X$ be a set of indeterminates, and define the ring
$$ T[X] = \mathbf{F}_2[X] / \langle x^2 - 1 \mid x \in X \rangle $$
$(T[X], +)$ is a vector space whose basis is the set of all finite subsets of $X$. For any $v \in T[X]$, let $\deg(v)$ be the sum of the coefficients of $v$.
For every $v \in T[X]$, $v^2 = \deg(v)$.
Therefore, for every $v \in T[X]$, we either have $v$ is zero divisor ($v^2 = 0$) or $v$ is a unit (with inverse $v$). Thus, $T[X]^\times$ is the set of all elements with $\deg(v) = 1$.
If $X$ is infinite, then $T[X]$ and $T[X]^\times$ have the same cardinality, and therefore $(T[X],+)$ is isomorphic to $(T[X]^\times, \cdot)$ as abelian groups.