Let $R$ be a ring and $A\subseteq R$. Let $R/A = \{x+A\mid x\in R\}$ be the set of cosets. Define relation $\equiv_A$ on $R$ with $$x\equiv_A y\iff x+A = y+A.$$
This relation is obviously equivalence relation on $R$. $R/A$ is a ring with well defined operations $$(x+A)+(y+A) = (x+y)+A\\(x+A)(y+A) = (xy+A)$$
induced by operations on $R$, if and only if $\equiv_A$ is a congruence, meaning that $\equiv_A$ is compatible with operations on $R$.
Denote with $\operatorname{Con}R$ the set of congruences on $R$ and with $\operatorname{Id}R$ the set of ideals of $R$. Let us define functions
\begin{align}
\varphi\colon \operatorname{Con}R&\to\operatorname{Id}R &\psi\colon \operatorname{Id}R&\to\operatorname{Con}R\\
\equiv\, &\mapsto \{x\in R\mid x\equiv 0\} &I&\mapsto \{(x,y)\in R\times R\mid x - y\in I\}.\end{align}
You can easily verify that these functions are well defined and that they are inverse to each other. Moreover, they are order preserving! If we extend $\psi$ to all subsets of $R$, then it is well known that the induced relation $\psi(I)$ is congruence if and only if $I$ is ideal.
Let us now return to our congruence $\equiv_A$ and its associated ideal $I_A$. It is clear that $x+A = y + A$ if and only if $x-y+A = A$, i.e. $x\equiv_A y$ if and only if $x - y\in I_A$, which means that $\equiv_A$ is equal to induced congruence by $I_A$. We conclude that $\equiv_A$ is congruence if and only if $I_A$ is ideal.
If $R/I_A$ is standard quotient by ideal, you can immediately verify that $R/I_A\cong R/A$, with isomorphism defined as $x+I\mapsto x+A$.
Returning to your examples.
Let $A = \{\pm p\mid p\ \text{odd prime}\}$. Instead of checking that $\equiv_A$ is congruence, by the above discussion it is enough to check that $I_A = \{x\in R\mid x + A = A\}$ is ideal. Indeed, we get that $I_A = \{0\}$, so $\mathbb Z/A$ is a ring isomorphic to $\mathbb Z$.
Let $B = \{2n+1\mid n\in\mathbb Z\}$. Then $I_B = 2\mathbb Z$, so $\mathbb Z/B$ is ring isomorphic to $\mathbb Z/2\mathbb Z$.
So far everything looks fine and dandy. However, working with general subsets of $R$ instead of ideals is very ill-behaved. The obvious reason is that we don't even know if $R/A$ is a ring or not. Furthermore, when $I\subseteq J$ are ideals of $R$, we get ring epimorphism $R/I\to R/J$. When $A\subseteq B$ are subsets of $R$, there might not even be nontrivial homomorphism $R/A\to R/B$.
For example, let $I$ a nonzero, proper ideal and $A = I\cup\{a\}$, $a\not\in I$. To prove that $R/A$ is a ring, we need to prove that $I_A = \{ x\in R\mid x + A = A\}$ is an ideal.
First, let me show that $I_A \subseteq I$. Let $x\in I_A$ and assume that $x\not\in I$. For all $y\in I$, we have $x + y \not\in I$ and in particular, $x + y = a$, i.e. $x = a - y$. This is contradiction, since $I$ is nonzero. Thus, $x\in I$.
Now, let $x\in I_A$. Since $a\not\in I$ and $x\in I$, we have $x+a\not\in I$, so $x+a = a$. Thus, $x = 0$. We conclude that $I_A = \{0\}$. In particular, $R/A$ is a ring isomorphic to $R$. Also, this example gives negative answer to your conjecture.
To answer your Question, let $C = \{2n+1\mid n\in\mathbb Z\}\setminus\{1\}$. To determine $I_C$, first note that $I_C \subseteq 2\mathbb Z$. Assume that $x\in I_C$, $x\neq 0$. Then, $x = 2n$, for nonzero $n$. Define $y = 1 - 2n\in C$. Then we have $x + y = 1\not\in C$. We conclude that $I_C = \{0\}$ and in particular $\mathbb Z/C$ is a ring isomorphic to $\mathbb Z$.
Finally, let me give an example of set $A$ such that $R/A$ is not a ring. If $A$ is additive subgroup of $R$, then it is easy to check that $I_A = A$. Thus, by the above general argument, $R/A$ is a ring if and only if $A$ is an ideal of $R$. So find additive subgroup that is not an ideal, and you will get an example of $R/A$ not being a ring.
However, if $A$ is additive subgroup, $R/A$ is abelian group. This is because, analogous to rings, there is bijective correspondence between congruences of a group and its normal subgroups.
