Let us apply the method of apt1002 to the OP's problem.
We use the definitions and notation of this question.
Let $D$ be a negative non-square integer such that $D \equiv 0$ (mod $4$) or $D \equiv 1$ (mod $4$). We denote the set of positive definite primitive binary quadratic forms of discriminant $D$ by $\mathfrak{F}^+_0(D)$.
For notational convenience, we denote an element $ax^2 + bxy + cy^2$ of $\mathfrak{F}^+_0(D)$ by $(a, b, c)$.
We define a map $\phi\colon \mathfrak{F}^+_0(D) \rightarrow \mathcal{H}(D)$ by $\phi((a,b,c)) = (-b + \sqrt D)/2a$.
By the question, $\phi$ is a bijection and $\phi(f^\sigma) = \sigma^{-1}\phi(f)$ for $f \in \mathfrak{F}^+_0(D),\ \sigma \in \Gamma$.
We denote $f^{\sigma^{-1}}$ by $\sigma f$.
Thus $\Gamma$ acts on $\mathfrak{F}^+_0(D)$ from left.
Then $\phi$ is an isomorphism between the $\Gamma$-sets.
Therefore it suffices to solve the corresponding problem in $\mathfrak{F}^+_0(D)$.
We denote $\phi((a,b,c))$ by $\phi(a,b,c)$ by abuse of notation.
Let $S = \left( \begin{array}{ccc}
0 & -1 \\
1 & 0 \end{array} \right)$,
$T = \left( \begin{array}{ccc}
1 & 1 \\
0 & 1 \end{array} \right)$.
It is easy to see that
$S\phi(a,b,c) = \phi(c, -b, a)$,
$T^{-n}\phi(a,b,c) = \phi(a, 2an + b, an^2 + bn + c)$ for $n \in \mathbb{Z}$.
Let $F = \{z \in \mathcal{H}\ |\ -1/2 \le Re(z) \lt 1/2, |z| \gt 1$ or $|z| = 1$ and $Re(z) \le 0\}$.
Proposition 3
Let $(a,b,c) \in \mathfrak{F}^+_0(D)$.
The necessary and sufficient conditions for $\phi(a,b,c) \in F$ are as follows.
$|b| \le a \le c$.
If $|b| = a$ or $a = c$, then $b \ge 0$.
Proof:
Since $Re(\phi(a,b,c)) = -b/2a, -1/2 \le -b/2a \lt 1/2$.
Hence $-a \le -b \lt a$.
Hence $|b| \le a$.
If $|b| = a$, $a = b$.
Since $|\phi(a,b,c)|^2 = (b^2 - D)/4a^2 = 4ac/4a^2 = c/a \ge 1, a \le c$.
If $|\phi(a,b,c)| = 1$, i.e. $a = c$, then $-1/2 \le -b/2a \le 0$.
Hence $-a \le -b \le 0$.
Hence $a \ge b \ge 0$.
QED
We denote the set of $(a, b, c)$ which satisfies the conditions of Proposition 3 by $\mathcal{F}(D)$.
$\mathcal{F}(D)$ is a finite set as shown in my answer to this question.
Let $(a,b,c) \in \mathfrak{F}^+_0(D)$.
We consider the following algorithm.
Algorithm 2
If $-a \le -b \lt a$, go to 2.
If not, set $(a,b,c) = (a, 2an + b, an^2 + bn + c)$, where $n = \lfloor -b/2a +1/2 \rfloor$.
If $a \lt c$, then stop.
If $a = c$, go to 3.
If $a \gt c$, set $(a,b,c) = (c, -b, a)$ and go to 1.
If $b \ge 0$, stop.
Otherwise set $(a,b,c) = (c, -b, a)$ and stop.
Proposition 4
The algorithm terminates in finite steps.
Moreover when it terminates, $(a, b, c) \in \mathcal{F}(D)$.
Proof:
This follows immediately from Proposition 1.
However, we will prove it directly.
If $|b| \gt a$ or $-b = a$, by executing step 1, $b$ satisfies $-a \le -b \lt a$, hence $|b| \le a$.Hence $|b|$ decreases at least by $1$.
If $-b = a$, by executing step 1, $b = a$.
Hence |b| does not change.
In step 2, the algorithm terminates or $|b|$ does not change and $a$ decreases and it goes to step 1.
Hence the algorithm must terminate in finite steps.
QED
Proposition 5
(1) Any two distinct forms of $\mathcal{F}(D)$ are not equivalent under the action of $\Gamma$.
(2) Let $\Gamma_f = \{\sigma \in \Gamma\ |\ \sigma f = f\}$ for $f \in \mathcal{F}(D)$.
Let $S = \left( \begin{array}{ccc}
0 & -1 \\
1 & 0 \end{array} \right)$,
$T = \left( \begin{array}{ccc}
1 & 1 \\
0 & 1 \end{array} \right)$.
If $D \ne -4, -3$, then $\Gamma_f = \{\pm1\}$ for every $f \in \mathcal{F}(D)$.
Let $g = x^2 + y^2$.
Let $h = x^2 + xy + y^2$.
$\mathcal{F}(-4) = \{g\}$.
$\mathcal{F}(-3) = \{h\}$.
$\Gamma_g = \{\pm 1, \pm S\}$.
$\Gamma_h = \{\pm 1, \pm ST, \pm (ST)^2\}$.
Proof:
This follows immediately from Proposition 2 except $\mathcal{F}(-4) = \{g\}$, $\mathcal{F}(-3) = \{h\}$.But this is easy to see by the method of my answer to this question.