Your "twisted commutator" is simply the normalizer of $\langle g\rangle$, because $g$ has finite order.
Recall that if $G$ is a group and $H$ is a subgroup, the normalizer of $H$ in $G$, $N_G(H)$, is the set
$$N_G(H) = \{g\in G\mid gHg^{-1}=H\}.$$
Proposition. For every subgroup $H$, $N_G(H)$ is a subgroup of $G$ that contains $H$. Moreover, $N_G(H)$ is the largest subgroup $M$ of $G$ such that $H\triangleleft M$.
Proof. It is clear that $e\in N_G(H)$, so $N_G(H)$ is nonempty. If $x,y\in N_G(H)$, then $(xy)H(xy)^{-1} x(yHy^{-1})x = xHx^{-1}=H$, so $xy\in N_G(H)$. Thus, $N_G(H)$ is closed under products. And if $x\in N_G(H)$, then $xHx^{-1}=H$ implies $H=x^{-1}Hx$, so $x^{-1}\in N_G(H)$, thus proving that $N_G(H)$ is a subgroup of $G$.
Now, $hHh^{-1}=H$ for every $h\in H$, so $H\leq N_G(H)$. Clearly, $H\triangleleft N_G(H)$. And if $H\triangleleft M$ for some $M\leq G$, then for every $m$ in $M$ we have $mHm^{-1}=H$, so $m\in N_G(H)$. Thus, $N_G(H)$ is the largest subgroup of $G$ containing $H$ where $H$ is normal. $\Box$
Corollary. If $g$ has finite order, then $T_g=N_G(\langle g\rangle)$. In particular, $T_g$ is a subgroup of $G$.
Proof. If $x\in T_g$, then $xgx^{-1}=g^k$ for some $k\in\mathbb{Z}$. Therefore, $x\langle g\rangle x^{-1} = \langle xgx^{-1}\rangle = \langle g^k\rangle\subseteq \langle g\rangle$. But because $g$ has finite order, and $|xgx^{-1}|=|g|$, we know that $\langle xgx^{-1}\rangle$ and $\langle g \rangle$ have the same finite order, so in fact $\langle xgx^{-1}\rangle = \langle g\rangle$. Thus, if $x\in T_g$, then $x\in N_G(\langle g\rangle)$.
Conversely, if $x\in N_G(\langle g\rangle)$, then $xgx^{-1}\in x\langle g\rangle x^{-1} = \langle g\rangle$, so $xgx^{-1}=g^k$ for some $k\in\mathbb{Z}$, so $x\in T_g$.
Thus, $T_g=N_G(\langle g\rangle)$, as claimed. $\Box$
If you want to prove this directly, it is easier to use the two-step subgroup test. Clearly, $e\in T_g$, and if $x,y\in T_g$, then $xy\in T_g$, since
$$(xy)g(xy)^{-1} = x(ygy^{-1})x^{-1} = xg^{k_2}x^{-1} = (xgx^{-1})^{k_2} = (g^{k_2})^{k_1} = g^{k_2k_1}.$$
Now, if $x\in T_g$, then $xgx^{-1}\in\langle g\rangle$. But because $\langle xgx^{-1}\rangle$ has the same number of finite elements as $\langle g\rangle$, it follows that $\langle xgx^{-1}\rangle = \langle g\rangle$. Thus, there exists $t\in\mathbb{Z}$ such that $g = (g^k)^t = g^{kt}$. Therefore,
$$x^{-1}gx = x^{-1}(g^{kt})x = (x^{-1}g^kx)^t = g^t$$
(since $xgx^{-1}=g^k$ implies $x^{-1}g^kx = g$), hence $x^{-1}\in T_g$. Thus, $T_g$ is a subgroup of $G$.
The result does not necessarily hold if $g$ has infinite order. This is because the collection of all $x$ such that $x\langle g\rangle x^{-1} \subseteq\langle g\rangle$ need not be closed under inverses when $g$ is infinite. An example can be extracted from this answer.