Let's consider the group extension $G$ s.t. $$1 \to A \to G \to B \to 1$$ where $A$ and $B$ are finite, non-abelian and simple groups. Let $C_G(A)$ be the centralizer of $A$. Since $A$ is normal, $C_G(A)$ is normal as well. Furthermore, the intersection of a subgroup of a group and its centralizer is just the center of the subgroup, and we know that non-abelian simple groups are centerless. So $A\cap C_G(A) = 1$ and $AC_G(A) = A\times C_G(A)$.
From here how can we conclude that $C_G(A)$ is isomorphic (as a group) to a normal subgroup of $B$? For one, $B$ might not be (isomorphic to) a subgroup of $G$! Second, I don't see why $C_G(A)$ should be normal in $B$. Moreover, $B$ might not even be a complement of $A$ unless $G$ is a split extension (cf. this).
I'm trying to understand the claim made in the second paragraph of this answer.