Indeed your idea to apply the orbit-stabilizer makes sense. But you seem to be thinking about the orbit and the stabilizer backwards. We say $SO(4)/I\simeq \mathbb{RP}^3$ as $SO(4)$-spaces, where $I$ is the stabilizer subgroup of $SO(4)$. That is, we quotient by the stabilizer to get the space, we don't quotient to get a subgroup. (That said, $\mathbb{RP}^3$ is diffeomorphic to $SO(3)$, but we can't use the orbit-stabilizer with these groups to conclude that very easily.)
But, your claim about the stabilizer is incorrect. Say $x=(0,0,0,1)^T$. If you write out the condition $Ax=x$ explicitly, you will see $A$ must be block diagonal with a $3\times3$ block in the upper left and a $1\times 1$ block in the lower right. But $A$ is an orthogonal matrix, which forces the $3\times 3$ and $1\times 1$ blocks to be orthogonal matrices as well. However, $\det A=+1$, so either both blocks have determinant $-1$ (to cancel out) or both blocks have determinant $+1$.
In other words, the stabilizer is $S(O(3)\times O(1))$ (this is the notation for it). It contains $SO(3)\times SO(1)$ as an index $2$ subgroup, with complement $[O(3)\setminus SO(3)]\times\{-1\}$ as its other coset. This group isn't the same set as $O(3)\times SO(1)$ (another block-diagonal subgroup of $O(4)$) however it is isomorphic to $O(3)$, with the isomorphism $O(3)\to S(O(3)\times O(1))$ given by $X\mapsto (X,\det X)$ (that is, we can turn a $3\times 3$ orthogonal matrix $X$ into a $4\times 4$ special orthogonal matrix that stabilizes $x$ by putting $X$ in the upper left corner and $\det X$ in the lower right corner).
The easiest way to see $SO(3)\simeq\mathbb{RP}^3$ in my opinion is with quaternions. The unit quaternions are $S^3$, the three-sphere, and there is a map $S^3\to SO(3)$ which is onto with kernel $S^0=\{\pm1\}$ (are you familiar with this map?). Then that means $S^3/S^0=SO(3)$, but $S^3/S^0$ is just $\mathbb{RP}^3$!
