Let's say we start with a vector $u = [0, 0, 1]$ which represents the positive z-axis.
Now, let's say we arbitrarily transform this vector via a rotation $z' = R \begin{bmatrix}0 \\ 0\\ 1 \end{bmatrix}$.
There are arbitrarily many rotations that can lead to the same $z'$.
I want to calculate a new rotation from $R$, $R_{undo}$, which inverts $R$ except it does not invert any portion of the rotation corresponding to a planar rotation around $z'$ (e.g. $z'$ is the normal vector of the plane of rotation).
For example, let's say in Euler angles with order zyx, $R = [90, 90, 0]$. Then, using the same zyx convention, $R_{undo} = [0, -90, 0]$, not $[-90, -90, 0]$.
How do we calculate $R_{undo}$ in general, for an arbitrary axis u?