The problem is given by
$$\begin{align*}
\arg \min_{R} \quad & {\left\| M - R \right\|}_{F}^{2} \\
\text{subject to} \quad & R {R}^{T} = I
\end{align*}$$
The Lagrangian is given by:
$$ L \left( R, \Lambda \right) = {\left\| M - R \right\|}_{F}^{2} + \operatorname{tr} \left( {\Lambda}^{T} \left( R {R}^{T} - I \right) \right) $$
The Gradient of the Lagrangian is given by:
$$ \nabla_{R} L \left( R, \Lambda \right) = 2 \left( M - R \right) + {\Lambda}^{T} R + {\Lambda} R $$
Now, since $ \Lambda $ is the Lagrangian multiplier of symmetric matrix equality it must be symmetric which means:
$$ \nabla_{R} L \left( R, \Lambda \right) = 2 \left( M - R \right) + 2 {\Lambda} R \Rightarrow M = \left( I + \Lambda \right) R $$
Using the above to calculate $ M {M}^{T} $ yields:
$$ M {M}^{T} = \left( I + \Lambda \right) R {R}^{T} \left( I + \Lambda \right) = {\left( I + \Lambda \right)}^{2} $$
The last move since by constraint $ R {R}^{T} = I $.
Since the Left Hand and the Right Hand are PSD matrices one could write:
$$ \left( I + \Lambda \right) = {\left( M {M}^{T} \right)}^{ \frac{1}{2} } $$
Taking from above:
$$ M = \left( I + \Lambda \right) R \Rightarrow R = {\left( I + \Lambda \right)}^{-1} M = {\left( M {M}^{T} \right)}^{ -\frac{1}{2} } M $$