One typical tool for this sort of thing is a Householder reflection (also, see this).
Let $e=(1,...,1)$.
We want to rotate ${1 \over \sqrt{n}}e$ to $e_n = (0,...,0,1)$ in the two dimensional subspace spanned by these vectors (note that both have the same length).
We can do this using a Householder reflection that reflects points across a hyperplane that 'reflects' ${1 \over \sqrt{n}}e$ to $e_n$.
Let $v={1 \over \sqrt{n}}e-e_n$, and $P=I-{2 \over v^T v} v v^T$. Check that $PP^T = I$ (hence an improper rotation, which preserves distances) and $P {1 \over \sqrt{n}}e = e_n$.
Check that $P$ maps the hyperplanes as required.
Alternative, proper rotation based answer:
Here is a derivation of a proper rotation that does the job, it is essentially a Givens rotation.
Use Gram Schmidt on $v_1={1 \over \sqrt{n}}e, e_n$ to get $v_2 = {1 \over \sqrt{1-{1 \over n}}} (e_n - (v_1^T e_n ) v_1)$, and note that
$e_n = {1 \over \sqrt{n}} v_1 + \sqrt{1-{1 \over n}} v_2 $. Let $R=\begin{bmatrix} {1 \over \sqrt{n}} & -\sqrt{1-{1 \over n}} \\ \sqrt{1-{1 \over n}} & {1 \over \sqrt{n}} \end{bmatrix}$, $V= \begin{bmatrix} v_1 & v_2\end{bmatrix}$ and define
$P = I - V V^T + V R V^T$, and check that $P^TP = I$, $Pv_1 = e_n$ and $\det P = \det R = +1$ (to see the latter, consider an orthonormal basis of the form $v_1, v_2, ...$).