If $U$ is a unitary on some Hilbert space $H$,
$U:H \to H, \; UU^\dagger = U^\dagger U = I, \tag 1$
then for any $x \in H$,
$\langle x, x \rangle = \langle x, Ix \rangle = \langle x, U^\dagger U x \rangle = \langle U x, U x \rangle; \tag 2$
if in addition $\mu$ is an eigenvalue of $U$, so that
$Ux = \mu x, \; x \ne 0, \tag 3$
(2) may be continued as
$\langle x, x \rangle = \langle U x, U x \rangle = \langle \mu x, \mu x \rangle = \bar \mu \mu \langle x, x \rangle, \tag 4$
from which, since $x \ne 0$ and hence $\langle x, x \rangle \ne 0$,
$\bar \mu \mu = 1; \tag 5$
that is,
$\mu, \bar \mu \in \Bbb S^1. \tag 6$
We note that, from (1),
$UU^\dagger = U^\dagger U = I \Longrightarrow (U^\dagger)^\dagger U^\dagger = U^\dagger (U^\dagger)^\dagger = I; \tag 7$
that is, $U^\dagger$ is also unitary; thus we see that (6) binds for the eigenvalues of $U^\dagger$ as well as for those of $U$; furthermore, we see via (3) and (5) that
$U^\dagger x = \bar \mu \mu U^\dagger x = \bar \mu U^\dagger \mu x = \bar \mu U^\dagger U x = \bar \mu (U^\dagger U)x = \bar \mu I x; = \bar \mu x; \tag 8$
that is, $\bar \mu$ is an eigenvalue of $U^\dagger$ whenever $\mu$ is of $U$, and with the same eigenvector $x$; and, in addition, (7) yields
$U^\dagger U = U^\dagger (U^\dagger)^\dagger \Longrightarrow U (U^\dagger U) = U(U^\dagger (U^\dagger)^\dagger)$
$\Longrightarrow (UU^\dagger)U = (UU^\dagger)(U^\dagger)^\dagger$
$\Longrightarrow IU = I(U^\dagger)^\dagger \Longrightarrow U = (U^\dagger)^\dagger. \tag 9$
These properties (8)-(9) of unitaries $U$ are not directly necessary here, though they may shed some useful light on unitary operators in general.
Returning to (2) we see that it implies
$\Vert x \Vert ^2 = \langle x, x \rangle= \langle U x, U x \rangle = \Vert Ux \Vert^2, \tag{10}$
or
$\Vert x \Vert = \Vert Ux \Vert, \tag{11}$
from which
$\Vert x \Vert = \Vert Ux \Vert \le \Vert U \Vert \Vert x \Vert, \tag{12}$
and hence
$1 \le \Vert U \Vert; \tag{13}$
furthermore, we may rule out the case
$\Vert U \Vert > 1, \tag{14}$
since this entails the existence of some $x$ with
$\Vert x \Vert < \Vert Ux \Vert \le \Vert U \Vert \Vert x \Vert \tag{15}$
by virtue of the definition of $\Vert U \Vert$ as the infimum of all $0 < C \in \Bbb R$ such that
$\Vert Ux \Vert \le C\Vert x \Vert, \; \forall x \in H; \tag{16}$
in addition, since
$U^\dagger = U^{-1} \tag{17}$
is also unitary, we have
$\Vert U^\dagger \Vert = \Vert U^{-1} \Vert = 1 \tag{18}$
as well.
Granted, then, that $\Vert U \Vert = 1$, we argue as follows: suppose
$0 < \vert \lambda \vert < 1; \tag{19}$
then (18) yields
$\Vert \lambda U^{-1} \Vert = \vert \lambda \vert \Vert U^{-1} \Vert < 1; \tag{20}$
but it is well-known that $I - A$ is invertible for any operator $A$ with
$\Vert A \Vert < 1; \tag{21}$
this is a standard result which is a consequence of the convergence of the series
$(I - A)^{-1} = \displaystyle \sum_0^\infty A^j = I + A + A^2 + \ldots, \tag{22}$
which is majorized term-by-term by
$(I - \Vert A \Vert)^{-1} = \displaystyle \sum_0^\infty \Vert A \Vert^j = I + \Vert A \Vert + \Vert A \Vert^2 + \ldots; \tag{23}$
we thus see that
$\exists (I - \lambda U^{-1})^{-1} \Longrightarrow \exists \lambda (I - \lambda U^{-1})^{-1} = (\lambda^{-1}I - U^{-1})^{-1}. \tag{24}$
If, on the other hand,
$\vert \lambda \vert > 1, \tag{25}$
then
$\Vert \lambda^{-1} U \Vert = \vert \lambda^{-1} \vert \Vert U \Vert < 1, \tag{26}$
whence
$\exists (I - \lambda^{-1}U)^{-1} \Longrightarrow \exists U(I - \lambda^{-1}U)^{-1} = (U^{-1} - \lambda^{-1} I)^{-1} \tag{27}$
as well.
We thus see that for all cases
$0 < \vert \lambda \vert \ne 1 \tag{28}$
the inverse of the operator
$U^{-1} - \lambda^{-1} I \tag{29}$
exists on the Hilbert space $H$.