Consider a Lie group $G$ and let it's Lie algebra be $\mathfrak{g}$. Let the exponential map be denoted by $\exp: \mathfrak{g} \to G$.
Given any $g \in G$, does there exist an open set $O \subset \mathfrak{g}$ with $g \in \exp(O)$ such that the exponential map restricted to $O$ is a diffeomorphism?
Note: I am not conversant with Riemannian geometry.
Not in general.
Let $G = S^3$, thought of as the unit length quaternions. Let $g = -1$. I claim that for any $x\in \exp^{-1}(g)$, and for any open neighborhood $O$ of $x$, that $\exp|_O:O\rightarrow G$ fails to be injective. In particular, this restricted map cannot be a diffeomoprhism onto its image.
Note that $\mathfrak{g}$ can be identified with the imaginary quaternions (where the Lie bracket is the commutator). If we declare the basis $\{i,j,k\}$ of $\mathfrak{g}$ to be orthonormal, then the adjoint action of $G$ is isometric. Also note that the adjoint action is simply the usual $SO(3)$ action on $\mathfrak{g}$ (a three dimensional vector space), so is transitive on any sphere centered at the origin.
Now, suppose $x\in \exp^{-1}(g)$. Then, for any $h\in G$, we have $\exp(Ad_h x) = h\exp(x)h^{-1} = h(-1)h^{-1} = -1$, so $Ad_h x\in \exp^{-1}(g)$ as well. Since the adjoint action is transitive on spheres, it follows that if $x\in \exp^{-1}(g)$, then all purely imaginary quaternions $y$ with $|x| = |y|$ are also in $\exp^{-1}(g)$. In other words, $\exp^{-1}(g)$ consists of a union of spheres centered at the origin.
Restricting $x$ to be purely imaginary and complex, the group exponential is just the usual complex exponential map. In particular, $\exp^{-1}(g)\cap \operatorname{Im}(\mathbb{C}) = \{ n\pi: n\text{is an odd integer}\}$.
Thus, $\exp^{-1}(g)$ consists of an infinite union of spheres centered at the origin, where there sphere each have radius an odd multiple of $\pi$.
Now, suppose $x\in \exp^{-1}(g)$ and let $O$ be any open neighborhood of $x$. Since $x\in\exp^{-1}(g)$ it is one some sphere $S$ whose radious is an odd multiple of $\pi$. But then $O\cap S$ is an infinite set. In particular, for any $y\in O\cap S$, $\exp(y) = g$, so $\exp|_O$ is not injective.
