My book is Connections, Curvature, and Characteristic Classes by Loring W. Tu (I'll call this Volume 3), a sequel to both Differential Forms in Algebraic Topology by Loring W. Tu and Raoul Bott (Volume 2) and An Introduction to Manifolds by Loring W. Tu (Volume 1).
Here are Section 2.1 and Section 2.2.
About Proposition 2.3, I believe what happens is that we show $s$ is an immersion (or equivalently a submersion or a local diffeomorphism, by this and by "Most of the concepts introduced for a manifold extend word for word to a manifold with boundary...", from Volume 1 Section 22) and conclude that its inverse is smooth. I believe we can show $s$ is an immersion without deducing $s$ is bijective.
Question 1: Is this right? Please verify.
$||c'||: [a,b] \to (0,\infty)$ is a smooth map (See here) because $c$ is regular/an immersion.
$||c'||$ is continuous by (1).
$\dot s = ||c'||$ by fundamental theorem of calculus and (2) (See here for $\dot s$ vs $s'$).
Use $t$ denote the standard coordinate (Volume 1 Section 8.6) on $[a,b]$ but $t_0$ to denote a point in $[a,b]$. Then $\dot s(t_0) = ||c'(t_0)||$, for each $t_0 \in [a,b]$.
$s'(t_0)=\dot s(t_0) \frac{d}{dx}|_{s(t_0)}$, where $x$ is the standard coordinate (Volume 1 Section 8.6) on $[0,l]$, by Volume 1 Exercise 8.14, where $s'(t_0) := s_{*,t_0}[\frac{d}{dx}|_{s(t_0)}] \in T_{s(t_0)}[0,l]$ and where $s_{*,t_0}$ is $s_{*,t_0}: T_{t_0}[a,b] \to T_{s(t_0)}[0,l]$.
$s'(t_0)=||c'(t_0)|| \frac{d}{dx}|_{s(t_0)}$, by (4) and (5).
$s$ is an immersion if and only if $s_{*,t_0}$ is injective if and only if the kernel of $s_{*,t_0}$ consists only of $0_{t_0}$, the zero tangent vector of $T_{t_0}[a,b]$.
The kernel of $s_{*,t_0}$ consists only of $0_{t_0}$, the zero tangent vector of $T_{t_0}[a,b]$, as shown below.
Therefore, $s$ is an immersion by (7) and (8).
Proof of (8):
8.1. Suppose there exists $X_{t_0} \in T_{t_0}[a,b]$ such that $s_{*,t_0}[X_{t_0}] = 0_{s(t_0)}$, the zero tangent vector of $T_{s(t_0)}[0,l]$.
8.2. Such $X_{t_0}$ in (8.1) is $X_{t_0)} = a(t_0) \frac{d}{dt}|_{t_0}$ for some real number $a(t_0)$.
8.3. $0_{s(t_0)} = s_{*,t_0}[X_{t_0}] = s_{*,t_0}[a(t_0) \frac{d}{dt}|_{t_0}] = a(t_0) s_{*,t_0}[\frac{d}{dt}|_{t_0}] = a(t_0)s'(t_0) = a(t_0)||c'(t_0)|| \frac{d}{dx}|_{s(t_0)}$, by (6), (8.1) and (8.2).
8.4 $||c'(t_0)|| = 0$ if and only if $c'(t_0) = 0_{c(t_0)}$, the zero tangent vector in $T_{c(t_0)}M$.
8.5. $c'(u_0)$ is not $0_{c(u_0)}$ for all $u_0 \in [a,b]$, which is the - definition of $c$ regular/an immersion.
8.6. $||c'(u_0)|| \ne 0$ for all $u_0 \in [a,b]$ by (8.4) and (8.5).
8.7. $0_{s(t_0)} = a(t_0) \frac{d}{dx}|_{s(t_0)}$ by (8.3) and (8.6).
8.8. $a(t_0) = 0$ by (8.7) and by definition of $0_{s(t_0)} := 0 \frac{d}{dx}|_{s(t_0)}$
8.9. $X_{t_0} = 0 \frac{d}{dt}|_{t_0} = 0_{t_0}$ by (8.1), (8.8) and by definition of $0_{t_0} := 0 \frac{d}{dt}|_{t_0}$.
Question 2: Do I use bijectiveness of s anywhere above?