I will answer to my own post. I posted these questions because I think that these questions and their answers will probably interest the boys reading MSE. The answers being the fruit of my reflexions, they may contain errors in them. If anyone has another interesting approach to the problem, feel free to post it.
============================================================
Convergence
These series are all convergent, for all $x \in \mathcal{R}$.
Case k = 1 :
The convergence of the series $f_1(x)$ towards $e^x$ is well-known, and some information about it can be found here.
Case k = 2 :
The sums over the $0 \text{ mod }2 $, $1 \text{ mod }2 $ terms are all convergent for all $x \geq 0$, because
\begin{align}
\sum_{n=0}^\infty \frac{x^{2n}}{(2n)!!} &= \sum_{n = 0}^\infty \frac{x^{2n}}{2^n n!} = e^{\frac{x^2}{2}}\\
\sum_{n=0}^\infty \frac{x^{2n+1}}{(2n+1)!!} &\leq \sum_{n = 0}^\infty \frac{x^{2n+1}}{(2n)!!} = xe^{\frac{x^2}{2}}
\end{align}
The two sums both converges when $x < 0$, because if a serie converges absolutely, then an alternate serie will be convergent. Since the two sums are convergent for all $x \in \mathcal{R}$, then $f_2(x)$ is convergent for all $x \in \mathcal{R}$.
Case k = 3
The sums over the $0 \text{ mod }3 $, $1 \text{ mod }3 $, $2 \text{ mod }3 $ terms are all convergent for all $x \geq 0$, because
\begin{align}
\sum_{n=0}^\infty \frac{x^{3n}}{(3n)!!!} &= \sum_{n = 0}^\infty \frac{x^{3n}}{3^n n!} = e^{\frac{x^3}{3}} \\
\sum_{n=0}^\infty \frac{x^{3n+1}}{(3n+1)!!} &\leq \sum_{n = 0}^\infty \frac{x^{3n+1}}{(3n)!!} = xe^{\frac{x^3}{3}} \\
\sum_{n=0}^\infty \frac{x^{3n+2}}{(3n+2)!!} &\leq \sum_{n = 0}^\infty \frac{x^{3n+2}}{(3n)!!} = x^2e^{\frac{x^3}{3}}
\end{align}
The three sums all converges when $x < 0$, because if a serie converges absolutely, then an alternate serie will be convergent. Since the three sums are convergent for all $x \in \mathcal{R}$, then $f_3(x)$ is convergent for all $x \in \mathcal{R}$.
General case
The same reasoning can be applied to the general case. It is not written here for simplicity.
=========================================================================
Closed form
The series can be written in a closed form as a function of the incomplete gamma function. An explicit formula can be found here (though only in the case $x = 1$).
I will here explain a method to convert these series into differential equations, and differential equations into integrals, which can then be evaluated into a closed form with the incomplete gamma function.
Series to differential equations
Let's write the series as,
\begin{align}
f_1(x) &= 1 + \sum_{n=1}^\infty \frac{x^n}{n!} \\
f_2(x) &= 1 + x + \sum_{n=2}^\infty \frac{x^n}{n!!} \\
f_3(x) &= 1 + x + \frac{x^2}{2} + \sum_{n=3}^\infty \frac{x^n}{n!!!} \\
f_4(x) &= 1 + x + \frac{x^2}{2} + \frac{x^3}{3} + \sum_{n=4}^\infty \frac{x^n}{n!!!!} \\
f_5(x) &= 1 + x + \frac{x^2}{2} + \frac{x^3}{3} + \frac{x^4}{4} + \sum_{n=5}^\infty \frac{x^n}{n!!!!!} \\
\vdots
\end{align}
By differentiating these equations, we obtain,
\begin{align}
f_1'(x) &= f_1(x) \\
f_2'(x) &= 1 + xf_2(x) \\
f_3'(x) &= 1 + x + x^2 f_3(x) \\
f_4'(x) &= 1 + x + x^2 + x^3 f_4(x) \\
f_5'(x) &= 1 + x + x^2 + x^3 + x^4 f_5(x) \\
\vdots
\end{align}
Differential equations to integral
We apply the classical method to solve a first order linear differential equation.
By multiplying the equation by the well-chosen exponential, we have,
\begin{align}
f_1'(x)e^{-x} - e^{-x}f_1(x) &= 0 \\
f_2'(x)e^{-\frac{x^2}{2}} - x e^{-\frac{x^2}{2}} f_2(x) &= e^{-\frac{x^2}{2}} \\
f_3'(x)e^{-\frac{x^3}{3}} - x^2e^{-\frac{x^3}{3}} f_3(x) &= e^{-\frac{x^3}{3}}(1 + x) \\
f_4'(x)e^{-\frac{x^4}{4}} - x^3e^{-\frac{x^4}{4}} f_4(x) &= e^{-\frac{x^4}{4}}(1 + x + x^2) \\
f_5'(x)e^{-\frac{x^5}{5}} - x^4e^{-\frac{x^5}{5}} f_5(x) &= e^{-\frac{x^5}{5}}(1 + x + x^2 + x^3) \\
\vdots
\end{align}
By integrating from 0 to $x$, and using that $f_i(0) = 1$ for all $i$, we obtain,
\begin{align}
f_1(x) &= e^x \\
f_2(x) &= e^{\frac{x^2}{2}} \Big[1 + \int_0^{x}e^{-\frac{x^2}{2}}dx \Big] \\
f_3(x) &= e^{\frac{x^3}{3}} \Big[1 + \int_0^{x}e^{-\frac{x^3}{3}}(1+x)dx \Big] \\
f_4(x) &= e^{\frac{x^4}{4}} \Big[1 + \int_0^{x}e^{-\frac{x^4}{4}}(1+x+x^2)dx \Big] \\
f_5(x) &= e^{\frac{x^5}{5}} \Big[1 + \int_0^{x}e^{-\frac{x^5}{5}}(1+x+x^2+x^3)dx \Big] \\
\vdots
\end{align}
Integrals to closed form
The integrals can then be evaluated in term of the incomplete gamma function by first doing the variable change $u = \frac{x^n}{n}$ in the integral associated to $f_n(x)$. Then after simplification, one should find,
$$f_n(x) = e^{\frac{x^n}{n}}\Big[1 + \sum_{k=1}^{n-1} \gamma\Big(\frac{k}{n}, \frac{x^n}{n}\Big)n^{\frac{k}{n}-1}\Big]$$
Note that $f_2(x)$ can be written in terms of the erf function in the following way,
$$ f_2(x) = e^{\frac{x^2}{2}}\Big[1 + \sqrt{\frac{\pi}{2}}\text{erf}\Big(\frac{x}{\sqrt{2}}\Big)\Big] $$