Since $\zeta(s)=\frac{\eta(s)}{1-2^{1-s}}$ and $\eta(0)=\frac{1}{2}$, we have $\zeta(0)=-\frac{1}{2}$. From:
$$ \zeta'(s) = \zeta(s)\cdot\frac{d}{ds}\log\zeta(s) = \zeta(s)\left(\frac{\eta'(s)}{\eta(s)}-\frac{2^{1-s}\log(2)}{1-2^{1-s}}\right)\tag{1}$$we get $\zeta'(0)=-\eta'(0)-\log(2)$, and the problem boils down to showing:
$$ \eta'(0) = \frac{1}{2}\log\frac{\pi}{2}\tag{2} $$
On the other hand, $\eta'(0)$ is directly related with the Wallis product through its series definition:
$$ \eta'(0)=\frac{1}{2}\sum_{n\geq 1}(-1)^{n+1}\log\left(\frac{n+1}{n}\right)=\frac{1}{2}\log\prod_{n\geq 1}\left(1-\frac{1}{4n^2}\right)^{-1}\tag{3}$$
but:
$$ \prod_{n=1}^{N}\left(1-\frac{1}{4n^2}\right)^{-1} = \frac{\Gamma\left(\frac{1}{2}\right)^2 \Gamma(N+1)^2}{2\,\Gamma\left(N+\frac{1}{2}\right)\Gamma\left(N+\frac{3}{2}\right)}\tag{4}$$
so by Gautschi's inequality the limit of the RHS of $(4)$ as $N\to +\infty$ is exactly $\frac{1}{2}\,\Gamma\left(\frac{1}{2}\right)^2$, and through the substitution $x=\sqrt{z}$ we have:
$$ \int_{0}^{+\infty}e^{-x^2}\,dx = \frac{1}{2}\int_{0}^{+\infty}z^{-1/2}e^{-z}\,dx = \frac{1}{2}\,\Gamma\left(\frac{1}{2}\right).\tag{5}$$
So, a summary of the connection:
$$ \zeta'(0)\mapsto \eta'(0)\mapsto \text{Wallis product}\mapsto \Gamma\left(\frac{1}{2}\right)\mapsto \text{Gaussian integral}.$$
The arrows can be reversed as you like. We also have a detour, since the value of $(3)$ is gladly provided by the Weierstrass product for the sine function:
$$ \frac{\sin x}{x}=\prod_{n\geq 1}\left(1-\frac{x^2}{\pi^2 n^2}\right) \tag{6}$$
through an evaluation at $x=\frac{\pi}{2}$. No wonder, since $\Gamma(x)\,\Gamma(1-x)=\frac{\pi}{\sin(\pi x)}$.
$$\frac{\mathrm{d}}{\mathrm{d}z}\zeta(z)=\frac{\mathrm{d}}{\mathrm{d}z}\sum_{n=1}^\infty n^{-z}=\sum_{n=1}^\infty\frac{\mathrm{d}}{\mathrm{d}z}n^{-z}=\sum_{n=1}^\infty-n^{-z}\log n$$
Hence
$$\zeta'(0)=-\frac{1}{2}\log{2\pi}=-\sum_{n=1}^\infty\log n$$
Now
$$\prod_{n=1}^\infty n=\exp\left(\log\prod_{n=1}^\infty n\right)=\exp\left(\sum_{n=1}^\infty\log n\right)=\exp(-\zeta'(0))=\exp\left(\frac{1}{2}\log{2\pi}\right)=\sqrt{2\pi}$$
– user76284 Jul 31 '14 at 16:15