I could be wrong about this, but I don't think the second formula is a good definition, pedagogically, because in order to motivate it you need to already have $\zeta(s)$ defined for $\mathrm{Re}(s)<1$ in the first place.
Here is the canonical story of how $\zeta(s)$ goes:
Let's start with $\zeta(s)=\sum_{n=1}^\infty \frac{1}{n^s}$. This is convergent for $\mathrm{Re}(s)>1$ and divergent otherwise. However, the Dirichlet eta function $\eta(s)=\sum_{n=1}^\infty\frac{(-1)^{n-1}}{n^s}$ converges for $\mathrm{Re}(s)>0$ and it is not hard to see that we can write $\eta(s)=(1-2^{1-s})\zeta(s)$ and we can use this to analytically continue $\zeta(s)$ so it is defined also for values $\mathrm{Re}(s)>0$ (except $s=1$, where it has a pole).
If you're unfamiliar with analytic continuation, consider the geometric series $(1-z)^{-1}=\sum_{n=0}^\infty z^n$ for a complex variable $z$. The function $(1-z)^{-1}$ is defined for all $z\ne1$ but the series only converges hence is only defined for $|z|<1$. So the function defined by the series can be "continued" beyond its original domain to a function $(1-z)^{-1}$ defined on all of $\mathbb{C}\setminus\{1\}$.
There is a really cool reason we ought to multiply $\zeta(s)$ by another function to "complete" it, which gives the function $\xi(s)=\binom{s}{2}\pi^{-s/2}\Gamma(s/2)\zeta(s)$. Interchanging summation and integration entailed respectively in defining the zeta and gamma function in this formula shows $\xi(s)$ is a Mellin transform of a theta function. The theta function has a functional equation deriving from the Poisson summation formula, which in turn implies the functional equation $\xi(s)=\xi(1-s)$, and we can use this to extend the domain of $\zeta(s)$ to all $s\ne 1$.
[Here is the cool reason BTW. The local factors of the Euler product factorization $\prod(1-p^{-s})^{-1}$ are in fact $p$-adic Mellin transforms of Gaussian functions (i.e. fixed points of the $p$-adic Fourier transform). Note the Mellin transform is actually the multiplicative version of the Fourier transform. The corresponding local factor for the real numbers is the Mellin transform of the usual Gaussian function, which gives us the stuff in front of $\zeta(s)$ in the definition of $\xi(s)$. This is a bumper sticker version of Tate's thesis.]
Now, we know that $\sum\frac{1}{n^s}$ only converges to $\zeta(s)$ for $\mathrm{Re}(s)>1$, so it seems like a reasonable question to ask how the partial sums grow (asymptotically) with $n$ otherwise. Note $H_{n,s}=\sum_{k=1}^n k^{-s}$ is called a generalized harmonic number. For $\mathrm{Re}(s)>0$ we can proceed by multiplying and dividing by $n^s$ - that is,
$$H_{n,s}=n^{1-s}\left(\frac{1}{n}\sum_{k=1}^n (k/n)^{-s}\right)\sim n^{1-s}\int_0^1 x^{-s}\,\mathrm{d}x=\frac{n^{1-s}}{1-s}. $$
In fact, the Euler-Maclaurin formula can be used to derive a full asymptotic series:
$$ H_{n,s}=\zeta(s)-\sum_{k=-1}^{\infty} \frac{B_{k+1}}{(k+1)!} (s)_k n^{-s-k}. $$
This series is divergent, but for a particular value of $s$ if we lop off all summands that $\to0$ with $n$ we get a finite sum. In particular, this shows your formula
$$ \zeta(s) = \lim_{n\to\infty} \left( H_{n,s}-\frac{n^{1-s}}{1-s} \right). $$
