$\newcommand{\multichoose}[2]{{#1}^{[\!\underline{#2}\!]}}$
If you want to show
$$\lim_{s\to 1}(s-1)\zeta(s)=1\text{,}$$
here's a way that doesn't invoke other special functions, built on estimating the zeta sum by certain integrals.
Start with the rectangle rule for integration:
$$\int_{n}^{n+1}f(x)\mathrm{d}x\approx f(n+1)\text{.}$$
If derive the error term using integration by parts, we get
$$\int_{n}^{n+1}f(x)\mathrm{d}x=f(n+1)-\int_{n}^{n+1}(x-n)f'(x)\mathrm{d}x\text{.}$$
This is just as much a relation expressing the value of $f$ in terms of its integrals:
$$f(n+1) = \int_{n}^{n+1}f(x)\mathrm{d}x+\int_{n}^{n+1}(x-n)f'(x)\mathrm{d}x\text{.}$$
Summing over $n$, we get
$$\sum_{n=1}^{\infty}f(n)=\int_{1}^{\infty}f(x)\mathrm{d}x +f(1) +\int_{1}^{\infty}(x-\lfloor x \rfloor)f'(x)\mathrm{d}x\text{.}$$
Let $f(k)=k^{-s}$. Then
$$\sum_{n=1}^{\infty}\frac{1}{k^s}=\int_{1}^{\infty}\frac{\mathrm{d}x}{x^s} +1 -s\int_{1}^{\infty}\frac{(x-\lfloor x \rfloor)}{x^{s+1}}\mathrm{d}x\text{,}$$
i.e.,
$$\boxed{\zeta(s)=\frac{1}{s-1} +1 -s\int_{1}^{\infty}\frac{x-\lfloor x \rfloor}{x^{s+1}}\mathrm{d}x}$$
(DLMF 25.2.8). The right side of this equation is defined for all $\Re s>0$, $s\neq 1$.
We can push this method farther. Rewrite the integral as
$$\begin{split}
\int_{1}^{\infty}\frac{x-\lfloor x \rfloor}{x^{s+1}}\mathrm{d}x&=\int_{1}^{\infty}\frac{x-\lfloor x \rfloor-\tfrac{1}{2}+\tfrac{1}{2}}{x^{s+1}}\mathrm{d}x\\
&=\frac{1}{2}\int_{1}^{\infty}\frac{\mathrm{d}x}{x^{s+1}}+\int_{1}^{\infty}\frac{x-\lfloor x \rfloor-\tfrac{1}{2}}{x^{s+1}}\mathrm{d}x \\
&=\frac{1}{2s}+(s+1)\int_{1}^{\infty}\frac{b_2(x-\lfloor x\rfloor)}{x^{s+2}}\mathrm{d}x
\end{split}$$
where $b_2(u)=\tfrac{1}{2}(u^2-u)$. In these steps, we separated out the mean value of $x-\lfloor x \rfloor$ then integrated by parts. Therefore
$$\boxed{\zeta(s)=\frac{1}{s-1} +1 -\frac{1}{2} -s(s+1)\int_{1}^{\infty}\frac{b_2(x-\lfloor x\rfloor)}{x^{s+2}}\mathrm{d}x}\text{.}$$
The right side of this equation is defined for all $\Re s > -1$, $s\neq 1$.
Then $\lim_{s\to 1}(s-1)\zeta(s)=1$ and $\zeta(0)=-\tfrac{1}{2}$ follow by direct substitution into the boxed equations.
The two expressions above are special cases of
$$\zeta(s)=\frac{1}{s-1}+1+\sum_{k=0}^{n-1}\multichoose{s}{k}\frac{B_{k+1}}{k+1}-\multichoose{s}{n+1}\int_1^{\infty}\frac{(B_{n+1}(x-\lfloor x \rfloor) -B_{n+1})\mathrm{d}x}{x^{s+n+1}}\text{,}$$
valid for $\Re s > -n$, $s\neq 1$ (DLMF 25.2.10); here $\multichoose{s}{k}$ are the multiset coefficients, $B_k(u)$ are the Bernoulli polynomials, and $B_k$ are the Bernoulli numbers:
$$\begin{align}
(1-t)^{-s}&=\sum_{k=0}^{\infty}\multichoose{s}{k}t^k \\
\frac{t\mathrm{e}^{tu}}{\mathrm{e}^t-1}&=\sum_{k=0}^{\infty}B_k(u) \frac{t^k}{k!} \\
\frac{t}{\mathrm{e}^t-1}&=\sum_{k=0}^{\infty}B_k \frac{t^k}{k!}\text{.}
\end{align}$$