The following would appear to be true.
For real $0 < \sigma < 1,$ we seem to have a very satisfying sum minus integral limit, $$ \zeta(\sigma) \; \; = \; \; \lim_{n \rightarrow \infty} \; \; \left( \sum_{k=1}^n \frac{1}{k^\sigma} \right) - \frac{n^{1-\sigma}}{1-\sigma} \; \; . $$
A little more detail in these examples:
$$ \lim_{n \rightarrow \infty} \; \; \left( \sum_{k=1}^n \frac{1}{\sqrt k} \right) - 2 \sqrt n - \frac{1}{2 \sqrt n} + \frac{1}{24 n^{3/2}} \; \; = \; \; \zeta \left(\frac{1}{2}\right) \approx -1.460354508809586 $$
$$ \lim_{n \rightarrow \infty} \; \; \left( \sum_{k=1}^n \frac{1}{\sqrt[3] k} \right) - \frac{3 n^{2/3}}{2} - \frac{1}{2 \sqrt[3] n} + \frac{1}{36 n^{4/3}} \; \; = \; \; \zeta \left(\frac{1}{3}\right) \approx -0.973360248350782 $$
So, are the above items really really true (rather than wishful thinking), and, if so, given $0 < \sigma < 1,$ what real numbers $A = A(\sigma), B = B(\sigma),$ make the short asymptotic expansion below correct?
$$ \left( \sum_{k=1}^n \frac{1}{k^\sigma} \right) - \frac{n^{1-\sigma}}{1-\sigma} - A n^{-\sigma} + B n^{-1-\sigma} \; \; = \; \; \zeta(\sigma) \; \; + \; \; O(n^{-2-\sigma}) \; \; ? $$
Here is an estimate of $\zeta\left( \frac{1}{5} \right)$ using Daniel's expansion as far as the $3+\sigma$ term, ignoring $5+\sigma$: $$ \zeta(\sigma) =\left( \sum_{k=1}^n \frac{1}{k^\sigma} \right) - \frac{n^{1-\sigma}}{1-\sigma} - \frac{1}{2 n^\sigma} + \frac{\sigma}{12 n^{1 + \sigma}} - \frac{\sigma (1 + \sigma)(2 + \sigma)}{720 n^{3+\sigma}} + O \left( \frac{1}{n^{5 + \sigma}} \right)$$ I think it will let me fit 33 lines in a "code" window without introducing a scroll bar.
1 1 -0.7340666666666666
2 1.870550563296124 -0.7339263390330826
3 2.673292125056355 -0.7339216399387463
4 3.431150408311554 -0.7339210905379737
5 4.155930071989249 -0.7339209776636304
6 4.854757190760828 -0.7339209455319796
7 5.532368104161309 -0.7339209342063361
8 6.192122059547756 -0.7339209295629763
9 6.836516074525011 -0.7339209274322004
10 7.467473419005204 -0.7339209263652269
11 8.086517339689049 -0.7339209257924113
12 8.694881681582254 -0.7339209254668952
13 9.293584537123753 -0.7339209252729478
14 9.883479099480143 -0.7339209251526775
15 10.46528985863283 -0.7339209250754953
16 11.03963903613135 -0.7339209250244763
17 11.60706632180293 -0.7339209249898626
18 12.16804389452603 -0.7339209249658375
19 12.72298804735438 -0.7339209249488197
20 13.27226831900744 -0.7339209249365428
21 13.81621476189632 -0.7339209249275397
22 14.35512379575268 -0.7339209249208374
23 14.8892629725969 -0.7339209249157863
24 15.41887489312131 -0.7339209249119263
25 15.94418045400206 -0.733920924908948
26 16.46538156214071 -0.7339209249066238
27 16.98266342011249 -0.7339209249047942
28 17.49619646365717 -0.7339209249033369
29 18.00613801451264 -0.7339209249021712
30 18.51263369862476 -0.7339209249012291
31 19.01581866962328 -0.733920924900462
32 19.51581866962328 -0.7339209248998368
33 20.0127509533112 -0.7339209248993227