SVP and GapSVP
For uniformly distributed $a_i \in \mathbb{Z}^n_q$ we are tasked with finding $s \in \mathbb{Z}^n_q$. This can be seen as the search form SVP, but by stating that the hardness derives from GapSVP the claim then becomes a decision form. GapSVP is also shown to reduce to GapCVP, and SVP is not harder than CVP (source). It has also been shown that SVP reduces to GapSVP (source)
Intuitive Sense
For the noisy linear $b_i$ w.r.t. $(a_i, \quad b_i = \langle s, a_i \rangle + e_i)$ can you show that $s$ is the shortest non-zero vector when you only know $(a_i, b_i)$ and $a$ is uniformly random? To put it another way, I task you with finding the shortest non-zero vector $s$, but to each vector I add slight noise using a Guassian distribution. You will not be able to determine if $s$ is indeed this short vector, since the noise used can lead to an $s$ modulo an arbitrary number of decimal places.
LWE and GapSVP
The immediate connection between LWE and GapSVP is that in both you are tasked with finding the shortest non-zero vector in the presence of a uniformly distributed noise. In both, LWE and GapSVP have the difficulty of finding the secret $s$ which is as hard as distinguishing pairs such that the shortest non-zero vector has at most length $d$ from the pairs. If I can't find the shortest non-zero vector, how can I decide if a pair of which the shortest vector is an element is at most length $d$? Since I can't find $s$, I can't show that $s$ has a length greater than $f(n)*d$, which is the GapSVP problem.
GapSVP and Lattices
In the work by Regev, cited in the question (source), Regev covers basic definitions of lattices and lattice problems on page 8. He defines the lattice in $\mathbb{R}^n$ with respect to the set of all integer combinations of $n$ linearly independent vectors. The set of $n$ linearly independent vectors is the basis of the lattice.
In LWE a number field $K$ is added to an $n$-dimensional vector space over $\mathbb{Q}$ which is used to introduce a cyclotomic polynomial. These properties allow for the canonical embedding described by Regev in section 2.3.2. One of the final requirements to establish the lattice geometry is to select a relevant norm. Regev uses the norm of $l_\infty$. This norm determines the expansion that occurs between elements under multiplication.
LWE uses a decision problem involving samples of LWE and a uniform distribution on $\mathbb{Z}^{n}_{q} \times \mathbb{Z}_{q}$ (Section 3, page 12).
Applied Lattices
Returning to the short basis generator and vector pairs $(a_i, \quad b_i = \langle s, a_i \rangle + e_i)$ and $e_i \ll q$, I use $s$ to encrypt text which produces $e$. Since $a_i$ is uniformly random, I take secret key $s$ as pair of $a$ and add it to the encrypted $e$. I now have $b_i$ which I pair with $a_i$. Now, any attempts to attack the scheme require finding $b_i$ compared to $a_i$, which is the SVP problem. To separate $b_i$ from $a_i$ after finding $b_i$, is the GapSVP problem. In the image below, the red vector represents the shortest non-zero vector, and the blue vectors are linearly independent of the red vectors and have basis $b_i$. The red vector is private key $s$, and the blue vectors represent the public key of $m$ samples $(\mathbf{a}_i, b_i)$ from LWE distribution with secret $s$, mod $q$, and error parameter $\alpha$. In the image you can easily see that neighboring points of the $s$ vector are approximate equidistant to the basis vector $b_i$, granting the security of private key $s$. To place final context on this answer, you have to determine if $(b_1,b_2)$ are at most distance $d$ from $s$.
