Question: "I am sorry if this is an extraordinarily basic question, but I find I am just going about using the definition without actually really knowing what or why I am doing it."
Answer: This is a much asked question by students: To "mod out" by the square of a maximal ideal corresponds to "taking derivatives".
Example: If $A:=k[x,y]$ and $I:=(x-a,y-b)$ a maximal ideal with $p:=(a,b)\in k^2$, and $k$ a field, there is a canonical projection map
$$T^1: A \rightarrow A/I^2.$$
Any polynomial $f(x,y)\in A$ may be written as
$$f(x,y)=f(a,b)+ \partial_x(f)(p)(x-a)+\partial_y(f)(p)(y-b) +\cdots \in A.$$
You get a map
$$D: A \rightarrow I/I^2$$
defined by
$$D(f):=T^1(f)-f(p)=\partial_x(f)(p)(x-a)+\partial_y(f)(p)(y-b) \in k\{(x-a),(y-b)\}.$$
Hence the map $D$ takes the derivative of $f(x,y)$ at $p=(a,b)$.
Note: You find an elementary and more detailed answer to your question below. You also find a discussion of the tangent cone that may provide intuition. It explains why it is natural to define the tangent space as the dual of the cotangent space. Note that if $k \rightarrow A$ is any $k$-algebra and $\mathfrak{m} \subseteq A$ is a maximal ideal with residue field $k$, it follows the fiber
$$\Omega^1_{A/k}\otimes_A A/\mathfrak{m}\cong \mathfrak{m}/\mathfrak{m}^2$$
is the cotangent space. Hence
"the cotangent space is the fiber of the cotangent module"
$\Omega^1_{A/k}$ at $k$-rational points.
Example: If $J \subseteq A\otimes_k A$ is the ideal of the diagonal, you may define $\Omega^1_{A/k}:=J/J^2$. The map $D$ defined above is the composed map
$$d: A \rightarrow J/J^2 \rightarrow J/J^2\otimes_A A/I \cong I/I^2$$
where $d: A\rightarrow \Omega^1_{A/k}:=J/J^2$ is the universal derivation:
$$d(f):=1\otimes f - f\otimes 1.$$
When $A$ is a regular $k$-algebra of finite type it follows $\Omega^1_{A/k}$ is a finite rank projective $A$-module: It is the "cotangent bundle" of $A/k$.
If $A_{\mathfrak{m}}$ is the local ring of $A$ at a maximal ideal $\mathfrak{m}$, there is an isomorphism of $A/\mathfrak{m}$-vector spaces
$$\mathfrak{m}_{\mathfrak{m}}/\mathfrak{m}_{\mathfrak{m}}^2 \cong \mathfrak{m}/\mathfrak{m}^2,$$
hence the cotangent space can be defined in terms of the local ring of $A$ at $\mathfrak{m}$. The local ring is intrinsic and does not depend on a "choice of coordinates" and one wants to define the cotangent and tangent space independently of choice of coordinates. If you choose a closed embedding
$$\phi: Spec(A) \subseteq \mathbb{A}^n$$
into affine $n$-space (equivalently: You choose a set of coordinates of $A$), you may define the "embedded tangent space $E(x)$ of $Spec(A)$ at $x$" using the embedding. Then you must prove that $E(x)$ is independent of choice of embedding $\phi$. When using the local ring, you avoid this problem. See the attached links for more information:
$aX + bY$ is an element of $M^2$ if and only if the line $aX + bY = 0$ is tangent to $W$ at $(0, 0)$
About the definition of tangent space and tangent cone.
Dimension of the tangent space in algebraic geometry
I guess what I am still confused about is how this more sophisticated notion of "tangent space" corresponds to the naïve undergraduate multivariable calculus notion of "tangent space". There, a tangent space to, say, a surface in $\mathbb{R}^3$ is a plane that touches the surface at one point. Can we view the space of derivations as being an actual geometric plane (or linear space) somehow lying tangent to a variety?
– Luke Jul 18 '21 at 03:29