Just like $y$ is a function of $x$, $dy$ is also a function of $x$, and is the best linear approximation of $y$ at some point, $p$. However, whereas the tangent line passes through $(p,f(p))$, the function $dy(x)$ is shifted so that $(p,f(p)$ becomes the origin, so $dy(0)=0$. This is done so that $dy$ is a linear function in the linear algebra sense, i.e, $dy(c_1x_1+c_2x_2)=c_1dy(x_1)+c_2dy(x_2)$.
Since the tangent line is given by $f'(p)(x-p)+f(p)$, this means that $$dy(x)=f'(p)x,$$ or put in a more suggestive form, $dy(x-p)=f'(p)(x-p)$.
What about $dx$? Well, $x$ itself is a function of $x$ ($x=1\cdot x$), so $dx$ is the best linear approximation of that function at $p$, shifted so $(p,p)$ becomes the origin. But that function is itself linear, so $$dx(x)=x.$$
Note that means $\frac{dy}{dx}$ is also a function of $x$, being the quotient of two functions. What function is it?
$$
\frac{dy(x)}{dx(x)}=\frac{f'(p)\cdot x}{x}=f'(p)
$$
so $\frac{dy}{dx}$ is a constant, and that constant is $f'(p)$.