You said:
- $\lim_{h \rightarrow 0} \dfrac{f(p+h)-f(p)}{||h||}$ represents the derivative at $p$
No, it doesn't. Next, you write:
- $\dfrac{f(p+h)-f(p)}{||h||}$ can be approximated as $f'(p)+\eta(h)$ where $\eta(h)$ represents an error function.
Again, this is false.
The definition of differentiability is that there exist a linear function $L:\Bbb{R}^m\to \Bbb{R}$ such that
\begin{align}
f(p+h) - f(p) &= L(h) + \lVert h\rVert\eta(h) \tag{$\ddot{\smile}$}
\end{align}
where $\lim_{h\to 0} \eta(h) = 0$. Typically, the notation used is that $L := Df_p$. Also, if we define $\Delta f_p(h) := f(p+h) - f(p)$, then the above equation becomes very memorable:
\begin{align}
\Delta f_p(h) &= Df_p(h) + \lVert h\rVert \eta(h).
\end{align}
This is exactly the formal way of saying that differentiable functions are locally approximately linear, because it says the actual change in the function (at the point $p$ by an amount $h$) $\Delta f_p(h)$ is equal to a linear part $Df_p(h)$ plus an error term $\lVert h\rVert\eta(h)$, and this error term is "small" in the sense $\eta(h)\to 0$ as $h\to 0$.
So, to address (1) above, it is $L= Df_p$ which is the derivative at $p$ (by definition).
For (2), we have
\begin{align}
\dfrac{f(p+h) - f(p)}{\lVert h \rVert} &= \dfrac{Df_p(h)}{\lVert h\rVert} + \eta(h) \\
&= Df_p\left(\dfrac{h}{\lVert h \rVert}\right) + \eta(h)
\end{align}
so, you can interpret this however you want. But the point remains: $L(\cdot) = Df_p(\cdot)$ is by definition a linear transformation which approximates changes in $f$ (i.e which is approximately equal to $\Delta f_p(\cdot)$).
In the comments you ask:
But, if we were to go by the basic definition of a derivative: it should be $f′(p)\cdot \lVert h\rVert$.
What do you mean by basic definition? The definition you wrote in the question is the definition of derivative in multivariable calculus. Do you mean the definition of single-variable calculus? If that's what you meant then the thing is you need to learn to re-interpret the definition in single variable calculus. We are often taught in the case of $f:\Bbb{R}\to \Bbb{R}$ to think of $f'(p)$ geometrically as "the instantaneous slope at $p$", because we define (if the limit exists)
\begin{align}
f'(p):= \lim_{h\to 0}\dfrac{f(p+h) - f(p)}{h}
\end{align}
so of course, geometrically this forces us to think in terms of slopes.
What I'm now suggesting to you is to think in terms of "local linear approximations (this allows for a much easier transition to multivariable calculus) and to rewrite this definition as
\begin{align}
\lim_{h\to 0}\dfrac{f(p+h) - f(p) - f'(p)\cdot h}{h} &= 0 \tag{$*$}
\end{align}
In this case, the mapping $\Bbb{R}\to \Bbb{R}$, $h\mapsto f'(p)\cdot h$ is a linear transformation which approximates the actual change $\Delta f_p(h):= f(p+h) - f(p)$.
Note that in this case, we are able to divide by $h$ because it is a real number and not a vector. But notice that $(*)$ is entirely equivalent to $(**)$:
\begin{align}
\lim_{h\to 0} \dfrac{|f(p+h) - f(p) - f'(p)\cdot h|}{|h|} &= 0 \tag{$**$}
\end{align}
and in this form, the relation with definition of differentiability in higher dimensions is much more clear, because $(\ddot{\smile})$ is entirely equivalent to the following statement (with appropriate domains and target spaces):
There exists a linear transformation $L$ such that
\begin{align}
\lim_{h\to 0}\dfrac{\lVert f(p+h) - f(p) - L(h)\rVert}{\lVert h\rVert} &= 0.
\end{align}
