Let $x^{T}=[X_{1}\ X_{2}], \ h(x)=x$. Then,
$$\dfrac{\partial h(x)}{\partial x} = \dfrac{\partial}{\partial h(x)}\begin{bmatrix}X_{1}\\ X_{2}\end{bmatrix} = \begin{bmatrix}\dfrac{\partial}{\partial X_{1}} X_{1} \dfrac{\partial}{\partial X_{2}} X_{1}\\ \dfrac{\partial}{\partial X_{1}}X_{2}\ \dfrac{\partial}{\partial X_{2}}X_{2}\end{bmatrix}$$
The derivative therefore is 2x2 while x is 2x1. If we were to therefore use this result for something like gradient descent, how would that work?
It is better if you use Mathjax because it is not so clear what you are asking. Anyway, if $f:\mathbb{R}^n \to\mathbb{R}^m $ is vector function $\mathbf{f}(\mathbf{x})= \left(f_1(\mathbf{x}),f_2(\mathbf{x}), \cdots, f_m(\mathbf{x}) \right)$, the derivative of $\mathbf {f}$ with respect to $\mathbf{x}$ is the matrix:
$$
\begin{bmatrix}
\frac{\partial f_1}{\partial x_1 }& \cdots & \frac{\partial f_1}{\partial x_n }\\
\frac{\partial f_2}{\partial x_1 }& \cdots & \frac{\partial f_2}{\partial x_n }\\
\vdots & \ddots & \vdots \\
\frac{\partial f_m}{\partial x_1 }& \cdots & \frac{\partial f_m}{\partial x_n }\\
\end{bmatrix}
$$
usually called the Jacobian matrix.
