2

I'm given that if
$A$ is a fixed $p\times p$ real matrix, and $x$ is a $p$-dimensional real vector, and if $f(x) = x^T Ax$, then
$$\frac{\partial f(x)}{\partial x} = Ax + A^T x = (A+A^T)x$$
which is $p \times 1$ dimensional real vector.

However, looking at the top answer here: Derivative of Quadratic Form

It says that the derivative is actually
$$\frac{\partial f(x)}{\partial x} = x^T A + x^T A^T = x^T(A+A^T)$$
which is $1 \times p$ real vector.
So these two results definitely conflict.

I can follow the proof in the link but aren't the $h$ terms just constants? As in, we take this limit:
$$\lim_{h\rightarrow 0}\frac{Q(x+h)-Q(x)}{h}$$
and $h$ is a real number.
If so, I'm not sure how they got from their first equality to the second.
If not, then for the first equality, then it looks like we divide by a vector which doesn't make sense (I think?)
Thanks for any clarifications on which is the right answer.

Rodrigo de Azevedo
  • 1
Twenty-six colours
  • 1,881

1 Answers1

0

If you have a function $f : \mathbb{R}^p \rightarrow \mathbb{R} $ then by definiton this function is differentiable at point $x$ if there exists a continuous linear functional (for $\mathbb{R}^p$ all linear functional are continuous) $L_x : \mathbb{R}^p \rightarrow \mathbb{R}$ such that $$f(x+h)-f(x) = L_x(h) +o(h) $$ If such a mapping $L_x$ exists we call that mapping the derivative of $f$ with respect to $x$ and there are all kinds of notation for this thing such as $Df(x): = L_x$ or $\frac{\partial f}{\partial x}:=L_x$.

Now looking at this definition your derivative is actually linear functional $\frac{\partial f}{\partial x}:\mathbb{R}^p \rightarrow \mathbb{R}$ so if you are expressing your "usual" vectors of $R^p$ as colum vectors then if you were to express this linear mapping as a matrix in the standard basis of $\mathbb{R}^p$ and $\mathbb{R}$ then it would become a matrix that transforms $p$ dimensional colum vectors into matrices with single entry (aka scalars, since single entry matrices are isomorphic to your field $\mathbb{R}$). Those matrices are the $p$ dimensional row vectors.

Keep in mind that matrices in general are just made to conveniently represent mappings between finite dimensional vector spaces. You could chose to represent your vectors of $R^p$ as row vectors and multiply them with the matrix from the other side and then your derivative would become a colum vector. Also the spaces of colum and row vectors are isomorphic to each other so you could decide to represent both the $R^p$ vectors and the derivatives as the same type of vector (colum or row) but this would mean that you can no longer use the standard matrix multiplication to transform your vectors under your linear functionals (derivatives in your case).

gcc-6.0
  • 695