0

I am trying to find the Jacobian matrix of $x^T A x$. Where $A \in \mathbb R^{n \times n}$ (and $A$ is symmetric) and $x\in \mathbb R^n$.

I have tried to write it in summation notation:

$$x^T A x=\sum_{i=1}^n \sum_{j=1}^n x_ia_{ij}x_j:=y$$

Since $x^TAx=y$ is a polynomial, the corresponding Jacobian matrix will look something like this:

$$J_y=\begin{pmatrix} \frac{\partial f_1}{\partial x_1} & \frac{\partial f_1}{\partial x_2} & ........ & \frac{\partial f_1}{\partial x_n} \end{pmatrix}$$

So the $k$-th component of my Jacobian matrix is:

$$ \frac{\partial y}{\partial x_k}=\frac{\partial }{\partial x_k} \left(\sum_{i=1}^n \sum_{j=1}^n x_ia_{ij}x_j \right) $$

I am stuck here. I have tried separating this into cases where $k=i \not=j$ but I am not really getting anywhere. Is there a way to simplify this further? ( I am not necessarily looking for a complete solution but rather a few hints that will enable me to figure it out by myself. Thanks.)

Jean-Claude Arbaut
  • 23,277
qmd
  • 4,275
  • 3
    Apply Leibniz rule and remember that $\partial x^i/\partial x^j = \delta_{ij}$, where $\delta_{ij} = 1$ if $i=j$ and $0$ if $i\neq j$. – Gibbs Oct 19 '19 at 13:47
  • @Gibbs So separating into cases where $k=j, k=i$ etc. was the right way to approach this if I am understanding you correctly? – qmd Oct 19 '19 at 14:14
  • 2
    You do not really need to separate this into cases. First of all, linearity of the partial derivative tells you can swap sums and derivative, so you end up with the derivative with respect to $x_k$ of $x_ia_{ij}x_j$. So applying Leibniz rule you have $(\partial_k x_i)a_{ij}x_j+x_i(\partial_k a_{ij})x_j+x_ia_{ij}\partial_k x_j$. Can you go on from here? – Gibbs Oct 19 '19 at 14:38
  • @Gibbs Of course! I forgot how to take derivatives for a second there. I think i can take it from here. Thank you for your help! – qmd Oct 19 '19 at 14:59
  • You are welcome. – Gibbs Oct 19 '19 at 15:03

1 Answers1

1

Define $f:\mathbb R^n \to \mathbb R$ as $f(x)= x^T A x$. Then $f(a+h)-f(a)= 2a^T A h+h^T A h$. As you can see, a candidate for the derivative $\mathrm df(a)$ is $2a^T A$ and one can indeed verify this guess by calculating \begin{equation} \lim _{{h} \to {0}} \frac{\|f({a}+{h}) - f({a}) - 2a^TAh\|}{\|{h}\|}. \end{equation}(Hint: Recall that $h^T A h=\langle h,Ah\rangle$ and use Cauchy Schwarz.)

If you believe $f$ is differentiable, an easier way to find out the derivative of $f$ at $a$ would be to use the result $$\mathrm df(a)y=\left.\frac{\mathrm df}{\mathrm dt}(a+ty)\right|_{t=0}.$$

cqfd
  • 12,219
  • 6
  • 21
  • 51