3

Let's define function $f : \mathbb{R}^n \to \mathbb{R}$ as

$$ f(x) = {1\over2}x'Ax + b'x $$

where matrix $A \in \mathbb{R}^{n\times n}$ and vector $b\in \mathbb{R}^n$ are given. Function $f$ is twice differentiable.

How can one prove the following?

$$\nabla f(x) = {1\over 2} (A+A')x + b$$

Rodrigo de Azevedo
  • 1
ecjb
  • 995

1 Answers1

1

Well you could expand it as $$f(x) = \frac{1}{2} \sum_i \sum_j x_ix_j A_{ij} + \sum_i b_ix_i$$ Derive with respect to $x_k$

$$\frac{\partial}{\partial x_k} f(x) =\frac{1}{2}(\sum_j x_j A_{kj} + \sum_i x_i A_{ik} ) + b_k$$

This means that

$$ \begin{bmatrix} \frac{\partial}{\partial x_1} f(x)\\ \frac{\partial}{\partial x_2} f(x)\\ \vdots\\ \frac{\partial}{\partial x_n} f(x) \end{bmatrix} = \begin{bmatrix} \frac{1}{2}(\sum_j x_j A_{1j} + \sum_i x_i A_{i1} ) + b_1\\ \frac{1}{2}(\sum_j x_j A_{2j} + \sum_i x_i A_{i2} ) + b_2\\ \vdots\\ \frac{1}{2}(\sum_j x_j A_{nj} + \sum_i x_i A_{in} ) + b_n \end{bmatrix}$$ The first term above it $\frac{1}{2} Ax$, the second is $\frac{1}{2} A^T x$ and the third is $b$.

Ahmad Bazzi
  • 12,076
  • thank you very much @AhmadBazzi for your answer. In the answer I got from my course I got the following first statement: $$ \forall k\leq n: \frac{\partial f}{\partial x_k}(x) =\frac{1}{2} \big[\sum_{i\neq k}x_i a_{ik} + \sum_{j\neq k}a_{kj}x_j + 2a_{kk}x_k\big] + b_k $$ now: where does $2a_{kk}x_k$ come from and why is there both $i\neq k$ and $j\neq k$ under both sum signs? – ecjb Mar 30 '19 at 09:53
  • it's the same: if you look at each of the summations that I wrote, you have an instance where $j =k $ and $i = k$. In your expression, the $k^{th}$ term is extracted out of each to emphasise that this term is in fact equal and thus sums up to $2a_{kk}x_k$. – Ahmad Bazzi Mar 30 '19 at 09:56