0

let $A\in \mathbb{R}^{N\times N}$ and $f: \mathbb{R}^N \to \mathbb{R}, f(x)=\langle Ax, x\rangle$. How can I compute $\partial _1 f$?

Analysis
  • 2,450
  • 1
    $(A(x+y),(x+y))-(Ax,x)=(Ax,y)+(Ay,x)+(Ay,y)$. Since $|(Ay,y)|\leq |A||y|^2$, then $(Ax,y)+(Ay,x)$ must be the differential. Since this is equal to $(Ax,y)+(A^Tx,y)=((A+A^T)x,y)$, then the gradient is the vector $(A+A^T)x$. – egorovik Dec 16 '19 at 00:15
  • Can you try the case when $N=2$? –  Dec 16 '19 at 00:44

1 Answers1

0

$f(x)=(Ax)^{\top}x=x^{\top}A^{\top}x$.

\begin{align*} f(x)=\sum_{i,j=1}^{n}x_{i}A_{i,j}x_{j} \end{align*}

\begin{align*} \frac{\partial f}{\partial x_{k}}&=\sum_{i,j=1}^{n}\left(\delta_{i,k}A_{i,j}x_{j}+x_{i}A_{i,j}\delta_{j,k}\right)\\ &=\sum_{j=1}^{n}A_{k,j}x_{j}+\sum_{i=1}^{n}x_{i}A_{i,k}\\ &=e_{k}^{\top}Ax+x^{\top}Ae_{k}\\ &=e_{k}^{\top}Ax+e_{k}^{\top}A^{\top}x \end{align*}

We note that

\begin{align*} e_{k}^{\top}\nabla f&=\frac{\partial f}{\partial x_{k}} \end{align*}

so that by uniqueness we have

\begin{align*} \nabla f &= Ax+ A^{\top}x\\&=\left(A+A^{\top}\right)x \end{align*}

Here I have used $\partial x_{i}/\partial x_{k}=\delta_{i,k}$, where $\delta_{i,k}=1$ when $i=k$ and is zero otherwise. Note also that I have used $e_{k}$ to denote the $k$th standard basis element of $\mathbb{R}^{n}$. I hope that helps.

Tucker
  • 2,090
  • 9
  • 12