Given
$$ f(X) = \frac{1}{2}X^TQX$$
Can anyone explain and show me step-by-step or point me to a good reference (or video), as to how $ \nabla f(X)$ is given by,
$$ \nabla f(X) = QX$$
Asked
Active
Viewed 657 times
0
-
There are some details missing. Are $X$ and $Q$ matrices of compatible dimensions? Is $X$ a vector? – parsiad Mar 15 '17 at 21:12
-
2$Q$ must be assumed symmetric. Otherwise it's $(Q + Q^T) X/2$. – Robert Israel Mar 15 '17 at 21:18
-
The notes I am following don't actually say. However, in the context it is talking about quadratic form. – user1011182 Mar 15 '17 at 21:53
-
That’s useful information. For a quadratic form, $Q$ is indeed symmetric. – amd Mar 15 '17 at 22:56
2 Answers
2
I presume $X$ is a $n$-dimensional vector and $Q$ is a $n \times n$ symmetric matrix.
Bare hands approach: $$f(X) = \frac{1}{2} X^\top Q X = \frac{1}{2} \sum_{i=1}^n \sum_{j=1}^n Q_{ij} X_i X_j$$ Compute partial derivative directly (use symmetry of $Q$): $$\frac{\partial}{\partial X_k} f(X) = \frac{1}{2}\sum_{i\ne k} Q_{ik} X_i + \frac{1}{2} \sum_{j\ne k} Q_{kj} X_j + Q_{kk} X_k = \sum_{j=1}^n Q_{kj} X_j.$$
If you then construct $\nabla f(X)$ as the vector of partial derivatives, you will get $QX$.
angryavian
- 89,882
1
$\nabla f$ is obtained by computing $f(X+H)-f(X)$ :
$$f(X+H) = \frac{1}{2}(X+H)^TQ(X+H)=\frac{1}{2}(X^TQX + H^TQX + X^TQH + H^TQH)$$
Since $H^TQX$ is a scalar, it is also equal to its own transpose $X^TQH$ (edit : indeed if $Q$ is symmetric)
Hence $f(X+H)-f(X)=H^TQX+\frac{1}{2}H^TQH$
By definition, $\nabla f$ is the vector such that $f(X+H)-f(X) = H^T\nabla f + o(H)$, the $o(H)$ meaning we can get rid of terms of second order in $H$ (everything which is $H$ squared or $H^TH$ for example, and $H^TQH$ is second order of $H$ too)
Therefore $\nabla f = QX$
M. Boyet
- 603