Let
$$f(x) := \| y - B x \|_2^2$$
where $y \in \mathbb R^k$, $x \in \mathbb R^n$, and $B \in \mathbb R^{k \times n}$. How to calculate the gradient of $f$ w.r.t. $x$?
Asked
Active
Viewed 89 times
0
-
I apologise for that. I didn't pay attention to the norms. I'll edit the question. – Shreyas Pimpalgaonkar Jul 05 '18 at 11:34
-
Yep I changed the question as well. Thanks for pointing that out. – Shreyas Pimpalgaonkar Jul 05 '18 at 11:36
-
As the dimensions of x are written, x is a vector. :) – Shreyas Pimpalgaonkar Jul 05 '18 at 11:37
-
Yes, I am using L2 norm. Also, thank you for your edits. – Shreyas Pimpalgaonkar Jul 05 '18 at 11:39
1 Answers
1
Since $f=\sum_i (bX-Y)_i^2=\sum_i (\sum_j b_{ij}X_j -Y_i)^2$, differentiating $f$ with respect to $X$ obtains a vector whose $k$th element is $\partial_{X_k} f$. Since $\partial_{X_k}\sum_j b_{ij}X_j=b_{ik}$, $\partial_{X_k}(\sum_j b_{ij}X_j -Y_i)^2=2b_{ik}(bX-Y)_i$ and $\partial_{X_k}f=[2b^T( bX-Y)]_k$ so $\partial_X f = 2b^T (bX-Y)$. As a sanity check, if everything were scalars you'd get $2b(bX-Y)$.
J.G.
- 115,835