1

Let $$f(x) = x^\top Q \, x$$ be a quadratic form, where $Q \in \mathbb R^{n×n}$ is NOT symmetric. Show that the Hessian matrix is $$H_f (x) = Q + Q^\top$$ Hint: $x^\top Q \, x = x^\top Q^\top x.$

If $Q$ is symmetric I know that $\nabla f(x) = 2 Q x$ and $H_f(x) = 2 Q$. However, I am not sure as to what I should do when Q is not symmetric. Also, the hint is somewhat misleading to me as it makes it appear that $Q$ is symmetric. Help would be really appreciated!

Rodrigo de Azevedo
  • 1
thisisme
  • 635
  • Were Q symmetric then $Q^T=Q$ which isn't the case here. – Triatticus Jan 21 '18 at 04:42
  • You can use this hint in the product rule, despite how it looks the two $x$ vectors have switched places this is important when you go to use the product rule as I'm assuming the differentiation is wrt either $x$ or $x^T$ – Triatticus Jan 21 '18 at 04:52
  • Last remark because I keep forgetting to edit my comments in time as I'm on mobile, do you have to do this via definition of derivative of are you allowed to differentiate the quicker way? – Triatticus Jan 21 '18 at 05:02
  • Let $y=x^TQx$. Now we can write $y^T=y$, because $y$ is a scalar. The hint provided in the question follows from this. –  Jan 21 '18 at 06:54
  • To use the hint, notice that $f(x)=x^TQx=x^TQ^Tx$, so $2f(x) = f(x)+f(x) = x^TQx+x^TQ^Tx$. Use this to write $f(x)$ with a symmetric matrix between $x^T$ and $x$. Then you know how to find its Hessian. –  Jan 21 '18 at 06:56
  • @Rahul thank you for your explanation! – thisisme Jan 21 '18 at 07:05

1 Answers1

2

HINT:

$$ Q = \frac{ 1}{2} \underbrace{( Q + Q^T)}_{sym} + \frac{1}{2} ( Q - Q^T)$$ If you use the hint provided then $$\begin{align*} (x,Qx) =& \frac{1}{2} (x,(Q+Q^T)x) + \frac{1}{2}(x,(Q-Q^T)x) \\ =& \frac{1}{2} (x,(Q+Q^T)x) + \frac{1}{2} \underbrace{(x,Qx)-(x,Q^Tx)}_{=0} \\ = & \frac{1}{2} (x,(Q+Q^T)x) \end{align*}$$

Jeb
  • 4,178
  • 1
  • 13
  • 13
  • Thank you for this explanation, looks beautiful! I would not be able to come up with the hint myself and was wondering did you use the hint provided to derive your hint? – thisisme Jan 21 '18 at 05:16
  • The hint is just a general fact that you can decompose any operator into a symmetric and anti-symmetric component. Its a good fact to remember. – Jeb Jan 21 '18 at 05:19
  • I see thank you. Just to clarify, is your hint completely independent of the hint provided in my problem, because I am still unable to connect the two? I just learned of this property that you mentioned, but I still don't understand how its tied to the hint in my problem. – thisisme Jan 21 '18 at 05:21
  • actually, the comment above answered my question, but thank you for your explanation! – thisisme Jan 21 '18 at 07:06