Let, $\mathbf{A}$ be a symmetric positive definite matrix. I want to minimize $$f(x) = \frac{1}{2}x^T \mathbf{A}x - b^{T}x.$$ How would I go about this? I think I have to recast this as a least squares problem.
Asked
Active
Viewed 152 times
0
-
My first thought was to diagonalize $A$, and write $x$ w.r.t. an eigenbasis of $A$. Then you have separated all the variables, and the rest is easy. Of course, an even simpler way may be to just find the critical point by solving the linear system $\nabla f(x)=\vec{0}$. – Jyrki Lahtonen Jul 15 '21 at 15:54
-
But, I also recommend that you study our guide for new askers. – Jyrki Lahtonen Jul 15 '21 at 16:00
1 Answers
1
Differentiate the expression as it was done here. You end up with the jacobian of $f$:
$$x^\top \mathbf{A} -b^\top. $$ To find the minimum, the expression above must be equal to $0$, and so we find $$x^\top \mathbf{A} = b^\top.$$ $\mathbf{A}$ is positive definite and thus invertible, and so we find an extremum of the function at $x = \mathbf{A}^{-1} b$. Finally, differentiate the first expression again to obtain the Hessian of $f$, i.e. $\mathbf{A}$. Since $\mathbf{A}$ is positive definite, the extremum you obtained is a minimum of your function.

Pengin
- 350
- 2
- 9