0

I want to solve the following equation: \begin{equation} \label{eq:approx_quad_2} \min_{{\bf x}} \| {\bf x}\|_2^2 \quad \mbox{s.t.} \quad \\ \quad {\bf x}^\top {\bf Q} {\bf x} + {\bf x}^\top {\bf b} = c, \end{equation} where ${\bf Q}$ is a matrix and ${\bf b}$ is a vector and $c$ is a scalar.

We can solve this equation using the method of Lagrange multipliers. \begin{align*} & L({\bf x}, \lambda) = \| {\bf x} \|_2^2 - \lambda({\bf x}^\top {\bf Q} {\bf x} + {\bf x}^\top {\bf b} - c) \\ \implies &\nabla_{\bf x} L = 2 {\bf x} - 2 \lambda ({\bf Q} {\bf x} + {\bf b}) = 0 \\ & \nabla_\lambda L = {\bf x}^\top {\bf Q} {\bf x} + {\bf x}^\top {\bf b} - c = 0 \end{align*}

Taking the inner product of the first constraint with ${\bf x}$, \begin{align*} 0 &= \| {\bf x} \|^2 - \lambda ({\bf x}^\top {\bf Q} {\bf x} + {\bf b}^\top {\bf x}) \\ &= \| {\bf x} \|^2 - \lambda c \\ \implies & \lambda = \frac{\| {\bf x} \|^2}{c} \end{align*}

Plugin this in the first constraint, \begin{align*} 0 &= {\bf x} - \lambda ({\bf Q} {\bf x} + {\bf b}) \\ &= {\bf x} - \frac{\| {\bf x} \|^2}{c} ({\bf Q} {\bf x} + {\bf b}) \\ \implies & \frac{{\bf x}}{\| {\bf x} \|^2} - \frac{1}{c} {\bf Q} {\bf x} = \frac{{\bf b}}{c} \\ \implies & {\bf x} = \left( \frac{{\bf I}}{\| {\bf x} \|^2} - \frac{1}{c} {\bf Q} \right)^{-1} \frac{{\bf b}}{c} \end{align*}

This gives us a transcendental equation, there might not be a closed-form solution.

Alternatively, I tried to complete the square to solve the second constraint (see e.g. Solving quadratic vector equation) but I also got to a transcendental equation. Is there another technique to solve this problem?

atownmath
  • 31
  • 3
  • Any known properties on $Q$? – Johan Löfberg Apr 28 '21 at 18:57
  • ${\bf Q}$ could be symmetric. – atownmath Apr 28 '21 at 22:03
  • Yes, that can always be assumed without loss of generality, more interesting about definiteness or semidefiniteness – Johan Löfberg Apr 28 '21 at 22:05
  • Ok, ${\bf Q}$ is not necessarily PSD in my problem, although I'm happy if we can already solve the PSD case. Does the PSD property help to solve this problem? – atownmath Apr 29 '21 at 07:14
  • With a positive definite $Q$ you should be able to make a variable change and write the problem as the classical trust-region problem. That is probably both good and bad news. Good in the sense that it puts it in a classical framework, bad in the sense that the trust-region problem does not have a closed form solution, but essentially arrives at what you have, and then performs a numerical search. – Johan Löfberg Apr 29 '21 at 07:23
  • I see, this is an interesting observation. Also bad news since I was hoping for a closed-form formula. – atownmath Apr 29 '21 at 07:27

0 Answers0