0

Let $A \in \mathbb{R}^{n \times n}$ be a positive semi-definite matrix. Let $$ f(x)=\frac{1}{2}x^TAx + c^Tx + b$$

It is possible to show that when $f$ is bounded below, and $c$ is in the range of $A$, then $f$ has a global minimizer.

Suppose only the following assumptions hold:

  1. $A$ is positive semi-definite matrix

  2. $c$ is in the range of $A$

Would it be possible to show that $f$ is bounded below? If so, find that lower bound.

Rodrigo de Azevedo
  • 1
Saeed
  • 175

3 Answers3

1

Let $x^*$ be such that $Ax^*=-c$. (You are missing a negative sign) Then $$ f(x) - f(x^*)= (x^*)^TA(x-x^*) + \frac12(x-x^*)^TA(x-x^*) + c^T(x-x^*) = \frac12(x-x^*)^TA(x-x^*) \ge0, $$ so $f(x^*)$ is the lower bound.

Saeed
  • 175
daw
  • 49,113
  • 2
  • 38
  • 76
0

This answer is for the first "possible" part in the question. We prove by contradiction. We need the result that $\mathrm{Null}(\mathbf{A}^T)^{\perp}=\mathrm{Range}(\mathbf{A})$ from here. Assume that $f(\mathbf{x})=\mathbf{x}^T\mathbf{A}\mathbf{x} + 2\mathbf{b}^T\mathbf{x} + c$ is lower bounded by a constant $d$. If $\mathbf{b}\notin \mathrm{Range}(\mathbf{A})$, that is to say $\mathbf{b}\notin\mathrm{Null}(\mathbf{A}^T)^{\perp}$, then $\mathbf{b}^T\mathbf{x}\neq0$ for any $\mathbf{A}^T\mathbf{x}=\mathbf{0}=\mathbf{A}\mathbf{x}$ since $\mathbf{A}$ is symmetric. In this case, $f(\mathbf{x})=2\mathbf{b}^T\mathbf{x} + c$. Let $t=\lambda\cdot\mathrm{sign}(\mathbf{b}^T\mathbf{x})$ where $\lambda>0$ and $\mathrm{sign}(u)=1$ if $u>0$, $\mathrm{sign}(u)=-1$ if $u<0$, otherwise $\mathrm{sign}(u)=0$, then \begin{equation} f(t\mathbf{x})=2\lambda\mathrm{sign}(\mathbf{b}^T\mathbf{x})\mathbf{b}^T\mathbf{x}\to -\infty, \end{equation} as $\lambda\to \infty$. This contradicts the assumption. Therefore, $\mathbf{b}\in\mathrm{Range}(\mathbf{A})$.

suineg
  • 385
0

By the linear approximation theorem, i.e. Theorem 1.24 in the Amir Beck's book Introduction to Nonlinear Optimization Theory, Algorithms, and Applications, at the stationary point $x^*$ satisfying $\mathbf{Ax}^*=-b$, there exists $\mathbf{z}\in[\mathbf{x}^*, \mathbf{x}]$ such that \begin{equation} f(\mathbf{x})-f(\mathbf{x}^*) = \frac{1}{2}(\mathbf{x}-\mathbf{x}^*)^T\nabla^2f(\mathbf{z})(\mathbf{x}-\mathbf{x}^*) = (\mathbf{x}-\mathbf{x}^*)^T\mathbf{A}(\mathbf{x}-\mathbf{x}^*)\ge 0 \end{equation} where the inequality follows from $\mathbf{A}\succeq\mathbf{0}$.

suineg
  • 385