Every local minimum of $f(x) = \frac{1}{2}x^tAx + b^tx +c$ is also a global minimum

Question

Consider the quadratic function $f(x) = \frac{1}{2}x^tAx + b^tx +c$, where $A\in\mathbb{R}^{n\times n}$ is symmetric, $b\in\mathbb{R}^n$ and $c\in\mathbb{R}$. Let $x$ be a local minimum of $f$. Prove that $x$ is a global minimum of $f$.

From here https://math.stackexchange.com/a/659982/166180 I know that if the hessian is positive then I know the function is convex. If we had this property, the function would have a unique minimum and therefore it'd be the global minimum because the function is convex. But we don't have any information on the hessian.

The only thing I can say is that

$$\nabla f(x) = Ax + b\\\nabla^2f(x) = A$$

since $x$ is a local minimum, $Ax+b = 0$ and $A$ is positive semidefinite. If it were positive definite, I think the problem would be solved, but it isn't. So how should I proceed?

Also where the symmetry of the matrix $A$ is used in all of this?

Answer 1

Once you know that $A$ is symmetric and positive semidefinite, there is an orthonormal basis transformation that makes it a diagonal matrix $D$ with non-negative real values in the diagonal. Another simplification is that yuo can forget about $c$.

By rewriting everything in the new basis, we have a very simple form of the function $f(x)= x^tDx+b^tx$ (with new $b$ and new variable $x$...), which is $f(x)= (d_1x_1^2 + b_1x_1)+ \ldots + (d_nx_n^2 + b_nx_n) $.

The condition that $x_0$ be a local minimum can be translated to all sections of the function: so for all $i$, $(d_1x_1^2 + b_1x_1)$ is a local minimum for the $i$-th coordinate of $x_0$.

If $d_i>0$, then we have a proper quadratic function with exactly one local minimum, and it is also a global minimum.

If $d_i=0$, then the only way $b_ix_i$ has a minimum is $b_i=0$. Nut then again, local minima and global minima coincide, as every rel number is a local and global minimum.

Answer 2

The symmetry of A is used in the fact that $\nabla f (x)= Ax + b$ otherwise it would be $\nabla f (x) = \frac{A + A^t}{2} x + b$.

Let $x_0$ be a minimizer of $f$, and let $k$ be in the kernel of $A$, then (here we use the fact that $A$ is symmetric) $f(x_0+ k) = b^t k + f(x_0) > f(x_0)$, when $k$ is sufficiently small. Therefore $b^t k >0$ for all $k$ in the kernel of $A$ sufficiently small. But since the sign of $b^t k $ depends only on the direction of $k$, this is true for all $k$ in the kernel of $A$. Now, by contradiction, suppose there exists another local minimizer of $f$, say $x$. It verifies $Ax = -b$. Therefore, $x = x_0 + k$ where $k$ is a vector in the kernel of $A$. Therefore $f(x) = b^tk + f(x_0) > f(x_0)$ Exchanging the roles of $x$ and $x_0$, we have $f(x_0)>f(x)$, which is absurd. Therefore $f$ possess only one local minimizer which must be a global minimizer, because $A$ is semidefinite and therefore $f(x)$ tends to infinity as $\| x\|$ goes to infinity.

Answer 3

If $A$ is positive semidefinite, $f$ is convex, whereas if $A$ is positive definite, $f$ is strictly convex. So your function is convex.