1

Let $A\in R^{n\times n}$ be a symmetric positive definite matrix and $b\in R^n$. Define $f:R^n\to R$ by $f(x)=\frac{1}{2}x^TAx-b^Tx$. Show that $f$ is strictly convex and has exactly one global minimum.

I've managed to prove that $f$ is strictly convex through some messy calculation. But I only need the fact that $A$ is positive definite to show convexity. So I suppose to prove there exists a global minimum I need to use $A$ being symmetric. How do I do that? And since $f$ strictly convex, if it has a minimum, it will be unique, right?

Fluffy Skye
  • 691

3 Answers3

1

Using Fréchet derivatives.

For any point $x_0 \in \mathbb R^n$, the second derivative of $f$ is

$$f^{\prime \prime}(x_0)(u,v) = u^T A v.$$

Which is positive definite as $A$ is. Also $f^\prime$ vanishes at only one point. Hence $f$ has a unique minimum.

mathcounterexamples.net
  • 70,018
1

If you are allowed to use an excursion into linear algebra, you can prove your claim quite easily as follows using:

  • $A$ is symmetric, so there is an orthogonal matrix $U$ with $$U^TAU = \operatorname{diag}(\lambda_1,\ldots , \lambda_n)$$
  • $A$ is positive definite, so $\lambda_i>0\,(i=1,\ldots , n)$
  • Using the fact $U^TU = UU^T = I_{n\times n}$, and setting $y= U^Tx$ and $c=U^Tb$, rewrite $f(x)$ as follows $$\frac{1}{2}x^TAx-b^Tx = \frac{1}{2}y^T\operatorname{diag}(\lambda_1,\ldots , \lambda_n)y-c^Ty = \frac{1}{2}\sum_{i=1}^n\left(\lambda_iy_i^2 -2c_iy_i \right)$$ $$= \frac{1}{2}\sum_{i=1}^n\left(\left(\sqrt{\lambda_i}y_i -c_i\right)^2-c_i^2\right) =:g(y)$$

$g(y)$ is easily checked to be strictly convex and has a unique global minimum at $y_i =\frac{c_i}{\sqrt{\lambda_i}}\, (i=1,\ldots , n)$.

trancelocation
  • 32,243
1

Complete the squares on $x$ $$ f(x)=\frac12(x-A^{-1}b)^TA(x-A^{-1}b)-\frac12b^TA^{-1}b. $$ Now the graph of $f$ is the shifted version of $g(x)=\frac12x^TAx$. Can you prove that the latter is strictly convex as well as the minimum exists and is unique?

A.Γ.
  • 29,518