Show that $\log |\Sigma|+\text{Tr}(\Sigma^{-1}V)$ is uniquely minimized

Question

Let $V$ be an $n\times n$ fixed positive definite matrix and let $f$ be the function defined by

$$f(\Sigma)=\log |\Sigma|+\text{Tr}(\Sigma^{-1}V)$$ over the set $P^+$ of $n\times n$ positive definite matrices $\Sigma$. Show that $f$ is uniquely minimized at $V$ over $P^+$.

My attempt:

Let $V=RR'$ be the Cholesky decomposition of $V$, and let $H:=R'\Sigma^{-1}R$. Then $H$ is positive definite and $\Sigma= RH^{-1}R'$. Hence $H\mapsto RH^{-1}R'$ is bijection from $P^+$ onto $P^+$ and, using the properties of determinant and trace, we see that the minimization problem is equivalent to minimizing

$$f(H):=\log |RH^{-1}R'|+\text{Tr}((R')^{-1}HR^{-1}V)$$

$$=\log |V|-\log |H|+\text{Tr}(H)$$

over $H\in P^+$. Now, Since every $H\in P^+$ has a unique Cholesky decomposition $LL'$ we see that the above problem is equivalent to minimizing

$$f(L)=\log |V|-\log |LL'|+\text{Tr}(LL')$$

$$=\log |V|-\sum_{i=1}^n\log(L_{ii}^2)+\sum_{i=1}^nL_{ii}^2+\sum_{i<j}L_{i,j}^2$$

$$=\log |V|+\sum_{i=1}^n\big(L_{ii}^2-\log(L_{ii}^2)\big)+\sum_{i<j}L_{i,j}^2$$

over $L$ lower triangular with positive diagonal entries. Since the map $x\mapsto x-\log x$ is uniquely minimized at $x=1$ it follows that $f$ is uniquely minimized at $L=I_n$, i.e. $\Sigma=V$.

Is this correct? Am I missing something?

Thanks a lot for your help.

It seems okay to me. However you can do it using the fact that $\Sigma \to \log |\Sigma|$ is strictly concave. — Kroki, Aug 03 '22 at 00:10
Every strictly convex function has one minimiser over a convex set. — Kroki, Aug 03 '22 at 03:02
@Youem the real exponential function on the convex set $\mathbb R$ has no minimizer. I think you mean such a function has at most one minimizer. — user8675309, Aug 03 '22 at 05:24

Kroki · Accepted Answer · 2022-08-03T16:10:54.903

1

Another argument you can use the concavity of $\Sigma \to \log \left|\Sigma\right|$ (see here). Since $$f\left(\Sigma^{-1}\right) = -\log \left|\Sigma\right| + \text{Tr}\left(\Sigma V\right)$$ then $\Sigma \to f\left(\Sigma^{-1}\right)$ is strictly convex function (sum of a strictly convex function and a convex function see here). Since every convex function has at most one minimiser on a convex set (see here), $f$ is at most uniquely minimized.

Now let's prove that $f$ has a local minimum (this will be a global minimum as well).

Let $g(t) = f\left(\left(V^{-1}+tH\right)^{-1}\right)$ for fixed $H \in \mathbb S^n$ this function is well defined for $t\in (-\delta, \epsilon)$ with $\delta, \epsilon > 0$. $g(t)$ is convex

\begin{align} g(t) &= -\log \left|V^{-1}+tH\right| + \text{Tr}\left(\left(V^{-1} + tH\right)V\right)\\ &= -\log \left|V^{-1}\right| - \log \left|I + tHV\right| + \text{Tr}(I) + \text{Tr}(tHV)\\ &= \log \left|V\right| - \left(\text{Tr}\left(tHV\right) + o\left(t^2\right)\right) + \text{Tr}(I) + \text{Tr}(tHV)\\ &= \log \left|V\right| + \text{Tr(I)} + o\left(t^2\right) \end{align}

This proves that $g'(0) = 0$ and $0$ is a local minimum of $g$ and hence global minimum of this function (since it is convex).

$$f\left(\Sigma^{-1}\right) = f\left(\left(V^{-1} + \left(\Sigma - V^{-1}\right)\right)^{-1}\right) = g(1) \ge g(0) = f(V)$$

for $H = \Sigma - V^{-1}$. Thus $V$ is a minimum of $f$.

edited Aug 03 '22 at 16:10

answered Aug 03 '22 at 03:14

Kroki

13,135

Two questions: 1) Don't we need to show that $-\log \left|I + tHV\right| + \text{Tr}(tHV)\geq 0$ in a neighborhood of $0$ 2) This shows local minimum along a specific line only. Is it really a local minimum? – Alphie Aug 03 '22 at 15:46
I am not sure I understand your question exactly. Are you asking about proving that $t\to f\left(\left(V^{-1} + tH\right)^{-1}\right)$ has a local minimum? – Kroki Aug 03 '22 at 15:48
Yes and also why a local minimum for $t\to f\left(\left(V^{-1} + tH\right)^{-1}\right)$ at $t=0$ is equivalent to a local minimum for $\Sigma \to f\left(\Sigma^{-1}\right)$ at $V$. – Alphie Aug 03 '22 at 15:54
Sorry I did not explain it well. I will add it to the answer. – Kroki Aug 03 '22 at 16:02
Let me know if it is clear now? – Kroki Aug 03 '22 at 16:11
Why is $g'(0)=0$? – Alphie Aug 03 '22 at 16:25
$g'(0) = 0$ because $g(t) = g(0) + o(t^2)$ and the second order conditions are given by the convexity. – Kroki Aug 03 '22 at 16:26
Ok I see and why $-\log \left|I + tHV\right| + \text{Tr}(tHV)=o(t^2)$? Taylor expansion? – Alphie Aug 03 '22 at 16:28
Let us continue this discussion in chat. – Kroki Aug 03 '22 at 16:29

Show that $\log |\Sigma|+\text{Tr}(\Sigma^{-1}V)$ is uniquely minimized

1 Answers1