Convex Optimization: Gradient of $\log \det (X)$

Question

In Boyd's CVX book, there is a step by step analysis of the gradient of so called log det function

Three confusions:

Is the determinant for positive definite matrix exactly equivalent to the sum of eigen values is equal to the trace?
There is the claim that because $\Delta x$ is small (what does it mean by small), therefore $\lambda_i$ are small, is there any justification to this claim? Because after all we are computing the eigenvalues of $X^{-1/2}\Delta X X^{-1/2}$ not simply $\Delta X$
By first order approximation of $\log(1+\lambda_i) \approx \lambda_i$ I am assuming first order Mac series?

Thanks!

If I remember right, eigenvalues depend continuously on the matrix, at least if the matrix is normal. (cf Terry Tao, 254A, Notes 3a: Eigenvalues and sums of Hermitian matrices, Equation (13)). — user251257, Sep 18 '15 at 01:38
@user251257 relevant? Prove $\frac{\partial \rm{ln}|X|}{\partial X} = 2X^{-1} - \rm{diag}(X^{-1})$.. Here I say 'We first note that for the case where the elements of X are independent, a constructive proof involving cofactor expansion and adjoint matrices can be made to show that $\frac{\partial ln|X|}{\partial X} = X^{-T}$ (Harville). This is not always equal to $2X^{-1}-diag(X^{-1})$. The fact alone that X is positive definite is sufficient to conclude that X is symmetric and thus its elements are not independent.' — BCLC, Apr 16 '21 at 09:59

score 1 · Accepted Answer · answered Sep 18 '15 at 01:08

1

If $A \in S_{++}$ then $\det A$ is the product of the eigenvalues and $\log \det A$ is the sum of their logarithms.
If $\Delta X$ is small then so is $X^{-1/2} \Delta X X^{-1/2}$. This is just a multiplication by two fixed matrices after all.
You got the MacLaurin series wrong, $\log (1+x) = x - \frac{x^2}{2} + \dots$ for $|x| < 1$. After that correction the argument is OK.

answered Sep 18 '15 at 01:08

Hans Engler

relevant? Prove $\frac{\partial \rm{ln}|X|}{\partial X} = 2X^{-1} - \rm{diag}(X^{-1})$.. Here I say 'We first note that for the case where the elements of X are independent, a constructive proof involving cofactor expansion and adjoint matrices can be made to show that $\frac{\partial ln|X|}{\partial X} = X^{-T}$ (Harville). This is not always equal to $2X^{-1}-diag(X^{-1})$. The fact alone that X is positive definite is sufficient to conclude that X is symmetric and thus its elements are not independent.' – BCLC Apr 16 '21 at 09:59

1 Answers1