3

In a text book I am following, there is a definition of positive definite matrices, which I did not see before:

$x^TAx \geq \alpha x^Tx$

where $\alpha$ is a positive scalar and $x \in \mathbb{R^n}$.

The usual definition I know is for every non-zero vector $x \in \mathbb{R^n}$, it is $x^TAx > 0$.

These definitions must be equivalent. It is trivial to show that the first definition implies $x^TAx > 0$. But I failed to see a way to show the other way around of the equivalence. I thought of applying SVD to the matrix $A$ and tried to find a way to show that $x^TAx$ is always lower bounded by a $\alpha x^Tx$ with $\alpha$ being positive, using singular values and vectors of $A$ but couldn't move forward. What is the correct way of approach here?

(The other question in the board assumes symmetric matrices. This one does not).

Ufuk Can Bicici
  • 3,026
  • 1
    I think we can prove that, assuming that $A$ is PD, there exists $a$ such that $A-aI$ is PSD, that is because the eigenvalues of $A-aI$ will be bigger than those of $A$ minus $a$ (this needs to be proven) and since they were strictly positive in the first place then you can find $a$ such that they still are positive. – P. Quinton Nov 26 '18 at 07:25

2 Answers2

2

Let $f(x)=x^TAx$. Furthermore let $f(x)>0$ for all non-zero $x$.

Let $C:=\{x \in \mathbb R^n: ||x||_2=1\}$. Then $C$ is compact and $f$ is continuous on $C$. Thus there is $x_0 \in C$ such that

$f(x_0) \le f(x)$ for all $x \in C$.

Now put $ \alpha =f(x_0)$. Then $\alpha >0$.

It is your turn to show that $x^TAx \geq \alpha x^Tx$ for all $x \in \mathbb R^n$.

Hint: for $t \in \mathbb R^n$ and $t \in \mathbb R$ we have $f(tx)=t^2f(x).$

Fred
  • 77,394
  • Thank you for the answer. I think we have the following: $C$ is compact and $f(x)$ is a continuous mapping, which implies $f(C)$ is also compact, which additionally means $f(C)$ is closed and bounded. This provides us with a $x_0 \in C$ such that $\alpha = f(x_0) = \inf f(C)$ and $f(x_0) \in f(C)$. For any $x \in \mathbb{R}^n$, we have $\left(\dfrac{x}{||x||_2} \right)^T A \left(\dfrac{x}{||x||_2} \right) \geq \alpha$. Seeing that $x^Tx = ||x||_2^2$ we multiply both sides with $x^Tx$ and then we are good to go: $x^T Ax \geq \alpha x^T x$. – Ufuk Can Bicici Nov 26 '18 at 14:44
  • 1
    Well done ! In $\left(\dfrac{x}{||x||_2} \right)^T A \left(\dfrac{x}{||x||_2} \right) \geq \alpha$ you should mention that $x \ne 0$. – Fred Nov 26 '18 at 14:57
1

The following is valid only if the matrix $A$ doesn't have complex values, which is at least true if it is also symmetric.

Since $A\succ 0$, we can use it's diagonalisation as $A=U\Sigma U^{\dagger}$ such that $U U^\dagger = U^\dagger U = I$ and $\Sigma$ is the diagonal matrix containing the eigenvalues of $A$, then for any $a>0$, $A-aI = U(\Sigma - aI) U^\dagger$ and so the eigenvalues of $A-aI$ are those of $A$ minus $a$.

Being positive definite means that the eigenvalues are strictly positive and so we can find $0<a\leq\min_{i} \lambda_i$ where $\lbrace\lambda_i\rbrace_{i\in[1{:}n]}$ are the eigen values such that the eigen values of $A-aI$ are positive which is $A\succ aI$ and hence $x^\dagger A x \geq a x^\dagger x$

P. Quinton
  • 6,031
  • $A$ can be nonsymmetric, so its eigenvalues do not have to be real; then how can we say that positive definite means the eigenvalues are strictly positive? – Ufuk Can Bicici Nov 26 '18 at 09:11
  • Well, I guess this is one of the properties of the positive definite matrix https://en.wikipedia.org/wiki/Positive-definite_matrix#Characterizations Also if an eigenvalue $\lambda$ is complex, then if $v$ is the associated eigenvector, then $v^\dagger A v = \lambda v^\dagger v$ which is a complex value and cannot be compared to $av^\dagger v$ so the problem is not solvable. – P. Quinton Nov 26 '18 at 09:17
  • Interestingly the question here: https://math.stackexchange.com/questions/83134/does-non-symmetric-positive-definite-matrix-have-positive-eigenvalues shows complex eigenvalues of a positive definite matrix. I think being symmetric is implicitly assumed in the text then. – Ufuk Can Bicici Nov 26 '18 at 09:19
  • 1
    Then I guess you would need to compare the real values in the question, otherwise it doesn't make sense. And yes you are right, in the wikipedia page they assume symetric. I think Fred's way is the way to go, but you should add real values everywhere. – P. Quinton Nov 26 '18 at 09:21