For $\rho$ and $A$ hermitian matrices, $t$ a scalar parameter, $f_i(\rho)$ denoting the $i-$th eigenvalue of the matrix $\rho$ and $|x_i\rangle$ the normalized eigenvector of $\rho$ corresponding to $f_i(\rho)$, does the following hold in the limit $t \to 0$
$f_i(\rho + tA)=f_i(\rho) + tr \{A |x_i\rangle\langle x_i|\}t +{\mathcal{O}}(t^2)$
I tried to find a reference for this but couldn't. Does anyone know whether this holds true?
This certainly isn't true in general. E.g. suppose $A$ is not a scalar multiple of $I$. Let $x$ be a unit eigenvector corresponding to the minimum eigenvalue of $A$. Then $x$ is also an eigenvector corresponding to the maximum eigenvalue of $I$. However, given any function $g$ such that $g(t)=O(t^2)$, \begin{aligned} \lambda_\max(I+tA) &=\lambda_\max(I)+t\lambda_\max(A)\\ &>\lambda_\min(I)+t\lambda_\min(A)+g(t)\\ &=\lambda_\min(I)+t\operatorname{tr}(Axx^\ast)+g(t)\\ &=\lambda_\max(I)+t\operatorname{tr}(Axx^\ast)+g(t) \end{aligned} when $t>0$ is sufficiently small.
The statement is true, however, when the eigenvalue is simple. If you assume that eigenvalues and eigenvectors are differentiable, the statement follows easily from implicit differentiation. See user7530's answer for instance.
An elementary proof of a more generalised version of the statement, without assuming differentiability of eigenvalues or eigenvectors, can be found in Stewart and Sun's Matrix Perturbation Theory. See theorem 2.3 on p.183. Their proof, when adapted to your case, is especially easy. By a change of orthonormal basis, we may assume that $\rho=\operatorname{diag}(\lambda_1(\rho),\ldots,\lambda_n(\rho))$ is a diagonal matrix whose eigenvalues are arranged in ascending/descending order. Now suppose $\lambda_1(\rho)$ is a simple eigenvalue (the other cases can be handled analogously.) The essential idea is to consider the following matrix $\widehat\rho_t$ that is similar to $\rho+tA$: \begin{aligned} \widehat\rho_t &=(tc\oplus I_{n-1})(\rho+tA)(\frac{1}{tc}\oplus I_{n-1})\\ &=\pmatrix{ \lambda_1(\rho)+ta_{11}&t^2ca_{12}&t^2ca_{13}&\cdots&t^2ca_{1n}\\ \frac{a_{21}}{c}&\lambda_2(\rho)+ta_{22}&ta_{23}&\cdots&ta_{2n}\\ \frac{a_{31}}{c}&ta_{32}&\ddots&\ddots&\vdots\\ \vdots&\vdots&\ddots&\ddots&ta_{n-1,n}\\ \frac{a_{n1}}{c}&ta_{n2}&\cdots&ta_{n,n-1}&\lambda_n(\rho)+ta_{nn} } \end{aligned} It follows that when $\lambda_1(\rho)$ is a simple eigenvalue, then when $c$ is sufficiently large and $t$ is sufficiently small, the radius of each Gerschgorin disc of $\widehat\rho_t$ can be made arbitrarily small. Since $\lambda_1(\rho)$ is different from other $\lambda_j(\rho)$s, the first Gerschgorin disc of $\widehat\rho_t$ is disjoint from the others. But by the continuity of eigenvalues, the eigenvalues contained in the first Gerschgorin disc must approach $\lambda_1(\rho)$ when $t$ tends to zero. Hence that eigenvalue must be $\lambda_1(\widehat\rho_t)$. Now, by Gerschgorin disc theorem, $$ |\lambda_1(\widehat\rho_t)-\lambda_1(\rho)-ta_{11}| \le \sum_{j\ne1}|t^2ca_{1j}| $$ and the result follows.
