Mathematics Stack Exchange
Mathematics Stack Exchange

Questions tagged [matrix-calculus]

Matrix calculus is about doing calculus, especially derivative and infinite series over spaces of vectors and matrices.

Matrix calculus studies derivatives and differentials of scalar, vector and matrix with respect to vector and matrix. It has been widely applied into different areas such as machine learning, numerical analysis, economics etc.

There are basically two methods.

  • Direct: Regard vectors and matrices as scalar so as to compute in the usual way in calculus. And The Matrix Cookbook provides a lot of basic facts.

  • Component-wise: Write everything in indices notation and compute in the usual way componentwisely. Einstein summation convention is frequently used.

3828 questions
8
votes
3 answers

Gradient of a matrix?

I was following Stephen Boyd's convex optimisation course and came across the following slide: Can somebody explain to me how the gradient was calculated for the quadratic and least-squares objective. Is there a general method to find the gradient…
humble
  • 430
7
votes
1 answer

derivative of inverse matrix by itself

Let $A$ be a matrix, supposedly $k\times k$ matrix. I know that $$\frac{\partial A^{-1}}{\partial A} = -A^{-2} $$ I do not know how I am supposed to obtain the following results using this fact. I want to know the step of $$\frac{\partial a^\top…
user1292919
  • 1,895
6
votes
2 answers

What is $\frac{\delta\log(\det(X))}{\delta X}$ where $X$ is a positive defnite $2\times2$ matrix?

I am reading a book and it says that $\frac{\delta \log(\det(X))}{\delta X} = 2X^{-1}-diag X^{-1}$ where $X$ is a $2\times 2$ positive definite matrix. However I have computed that $\frac{\delta \log(\det(X))}{\delta X} = (X^{-1})^T$. Now the book…
user276611
5
votes
1 answer

What is the first order approximation to a differentiable function $F:M_{n\times n}(\mathbb{R})\to\mathbb{R}$

Given a differentiable function $$F:M_{n\times n}(\mathbb{R})\to\mathbb{R}$$ How am I to interpret its first order approximation? The derivative of a real valued function of a matrix is a matrix valued function of a matrix. So what I did was treat…
Set
  • 7,600
4
votes
1 answer

Meaning of power matrix function

Suppose $t\in \mathbb{C}$, $\Lambda=\begin{bmatrix} 2 & 0 \\ 0 & 1 \end{bmatrix}$, what does $t^\Lambda$ mean? Is it $\begin{bmatrix} t^2 & 0 \\ 0 & t \end{bmatrix}$?
Dasheng Wang
  • 899
4
votes
1 answer

Derivative with respect to Symmetric Matrix

I realize that derivatives with respect to symmetric matrices have been well covered in prior questions. Still, I find that the numerical results do not agree with what I understand as the theory. Minka states (page 4) that for a symmetric matrix…
jds
  • 237
  • 1
  • 8
4
votes
1 answer

Derivative of sigmoid assuming input is a matrix

I do know how to calculate the derivative of sigmoid function assuming the input is a scalar. How to properly derive the derivative of sigmoid function assuming the input is a matrix - i.e. using matrix calculus? The fraction (a sort of division)…
Szpilona
  • 151
4
votes
1 answer

Operations order in matrix calculus

I have just started learning about matrix and tensor calculus. I was introduced to different products: Kronecker, Hadamard, Frobenius, row-wise products and obviously standard matrix and scalar multiplications. I am not sure yet what is the order in…
Szpilona
  • 151
4
votes
1 answer

Gradient of quadratic forms involving matrix powers

Let $f:\mathbb{R}^{n \times n} \to \mathbb{R}$ be defined as: $$ f(A)= x^T (A^2)^i y + v^T A^i w, $$ where $i \in \mathbb{N}$ and $x,y,v,w$ are some fixed column vectors. One can assume that $A$ is a symmetric matrix. I am interested in computing…
pikachuchameleon
  • 2,809
4
votes
1 answer

A contraction operator shows some commutivity

Let $A$ be a $n\times n$ matrix, we define its operator norm (or just spectral norm) as $$||A||=\max_{||x||=1}||Ax||.$$ If $||A||\leq 1$, we say $A$ is a contraction. Show that $$||A||\leq 1\Rightarrow A^*(I-AA^*)^{1/2}=(I-A^*A)^{1/2}A^*.$$ This…
xldd
  • 3,485
3
votes
3 answers

Derivative of the inverse of a symmetric matrix

I am quite a beginner in linear algebra and matrix calculus. I was wondering what is the derivative of the matrix inverse when the matrix is symmetric. More precisely, I'm looking for $\frac{\partial}{\partial \mathbf{X}} \mathbf{X}^{-1}$ when…
MikeL
  • 627
3
votes
2 answers

Derivative of matrix-value function in integral representation

I'm learning about matrices and matrix calculus. In Matrix Monotone Functions and Convexity. In: Introduction to Matrix Analysis and Applications there is written that integral representations of matrices are often helpful with calculating…
Barabara
  • 690
3
votes
1 answer

Divide two matrices

Cosider the two following matrices: \begin{align} E=\begin{bmatrix} e_{11}\;\; e_{12} \\ e_{21}\;\; e_{22} \\ \end{bmatrix} \end{align} and \begin{align} X=\begin{bmatrix} x_{11}\;\; x_{12} \\ x_{21}\;\; x_{22} \\ \end{bmatrix} \end{align} I want…
Michael
  • 131
3
votes
1 answer

Gradient of $f(x) = a \oslash (b + a \odot x)$ w.r.t. $x \in \mathbb{R}^n$

How to compute the gradient of $f(x) := a \oslash (a \odot x + b) $, with respect to $x \in \mathbb{R}^n$, where $\oslash$ is element-wise division, $\odot$ is element-wise multiplication, and $a , b \in \mathbb{R}^n$. The gradient should be a…
learning
  • 661
3
votes
2 answers

Eigenvalues of the absolute value of the matrix $A$

Can we say that there is a relationship between the eigenvalues of the matrix $A$ and its absolute value as $B$, where $b_{ij}=|a_{ij}|$? Consequently, can we say that there is a relationship between the spectral radius of them?
M. Raha
  • 69
1
2 3
…
13 14