Derivative of transpose of matrix

Question

Let $X$ be a $n*n$ matrix with its entry indices increasing along every column.For example,when $n = 2$, $X =\left( \begin{matrix}x_1 & x_3\\ x_2 & x_4\end{matrix} \right)$.Let $\rm{vec}(X)$ denotes puting every column of X (one by one) into a new vector.It is easy to see that $\rm{vec}(X)$ is a vector with $n^2$ elements.So is there any closed-formed formula (like the result of kronecker product) for $\frac{d\rm{vec}(X^T)}{d\rm vec(X)}$?I have already calculated the cases for $n=2$ and $n=3$.For $n=2$,the answer is $\left( \begin{matrix}1&0&0&0\\0&0&1&0\\0&1&0&0\\0&0&0&1\end{matrix} \right)$.I have seen this question Derivative of transpose of a matrix which seems to be unsolved.And also this one What is the derivative of a vector with respect to its transpose? where X is a vector instead of a matrix. I attempted to use this formula: $\rm{vec}(X^T) = (X \otimes I)\rm{vec}(I)$,but I don't know how to get the derivative of the right side.

Answer 1

$X↦X^⊤$ is a linear map and so equal to its own derivative.

A better question to ask is can we express this function in the standard form of a linear map, which is as a matrix-vector product in the case of a function $ℝⁿ→ℝᵐ$. Here, we instead have a function $ℝⁿ⊗ℝᵐ → ℝᵐ⊗ℝⁿ$, which can be encoded as a $(ℝᵐ⊗ℝⁿ)⊗(ℝⁿ⊗ℝᵐ)^*$ tensor. This 4-d tensor $_{ij, kl}=δ_{kj}⋅δ_{il}$ is referred to as the transpose tensor.

$$ X^⊤ = (X_{ji})_{ij} = \Big(∑_{kl} \underbrace{δ_{kj}⋅δ_{il}}_{=_{ij, kl}} ⋅ X_{kl}\Big)_{ij} = _{ij, kl}X_{kl} = ⋅X$$

where the last $"⋅"$ is the tensor-contraction along the last 2 axes.

Answer 2

The set of real $n \times n$ matrices forms a real vector space with respect to the usual addition of matrices. Your identification of this space with the canonical vector space $\mathbb{R}^{n^2}$ is a linear isomorphism between vector spaces.

You can quickly check that the transposition map on $\mathbb{R}^{n \times n}$ is a linear automorphism. Thus, the derivative of the map does not only exist for every point (it is constant), but the Jacobian coincides with the matrix representing the transposition map with respect to a chosen basis.

Thus, you find the Jacobian of the map once you have found a matrix representation of your map with respect to, for instance, the canonical basis.