Inverse of partial derivative

Question

Good afternoon everyone,

I'm posting a similar but different question with respect to this post.

I have two vectors $a \in \mathbb{R}^M$ and $b \in \mathbb{R}^N$. Assume that I know the partial derivative $\frac{\partial a}{\partial b} \in \mathbb{R}^{M\times N}$ and, from that, I would like to know its inverse $\frac{\partial b}{\partial a} \in \mathbb{R}^{N \times M}$. Also assume that the functions are continuous and differentiable.

I would like to know if there is a relationship between the two maps because I know that, if both vectors are of the same dimensions, it is valid that $\frac{\partial b}{\partial a} = \left(\frac{\partial a}{\partial b}\right)^{-1}$. In my case, do I have to use the Moore-Penrose pseudoinverse? Or is there any more elegant solution?

Moreover, I personally think it is wrong to adjust the previous computation for vectors.

$\frac{\partial b_j}{\partial a} = \left(\frac{\partial a}{\partial b_j}\right)^{-1}$

for any component of $b$. Here I obtain a vector $\in \mathbb{R}^{1 \times M}$ that I should obtain with the pseudoinverse. I think that stacking these vectors ($j \in 1,\dots,N$) is different from the first solution I proposed at the beginning. I think this is wrong. Am I correct?

I also think that computing $\frac{\partial b_j}{\partial a_i} = \left(\frac{\partial a_i}{\partial b_j}\right)^{-1}, \forall i \in \{1,\dots,M\}, j \in \{1,\dots,M\}$ and composing properly the derivative is wrong. In other words: $\left(\begin{matrix} \frac{\partial b_1}{\partial a_1} & \dots & \frac{\partial b_1}{\partial a_M} \\ \dots & \dots & \dots \\ \frac{\partial b_N}{\partial a_1} & \dots & \frac{\partial b_N}{\partial a_M} \end{matrix}\right)$. Am I right that this is wrong?

Thank you in advance for any reply. Best regards, Neostek

P.S. just for knowledge, the reason for which the two vectors are of different dimensionality is that $a$ is a unit quaternion in $\mathbb{R}^4$ (even if it has a 1-dim constraint) while $b$ is a standard vector in $\mathbb{R}^3$.

Answer 1

First, I think you should be a little more precise: If you have two plain old vectors $a$ and $b$ of some dimension, then $\partial a/\partial b$ is either undefined or can be considered the zero matrix, because differentiation always needs a function that is being differentiated.

But I assume you meant something like: You have a differentiable function $b\colon \mathbb R^m \to \mathbb R^n$, so that you can consider the differential $d b(a)/da$. (I just write differential '$d$' instead of '$\partial$', because there are no other arguments to $b$ in my considerations here. But the same holds, if $b$ has more arguments).

You asked about '$\partial a/\partial b$' which I assume means something like: There is an inverse function $b^{-1}\colon U\subset\mathbb R^n \to \mathbb R^m$ to the function $b$ and you want to know something about $db^{-1}/dc$ with $c\in U$. But we know that in order for $db^{-1}/dc$ to exist, $b$ must be injective and $b^{-1}$ has to be differentiable. This means that $b$ (or its restriction to some subset of $\mathbb R^m$) must be a diffeomorphism. But this means that $\dim \mathbb R^m = \dim b(\mathbb R^m)$. If $n<m$ this is impossible. If $n=m$, you know what to do. If $n>m$ then you know that $\dim b(\mathbb R^m) = m$, so $b$ maps to an $m$-dimensional space. This must be a manifold. So you can apply the inverse function theorem for manifolds.

I'm sorry that it's not that easy.