I wish to compute the derivative and the Hessian of the function ${\bf f}(X)$ where
$$ {\bf f}({\bf X}) = {\bf X} \, {\bf a}, $$
${\bf X}$ is an $(m \times n)$ matrix and ${\bf a}$ is a vector of constants of size $n$. From "Matrix Differential Calculus" by Magnus and Neudecker, it is relatively straightforward to obtain the derivative and the Hessian of ${\bf f}({\bf X})$ w.r.t. ${\bf X}$. For example, the first differential is
\begin{align} \partial {\bf f}({\bf X}) &= (\partial {\bf X}) {\bf a} = {\rm vec} (\partial {\bf X}) {\bf a} = ({\bf a}^\prime \otimes {\bf I}_m) \,\partial {\rm vec} {\bf X} \\ \end{align}
In my case however the matrix ${\bf X}$ has a special structure. Specifically, I know that that ${\bf X}^\prime {\bf X}$ is a diagonal matrix, say, ${\bf X}^\prime {\bf X} = {\rm diag}(d_1, \ldots, d_n)$, which is not necessarily equal to the identity matrix. In other words, I know that the columns of ${\bf X}$ are mutually orthogonal, but each column vector can be of any length. How do I compute the derivative and the Hessian of $f({\bf X})$ w.r.t. ${\bf X}$ to take this structure into account?
Any references on how to proceed would be greatly appreciated.
I could not find a similar question posted before. The closest I found was Hessian of $f(X)$ when $X$ is a symmetric matrix, but I was not able to apply it to my problem.