A (normalized) gradient vector $\hat{g}$ at a point $x$ defines a hyperplane. Given a vector $v$ the operator below reflects it against the hyperplane $$ R(v, \hat{g}) = v - 2\hat{g}\hat{g}^\top v $$
Now suppose I have a Jacobian matrix $J_f$ instead of a gradient vector. What is the generalization of this operation?
My Attempt
Let $f:\mathbb{R}^n\to\mathbb{R}^m$ be a function such that $f(x) = (f_1(x), \ldots, f_m(x))^\top$. Its Jacobian is a matrix $$ J_f(x) = \begin{pmatrix} \nabla f_1(x)^\top \\ \vdots \\ \nabla f_m(x)^\top \end{pmatrix} $$ Now each of its rows could be used to do a reflection with respect to that gradient $R(v, \widehat{\nabla f}_i(x))$, as shown in the picture above. However, I would like to generalize this operation to the whole Jacobian.
Idea 1
One way I thought of doing this is by using the reflection with respect to each of the coordinates sequentially. That is I keep on iterating $$ v \longleftarrow R(v, \widehat{\nabla f}_i(x)) $$ for $i=1, \ldots, m$. However the order in which you do this operation seems to matter.
Idea 2
Perhaps rather than reflecting against each gradient, one could sum up all the gradients, normalize and reflect against a "mean" gradient? That is one would compute $$ \overline{\nabla f}(x) = \sum_{i=1}^m \nabla f_i(x) $$ then normalize it and compute $R(v, \widehat{\overline{\nabla f}})$
$$ Now for $f:\mathbb{R}^n\to\mathbb{R}^m$ the Jacobian is a matrix $J_f\in\mathbb{R}^{m\times n}$. I am not sure what mathematical object this matrix defines. What do you mean by $\nabla h(x)^\top v$?
– Euler_Salter Jan 17 '22 at 16:08