0

This thread develops the partial derivative of cosine similarity between two vectors.

Given a scalar $k$ and the cosine similarity function

$$cossim (\mathbf{v},\mathbf{w}) = \frac{{\mathbf{v} \cdot \mathbf{w}}} {{\left| {\mathbf{v} } \right|\;\left| \mathbf{w} \right|}}$$

how can we find the following partial derivative?

$$\frac{\partial}{\partial{\mathbf{v}}}|k - cossim(\mathbf{v},\mathbf{w})|$$

My attempt based on Andrew L's comment is as below. Is it correct?

$$y := k - cossim(\mathbf{v},\mathbf{w})$$

$$\frac{\partial}{\partial \mathbf{v}}|y| = \frac{y}{|y|} \frac{\partial y}{\partial \mathbf{v}} $$,

and from the accepted answer in above thread:

$$\frac{\partial y}{\partial \mathbf{v}} = - \frac{\partial}{\partial \mathbf{v}} cossim(\mathbf{v}, \mathbf{w}) \\\ = - \frac{\mathbf{w}}{|\mathbf{v}|\cdot{|\mathbf{w}|}} + cossim(\mathbf{v},\mathbf{w})\cdot{\frac{\mathbf{v}} {|\mathbf{v}|^2}}$$.

When put together, the solution is:

$$\frac{\partial}{\partial \mathbf{v}}|k - cossim(\mathbf{v},\mathbf{w})| = \frac{k - cossim(\mathbf{v},\mathbf{w})}{|k - cossim(\mathbf{v},\mathbf{w})|} ( cossim(\mathbf{v},\mathbf{w})\cdot{\frac{\mathbf{v}} {|\mathbf{v}|^2}} - \frac{\mathbf{w}}{|\mathbf{v}|\cdot{|\mathbf{w}|}}) $$

bzt3428
  • 1
  • 1
  • Question: what is $v$ in this case? If you mean a component of $\mathbf{v}$ you should specify that. Barring that, if $k$ is nothing more than a constant, then you can easily just use the chain rule. The derivative of $|x|$ isn't hard, it's just $\frac{d}{dx}|x|=\frac{x}{|x|}$, so by the chain rule, $\frac{d}{dx}|u|=\frac{u}{|u|}\frac{du}{dx}$ – Andrew L May 07 '21 at 03:49
  • $v$ was a typing error and is now replaced with $\mathbf{v}$ in the original question. I also added an attempted solution based on your comment. Thank you. – bzt3428 May 07 '21 at 19:45

0 Answers0