I just discovered recently this tool of automatic differentiation. I was wondering when we use forward AD and when we use backward AD. As I understand it forward AD is nice for computing directional derivatives (just one traversal of the AD graph) and backward AD is nice for computing gradients (just one backward traversal of the AD graph). Is that right? Of course Once we have the gradient from the backward AD we can compute the product with the direction to get the directional derivative.
Asked
Active
Viewed 511 times
1
-
See the accepted answer here, https://math.stackexchange.com/a/3119199/280172 "duplicate"!! – I.Omar May 03 '21 at 13:13