Automatic differentiation can be accomplished using forward or reverse accumulation.
Quoting Wikipedia :
which mode is used in DL frameworks is used for implementation and why? Does it have any motivation from the issue of complexity as in the above article?
