For questions about the back-propagation (aka "backprop", and often abbreviated as "BP") algorithm, which is used to compute the gradient of the objective function (e.g. the mean squared error) with respect to the parameters (or weights) of the neural network, when trained with gradient descent.
Questions tagged [backpropagation]
282 questions
1
vote
1 answer
Please, could someone help me understand if the backpropagation explanations in these two articles about calculate the error are equivalent?
I have a question about backpropagation, I'm a beginner, I'm studying the formulas to calculate the delta of neurons, there are several sources on the internet, which teach in different ways, so I'm confused about the formulas presented in the…
will The J
- 267
- 6
1
vote
1 answer
what is the backpropagation formula to calculate delta and update weights?
I'm trying to study how backpropagation works step by step in a MultiLayer Perceptron neural network. I would really like to be able to understand how these calculations work. And I have a specific question I would like to ask. The formulas I'm…
will The J
- 267
- 6
1
vote
2 answers
Back propagation activation function derivative
I am reading about backpropagation for fully connected neural networks and I found a very interesting article by Jeremy Jordan. It explains the process from start to finish. There is a section though that confused me a bit. The partial derivative of…
Kyriafinis Vasilis
- 13
- 4
1
vote
1 answer
Parallelize Backpropagation - How to synchronize the weights of each thread?
I implemented a parallel backpropagation algorithm that uses $n$ threads. Now every thread gets $\dfrac{1}{n}$ examples of the training data and updates its instance of the net with it. After every epoch the different threads share their updated…
CptK
- 11
- 1
0
votes
2 answers
Does this article make use of the chain rule? And where?
References:
Chain rule in Wikipedia: https://en.wikipedia.org/wiki/Chain_rule
Chain rule in Towards Data Science:…
will The J
- 267
- 6
0
votes
1 answer
About the requirement to compute the gradient at layer $l$
I'm trying to understand a line of my note.
Let's say there is a simple feedforward neural network that has $N$ layers, and for a given layer $l$, it has weight $W^l$, and $g^l$ is the gradient to update it. Now the problem is:
From the Wiki page…
NRain
- 121
- 5
0
votes
1 answer
How would I use backpropagation with a changing cost function
I have a neural network that is being trained with a changing cost function. Could I use backpropagation at all? If yes, how would I do this?
922073
- 11
- 2
0
votes
1 answer
Is it possible to reverse engineer out the loss based on weights update when data is unknown?
Assume the gradient updates (both $W_t$ and $W_{t+1}$) and learning rate are known while data $X$ is unknown, is it possible to deduce the loss $L$ used in backprop algorithm that gave rise to the gradient update $W_{t+1} - W_{t}$? If not, is it…
Sam
- 185
- 5