3

In a typical neural network , bias is usally added like this

 v = activation ( w1*x1+ ... + Wb*b)

However, I am not really sure how it is done in convolutional layer. My one thought is that it is added with each convoltional operation for a neuron. Is that correct?

user58529
  • 31
  • 1
  • 2

1 Answers1

2

There are two ways bias is usually added to a convolutional layer:

  • Tied bias: where you share one bias per kernel

  • Untied bias: where you use use one bias per kernel and output

Check this blog post comparing the two.

Other than that, bias isn't really necessary in CNNs. There are many popular implementations which ignore it altogether (e.g. ResNet-152).

Djib2011
  • 7,968
  • 5
  • 27
  • 37