0

Here is a code snippet wherein I add two convolution layers one with 3x3 filter followed by a layer with 1x1 filter. While I am sure how the parameters are calculated for 3x3 filter, I could not figure out the learnable parameter calculation for 1x1 convolution. Could someone help me understand?

model = Sequential()
inputShape = (227, 227, 3)
model.add(Conv2D(32, (3, 3), padding="same",
 input_shape=inputShape))
model.add(Conv2D(32, (1,1)))

Upon running model.summary(), the output is like:

Model: "sequential_2"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_5 (Conv2D)            (None, 227, 227, 32)      896       
_________________________________________________________________
conv2d_6 (Conv2D)            (None, 227, 227, 32)      1056      
=================================================================
Total params: 1,952
Trainable params: 1,952
Non-trainable params: 0

How did it arrive at 1056 parameters for the 1x1 convolution?

Oxbowerce
  • 7,472
  • 2
  • 8
  • 23
Ananth Subramanian
  • 1
  • The number of parameters is a sum of the number of weights (n_channels_in x n_channels_out = 32 x 32 = 1024) and the number of biases (n_channels_out = 32), which adds up to 1056. – Oxbowerce Jul 07 '21 at 16:16
  • @Oxbowerce mind posting this as an answer? – desertnaut Jul 07 '21 at 19:33

1 Answers1

0

The number of parameters in a convolutional layer can be calculated using the following formula: (filter_width * filter_height * n_filters_in) + 1) * n_filters_out. Filling in the numbers for your example we get ((1 * 1 * 32) + 1) * 32 = 33 * 32 = 1056, which aligns with the number you get from model.summary(). Of these 1056 parameters, 1024 (1 * 1 * 32 * 32) are the layer weights, the other 32 (1 * 32) are for the layer biases.

Oxbowerce
  • 7,472
  • 2
  • 8
  • 23