0

In many research papers there are 'projection layers' related to BLSTM layers. For example, from here:

"we trained an 8-layer BLSTM encoder including 320 cells in each layer and direction, and the linear projection layer with 320 units followed by each BLSTM layer"

I can't understand what this means and how it works.

Any help on this topic would be appreciated. Thanks in advance!

Ethan
  • 1,633
  • 9
  • 24
  • 39
Selma_KA
  • 13
  • 1

1 Answers1

0

A "projection" is a simple linear/dense layer, that is, a matrix multiplication and a bias vector addition.

It is called projection because you "project" a representation of dimensionality $M$ into a representation space of dimensionality $N$.

Sometimes, especially for sequences or 2D data, these projections are implemented as a convolution of size 1, which is equivalent to the computations I described above.

noe
  • 26,410
  • 1
  • 46
  • 76
  • Thanks for your answer, but I have another question because it's a little bit ambiguous for me, are they used for the dimensionality reduction? – Selma_KA Dec 24 '20 at 14:19
  • A projection can be used to decrease the dimensionality of the representation, but it can as well be used to increase the dimensionality. – noe Dec 24 '20 at 14:22