-1

What is the state of the art of transforming input data for neural networks? They need to have constant width, and that's why I'm trying to wrap my head around that.

Let's say that we want to classify some books (for example, into some categories). Books have many attributes of different type, like:

  • short strings (title)
  • longer strings/documents (description)
  • dates (publishing date, author's birth date)
  • simple arrays (authors)
  • longitude/latitude (place where the book was finished, author's birth place)

How can one handle these attributes? I've read already a little about handling long strings here, but the rest, in especially small arrays of attributes are a mystery for me.

Świstak35
  • 99
  • one way to solve this is to use recurrent networks (like LSTM/GRU) instead of feedforward networks, so you process character by character (or block of character). Then you take the output of the last (block) character(s) as classification output – Thomas Wagenaar Jun 28 '17 at 12:12
  • Yes, I know about RNN, but they seemed to me like best fit for the "streams" of data, like audio, texts, character stream and such. Aren't they not so good for only simple array structures? I'm talking about, for example books, which have only a few authors, or something like that. Not that order of magnitude, I feel, to use the RNNs – Świstak35 Jun 28 '17 at 13:23
  • You need to treat each case separately. Strings of any length can be embedded into fixed-length vectors using doc2vec. Dates can be represented as tuples of (epoch time, day of week, time of day, etc.) Longitude/latitude can be represented in various forms; as is, polar, Euclidean, or discretized into (country, city), etc. The authors can be a bit set. – Emre Jun 29 '17 at 02:52

1 Answers1

-1

There is a trend towards implementations that don't need input sizes known in advance. Check out DyNet or Chainer for example.

From DyNet's technical paper:

In DyNet's dynamic declaration strategy, computation graph construction is mostly transparent, being implicitly constructed by executing procedural code that computes the network outputs, and the user is free to use different network structures for each input.

Ronen

RonenKi
  • 101
  • 1