Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [rnn]

A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle.

744 questions
15
votes
1 answer

How many LSTM cells should I use?

Are there any rules of thumb (or actual rules) pertaining to the minimum, maximum and "reasonable" amount of LSTM cells I should use? Specifically I am relating to BasicLSTMCell from TensorFlow and num_units property. Please assume that I have a…
user27994
3
votes
1 answer

Difference between Jordan, Elman and normal RNN

As far as I know for history, the Jordan network was proposed first in 1986 as a form of RNN with this diagram: Actually, this is the solution that makes sense when thinking about sequence data that the current output is an input in the next time…
Osama El-Ghonimy
  • 241
  • 3
  • 6
2
votes
1 answer

How to compare the complexity of different RNN cells?

I want to compare three different types of RNNs to decide which architecture can handle my data best. To do that, I want them to have the same complexity. Can I simply define the complexity by the number of trainable parameters? If not, why? Suppose…
Gilfoyle
  • 141
  • 4
2
votes
1 answer

Neural Network - distinguishing between several normalized values is impossible?

It's a common practice to normalize inputs to the neural Network. Let's assume we have a vector of activations. One of techniques, the Layer Normalization simply looks at the vector's components, re-centers this activated vector from $\mu$ to…
Kari
  • 2,726
  • 2
  • 20
  • 49
1
vote
1 answer

Number of parameters in Simple RNNs

Please, I am stuck, I can not understand the number of parameters of a simple RNN, here the example and the model summary. the example is simple: x = np.linspace(0,50,501) y= np.sin(x) df= pd.DataFrame(data=y, index=x, columns=['Sinus']) Then I…
Abir ELTAIEF
  • 11
  • 3
1
vote
1 answer

How does "one-to-many" RNNs work?

I recently came across an article about RNNs here. Which describes different types of RNNs like: The first figure makes sense. A regular feedforward network. The second is a big question for me. Is it one timestep cloned three times to make it…
Gergő Horváth
  • 109
  • 7
1
vote
1 answer

Clarify recurrent neural networks

I'm in the beginning to learn and understand recurrent neural networks. As far as I can imagine, its multiple feed-forward neural networks with one neuron at each layer put next to each other, and connected from left to right, where each neuron is…
Gergő Horváth
  • 109
  • 7
1
vote
1 answer

how to apply feature selection on LSTM-RNN?

am doing my research using lstm-rnn algorithm. i have time-series and non time-series features. how to apply lstm on my dataset? and also how to apply feature selection mechanism to select features?
Meron
  • 11
  • 2
1
vote
1 answer

Why cant RNN learn long term dependencies=?

In Colah's blog, he explain this. In theory, RNNs are absolutely capable of handling such “long-term dependencies.” A human could carefully pick parameters for them to solve toy problems of this form. Sadly, in practice, RNNs don’t seem to be…
abdoulsn
  • 165
  • 1
  • 5
1
vote
1 answer

Where can I download the toy benchmark dataset for RNNs?

I have read the paper: Simple Way to Initialize Recurrent Networks of Rectified Linear Units Where can I download the toy benchmark dataset for RNNs this paper mentions? I need addition problem benchmark.
Yan-Jen Huang
  • 113
  • 2
1
vote
0 answers

Catching hidden state activations of an RNN on prediction.

I'm interested in doing research similar to the work done in this blog, The Unreasonable Effectiveness of Recurrent Neural Networks by Andrej Karpathy. Specifically, where the author explores the rules that individual neurons have learned by…
Jay Speidell
  • 581
  • 4
  • 13
1
vote
1 answer

LSTMs - Carousel of Error

Is there any other materials that derives the LSTM back propagation and carousel of error except the original paper? (I could not understand it, sorry). I tried deriving and got stuck, and asked the following question: LSTMs - Data Science Stack…
user1157751
  • 689
  • 1
  • 8
  • 22
1
vote
0 answers

LSTM for multidimensional data

Is there any ML framework that readily supports LSTM for multi-dimentionsal data ? Best would be in Pytorch. But I don't think the official API supports it and I have found an implementation.
Quan Vuong
  • 141
  • 3
0
votes
1 answer

Data preprocessing for time series prediction

I have a dataset that has the following structure [ [ [ product 1 ,shelf number, position on the tray, time of stay on the shelf, was sold?], # Hour 1 [ product 1 ,shelf number, position on the tray, time of stay on the shelf, was sold?], #…
skrrrt
  • 304
  • 2
  • 13
0
votes
1 answer

About batches in stateful RNN

..., to create proper consecutive batches, where the nth input sequence in a batch starts off exactly where the nth input sequence ended in the previous batch. Géron, Aurélien. Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow…
Crispy13
  • 133
  • 4
1
2