6

I'm wondering how RNN can be used when doing sentiment analysis.

It seems that the characteristic of RNN is to remember what appeared in the past and determine the value of the present (future), but I think this RNN's characteristic is useless when analyzing sentiment.

I am curious to see how it is possible to do emotional analysis by remembering the words that have appeared so far.

I think the way to train vectors that show how many times specific words appear is the only way to analyze sentiment.

If some RNN model train a sentence "good movie" and label "positive", can it predict a new sentence "movie good" as a "positive"?

Also, if a normal model(just fully connected deep neural network) train a sentence "good movie" and label "positive", is it hard for this model to predict a new sentence "movie good" as a "positive"?

desertnaut
  • 1,988
  • 2
  • 14
  • 23
WooseokChoi
  • 61
  • 1

2 Answers2

4

RNNs do not learn to predict sentiment. They learn correlations between the input data and the target labels. If they see that every time the input contains the word "bad" they have to generate the label "negative", then they will learn it. If they see in the training data that the previous phenomenon happens always except when there is a "not" before "bad", then they will learn it. Depending on how the data distribution is, they may generalize the negation to any combination of verbs and adjectives, or maybe not, and they only handle negation appropriately when the input data is very very similar to the training data.

Your question seems to aim at understanding the "internal dynamics" of the RNN when it is predicting sentiment. While there is some research in that direction, I think they can still be considered as "black boxes", in that we do not actually understand the functions modeled by neural networks and therefore their outputs are not "explainable"

noe
  • 26,410
  • 1
  • 46
  • 76
1

It depends on how you configure your model and the layers you're using. You can add other layer's of DNN and experiment with those.

You should also consider using LSTM(Long short-term memory), since the past information do not get decayed quickly like in RNN, and you keep the present values, thus having a better model based on RNN.

You can also you a Bidirectional LSTM (BLSTM), which improve the learning process by using a cell state that runs through all your past data and generalize well for sentiment analysis.

Keep in mind that RNN cannot detect negation in sentences, like saying "This movie was not good", usually ANN need more configuration to understand those kind of aspects.

Another trick is to add a CNN layer to your model, so it can extract key words, thus when your data is forwarded to the RNN based layers, it will less noisy and more oriented to specific keywords.

MXK
  • 184
  • 1
  • 15
  • Why do you think that "RNN cannot detect negation in sentences"? – noe Oct 26 '20 at 10:44
  • @ncasas in NLP some concepts involving irony, sarcams, negations, jokes to name a few needs a lot of context information, and coding them is not straightforward. For example if you have the word "good" in sentence your model will likely to determine it as positive, and for the example "not as good as", what do you think you're model will classify it as? so you need to explicitly code some negation terms yourself, then even with a simple RNN you'll have a better reults. – MXK Oct 26 '20 at 12:00
  • @ncasas I agree with you that the ANN's are like black boxes, but in you choose a fairly simple dataset, then have a few layers and neurons, this way you can track down the ANN's behavior. – MXK Oct 26 '20 at 12:15
  • I don't think any natural language dataset can be simple enough to "track" it. Of course, you can measure how the model behaves under certain input patterns, but that won't enable you to reason about the model's behavior on unseen data, so any explanation you derive from the "tracking" would probably just be post hoc . – noe Oct 26 '20 at 12:44
  • 1
    And the statements RNN cannot detect negation in sentences and in NLP some concepts involving irony, sarcams, negations, jokes to name a few needs a lot of context information, and coding them is not straightforward are very different. I agree with the second one and strongly disagree on the first one. – noe Oct 26 '20 at 12:46