How to deal with class imbalance in a neural network?

Question

Suppose we have a game and its action space contains two possible actions: A and B.

We have a labelled dataset of state-action pairs but 95% of actions are A and only 5% are B

If we train a neural network, it will always output A as it will choose the most probable class to decrease its loss.

Are there ways to solve this problem?

One approach would be to "oversample" the minority class, e.g. for each sample from A that is shown to the network, 10 samples from Bare shown. — Peter, May 07 '20 at 12:07
@ValentinCalomme For a classifier we can split our data and make a balance between two classes but if we have RL problem it is harder to split the data. suppose we have a continuous q-table and we can't manipulate it. can we use a custom loss function that it is more sensitive to B or using different network architecture. I trained a network on such a problem like this and it's output always was the first class. i just speculate it that network consider the second class as noise. — amin msh, May 07 '20 at 20:03

score 3 · Accepted Answer · edited Dec 19 '23 at 09:28

3

This is for classification, and I am not sure if it is possible to extend them to reinforcement learning.

As you figured out, accuracy should not be used as a metric for a dataset as imbalanced as the one you have. Instead, you should look at a metric such as Area Under Curve(AUC). If you would have infinite data, then you could just rebalance and remove some of the data from the class that has the most samples. However, in many cases data is sparse and you want to use as much of it as possible. Removing data can have a disastrous effect on many applications.

So what are good and convenient ways of handling this?

Add weights to the loss function. One weight for class A and one for B. By increasing the magnitude of the loss for the B class the model should not get stuck in a suboptimal solution that just predicts one class.
Use another objective(loss) function. F1-score can, for example, be implemented and used as an objective(loss) function. For a differentiable version see https://stackoverflow.com/a/65320239/12229416

What is great with these approaches is that it will allow you to use all the data.

edited Dec 19 '23 at 09:28

Ggjj11

216
2
6

answered May 10 '20 at 00:04

Carl Rynegardh

876
6
16

Thank you for the answer. I have one question, can i use conditional loss function instead of weighted loss function? for example use two loss function and check the the true output and choose the loss for this case with if statement. – amin msh May 12 '20 at 11:21
Suppose there is three possible actions A, B and C , if we always output A the loss doesn't change a lot but by choosing B and C make the loss more sensitive to true or wrong prediction. – amin msh May 12 '20 at 11:26
Yes! So then you would multiply the loss with a different weight depending on the ground truth label. – Carl Rynegardh May 14 '20 at 12:04
@CarlRynegardh "F1-score can, for example, be implemented and used as an objective(loss) function". This isn't true. F1-score is a non-differentiable function so I don't see how you could use it as an objective/loss function in a neural network. -1 – ATP Jun 13 '21 at 17:52
@ATP it can be approximated (but still a bad idea to use this). – Lucas Morin Dec 19 '23 at 09:27

noe · Answer 2 · 2020-05-11T14:36:38.060

You have tagged the question with reinforcement-learning, but you describe a labeled dataset, suggesting supervised learning. I will try to cover both cases.

There are some techniques that are applicable in both supervised learning and reinforcement learning:

Sample from the buffer conditioning on the action, to have a balanced dataset regarding action taken.
Apply data augmentation techniques on the minority action class. The Synthetic Minority Oversampling TEchnique (SMOTE) algorithm may be an option for that. The problem with data augmentation is that you would need to do it in the RL loop, which can enlarge the needed computation time.

Note that to apply them for reinforcement learning, you should use a replay buffer, like they do in the DeepMind Atari paper.

If you are in a supervised learning scenario, you can apply class weights, e.g. this example in Keras.

score 2 · Answer 3 · answered May 14 '20 at 11:05

As people have mentioned above you want to try and up-sample / bootstrap. In other words you want to try and get the classes to have similar proportions. One way to do this is to simply randomly select the less likely sample.

More complicated solutions: 1. involve adding realistic noise to the less likely class to increase the number of data points. 2. Using a different score/error function - look at balanced accuracy 3. Initiate the training with 50% A and 50% B - once it converges start training it gradually on a larger part of the data set which will gradually become 95% A and 5% B.

score 0 · Answer 4 · answered May 11 '20 at 14:14

For Imbalanced classes, the method which I prefer the most is bootstrapping.

Lets say you have n classes with number of examples as m , 2m, 3m (this is just to tell which is the minimum).
create multiple dataset with m samples from each classes. (randomly)
keep training on each one of them .

How to deal with class imbalance in a neural network?

4 Answers4

Linked