0

My training data consists of 3 variables X1, X2, X3 such that

Y = (X1 * X2 / 1e6) * X3

Example of data

X1 X2 X3 Y
310 768 952 0.2267
157 889 860 0.1200
157 787 610 0.0754
97 385 118 0.044

using data size of 80000 rows trained over 500 epochs with early stopping.

I structured the network like so

model = Sequential()
model.add(Dense(32, input_dim=3, activation='relu'))
model.add(Dense(16, input_dim=3, activation='relu'))
model.add(Dense(8, input_dim=3, activation='relu'))
model.add(Dense(1, activation='linear'))

model.compile(loss='mean_squared_error',optimizer='adam')

However, after training the output network were unable to predict even seen training examples.

Training loss stalled at about 0.0145

model.predict([
    [310, 768, 952], 
    [157,787,610],
    [1,1,1],
    [5,5,5]
])

output:


array([[0.07321107],


[0.07321107],


[0.07321107],


[0.07321107]], dtype=float32)

Which is wrong. Wondering what should I do to correct this?

40pro
  • 111
  • 4
  • Have you tried smaller learning rate (particularly if and when the loss plateaus)? – learner Jan 10 '23 at 05:10
  • Okay I will try that. Thanks. – 40pro Jan 10 '23 at 07:16
  • seems like the model didn't learn much. are you sure you aren't stopping too early ? – Lucas Morin Jan 10 '23 at 08:50
  • I tried removing early stop callback, seems to have no impact on loss. Though, reducing learning rate seems to have good effect. It gives output that is closer to expected output. – 40pro Jan 10 '23 at 09:46
  • 1
    The next to try is to add more neurons to the hidden layers. In the meantime, I want to still emphasize the importance of having a sufficiently small learning rate and monitoring the loss-vs-epoch variations. – learner Jan 10 '23 at 12:19
  • Related: https://datascience.stackexchange.com/q/47787/64377 (check the most upvoted answer). – Erwan Jan 10 '23 at 19:42
  • Tried changing neuron count to 64 --> 1024 --> 1. Seems okay. But whats good rule of thumb for number of nueurons. – 40pro Jan 11 '23 at 04:00

0 Answers0