0

I am a newbie here and trying to make sense out of the scores from model.evaluate from what I am actually seeing in model.predict

I have a created a CNN model for the Google Audio Set data and achieved a 99%+ accuracy on training.

Here is how I do the prediction

model = load_model('model_audioset.h5')
for x, y in unbal_generator:
  score = model.evaluate(x, y, verbose=0)
  pred_y = model.predict(normalized_x)

Here is what I am seeing for one specific iteration of x and y from model.evaluate

model.metrics_names = {list: 2} ['loss', 'acc']
 0 = {str} 'loss'
 1 = {str} 'acc'

score = {list: 2} [0.03851451724767685, 0.9905123114585876]
 0 = {float64} 0.03851451724767685
 1 = {float64} 0.9905123114585876

Here is a readable output from model.predict and comparing it to y

Actual vs. Expected

Wondering how Keras came up with an accuracy score of 99.05 for this output? Clearly, the predicted classes are not the same as expected

I am using a binary_crossentropy loss function and sigmoid activation in the predictions layer as classes are NOT mutually exclusive

infinity
  • 111
  • 2

1 Answers1

1

Actually, I think accuracy is not a good metric for this case as y_actual and y_expected have most values 0 and the length of y_actual is quite big too.

So the accuracy calculation using an equation like K.mean(K.equal(K.argmax(y_true, axis=-1), K.argmax(y_pred, axis=-1))) (Reference) is not a good metric on model performance

infinity
  • 111
  • 2