1

Let's assume I have 2 models

Model 1:

  • Train Accuracy = 92.4%
  • Validation Accuracy = 37.6%
  • Test Accuracy = 35.3%

Model 2:

  • Train Accuracy = 37.0%
  • Validation Accuracy = 34.2%
  • Test Accuracy = 34.1%

Which is the best model ? Model 1 is heavily overfitting but the final performance is better

Daniel_Fortesque
  • 33
  • 5
  • Oof, I forgot that I can't just propose a duplicate now. Let me know if the linked question doesn't help, and I'll reopen. – Ben Reiniger Sep 07 '22 at 13:35

1 Answers1

0

Deep learning models heavily rely on stochastic processes such as weight initialization, back-propagation, etc. For evaluation and comparison of different models, there are methods that are generally referred to as Cross-Validation. The most popular type of CV is the k-Fold CV, and if your model training comprises hyperparameter tuning, you must use the Nested k-Fold CV.

Amirhossein Rezaei
  • 572
  • 2
  • 13