In the question of What is the relationship between the accuracy and the loss in deep learning?, @Jérémy Blain gave a fantastic interpretation of 'relationship' between accuracy and loss:
- 1 - low accuracy and big loss means you made huge errors on a lot of data
- 2 - low accuracy but small loss means you made little errors on a lot of data
- 3 - high accuracy with small loss means you made low errors on a few data (best case)
- 4 - high accuracy but a big loss, means you made huge errors on a few data.
As I understand,
- 1 - implies bad algorithm 'in most of time'
- 2 - implies overfitting
- 3 - is what we want
Questions are:
1 Any other reasons caused 1 and 2?
2 what will lead to case 4 : high accuracy but a big loss?
