How to choose the best optimiser in Deep Learning algorithms?

Question

I have been training a CNN to classify 4 faulty (acoustic emission/250KhZ) signals. I have no problem in implementing the algorithm using tensorflow libraries, but I am confused with which optimizer to choose. I have been trying out different optimizers, for Adam optimizer and the gradient descent optimizer algorithm gave the same results in test classification accuracy(~84% to 85%) on k-fold cross-validation. Results are same for both optimizers, is it because of the data or because of training on huge data(64000 samples/fault condition)?

Are there any specific guidelines to choose a particular optimizer for kind of data? Or else, is it randomly by trial and error?

Answer 1

I found this helpful.

I usually stick with Adam, and rarely attempt SGD. Simple words: try to go with the recent optimiser implemented on popular datasets and that has been implemented by the framework which claims to be better than others.

To specifically say that this optimiser is best for this kind of data is subjective, and I haven't read any relevant material related to that. In future, If anyone finds it, please do comment or edit this post.

Hope this helps.