2

I have about 6 months of experience in building and using Neural Networks with no prior/formal training. As I explore this field further, I see a lot of discussions about determining how many layers/neurons to use and some rules of thumb of where to start. In development of my network models, I use a brute-force approach by incrementing the layers and neurons by 1 for several cycles of epoch training and then select the "best" model out of those. My understanding at this point, is that the layers/neurons represent the relationship between the NN inputs and the NN outputs, and while training my networks, I can determine the optimal number of layers to be 5 or more. I determine optimal based on using cross validation data the network was not trained on.

Given the above, I have read from various sources statements like "you never need more than 2 layers" and "you don't get better performance out of more than 2 layers", etc. I have also read comments that indicate more than 2 layers is expected. Is the idea of 2 layers could be "too much" really correct and I should be focusing on expanding the number of neurons used while capping the number of layer at 2 for my brute-force determination of optimal layers?

EDIT:

Is it is true that in the vast majority of cases no more than 2 layers is warranted as it seems some are claiming? My specific interest is in the area of numeric prediction, however, comments about other segments are welcome as well.

user7226068
  • 61
  • 3
  • Noted, this was one of the posts I read that implies using more than 2 layers is valid, however, the underlying question is if it is true that in the vast majority of cases no more than 2 layers is warranted as it seems some are claiming. – user7226068 Sep 23 '18 at 14:26
  • The #layers depends on your needs... – Aditya Sep 23 '18 at 14:33
  • 1
    The general conclusion that "you never need more than 2 layers" for neural network is absolutely incorrect. In computer vision, almost every modern NN model uses way more than 2 layers: VGG, Inception, AlexNet, ResNet, you name it. In some specific area, it might be true (although still unlikely). – user12075 Sep 24 '18 at 01:05
  • Here is the REAL answer to my question: https://www.heatonresearch.com/2017/06/01/hidden-layers.html: "Problems that require more than two hidden layers were rare prior to deep learning. Two or fewer layers will often suffice with simple data sets. However, with complex datasets involving time-series or computer vision, additional layers can be helpful." - Thank You Jeff Heaton! – user7226068 Oct 15 '18 at 12:56

0 Answers0