I'm just doing some reacher for my machine learning class, I found out it must be three layers MLP. But why? Is there a proof/paper or I'm wrong?
Asked
Active
Viewed 70 times
1
-
Make your question more precise. Give the exact statement that is the source of your confusion. – Vikash Balasubramanian Dec 21 '20 at 05:52
-
Please provide more context for the question. – Academic Dec 21 '20 at 07:29
-
https://datascience.stackexchange.com/questions/26597/how-to-set-the-number-of-neurons-and-layers-in-neural-networks/26642#26642 – Green Falcon Dec 21 '20 at 11:22
1 Answers
2
I understand that the fact that MLP allows for arbitrary decision boundaries derives from the fact that an MLP is a universal approximator.
The theoretical area addressing the capability of neural networks to be able to approximate any function are the different flavors of the universal approximation theorem, for which there are two directions: the arbitrary width case (which you refer to when you assume that the ability comes from the three layers), and the arbitrary depth case.
The wikipedia page contains the mathematical formulation of both directions, as well as reference to their respective proof for different assumptions.
noe
- 26,410
- 1
- 46
- 76