-4

Let the set of instances be generated by the function , where $ε$ is random, uniformly distributed noise. Also, assume that you are using a fifth degree polynomial for regression.

Math forever
  • 1
  • 1
    what are your thoughts? Why would the weights $w$ be different for different number of instances and different amount of noise? – Steven01123581321 Apr 21 '21 at 11:34
  • @Steven01123581321 I have no idea how to start giving a task. The only thing I know is that the hypothesis will perform the best approximation if we have a smaller amount of noise – Math forever Apr 21 '21 at 11:37
  • So you don’t know what effect having 6 MILLION training instances might have, as opposed to 6 training instances? – Joe Apr 21 '21 at 11:41

1 Answers1

1

Since we are using a fifth degree polynomial, which bends closely to the shape of the points, if there is a lot of noise then we could get overfitting. That is, the optimal hypothesis will have parameters $w$ not reflective of the actual 2nd degree polynomial, and would include higher order terms such as cubics, quartile, and quintics. If there are many data points, the estimated parameters $w$ would generally be closer to the true quadratic coefficients, hopefully getting small values for the third degree and higher terms. This is especially true if there is not much noise. However, if there were a lot of noise, having more instances would tend to mitigate the effects. In the presence of much noise, we expect $w$ to converge to the true parameter estimates as the number of instances increases, though more slowly for higher noise. That is,

$$\lim_{n\rightarrow\infty} w_n=(0,0,0,4,-4,-2)$$

Vons
  • 11,004