4

I have been reading a couple of articles regarding polynomial regression vs non-linear regression, but they say that both are a different concept. I mean when you say polynomial regression, in fact, it implies that its Nonlinear right. Then why there is a difference in the Data Science world regarding both the concepts?

Amin Soheyli
  • 103
  • 4
jsr
  • 43
  • 1
  • 6

2 Answers2

7

the difference is probably easily seen with an example. Linear regression assumes a form $$f(x, \beta) = \beta_0 + \beta_1 x_1 + \cdots + \beta_n x_n$$ with some covariates $x_i$ and some parameters to estimate $\beta_i$. An example of non-linear regression would be something like $$f(x,\beta) = \frac{\beta_1 x_1 + \beta_2 x_2}{\beta_3 x_3 + \cdots + \beta_n x_n}.$$ Essentially you are assuming your model to be of a nonlinear form. Polynomial regression on the other hand is a fixed type of regression where the model follows a fixed form $$f(x, \beta) = \beta_0 + \beta_1 x + \beta_2 x^2 + \cdots + \beta_n x^n$$ which is a nonlinear function, however it is still linear in the parameters $\beta$ you are trying to estimate.

That is to say,

Polynomial regression is non-linear in the way that $x$ is not linearly correlated with $f(x, \beta)$; the equation itself is still linear.

In the other hand, non-linear regression is both non-linear in equation and $x$ not linearly correlated with $f(x, \beta)$.

1tan Wang
  • 358
  • 1
  • 14
Marvin Purtorab
  • 156
  • 1
  • Nice answer, could you also tell if it's preferable to use non linear regression using RF or Boosting etc. than polynomial regression due to difficulty in finding right polynomial degree. – Vikrant Arora Oct 10 '19 at 21:00
  • can we say that polynomial is a special case (form) of non-linear regression? – Abdulkarim Kanaan Dec 22 '21 at 04:43
  • @AbdulkarimKanaan Linearity refers to linearity in the parameters: a linear combination of features. Since each polynomial term is its own feature, polynomial regression is a linear regression. – Dave Dec 03 '22 at 19:59
0

Well elaborated by Marvin,To add on that-We define linear model as "A model that expresses the target output value in terms of a sum of weighted input variables" taking this definition into consideration for polymonial regression- the features are just numbers within a weighted sum

IQBAL SINGH
  • 1