Is a "curve" considered "linear"?

Question

In linear regression, we are fitting a polynomial to a set of data points. In Bishop's book of Pattern Recognition & Machine Learning, there are a few examples where the fit is a curve or a straight line. I am a bit confused if a curve is linear or not. The term linear means that the fit should be a linear function or a polynomial of degree 1 i.e., a straight line. But in many resources, examples are shown where the fit can be a polynomial of degree 3,9 etc. So, are these higher order polynomials linear?

Answer 1

Polynomial regression (for nth degree polynomial) in statistics is a special case of linear regression. Lets give an example for square function:

1. y = w*x           

This is linear in terms of both weight (w) and data (x).

2. y = w*(x^2)    OR        y = w*z ; where z = x^2     

This is still linear in terms of weight (w) and still treated as a linear regression for the transformed data (z). While the modeled relationship between y and x is certainly non-linear.

As you can notice above: The commonality in (1) and (2) is the linearity with the weight/ coefficient of linear regression.

Answer 2

Linear in linear regression means linear in parameters.

It refers to the relationship between the parameters that you are estimating (e.g., $\beta$) and the dependent variable (e.g., $y_i$). Hence, $y=e^x\beta+\epsilon$ is linear, but $y=e^\beta x + \epsilon$ is not.

This has nothing to do with the powers of the independent variables.

there are a few examples where the fit is a curve or a straight line.

The fit can be a curve and can incorporate higher powers of independent variables and be linear in parameters -- the betas.

Answer 3

If instead of using feature x, you use its square, you get a curve. It is a linear function of its variables, but you may enter the square or a cube of a variable, therefore making the graph appear as a curve. In this sense it is still linear while in essence it is a polynomial curve.