Questions tagged [regression]

This tag is for questions on (linear or nonlinear) regression, which is a way of describing how one variable, the outcome, is numerically related to predictor variables. The dependent variable is also referred to as $~Y~$, dependent or response and is plotted on the vertical axis (ordinate) of a graph.

Regression is a statistical measurement used in finance, investing and other disciplines that attempts to determine the strength of the relationship between one dependent variable (usually denoted by $~Y~$) and a series of other changing variables (known as independent variables).

Types of Regression –

  • Linear regression
  • Logistic regression
  • Polynomial regression
  • Stepwise regression
  • Stepwise regression
  • Ridge regression
  • Lasso regression
  • ElasticNet regression

The two basic types of regression are linear regression and multiple linear regression.

The general form of each type of regression is:

  • Linear regression: $~Y = a + b~X + u~$
  • Multiple regression: $~Y = a + b_1~X_1 + b_2~X_2 + b_3~X_3 + ... + b_t~X_t + u~$

Where:

  • $Y =~$ the variable that you are trying to predict (dependent variable).
  • $X =~$ the variable that you are using to predict Y (independent variable).
  • $a =~$ the intercept.
  • $b =~$ the slope.
  • $u =~$ the regression residual.

There are multiple benefits of using regression analysis. They are as follows:

$1.~$ It indicates the significant relationships between dependent variable and independent variable.

$2.~$ It indicates the strength of impact of multiple independent variables on a dependent variable.

Reference:

https://en.wikipedia.org/wiki/Regression_analysis

This tag often goes along with the tag.

2700 questions
15
votes
3 answers

Fitting exponential curve to data

If I have a collection of data points that follow an exponential curve relationship, how can I manually construct the equation that defines the best-fit exponential curve for the data?
Nakano
  • 655
13
votes
1 answer

How to find curve equation from data?

How do I find the formula when I only know some data points ? Usually I just use the Trendline option for diagrams in Excel, but this one eludes me. I expect it to be something like : Ax^2 + or - Bx + or - C. Sample data: X Y 1 4 2 8 3 …
Kim
  • 233
9
votes
9 answers

Why work with squares of error in regression analysis?

In regression analysis one finds a line that fits best by minimizing the sum of squared errors. But why squared errors? Why not use the absolute value of the error? It seems to me that with squared errors the outlyers gain more weight. Why is that…
8
votes
1 answer

Linear regression is wrong?

I tried asking this in electronics as a question related to oscillators, but I wasn't able to get a satisfactory answer. I think the more math-y types here may shed some additional light on the phenomenon I am observing. I am using two clocks…
ajs410
  • 181
8
votes
1 answer

Variant of linear regression using perpendicular distance instead of vertical

Normally, linear regression asks for a pair of parameters m,b such that for a set of given points $\{x_i,y_i\}$ the variance of $y-m\cdot x-b$ is minimized (this minimizes the distance in y-direction only). Instead, I would like to find a line…
7
votes
1 answer

How to find line that has least distance to all points?

I need to find the line having minimal distance to all points. I found linear regression and linear interpolation algorithms. But their minimal distance is only in y-axis: $D = y - f(x)$. But I need to find $a,b,c$ for line: $ax + by + c = 0$ where…
kravemir
  • 431
7
votes
1 answer

What is $\operatorname{Cov}(\widehat{Y},Y)$?

If $\hat{Y}$ is the OLS linear regression model for $Y$, what can I say about $\operatorname{Cov}(\hat{Y},Y)$? Is this value $0$?
Peter
  • 1,497
  • 1
  • 16
  • 28
5
votes
1 answer

Isn't the Hat Matrix just an identity matrix?

In Linear regression $y = X\beta + \epsilon$ The Hat matrix is defined to be $H = X(X^TX)^{-1} X^T$ . However. If I compute the equation for Hat matrix, I just get an identity matrix. My calculation is the following: $X(X^TX)^{-1} X^T $ $= X(X^{-1}…
5
votes
4 answers

How to fit logarithmic curve to data, in the least squares sense?

How to fit logarithmic curve to data, in the least squares sense? I have simple data of the type $(x,y)$, that is 2D. I need to fit curve of the type: $y = c_1 + c_2\ln(x)$. So I have the $x$'s and the $y$'s but I need to learn the $c_1$ and $c_2$…
5
votes
3 answers

Unconditional expectation vs conditional expectation in regressions - does it really matter?

I refer here to a simple linear regression whose true representation is given by the equation: $y_i=x_i'\beta+u_i$, where as usual $x_i$ is a $K\times1$ vector of independent explanatory variables, $\beta$ is a $K\times1$ vector of parameters and…
Majte
  • 303
5
votes
2 answers

How to calculate a correction factor for two sets of numbers

Suppose one has a set of numbers. To help understand my question, suppose that these numbers are from two different temperature sensors. In this first example, both sensors are placed in the same environment and should read the same temp Col 1 …
Vinterwoo
  • 199
  • 2
  • 3
  • 7
4
votes
1 answer

Projection Pursuit Regression

This is with reference to projection pursuit regressions. I kind of get the idea behind approximating a continuous function using weighted sums of ridge functions. I am not sure why ridge functions work in the model. Or put another way, why work…
Roy
  • 41
4
votes
4 answers

Curve fitting with derivatives

Is there any tool to do curve fitting with derivative values? I.e. I have a bunch of values of the function at certain points, a bunch of values of the function's derivative at certain points, a bunch of values of the function's second derivative at…
Red
  • 950
4
votes
4 answers

How to model prices?

This is my first question here. As I'm not a matematician I thought I'd ask here for advice how to approach something I'm working on as a hobby project. A bit of context Let's say there is a collection of items with a description of features and a…
murrekatt
  • 143
4
votes
1 answer

Simple Regression ~ House Price Prediction

I am stuck with this question. "You have a data set consisting of the sales prices of houses in your neighborhood, with each sale time-stamped by the month and year in which the house sold. You want to predict the average value of houses in your…
1
2 3
15 16