Questions about (linear or nonlinear) least-squares, an estimation method used in statistics, signal processing and elsewhere.
Questions tagged [least-squares]
1853 questions
6
votes
1 answer
Power curve fitting
Least Squares can be used to fit the following power curve to given data points.
$y=ax^b$ where $a,b$ are constants to be determined by the fitting process as seen here.
Is there a way to fit $y=a\left(x-c\right)^b$ to given data points? where…
Nathan
- 177
5
votes
1 answer
For fat, full-rank matrix $A$, why does $I - A^T(AA^T)^{-1}A$ give projection onto $\mathcal{N}(A)$?
Suppose we are trying to solve the least-norm problem for underdetermined equations, i.e. we want to minimize $\|x\|$ given that $Ax = y$ with $A \in \mathbf{R}^{m \times n}$ and $m < n$. I know that (and can prove) the least-norm solution is given…
user83387
4
votes
2 answers
Exponential least squares equation
I want to fit a function in the form of $y=Ae^{bx}$ using least squares regression. This page says that there's an alternative to applying least squares to $\ln y=\ln A+Bx$, but I can't figure out why $\sum_{i=1}^ny_i(\ln y_i-a-bx_i)^2$ is…
John
- 43
4
votes
3 answers
Is the least-squares solution unique?
I am looking for a line closest to $(-5, -2)$, $(-2, 0)$, $(-1, 0)$, $(2, 3)$, $(5, 4)$ using the least square solution. So I set the line as $$ax+by+c=0$$ let $a=1$ (where $a$ is not $0$ obviously) and got
$$\begin{pmatrix}
-2 & 1 \\
0 & 1 \\
0 &…
Mike Park
- 145
3
votes
0 answers
Least Squares with experimental data
HI everyone I have a question on my study guide I dont know how to attack. Here is the question:
My attempt at making sense of the problem:
The problem provides us with the sum of squares (SS) (I believe that's the 18.1). We can use the SS along…
user126582
3
votes
1 answer
linear regression - least squares fit error behaviour
I'm trying to be as clear as possible but please be patient as I am very new to the subject of curve fitting.
I come from a specific type of problem. I have an input/output relationship I get from a circuital level simulator.
At the most basic…
Mike Anblips
- 183
3
votes
1 answer
$x^TAx-b^Tx\leq 0\Rightarrow\|x\|_2\leq\|A^{-1}b\|_2$?
Let $x$ be a vector, $A\succ 0$ an inverse matrix and $b$ a vector with proper dimensions. If $$x^TAx-b^Tx\leq 0,$$ do we have $$\|x\|_2\leq\|A^{-1}b\|_2?$$
I don't think it's a hard problem, but I cannot think of a proof or a counterexample right…
ZhuShY
- 224
2
votes
2 answers
Least square best solution
The statement is "This least square problem can be solved efficiently, when A is of full rank. We can prove that the best
vector has to satisfy the equation (A T A)x = A T b."
Can anybody explain this?
thetatheta
- 227
2
votes
0 answers
reformulate optimization problems as a standard least square problem
The first summand in the objective represents data fidelity; the second summand in the objective promotes smoothness.
I have no idea how to understand this equation. Why xhat will be equal to the sum of two constants? What does each part mean in…
melonmelon
- 21
2
votes
1 answer
Help me get my head around this simple linear regression problem
Given a very simple linear projection model $Y=X\beta+e$ with $E(e|X)=0$ and $X$ a scalar. Notice that this is a simple linear model with no intercept.
Then $\beta = E(XY)/E(X^2)$ from the least squares formula.
On the other hand, if I take…
Bunbury
- 395
2
votes
1 answer
Deriving the least square system equations from calculus and the normal equations
My question is similar to this question.
For least squares, if $E=e^2 = \sum_{i=1}^m(a_0 + a_1 x_{1}^i+a_2x_2^i+...a_nx_n^i-y_i)^2$ ($n$ features and $m$ observations), if I set up a system of equations of $dE/da_i = 0$, is there a way to show…
Yandle
- 885
2
votes
1 answer
Why Does Ordinary Least Squares Need The Independent Variable Normally Distributed?
In wikipedia, in reference to generalized linear models, I read:
Ordinary linear regression predicts the expected value of a given unknown quantity (the response variable, a random variable) as a linear combination of a set of observed values…
Frank
- 880
2
votes
4 answers
How can I solve large size Least norm-2 Problem?
As shown in the title, the least norm-2 problem can be formulated as
$$\min_{x}{\|Ax-b\|_2^2}$$
where $A\in\mathbb R^{m\times n},b\in\mathbb R^m$ are parameters with $\operatorname{rank}(A)=n$ and $x\in\mathbb R^n$ is variable.
As the above problem…
eeliubin
- 31
2
votes
0 answers
Complex Least Squares Derivation
Considering the Complex Least Squares Problem, one can write the cost function for estimating H (where Y = XH + Z) as:
Reference: Mimo-OFDM Wireless Communications with MATLAB book (page 190)
$J(\boldsymbol H) = ||\boldsymbol Y - \boldsymbol X…
Bruno A
- 81
2
votes
2 answers
How to formulate a data fitting problem as a least squares problem
Formulate the data fitting problem as a least squares problem
$\frac {1}{2} \Vert Ax-b \Vert_2^2 $
I thought I was supposed to wrote it like this:
$ \frac {1}{2} x^THx + g^T+ \gamma$ but actually that's an unconstrained
quadratic program; any help?
user158013
- 215
- 1
- 9