1

I have points $(1,1) (2,2)$ and $(3,2)$. Now I wish use least-squares to fit a line through it.

The equations I get, $$ C+D=1$$ $$ C+2D=2$$ $$ C+3D=2$$ Now, of course there exists no solution to this system($Ax=b$). We are obtaining the best fit hyperplanein $\mathbb R^{2}$(,i.e., a line) for the points. This plane has the closest projections of vector $b$.

Questions:

  1. I can't understand what the plane in $\mathbb R^{3}$ represents on which we are projecting on. What's does it signify?
  2. How the the plane in $\mathbb R^{3}$ relate to the best-fit line we obtain?
Abhishek Bhatia
  • 1,648

1 Answers1

0

The linear system is $$ \begin{align} % \mathbf{A} c &= b \\[2pt] % \left[ \begin{array}{cc} 1 & 1 \\ 1 & 2 \\ 1 & 3 \end{array} \right] % \left[ \begin{array}{c} C \\ D \end{array} \right] % & = % \left[ \begin{array}{c} 1 \\ 2 \\ 2 \end{array} \right] % \end{align} $$ The solution is $$ \left[ \begin{array}{c} C \\ D \end{array} \right]_{LS} = \frac{1}{6} \left[ \begin{array}{c} 4 \\ 3 \end{array} \right] $$ The matrix $\mathbf{A}$ is a map which takes vectors from the domain $\mathbb{C}^{n}$ and maps them to vectors in the image $\mathbb{C}^{m}$.

The matrix $\mathbf{A}$ is a map which connects vectors from the solution space $\mathbb{C}^{n}$ and to vectors in the data space $\mathbb{C}^{m}$.

If you are doing a linear regression with 10 data points, the solution space is $\mathbb{C}^{2}$ where your slope and intercept live. Your data space is $\mathbb{C}^{10}$ where the measurements live.

More about the fundamental projectors: Least squares solutions and the orthogonal projector onto the column space

Community
  • 1
dantopa
  • 10,342