2

Assuming the conventional set up y-vertical and x-horizontal. Considering a data set of n pairs $(x_1,y_1),(x_2,y_2),\dots(x_n,y_n)$. $\mu_X,\mu_Y,\sigma_X,\sigma_Y,\sigma_{XY},\rho$ denote, in order: the mean of x, the mean of y, standard deviation of x, standard deviation of y, covariance of x and y, and coefficient of correlation of x and y.

When performing a regression y on x, model $y=\alpha_0+\alpha_1x$, vertical distance is minimized, i.e. minimizing $$\text{SSE}_Y=\sum_{k=1}^{n}(y_k-\alpha_0-\alpha_1x_k)^2$$

The resulting formula is: $$(y-\mu_Y)=\rho\frac{\sigma_Y}{\sigma_X}(x-\mu_X)$$

Similarly, when performing a regression x on y, model $x=\beta_0+\beta_1y$, horizontal distance is minimized, i.e. minimizing $$\text{SSE}_X=\sum_{k=1}^{n}(x_k-\beta_0-\beta_1y_k)$$

The resulting formula is: $$(y-\mu_Y)\rho\frac{\sigma_X}{\sigma_Y}=(x-\mu_X)$$

These two regressions are different, yielding different fitted-lines. Quote: Regression x on y and y on x

I was wondering about a linear model that minimizes both vertical and horizontal distances. For example, consider a model $$y=\gamma_0+\gamma_1x \Leftrightarrow x=\frac{-\gamma_0}{\gamma_1}+\frac{1}{\gamma_1}y\mid\gamma_1\neq0$$ minimizing $$\text{SSE}_{XY}=\sum_{k=1}^{n}\left((y_k-\gamma_0-\gamma_1x_k)^2+(x_k+\frac{\gamma_0}{\gamma_1}-\frac{1}{\gamma_1}y_k)^2\right)$$

I tried to search the literature for such model; yet I could not find it. Instead, I found the Error in variable model, which might have been developed on a similar idea. However, the model is different.

I attempted to minimize $\text{SSE}_{XY}$, following conventional approach. Eventually, the original problem seems to be reduced to: $$ \begin{cases} \gamma_0=\mu_Y-\gamma_1\mu_X \\ \text{minimize }(\gamma_1 - \rho\frac{\sigma_Y}{\sigma_X})^2+(\frac{1}{\gamma_1} - \rho\frac{\sigma_X}{\sigma_Y})^2 \end{cases} $$ Which is equivalent to minimizing the polynomial: $$(z-a)^2+(\frac{1}{z}-b)^2\mid\text{a, b are const.}$$ This polynomial perhaps can be minimized by finding its global critical values/minimum from its derivative with respect to z.

Hence, I would like to ask these questions:

  1. Whether anyone has seen the model I described above in literature?

  2. Is my approach in minimizing $\text{SSE}_{XY}$ correct?

  3. Is there a better/faster/more elegant method to solve the minimizing problem above that I missed?

I thank you for your interest and assistance.

Nam Tran
  • 31
  • You could be interested by orthogonal distance regression. Have a look at https://math.stackexchange.com/questions/1530298/variant-of-linear-regression-using-perpendicular-distance-instead-of-vertical?rq=1 – Claude Leibovici Oct 24 '17 at 07:22
  • Dear @ClaudeLeibovici Thank you for the link. I was reading about the Error in variables model, yet I did not realize that it is effectively what I am looking for (orthogonal distance regression when the variances are equal). – Nam Tran Oct 24 '17 at 14:40
  • 1
    This is really the key topic without speaking about errors or whatever. It gives the "same" regression for $y(x)$ and $x(y)$. – Claude Leibovici Oct 24 '17 at 14:43

0 Answers0