1

I have the following equation for a decision boundary line: $-w_0 = w_1x_1 + w_2x_2$ and I want to prove that the distance from the decision boundary to the origin is $l = \frac{w^Tx}{||w||}$. I am having trouble wrapping my mind around how I can just get the distance from a line to a point. Am I supposed to be averaging the distances of all the points on the line to the point?

Jonathan
  • 736
  • 2
    The distance from a point to a line (or plane), is the shortest distance / distance to the nearest point on the line (or plane). Further it works out such a shortest line segment is perpendicular to the line (or plane). – Macavity Oct 11 '16 at 04:12
  • The equation for your boundary “line” looks like that of a plane instead. – amd Oct 11 '16 at 06:01
  • I have edited the question so it is a line, and not a plane. Thanks for pointing that out. – Jonathan Oct 11 '16 at 06:29
  • @Christian Actually, it doesn’t matter. The boundary hyperplane equation and formula for distance to the origin have the same form in any dimensional space. – amd Oct 11 '16 at 06:49

2 Answers2

1

The projection of one vector $\vec{u}$ onto another vector $\vec{v}$ in the same Hilbert space is related to their inner (dot, scalar) product as per the following formula obtained from an easy-to-understand YouTube proof:

$\textrm{proj}_{\vec{v}}(\vec{u}) = \left(\frac{\vec{u}\cdot\vec{v}}{{\left|\vec{v}\right|}^2}\right)\vec{v}$,

where $\left|\cdot\right|$ is the norm induced by the dot (inner, scalar) product, defined as $\left|\vec{x}\right| \equiv \left(\vec{x} \cdot \vec{x}\right)^{1/2}$.

There are many ways to view the dot product (a.k.a. inner product or scalar product) in a geometric way. For example, MapleSoft's explanation and that of Sangaku maths; these "prove" the concept more effectively than any attempt without pictures.

brethvoice
  • 132
  • 1
    While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - From Review – Integrand Jul 07 '20 at 15:06
  • @Integrand unfortunately the proof is mostly visual/geometric and I am unable to render such artistic representations at this time. To double down on the links, here is another one just in case the YouTube channel goes defunct in the future: https://www.maplesoft.com/support/help/Maple/view.aspx?path=MathApps/ProjectionOfVectorOntoPlane – brethvoice Jul 07 '20 at 16:27
0

The distance of a line to a point (or from a point to a line) is the minimum distance (in your case euclidean distance) of the points in the line from the specific point (here the origin).

In other words, the distance is the length of the perpendicular line from origin to your decision boundary.

Also, $\frac{w^Tx}{||W||} $ is not the distance from the origin to your line. You might want to re check your calculations.

As @amd mentioned, $\frac{w^Tx}{||w||}$ is the distance from a point $x$ to the line defined by $w$. Hence replacing $x$ by origin the signed distance should be $\frac{-w_0}{||w||}$.

alili2050
  • 34
  • I meant the length from the decision boundary to the origin. Hence, why is it not correct? Or should it be something else? – Jonathan Oct 11 '16 at 04:38
  • I don't understand why the length of the line is the line divided by the magnitude either. – Jonathan Oct 11 '16 at 05:32
  • 1
    The distance formula in the question is correct. The equation of the line says that, with $w=(w_1,w_2)^T$, $w^Tx=-w_0$ for all $x=(x_1,x_2)^T$ on the line, and the (signed) distance from the origin to the line is $-w_0/|w|$. – amd Oct 11 '16 at 08:06
  • @amd You are right. I will update my answer to reflect your point more clearly for others that might stumble upon this question. When writing the answer, I just checked that since s/he is looking for the distance to the origin, hence the formula should not contain $x$. – alili2050 Oct 13 '16 at 09:59
  • You still appear to be misunderstanding the OP’s formula. The equation of the line (hyperplane) is $w^Tx=-w_0$. Normalizing makes this $w^Tx/|w|=-w_0/|w|$. The quantity on the right is, as you say, the distance from the origin to this hyperplane. Observe, however, that this equation says that for all points $x$ on the hyperplane $w^Tx/|w|$ is constant and equal to that same distance (it’s the length of the projection of $x$ onto the normal). See the OP’s follow-up question for a more detailed explanation. – amd Oct 13 '16 at 18:09
  • @amd, I don't quite get your point. I didn't say the distance from origin to point x. I claimed (from your comment) the distance from point P to line (hyper plane) $w^T x$ is $w^T P / ||w||$. This might be wrong though? – alili2050 Oct 15 '16 at 18:10
  • You write: “Also, $w^tx/|W|$ is not the distance from the origin to your line.” This is patently false, as a glance at the equation of the line will show you. From that equation, we can see that this expression equals the distance of the line (hyperplane) to the origin for every point $x$ on it. – amd Oct 16 '16 at 21:04
  • The equation of a hyperplane is $w^Tx = 0$ (where $w = (w_0,w_1,w_2,...,w_n)$ and $x = (1,x_1,x_2,...,x_n)$). – alili2050 Oct 18 '16 at 06:48
  • That explains all the confusions, I was using a different notation, didn't read your comment carefully (or it was edited) and didn't get what you were saying, you are right. – alili2050 Oct 18 '16 at 06:59