There's a geometric way to do it, too.
Draw a rectangle and a point inside it. You can make four oriented triangles by connecting the point to the diagonals, and then uniformly drawing arrows on the edges so that they all "flow clockwise" or "flow counterclockwise".
Now if you imagine dragging the point over an edge, you'll see that three of the triangles retain their orientation, but the one whose edge has been crossed will flip orientation. If you go over a corner instead, two of the four triangles will flip orientation.
So, by uniformly representing these four triangles with vectors, you just have to check their cross products to see if they are all the same sign. If two disagree with the other two, you know the point lies on a diagonal outside. If one disagrees with the rest, it lies outside of one of the edges. If they all agree, the point is inside.
Here's the setup. Suppose you are given $(a_1,a_2),(b_1,b_2),(c_1,c_2),(d_1,d_2)$ as the corners of the rectangle (in cyclic order, say, clockwise), and $(e_1,e_2)$ is the point.
One triangle will have vector edges (in this order) $(a_1,a_2)-(e_1,e_2)$ and $(b_1,b_2)-(a_1,a_2)$.
The other three will be:
$(b_1,b_2)-(e_1,e_2)$ and $(c_1,c_2)-(b_1,b_2)$
$(c_1,c_2)-(e_1,e_2)$ and $(d_1,d_2)-(c_1,c_2)$
$(d_1,d_2)-(e_1,e_2)$ and $(a_1,a_2)-(d_1,d_2)$
They've all been set up so that if the point is inside, then taking the cross product of the first vector with the second in any of these four pairs, they will all have the same sign (all positive or all negative).
Given any four points in cyclic order like this, you can fill out the four equations and check all their cross products.
dist
? – Maesumi Oct 09 '12 at 10:32