I've got two rectangles in 3D space, each given by the coordinates of their 4 corners. They are not axis aligned, meaning their edges are not necessarily parallel/perpendicular to the world axes. Each rectangle can have any orientation.
Is there an easy way to know whether or not the two rectangles are intersecting?
Find the intersection points of both rectangles in the plane of the other. You achieve that by plugging the coordinates of the vertices in the implicit equation of the plane and detecting a change of sign.
If you obtain four intersection points, which define two line segments, it suffices to test if these segments overlap (and where if you need the endpoints). Project the points on the direction of the common line (cross-product of the normal vectors), and you will get scalar values corresponding to abscissas along that line.
Checking for overlap in 1D is trivial.
If you can test for line-rectangle intersection, then two rectangles A and B intersect if and only if either a line from A intersects B or a line from B intersects A (inclusive of the endpoints). Each of A and B are only four lines, so this shouldn't be too bad to check.
A point in each rectangle is individuated vectorially by
$$
\eqalign{
& {\bf P}_{\,1} = {\bf t}_{\,1} + a{\bf u}_{\,1} + b{\bf v}_{\,1} \quad \left| {\;0 \le a,b \le 1} \right. \cr
& {\bf P}_{\,2} = {\bf t}_{\,2} + c{\bf u}_{\,2} + d{\bf v}_{\,2} \quad \left| {\;0 \le c,d \le 1} \right. \cr}
$$
with an obvious meaning of the vectors
- ${\bf t} _k$: vector of a chosen vertex;
- ${\bf u} _k$: vector parallel to a side from the chosen vertex;
- ${\bf v} _k$: vector parallel to the other side from the chosen vertex;
Therefore, for the points to coincide we shall have $$ {\bf P}_{\,1} = {\bf P}_{\,2} \quad \Rightarrow \quad a{\bf u}_{\,1} + b{\bf v}_{\,1} - c{\bf u}_{\,2} - d{\bf v}_{\,2} = {\bf t}_{\,2} - {\bf t}_{\,1} $$ which is a linear system of $3$ equations in $4$ unknowns .
For the rectangles to intersecate each other, we shall verify that
- the system is solvable (in $a,b,c,d$);
- the set of solutions intersect the domain $[0,1]^4$.
There are various approaches to verify the first condition.
We can for instance go algebraically and check the the rank of the coefficients matrix, and that of the complete matrix:
- if both have rank $3$, we have a set of $\infty ^1$ solutions (a line), depending on one of the unknowns taken as a parameter,
geometrically that means that the planes of the ractangles intersect on a line;
- if the coefficients matrix has rank $2$, then the rectangles planes are parallel,
and they will be coincident or not depending if the complete matrix has rank $2$ or $3$,
if they have both rank $2$ then we have $\infty ^2$ solutions (two parameters);
- if the coefficients matrix has rank $1$, one or both the rectangles are degenerated
(and you can check this case in advance).
There are easy ways to decide (in some cases) the complement of this question, which is why you can devise a test to eliminate false positives. The problem is all the pathological fencepost cases, which you'll have to tackle systematically (one common vertex, one common edge etc) and floating point / precision issues, which are the bane of computational geometry.
For the first elimination test, I'd use the bounding boxes of the rectangles. If no more than two of the three coordinate ranges $\{min(x), max(x)\},\{min(y), max(y)\},\{min(z), max(z) \}$ of the two rectangles overlap, there is no intersection. You can convince yourself of that. You can check the x-, y- and z-ranges in succession eliminating false positives this way. ("Overlap" has to be understood as a non-empty set produced when you take the set intersection of the respective intervals).
After one round of elimination comes the tougher part. Even if there is overlap in the x,y, and z- ranges of two space rectangles, it does not mean that they intersect (think parallel rectangles). However, you can easily find the equations of the planes containing the two rectangles (taking three points each), so you can eliminate parallel planes.
Pairs of rectangles that pass both tests still need not intersect. At this stage you can try solving for the equation of the line in which both planes containing the rectangles intersect. Now you have to test the (endpoints of the) line against your intersection bounding box. If it lies outside, it is another false positive eliminated. If it lies inside, well, you still have more work to do.
