4

I'm trying to create an Augmented Reality app. I'm using OpenCV to get 4 points from a checkerboard pattern that represent the four corners of the pattern. This should be used to create a plane in my app, but the points are in 2D.

Anyone here have an idea how one is to use these 4 points to create a plane and/or map a 3D object to the pattern

Joey Green
  • 2,509
  • 4
  • 26
  • 46
  • 1
    I'm not an expert in 3D math unfortunately so I can't give an exact answer, but I think the solution will involve something like creating vectors out of lines connecting those points and using the cross product of those vectors to determine the orientation of the plane. Refer here: http://blog.wolfire.com/2009/07/linear-algebra-for-game-developers-part-2/ – jhocking Apr 18 '11 at 21:37

3 Answers3

1

Having found the corners in 2D, you assign them 3D coordinates yourself. (You might know the actual physical size of the board squares, or you might simply do everything relative to these sizes e.g. assign them a size of 1).

You then calibrate the camera giving the chessboard positions and your assigned 3D coordinates. OpenCV then computes a camera model. With this camera model, you can project 3D points.

There is a good example in Tutorial 10. This also shows how to undistort the image.

Will
  • 6,977
  • 4
  • 36
  • 59
0

There was a question on Stack Overflow that also has a link to the ARToolkit which describes the process they use a little bit.

Community
  • 1
Roger Perkins
  • 3,377
  • 1
  • 18
  • 14
0

So, haven't tried this before, but here is my guess at the approach.

You've got four points in screen space, in 2D. Let's refer to those as the UL,UR,LL,LR corner points (upper left, upper right, etc.).

Create a vector from the LL to the UL (so, along the left edge), and another vector from the LL to the LR (along the bottom edge). The cross product of these two vectors (with normalization applied where needed) give you the normal of the plane for the checkerboard.

You've still got to figure out the distance of the plane from the camera; to do this, you need to figure out how the edge distances compare with what you believe they should be, and so back out the perspective scaling that's been undergone. To get a feel for the scales, you'll probably need a calibration phase for your code where you place the checkerboard facing the camera straight on (perpendicular to the camera) and move it to different (measured and known!) distances, observing the detected lengths of the edges. From that information you can figure out the perspective transform of the camera.

That should be a good start, anyways.

Update

I made a mistake here and failed to address that the actual depth of the points/vectors would still need to be worked out. The cross-product as spec'd above would be useless, as it could only give you a vector perp. to the camera, which is useless.

I'm leaving this up in case it spurs any ideas, but the approach I quickly sketched is, at best, deeply flawed.

ChrisE
  • 2,327
  • 16
  • 17
  • Won't the vectors LL,UL and LL,LR need to be perpendicular to each other( 90 degree angle ) in order for the cross product to give the normal to the plane? In 3D space they are, but in 2D space they are not. I could be wrong though. – Joey Green Apr 19 '11 at 00:18
  • They don't need to be perpendicular but your real problem is that getting the vectors for the edges (3D vectors that is) is not addressed at all by Chris' answer and that task will probably be tricky. – jhocking Apr 19 '11 at 00:56
  • @Joey: Gah, absolutely correct--looks like my brain shut off. I was answering this as I left work; mea culpa. Updated answer to reflect this. – ChrisE Apr 19 '11 at 01:03