1

I have been using this answer as a guide to find specific point within a shape.

I am using a camera to read the area that is projected on a wall. This first drawing shows the boundary of the image as detected by the camera. Using the coordinates p1, I want to figure out what the coordinate would be for p2 (in the second image).

projected image

original image

This is my work. Something is wrong, and I am not sure where I went off the rails. You can see that the values for p2 are negative. That is definitely not anticipated.

enter image description here

Here is more detail on how I calculated lambda, mu, and tao for Gameboard. The formula is repeated for the next two variables.

Gameboard Matrix detail

The matrix for the first image is what I refer to as the boundary (b). The matrix for the second image is referred to as gameboard (g). From there, hopefully you can see where I tried to follow the process listed in the fore mentioned answer.

I am in way over my head on this problem. Can someone please help me find the issue in my logic?

Update: I found several errors in my matrix multiplication. I have followed the steps listed in the original answer and suggestions from this question, and while I am getting closer, I am still not there. As you can see in the following image, my coordinates are at least in the range of the gameboard, but they appear to be inverse of the expected results. enter image description here enter image description here

fizch
  • 11
  • how are you calculating lambda, mu, and tau? – TomKern Dec 06 '22 at 06:02
  • lambda, mu, and tau are calculated from step 1 in https://math.stackexchange.com/questions/296794/finding-the-transform-matrix-from-4-projected-points-with-javascript/339033#339033. I multiply the bottom left of the "boundary" by the inverse of matrix b. – fizch Dec 06 '22 at 16:24
  • 2
    check over your lambda, mu and tau for the gameboard g: they don't seem to solve the matrix equation – TomKern Dec 06 '22 at 19:50
  • You can debug this step by step. The way you checked the matrix inverse is already good. Next check that the four basis vectors actually map to the for given points for A and B. Check that the four preimage points map to the image points for C. If all of this works then C should be good. If not, you know the step that fails and can investigate that in more detail. – MvG Dec 06 '22 at 19:59
  • @MvG I am glad that you saw this considering I based this off of your answer. I am not sure how to do what you asked. How do I look for the basis vectors in A and B? – fizch Dec 06 '22 at 20:45
  • @TomKern I was wondering if that was a problem, but I have checked my math a few times. I am not 100% sure of what I am doing here. – fizch Dec 06 '22 at 20:45
  • I have updated the question to show further details about calc'ing lambda, mu, and tao for Gameboard. – fizch Dec 06 '22 at 21:39

1 Answers1

1

Your second screenshot shows that to compute $\lambda$ you multiply the first coordinate of the fourth (bottom left) point with each element in the first row of the inverse matrix. That's not how matrix times vector multiplication works: you need to multiply the first column entry with the first point coordinate, the second column entry with the second point coordinate and the third column entry with the third point coordinate. For $\mu$ and $\tau$ you do the same with the second and third row of the matrix.

As I have described in my answer, you can avoid the division by the determinant, and just work with the transposed cofactor matrix (adjugate matrix). So at that point I would compute

$$ \begin{pmatrix}\lambda\\\mu\\\tau\end{pmatrix} = \begin{pmatrix} -667 & -296 & 912919 \\ 528 & -297 & -124410 \\ 139 & 593 & -434122 \end{pmatrix}\cdot \begin{pmatrix}497\\1145\\1\end{pmatrix} = \begin{pmatrix}242500\\-202059\\313946\end{pmatrix} \\ A = \begin{pmatrix} 572\lambda & 1165\mu & 869\tau \\ 598\lambda & 459\mu & 1126\tau \\ 1\lambda & 1\mu & 1\tau \end{pmatrix} = \begin{pmatrix} 138710000 & -235398735 & 272819074 \\ 145015000 & -92745081 & 353503196 \\ 242500 & -202059 & 313946 \end{pmatrix} $$

Any multiple of the above matrix will work just as well. In particular if you divide by the determinant to get the actual inverse instead of the adjugate, you get that division applied to all the entries and end up with just a different multiple of that matrix.

At that point you can verify that all four basis points map as expected. $A\cdot(1,0,0)^T$ must be a multiple of your first point, for $(0,1,0)$ a multiple of the second, and for $(0,0,1)$ a multiple of the third. The most interesting one is probably $(1,1,1)$ yielding a multiple of the fourth. In a spreadsheet you could check this by computing these:

$$ \frac1{A_{31}}\left(A\cdot\begin{pmatrix}1\\0\\0\end{pmatrix}\right) = \frac1{A_{31}}\begin{pmatrix}A_{11}\\A_{21}\\A_{31}\end{pmatrix} = \begin{pmatrix}572\\598\\1\end{pmatrix} \\ \frac1{A_{32}}\left(A\cdot\begin{pmatrix}0\\1\\0\end{pmatrix}\right) = \frac1{A_{32}}\begin{pmatrix}A_{12}\\A_{22}\\A_{32}\end{pmatrix} = \begin{pmatrix}1165\\459\\1\end{pmatrix} \\ \frac1{A_{33}}\left(A\cdot\begin{pmatrix}0\\0\\1\end{pmatrix}\right) = \frac1{A_{33}}\begin{pmatrix}A_{13}\\A_{23}\\A_{33}\end{pmatrix} = \begin{pmatrix}869\\1126\\1\end{pmatrix} \\ \frac1{A_{31}+A_{32}+A_{33}}\left(A\cdot\begin{pmatrix}1\\1\\1\end{pmatrix}\right) = \frac1{A_{31}+A_{32}+A_{33}}\begin{pmatrix} A_{11}+A_{12}+A_{13}\\ A_{21}+A_{22}+A_{23}\\ A_{31}+A_{32}+A_{33} \end{pmatrix} = \begin{pmatrix}497\\1145\\1\end{pmatrix} $$

Mind you, all of the above is just the matrix $A$ from step 2 of the answer you're citing. You still have to do the same for the target points, then multiply the adjugate of one with the other.

I'm willing to share a spreadsheet with the computation for $A$, including the checks.

MvG
  • 42,596
  • Thank you so much. I am looking into it. Your spreadsheet will help tremendously. – fizch Dec 07 '22 at 17:25
  • So I have matrix b working with your help. I tried doing the same (using the adjugate and then checking) with matrix g and it is not checking out correctly. I have made a copy of my spreadsheet to share. https://docs.google.com/spreadsheets/d/1d-3ce9YtgxxVtdUTWH_hZdkAki1J_n1ehikBTCqJKAc/edit?usp=sharing – fizch Dec 07 '22 at 19:58
  • I found the issues with the adjugates. I fixed that and then proceeded to try completing the problem. However, my results still are not correct. – fizch Dec 07 '22 at 22:55
  • did you lose interest in helping with this problem? – fizch Dec 16 '22 at 22:53