0

I have a set of n ($n\geq4$) or more unique point correspondences $[x_i, y_i] \to [x_i',y_i']$ and want to find the projective transform matrix A that relates each point as follows: $$\begin{pmatrix}x_i'\\y_i'\\1\end{pmatrix} \sim A\begin{pmatrix}x_i\\y_i\\1\end{pmatrix}$$ Our professor suggested and worked out a method with DLT. I thought that it might be possible to find A in a simpler way. It is as follows:

$$\begin{pmatrix}x'_1&x'_2&x'_3&x'_4&\ldots&x'_n\\y'_1&y'_2&y'_3&y'_4&\dots&y'_n\\1&1&1&1&\ldots&1\end{pmatrix}=\begin{pmatrix}a&b&c\\d&e&f\\g&h&i\end{pmatrix}\begin{pmatrix}x_1&x_2&x_3&x_4&\ldots&x_n\\y_1&y_2&y_3&y_4&\ldots&y_n\\1&1&1&1&\ldots&1\end{pmatrix}$$ $$B=AC$$ $$BC^+=A$$ where $C^+$ is the pseudoinverse of $C$. It seems to work for transforming a random quadrilateral from an image to a square, but I am wondering what's the mathematical background is. I've namely replaced the equivalent ($\sim$) with an ($=$). I know that $C^+$ coincides with the right inverse of C if all point correspondences are unique.

Ruben23630
  • 83
  • Another problem with your approach is that it doesn’t account for homogeneity. In general, the actual result of multiplying $(x,y,1)^T$ by a matrix representing a projective transformation won’t have a $1$ for its last coordinate. You’re essentially computing the unique affine transformation between the two given triangles. That is technically a projective transformation, but it doesn’t seem like that’s what you’re trying to do. – amd May 30 '18 at 23:35
  • There was a mistake in the wording, there is supposed to be a fourth point. Your clarification regarding the homogeneity is helpfull, I now understand why this method isn't correct. – Ruben23630 May 31 '18 at 08:21
  • The updated method is better, but it’s still not quite right. You’re computing a best-fit affinity, not a true projectivity. Essentially, replacing $\sim$ by $=$ is tripping you up: the way that you’re computing A, its last row will be $(0,0,1)$, which is a hallmark of an affine transformation. This method will not produce a matrix that maps an arbitrary quad to even the unit square. – amd May 31 '18 at 18:56
  • I’ll write up a more detailed explanation of why this doesn’t really do what you want later today when I have some more time. – amd May 31 '18 at 19:08
  • I think I understand what you mean. I'll post an answer myself to save you some effort. Please correct me if I'm wrong on some point. It's too bad however this method doesn't work as it's way more concise than the DLT. – Ruben23630 May 31 '18 at 19:14
  • You lose one degree of freedom by sticking all of the points into a single array and another by forcing the last coordinate of the output to be $1$. Still, if what you need is an affinity, you’ve got a nice method here. – amd May 31 '18 at 19:17
  • Oh, I don't really understand what you mean by losing one degree of freedom by sticking all coordinates in the same array. Tbh I don't really understand the concept of degrees of freedom. Maybe it's better if you write an answer later. – Ruben23630 May 31 '18 at 19:18
  • That was wrong. I miscounted—forcing a $1$ in the last coordinate of every input costs two degrees of freedom. – amd May 31 '18 at 23:56

1 Answers1

2

It’s a good idea that unfortunately doesn’t really do what you want: you’re computing a best-fit affinity instead of a proper projectivity. The way you’ve set it up, the matrix $A$ must preserve the last coordinate of every input vector, which means that its last row must be $(0,0,1)$. This is the hallmark of an affine transformation.

Given pairings between the vertices of two nondegenerate quadrilaterals, there’s a unique projectivity that maps one onto the other. Let’s see what happens when we try to compute this using your method. Let the source vertices be, say, $(-2,1)$, $(1,-3)$, $(3,2)$, $(2,5)$ and the destination be the unit square with vertices enumerated counterclockwise from the origin. Then $$A=\begin{bmatrix} 0.204734 & -0.144379 & 0.47574 \\ 0.168639 & 0.106509 & 0.198225 \\ 0&0&1 \end{bmatrix}$$ and the images of the four vertices are $(-0.0781065, -0.0325444)$, $(1.11361, 0.0473373)$, $(0.801183, 0.91716)$, $(0.163314, 1.06805)$, not the vertices of the unit square. If the four points formed a paralellogram, on the other hand, the mapping would’ve been exact.

What’s going wrong is that the way that you’ve formulated the mappings gives up two of the eight degrees of freedom of a projectivity. In general, the raw output of the product of a projective matrix and a point in the form $(x,y,1)^T$ is not going to be of that form. Its last coordinate will be some other value (think of it as a scale factor) that depends on the values of $x$ and $y$. So, by forcing the last coordinate of every output vector to be $1$—replacing $\sim$ by $=$, as you put it—you’ve lost two degrees of freedom. Not coincidentally, affinities have only six degrees of freedom.

To fix this, you have to allow the result of $A\mathbf x_i$ to be a scalar multiple of its desired image $\mathbf x_i'$. In $\mathbb P^2$, this can be expressed without introducing another variable via a cross product: $\mathbf x_i'\times A\mathbf x_i=0$. This results in a system of homogeneous linear equations in the elements of $A$, but this is pretty much the DLT method.

If you have exactly four point correspondences, then there is a method described here by MvG that’s similar to yours which allows you to compute the transformation matrix more-or-less directly. For the above example, the resulting matrix after normalization is $$H = \begin{bmatrix} 0.127072 & -0.127072 & 0.381215 \\ 0.0883978 & 0.0662983 & 0.110497 \\ -0.154194 & -0.0145655 & 1. \end{bmatrix}.$$ You can verify that the images of the source data points are in fact the corners of the unit square.

amd
  • 53,693