I'm using the getPerspectiveTransform() method of OpenCV (description here). This method calculates a perspective transform from four pairs of the corresponding points. So we have to specify four start (source) points and 4 end (destination) points.
Let's assume we have the 4 start points $p_{s1}=(x_{s1}, y_{s1})$, $p_{s2}=(x_{s2}, y_{s2})$, $p_{s3}=(x_{s3}, y_{s3})$, $p_{s4}=(x_{s4}, y_{s4})$ and the 4 end points $p_{e1}=(x_{e1}, y_{e1})$, $p_{e2}=(x_{e2}, y_{e2})$, $p_{e3}=(x_{e3}, y_{e3})$, $p_{e4}=(x_{e4}, y_{e4})$. (The points can also be named differently, I don't know if I chose the best naming).
How is the $3\times3$ transformation matrix calculated from the 4 start and end points?