$
\newcommand\R{\mathbb R}
\newcommand\Ext{\mathop{\textstyle\bigwedge}}
\newcommand\MVects[1]{\mathop{\textstyle\bigwedge^{\mkern-0.5mu#1}}}
\newcommand\form[1]{\langle#1\rangle}
$The determinant of a linear transformation $T : \R^n \to \R^n$ is the factor by which $T$ scales $n$-volumes. This is shown concretely by the change-of-variables formula for integrals: if $S$ is some volume then
$$
\int_{T(S)}\mathrm dV = \int_S\det(\mathrm DT)\,\mathrm dV = \det(T)\int_s\mathrm dV
\tag{Int}
$$
where $\mathrm DT$ is the total differential of $T$ (making $\det(\mathrm DT)$ the Jacobian determinant) and the second equality follows since the differential at any point of a linear transformation is the transformation itself.
This fact takes a more abstract form within the exterior algebra $\Ext\R^n$. The space of $n$-vectors $\MVects n\R^n$ is one-dimensional, corresponding to the fact that $\R^n$ has exactly one $n$-dimensional subspace; it follows that there is a unique scalar $\delta$ such that $T(I) = \delta I$ for all $I \in \MVects n\R^n$ (where here we are conflating $T$ with its outermorphism: its natural extension to a homomorphism over $\Ext\R^n$, which is much like its action over subspaces of $\R^n$). This scalar is precisely $\delta = \det T$.
Now suppose $n = 3$, we have planes-through-the-origin $P_a$ and $P_b$ with unit normals $a, b$, and $\mathcal P : \R^3 \to P_b$ is the orthogonal projection onto $P_b$. More properly we can represent $P_a$ and $P_b$ with bivectors $p_a, p_b \in \MVects 2\R^3$, related to the normals through the Hodge star:
$$
a = \star p_a,\quad b = \star p_b.
$$
Though much more could be said, we will say that $p_a, p_b$ are unit whenever $a, b$ are. Requiring $p_a$ and $p_b$ to be units makes each unique up to a sign. Because $\mathcal P$ maps $P_a$ to $P_b$, the outermorphism $\Ext\R^ 3\to \Ext P_b$ of $\mathcal P$ maps $p_a$ to a multiple of $p_b$, i.e. there is some scalar $\alpha$ such that
$$
\mathcal P(p_a) = \alpha p_b.
\tag{$*$}
$$ Because $p_a, p_b$ are unique up to a sign, so is $\alpha$.
The analogy of this $\alpha$ with a determinant should be clear. Let's make this more precise. Intuitively, there are exactly two rotations about $P_a\cap P_b$ taking $P_b$ to $P_a$; WLOG choose one and call it $R$. Then $R\circ\mathcal P : P_a \to P_a$ has a well-defined determinant. Because $R$ is an isometry, the equation (Int) gives this determinant a clear interpretation as the factor by which $\mathcal P$ scales areas within $P_a$ when they are projected onto $P_b$. Again because $R$ is an isometry we can say $R(p_b) = \pm p_a$; then we see
$$
R(\mathcal P(p_a)) = \alpha R(p_b) = \pm\alpha p_a
$$
and so $\det(R\circ\mathcal P) = \pm\alpha$ as desired.
Finally we concern ourselves with $a\cdot b$, which is the factor in the projection of $a$ onto $b$; our last task is to show that this is $\pm\alpha$ since scaling $a$ by an area factor $A$ then gives $b$ a magnitude of $\alpha A$. This is simplest in the formalism of geometric algebra. Let $I$ be our right-handed unit pseudoscalar (i.e. 3-vector). Then $\star B = -BI$ for any bivector $B$, and we will also take it as fact that $\mathcal P(B) = -\form{Bp_b}p_b$ where $\form{\cdot}$ is the scalar part operator. From ($*$) we see that
$$
-\form{p_ap_b}p_b = \alpha p_b
\implies \alpha = -\form{p_ap_b} = \form{p_aI^2p_b} = \form{(-p_aI)(-p_bI)} = (\star p_a)\cdot(\star p_b) = a\cdot b
$$
using the fact that $I^2 = -1$, that $I$ is in the center of the algebra, and that for two vector $x, y$ we have $xy = x\cdot y + x\wedge y$.
