I have the points:
$A(0,0,1), B(0,1,0), C(2,0,0)$
I am asked to find the scalar equation of the plane but I cannot use cross product to compute the normal vector. How do I do it without it?
Let:
$AB = [0,1,-1], AC = [2,0,-1]$
Let $n$ be the normal vector $[n_{1}, n_{2}, n_{3}]$
I know, $AB \cdot n = 0, AC \cdot n = 0 $.
How can I find the normal vector?
If $n=(n_1,n_2,n_3)$, then $$\begin{cases} \overrightarrow{AB}\cdot n=0 \\ \overrightarrow{AC}\cdot n=0\end{cases}\Leftrightarrow \begin{cases} n_2-n_3=0 \\ 2n_1-n_3=0\end{cases}$$ $$\Leftrightarrow \begin{cases}n_1=\lambda \\ n_2=2\lambda\\n_3=2\lambda\end{cases}\quad (\lambda\in\mathbb{R})\Leftrightarrow n=\lambda(1,2,2)\quad (\lambda\in\mathbb{R}).$$ Choosing for example $\lambda=1$, we get $n=(1,2,2).$
You don't need normal vectors. Clearly $\overrightarrow{AB}$ and $\overrightarrow{AC}$ are linearly independent. So, they span the plane containing $A,B,C$, and a point $X=(x,y,z)$ lies on the plane if and only if $\overrightarrow{AX}=(x,y,z-1),\ \overrightarrow{AB}=(0,1,-1)$ and $\overrightarrow{AC}=(2,0,-1)$ are linearly dependent. Hence the equation of the plane is given by $$ \left|\begin{matrix}x&y&z-1\\ 0&1&-1\\ 2&0&-1\end{matrix}\right|=0. $$
Another way is to use the Gram–Schmidt process. This is more complicated (unnecessarily so) but offers another perspective, constructing the normal vector starting with any vector not parallel to the plane by subtracting the parts parallel to the plane using vector projections.
With $\vec v_1=\overrightarrow{AB}=[0,1,-1]$, $\vec v_2=\overrightarrow{AC}=[2,0,-1]$, and $\vec v_3$ any vector not in the span of $\{\vec v_1,\vec v_2\}$, Gram–Schmidt produces mutually perpendicular vectors $\{\vec u_1,\vec u_2,\vec u_3\}$ such that the span of $\{\vec u_1,\vec u_2\}$ is the same as the span of $\{\vec v_1,\vec v_2\}$, and such that $\vec u_3$ is perpendicular to $\vec u_1$ and $\vec u_2$, and hence to the plane.
E.g., take $v_3=[1,0,0]$ and get
$$\begin{align*} \vec u_1=\vec v_1&=[0,1,-1];\\ \vec u_2=\vec v_2-\dfrac{\vec v_2\cdot \vec u_1}{\|\vec u_1\|^2}\vec u_1 &=\left[2,-\frac12,-\frac12\right];\\ \vec u_3=\vec v_3-\dfrac{\vec v_3\cdot \vec u_1}{\|\vec u_1\|^2}\vec u_1 -\dfrac{\vec v_3\cdot \vec u_2}{\|\vec u_2\|^2}\vec u_2 &=\left[\frac19,\frac29,\frac29\right]. \end{align*}$$
So you can take as normal vector any multiple of the last vector, e.g. $[1,2,2]$ as the simpler answer concluded.
