My attempt and thoughts:
I was able to show that $E\otimes_F E$ is not a field: Consider the multiplication map $\phi: E \otimes_F E \rightarrow E$ where $a\otimes b \mapsto ab$. Observe that $0 \neq \ker \phi \subsetneq E \otimes_F E$ since we can take an element $x\in E \setminus F$ and observe that $0\neq x \otimes 1 - 1 \otimes x \in \ker \phi$. Thus, it is not a field.
A comment on this post (Tensor products of fields) claims that $E \otimes_F E$ is a field if and only if it is a domain. But I am not sure how one would start this proof.
I have also seen another post that claims that the Tensor product $L \otimes_K L$ has no nilpotent elements iff $I/I^2=0$ where $I=\ker \phi$ from above. However, they rely on an argument using Kahler differentials $\Omega_L$ which I do not have. I tried studying $i \otimes 1 - 1 \otimes i \in \mathbb{Q}(i) \otimes_\mathbb{Q} \mathbb{Q}(i)$ but I can't see how it is nilpotent? I am currently in the belief that it is not actually nilpotent in this ring.
I also thought about possibly considering an element $\alpha \in E \setminus F$ and its minimal polynomial $f(x) \in F$. Then, $f(x) = (x-\alpha)g(x) \in F(\alpha)[x]$ and possibly cooking up a zero divisor using $g(\alpha)$ but I failed so far.