Denote the probability for the convex hull of the five points to consist of $k$ points by $x_k$. The convex hull has five points if and only if the five points form a convex pentagon, so $x_5=p_5$.
Now let's determine the expected number of subsets of four of the five points that form a convex quadrilateral in two different ways. There are $5$ such subsets, and each has probability $p_4$ to form a convex quadrilateral, so the expected number is $5p_4$. On the other hand, if the convex hull has $5$ points, all $5$ subsets form a convex quadrilateral; if it has $4$ points, the convex hull itself and two of the other four quadrilaterals are convex, for a total of $3$, and if the convex hull has $3$ points, exactly one of the five quadrilaterals is convex (the one not including the hull vertex that the line joining the two inner points separates from the other two hull vertices). Thus we have
$$
5p_4=5x_5+3x_4+x_3\;.
$$
Together with $x_5=p_5$ and $x_3+x_4+x_5=1$, that makes three linear equations for the three unknowns. The solution is
$$
\begin{align}
x_3&=\frac32-\frac52p_4+p_5\;,\\
x_4&=-\frac12+\frac52p_4-2p_5\;,\\
x_5&=\vphantom{\frac12}p_5\;.
\end{align}
$$
MathWorld gives $p_4$ and $p_5$ for points uniformly selected in a triangle and a parallelogram; here are the corresponding distributions:
$$
\begin{array}{c|c|c|c}
\text{shape}&p_4&p_5&x_3&x_4&x_5\\\hline
\text{triangle}&\frac23&\frac{11}{36}&\frac5{36}&\frac59&\frac{11}{36}\\\hline
\text{parallelogram}&\frac{25}{36}&\frac{49}{144}&\frac5{48}&\frac59&\frac{49}{144}
\end{array}
$$
The probability $x_4$ that the convex hull consists of four of the five points is the same in both cases; however, this probability is different for an ellipse. Here's code to check these results and estimate the values for an ellipse.
An addition eleven years later: I referred to this in answering Probability of each type of inscribed octahedron. Interestingly, it turns out that for points obtained by stereographically projecting points uniformly randomly chosen on the sphere $p_4$ can be shown to be $\frac35$ just by considering the possible degree sequences of polyhedra with $5$ vertices. This value is considerably lower than the two values in the table above: $\frac35=0.6\lt\frac23=0.\overline6\lt\frac{11}{36}=0.69\overline4$.