The key fact in the formulae above is Wilson's theorem:
If $n$ is prime, then $(n-1)!+1$ is divisible by $n$.
And the slightly more obvious:
If $n$ is composite and $n\neq4,$ then $(n-1)!$ is divisible by $n$.
Let's state upfront, though: These formula are essentially "nonsense," in that they add zero knowledge beyond our knowledge from Wilson's theorem, and are certainly not computationally useful.
Formula 1
$$\theta_j = \pi\frac{(j-1)!+1}{j}$$
will be a multiple of $\pi$ if $j$ is prime, and something else if $j$ is not prime. So $\cos^2\theta_j = 1$ if $j$ is prime and otherwise it is some value in $[0,1)$ so
$$\left\lfloor \cos^2\theta_j \right\rfloor = \begin{cases}1&\text{if $j$ is prime or $j=1$}\\
0&\text{if $j$ composite}
\end{cases}$$
The case $j=1$ is why there is a $-1$ added to the sum above.
Formula 2
The second case has $H(j)=F(j)$ for $j>1$. It's not actually defined when $j=1$, so you'd have to deal with that to fix it.
If $j$ is composite, then $\pi\frac{(j-1)!^2}{j}$ is a multiple of $\pi$, so the numerator is zero, so $H(j)=0$.
When $j$ is prime, then $\pi\frac{(j-1)!^2}{j} = \frac{\pi}{j}+M\pi$ for some integer $M$, so the sine of this angle is $\pm\sin \frac{\pi}{j}$, and therefore the square of the quotient is $1$.
Formula 3
The last formula can be rewritten as:
$$p_n=1+\sum_{m=1}^{2^n} \left\lfloor\left(\frac{n}{1+\pi(m)} \right )^{1/n} \right\rfloor$$
But it is easy to show that $n^{1/n}<2$ for all $n$, and thus
$$\left\lfloor\left(\frac{n}{1+\pi(m)} \right )^{1/n}\right\rfloor=\begin{cases}1&\text{if }\pi(m)<n\\
0&\text{otherwise}
\end{cases}$$
Since the first $n$ primes are known to be less than $2^n$, this sum counts all $m$ with $\pi(m)<n$, which is is $p_n-1$. Adding $1$ gives $p_n$.
Excuse me.
– benjamin_ee Oct 27 '15 at 13:34\left\lfloor ...\right\rfloor
than with brackets. – Thomas Andrews Oct 27 '15 at 13:57