I tried to solve & got somewhere & I am presenting the same. This may be very long way, but it may be someway ...
We want prime numbers only by performing $x^2 - y^2$. Since all primes (except $2$) are odd, this suggests that both $x$ and $y$ are neither even nor odd. One is odd and other is even
So, Let us take $x = 2k_2 + 1$ and $y = 2k_1$, with $k_2 \geq k_1$, $k_1,k_2 \in \mathbb{N}$ (This is necessary since in $x^2 - y^2, y^2$ must not be $0$, so $y$ must not be $0$, so $k_1$ must not be $0$) (And this assumption does not take all odd numbers and so not all primes, but later we can reverse the role of $x$ and $y$)
Now, our final result $N = x^2 - y^2$ is a prime number. And we need to prove that for every prime number, we only have a unique $x$ and $y$.
Let $k_2 = k_1 + m, m \in \{0,1,2,\cdots\}$
Replacing $k_2$,
we get $N = 4m^2 + 8k_1m + 4k_1 + 4m + 1$ which is a prime number.
It is observed that the unique solution for all primes should have $x$ and $y$ as consecutive numbers. So if we prove that is $N$ has to prime, then $m$ has to $0$ for any $k_1$ that makes $N$ prime then we are done.
Taking $k_1$ as $k$,
$N = 4m(m + 2k + 1) + 4k + 1$
If we take $m = 0, N = 4k + 1$ which is superset of all primes (except $2$).
But then we need to prove that above $N$ can never be prime for any other value of $m$.