How to recover small messages from textbook RSA?

Question

In the answer to the question Random unique key per message, safe to use key as IV? it was claimed that one can recover a random 128-bit textbook-RSA encrypted value using about $2^{64}$ steps.

How and why does this attack work?

Answer 1

Disclaimer: I did not come up originally with this attack, but rather it was first described in "Why Textbook ElGamal and RSA Encryption Are Insecure" by Boneh, Joux and Nguyen (PDF).

This answer will give a summary of the attack, similar to what is documented in Introduction to Modern Cryptography by Katz and Lindell.

First for the actual attack:
Let $c=m^e\bmod N$ be given as well as $(N,e)$ and let $m<2^l$ for some $l\in\mathbb N$. Then there exists $m=r\cdot s$ with $r,s<2^{\alpha\cdot l}$ for some $1/2<\alpha<1$ with decent probability:

1. Set $T:=2^{\alpha\cdot l}$.
2. For all $r$ from $1$ to $T$ compute and store $x_i=r^e\bmod N$.
3. Sort the pairs $(r,x_i)$ by the second component.
4. For all $s$ from $1$ to $T$ check (e.g. using binary search) whether $c\cdot s^{-e}\bmod N=x_i$ for some $i$, if so return $r\cdot s \bmod N$ as the message $m$.

(If $s^{-e}$ cannot be computed, then $s$ shares a non-trivial common divisor with $N$, and so one can trivially factor $N$ by calculating $\gcd(s,N)$.)

Now for the run-time of this algorithm: One needs about $2^{\alpha\cdot l}$ modular exponentiations for step 2, $\mathcal O(l\cdot 2^{\alpha\cdot l})$ for step three and about $\mathcal O(l\cdot 2^{\alpha\cdot l})$ for step four. So overall one does get away with "little more" than $2^{\alpha\cdot l}$ steps. Given that there's at least some chance (a few percent) that $\alpha=1/2$ will work, this would result in an attack with about $2^{l/2}$ run-time.

Note that steps 1 to 3 above don't depend on the ciphertext $c$, so they can be done in advance, and the resulting table reused for several ciphertext.

As for why this attack works, note that we effectively brute-force factor the message by exploiting the homomorphic property of RSA and that $c=m^e=(rs)^e=r^es^e\bmod N$.