Another way to ask this question: when block ciphers were invented, why did they add the restriction that the block cipher would be restricted to blocks of a fixed size?
From my laymen's point of view, I don't see why rounds of S-box and P-box operations couldn't be performed with inputs of variable size.
Let us begin with what is a block cipher;
A block cipher is a family of permutations $$F:\{0,1\}^\ell\times\{0,1\}^b \to\{0,1\}^b$$ where $\ell$ is the key lenght and $b$ is the block size ( DES has 64, AES has 128 bit block sizes). From the key space $\mathcal{K}$ with $2^\ell$-size, a block cipher with the given $k \in \mathcal{K}$ selects one permutation form all possible permutations from $\{0,1\}^b \to\{0,1\}^b$. That makes at most $2^b!$ permutations. With the Stirling Formula
$$2^{b}! \approx \sqrt{2\pi 2^{b}} \bigl(2^{b}/e\bigr)^{2^{b}}$$ if you plug some numbers like AES-256 you will see how small the keyspace is compared to number of possible permutations;
$$2^{256}! \approx \sqrt{2\pi 2^{126}} \bigl(2^{256}/e\bigr)^{2^{256}} \!= \sqrt{2\pi 2^{256}} e^{-2^{256}} \bigl(2^{256}\bigr)^{2^{256}} \gg 2^{256}$$
Still what we want from a block-cipher like AES that they are a Pseudo-Random Permutation (PRP), not distinguishable from a random permutation. No one is able to show a distinguisher for AES yet in more than 20 years and no-one is able to prove it. We strongly believe it is a PRP.
Why are iterated substitution-permutation product ciphers only used on fixed-size plaintext blocks?
The substitution (Sbox in AES) can be used to design a larger block size, however, when you changed the block size from 64 to 128 bits you need a new permutation. Besides, for a better design, you will need to change the substitution, too.
When block size increased more and more, the control of diffusion (MixColumns+ShiftRows in AES) will be harder, too.
The above are some theoretical aspects of such design. There are other problems with the software and hardware. You need to implement many different sizes for software and hardware. This will increase the cost and area. Besides, in the hardware, we want small designs to reduce the area size and reduce power consumption. Even using the CTR mode removes the decryption implementation of the block cipher.
Although one possibly easily implement these in software, the problems are multiplied. You need to provide countermeasures for each implementation against the side-channeled attacks. The same applies to the hardware, too. Yet we are not perfectly able to do one, we will face multiple problems. See AES side-channel attacks, including the cache-attacks on AES.
Instead, it is better to design one with a fixed block size so that we can analyze it with more concentration. There is a nice post about the block cipher analyzing methods;
We can use a block cipher with various modes of operation to achieve the goal of encryption of large messages up to a limit, of course. I'll not list all of them here, each has pros and cons. Some to be mentioned
the CBC mode is a chaining based mode that uses padding to achieve encrypting the long messages. You need to be careful about selecting the key uniform randomly, selecting an unpredictable IV, and applying an HMAC to authenticate against the bit flipping attacks and padding oracle attacks when possible.
The CTR mode is a counter bases mode that turns a block cipher into a stream cipher, Instead of listing all pros and cons please read here;Disadvantages of AES-CTR?. In your case, the CTR mode doesn't need padding at all, one can encrypt arbitrary long messages ( not that so long, CTR designed for PRF when using a PRP be careful).
With the above modes you can only achieve CPA security and in modern cryptography, we require more than that. One should use Authenticated Encryption modes (AE) ( and possibly with Associated Data (AEAD)), like AES-GCM and ChaCha20-Poly1305. You will achieve more than CCA (AEAD > CCA). Each has pros and cons,
Does ChaCha20/Salsa have the same bit strength as AES for identical key sizes?
The one real problem with these two modes is the reuse of a (key, IV) pair. Both modes have an extension;
Daence: Salsa20 and ChaCha in Deterministic Authenticated Encryption with no noNCEnse
Arbitrary block sizes are also problematic if the block size is less than 128. There is a nice page about this, see
Another solution for your case the stream ciphers, they are designed to fast and can encrypt for large block sizes. They doesn't need paddings and they are encryption bit-by-bit or more in some designs.
Another way to ask this question: when block ciphers were invented, why did they add the restriction that the block cipher would be restricted to blocks of a fixed size?
The above reasons, especially the design and implementation. This will be more clear if you consider that during that age, people hadn't advanced as today. And the time showed that some decisions were right.
From my laymen point of view, I don't see why rounds of S-box and P-box operations couldn't be performed with inputs of variable size.
Hold on! they are not magical tools that like in LEGO that you may replace a 2x8 block with 4 2x2 blocks. Even there in some places, the structure will fail!
Each variation in the design needs to be designed, analyzed, and needs good implementation.