Does there exist an “almost-ideal” compression function to compress an arbitrarily long bitstring into a single pseudo-random (unbiased) byte?

Question

I need a function $F$ that compresses an arbitrary (unbounded from above) number of bytes into a single byte. The goal is creating a single unpredictable byte from an arbitrary bitstring.

Here are the requirements:

1. Pseudo-randomness: the resulting byte should look uncorrelated and unbiased in comparison with the input. This is why Pearson hashing is unacceptable (because it is not even needed to compute this hash to know that the outputs for 0000 and FF00 will be different). Note: see the detailed clarification of Property 1 in Edit #1 below.
2. Let $S$ denote the collection of all possible (different) bitstrings, assuming that all bitstrings are of the same non-zero length and this length is a multiple of 8. Let $L$ denote the number of elements of $S$. Let $T$ denote a collection of all outputs that correspond to each element of $S$. Then $T$ must contain $L/256$ occurrences of each byte. That is, for example, if I will compress all possible two-byte inputs, I will have 256 occurrences of each byte in the collection of all outputs. This requirement makes standard cryptographic hash functions unacceptable.
3. Table lookups are allowed, but if and only if all tables are “Nothing up my sleeve” and are generated/precomputed by an open algorithm, so everyone can observe the entire process of their creation.
4. The algorithm should be as fast/efficient as possible. This is another reason why standard cryptographic hash functions are not acceptable. Consider a situation when $F$ is used only for 8/16/24/32-bit inputs. Then I think that using 128/256/512-bit hash functions to compress such inputs does not seem efficient.
5. The algorithm is not required to be online. The entire input is allowed to be stored in RAM.

Is there any algorithm that matches the above requirements?

EDIT #1

I will clarify the property 1 as follows.

Let $G(V, W)$ denote an infinite family of all possible functions where any function $G_i$, given a bitstring $V$, generates a collection (array) that contains $W$ different bitstrings of equal length, assuming that $V$ will be the first element of the generated collection and the length (in bits) of each bitstring in this collection is a multiple of 8.

Let $G_1$ denote any chosen (fixed) function from the family of $G$. The algorithm of $G_1$ is open for everyone (as well as the algorithm of $F$). The simplest example of $G_1$ is a function that creates a collection where each successive element $b_i$ is a binary representation of $(b_{i-1} + 1) \bmod{2^N}$, where $N$ is the length of $V$.

Alice chooses any bitstring $B_1$ such that its length is equal to $8X$. Then Alice obtains the collection $S_1$: $$S_1 = G_1(B_1, X) = (B_1, B_2, \ldots, B_{X-1}, B_X).$$ Then Alice computes the bitstring $Y_1$: $$Y_1 = F(B_1) \mathbin\Vert F(B_2) \mathbin\Vert \ldots \mathbin\Vert F(B_{X-1}) \mathbin\Vert F(B_X).$$

Then $Y_1$ is given to the attackers, so the exact value of $X$ is open to them.

The attackers know that $G_1$ was used to generate each $i$-th element $B_i$, and they are allowed to choose any function from the family of $G$, denote it by $G_k$, then choose any $8X$-bit sequence $B_n$, then generate the collection $$S_2 = G_k(B_n, X) = (C_1, C_2, \ldots, C_{X-1}, C_X),$$

and then compute the corresponding bitstring $Y_2$: $$Y_2 = F(C_1) \mathbin\Vert F(C_2) \mathbin\Vert \ldots \mathbin\Vert F(C_{X-1}) \mathbin\Vert F(C_X)$$

(the attackers are not allowed to replace $F$ with another function).

The desirable property of $F$ is that the cost of finding any suitable combination of $G_k$ and $B_n$ that corresponds to $Y_2 = Y_1$ should be as close to $\sqrt{2^{8X}}$ as possible.

SUMMARY

This question implies a bijective function (or a Permutation-Based Compression Function) that can be described by the following quote (taken from the Abstract of “The MD6 hash function: A proposal to NIST for SHA-3”):

The compression function can be viewed as encryption with a fixed key (or equivalently, as applying a fixed random permutation of the message space) followed by truncation.

and optimized for 8-bit words architecture and for all bitstrings such that their lengths are multiples of 8 (similar to how MD6 compression function operates, so that no unnecessary computations are performed if all inputs are short, and all parameters are configurable for different input sizes).

Answer 1

I read property 2 to mean "every output byte value has exactly equal frequency". That's an uncommon design goal in cryptography. If that's what you really want then I recommend choosing a family of permutation functions with a separate function for each possible input length. Apply the appropriate permutation function to the input string (the one that matches the input length), then truncate the resulting bit string to 8 bits.

This grants you exactly equal frequencies. Note that you have an $n$ bit input string, and a bijective function that returns an $n$ bit output. Consider how many outputs are possible. That number is $2^n$. All possible bit strings are required to be possible output strings by the definition of a permutation function.

Next consider how many of those strings begin with a given 8 bit prefix. Well, the first 8 bits are constant and the remaining $n - 8$ bits can take on any value. (And must be able to take on any value.) Easy. There are $2^{n-8}$ possible output strings with a given 8 bit prefix. That's independent of the value of the prefix, so you can see that every prefix within all $2^n$ possible output strings occur with precisely equal frequency.

Use a pseudorandom permutation for your permutation function. This let's you conclude the prefix is pseudorandom as well. The tough part is finding fast PRPs for each possible input length. I suggest looking at XXTEA. It is a block cipher (therefore a PRP family) and is unusual in the property that it is defined for a variable block length. That's not to imply that it's secure or practical for cryptographic purposes. But if you don't need security it's fine.

One big problem with this method is that it has no preimage resistance. Simply choose any bitstring that begins with the 8 bit image value, then apply the inverse of the permutation function to get your preimage. I assume you know that an 8 bit output function is too small to qualify as secure even if it weren't so straightforward to compute preimages. (Just brute force preimages.) Even if you managed to find a fast one way permutation it wouldn't matter because brute force is an option.

---

This set of properties are rather unusual. If you're not sure if you need precisely-equal frequencies then there are things like hash functions which are fast. You can revise your question if you want those types of options. Property 2 actually reduces the randomness of the function. Property 3 implies you care about side channel attacks, but the 8 bit output is too small to rely on for security purposes. Property 4 suggests speed is important, but have you done any benchmarking to find out if optimizing this one function for speed will give you good returns? And if speed does matter, have you considered other designs with relaxed requirements? Those might be faster overall especially if you can use fast insecure hash functions.

Answer 2

Yes, it is possible to devise a function as asked.

Existence and theoretical construction

Define ${\mathcal F}_{n,m}$ to be the set of function $\Bbb Z_n\to\Bbb Z_m$. It has $m^n$ elements. The question is about functions of bytestrings of $l>0$ byte(s) to a single byte, which we can assimilate to functions $\{0,1\}^{8l}\to\{0,1\}^8$, and thus to members of ${\mathcal F}_{2^{8l},2^8}$.

Define ${\mathcal G}_{n,m}$ as the subset of ${\mathcal F}_{n,m}$ matching the question's property 2: each of the $m$ output elements is reached exactly $n/m$ times; otherwise said, the output is perfectly equidistributed. If $n\bmod m\ne0$, ${\mathcal G}_{n,m}$ is empty. With $n\bmod m=0$ (which is the case for $n=2^{8l}$ and $m=2^8$ of the question and assumed from now on), that set has $n!/((n/m)!)^m$ elements. As $l$ grows, that's an increasingly severe restriction, but even for a single byte input, tremendously many functions remain: $$\begin{array}{c|ccc} l\text{ (bytes)}&|{\mathcal F}_{2^{8l},2^8}|&|{\mathcal G}_{2^{8l},2^8}|l&|{\mathcal F}_{2^{8l},2^8}|/|{\mathcal G}_{2^{8l},2^8}|\\ \hline 1&2^{2048}&2^{1683.996\dots}&2^{364.003\dots}\\ 2 &2^{524288}&2^{522933.814\dots}&2^{1354.186\dots}\\ 3 &2^{134217728}&2^{134215353.934\dots}&2^{2374.066\dots}\\ \ge3 &2^{(2^{8l+3})}&2^{\approx2^{8l+3}-1020l+k}&2^{\approx1020l-k}& \end{array}$$ with $k=\displaystyle\frac{1793-255\log_2(\pi)}2=685.934\dots$.

A method to build some function $G$ in ${\mathcal G}_{n,m}$ is to pick a permutation $P$ of $\Bbb Z_n$, and set $G(x)=P(x)\bmod m$; that $G$ is perfectly equidistributed. Conversely, from any function $G$ in ${\mathcal G}_{n,m}$, we can build a permutation $P$ of $\Bbb Z_n$ with $G(x)=P(x)\bmod m$. Proof: define $I(x)$ to be the number of $x'\in\Bbb Z_n$ with $x'<x$ and $G(x')=G(x)$, then define $P(x)=m\,I(x)+G(x)$.

It follows that to pick a uniformly random member of ${\mathcal G}_{n,m}$ (which gives ideal security per any reasonable criteria), we can pick a uniformly random permutation $P$ of $\Bbb Z_n$ among the $n!$ there is, and define $G(x)=P(x)\bmod m$.

Translated to the question's ${\mathcal G}_{2^{8l},2^8}$, that's picking an arbitrary permutation of the $l$-byte input space among the $2^{8l}!$ ones, and keeping (say) the last byte.

Thus, as stated by Future Security's answer, we plausibly get a secure construction (per some reasonable definition) by picking a good block cipher of variable block size $l$-byte; fixing a nothing-up-my-sleeve key for that block cipher; and defining the function as the last byte of the output of that block cipher.

For $l=1$, our function is the block cipher. As $l$ grows, a secure block cipher arguably becomes increasingly overkill, because so much output is discarded.

Towards a practical construction

For small $l$, we could use some ARX $8l$-bit block cipher with a balanced Feistel structure, perhaps enhanced to use addition modulo $2^{4l}$ when mixing the round function's output (using XOR always generates an even permutation, which is recognizable for $l=1$; that's no big deal if we truly fix the key, but if we used it as a tweak that would be a distinguishable weakness).

Constructing an exactly equidistributed function for large $l$ is needed only for the sake of having something that formally answers the question: for large input (for sure for $l\ge32$), it is computationally impossible to recognize that a secure hash truncated to its last byte in not exactly equidistributed, and that gives a practical construction.

---

Despite the above, the following constructs an exactly equidistributed function for large $l$, with the convenient property of processing the input only once. I conjecture that for $b/2$-bit security and $8l\ge2b$ we can process the input by $b$-bit blocks with a $2b$-bit clock cipher, as follows:

• init a $2b$-bit block to the first $2b$ bits of input
• encipher the block
• while there remains at least $b$ bits of input
  • XOR these $b$ bits into the right side of the block
  • encipher the block
• append to the remaining input a single one bit then just enough zero bit(s) to form a $b$-bit block [with big-endian convention for bytes and $b\bmod8=0$, that's equivalent to appending one 0x80 byte and $b/8-1-(l\bmod(b/8))$ 0x00 byte(s)]
• XOR these $b$ bits into the right side of the block
• XOR a bitstring representing the input length into the other side of the block
• encipher the block
• output the rightmost byte of the block.

It can be proved by induction that when $(c+2)b$ bits of input have been processed, the block is the image of the processed input by a function in ${\mathcal G}_{2^{(c+2)b},2^{2b}}$. If follows that the final outcome is the image of the input by a function in ${\mathcal G}_{2^{8l},2^8}$ as thought.

For 128-bit conjectured security we could use the 512-bit block cipher in the round function of SHA-512, without the final 8 modular additions so that it remains a permutation, and with a message/key all-zero (saving all message expansion and mixing operations) since SHA-512 has enough nothing-up-my-sleeves constants (but if we need a tweak, we can use the message for that; entering the round number there could give extra security margin).

Note: The best attack is conjectured to be trying to encipher $2b$-bit values until finding a collision in the left half of the result (requiring $\mathcal O(b/2)$ effort), which then trivially allows to find $3b$-bit distinct prefixes $M_0$ and $M_1$ such that for all $M$, $M_0\|M$ and $M_1\|M$ yield the same result, which is a distinguishable property making no obvious reference to the nothing-up-my-sleeve constant.

Note: I we processed input by blocks of the full width $2b$ of the block cipher rather than half, it would require only two encryptions to exhibit $4b$-bit distinct prefixes $M_0$ and $M_1$ with that property.