How long to wait to feed hashing using SHA 256?

Question

I have a source which generates secrets at $8-10 bits $ per second. I need to use this secret to feed another generator every second. I am wondering how long should I wait while I am concatenating 8 bits every second and then use longer bits as input to hash.

I have two options:

Use 8 bit that I get every second, hash it and send to next level every second.
Wait for say 10 seconds then hash the final say 100 bit. In between for 9 seconds keep sending hashed value of previous 100 bit using public informations.

Which one is more secure? I am using HKDF for hashing.

What do you mean that your source generates secrets? Do you possibly mean entropy as input for a PRNG / DRBG, i.e. random number generator? In that case, why not use a DRBG algorithm instead of HKDF? Those have specific methods to update the entropy. Secret bits are bits that you tend to distribute and store so that you can reuse them, e.g. as key for a symmetric cipher. — Maarten Bodewes, Jun 19 '17 at 10:19
I suggest you read the Fortuna paper and following works. It speaks to your situation quite well. — Thomas M. DuBuisson, Jun 19 '17 at 13:38

Maarten Bodewes · Answer 1 · 2017-06-19T12:34:27.273

10

You should aggregate bits until you have enough seed bits to start your DRBG using an initial seed. Your seed should contain at least 128 bits of entropy (the amount of uncertainty to an attacker).

Then you could either add the bits as additional seed when you get it or you could first aggregate some entropy. With only an update rate of about 8 bits per second you might as well directly add the entropy to your DRBG.

To create the same pseudorandom bit stream on both sides you should:

make sure you got the same algorithm on both sides;
configure the DRBG identically on both sides;
update your entropy at exactly the same time;
request data from the DRBG using the same methods and parameters.

Note that HKDF is not a pseudo random number generator (PRNG) or deterministic random bit generator (DRBG, same thing). It may be a pseudo random function but that's not the same.

HKDF turns an already secure key into more key material, possibly given additional information. HKDF / HMAC could be used to build a deterministic random bit generator. If you want to do that then you need to create a paper and prove it is correct.

I would recommend having a look at NIST SP 800-90A (revision 1) and pick a DRBG from there.

edited Jun 19 '17 at 12:34

answered Jun 19 '17 at 11:09

Maarten Bodewes

92,551
13
161
313

1

And HKDF is not a hash either. If you cannot get your definitions straight then the result of your endeavor may well end up in disaster. – Maarten Bodewes Jun 19 '17 at 11:13
2

+1 for "aggregate bits until you have enough seed bits to start your DRBG". Note, however, that constantly mixing in more bits afterwards could open you to attacks if your entropy source is partially compromised. If you cannot confidently estimate when you have enough true entropy for a secure seed, or if you want to do continuous reseeding for other reasons, you may want to consider using a tiered entropy pool design like Fortuna. – Ilmari Karonen Jun 19 '17 at 11:24
Just for reference, could you point out such an attack when the entropy source is compromised? I presume an attacker would still have to know the inner state of the DRBG, at least after it has been initialized with seed material containing enough - non compromised - seed material. – Maarten Bodewes Jun 19 '17 at 11:37
2

This blog post by DJB describes one class of such attacks. Another general attack type, which doesn't even require active tampering but just throttling down the entropy feed rate, is to first guess the initial DRBG state (which can be easy, if you know/control most of the entropy input) and then continually update the state to match the observed output, guessing any injected entropy (which, again, is easy if you only need to actually guess a few bits of entropy per update) as needed. – Ilmari Karonen Jun 19 '17 at 12:00
1

But yes, after you've managed to securely seed the DRBG once, and keep the internal state secret, such attacks are (AFAIK) generally not possible. The Fortuna algorithm I mentioned above is designed to ensure that such a secure reseeding will eventually happen, even if you've grossly overestimated the amount of entropy you're getting from your entropy source. – Ilmari Karonen Jun 19 '17 at 12:02

Paul Uszak · Accepted Answer · 2017-06-19T18:01:40.090

There are three classes of random number generators (my own informal classes), dependant on the relative entropies that flow in and out:-

Class 1. Hout < Hin, which is a true random number generator (the most secure and best) such as those by Swiss company ID Quantique. This is the class of generator that can be used for creating one time pad material.
Class 2. Hout > Hin, the most common commercial type like ultra fast beam splitters and ostensibly /dev/random.
Class 3. Hout >> Hin, which is a reseeded psuedo random number generator like /dev/urandom.

where H is the entropy.

Let's try and make a Class 1 type which is the most pure and secure.

Capture 1MB from your entropy source over 28 hours. Why rush?
Aquire a good compression tool like fp8.exe.
Compress the entropy sample, divide by two for safety and you have a very good estimate of usable cryptographic entropy rate from the source. This step is CRUCIAL in determining the entropy rate. Do not try to use the fancy Shannon formula as it won't work if there is any auto correlation in the entropy stream.
I'm unfamiliar with the input width to HKDF, but simply divide it by your generation rate and that's the time between re seeding.
You don't need the PRNG. The HKDF whilst exorbitant, will act perfectly as an entropy extractor.

As an example, if you're using SHA-256 it would need 256 bits of entropy as an input. If your raw entropy rate is 10 bits /s, it might be say 4 when you measure it using my technique. So feed SHA-256 every 64 seconds.

64 seconds will create a class one generator which is a true random number generator. Reducing the feed rate will reduce the security and you will have a class two which might be acceptable to you.

PS. I find it a bad idea to use a cryptographic function as a randomness extractor. It's mathematically unnecessary and dangerous in that if the entropy source misbehaves or you've underestimated the entropy generator rate, you'll never know. If you use a much simpler randomness extractor such as a compression function /matrix, any mistake you've made will immediately become obvious in the quality of the final output. HDKF will continue to produce uniformly distributed random numbers with any kind of rubbish input and you'll never know which class the generator is running within.

PPS. Don't follow any of the NIST guidance. It's specifically crippled to preclude the creation of true random number generators and therefore not as secure as it could be. Unfortunately (but not unexpectedly) you will find a heavy bias towards NIST on this forum. If you need further guidance, look to Europe like AIS or ISO standards. Maybe the French too :-(

... like AIS 31 (which, by the way, can be implemented using schemes from SP 800 90A). I'm sure this will result in a very strong RNG. But I'm not sure if this fits the experience nor the requirements of Jay. I'll upvote none-the-less to indicate that this is probably the most secure if not slightly paranoid option (a bit of paranoia never hurts when it comes to crypto). — Maarten Bodewes, Jun 19 '17 at 11:44
When you say feed SHA256 every 64 seconds, is it because 256 input length is more secure than lower length 4 or 10 bit which we get every second? — Jay, Jun 19 '17 at 14:18
@JayPrakash No, it's simple pragmatism. The hash takes an input of 256 bits. Therefore you need to put 256 bits in. If you only entered 4 bits at say bit positions 0, 1,2 and 3, your output would only range through 2^4 (16) possible output combinations. — Paul Uszak, Jun 19 '17 at 14:30
I guess there would be padding before hashing. I also believe that hash here should take 256 bits but this answer confuses a bit: https://stackoverflow.com/questions/4676828/when-generating-a-sha256-512-hash-is-there-a-minimum-safe-amount-of-data-to — Jay, Jun 19 '17 at 14:33
Piecemeal inputs to a randomness extractor like this require an fakakta entropy pool. It's done with /dev/*random and is a sure sign of dodgy entropy measurement /handling. — Paul Uszak, Jun 19 '17 at 14:33
@JayPrakash Yes, your linked question highlights exactly what I was talking about with inputs to a hash below it's block width. You'd have a collision every ~8 hashes due to the birthday problem. — Paul Uszak, Jun 19 '17 at 14:40
I doubt that you can realistically measure the entropy by e.g. a compression function. For example, the stream of length 1 MB produced by $AES-CTR$ with the zero key has effective entropy 0 (because I fixed the key), but every compression function (which is not specifically designed for this stream) will say it has full entropy of 1 MB. — Paŭlo Ebermann, Jun 19 '17 at 18:36
yeah. what you want to measure here is kolmogorov complexity, and it's proven to be not computable. — Display Name, Jun 20 '17 at 05:51

How long to wait to feed hashing using SHA 256?

2 Answers2

Linked