key exchange using hashing with preshared secret

Question

I came upon an idea the other day that I had some questions about. I am not a security professional but I do study with an amateur interest and have not heard of this particular idea before.

There were a few similar questions here, but :

• Using one-way hash functions as the encryption method
• Construct an encryption algorithm using hash function
• Can I use HMAC-SHA1 in counter mode to make a stream cipher?

Basically the idea is:

• presume the existence of a pre shared secret between client and server of appropriate size
• the server can send secret data to the client by sending a hash of the secret data prefixed to the shared secret.
• amount of data sent per hash is small, a handful of bytes (i.e. a key fragment)
• The client receives the hash. We can assume an attacker does as well
• The length of the hash input material is length(secret_data) + length(shared_secret)
• The client begins cracking the hash. The client must crack the first length(secret_data) bytes of data, as length(shared_secret) bytes of the input are already known.
• Cracking a few bytes should be trivial. Time taken can be made to scale with the quantity of secret bytes sent.
• Attacker must crack length(secret_data) + length(shared_secret) to recover the secret data. This should be infeasible.

The above is the idea in it's most simple form.

The following is a python implementation with slightly more detail, which I will mention afterwards:

""" Provides a simple protocol for establishing a new shared secret from an old one.
    This protocol is only valid for models that utilize initial registration through
    a secured channel (i.e. tls) or better, an out of band communication. """
import random
import itertools
import hashlib

# generate and establish an initial secret, for example via:
_initial_value = random._urandom(32)
# ... skipping how the exchange actually occurs, for brevity
# the initial_value MUST be transmitted securely (i.e. via tls or similar) or out of band!

# assuming both client and server have knowledge of the secret initial_value
def get_challenge(secret, key_size=32, bytes_per_hash=1, hash_function="sha256"):
    """ Usage: get_challenge(secret, key_size=32, 
                                   bytes_per_hash=1, 
                                   hash_function="sha256") => key, challenge, new_secret
        Create a challenge that only the holder of the shared secret (i.e. the client) can
        answer. Returns a randomly generated shared secret of key_size, and the challenge 
        with a message authentication code prefixed.

        Increasing the bytes_per_hash will increase the time required to
        crack a single hash exponentially. 

        Increasing key_size will increase the time required to complete the
        challenge by a multiple of how long it takes to crack a single hash"""          
    # on login:
    #    generate x random bytes and append the secret + the previous hash (or len(digest) null bytes for the first hash)
    #    hash the resultant string of length x + len(secret), buffer the bytes
    #    repeat the above, appending new hashes to the buffer while
    #    ensuring to hash the secret at each iteration
    #    send hashes to client
    challenge = key = ''
    hash_function = getattr(hashlib, hash_function)
    hash_output = '\x00' * 32

    # below is just for simplicity sake; DO NOT use the secret as the mac key directly
    # use hkdf to derive a mac key from the secret
    mac_key = secret 
    while len(key) < key_size:
        random_bytes = random._urandom(bytes_per_hash)
        key += random_bytes
        hash_input = key + secret + hash_function(hash_output + ':' + secret).digest()
        hash_output = hash_function(hash_input).digest()
        challenge += hash_output
   #     print "Created challenge: ", hash_function(hash_input).digest()
        secret = hash_function(secret).digest()        
    mac = hash_function(challenge + mac_key).digest()
  #  print "Generated key: ", key
    return key, mac + challenge, secret

def solve_challenge(secret, challenge, key_size=32, bytes_per_hash=1, hash_size=32, hash_function="sha256"):
    # client begins cracking hashes:
    #    number of unknown bytes: x; number of known bytes len(secret)
    #    x bytes are cracked in trivial amount of time and returned as part of the key
    #    repeat until all hashes are cracked, ensuring to hash the secret each round
    #    hashes must be cracked in order
    hash_function = getattr(hashlib, hash_function)
    assert hash_function(challenge[32:] + secret).digest() == challenge[:32], "Message Authentication Code Invalid"
    challenge = challenge[32:] # remove the mac
    key = ''
    range_256 = [chr(x) for x in range(256)]
    previous_hash = hash_function("\x00" * 32 + ':' + secret).digest()

    while challenge:
        current_hash = challenge[:hash_size]
    #    print "Cracking challenge: ", current_hash
        for permutation in itertools.permutations(range_256, bytes_per_hash):
            key_guess = ''.join(permutation)
            #print "Guessing: ", key_guess
            hash_output = hash_function(key + key_guess + secret + previous_hash).digest()
            if hash_output == current_hash:
                key += key_guess
                new_key_length = len(key)
                secret = hash_function(secret).digest()
                previous_hash = hash_function(hash_output + ':' + secret).digest()                
                break          
        else:            
            raise ValueError("Unable to recover bytes from hash")        

        challenge = challenge[hash_size:]
    #print "Recovered key ", len(key)
    return key, secret

# assume attacker receives hashes also...
# attacker begins cracking hash:
#    number of unknown bytes: x + len(secret) per hash
#    theoretically cannot crack a single hash, assuming the security of the 
#    underlying hash function holds true
#    UNLESS the attacker observed or obtained the secret value!
#    the hashes must be cracked in order

if __name__ == "__main__":
    key, mac_challenge, server_secret = get_challenge(_initial_value)
    _key, client_secret = solve_challenge(_initial_value, mac_challenge)
    assert key == _key
    assert server_secret == client_secret    

In the above example, a random 32 byte key is generated and split into 16 hashed secret messages and sent to the client. This key would be used to derive session keys.

My first question: Is it correct that without the shared secret, assuming the strength of the underlying hash function, the secret data should be unrecoverable?

Second question: Is data manipulated this way implicitly authenticated and integrity assured without the need for an additional external mac? The example includes one, is it necessary? My guess is yes because you wouldn't want to "decrypt" anything before verifying the intergrity.

The input for each hash includes the cumulative key fragments obtained so far and the hashed output of the last hash + secret. I had the idea this would ensure the hashes must be cracked in order. Is this correct?

The secret used in each challenge is also hashed before the next challenge is generated. Does this help or harm anything or should it be omitted?

Another potential application I considered was rate limiting/proof of work (unrelated to key exchange). The ability of a client to make requests could be slowed by a desired factor by adjusting the amount of secret data to crack and using the secret data as a capability/session id to perform the desired request. This would shift the rate limiting from server to clients by throttling to the physical capability of the client to make requests. At least, that is my idea; Are there already better designed/implemented schemes, and if so, may I be pointed towards them?

Presuming there is no way to crack the hashes substantially faster then brute force, is the best advantage an adversary could do in terms of advantage usage of gpu's and fpga's?

I gathered that the general consensus from the other questions was basically not to use such a construct while more thoroughly researched ones are readily available. No surprises there. Since there's a shared secret already there's little reason not to properly encrypt further secrets.