60

I have been trying to understand exactly how a length extension attack works on SHA-1. I'll detail below what I've understood so far so that I can convey my understanding of the same and hopefully get advice on where I'm going wrong.

Let's assume that for a message $m$, a secret $s$ is prepended to it and the SHA-1 hash is calculated. If $\textrm{len}(m) + \textrm{len}(s)$ is not a multiple of the block size, a certain amount of padding is added to it to make it a multiple, and the hash is generated using what is essentially $\textrm{H}(s \mathbin\| m \mathbin\| \textrm{padding})$.

Now SHA-1 internally uses 5 registers — it takes in a block, generates new register values, turns the crank, takes in another block, generates new register values, turns the crank ... and so on till the end. When all blocks are exhausted, the register values are concatenated and spit out.

Assume that an attacker knows a certain value of $m$, for which he also knows the hashed key. In such a case, he knows the state of the 5 registers when $(s \mathbin\ m \mathbin\ \textrm{padding})$ was processed. Let's also assume that the attacker knows $\textrm{len}(s)$ so that he can predict how much padding was added.

Here is where I get confused. From what I've read so far, the attack involves taking the known hash, adding a certain amount of padding, and then the value that we want. However, isn't the padding already used up in creating the hash? Why are we using padding in our attack?

DannyNiu
  • 9,207
  • 2
  • 24
  • 57
  • Been wanting to ask the same but forgot! Is this by any chance in response to some blogpost unclearly explaining the 7th level of Stripe's CTF challenge? – Luc Oct 07 '12 at 18:08

2 Answers2

85

SHA-1 processes data by 512-bit blocks (64 bytes). For a given input message m, it first appends some bits (at least 65, at most 576) so that the total length is a multiple of 512. Let's call p the added bits (that's the padding). The padding bits depend only on the length of m (these bits include an encoding of that length, but they do not depend on the value of the actual bits).

The padded message m||p is then split into successive 512-bit blocks, which are processed one after the other. SHA-1 uses an internal compression function (that's the traditional term); it also has a running state consisting of five 32-bit words. The compression function takes as input two values of 160 and 512 bits, respectively, and outputs 160 bits. The processing goes like this:

  • The running state is initialized to a fixed, conventional value (which is given in the SHA-1 specification).
  • For each input block, the compression function is evaluated, with as input the current running state, and the input block; the output of the function is the new running state.
  • The running state after processing the last block is the hash output.

So now, the length-extension attack. Suppose that you give me a hash value h, computed over a message m that is unknown to me. I know the length of m, but not its contents. Since I know the length of m, I can easily compute the padding p that you used. Then I imagine a message m', which begins with m||p; that is, m' = m||p||z where z is a sequence of bits that I can choose arbitrarily. I will now proceed to computing the SHA-1 hash of m', even though I do not know part of it (it begins with m, which I do not know).

When SHA-1 is computed over m', the latter is first padded with p', which depends on the length of m' (which I know). The resulting stream is m||p||z||p'. Then, the 512-bit blocks are processed one by one. I cannot do it for the first blocks, since I do not know m. However, I can imagine myself doing it. At some point, I would reach precisely the end of the p string (since the length of m||p is a multiple of 512). What would be the value of the running state at that point ? Well, that's exactly the hash value h that you gave me ! Therefore, I can stop imagining; I can start my SHA-1 computation of SHA-1(m') right at that point, at the beginning of z, using your hash value h as value for the running state.

That's the core of it. I can use SHA-1(m) to compute SHA-1(m'), a message which begins with the contents of m, and I can do that without knowing m. This property of SHA-1 (which applies to other Merkle-Damgård hash functions such as MD5 or SHA-256) is not in contradiction with the usual hash function security features (resistance to collisions, preimages and second preimages), but it shows that SHA-1 is not a random oracle.

What good can it do to the attacker ? Well, consider the following (flawed) construction for a Message Authentication Code: for a given key k and data to protect d, compute SHA-1(k||d) as the MAC value. This scheme breaks in the presence of the length-extension attack: if I, as an attacker, see a MAC for a message d, then I can compute the MAC for a message d' which extends d -- and I can do that without knowing k||d (so, in particular, without knowing the key). This allows me to forge messages with a valid MAC.

The length-extension attack is the reason why, when building a MAC out of a hash function, we need something a bit more convoluted, namely HMAC (which is safe against it).

HorstKevin
  • 244
  • 1
  • 8
Thomas Pornin
  • 86,974
  • 16
  • 242
  • 314
  • 1
    How you can imagine a message starting with m||p since you don't know m? Do you mean that i feed the SHA-1 with h(m)||p||z? – curious Oct 10 '12 at 10:23
  • 4
    I imagine with my brain. That's the point of imagining: I don't have to do it for real. If I knew m, I could build the larger message. Application is given in the end of my answer: I don't know the MAC key but I can still forge a message by imagining I know it, and computing the MAC value that I would have obtained. – Thomas Pornin Oct 10 '12 at 11:11
  • It's really lot of text, hard to understand. Would help if you explain yourself more shortly. – Smit Johnth May 30 '13 at 21:55
  • Question: HMAC-SHA-1 and HMAC-SHA2family is safe against length extension, and is better. However, as an alternative for, say, SHA-512, would simply truncating the output by N bytes prevent length extension, since an attacker no longer has the complete internal state? If so, for what values of N is this even possibly worthwhile? I assume there are other benefits HMAC gives outside the scope of this question, of course. – Anti-weakpasswords Feb 23 '14 at 07:39
  • Being the attacker, we didn't use $p$ padding anywhere, so is it necessary for attacker to know length of $m$ ? without knowing length of $m$, attacker can also calculate $H(m || p || z)$, can't ? – Mehran Torki Apr 18 '16 at 19:17
  • @MehranTorki: to complete the hash computation, the attacker must append its own padding, that encodes the length of m||p||z, so the attacker must know the length of m||p. Thus, he must know the length of m at least approximately (within one block of the true length). – Thomas Pornin Apr 18 '16 at 19:21
  • @ThomasPornin Thanks for the reply. I think i didn't explain what i meant. For example, when using $SHA-1$, we know that $m || p$ is a multiple of 512 bits. right ? so if we put $SHA-1(m)$ as $IV$ in MD structure and put $z$ as input message, the output would be the has for $m || p || z$, isn't it ? – Mehran Torki Apr 18 '16 at 19:33
  • @MehranTorki: to actually compute SHA-1 of m||p||z you need to process m||p||z||p' where p' is the padding that corresponds to the length of m||p||z. You don't have to know m||p because you (as the attacker) start from the known hash value (SHA-1 of m) but you still need to assemble a proper p', and that entails knowing the length of m||p||z -- which basically requires knowing the length of m within a few bytes. – Thomas Pornin Apr 18 '16 at 19:38
  • @ThomasPornin What do you mean by "but you still need to assemble a proper p' ", m|| p is a multiple of 512 bits so p' length is just depends on the length of z, is this correct? sorry bothering you – Mehran Torki Apr 18 '16 at 19:43
  • 1
    @MehranTorki: the length which is encoded in the padding is the length. Not "the length modulo the block size". It is a multiple of 512, but you must still know which multiple of 512 it is. Is it 512, 1024, 1049088? – Thomas Pornin Apr 18 '16 at 20:01
  • @ThomasPornin I still don't get it. Suppose $y = h(k||x)$. $x,y$ are known to the attacker. The purpose is to create $x'$ so that its MAC is valid. The validity is checked (by someone else) by taking $x'$, prepending $k$, and hashing it (in this case). If $k||x$ is processed into $x_1 ... x_t$ so that $y = f(H_{t-1},x_t)$, then making up your own extra block $x_{t+1}$ and calculating $f(y,x_{t+1})$ accomplishes nothing. You have a valid MAC, but you don't have a message. What am I missing? – Erik Vesterlund Apr 20 '16 at 02:00
  • @ErikVesterlund Why do you not have a message? You know x, and you know x' (sorry, I don't know how to make the MathML symbols here), therefore you know the complete message (just not the key). – Stavros Korokithakis Aug 24 '16 at 21:37
  • @StavrosKorokithakis I'm not sure what you mean. $x'$ is not known, as stated the purpose is to create $x'$ and a new MAC from the available information, i.e. $x,y$ and possibly the MAC-algorithm. – Erik Vesterlund Aug 25 '16 at 16:14
  • $x'$ is a concatenation of $x$ with some of your data. You may or may not know $x$ (but need to know its length). You will know your half of $x'$, as you constructed it. The purpose is not to get the contents of the original message, but to construct a new valid MAC by extending the original message. – Stavros Korokithakis Aug 25 '16 at 19:34
  • @StavrosKorokithakis Could you show me an algorithm on how to do it, then? I've gone over the specifications of SHA-1 and SHA-2 and for neither of them am I able to construct a length extension. IF the key length is known, then length extension is trivial, but that's a big if, ain't it? – Erik Vesterlund Aug 29 '16 at 08:53
  • @ErikVesterlund I don't understand your comment. Length extension attacks require you to know the message length. Yes, it's a big if, but we aren't discussing the size of the "if" here, we're supposing it's true. – Stavros Korokithakis Aug 29 '16 at 12:38
  • @stavros Read again :) of course the message length is known, I asked how to do the attack without knowing the key length. If at all possible it is not trivial (hence my asking for an algorithm). – Erik Vesterlund Aug 30 '16 at 13:21
  • @ErikVesterlund As per the post above, $m = k || x$, where m is the message, so if you know the message length, and you know $x$, you also know $k$. If you don't know the message length, I'm not aware of an algorithm that can extend the hash. – Stavros Korokithakis Aug 30 '16 at 19:16
  • @StavrosKorokithakis Finally it is apparent that you have not read my posts at all. The notation "$k$" that you reference was introduced by me (apr 20), then you claim, or at least strongly imply, that a proper preimage ($x'$) is knownn (false, which I pointed out on aug 25). You then (also aug 25) suggest $x'$ is concatenation of $x$ with unspec. data, and strongly imply that you have no idea what a MAC protocol entails ("may or may not know", etc). Further discussion is impossible due to your lacking familiarity with hash functions, relevant protocols and, seemingly, math notation. – Erik Vesterlund Aug 30 '16 at 20:56
  • Could this kind of attack be thwarted by making the running state larger than the reported hash? Are there any disadvantages to using a running state which is bigger than the reported hash value? – supercat May 29 '18 at 22:02
  • But this way you have a hash for m||p||z not m||z, right? And they are different? – Yola Sep 17 '21 at 02:13
  • How does double hashing solve the length extension attack? – Rony Tesler Sep 22 '21 at 01:55
  • @ThomasPornin You are a hero and a scholar – Tobias Wilfert Oct 27 '21 at 16:25
38

The key to understanding hash extension attacks is to understand that the hash output isn't just the output of the machine generating the hash, it's also the state of the machine up till that point. In other words, just the hash output alone contains enough information for you to keep going and append more content to the hashed input.

The catch is that since hashes work on fixed-size blocks, if the data size doesn't line up properly along one of these block boundaries, it will be padded with zeros until it does. So we have to account for that in our attack. (Yes, SHA-1 has additional padding rules, which we'll ignore for simplicity)

An example

Let's say your super-secure authentication protocol is to take a given message, prepend a password to it, and send the hashed result as the "signature". I can compute the hash of your string plus an additional string just by knowing your hash output.

So lets say I have your "signature" for the string count=1&price=100.2, but I want to change the price to 1. I know your parsing algorithm replaces earlier values with later ones and it doesn't choke on embedded nulls. So theoretically I can compute the hash for my desired string, count=1&price=100.2&price=1.0.

But I can't just append my string to yours and expect it to work. I have to account for padding. We'll assume a block size of 10 for simplicity. (nb: "•" = null)

   123456789|123456789|123456789|123456789|             <- ruler
A: [mysecurepassword]count=1&price=100.2                
B: [mysecurepassword]count=1&price=100.2•••

Ostensibly I have the hash for string A, but what I really have is the hash for string B -- with the three nulls appended at the end for padding.

So instead of sending the string count=1&price=100.2&price=1, I would have to send the string count=1&price=100.2•••&price=1.0.

To generate my signature, I just initialize the hash engine with the signature you gave me, add &price=1.0, and output the result.

Of course, the rules of SHA-1 are somewhat different than my pretend 10-byte hash, but the principle still applies, it's just a matter of getting the bytes to line up.

Community
  • 1
tylerl
  • 857
  • 6
  • 5
  • excellent answer. i think i get it now – wlad May 03 '15 at 14:33
  • +1 for state and modifying the initial state (initialization vector) of the hash function – chrisamanse Sep 05 '16 at 13:53
  • 1
    "and it doesn't choke on embedded nulls" --> if embedded nulls are not possible, is length extension attack negated? – chux - Reinstate Monica Jan 26 '17 at 02:39
  • I still don't fully get it. 1. Could the attacker computer the hash for any data, even one that completely replaces the original data? Or can they just append stuff to the end of the original data? If it's the latter, isn't the attack very limited? If the message is a JSON object for example, the opening and closing tags won't match, and the data will be invalid? 2. All explanations I've seen concatenated the secret key to the front of the data. Does that matter, or would the attack still work the same even if key is concatenated at the end of the data, in the middle of it or at both ends? – Magnus Sep 09 '22 at 07:53
  • Yeah, length extension attacks are only possible if the victim allows additional information to be accepted when it's tacked onto the end of the otherwise-authenticated string, as long as a new authentication hash can be computed. One way to thwart this is by making the hash not so dumb (use HMAC for example), another is to make the data itself non expandable, e.g. by adding an end marker or a length declaration or something. – tylerl Sep 18 '22 at 15:54
  • "To generate my signature, I just initialize the hash engine with the signature you gave me, add &price=1.0, and output the result.", how can you initialize SHA1 from the outside when it is initialized from the inside with fixed five 32-bit words? From my understanding, you can not manipulate initialization to SHA1 because it is already predetermined inside the code. – Oli Oct 19 '22 at 03:03
  • @Oli: one can take an implementation of SHA-1 and add an additional input for the "fixed five 32-bit words". Other details are important, like adding another input for the initial value of the message length counter. And the attack on SHA-1 can not append any data, due to numerous details including 512-bit block alignment issues, and padding issues. – fgrieu Oct 19 '22 at 06:32