AES + CBC encryption for a stream of UDP packets

Question

I am developing an application that is based on UDP, and I need to send a stream of packets. As you can imagine, packets can get lost or corrupted. I need to make sure that the content of those packets is encrypted, and I also need to be able to certify that the packet is correct when I receive it.

Finally, since performance is crucial in my application, I would like to keep the encryption overhead as small as possible. Often times, packets are relatively small, so even a few blocks can build up a relatively big overhead.

I don't have a reliable previous experience with cryptography, so I thought it was better not to trust myself on this.

Here is the strategy I have used until now (my application is far from being released, so having to change it more or less completely won't be so much of a problem).

• First of all, packets are of fixed size, which is usually multiple of the block size of AES. I have no need whatsoever for padding, since messages aren't variable in length.

• I use a shared key (obviously) and two fixed IVs, each endpoint uses one of the two to encrypt its messages. I use AES in CBC mode.

• Since obviously using a fixed IV leads to a whole bunch of security flaws, I need to make my first block vary in a uniformly random fashion: that will act as IV for the next blocks.

• Therefore, I prepend a block at the beginning of my packet. Its content goes as follows:

  • First four bytes: current timestamp in seconds
  • Next 12 bytes: zeros
  • I compute the sha256 hash of the message (32 bytes)
  • I xor the timestamp + zeros block with the first half of the hash
  • I xor the result with the second half of the hash
• When I receive the packet on the other endpoint, I decrypt it with the key and the remote iv.
• I compute the sha256 hash of the message (without the first block), xor the first half and the second half.
• I xor the result of the hashing procedure with the first block.
• If I get a timestamp that is no more than a few seconds ago, and 12 bytes of zeros, then the packet is correct.
• If the packet is not correct, I just drop it, exactly as if I didn't receive it. No answer is sent back.

What are the flaws in this procedure?

Answer 1

At a high level, the major flaw is that you are rolling your own crypto protocol. You should strongly consider using a standardized protocol like DTLS.

Some specific problems:

1. Symmetric key distribution is left unspecified. Keys must be changed occasionally to thwart distinguishers.
2. No way to recover from symmetric key compromise.
3. Your message authentication code follows no existing standards and therefore is likely not secure.
4. If the attacker can get the IV, which is fixed, so likely not too hard, and can influence the first block of plaintext (which wouldn't be too hard since you are using timestamps), they can do some nasty things.
5. The fixed IVs add complexity that is unnecessary. For example, you have to trust the endpoints to not get the IVs mixed up.

There are probably a lot more issues that I haven't thought of in the short time I've been thinking about it. To solve this, use something standardized.

Answer 2

The "interesting" part of your encryption is here:

Therefore, I prepend a block at the beginning of my packet. Its content goes as follows:

• First four bytes: current timestamp in seconds
• Next 12 bytes: zeros
• I compute the sha256 hash of the message (32 bytes)
• I xor the timestamp + zeros block with the first half of the hash
• I xor the result with the second half of the hash

Since XOR is commutative and associative, order does not matter and you are XORing the zero-padded timestamp with a hash from a function defined as the XOR of the two halves of SHA-256.

This should ring some warning bells. There is a reason SHA-256 is 256 bits long, and that's because collision resistance of a hash is only $n/2$ bits. By halving the length of the hash (which you could equally strongly do by truncation) you reduce its collision resistance to 64 bits, which is not considered strong enough.

Now, if you only relied on the preimage or second preimage resistance of the hash function, you would be OK. However, the security of encrypted hash as a MAC relies on its collision resistance.

Instead, you could just use a normal MAC (like HMAC), preferably in encrypt-then-MAC. Or better yet, an authenticated encryption mode (like GCM).

That does not deal with the lack of IV, however. If you continue using CBC, you can just generate a random number for that purpose, if you must, no need to get fancy. CTR-based modes, like GCM, require a unique nonce and cannot recover this way from a duplicate.

---

Your options, from most to least recommendable are:

1. Don't roll your own protocol, but use (D)TLS.
2. If you must roll your own protocol, use an authenticated encryption mode.
3. If you can't, at least use standard encryption and MAC.