7

I use Backblaze to back up my computer. You restore files from your backups by selecting files to restore, which are then packed into large zip files. Of course, it's fairly rare to be able to download a 500GB zip file without a connection interruption, so a sane developer would implement support for the HTTP Range header to allow users to resume downloads.

They have not done so. Instead, they have a boutique download utility that specifies the requested byte ranges by emulating a POSTed HTML form. This utility does all the stuff you'd expect a normal download manager to do, like downloading with multiple connections at a time and resuming partially completed downloads, but due to some dodgy design issues (like opening a fully-fledged process, not a thread, for each 40MB block) it is rather inefficient on fast (>100 Mbps) connections. It also is Windows-exclusive.

I'm trying to write an open source replacement in Node.js that removes some of the suck, but I'm up against a roadblock: one of the fields the utility sends in its POST requests is called "bzsanity" and is a 16-bit checksum over the account email address. Unfortunately, I can't figure out what the algorithm is. Maybe I'm just dumb, but I'm hoping you guys can help me out.

Here are some checksum values:

If you want more test vectors, I can probably deliver. I've tried adding the bytes in an accumulator and a few variants of CRC-16, and those approaches don't work.

Reid Rankin
  • 173
  • 3
  • Maybe a few years too late but... Did you ever finish making a replacement in node.js? I've got several 300-500GB zip files I need to get down, and even with gigabit internet download speeds are slow. (I'm using aria2c 1 thread, cos as you said, no HTTP Range header.. I'm on Linux so...can't use there DL apps.) – Mint Aug 28 '19 at 09:13
  • 1
    @Mint I did, and then I promptly forgot about it for three years. Thanks for making sure I followed through, if a bit belatedly. – Reid Rankin Aug 28 '19 at 18:47
  • legend!!! I had little hope that I'd ever hear back let along a fully working node.js app! Gave it a try just now and it does indeed work, "(513.61 Mbps instantaneous, 491.52 Mbps total, ETA: an hour)" An hour sure beats like ~10h I was getting. Thank you so much for sharing. – Mint Aug 29 '19 at 00:11

2 Answers2

10

I'll use "[email protected]" for the sake of example.

Algorithm

  1. Convert the email address to its ASCII bytes. For example, the ASCII bytes for "[email protected]" are 74 65 73 74 40 74 65 73 74 2E 63 6F 6D.
  2. Make a lowercase string out of those hex bytes. Using the running example, this would produce "7465737440746573742e636f6d".
  3. Compute the SHA-1 hash of that string. For example, SHA1("7465737440746573742e636f6d") = 90 A2 78 5A 31 39 E2 2A 3D F7 56 90 0A F3 79 87 A9 35 03 16.
  4. Make a lowercase string out of those hex bytes. Using the running example, this would produce "90a2785a3139e22a3df756900af37987a9350316".
  5. Concatenate the 2nd, 4th, 6th, and 8th characters in that string to produce the bzsanity value. Using the running example, this would be '0'+'2'+'8'+'a' = "028a".

And just for fun, here's a one-line snippet in Python to compute the bzsanity value:

import hashlib
"".join(map(lambda i: hashlib.sha1("".join(map(hex, map(ord, "[email protected]"))).replace("0x", "")).hexdigest()[i], [1, 3, 5, 7]))
Jason Geffner
  • 20,681
  • 1
  • 36
  • 75
  • ... How did you figure this out? I'm sitting here with my bit twiddling and XORing and trying to determine CRC coefficients, and SHA1 of all things... – Reid Rankin Mar 09 '16 at 22:26
  • 6
    I am shocked, he may be not human. – Ta Thanh Dinh Mar 09 '16 at 22:47
  • 2
    @MrNerdHair: I reverse engineered the client software. – Jason Geffner Mar 09 '16 at 22:50
  • @JasonGeffner Well, thanks! May I ask what tools you used? I looked at it under IDA Pro, but I haven't used it extensively before, so I didn't get far. – Reid Rankin Mar 09 '16 at 22:52
  • 6
    IDA Pro + OllyDbg. Because of the compiler they used, it was nearly impossible to figure this out without dynamic analysis, so don't kick yourself too hard ;) – Jason Geffner Mar 09 '16 at 22:55
  • I added to the answer a one-line snippet in Python to compute the bzsanity value. – Jason Geffner Mar 10 '16 at 17:35
  • 1
    @JasonGeffner awesome work, although IMO it is better if this site does not become "do my reversing of me" site, but rather provides advice on how to tackle difficulties with reversing. – Vitaly Osipov Mar 11 '16 at 20:25
  • 4
    @VitalyOsipov: My hope is to show people that for all these "figure out the checksum by looking at sample inputs and outputs" questions, you almost always need to reverse engineer the code itself. Expect me to point people to this answer (and this specific comment) going forward ;) – Jason Geffner Mar 11 '16 at 20:27
4

As CTO and founder of Backblaze, I wrote the original source code of the Backblaze client, and Jason Geffner above is correct. That is:

  1. hexencode the email address (all lowercase, email addresses are not case sensitive)
  2. take the sha1 - the result should be a 40 byte human readable all lowercase string
  3. if the sha1 characters have "zero" for the index of the very initial character, then take the characters at index 1, 3, 5, and 7.

-- BrianW

brianwski
  • 41
  • 2