How does ccrypt work?

Question

On many Linux/GNU distributions CCrypt (ccrypt) by Peter Selinger is available as a command line utility to encrypt files. How does the protocol work and is it secure?

Note: this should not be confused with the CCCrypt API by Apple.

Answer 1

Initialization

Key hash function hashstring

Configuration: E is Rijndael with a block size and key size of 256 bits
Input: keystring
Output: hash

1. The keystring is padded up with bytes valued zero and split up in blocks of 256 bits $S_0$ to $S_n$;
2. The key blocks $K_0$ to $K_n$ are generated, where $K_0$ consist of $S_0$. The following blocks - if any, most passwords are shorter than 32 bytes - are defined as follows: $K_i = r(K_{i-1})\oplus S_i$ where $r$ is the full subkey derivation of Rijndael-256/256;
3. A hash block of 256 bits is initialized to zero: A hash block of 256 bits is initialized to all zero's, lets call this $H_{zero}$, then $H_0 = E(K_0, H_{zero})$, $H_i = E(K_i, E(K_{i-1}))$ for $i > 0$. In other words, the hash block is repeatedly encrypted with itself using the generated key stream $k_i$;
4. The output is $H_n$.

An answer by Peter Selinger - the creator and current maintainer - showed that the subkey derivation in step 2 is included on purpose. The source code from which this description has been derived does not show any comment or indication that this is the case. To replicate the functionality in another language it may be required to alter an existing Rijndael implementation as the result of the subkey derivation may not be returned to the caller.

Key derivation from the password

Key derivation simply consists of applying hashstring to the password. The password simply consists of a null terminated string of C char. Lets call this key $R$.

Nonce derivation

The nonce derivation consists of the performing the hashstring method over an ASCII string consisting of the following:

gethostname(host, 256);   /* ignore failures */
host[255] = 0;
gettimeofday(&tv, NULL);  /* ignore failures */
sprintf(acc, "%s,%ld,%ld,%ld,%d", host, (long)tv.tv_sec, (long)tv.tv_usec,
   (long)getpid(), count++);

in other words the input of haststring consists of a comma separated ASCII string of the hostname (255 characters max), the seconds in time-of-day, the microseconds in the time-of-day, the process ID and a static / process-wide counter.

Lets call the result $N$.

Creation of IV

The first 4 bytes/characters of the nonce are replaced by the following magic: c051, lets call this $N'$. Then the resulting value is encrypted using key $R$: ${IV} = E(R, N')$

Encryption

Encryption consists of a simple CFB mode using key $R$ and initialization vector $IV$, where the IV is prefixed to the ciphertext.

Decryption

The IV is first decrypted using key $R$ to validate if the if the magic is included in the nonce. The magic therefore acts as a key check value. After that the ciphertext is simply decrypted in CFB mode.