Is encrypting a file with AES-CTR and then encrypting the AES key with RSA-4096 secure?

Question

I'm using microsoft visual studio 2019 c++. I have aes-ctr-128 implementation.

---

Encryption process:

1. Generate key pair for aes-ctr-128 using CryptGenKey() function
2. Encrypt whole 16000 bytes array using key\nonce that was generated previously
3. Encrypt key and nonce with RSA master key
4. Write encrypted key\nonce in "key.txt" and "nonce.txt"

Decryption process:

1. Read encrypted key\nonce
2. Decrypt encrypted key\nonce with RSA master key
3. Decrypt whole 16000 bytes array using key\nonce that was decrypted from file

The question is: does this algorithm provide secure encryption process?

Answer 1

First of all, CTR mode is an archaic mode of operation that only provides you confidentiality. In modern standards, we use Authenticated Encryption (AE) modes like AES-GCM or ChaCha20-Poly1305. AE provides you confidentiality, Integrity, and authentication in a bundle.

There is no need to encrypt the IV. They are never designed to be encrypted. They provide probabilistic encryption that is very useful if a key is used more than once.

To encrypt with RSA-4096 you need a proper padding scheme like PKCS#1.5 or OAEP. The latter, OAEP, is better since the former has more issues. Once you use proper padding you are fine (not exactly). Note that RSA is totally broken if someone is able to build Shor's algorithm with enough Q-bits.

It is not clear how are you going to keep your RSA-private keys safely. Is it turning into a chicken-egg problem? Another problem is what if someone plays with the key.txt file. Alas, you can no longer access your files! You need backups.

In these kinds of problems, we prefer passwords based key derivation function like PBKDF2 or Argon2id. Have a good password and derive the encryption key with good parameters and from there encrypt with a AES-256-GCM ^[*]. It actually uses CTR mode inside. You will only get an additional tag this is up to 128-bit storage overhead and this is nothing compared to your file size. The speed lost, see yourself below from the OpenSSL speed tool.

$openssl speed -evp aes-256-ctr
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-256-ctr     474901.33k  1490379.07k  2337490.52k  3131689.64k  3592602.97k  3086712.83k


$openssl speed -evp aes-128-gcm
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-256-gcm     435559.37k  1178543.34k  2184926.89k  3160146.94k  3705495.55k  3713903.27k

---

In the end, a Veracrypt may be good for you. Have a good password, generate it with diceware or Bip39 then you will have a good entropy. Use it while creating your volume.

• Diceware passphrase has an entropy around

  • A five-word has 64.6 bits
  • six words have 77.5 bits
  • seven words 90.4 bits
  • eight words 103 bits
  • nine words 116 bits
  • ten words 129 bits.

• Bip39 creates $\approx 2^{263}$ with 256 coins tossing and selecting 24 words.

And, don't forget to back up your files.

---

^{[*] Since we talked about Shor, we have to consider Grover, too.}