2

According to specification, to create data that will be signed with server private key for CertificateVerify message, we need to concatenate 64 space characters, the string "TLS 1.3, server CertificateVerify", null and the hash of all handshake messages to this point. During the handshake process, we choosing the cipher suite and signature scheme...

So, if the cipher suite is, for example, TLS_AES_256_GCM_SHA384 and the signature scheme is rsa_pss_rsae_sha256 (different hash algorithms), hash of which algorithm should be used as handshake hash in data to sign, SHA-384 or SHA-256?

kelalaka
  • 48,443
  • 11
  • 116
  • 196
Iceman
  • 137
  • 3

1 Answers1

2

rsa_pss_rsae_sha256 is the signature scheme that defines to use of the SHA-256 during the RSA-PSS signature algorithm.

Since Rabin-Signature Scheme, which is the first true signature scheme, the security of the signature is a part of the hashing a message and that enables us to sign arbitrary long messages, too. The rsa_pss_rsae_sha256 will use SHA-256 to hash the message before signing.

The RSA-PSS also needs a hash function to use in its Message Generation Function 1 (MGF1). RSA-PSS defined in RFC 8017 also known as RSASSA-PSS. The SHA-256 will be used in MFG1, too.

TLS_AES_256_GCM_SHA384: AES-GCM doesn't need the SHA-384. The SHA-384 suffix;

  • In Pre TLS 1.3, the appending hash function SHA-384 defines PRF to use in key derivation and Finished and HMAC when the cipher suites required.

    For cipher suites ending with _SHA384, the PRF is the TLS PRF [RFC5246] with SHA-384 as the hash function.

  • In TLS 1.3: TLS 1.3 finally defined a KDF; HKDF that replaces PRF and there is no more need for HMAC since all modes are authenticated encryption modes that use CCM, GCM, and Poly1305 message authenticators.

    The hash function in the cipher suite definition represents the hash function that is used to hash the handshake messages for the Finished message and HKDF.

What hash is concatenating with the data to be signed in TLS 1.3 CertificateVerify message?

Therefore it is SHA-384

Community
  • 1
kelalaka
  • 48,443
  • 11
  • 116
  • 196
  • 1
    rsa_pss_rsae_sha256 (and also rsa_pss_pss_sha256 with different OID in the cert) uses sha256 both for MGF1 and to hash the data (which in 1.3 includes the handshake transcript); see rfc8446 4.2.3. In 1.2 the hash in the ciphersuite is used in PRF which is used for key derivation and Finished, and HMAC if applicable, but not signature. In 1.3 it is used in HKDF which replaces PRF and is again used for key derivation and Finished and never HMAC, but not signature. – dave_thompson_085 Mar 08 '21 at 01:28
  • @dave_thompson_085 thanks for the comment. Better now? – kelalaka Mar 08 '21 at 09:41
  • 1
    Yes. I later (overnight) realized I overstated it: 1.2 uses the ciphersuite hash for PRF in new-1.2 suites but not pre-1.2 ones; you stated it for SHA384 only which implies new-1.2 and thus is correct. And I missed, and you got right, that 1.3 uses ciphersuite hash for transcript-hash as well as HKDF (which in turn is used for key derivation and Finished, and optionally export), – dave_thompson_085 Mar 10 '21 at 04:40