6.6 TLS handshake

Once a session has been initialized and a network connection has been set up, TLS and DTLS protocols perform a handshake. The handshake is the actual key exchange.

Function: int gnutls_handshake (gnutls_session_t session)

session: is a gnutls_session_t type.

This function does the handshake of the TLS/SSL protocol, and initializes the TLS connection.

This function will fail if any problem is encountered, and will return a negative error code. In case of a client, if the client has asked to resume a session, but the server couldn’t, then a full handshake will be performed.

The non-fatal errors expected by this function are: GNUTLS_E_INTERRUPTED , GNUTLS_E_AGAIN , GNUTLS_E_WARNING_ALERT_RECEIVED , and GNUTLS_E_GOT_APPLICATION_DATA , the latter only in a case of rehandshake.

The former two interrupt the handshake procedure due to the lower layer being interrupted, and the latter because of an alert that may be sent by a server (it is always a good idea to check any received alerts). On these errors call this function again, until it returns 0; cf. gnutls_record_get_direction() and gnutls_error_is_fatal() . In DTLS sessions the non-fatal error GNUTLS_E_LARGE_PACKET is also possible, and indicates that the MTU should be adjusted.

If this function is called by a server after a rehandshake request then GNUTLS_E_GOT_APPLICATION_DATA or GNUTLS_E_WARNING_ALERT_RECEIVED may be returned. Note that these are non fatal errors, only in the specific case of a rehandshake. Their meaning is that the client rejected the rehandshake request or in the case of GNUTLS_E_GOT_APPLICATION_DATA it could also mean that some data were pending. A client may receive that error code if it initiates the handshake and the server doesn’t agreed.

Returns: GNUTLS_E_SUCCESS on success, otherwise a negative error code.

Function: void gnutls_handshake_set_timeout (gnutls_session_t session, unsigned int ms)

session: is a gnutls_session_t type.

ms: is a timeout value in milliseconds

This function sets the timeout for the TLS handshake process to the provided value. Use an ms value of zero to disable timeout, or GNUTLS_DEFAULT_HANDSHAKE_TIMEOUT for a reasonable default value. For the DTLS protocol, the more detailed gnutls_dtls_set_timeouts() is provided.

This function requires to set a pull timeout callback. See gnutls_transport_set_pull_timeout_function() .

Since: 3.1.0

In GnuTLS 3.5.0 and later it is recommended to use gnutls_session_set_verify_cert for the handshake process to ensure the verification of the peer’s identity.

In older GnuTLS versions it is required to manually verify the peer’s certificate during the handshake by using gnutls_certificate_set_verify_function, and gnutls_certificate_verify_peers2. See Certificate authentication for more information.

void gnutls_session_set_verify_cert (gnutls_session_t session, const char * hostname, unsigned flags)

int gnutls_certificate_verify_peers2 (gnutls_session_t session, unsigned int * status)