MDEV-10332 support for OpenSSL 1.1 and LibreSSL

Initial support

tested against OpenSSL 1.0.1, 1.0.2, 1.1.0, Yassl and LibreSSL
not working on Windows with native SChannel support, due to wrong cipher
mapping: Latter one requires push of CONC-241 fixes.
Please note that OpenSSL 0.9.8 and OpenSSL 1.1.0 will not work: Even if
the build succeeds, test cases will fail with various errors, especially
when using different tls libraries or versions for client and server.

1 files changed, 10 insertions, 0 deletions

diff --git a/extra/yassl/src/handshake.cpp b/extra/yassl/src/handshake.cpp
index 407e4092ccc..6e181a997bd 100644
--- a/extra/yassl/src/handshake.cpp
+++ b/extra/yassl/src/handshake.cpp
@@ -788,6 +788,16 @@ int DoProcessReply(SSL& ssl)
             needHdr = true;
         else {
             buffer >> hdr;
+            /*
+              According to RFC 4346 (see "7.4.1.3. Server Hello"), the Server Hello
+              packet needs to specify the highest supported TLS version, but not
+              higher than what client requests. YaSSL highest supported version is
+              TLSv1.1 (=3.2) - if the client requests a higher version, downgrade it
+              here to 3.2.
+              See also Appendix E of RFC 5246 (TLS 1.2)
+            */
+            if (hdr.version_.major_ == 3 && hdr.version_.minor_ > 2)
+              hdr.version_.minor_ = 2;
             ssl.verifyState(hdr);
         }