Fix all warnings given by UBSAN

The easiest way to compile and test the server with UBSAN is to run:
./BUILD/compile-pentium64-ubsan
and then run mysql-test-run.
After this commit, one should be able to run this without any UBSAN
warnings. There is still a few compiler warnings that should be fixed
at some point, but these do not expose any real bugs.

The 'special' cases where we disable, suppress or circumvent UBSAN are:
- ref10 source (as here we intentionally do some shifts that UBSAN
  complains about.
- x86 version of optimized int#korr() methods. UBSAN do not like unaligned
  memory access of integers.  Fixed by using byte_order_generic.h when
  compiling with UBSAN
- We use smaller thread stack with ASAN and UBSAN, which forced me to
  disable a few tests that prints the thread stack size.
- Verifying class types does not work for shared libraries. I added
  suppression in mysql-test-run.pl for this case.
- Added '#ifdef WITH_UBSAN' when using integer arithmetic where it is
  safe to have overflows (two cases, in item_func.cc).

Things fixed:
- Don't left shift signed values
  (byte_order_generic.h, mysqltest.c, item_sum.cc and many more)
- Don't assign not non existing values to enum variables.
- Ensure that bool and enum values are properly initialized in
  constructors.  This was needed as UBSAN checks that these types has
  correct values when one copies an object.
  (gcalc_tools.h, ha_partition.cc, item_sum.cc, partition_element.h ...)
- Ensure we do not called handler functions on unallocated objects or
  deleted objects.
  (events.cc, sql_acl.cc).
- Fixed bugs in Item_sp::Item_sp() where we did not call constructor
  on Query_arena object.
- Fixed several cast of objects to an incompatible class!
  (Item.cc, Item_buff.cc, item_timefunc.cc, opt_subselect.cc, sql_acl.cc,
   sql_select.cc ...)
- Ensure we do not do integer arithmetic that causes over or underflows.
  This includes also ++ and -- of integers.
  (Item_func.cc, Item_strfunc.cc, item_timefunc.cc, sql_base.cc ...)
- Added JSON_VALUE_UNITIALIZED to json_value_types and ensure that
  value_type is initialized to this instead of to -1, which is not a valid
  enum value for json_value_types.
- Ensure we do not call memcpy() when second argument could be null.
- Fixed that Item_func_str::make_empty_result() creates an empty string
  instead of a null string (safer as it ensures we do not do arithmetic
  on null strings).

Other things:

- Changed struct st_position to an OBJECT and added an initialization
  function to it to ensure that we do not copy or use uninitialized
  members. The change to a class was also motived that we used "struct
  st_position" and POSITION randomly trough the code which was
  confusing.
- Notably big rewrite in sql_acl.cc to avoid using deleted objects.
- Changed in sql_partition to use '^' instead of '-'. This is safe as
  the operator is either 0 or 0x8000000000000000ULL.
- Added check for select_nr < INT_MAX in JOIN::build_explain() to
  avoid bug when get_select() could return NULL.
- Reordered elements in POSITION for better alignment.
- Changed sql_test.cc::print_plan() to use pointers instead of objects.
- Fixed bug in find_set() where could could execute '1 << -1'.
- Added variable have_sanitizer, used by mtr.  (This variable was before
  only in 10.5 and up).  It can now have one of two values:
  ASAN or UBSAN.
- Moved ~Archive_share() from ha_archive.cc to ha_archive.h and marked
  it virtual. This was an effort to get UBSAN to work with loaded storage
  engines. I kept the change as the new place is better.
- Added in CONNECT engine COLBLK::SetName(), to get around a wrong cast
  in tabutil.cpp.
- Added HAVE_REPLICATION around usage of rgi_slave, to get embedded
  server to compile with UBSAN. (Patch from Marko).
- Added #ifdef for powerpc64 to avoid a bug in old gcc versions related
  to integer arithmetic.

Changes that should not be needed but had to be done to suppress warnings
from UBSAN:

- Added static_cast<<uint16_t>> around shift to get rid of a LOT of
  compiler warnings when using UBSAN.
- Had to change some '/' of 2 base integers to shift to get rid of
  some compile time warnings.

Reviewed by:
- Json changes: Alexey Botchkov
- Charset changes in ctype-uca.c: Alexander Barkov
- InnoDB changes & Embedded server: Marko Mäkelä
- sql_acl.cc changes: Vicențiu Ciorbaru
- build_explain() changes: Sergey Petrunia

6 files changed, 17 insertions, 14 deletions

diff --git a/storage/mroonga/lib/mrn_multiple_column_key_codec.cpp b/storage/mroonga/lib/mrn_multiple_column_key_codec.cpp
index 5536eecb255..73639685d0e 100644
--- a/storage/mroonga/lib/mrn_multiple_column_key_codec.cpp
+++ b/storage/mroonga/lib/mrn_multiple_column_key_codec.cpp
@@ -675,7 +675,8 @@ namespace mrn {
                                 &normalized, &normalized_length, NULL);
       uint16 new_blob_data_length;
       if (normalized_length <= UINT_MAX16) {
-        memcpy(grn_key, normalized, normalized_length);
+        if (normalized_length)
+          memcpy(grn_key, normalized, normalized_length);
         if (normalized_length < *mysql_key_size) {
           memset(grn_key + normalized_length,
                  '\0', *mysql_key_size - normalized_length);
diff --git a/storage/mroonga/vendor/groonga/lib/alloc.c b/storage/mroonga/vendor/groonga/lib/alloc.c
index 2e28431595a..5e556b83712 100644
--- a/storage/mroonga/vendor/groonga/lib/alloc.c
+++ b/storage/mroonga/vendor/groonga/lib/alloc.c
@@ -310,13 +310,13 @@ grn_alloc_info_free(grn_ctx *ctx)
 }
 #endif /* USE_MEMORY_DEBUG */
 
-#define GRN_CTX_SEGMENT_SIZE    (1<<22)
+#define GRN_CTX_SEGMENT_SIZE    (1U <<22)
 #define GRN_CTX_SEGMENT_MASK    (GRN_CTX_SEGMENT_SIZE - 1)
 
-#define GRN_CTX_SEGMENT_WORD    (1<<31)
-#define GRN_CTX_SEGMENT_VLEN    (1<<30)
-#define GRN_CTX_SEGMENT_LIFO    (1<<29)
-#define GRN_CTX_SEGMENT_DIRTY   (1<<28)
+#define GRN_CTX_SEGMENT_WORD    (1U <<31)
+#define GRN_CTX_SEGMENT_VLEN    (1U <<30)
+#define GRN_CTX_SEGMENT_LIFO    (1U <<29)
+#define GRN_CTX_SEGMENT_DIRTY   (1U <<28)
 
 void
 grn_alloc_init_ctx_impl(grn_ctx *ctx)
@@ -400,8 +400,8 @@ grn_ctx_alloc(grn_ctx *ctx, size_t size, int flags,
       header[0] = i;
       header[1] = (int32_t) size;
     } else {
-      i = ctx->impl->currseg;
-      mi = &ctx->impl->segs[i];
+      if ((i = ctx->impl->currseg) >= 0)
+        mi = &ctx->impl->segs[i];
       if (i < 0 || size + mi->nref > GRN_CTX_SEGMENT_SIZE) {
         for (i = 0, mi = ctx->impl->segs;; i++, mi++) {
           if (i >= GRN_CTX_N_SEGMENTS) {
diff --git a/storage/mroonga/vendor/groonga/lib/db.c b/storage/mroonga/vendor/groonga/lib/db.c
index f3769f9aa4c..418335aaf00 100644
--- a/storage/mroonga/vendor/groonga/lib/db.c
+++ b/storage/mroonga/vendor/groonga/lib/db.c
@@ -12494,7 +12494,7 @@ grn_db_init_builtin_types(grn_ctx *ctx)
                 GRN_OBJ_KEY_VAR_SIZE, 1 << 16);
   if (!obj || DB_OBJ(obj)->id != GRN_DB_TEXT) { return GRN_FILE_CORRUPT; }
   obj = deftype(ctx, "LongText",
-                GRN_OBJ_KEY_VAR_SIZE, 1 << 31);
+                GRN_OBJ_KEY_VAR_SIZE, 1U << 31);
   if (!obj || DB_OBJ(obj)->id != GRN_DB_LONG_TEXT) { return GRN_FILE_CORRUPT; }
   obj = deftype(ctx, "TokyoGeoPoint",
                 GRN_OBJ_KEY_GEO_POINT, sizeof(grn_geo_point));
diff --git a/storage/mroonga/vendor/groonga/lib/pat.c b/storage/mroonga/vendor/groonga/lib/pat.c
index 642173e2fdc..01f6108fbd0 100644
--- a/storage/mroonga/vendor/groonga/lib/pat.c
+++ b/storage/mroonga/vendor/groonga/lib/pat.c
@@ -899,7 +899,7 @@ chop(grn_ctx *ctx, grn_pat *pat, const char **key, const char *end, uint32_t *lk
   case GRN_OBJ_KEY_FLOAT :\
     if ((size) == sizeof(int64_t)) {\
       int64_t v = *(int64_t *)(key);\
-      v ^= ((v >> 63)|(1LL << 63));\
+      v ^= ((v >> 63)|(1ULL << 63));\
       grn_hton((keybuf), &v, (size));\
     }\
     break;\
@@ -924,7 +924,7 @@ chop(grn_ctx *ctx, grn_pat *pat, const char **key, const char *end, uint32_t *lk
     if ((size) == sizeof(int64_t)) {\
       int64_t v;\
       grn_hton(&v, (key), (size));\
-      *((int64_t *)(keybuf)) = v ^ (((v^(1LL<<63))>> 63)|(1LL<<63));  \
+      *((int64_t *)(keybuf)) = v ^ ((((int64_t)(v^(1ULL<<63)))>> 63)|(1ULL<<63)); \
     }\
     break;\
   }\
diff --git a/storage/mroonga/vendor/groonga/lib/proc/proc_select.c b/storage/mroonga/vendor/groonga/lib/proc/proc_select.c
index 605fd42239f..1f2a5005401 100644
--- a/storage/mroonga/vendor/groonga/lib/proc/proc_select.c
+++ b/storage/mroonga/vendor/groonga/lib/proc/proc_select.c
@@ -2989,7 +2989,8 @@ grn_select(grn_ctx *ctx, grn_select_data *data)
     char *cp = cache_key;
 
 #define PUT_CACHE_KEY(string)                                   \
-    grn_memcpy(cp, (string).value, (string).length);            \
+    if ((string).value)                                         \
+      grn_memcpy(cp, (string).value, (string).length);          \
     cp += (string).length;                                      \
     *cp++ = '\0'
 
diff --git a/storage/mroonga/vendor/groonga/lib/str.c b/storage/mroonga/vendor/groonga/lib/str.c
index 6b2d17769ca..4f0a3a98699 100644
--- a/storage/mroonga/vendor/groonga/lib/str.c
+++ b/storage/mroonga/vendor/groonga/lib/str.c
@@ -46,7 +46,7 @@ grn_str_charlen_utf8(grn_ctx *ctx, const unsigned char *str, const unsigned char
   if (*str & 0x80) {
     int i;
     int len;
-    GRN_BIT_SCAN_REV(~(*str << 24), len);
+    GRN_BIT_SCAN_REV(~(((uint) *str) << 24), len);
     len = 31 - len;
     if ((unsigned int)(len - 2) >= 3) {  /* (len == 1 || len >= 5) */
       GRN_LOG(ctx, GRN_LOG_WARNING,
@@ -1963,7 +1963,8 @@ grn_bulk_write(grn_ctx *ctx, grn_obj *buf, const char *str, unsigned int len)
     if ((rc = grn_bulk_resize(ctx, buf, GRN_BULK_VSIZE(buf) + len))) { return rc; }
   }
   curr = GRN_BULK_CURR(buf);
-  grn_memcpy(curr, str, len);
+  if (str)
+    grn_memcpy(curr, str, len);
   GRN_BULK_INCR_LEN(buf, len);
   return rc;
 }