Bug#11756764 48726: MYSQLD KEEPS CRASHING WITH SIGSEGV

                    WITH MYISAM_USE_MMAP ENABLED

MySQL server can crash due to segmentation fault when
started with myisam_use_mmap.

The reason behind this being, while making a request to
unmap (munmap) the previously mapped memory (mmap), the
size passed was 7 bytes larger than the size requested at
the time of mapping. This can eventually unmap the adjacent
memory mapped block, belonging to some other memory-map pool.
Hence the subsequent call to mmap can map a region which was
still a valid memory mapped area.

Fixed by removing the extra 7-byte margin which was erroneously
added to the size, used for unmappping.


storage/myisam/mi_close.c:
  Bug#11756764 48726: MYSQLD KEEPS CRASHING WITH SIGSEGV
                      WITH MYISAM_USE_MMAP ENABLED
  
  Added a condition to call _mi_unmap_file() in case
  of compressed records. mi_munmap_file() is called
  otherwise.
storage/myisam/mi_packrec.c:
  Bug#11756764 48726: MYSQLD KEEPS CRASHING WITH SIGSEGV
                      WITH MYISAM_USE_MMAP ENABLED
  
  mi_dynmap_file() function, after successfully executing
  mmap, stores the total size in info->s->mapped_length
  variable. Now, if mi_dynmap_file() is invoked with a size
  with an extra 7-byte margin (MEMMAP_EXTRA_MARGIN),
  the margin will eventually also get stored in mapped_length.
  So, un-mapping function can simply use the value stored in
  mapped_length in order to unmap the previously mapped
  region.

1 files changed, 4 insertions, 3 deletions

diff --git a/storage/myisam/mi_packrec.c b/storage/myisam/mi_packrec.c
index e6ad4c3defc..005f8c65d71 100644
--- a/storage/myisam/mi_packrec.c
+++ b/storage/myisam/mi_packrec.c
@@ -1553,13 +1553,14 @@ my_bool _mi_memmap_file(MI_INFO *info)
 
 void _mi_unmap_file(MI_INFO *info)
 {
-  VOID(my_munmap((char*) info->s->file_map, 
-                 (size_t) info->s->mmaped_length + MEMMAP_EXTRA_MARGIN));
+  DBUG_ASSERT(info->s->options & HA_OPTION_COMPRESS_RECORD);
+
+  VOID(my_munmap((char*) info->s->file_map, (size_t) info->s->mmaped_length));
 
   if (myisam_mmap_size != SIZE_T_MAX)
   {
     pthread_mutex_lock(&THR_LOCK_myisam_mmap);
-    myisam_mmap_used-= info->s->mmaped_length + MEMMAP_EXTRA_MARGIN;
+    myisam_mmap_used-= info->s->mmaped_length;
     pthread_mutex_unlock(&THR_LOCK_myisam_mmap);
   }
 }