Fix for BUG#17486: IM: race condition on exit.

The problem was that IM stoped guarded instances on shutdown,
but didn't wait for them to stop.

The fix is to wait for guarded instances to stop before exitting
from the main thread.

The idea is that Instance-monitoring thread should add itself
to Thread_registry so that it will be taken into account on shutdown.
However, Thread_registry should not signal it on shutdown in order to
not interrupt wait()/waitpid().


server-tools/instance-manager/guardian.cc:
  Be more verbose.
server-tools/instance-manager/instance.cc:
  Register mysqld-monitoring thread in Thread_registry.
server-tools/instance-manager/instance.h:
  Add reference to Thread_registry.
server-tools/instance-manager/instance_map.cc:
  Pass Thread_registry reference to Instance.
server-tools/instance-manager/instance_map.h:
  Add reference to Thread_registry.
server-tools/instance-manager/listener.cc:
  Be more verbose.
server-tools/instance-manager/manager.cc:
  Be more verbose.
server-tools/instance-manager/mysql_connection.cc:
  Eliminate type-conversion warnings.
server-tools/instance-manager/thread_registry.cc:
  Be more verbose.
server-tools/instance-manager/thread_registry.h:
  Eliminate copy&paste, make impl-specific constructor private.

1 files changed, 22 insertions, 1 deletions

diff --git a/server-tools/instance-manager/manager.cc b/server-tools/instance-manager/manager.cc
index 3fb967fc352..4bd298eedec 100644
--- a/server-tools/instance-manager/manager.cc
+++ b/server-tools/instance-manager/manager.cc
@@ -156,7 +156,8 @@ void manager()
   */
 
   User_map user_map;
-  Instance_map instance_map(Options::Main::default_mysqld_path);
+  Instance_map instance_map(Options::Main::default_mysqld_path,
+                            thread_registry);
   Guardian_thread guardian_thread(thread_registry,
                                   &instance_map,
                                   Options::Main::monitoring_interval);
@@ -308,6 +309,8 @@ void manager()
   */
   pthread_cond_signal(&guardian_thread.COND_guardian);
 
+  log_info("Main loop: started.");
+
   while (!shutdown_complete)
   {
     int signo;
@@ -320,6 +323,20 @@ void manager()
       goto err;
     }
 
+    /*
+      The general idea in this loop is the following:
+        - we are waiting for SIGINT, SIGTERM -- signals that mean we should
+          shutdown;
+        - as shutdown signal is caught, we stop Guardian thread (by calling
+          Guardian_thread::request_shutdown());
+        - as Guardian_thread is stopped, it sends SIGTERM to this thread
+          (by calling Thread_registry::request_shutdown()), so that the
+          my_sigwait() above returns;
+        - as we catch the second SIGTERM, we send signals to all threads
+          registered in Thread_registry (by calling
+          Thread_registry::deliver_shutdown()) and waiting for threads to stop;
+    */
+
 #ifndef __WIN__
 /*
   On some Darwin kernels SIGHUP is delivered along with most
@@ -336,6 +353,8 @@ void manager()
     else
 #endif
     {
+      log_info("Main loop: got shutdown signal.");
+
       if (!guardian_thread.is_stopped())
       {
         guardian_thread.request_shutdown();
@@ -349,6 +368,8 @@ void manager()
     }
   }
 
+  log_info("Main loop: finished.");
+
 err:
   /* delete the pid file */
   my_delete(Options::Main::pid_file_name, MYF(0));