MDEV-11853: semisync thread can be killed after sync binlog but before ACK in the sync state10.4-MDEV-11853

Problem:
========
If a primary is shutdown during an active semi-sync connection
during the period when the primary is awaiting an ACK, the primary
hard kills the active communication thread and does not ensure the
transaction was received by a replica. This can lead to an
inconsistent replication state.

Solution:
========
During shutdown, the primary should wait for an ACK or timeout
before hard killing a thread which is awaiting a communication. We
extend the `SHUTDOWN WAIT FOR SLAVES` logic to identify and ignore
any threads waiting for a semi-sync ACK in phase 1. Then, before
stopping the ack receiver thread, the shutdown is delayed until all
waiting semi-sync connections receive an ACK or time out. The
connections are then killed in phase 2.

Reviewed By:
============

1 files changed, 14 insertions, 0 deletions

diff --git a/sql/sql_class.h b/sql/sql_class.h
index 7570211f586..a65e21f80c1 100644
--- a/sql/sql_class.h
+++ b/sql/sql_class.h
@@ -4826,12 +4826,26 @@ public:
   LOG_INFO *current_linfo;
   Slave_info *slave_info;
 
+  /*
+    Indicates if this thread is suspunded due to awaiting an ACK from a
+    replica. True if suspended, false otherwise.
+  */
+  bool awaiting_semisync_ack;
+
   void set_current_linfo(LOG_INFO *linfo);
   void reset_current_linfo() { set_current_linfo(0); }
+  void set_awaiting_semisync_ack(bool status) { awaiting_semisync_ack= status; }
 
   int register_slave(uchar *packet, size_t packet_length);
   void unregister_slave();
   bool is_binlog_dump_thread();
+  bool is_awaiting_semisync_ack()
+  {
+    mysql_mutex_lock(&LOCK_thd_data);
+    bool res= awaiting_semisync_ack;
+    mysql_mutex_unlock(&LOCK_thd_data);
+    return res;
+  }
 #endif
 
   inline ulong wsrep_binlog_format() const