BUG#40611: MySQL cannot make a binary log after sequential number

beyond unsigned long.
BUG#44779: binlog.binlog_max_extension may be causing failure on 
next test in PB
      
NOTE1: this is the backport to next-mr.
NOTE2: already includes patch for BUG#44779.
      
Binlog file extensions would turn into negative numbers once the
variable used to hold the value reached maximum for signed
long. Consequently, incrementing value to the next (negative) number
would lead to .000000 extension, causing the server to fail.
                  
This patch addresses this issue by not allowing negative extensions
and by returning an error on find_uniq_filename, when the limit is
reached. Additionally, warnings are printed to the error log when the
limit is approaching. FLUSH LOGS will also report warnings to the
user, if the extension number has reached the limit. The limit has been
set to 0x7FFFFFFF as the maximum.

mysql-test/suite/binlog/t/binlog_max_extension.test:
  Test case added that checks the maximum available number for
  binlog extensions.
sql/log.cc:
  Changes to find_uniq_filename and test_if_number.
sql/log.h:
  Added macros with values for MAX_LOG_UNIQUE_FN_EXT and
  LOG_WARN_UNIQUE_FN_EXT_LEFT, as suggested in review.

2 files changed, 100 insertions, 0 deletions

diff --git a/mysql-test/suite/binlog/r/binlog_max_extension.result b/mysql-test/suite/binlog/r/binlog_max_extension.result
new file mode 100644
index 00000000000..af341db4536
--- /dev/null
+++ b/mysql-test/suite/binlog/r/binlog_max_extension.result
@@ -0,0 +1,8 @@
+call mtr.add_suppression("Next log extension: 2147483647. Remaining log filename extensions: 0.");
+call mtr.add_suppression("Log filename extension number exhausted:");
+call mtr.add_suppression("Can't generate a unique log-filename");
+RESET MASTER;
+FLUSH LOGS;
+Warnings:
+Warning	1098	Can't generate a unique log-filename master-bin.(1-999)
+
diff --git a/mysql-test/suite/binlog/t/binlog_max_extension.test b/mysql-test/suite/binlog/t/binlog_max_extension.test
new file mode 100644
index 00000000000..9f52d195e21
--- /dev/null
+++ b/mysql-test/suite/binlog/t/binlog_max_extension.test
@@ -0,0 +1,92 @@
+#  BUG#40611: MySQL cannot make a binary log after sequential number beyond 
+#             unsigned long.
+# 
+#  Problem statement
+#  =================
+#
+#    Extension for log file names might be created with negative 
+#    numbers (when counter used would wrap around), causing server 
+#    failure when incrementing -00001 (reaching number 000000 
+#    extension).
+#
+#  Test
+#  ====
+#    This tests aims at testing the a patch that removes negatives 
+#    numbers from log name extensions and checks that the server 
+#    reports gracefully that the limit has been reached.
+#
+#    It instruments index file to point to a log file close to
+#    the new maximum and calls flush logs to get warning.
+#
+
+call mtr.add_suppression("Next log extension: 2147483647. Remaining log filename extensions: 0.");
+call mtr.add_suppression("Log filename extension number exhausted:");
+call mtr.add_suppression("Can't generate a unique log-filename");
+
+
+-- source include/have_log_bin.inc
+RESET MASTER;
+
+-- let $MYSQLD_DATADIR= `select @@datadir`
+
+###############################################
+# check hitting maximum file name extension:
+###############################################
+
+##########
+# Prepare
+##########
+
+# 1. Stop master server
+-- write_file $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
+wait
+EOF
+-- shutdown_server 10
+-- source include/wait_until_disconnected.inc
+
+# 2. Prepare log and index file
+-- copy_file $MYSQLD_DATADIR/master-bin.index $MYSQLD_DATADIR/master-bin.index.orig
+-- copy_file $MYSQLD_DATADIR/master-bin.000001 $MYSQLD_DATADIR/master-bin.2147483646
+-- append_file $MYSQLD_DATADIR/master-bin.index
+master-bin.2147483646
+EOF
+
+# 3. Restart the server
+-- append_file $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
+restart
+EOF
+-- enable_reconnect
+-- source include/wait_until_connected_again.inc
+
+###########
+# Assertion
+###########
+
+# assertion: should throw warning
+FLUSH LOGS;
+
+##############
+# Clean up
+##############
+
+# 1. Stop the server
+-- write_file $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
+wait
+EOF
+-- shutdown_server 10
+-- source include/wait_until_disconnected.inc
+
+# 2. Undo changes to index and log files
+-- remove_file $MYSQLD_DATADIR/master-bin.index
+-- copy_file $MYSQLD_DATADIR/master-bin.index.orig $MYSQLD_DATADIR/master-bin.index
+-- remove_file $MYSQLD_DATADIR/master-bin.index.orig
+
+-- remove_file $MYSQLD_DATADIR/master-bin.2147483646
+-- remove_file $MYSQLD_DATADIR/master-bin.2147483647
+
+# 3. Restart the server
+-- append_file $MYSQLTEST_VARDIR/tmp/mysqld.1.expect
+restart
+EOF
+-- enable_reconnect
+-- source include/wait_until_connected_again.inc