MDEV-14051 'Undo log record is too big.' error occurring in very narrow range of string lengthsbb-5.5-marko

InnoDB was writing unnecessary information to the
update undo log records. Most notably, if an indexed column is updated,
the old value of the column would be logged twice: first as part of
the update vector, and then another time because it is an indexed column.

Because the InnoDB undo log record must fit in a single page,
this would cause unnecessary failure of certain updates.

Even after this fix, InnoDB still seems to be unnecessarily logging
indexed column values for non-updated columns. It seems that non-updated
secondary index columns only need to be logged when a PRIMARY KEY
column is updated. To reduce risk, we are not fixing this remaining flaw
in GA versions.

trx_undo_page_report_modify(): Log updated indexed columns only once.

1 files changed, 23 insertions, 1 deletions

diff --git a/storage/xtradb/trx/trx0rec.c b/storage/xtradb/trx/trx0rec.c
index ef42152aeb7..fc38f81c594 100644
--- a/storage/xtradb/trx/trx0rec.c
+++ b/storage/xtradb/trx/trx0rec.c
@@ -1,6 +1,7 @@
 /*****************************************************************************
 
 Copyright (c) 1996, 2012, Oracle and/or its affiliates. All Rights Reserved.
+Copyright (c) 2017, MariaDB Corporation.
 
 This program is free software; you can redistribute it and/or modify it under
 the terms of the GNU General Public License as published by the Free Software
@@ -786,7 +787,25 @@ trx_undo_page_report_modify(
 			const dict_col_t*	col
 				= dict_table_get_nth_col(table, col_no);
 
-			if (col->ord_part) {
+			if (!col->ord_part) {
+				continue;
+			}
+
+			if (update) {
+				for (i = 0; i < update->n_fields; i++) {
+					const dict_field_t* f
+						= dict_index_get_nth_field(
+							index,
+							upd_get_nth_field(
+								update, i)
+							->field_no);
+					if (f->col == col) {
+						goto already_logged;
+					}
+				}
+			}
+
+			if (TRUE) {
 				ulint	pos;
 
 				/* Write field number to undo log */
@@ -836,6 +855,9 @@ trx_undo_page_report_modify(
 					ptr += flen;
 				}
 			}
+
+already_logged:
+			continue;
 		}
 
 		mach_write_to_2(old_ptr, ptr - old_ptr);