NULL MERGE UP to 5.1.

Apply innodb-5.0-* snapshots:  ss1489 and ss1547.

Fixes:
Bug#9709:  InnoDB inconsistensy causes "Operating System Error 32/33"
Bug#22819: SHOW INNODB STATUS crashes the server with an assertion failure under high load
Bug#25645: Assertion failure in file srv0srv.c
Bug#27294: insert into ... select ... causes crash with innodb_locks_unsafe_for_binlog=1
Bug#28138: indexing column prefixes produces corruption in InnoDB


innobase/btr/btr0btr.c:
  Apply innodb-5.0-* snapshots:  ss1489 and ss1547.
  
  Revision r1547:
  branches/5.0: Merge r1546 from trunk:
  
  When buffering an insert to a prefix index of a variable-length column,
  do not incorrectly mark the column as fixed-length.  (Bug #28138)
  
  ibuf_entry_build(): Instead of prefix_len, pass fixed_len to
  dtype_new_store_for_order_and_null_size().  Add debug assertions.
  
  btr_index_rec_validate(): Correct a comment about prefix indexes.
  
  rec_get_converted_size_new(), rec_convert_dtuple_to_rec_new(): Add
  debug assertions and comments.
  
  dict_col_type_assert_equal(): New debug function.
innobase/buf/buf0buf.c:
  Apply innodb-5.0-* snapshots:  ss1489 and ss1547.
  
  Revision r1502:
  branches/5.0: Add debug code for Bug 26081. This change has some debug
  assertions that have been promoted to normal assertions. These will need
  to be undone once we've nailed this bug.
innobase/buf/buf0lru.c:
  Apply innodb-5.0-* snapshots:  ss1489 and ss1547.
  
  Revision r1502:
  branches/5.0: Add debug code for Bug 26081. This change has some debug
  assertions that have been promoted to normal assertions. These will need
  to be undone once we've nailed this bug.
innobase/ibuf/ibuf0ibuf.c:
  Apply innodb-5.0-* snapshots:  ss1489 and ss1547.
  
  Revision r1547:
  branches/5.0: Merge r1546 from trunk:
  
  When buffering an insert to a prefix index of a variable-length column,
  do not incorrectly mark the column as fixed-length.  (Bug #28138)
  
  ibuf_entry_build(): Instead of prefix_len, pass fixed_len to
  dtype_new_store_for_order_and_null_size().  Add debug assertions.
  
  btr_index_rec_validate(): Correct a comment about prefix indexes.
  
  rec_get_converted_size_new(), rec_convert_dtuple_to_rec_new(): Add
  debug assertions and comments.
  
  dict_col_type_assert_equal(): New debug function.
innobase/include/buf0buf.ic:
  Apply innodb-5.0-* snapshots:  ss1489 and ss1547.
  
  Revision r1502:
  branches/5.0: Add debug code for Bug 26081. This change has some debug
  assertions that have been promoted to normal assertions. These will need
  to be undone once we've nailed this bug.
innobase/include/buf0lru.h:
  Apply innodb-5.0-* snapshots:  ss1489 and ss1547.
  
  Revision r1502:
  branches/5.0: Add debug code for Bug 26081. This change has some debug
  assertions that have been promoted to normal assertions. These will need
  to be undone once we've nailed this bug.
innobase/include/dict0dict.h:
  Apply innodb-5.0-* snapshots:  ss1489 and ss1547.
  
  Revision r1547:
  branches/5.0: Merge r1546 from trunk:
  
  When buffering an insert to a prefix index of a variable-length column,
  do not incorrectly mark the column as fixed-length.  (Bug #28138)
  
  ibuf_entry_build(): Instead of prefix_len, pass fixed_len to
  dtype_new_store_for_order_and_null_size().  Add debug assertions.
  
  btr_index_rec_validate(): Correct a comment about prefix indexes.
  
  rec_get_converted_size_new(), rec_convert_dtuple_to_rec_new(): Add
  debug assertions and comments.
  
  dict_col_type_assert_equal(): New debug function.
innobase/include/dict0dict.ic:
  Apply innodb-5.0-* snapshots:  ss1489 and ss1547.
  
  Revision r1547:
  branches/5.0: Merge r1546 from trunk:
  
  When buffering an insert to a prefix index of a variable-length column,
  do not incorrectly mark the column as fixed-length.  (Bug #28138)
  
  ibuf_entry_build(): Instead of prefix_len, pass fixed_len to
  dtype_new_store_for_order_and_null_size().  Add debug assertions.
  
  btr_index_rec_validate(): Correct a comment about prefix indexes.
  
  rec_get_converted_size_new(), rec_convert_dtuple_to_rec_new(): Add
  debug assertions and comments.
  
  dict_col_type_assert_equal(): New debug function.
innobase/include/dict0mem.h:
  Apply innodb-5.0-* snapshots:  ss1489 and ss1547.
  
  Revision r1537:
  branches/5.0: merge r1536 and partially r1535
  
  Change the comment to a more appropriate one. Discussed with Heikki on IM.
  
  Document that DICT_MAX_INDEX_COL_LEN should not be changed.
innobase/include/os0file.h:
  Apply innodb-5.0-* snapshots:  ss1489 and ss1547.
  
  Revision r1417:
  branches/5.0: Fix Bug#9709 by retrying (forever) if ERROR_SHARING_VIOLATION or
  ERROR_LOCK_VIOLATION is encountered during file operation.
  This is caused by backup software, so InnoDB should retry while the backup
  software is done with the file.
  
  Approved by:	Heikki
innobase/include/trx0trx.h:
  Apply innodb-5.0-* snapshots:  ss1489 and ss1547.
  
  Revision r1463:
  branches/5.0: merge r1462 from trunk:
  
  Fix typo in comment.
innobase/lock/lock0lock.c:
  Apply innodb-5.0-* snapshots:  ss1489 and ss1547.
  
  Revision r1458:
  branches/5.0: Fix Bug#22819, remove assertion.  (http://bugs.mysql.com/bug.php?id=22819)
innobase/log/log0log.c:
  Apply innodb-5.0-* snapshots:  ss1489 and ss1547.
  
  Revision r1520:
  Patch to allow monitor threads to stop before proceeding with normal shutdown. 
  Also have a separate time counter for tablespace monitor.
  
  reviewed by: Heikki
  
  
  Revision r1525:
  backport of r1524
  
  Log:
  Undo bad space formatting introduced in earlier commit r1521
  
  spotted by: Marko
innobase/mtr/mtr0mtr.c:
  Apply innodb-5.0-* snapshots:  ss1489 and ss1547.
  
  Revision r1502:
  branches/5.0: Add debug code for Bug 26081. This change has some debug
  assertions that have been promoted to normal assertions. These will need
  to be undone once we've nailed this bug.
innobase/os/os0file.c:
  Apply innodb-5.0-* snapshots:  ss1489 and ss1547.
  
  Revision r1417:
  branches/5.0: Fix Bug#9709 by retrying (forever) if ERROR_SHARING_VIOLATION or
  ERROR_LOCK_VIOLATION is encountered during file operation.
  This is caused by backup software, so InnoDB should retry while the backup
  software is done with the file.
  
  Approved by:	Heikki
innobase/rem/rem0rec.c:
  Apply innodb-5.0-* snapshots:  ss1489 and ss1547.
  
  Revision r1547:
  branches/5.0: Merge r1546 from trunk:
  
  When buffering an insert to a prefix index of a variable-length column,
  do not incorrectly mark the column as fixed-length.  (Bug #28138)
  
  ibuf_entry_build(): Instead of prefix_len, pass fixed_len to
  dtype_new_store_for_order_and_null_size().  Add debug assertions.
  
  btr_index_rec_validate(): Correct a comment about prefix indexes.
  
  rec_get_converted_size_new(), rec_convert_dtuple_to_rec_new(): Add
  debug assertions and comments.
  
  dict_col_type_assert_equal(): New debug function.
innobase/row/row0sel.c:
  Apply innodb-5.0-* snapshots:  ss1489 and ss1547.
  
  Revision r1456:
  branches/5.0: merge r1452 from trunk:
  
  Fix phantom reads (http://bugs.mysql.com/27197) following Heikki's
  patch in the bug followup.
innobase/srv/srv0srv.c:
  Apply innodb-5.0-* snapshots:  ss1489 and ss1547.
  
  Revision r1460:
  branches/5.0: Merge r1459 from trunk:
  
  Fix typo in the comment.
  
  
  Revision r1520:
  Patch to allow monitor threads to stop before proceeding with normal shutdown. 
  Also have a separate time counter for tablespace monitor.
  
  reviewed by: Heikki
sql/ha_innodb.cc:
  Apply innodb-5.0-* snapshots:  ss1489 and ss1547.
  
  Revision r1436:
  branches/5.0: Fix Bug#27294 by using trx returned by check_trx_exists()
  instead of prebuilt->trx. This has been fixed in 5.1 in r782.
  
  Approved by:	Heikki
  
  
  Revision r1443:
  branches/5.0: merge r1442 from trunk:
  
  Potential fix for Bug#25645:
  
  "Move innobase_release_stat_resources(trx) outside the 'if' in
  ha_innobase::external_lock(). That would add more safety that whatever
  MySQL does at a query end, there would be no risk of a hang on the btr
  search latch."
  
  Also call innobase_release_temporary_latches() in the beginning of
  ha_innobase::close().
  
  Approved by:	Heikki
  
  
  Revision r1454:
  branches/5.0: merge r1453 from trunk:
  
  Bugfix: only call innobase_release_temporary_latches() in case of current_thd
  is not NULL, otherwise we get NULL pointer dereferencing.
  
  
  
  Revision r1504:
  branches/5.0: Apply patch for Bug 27650 from MySQL.
  
  
  Revision r1539:
  Backport of r1538 from 5.1
  Do not return error in ha_innobase::info if srv_force_recovery >= 4. This is to allow for
  normal processing of the query by MySQL instead of generating an error.
  
  Reviewed by: Heikki

18 files changed, 286 insertions, 63 deletions

diff --git a/innobase/btr/btr0btr.c b/innobase/btr/btr0btr.c
index 07bc04feae6..a339405372d 100644
--- a/innobase/btr/btr0btr.c
+++ b/innobase/btr/btr0btr.c
@@ -2542,8 +2542,11 @@ btr_index_rec_validate(
 
 		rec_get_nth_field(rec, offsets, i, &len);
 
-		/* Note that prefix indexes are not fixed size even when
-		their type is CHAR. */
+		/* Note that if fixed_size != 0, it equals the
+		length of a fixed-size column in the clustered index.
+		A prefix index of the column is of fixed, but different
+		length.  When fixed_size == 0, prefix_len is the maximum
+		length of the prefix index column. */
 
 		if ((dict_index_get_nth_field(index, i)->prefix_len == 0
 		    && len != UNIV_SQL_NULL && fixed_size
diff --git a/innobase/buf/buf0buf.c b/innobase/buf/buf0buf.c
index f24f1744363..9df48495355 100644
--- a/innobase/buf/buf0buf.c
+++ b/innobase/buf/buf0buf.c
@@ -1220,7 +1220,8 @@ loop:
 		ut_ad(buf_validate());
 	}
 #endif
-	ut_ad(block->buf_fix_count > 0);
+	/* This debug code has been upgraded to narrow down Bug# 26081 */
+	ut_a(block->buf_fix_count > 0);
 	ut_ad(block->state == BUF_BLOCK_FILE_PAGE);
 
 	if (mode == BUF_GET_NOWAIT) {
@@ -1238,6 +1239,13 @@ loop:
 		if (!success) {
 			mutex_enter(&block->mutex);
 
+			/* This debug code has been added to narrow
+			down Bug# 26081 */
+			if (UNIV_UNLIKELY(block->buf_fix_count == 0)) {
+				buf_print_diagnostic(mtr, block);
+				ut_error;
+			}
+
 			block->buf_fix_count--;
 
 			mutex_exit(&block->mutex);
@@ -1362,7 +1370,14 @@ buf_page_optimistic_get_func(
 
 	if (UNIV_UNLIKELY(!success)) {
 		mutex_enter(&block->mutex);
-		
+
+		/* This debug code has been added to narrow
+		down Bug# 26081 */
+		if (UNIV_UNLIKELY(block->buf_fix_count == 0)) {
+			buf_print_diagnostic(mtr, block);
+			ut_error;
+		}
+
 		block->buf_fix_count--;
 
 		mutex_exit(&block->mutex);
@@ -1384,7 +1399,14 @@ buf_page_optimistic_get_func(
 		}
 
 		mutex_enter(&block->mutex);
-		
+
+		/* This debug code has been added to narrow
+		down Bug# 26081 */
+		if (UNIV_UNLIKELY(block->buf_fix_count == 0)) {
+			buf_print_diagnostic(mtr, block);
+			ut_error;
+		}
+
 		block->buf_fix_count--;
 
 		mutex_exit(&block->mutex);
@@ -1404,7 +1426,10 @@ buf_page_optimistic_get_func(
 		ut_ad(buf_validate());
 	}
 #endif
-	ut_ad(block->buf_fix_count > 0);
+	/* This debug code has been upgraded from ut_ad() to narrow
+	down Bug# 26081 */
+	ut_a(block->buf_fix_count > 0);
+
 	ut_ad(block->state == BUF_BLOCK_FILE_PAGE);
 
 #ifdef UNIV_DEBUG_FILE_ACCESSES
@@ -1493,7 +1518,14 @@ buf_page_get_known_nowait(
 	
 	if (!success) {
 		mutex_enter(&block->mutex);
-		
+
+		/* This debug code has been added to narrow
+		down Bug# 26081 */
+		if (UNIV_UNLIKELY(block->buf_fix_count == 0)) {
+			buf_print_diagnostic(mtr, block);
+			ut_error;
+		}
+
 		block->buf_fix_count--;
 
 		mutex_exit(&block->mutex);
@@ -1514,7 +1546,8 @@ buf_page_get_known_nowait(
 		ut_ad(buf_validate());
 	}
 #endif
-	ut_ad(block->buf_fix_count > 0);
+	/* This debug code has been upgraded to narrow down Bug# 26081 */
+	ut_a(block->buf_fix_count > 0);
 	ut_ad(block->state == BUF_BLOCK_FILE_PAGE);
 #ifdef UNIV_DEBUG_FILE_ACCESSES
 	ut_a(block->file_page_was_freed == FALSE);
@@ -2027,7 +2060,6 @@ buf_pool_invalidate(void)
 	mutex_exit(&(buf_pool->mutex));
 }
 
-#ifdef UNIV_DEBUG
 /*************************************************************************
 Validates the buffer buf_pool data structure. */
 
@@ -2227,7 +2259,6 @@ buf_print(void)
 
 	ut_a(buf_validate());
 }	
-#endif /* UNIV_DEBUG */
 
 /*************************************************************************
 Returns the number of latched pages in the buffer pool. */
@@ -2483,3 +2514,64 @@ buf_get_free_list_len(void)
 
 	return(len);
 }
+
+/*************************************************************************
+Print the block fields. */
+
+void
+buf_block_print(
+/*============*/
+	const buf_block_t*	block)		/* in: block to print */
+{
+	fprintf(stderr,
+		"BLOCK fields\nmagic_n: 0x%lx\n"
+		"state: %lu frame: 0x%lx space:offset: 0x%lx:0x%lx\n"
+		"hash value: 0x%lx  check_index_page_at_flush: %ld\n"
+		"newest_modification %lu:%lu oldest_modification %lu:%lu\n"
+		"flush_type: %lu in_free_list: %ld in_LRU_list: %ld\n"
+		"LRU_position: %ld freed_page_clock: %lu old: %ld\n"
+		"accessed: %ld buf_fix_count: %lu io_fix: %ld "
+		"modify_clock: %lu:%lu\n"
+		"n_hash_helps: %lu n_fields: %ld n_bytes: %lu side: %lu\n"
+		"is_hashed: %lu n_pointers: %lu curr_n_fields: %lu\n"
+		"curr_n_bytes: %lu curr_side: %lu index name: %s\n"
+		"file_page_was_freed: %lu\n",
+		block->magic_n, block->state, (ulint) block->frame,
+		block->space, block->offset, block->lock_hash_val,
+		block->check_index_page_at_flush,
+		ut_dulint_get_high(block->newest_modification),
+		ut_dulint_get_low(block->newest_modification),
+		ut_dulint_get_high(block->oldest_modification),
+		ut_dulint_get_low(block->oldest_modification),
+		block->flush_type, block->in_free_list, block->in_LRU_list,
+		block->LRU_position, block->freed_page_clock,
+		block->old, block->accessed, block->buf_fix_count,
+		block->io_fix,
+		ut_dulint_get_high(block->modify_clock),
+		ut_dulint_get_low(block->modify_clock),
+		block->n_hash_helps, block->n_fields, block->n_bytes,
+		block->side, block->is_hashed, block->n_pointers,
+		block->curr_n_fields, block->curr_n_bytes,
+		block->curr_side, block->index->name,
+		block->file_page_was_freed);
+}
+
+/************************************************************************
+Print some diagnostics related to the buffer pool.*/
+
+void
+buf_print_diagnostic(
+/*=================*/
+	mtr_t*			mtr,	/* in: mtr to print */
+	const buf_block_t*	block)	/* in: block to print */
+{
+	fprintf(stderr, "=== MTR ===\n");
+	mtr_print(mtr);
+	buf_LRU_print();
+	buf_print();
+	buf_LRU_validate();
+	buf_print_io(stderr);
+	fprintf(stderr, "=== BLOCK ===\n");
+	buf_block_print(block);
+}
+
diff --git a/innobase/buf/buf0lru.c b/innobase/buf/buf0lru.c
index dfee2add045..0f632f0752a 100644
--- a/innobase/buf/buf0lru.c
+++ b/innobase/buf/buf0lru.c
@@ -945,7 +945,6 @@ buf_LRU_block_free_hashed_page(
 	buf_LRU_block_free_non_file_page(block);
 }
 
-#ifdef UNIV_DEBUG
 /**************************************************************************
 Validates the LRU list. */
 
@@ -1076,4 +1075,3 @@ buf_LRU_print(void)
 
 	mutex_exit(&(buf_pool->mutex));
 }
-#endif /* UNIV_DEBUG */
diff --git a/innobase/ibuf/ibuf0ibuf.c b/innobase/ibuf/ibuf0ibuf.c
index eb10a88d1d1..031add1c9ba 100644
--- a/innobase/ibuf/ibuf0ibuf.c
+++ b/innobase/ibuf/ibuf0ibuf.c
@@ -1436,6 +1436,9 @@ ibuf_entry_build(
 		*buf2++ = 0; /* write the compact format indicator */
 	}
 	for (i = 0; i < n_fields; i++) {
+		ulint			fixed_len;
+		const dict_field_t*	ifield;
+
 		/* We add 4 below because we have the 4 extra fields at the
 		start of an ibuf record */
 
@@ -1443,11 +1446,30 @@ ibuf_entry_build(
 		entry_field = dtuple_get_nth_field(entry, i);
 		dfield_copy(field, entry_field);
 
+		ifield = dict_index_get_nth_field(index, i);
+		/* Prefix index columns of fixed-length columns are of
+		fixed length.  However, in the function call below,
+		dfield_get_type(entry_field) contains the fixed length
+		of the column in the clustered index.  Replace it with
+		the fixed length of the secondary index column. */
+		fixed_len = ifield->fixed_len;
+
+#ifdef UNIV_DEBUG
+		if (fixed_len) {
+			/* dict_index_add_col() should guarantee these */
+			ut_ad(fixed_len <= (ulint) entry_field->type.len);
+			if (ifield->prefix_len) {
+				ut_ad(ifield->prefix_len == fixed_len);
+			} else {
+				ut_ad(fixed_len
+				      == (ulint) entry_field->type.len);
+			}
+		}
+#endif /* UNIV_DEBUG */
+
 		dtype_new_store_for_order_and_null_size(
-				buf2 + i * DATA_NEW_ORDER_NULL_TYPE_BUF_SIZE,
-				dfield_get_type(entry_field),
-				dict_index_get_nth_field(index, i)
-				->prefix_len);
+			buf2 + i * DATA_NEW_ORDER_NULL_TYPE_BUF_SIZE,
+			dfield_get_type(entry_field), fixed_len);
 	}
 
 	/* Store the type info in buf2 to field 3 of tuple */
diff --git a/innobase/include/buf0buf.ic b/innobase/include/buf0buf.ic
index ddc91b8d66c..844d3fbd6d1 100644
--- a/innobase/include/buf0buf.ic
+++ b/innobase/include/buf0buf.ic
@@ -11,6 +11,15 @@ Created 11/5/1995 Heikki Tuuri
 #include "buf0rea.h"
 #include "mtr0mtr.h"
 
+/*************************************************************************
+Print the mtr and block fields. */
+
+void
+buf_print_diagnostic(
+/*=================*/
+	mtr_t*			mtr,		/* in: mtr to print */
+	const buf_block_t*	block);		/* in: block to print */
+
 #ifdef UNIV_DEBUG
 extern ulint		buf_dbg_counter; /* This is used to insert validation
 					operations in execution in the
@@ -618,12 +627,9 @@ buf_page_release(
 					RW_NO_LATCH */
 	mtr_t*		mtr)		/* in: mtr */
 {
-	ulint	buf_fix_count;
-	
 	ut_ad(block);
 
 	ut_a(block->state == BUF_BLOCK_FILE_PAGE);
-	ut_a(block->buf_fix_count > 0);
 
 	if (rw_latch == RW_X_LATCH && mtr->modifications) {
 		mutex_enter(&buf_pool->mutex);
@@ -633,11 +639,16 @@ buf_page_release(
 
 	mutex_enter(&block->mutex);
 
+	/* This debug code has been added to narrow down Bug# 26081 */
+	if (UNIV_UNLIKELY(block->buf_fix_count == 0)) {
+		buf_print_diagnostic(mtr, block);
+		ut_error;
+	}
+
 #ifdef UNIV_SYNC_DEBUG
 	rw_lock_s_unlock(&(block->debug_latch));
 #endif
-	buf_fix_count = block->buf_fix_count;
-	block->buf_fix_count = buf_fix_count - 1;
+	--block->buf_fix_count;
 
 	mutex_exit(&block->mutex);
 
diff --git a/innobase/include/buf0lru.h b/innobase/include/buf0lru.h
index 790bf65c698..e0f511cf7b8 100644
--- a/innobase/include/buf0lru.h
+++ b/innobase/include/buf0lru.h
@@ -122,7 +122,6 @@ void
 buf_LRU_make_block_old(
 /*===================*/
 	buf_block_t*	block);	/* in: control block */
-#ifdef UNIV_DEBUG
 /**************************************************************************
 Validates the LRU list. */
 
@@ -135,7 +134,6 @@ Prints the LRU list. */
 void
 buf_LRU_print(void);
 /*===============*/
-#endif /* UNIV_DEBUG */
 
 #ifndef UNIV_NONINL
 #include "buf0lru.ic"
diff --git a/innobase/include/dict0dict.h b/innobase/include/dict0dict.h
index 642037494b5..7fd9be1ebd6 100644
--- a/innobase/include/dict0dict.h
+++ b/innobase/include/dict0dict.h
@@ -84,6 +84,17 @@ dtype_t*
 dict_col_get_type(
 /*==============*/
 	dict_col_t*	col);
+#ifdef UNIV_DEBUG
+/*************************************************************************
+Assert that a column and a data type match. */
+UNIV_INLINE
+ibool
+dict_col_type_assert_equal(
+/*=======================*/
+					/* out: TRUE */
+	const dict_col_t*	col,	/* in: column */
+	const dtype_t*		type);	/* in: data type */
+#endif /* UNIV_DEBUG */
 /*************************************************************************
 Gets the column number. */
 UNIV_INLINE
diff --git a/innobase/include/dict0dict.ic b/innobase/include/dict0dict.ic
index 861da5d057a..cbd7285e20f 100644
--- a/innobase/include/dict0dict.ic
+++ b/innobase/include/dict0dict.ic
@@ -23,6 +23,30 @@ dict_col_get_type(
 	return(&col->type);
 }
 
+#ifdef UNIV_DEBUG
+/*************************************************************************
+Assert that a column and a data type match. */
+UNIV_INLINE
+ibool
+dict_col_type_assert_equal(
+/*=======================*/
+					/* out: TRUE */
+	const dict_col_t*	col,	/* in: column */
+	const dtype_t*		type)	/* in: data type */
+{
+	ut_ad(col);
+	ut_ad(type);
+
+	ut_ad(col->type.mtype == type->mtype);
+	ut_ad(col->type.prtype == type->prtype);
+	ut_ad(col->type.len == type->len);
+	ut_ad(col->type.mbminlen == type->mbminlen);
+	ut_ad(col->type.mbmaxlen == type->mbmaxlen);
+
+	return(TRUE);
+}
+#endif /* UNIV_DEBUG */
+
 /*************************************************************************
 Gets the column number. */
 UNIV_INLINE
diff --git a/innobase/include/dict0mem.h b/innobase/include/dict0mem.h
index 3c10e82342b..7d8dc404290 100644
--- a/innobase/include/dict0mem.h
+++ b/innobase/include/dict0mem.h
@@ -159,10 +159,13 @@ struct dict_col_struct{
 				in some of the functions below */
 };
 
-/* DICT_MAX_INDEX_COL_LEN is measured in bytes and is the max index column
-length + 1. Starting from 4.1.6, we set it to < 3 * 256, so that one can
-create a column prefix index on 255 characters of a TEXT field also in the
-UTF-8 charset. In that charset, a character may take at most 3 bytes. */
+/* DICT_MAX_INDEX_COL_LEN is measured in bytes and is the maximum
+indexed column length (or indexed prefix length). It is set to 3*256,
+so that one can create a column prefix index on 256 characters of a
+TEXT or VARCHAR column also in the UTF-8 charset. In that charset,
+a character may take at most 3 bytes.
+This constant MUST NOT BE CHANGED, or the compatibility of InnoDB data
+files would be at risk! */
 
 #define DICT_MAX_INDEX_COL_LEN		768
 
diff --git a/innobase/include/os0file.h b/innobase/include/os0file.h
index d5bc5a2b115..49460dbaf19 100644
--- a/innobase/include/os0file.h
+++ b/innobase/include/os0file.h
@@ -94,7 +94,8 @@ log. */
 #define	OS_FILE_PATH_ERROR		74
 #define	OS_FILE_AIO_RESOURCES_RESERVED	75	/* wait for OS aio resources
 						to become available again */
-#define	OS_FILE_ERROR_NOT_SPECIFIED	76
+#define	OS_FILE_SHARING_VIOLATION	76
+#define	OS_FILE_ERROR_NOT_SPECIFIED	77
 
 /* Types for aio operations */
 #define OS_FILE_READ	10
diff --git a/innobase/include/trx0trx.h b/innobase/include/trx0trx.h
index 0dc82893ad1..cf9d1788ad8 100644
--- a/innobase/include/trx0trx.h
+++ b/innobase/include/trx0trx.h
@@ -604,7 +604,7 @@ struct trx_struct{
 					NULL */
 	ibool		was_chosen_as_deadlock_victim;
 					/* when the transaction decides to wait
-					for a lock, this it sets this to FALSE;
+					for a lock, it sets this to FALSE;
 					if another transaction chooses this
 					transaction as a victim in deadlock
 					resolution, it sets this to TRUE */
diff --git a/innobase/lock/lock0lock.c b/innobase/lock/lock0lock.c
index 77dfca5fdf4..e0f3f58f737 100644
--- a/innobase/lock/lock0lock.c
+++ b/innobase/lock/lock0lock.c
@@ -4438,14 +4438,11 @@ lock_table_queue_validate(
 	dict_table_t*	table)	/* in: table */
 {
 	lock_t*	lock;
-	ibool	is_waiting;
 
 #ifdef UNIV_SYNC_DEBUG
 	ut_ad(mutex_own(&kernel_mutex));
 #endif /* UNIV_SYNC_DEBUG */
 
-	is_waiting = FALSE;
-
 	lock = UT_LIST_GET_FIRST(table->locks);
 
 	while (lock) {
@@ -4455,12 +4452,9 @@ lock_table_queue_validate(
 	
 		if (!lock_get_wait(lock)) {
 
-			ut_a(!is_waiting);
-		
 			ut_a(!lock_table_other_has_incompatible(lock->trx, 0,
 						table, lock_get_mode(lock)));
 		} else {
-			is_waiting = TRUE;
 
 			ut_a(lock_table_has_to_wait_in_queue(lock));
 		}
diff --git a/innobase/log/log0log.c b/innobase/log/log0log.c
index 2d3bff522e2..26c0ce60c69 100644
--- a/innobase/log/log0log.c
+++ b/innobase/log/log0log.c
@@ -3061,9 +3061,22 @@ loop:
 
 	mutex_enter(&kernel_mutex);
 
-       /* Check that there are no longer transactions. We need this wait even
-        for the 'very fast' shutdown, because the InnoDB layer may have
-        committed or prepared transactions and we don't want to lose them. */
+	/* We need the monitor threads to stop before we proceed with a
+	normal shutdown. In case of very fast shutdown, however, we can
+	proceed without waiting for monitor threads. */
+
+	if (srv_fast_shutdown < 2
+	   && (srv_error_monitor_active
+	      || srv_lock_timeout_and_monitor_active)) {
+		
+		mutex_exit(&kernel_mutex);
+		
+		goto loop;
+	}
+
+	/* Check that there are no longer transactions. We need this wait even
+	for the 'very fast' shutdown, because the InnoDB layer may have
+	committed or prepared transactions and we don't want to lose them. */
 
 	if (trx_n_mysql_transactions > 0
 			|| UT_LIST_GET_LEN(trx_sys->trx_list) > 0) {
@@ -3184,22 +3197,8 @@ loop:
 		goto loop;
 	}
 
-	/* The lock timeout thread should now have exited */
-
-	if (srv_lock_timeout_and_monitor_active) {
-
-		goto loop;
-	}
-
-	/* We now let also the InnoDB error monitor thread to exit */
-	
 	srv_shutdown_state = SRV_SHUTDOWN_LAST_PHASE;
 
-	if (srv_error_monitor_active) {
-
-		goto loop;
-	}
-
 	/* Make some checks that the server really is quiet */
 	ut_a(srv_n_threads_active[SRV_MASTER] == 0);
        ut_a(buf_all_freed());
diff --git a/innobase/mtr/mtr0mtr.c b/innobase/mtr/mtr0mtr.c
index da045be1f62..143db0ce0dd 100644
--- a/innobase/mtr/mtr0mtr.c
+++ b/innobase/mtr/mtr0mtr.c
@@ -326,7 +326,16 @@ mtr_print(
 	mtr_t*	mtr)	/* in: mtr */
 {
 	fprintf(stderr,
-	"Mini-transaction handle: memo size %lu bytes log size %lu bytes\n",
+		"Mini-transaction handle: memo size %lu bytes log "
+		"size %lu bytes\n"
+		"state: %lu modifications: %lu n_lock_recs %lu\n"
+		"log_mode: %lu start_lsn: %lu:%lu end_lsn: %lu:%lu\n",
 		(ulong) dyn_array_get_data_size(&(mtr->memo)),
-		(ulong) dyn_array_get_data_size(&(mtr->log)));
+		(ulong) dyn_array_get_data_size(&(mtr->log)),
+		mtr->state, (ulint) mtr->modifications,
+		mtr->n_log_recs, mtr->log_mode,
+		ut_dulint_get_high(mtr->start_lsn),
+		ut_dulint_get_low(mtr->start_lsn),
+		ut_dulint_get_high(mtr->end_lsn),
+		ut_dulint_get_low(mtr->end_lsn));
 }
diff --git a/innobase/os/os0file.c b/innobase/os/os0file.c
index 7133b28ecb2..3030b20a685 100644
--- a/innobase/os/os0file.c
+++ b/innobase/os/os0file.c
@@ -243,7 +243,13 @@ os_file_get_last_error(
 			fprintf(stderr,
   "InnoDB: The error means mysqld does not have the access rights to\n"
   "InnoDB: the directory. It may also be you have created a subdirectory\n"
-  "InnoDB: of the same name as a data file.\n"); 
+  "InnoDB: of the same name as a data file.\n");
+		} else if (err == ERROR_SHARING_VIOLATION
+			   || err == ERROR_LOCK_VIOLATION) {
+			fprintf(stderr,
+  "InnoDB: The error means that another program is using InnoDB's files.\n"
+  "InnoDB: This might be a backup or antivirus software or another instance\n"
+  "InnoDB: of MySQL. Please close it to get rid of this error.\n");
 		} else {
 			fprintf(stderr,
   "InnoDB: Some operating system error numbers are described at\n"
@@ -260,6 +266,9 @@ os_file_get_last_error(
 		return(OS_FILE_DISK_FULL);
 	} else if (err == ERROR_FILE_EXISTS) {
 		return(OS_FILE_ALREADY_EXISTS);
+	} else if (err == ERROR_SHARING_VIOLATION
+		   || err == ERROR_LOCK_VIOLATION) {
+		return(OS_FILE_SHARING_VIOLATION);
 	} else {
 		return(100 + err);
 	}
@@ -369,6 +378,12 @@ os_file_handle_error_cond_exit(
 			|| err == OS_FILE_PATH_ERROR) {
 
 		return(FALSE);
+
+	} else if (err == OS_FILE_SHARING_VIOLATION) {
+
+		os_thread_sleep(10000000);  /* 10 sec */
+		return(TRUE);
+
 	} else {
 	        if (name) {
 	                fprintf(stderr, "InnoDB: File name %s\n", name);
diff --git a/innobase/rem/rem0rec.c b/innobase/rem/rem0rec.c
index 9480c978755..65da8352dc7 100644
--- a/innobase/rem/rem0rec.c
+++ b/innobase/rem/rem0rec.c
@@ -462,7 +462,7 @@ rec_get_converted_size_new(
 	case REC_STATUS_INFIMUM:
 	case REC_STATUS_SUPREMUM:
 		/* infimum or supremum record, 8 bytes */
-		return(size + 8); /* no extra data needed */
+		return(8); /* no extra data needed */
 	default:
 		ut_a(0);
 		return(ULINT_UNDEFINED);
@@ -473,23 +473,34 @@ rec_get_converted_size_new(
 		ulint	len	= dtuple_get_nth_field(dtuple, i)->len;
 		field = dict_index_get_nth_field(index, i);
 		type = dict_col_get_type(dict_field_get_col(field));
-		ut_ad(len != UNIV_SQL_NULL ||
-			!(dtype_get_prtype(type) & DATA_NOT_NULL));
+
+		ut_ad(dict_col_type_assert_equal(
+			      dict_field_get_col(field),
+			      dfield_get_type(dtuple_get_nth_field(
+						      dtuple, i))));
 
 		if (len == UNIV_SQL_NULL) {
 			/* No length is stored for NULL fields. */
+			ut_ad(!(dtype_get_prtype(type) & DATA_NOT_NULL));
 			continue;
 		}
 
 		ut_ad(len <= dtype_get_len(type)
 			|| dtype_get_mtype(type) == DATA_BLOB);
-		ut_ad(!field->fixed_len || len == field->fixed_len);
 
 		if (field->fixed_len) {
+			ut_ad(len == field->fixed_len);
+			/* dict_index_add_col() should guarantee this */
+			ut_ad(!field->prefix_len
+			      || field->fixed_len == field->prefix_len);
 		} else if (len < 128 || (dtype_get_len(type) < 256
 				&& dtype_get_mtype(type) != DATA_BLOB)) {
 			size++;
 		} else {
+			/* For variable-length columns, we look up the
+			maximum length from the column itself.  If this
+			is a prefix index column shorter than 256 bytes,
+			this will waste one byte. */
 			size += 2;
 		}
 		size += len;
@@ -885,6 +896,11 @@ rec_convert_dtuple_to_rec_new(
 		len = dfield_get_len(field);
 		fixed_len = dict_index_get_nth_field(index, i)->fixed_len;
 
+		ut_ad(dict_col_type_assert_equal(
+			      dict_field_get_col(dict_index_get_nth_field(
+							 index, i)),
+			      dfield_get_type(field)));
+
 		if (!(dtype_get_prtype(type) & DATA_NOT_NULL)) {
 			if (len == UNIV_SQL_NULL)
 				continue;
diff --git a/innobase/row/row0sel.c b/innobase/row/row0sel.c
index f8a65e6ff82..c956492900d 100644
--- a/innobase/row/row0sel.c
+++ b/innobase/row/row0sel.c
@@ -3451,6 +3451,31 @@ shortcut_fails_too_big_rec:
 					pcur, 0, &mtr);
 
 		pcur->trx_if_known = trx;
+
+		rec = btr_pcur_get_rec(pcur);
+
+		if (!moves_up
+		    && !page_rec_is_supremum(rec)
+		    && set_also_gap_locks
+		    && !srv_locks_unsafe_for_binlog
+		    && prebuilt->select_lock_type != LOCK_NONE) {
+
+			/* Try to place a gap lock on the next index record
+			to prevent phantoms in ORDER BY ... DESC queries */
+
+			offsets = rec_get_offsets(page_rec_get_next(rec),
+						  index, offsets,
+						  ULINT_UNDEFINED, &heap);
+			err = sel_set_rec_lock(page_rec_get_next(rec),
+					       index, offsets,
+					       prebuilt->select_lock_type,
+					       LOCK_GAP, thr);
+
+			if (err != DB_SUCCESS) {
+
+				goto lock_wait_or_error;
+			}
+		}
 	} else {
 		if (mode == PAGE_CUR_G) {
 			btr_pcur_open_at_index_side(TRUE, index,
diff --git a/innobase/srv/srv0srv.c b/innobase/srv/srv0srv.c
index 96c0f05111b..e4ce29f445c 100644
--- a/innobase/srv/srv0srv.c
+++ b/innobase/srv/srv0srv.c
@@ -1037,7 +1037,7 @@ retry:
 	if (!has_slept && !trx->has_search_latch
 	    && NULL == UT_LIST_GET_FIRST(trx->trx_locks)) {
 
-	        has_slept = TRUE; /* We let is sleep only once to avoid
+		has_slept = TRUE; /* We let it sleep only once to avoid
 				  starvation */
 
 		srv_conc_n_waiting_threads++;
@@ -1858,6 +1858,7 @@ srv_lock_timeout_and_monitor_thread(
 	double		time_elapsed;
 	time_t          current_time;
 	time_t		last_table_monitor_time;
+	time_t		last_tablespace_monitor_time;
 	time_t		last_monitor_time;
 	ibool		some_waits;
 	double		wait_time;
@@ -1870,6 +1871,7 @@ srv_lock_timeout_and_monitor_thread(
 	UT_NOT_USED(arg);
 	srv_last_monitor_time = time(NULL);
 	last_table_monitor_time = time(NULL);
+	last_tablespace_monitor_time = time(NULL);
 	last_monitor_time = time(NULL);
 loop:
 	srv_lock_timeout_and_monitor_active = TRUE;
@@ -1905,9 +1907,9 @@ loop:
 	    }
 
 	    if (srv_print_innodb_tablespace_monitor
-		&& difftime(current_time, last_table_monitor_time) > 60) {
+		&& difftime(current_time, last_tablespace_monitor_time) > 60) {
 
-		last_table_monitor_time = time(NULL);	
+		last_tablespace_monitor_time = time(NULL);	
 
 		fputs("================================================\n",
 			stderr);
@@ -2103,7 +2105,7 @@ loop:
 
 	os_thread_sleep(2000000);
 
-	if (srv_shutdown_state < SRV_SHUTDOWN_LAST_PHASE) {
+	if (srv_shutdown_state < SRV_SHUTDOWN_CLEANUP) {
 
 		goto loop;
 	}