InnoDB: Performance optimizations based on OProfile analysis

innobase/btr/btr0btr.c:
  Eliminate some buf_frame_align() calls.
  Make use of the page_rec_is_infimum(), page_rec_is_supremum()
  and page_rec_is_user_rec() functions.
  Replace some index->table->comp with page_is_comp().
  Eliminate some variables to reduce register spilling on x86.
  Note that page_is_comp() may return nonzero instead of TRUE.
  Note that rec_offs_comp() may return nonzero instead of TRUE.
innobase/btr/btr0cur.c:
  Eliminate some buf_frame_align() calls.
  Replace some index->table->comp with
  page_is_comp() or rec_offs_comp().
  Eliminate some variables to reduce register spilling on x86.
  Note that page_is_comp() may return nonzero instead of TRUE.
  Note that rec_offs_comp() may return nonzero instead of TRUE.
  Remove an extra mem_heap_create() call from btr_cur_update_in_place().
  Add "page" parameter to lock_rec_store_on_page_infimum().
  Add some UNIV_LIKELY() and UNIV_UNLIKELY() hints.
  
  btr_estimate_number_of_different_key_vals(): Rename the
  offsets_* variables to be more descriptive and eliminate one
  rec_get_offsets() and one page_rec_get_next() call in the loop.
innobase/btr/btr0pcur.c:
  Eliminate some buf_frame_align() calls.
  Make use of the page_rec_is_infimum(), page_rec_is_supremum()
  and page_rec_is_user_rec() functions.
  Replace some index->table->comp with page_is_comp().
  Eliminate some variables to reduce register spilling on x86.
  Note that page_is_comp() may return nonzero instead of TRUE.
  Make some ut_a() assertions ut_ad() ones to improve performance.
  Add some UNIV_LIKELY() and UNIV_UNLIKELY() hints.
innobase/btr/btr0sea.c:
  Make use of page_rec_is_infimum() and page_rec_is_supremum().
  Eliminate some buf_frame_align() calls.
  Add some UNIV_UNLIKELY and UNIV_LIKELY hints.
  Turn some assertions into debug assertions.
innobase/dict/dict0crea.c:
  Replace FALSE/TRUE ibool comp with zero/nonzero ulint comp
innobase/ibuf/ibuf0ibuf.c:
  Make use of page_rec_is_infimum() and page_rec_is_supremum().
  Add some UNIV_UNLIKELY and UNIV_LIKELY hints.
  ibuf_get_merge_page_nos(): Rename parameter "first_rec" to "rec"
  and eliminate local variable "rec".
innobase/include/btr0btr.h:
  Replace FALSE/TRUE ibool comp with zero/nonzero ulint comp
innobase/include/buf0buf.h:
  Rename buf_frame_get_modify_clock()
  to buf_block_get_modify_clock().
innobase/include/buf0buf.ic:
  Rename buf_frame_get_modify_clock()
  to buf_block_get_modify_clock() and
  remove the buf_block_align() call.
innobase/include/lock0lock.h:
  lock_rec_store_on_page_infimum(): Add parameter "page"
innobase/include/mach0data.h:
  Add mach_encode_2() and mach_decode_2().
innobase/include/mach0data.ic:
  Add mach_encode_2() and mach_decode_2().
innobase/include/page0cur.h:
  Add const qualifier to page_cur_is_before_first()
  and page_cur_is_after_last().
innobase/include/page0cur.ic:
  Make use of page_rec_is_infimum() and page_rec_is_supremum().
innobase/include/page0page.h:
  Remove page_rec_is_first_user_rec() and page_rec_is_last_user_rec().
  Add page_rec_is_infimum() and page_rec_is_supremum().
  Replace FALSE/TRUE ibool comp with zero/nonzero ulint comp.
innobase/include/page0page.ic:
  Remove page_rec_is_first_user_rec() and page_rec_is_last_user_rec().
  Add page_rec_is_infimum() and page_rec_is_supremum().
  Replace FALSE/TRUE ibool comp with zero/nonzero ulint comp.
  Add UNIV_UNLIKELY, UNIV_LIKELY and UNIV_EXPECT hints.
  Reduce the number of buf_frame_align() calls.
innobase/include/rem0rec.ic:
  rec_offs_comp(): Return zero or nonzero instead of FALSE or TRUE.
innobase/include/row0mysql.h:
  Replace FALSE/TRUE ibool comp with zero/nonzero ulint comp.
innobase/lock/lock0lock.c:
  Replace FALSE/TRUE ibool comp with zero/nonzero ulint comp.
  Remove parameter "comp" from lock_rec_get_next(),
  lock_rec_has_expl() and lock_rec_other_has_expl_req().
  Add parameter "page" to lock_rec_store_on_page_infimum().
  Add UNIV_UNLIKELY hints.
  Reduce the number of buf_frame_align() calls.
  Make use of page_rec_is_infimum(), page_rec_is_supremum() and
  page_rec_is_user_rec().
  Move the "comp" flag outside some loops.
innobase/mtr/mtr0log.c:
  Replace index->table->comp with page_rec_is_comp().
innobase/page/page0cur.c:
  Replace index->table->comp with page_is_comp() or page_rec_is_comp().
  Eliminate some buf_frame_align() calls.
  Add some debug assertions.
innobase/page/page0page.c:
  Optimize page_dir_find_owner_slot(). Compare the record offset
  16 bits at a time, because that seems to be the only way to avoid
  register spilling on x86.
  Replace FALSE/TRUE ibool comp with zero/nonzero ulint comp.
  Remove parameter "page" of page_delete_rec_list_write_log().
  Make use of page_rec_is_infimum().
innobase/rem/rem0cmp.c:
  Replace FALSE/TRUE ibool comp with zero/nonzero ulint comp.
innobase/row/row0ins.c:
  Make use of page_rec_is_infimum() and page_rec_is_supremum().
  Reduce the amount of buf_frame_align() calls.
  row_ins_index_entry_low(): Disable assertion about column count
  unless #ifdef UNIV_DEBUG.
innobase/row/row0mysql.c:
  Replace FALSE/TRUE ibool comp with zero/nonzero ulint comp.
innobase/row/row0row.c:
  Eliminate some buf_frame_align() calls.
  Make use of page_rec_is_infimum().
innobase/row/row0sel.c:
  Make use of page_rec_is_supremum() and page_rec_is_infimum().
  Turn some assertions into debug assertions.
  Add UNIV_LIKELY and UNIV_UNLIKELY hints.
  
  row_search_for_mysql(): Eliminate local variables "moved",
  "cons_read_requires_clust_rec", "was_lock_wait", "shortcut",
  "success" and "comp". Replace some of them with goto's.
  Disable variable "cnt" unless #ifdef UNIV_SEARCH_DEBUG.
innobase/row/row0vers.c:
  Replace FALSE/TRUE ibool comp with zero/nonzero ulint comp.
  Replace index->table->comp with page_rec_is_comp().
  Eliminate some buf_frame_align() calls.

4 files changed, 246 insertions, 258 deletions

diff --git a/innobase/btr/btr0btr.c b/innobase/btr/btr0btr.c
index 961b9eb696d..75c99e04f7b 100644
--- a/innobase/btr/btr0btr.c
+++ b/innobase/btr/btr0btr.c
@@ -138,13 +138,13 @@ btr_root_get(
 	ulint	space;
 	ulint	root_page_no;
 	page_t*	root;
-	ibool	comp = UT_LIST_GET_FIRST(tree->tree_indexes)->table->comp;
 	
 	space = dict_tree_get_space(tree);
 	root_page_no = dict_tree_get_page(tree);
 
 	root = btr_page_get(space, root_page_no, RW_X_LATCH, mtr);
-	ut_a(page_is_comp(root) == comp);
+	ut_a(!!page_is_comp(root) ==
+			UT_LIST_GET_FIRST(tree->tree_indexes)->table->comp);
 	
 	return(root);
 }
@@ -164,21 +164,19 @@ btr_get_prev_user_rec(
 	page_t*	page;
 	page_t*	prev_page;
 	ulint	prev_page_no;
-	rec_t*	prev_rec;
 	ulint	space;
 
-	page = buf_frame_align(rec);
-	
-	if (page_get_infimum_rec(page) != rec) {
+	if (!page_rec_is_infimum(rec)) {
 
-		prev_rec = page_rec_get_prev(rec);
+		rec_t*	prev_rec = page_rec_get_prev(rec);
 
-		if (page_get_infimum_rec(page) != prev_rec) {
+		if (!page_rec_is_infimum(prev_rec)) {
 
 			return(prev_rec);
 		}
 	}
 	
+	page = buf_frame_align(rec);
 	prev_page_no = btr_page_get_prev(page, mtr);
 	space = buf_frame_get_space_id(page);
 	
@@ -193,9 +191,7 @@ btr_get_prev_user_rec(
 		      				MTR_MEMO_PAGE_X_FIX)));
 		ut_a(page_is_comp(prev_page) == page_is_comp(page));
 
-		prev_rec = page_rec_get_prev(page_get_supremum_rec(prev_page));
-
-		return(prev_rec);
+		return(page_rec_get_prev(page_get_supremum_rec(prev_page)));
 	}
 
 	return(NULL);
@@ -216,21 +212,19 @@ btr_get_next_user_rec(
 	page_t*	page;
 	page_t*	next_page;
 	ulint	next_page_no;
-	rec_t*	next_rec;
 	ulint	space;
 
-	page = buf_frame_align(rec);
-	
-	if (page_get_supremum_rec(page) != rec) {
+	if (!page_rec_is_supremum(rec)) {
 
-		next_rec = page_rec_get_next(rec);
+		rec_t*	next_rec = page_rec_get_next(rec);
 
-		if (page_get_supremum_rec(page) != next_rec) {
+		if (!page_rec_is_supremum(next_rec)) {
 
 			return(next_rec);
 		}
 	}
 	
+	page = buf_frame_align(rec);
 	next_page_no = btr_page_get_next(page, mtr);
 	space = buf_frame_get_space_id(page);
 	
@@ -245,9 +239,7 @@ btr_get_next_user_rec(
 		      				MTR_MEMO_PAGE_X_FIX)));
 
 		ut_a(page_is_comp(next_page) == page_is_comp(page));
-		next_rec = page_rec_get_next(page_get_infimum_rec(next_page));
-
-		return(next_rec);
+		return(page_rec_get_next(page_get_infimum_rec(next_page)));
 	}
 
 	return(NULL);
@@ -574,8 +566,7 @@ btr_page_get_father_for_rec(
 
 	ut_ad(mtr_memo_contains(mtr, dict_tree_get_lock(tree),
 							MTR_MEMO_X_LOCK));
-	ut_a(user_rec != page_get_supremum_rec(page));
-	ut_a(user_rec != page_get_infimum_rec(page));
+	ut_a(page_rec_is_user_rec(user_rec));
 	
 	ut_ad(dict_tree_get_page(tree) != buf_frame_get_page_no(page));
 
@@ -599,6 +590,7 @@ btr_page_get_father_for_rec(
 
 	if (btr_node_ptr_get_child_page_no(node_ptr, offsets) !=
                                                 buf_frame_get_page_no(page)) {
+		rec_t*	print_rec;
 		fputs("InnoDB: Dump of the child page:\n", stderr);
 		buf_page_print(buf_frame_align(page));
 		fputs("InnoDB: Dump of the parent page:\n", stderr);
@@ -613,11 +605,10 @@ btr_page_get_father_for_rec(
 			(ulong)
 			btr_node_ptr_get_child_page_no(node_ptr, offsets),
 			(ulong) buf_frame_get_page_no(page));
-		offsets = rec_get_offsets(page_rec_get_next(
-				page_get_infimum_rec(page)), index,
+		print_rec = page_rec_get_next(page_get_infimum_rec(page));
+		offsets = rec_get_offsets(print_rec, index,
 				offsets, ULINT_UNDEFINED, &heap);
-		page_rec_print(page_rec_get_next(page_get_infimum_rec(page)),
-					offsets);
+		page_rec_print(print_rec, offsets);
 		offsets = rec_get_offsets(node_ptr, index, offsets,
 					ULINT_UNDEFINED, &heap);
 		page_rec_print(node_ptr, offsets);
@@ -664,7 +655,7 @@ btr_create(
 	ulint	type,	/* in: type of the index */
 	ulint	space,	/* in: space where created */
 	dulint	index_id,/* in: index id */
-	ibool	comp,	/* in: TRUE=compact page format */
+	ulint	comp,	/* in: nonzero=compact page format */
 	mtr_t*	mtr)	/* in: mini-transaction handle */
 {
 	ulint		page_no;
@@ -855,11 +846,12 @@ btr_page_reorganize_low(
 
 	ut_ad(mtr_memo_contains(mtr, buf_block_align(page),
 			      				MTR_MEMO_PAGE_X_FIX));
+	ut_ad(!!page_is_comp(page) == index->table->comp);
 	data_size1 = page_get_data_size(page);
 	max_ins_size1 = page_get_max_insert_size_after_reorganize(page, 1);
 
 	/* Write the log record */
-	mlog_open_and_write_index(mtr, page, index, index->table->comp
+	mlog_open_and_write_index(mtr, page, index, page_is_comp(page)
 			? MLOG_COMP_PAGE_REORGANIZE
 			: MLOG_PAGE_REORGANIZE, 0);
 
@@ -878,7 +870,7 @@ btr_page_reorganize_low(
 	/* Recreate the page: note that global data on page (possible
 	segment headers, next page-field, etc.) is preserved intact */
 
-	page_create(page, mtr, index->table->comp);
+	page_create(page, mtr, page_is_comp(page));
 	buf_block_align(page)->check_index_page_at_flush = TRUE;
 	
 	/* Copy the records from the temporary space to the recreated page;
@@ -1071,7 +1063,7 @@ btr_root_raise_and_insert(
 	as there is no lower alphabetical limit to records in the leftmost
 	node of a level: */
 
-	btr_set_min_rec_mark(node_ptr_rec, cursor->index->table->comp, mtr);
+	btr_set_min_rec_mark(node_ptr_rec, page_is_comp(root), mtr);
 		
 	/* Free the memory heap */
 	mem_heap_free(heap);
@@ -1152,7 +1144,6 @@ btr_page_get_split_rec_to_right(
 {
 	page_t*	page;
 	rec_t*	insert_point;
-	rec_t*	supremum;
 
 	page = btr_cur_get_page(cursor);
 	insert_point = btr_cur_get_rec(cursor);
@@ -1161,13 +1152,22 @@ btr_page_get_split_rec_to_right(
 	the previous insert on the same page, we assume that there is a
 	pattern of sequential inserts here. */
 
-	if (page_header_get_ptr(page, PAGE_LAST_INSERT) == insert_point) {
+	if (UNIV_LIKELY(page_header_get_ptr(page, PAGE_LAST_INSERT)
+				== insert_point)) {
 
-	     	supremum = page_get_supremum_rec(page);
-	     	 			
-		if (page_rec_get_next(insert_point) != supremum
-		    && page_rec_get_next(page_rec_get_next(insert_point))
-			!= supremum) {
+		rec_t*	next_rec;
+
+		next_rec = page_rec_get_next(insert_point);
+
+		if (page_rec_is_supremum(next_rec)) {
+		split_at_new:
+			/* Split at the new record to insert */
+	     		*split_rec = NULL;
+		} else {
+			rec_t*	next_next_rec = page_rec_get_next(next_rec);
+			if (page_rec_is_supremum(next_next_rec)) {
+				goto split_at_new;
+			}
 
 			/* If there are >= 2 user records up from the insert
 			point, split all but 1 off. We want to keep one because
@@ -1176,12 +1176,8 @@ btr_page_get_split_rec_to_right(
 			search position just by looking at the records on this
 			page. */
 		
-			*split_rec = page_rec_get_next(
-					page_rec_get_next(insert_point));
-		} else {
-			/* Else split at the new record to insert */
-	     		*split_rec = NULL;
-	     	}
+			*split_rec = next_next_rec;
+		}
 
 		return(TRUE);
 	}
@@ -1221,7 +1217,7 @@ btr_page_get_sure_split_rec(
 	page = btr_cur_get_page(cursor);
 	
 	insert_size = rec_get_converted_size(cursor->index, tuple);
-	free_space  = page_get_free_space_of_empty(cursor->index->table->comp);
+	free_space  = page_get_free_space_of_empty(page_is_comp(page));
 
 	/* free_space is now the free space of a created new page */
 
@@ -1283,11 +1279,8 @@ btr_page_get_sure_split_rec(
 				} else {
 					next_rec = page_rec_get_next(rec);
 				}
-				if (next_rec != page_get_supremum_rec(page)) {
-					if (UNIV_LIKELY_NULL(heap)) {
-						mem_heap_free(heap);
-					}
-					return(next_rec);
+				if (!page_rec_is_supremum(next_rec))) {
+					rec = next_rec;
 				}
                     	}
 
@@ -1330,13 +1323,12 @@ btr_page_insert_fits(
 
 	ut_ad(!split_rec == !offsets);
 	ut_ad(!offsets
-		|| cursor->index->table->comp == rec_offs_comp(offsets));
+		|| page_is_comp(page) == !!rec_offs_comp(offsets));
 	ut_ad(!offsets
 		|| rec_offs_validate(split_rec, cursor->index, offsets));
-	ut_ad(page_is_comp(page) == cursor->index->table->comp);
 
 	insert_size = rec_get_converted_size(cursor->index, tuple);
-	free_space  = page_get_free_space_of_empty(cursor->index->table->comp);
+	free_space  = page_get_free_space_of_empty(page_is_comp(page));
 
 	/* free_space is now the free space of a created new page */
 
@@ -1833,14 +1825,15 @@ void
 btr_set_min_rec_mark_log(
 /*=====================*/
 	rec_t*	rec,	/* in: record */
-	ibool	comp,	/* TRUE=compact record format */
+	ulint	comp,	/* nonzero=compact record format */
 	mtr_t*	mtr)	/* in: mtr */
 {
 	mlog_write_initial_log_record(rec,
 		comp ? MLOG_COMP_REC_MIN_MARK : MLOG_REC_MIN_MARK, mtr);
 
 	/* Write rec offset as a 2-byte ulint */
-	mlog_catenate_ulint(mtr, rec - buf_frame_align(rec), MLOG_2BYTES);
+	mlog_catenate_ulint(mtr, ut_align_offset(rec, UNIV_PAGE_SIZE),
+								MLOG_2BYTES);
 }
 
 /********************************************************************
@@ -1853,7 +1846,7 @@ btr_parse_set_min_rec_mark(
 			/* out: end of log record or NULL */
 	byte*	ptr,	/* in: buffer */
 	byte*	end_ptr,/* in: buffer end */
-	ibool	comp,	/* in: TRUE=compact page format */
+	ulint	comp,	/* in: nonzero=compact page format */
 	page_t*	page,	/* in: page or NULL */
 	mtr_t*	mtr)	/* in: mtr or NULL */
 {
@@ -1865,6 +1858,8 @@ btr_parse_set_min_rec_mark(
 	}
 
 	if (page) {
+		ut_a(!page_is_comp(page) == !comp);
+
 		rec = page + mach_read_from_2(ptr);
 
 		btr_set_min_rec_mark(rec, comp, mtr);
@@ -1880,7 +1875,7 @@ void
 btr_set_min_rec_mark(
 /*=================*/
 	rec_t*	rec,	/* in: record */
-	ibool	comp,	/* in: TRUE=compact page format */
+	ulint	comp,	/* in: nonzero=compact page format */
 	mtr_t*	mtr)	/* in: mtr */
 {
 	ulint	info_bits;
@@ -2009,11 +2004,12 @@ btr_compress(
 	ulint		max_ins_size;
 	ulint		max_ins_size_reorg;
 	ulint		level;
-	ibool		comp	= cursor->index->table->comp;
+	ulint		comp;
 
 	page = btr_cur_get_page(cursor);
 	tree = btr_cur_get_tree(cursor);
-	ut_a(comp == page_is_comp(page));
+	comp = page_is_comp(page);
+	ut_a(!!comp == cursor->index->table->comp);
 
 	ut_ad(mtr_memo_contains(mtr, dict_tree_get_lock(tree),
 							MTR_MEMO_X_LOCK));
@@ -2056,7 +2052,7 @@ btr_compress(
 	
 	n_recs = page_get_n_recs(page);
 	data_size = page_get_data_size(page);
-	ut_a(page_is_comp(merge_page) == page_is_comp(page));
+	ut_a(page_is_comp(merge_page) == comp);
 
 	max_ins_size_reorg = page_get_max_insert_size_after_reorganize(
 							merge_page, n_recs);
@@ -2251,10 +2247,9 @@ btr_discard_page(
 
 		node_ptr = page_rec_get_next(page_get_infimum_rec(merge_page));
 
-		ut_ad(node_ptr != page_get_supremum_rec(merge_page));
+		ut_ad(page_rec_is_user_rec(node_ptr));
 
-		btr_set_min_rec_mark(node_ptr,
-					cursor->index->table->comp, mtr);
+		btr_set_min_rec_mark(node_ptr, page_is_comp(merge_page), mtr);
 	}	
 	
 	btr_node_ptr_delete(tree, page, mtr);
@@ -2499,8 +2494,8 @@ btr_index_rec_validate(
 	*offsets_ = (sizeof offsets_) / sizeof *offsets_;
 
 	page = buf_frame_align(rec);
-	
-	if (index->type & DICT_UNIVERSAL) {
+
+	if (UNIV_UNLIKELY(index->type & DICT_UNIVERSAL)) {
 	        /* The insert buffer index tree can contain records from any
 	        other index: we cannot check the number of fields or
 	        their length */
@@ -2508,9 +2503,18 @@ btr_index_rec_validate(
 	        return(TRUE);
 	}
 
+	if (UNIV_UNLIKELY(!!page_is_comp(page) != index->table->comp)) {
+		btr_index_rec_validate_report(page, rec, index);
+		fprintf(stderr, "InnoDB: compact flag=%lu, should be %lu\n",
+			(ulong) !!page_is_comp(page),
+			(ulong) index->table->comp);
+		return(FALSE);
+	}
+
 	n = dict_index_get_n_fields(index);
 
-	if (!index->table->comp && rec_get_n_fields_old(rec) != n) {
+	if (!page_is_comp(page)
+			&& UNIV_UNLIKELY(rec_get_n_fields_old(rec) != n)) {
 		btr_index_rec_validate_report(page, rec, index);
 		fprintf(stderr, "InnoDB: has %lu fields, should have %lu\n",
 			(ulong) rec_get_n_fields_old(rec), (ulong) n);
@@ -2774,7 +2778,7 @@ loop:
 	if (level > 0 && left_page_no == FIL_NULL) {
 		ut_a(REC_INFO_MIN_REC_FLAG & rec_get_info_bits(
 			page_rec_get_next(page_get_infimum_rec(page)),
-				index->table->comp));
+				page_is_comp(page)));
 	}
 
 	if (buf_frame_get_page_no(page) != dict_tree_get_page(tree)) {
@@ -2930,7 +2934,7 @@ node_ptr_fails:
 	mtr_commit(&mtr);
 
 	if (right_page_no != FIL_NULL) {
-		ibool	comp = page_is_comp(page);
+		ulint	comp = page_is_comp(page);
 		mtr_start(&mtr);
 	
 		page = btr_page_get(space, right_page_no, RW_X_LATCH, &mtr);
diff --git a/innobase/btr/btr0cur.c b/innobase/btr/btr0cur.c
index 80a2600c7f9..4a68ef1d42e 100644
--- a/innobase/btr/btr0cur.c
+++ b/innobase/btr/btr0cur.c
@@ -505,8 +505,9 @@ retry_page_get:
 
 			if (level > 0) {
 				/* x-latch the page */
-				ut_a(page_is_comp(btr_page_get(space,
-						page_no, RW_X_LATCH, mtr))
+				page = btr_page_get(space,
+						page_no, RW_X_LATCH, mtr);
+				ut_a(!!page_is_comp(page)
 						== index->table->comp);
 			}
 
@@ -961,7 +962,7 @@ calculate_sizes_again:
 	rec_size = rec_get_converted_size(index, entry);
 
 	if (rec_size >=
-		ut_min(page_get_free_space_of_empty(index->table->comp) / 2,
+		ut_min(page_get_free_space_of_empty(page_is_comp(page)) / 2,
 		REC_MAX_DATA_SIZE)) {
 
 		/* The record is so big that we have to store some fields
@@ -1027,7 +1028,7 @@ calculate_sizes_again:
 
 	*rec = page_cur_insert_rec_low(page_cursor, entry, index,
 							NULL, NULL, mtr);
-	if (!(*rec)) {
+	if (UNIV_UNLIKELY(!(*rec))) {
 		/* If the record did not fit, reorganize */
 		btr_page_reorganize(page, index, mtr);
 
@@ -1039,7 +1040,7 @@ calculate_sizes_again:
 
 		*rec = page_cur_tuple_insert(page_cursor, entry, index, mtr);
 
-		if (!*rec) {
+		if (UNIV_UNLIKELY(!*rec)) {
 			fputs("InnoDB: Error: cannot insert tuple ", stderr);
 			dtuple_print(stderr, entry);
 			fputs(" into ", stderr);
@@ -1166,7 +1167,7 @@ btr_cur_pessimistic_insert(
 	}
 
 	if (rec_get_converted_size(index, entry) >=
-		ut_min(page_get_free_space_of_empty(index->table->comp) / 2,
+		ut_min(page_get_free_space_of_empty(page_is_comp(page)) / 2,
 		REC_MAX_DATA_SIZE)) {
 
 		/* The record is so big that we have to store some fields
@@ -1293,9 +1294,11 @@ btr_cur_update_in_place_log(
 	mtr_t*		mtr)		/* in: mtr */
 {
 	byte*	log_ptr;
+	page_t*	page	= ut_align_down(rec, UNIV_PAGE_SIZE);
 	ut_ad(flags < 256);
+	ut_ad(!!page_is_comp(page) == index->table->comp);
 
-	log_ptr = mlog_open_and_write_index(mtr, rec, index, index->table->comp
+	log_ptr = mlog_open_and_write_index(mtr, rec, index, page_is_comp(page)
 			? MLOG_COMP_REC_UPDATE_IN_PLACE
 			: MLOG_REC_UPDATE_IN_PLACE,
 			1 + DATA_ROLL_PTR_LEN + 14 + 2 + MLOG_BUF_MARGIN);
@@ -1317,7 +1320,7 @@ btr_cur_update_in_place_log(
 
 	log_ptr = row_upd_write_sys_vals_to_log(index, trx, roll_ptr, log_ptr,
 									mtr);
-	mach_write_to_2(log_ptr, rec - buf_frame_align(rec));
+	mach_write_to_2(log_ptr, ut_align_offset(rec, UNIV_PAGE_SIZE));
 	log_ptr += 2;
 
 	row_upd_index_write_log(update, log_ptr, mtr);
@@ -1374,18 +1377,11 @@ btr_cur_parse_update_in_place(
 	
 	ptr = row_upd_index_parse(ptr, end_ptr, heap, &update);
 
-	if (ptr == NULL) {
-		mem_heap_free(heap);
-		
-		return(NULL);
+	if (!ptr || !page) {
+		goto func_exit;
 	}
 
-	if (!page) {
-		mem_heap_free(heap);
-
-		return(ptr);
-	}
-	
+	ut_a(!!page_is_comp(page) == index->table->comp);
 	rec = page + rec_offset;
 	
 	/* We do not need to reserve btr_search_latch, as the page is only
@@ -1400,6 +1396,7 @@ btr_cur_parse_update_in_place(
 
 	row_upd_rec_in_place(rec, offsets, update);
 
+func_exit:
 	mem_heap_free(heap);
 
 	return(ptr);
@@ -1438,7 +1435,6 @@ btr_cur_update_in_place(
 	rec = btr_cur_get_rec(cursor);
 	index = cursor->index;
 	trx = thr_get_trx(thr);
-	heap = mem_heap_create(100);
 	offsets = rec_get_offsets(rec, index, offsets, ULINT_UNDEFINED, &heap);
 
 	if (btr_cur_print_record_ops && thr) {
@@ -1449,7 +1445,7 @@ btr_cur_update_in_place(
 	/* Do lock checking and undo logging */
 	err = btr_cur_upd_lock_and_undo(flags, cursor, update, cmpl_info,
 							thr, &roll_ptr);
-	if (err != DB_SUCCESS) {
+	if (UNIV_UNLIKELY(err != DB_SUCCESS)) {
 
 		if (UNIV_LIKELY_NULL(heap)) {
 			mem_heap_free(heap);
@@ -1458,6 +1454,8 @@ btr_cur_update_in_place(
 	}
 
 	block = buf_block_align(rec);
+	ut_ad(!!page_is_comp(buf_block_get_frame(block))
+				== index->table->comp);
 
 	if (block->is_hashed) {
 		/* The function row_upd_changes_ord_field_binary works only
@@ -1481,7 +1479,8 @@ btr_cur_update_in_place(
 	/* FIXME: in a mixed tree, all records may not have enough ordering
 	fields for btr search: */
 
-	was_delete_marked = rec_get_deleted_flag(rec, index->table->comp);
+	was_delete_marked = rec_get_deleted_flag(rec,
+				page_is_comp(buf_block_get_frame(block)));
 
 	row_upd_rec_in_place(rec, offsets, update);
 
@@ -1491,7 +1490,8 @@ btr_cur_update_in_place(
 
 	btr_cur_update_in_place_log(flags, rec, index, update, trx, roll_ptr,
 									mtr);
-	if (was_delete_marked && !rec_get_deleted_flag(rec, index->table->comp)) {
+	if (was_delete_marked && !rec_get_deleted_flag(rec,
+				page_is_comp(buf_block_get_frame(block)))) {
 		/* The new updated record owns its possible externally
 		stored fields */
 
@@ -1597,7 +1597,7 @@ btr_cur_optimistic_update(
 	new_rec_size = rec_get_converted_size(index, new_entry);
 	
 	if (new_rec_size >=
-			page_get_free_space_of_empty(index->table->comp) / 2) {
+			page_get_free_space_of_empty(page_is_comp(page)) / 2) {
 
 		mem_heap_free(heap);		
 
@@ -1644,7 +1644,7 @@ btr_cur_optimistic_update(
 	explicit locks on rec, before deleting rec (see the comment in
 	.._pessimistic_update). */
 
-	lock_rec_store_on_page_infimum(rec);
+	lock_rec_store_on_page_infimum(page, rec);
 
 	btr_search_update_hash_on_delete(cursor);
 
@@ -1665,7 +1665,7 @@ btr_cur_optimistic_update(
 
 	ut_a(rec); /* <- We calculated above the insert would fit */
 
-	if (!rec_get_deleted_flag(rec, index->table->comp)) {
+	if (!rec_get_deleted_flag(rec, page_is_comp(page))) {
 		/* The new inserted record owns its possible externally
 		stored fields */
 
@@ -1814,7 +1814,7 @@ btr_cur_pessimistic_update(
 		}
 		
 		success = fsp_reserve_free_extents(&n_reserved,
-						cursor->index->space,
+						index->space,
 						n_extents, reserve_flag, mtr);
 		if (!success) {
 			err = DB_OUT_OF_FILE_SPACE;
@@ -1858,14 +1858,14 @@ btr_cur_pessimistic_update(
 
 	ext_vect = mem_heap_alloc(heap, sizeof(ulint)
 					* dict_index_get_n_fields(index));
-	ut_ad(!cursor->index->table->comp || !rec_get_node_ptr_flag(rec));
+	ut_ad(!page_is_comp(page) || !rec_get_node_ptr_flag(rec));
 	offsets = rec_get_offsets(rec, index, offsets,
 					ULINT_UNDEFINED, &heap);
 	n_ext_vect = btr_push_update_extern_fields(ext_vect, offsets, update);
 
-	if (rec_get_converted_size(index, new_entry) >=
-		ut_min(page_get_free_space_of_empty(index->table->comp) / 2,
-		REC_MAX_DATA_SIZE)) {
+	if (UNIV_UNLIKELY(rec_get_converted_size(index, new_entry) >=
+		ut_min(page_get_free_space_of_empty(page_is_comp(page)) / 2,
+		REC_MAX_DATA_SIZE))) {
 
                 big_rec_vec = dtuple_convert_big_rec(index, new_entry,
                 					ext_vect, n_ext_vect);
@@ -1887,7 +1887,7 @@ btr_cur_pessimistic_update(
 	delete the lock structs set on the root page even if the root
 	page carries just node pointers. */
 
-	lock_rec_store_on_page_infimum(rec);
+	lock_rec_store_on_page_infimum(buf_frame_align(rec), rec);
 
 	btr_search_update_hash_on_delete(cursor);
 
@@ -1965,8 +1965,7 @@ return_after_reservations:
 	mem_heap_free(heap);
 
 	if (n_extents > 0) {
-		fil_space_release_free_extents(cursor->index->space,
-							n_reserved);
+		fil_space_release_free_extents(index->space, n_reserved);
 	}
 
 	*big_rec = big_rec_vec;
@@ -1995,7 +1994,10 @@ btr_cur_del_mark_set_clust_rec_log(
 	ut_ad(flags < 256);
 	ut_ad(val <= 1);
 
-	log_ptr = mlog_open_and_write_index(mtr, rec, index, index->table->comp
+	ut_ad(!!page_rec_is_comp(rec) == index->table->comp);
+
+	log_ptr = mlog_open_and_write_index(mtr, rec, index,
+			page_rec_is_comp(rec)
 			? MLOG_COMP_REC_CLUST_DELETE_MARK
 			: MLOG_REC_CLUST_DELETE_MARK,
 			1 + 1 + DATA_ROLL_PTR_LEN + 14 + 2);
@@ -2012,7 +2014,7 @@ btr_cur_del_mark_set_clust_rec_log(
 
 	log_ptr = row_upd_write_sys_vals_to_log(index, trx, roll_ptr, log_ptr,
 									mtr);
-	mach_write_to_2(log_ptr, rec - buf_frame_align(rec));
+	mach_write_to_2(log_ptr, ut_align_offset(rec, UNIV_PAGE_SIZE));
 	log_ptr += 2;
 
 	mlog_close(mtr, log_ptr);
@@ -2039,6 +2041,8 @@ btr_cur_parse_del_mark_set_clust_rec(
 	ulint	offset;
 	rec_t*	rec;
 
+	ut_ad(!!page_is_comp(page) == index->table->comp);
+
 	if (end_ptr < ptr + 2) {
 
 		return(NULL);
@@ -2087,7 +2091,7 @@ btr_cur_parse_del_mark_set_clust_rec(
 		is only being recovered, and there cannot be a hash index to
 		it. */
 
-		rec_set_deleted_flag(rec, index->table->comp, val);
+		rec_set_deleted_flag(rec, page_is_comp(page), val);
 	}
 	
 	return(ptr);
@@ -2161,7 +2165,7 @@ btr_cur_del_mark_set_clust_rec(
 		rw_lock_x_lock(&btr_search_latch);
 	}
 
-	rec_set_deleted_flag(rec, index->table->comp, val);
+	rec_set_deleted_flag(rec, rec_offs_comp(offsets), val);
 
 	trx = thr_get_trx(thr);
 	
@@ -2486,6 +2490,7 @@ btr_cur_pessimistic_delete(
 	ulint		n_reserved;
 	ibool		success;
 	ibool		ret		= FALSE;
+	ulint		level;
 	mem_heap_t*	heap;
 	ulint*		offsets;
 	
@@ -2522,15 +2527,15 @@ btr_cur_pessimistic_delete(
 	/* Free externally stored fields if the record is neither
 	a node pointer nor in two-byte format.
 	This avoids an unnecessary loop. */
-	if (cursor->index->table->comp
+	if (page_is_comp(page)
 			? !rec_get_node_ptr_flag(rec)
 			: !rec_get_1byte_offs_flag(rec)) {
 		btr_rec_free_externally_stored_fields(cursor->index,
 					rec, offsets, in_rollback, mtr);
 	}
 
-	if ((page_get_n_recs(page) < 2)
-	    && (dict_tree_get_page(btr_cur_get_tree(cursor))
+	if (UNIV_UNLIKELY(page_get_n_recs(page) < 2)
+	    && UNIV_UNLIKELY(dict_tree_get_page(btr_cur_get_tree(cursor))
 					!= buf_frame_get_page_no(page))) {
 
 		/* If there is only one record, drop the whole page in
@@ -2545,9 +2550,13 @@ btr_cur_pessimistic_delete(
 	}
 
 	lock_update_delete(rec);
+	level = btr_page_get_level(page, mtr);
 
-	if ((btr_page_get_level(page, mtr) > 0)
-	    && (page_rec_get_next(page_get_infimum_rec(page)) == rec)) {
+	if (level > 0
+	    && UNIV_UNLIKELY(rec == page_rec_get_next(
+				page_get_infimum_rec(page)))) {
+
+		rec_t*	next_rec = page_rec_get_next(rec);
 
 		if (btr_page_get_prev(page, mtr) == FIL_NULL) {
 
@@ -2555,8 +2564,8 @@ btr_cur_pessimistic_delete(
 			non-leaf level, we must mark the new leftmost node
 			pointer as the predefined minimum record */
 
-			btr_set_min_rec_mark(page_rec_get_next(rec),
-					cursor->index->table->comp, mtr);
+			btr_set_min_rec_mark(next_rec, page_is_comp(page),
+						mtr);
 		} else {
 			/* Otherwise, if we delete the leftmost node pointer
 			on a page, we have to change the father node pointer
@@ -2566,13 +2575,12 @@ btr_cur_pessimistic_delete(
 			btr_node_ptr_delete(tree, page, mtr);
 
 			node_ptr = dict_tree_build_node_ptr(
-					tree, page_rec_get_next(rec),
+					tree, next_rec,
 					buf_frame_get_page_no(page),
-       					heap, btr_page_get_level(page, mtr));
+						heap, level);
 
 			btr_insert_on_non_leaf_level(tree,
-					btr_page_get_level(page, mtr) + 1,
-					node_ptr, mtr);
+					level + 1, node_ptr, mtr);
 		}
 	} 
 
@@ -2812,12 +2820,13 @@ btr_estimate_number_of_different_key_vals(
 	ulint		add_on;
 	mtr_t		mtr;
 	mem_heap_t*	heap		= NULL;
-	ulint		offsets1_[REC_OFFS_NORMAL_SIZE];
-	ulint		offsets2_[REC_OFFS_NORMAL_SIZE];
-	ulint*		offsets1	= offsets1_;
-	ulint*		offsets2	= offsets2_;
-	*offsets1_ = (sizeof offsets1_) / sizeof *offsets1_;
-	*offsets2_ = (sizeof offsets2_) / sizeof *offsets2_;
+	ulint		offsets_rec_[REC_OFFS_NORMAL_SIZE];
+	ulint		offsets_next_rec_[REC_OFFS_NORMAL_SIZE];
+	ulint*		offsets_rec	= offsets_rec_;
+	ulint*		offsets_next_rec= offsets_next_rec_;
+	*offsets_rec_ = (sizeof offsets_rec_) / sizeof *offsets_rec_;
+	*offsets_next_rec_ =
+			(sizeof offsets_next_rec_) / sizeof *offsets_next_rec_;
 
 	n_cols = dict_index_get_n_unique(index);
 
@@ -2830,6 +2839,7 @@ btr_estimate_number_of_different_key_vals(
 	/* We sample some pages in the index to get an estimate */
 	
 	for (i = 0; i < BTR_KEY_VAL_ESTIMATE_N_PAGES; i++) {
+		rec_t*	supremum;
 		mtr_start(&mtr);
 
 		btr_cur_open_at_rnd_pos(index, BTR_SEARCH_LEAF, &cursor, &mtr);
@@ -2842,26 +2852,29 @@ btr_estimate_number_of_different_key_vals(
 
 		page = btr_cur_get_page(&cursor);
 
-		rec = page_get_infimum_rec(page);
-		rec = page_rec_get_next(rec);
+		supremum = page_get_supremum_rec(page);
+		rec = page_rec_get_next(page_get_infimum_rec(page));
 
-		if (rec != page_get_supremum_rec(page)) {
+		if (rec != supremum) {
 			not_empty_flag = 1;
+			offsets_rec = rec_get_offsets(rec, index, offsets_rec,
+						ULINT_UNDEFINED, &heap);
 		}
-		
-		while (rec != page_get_supremum_rec(page)
-		       && page_rec_get_next(rec)
-					!= page_get_supremum_rec(page)) {
+
+		while (rec != supremum) {
 			rec_t*	next_rec = page_rec_get_next(rec);
+			if (next_rec == supremum) {
+				break;
+			}
+
 			matched_fields = 0;
 			matched_bytes = 0;
-			offsets1 = rec_get_offsets(rec, index, offsets1,
-						ULINT_UNDEFINED, &heap);
-			offsets2 = rec_get_offsets(next_rec, index, offsets2,
+			offsets_next_rec = rec_get_offsets(next_rec, index,
+						offsets_next_rec,
 						n_cols, &heap);
 
 			cmp_rec_rec_with_match(rec, next_rec,
-						offsets1, offsets2,
+						offsets_rec, offsets_next_rec,
 						index, &matched_fields,
 						&matched_bytes);
 
@@ -2874,9 +2887,17 @@ btr_estimate_number_of_different_key_vals(
 
 			total_external_size +=
 				btr_rec_get_externally_stored_len(
-								rec, offsets1);
+								rec, offsets_rec);
 			
-			rec = page_rec_get_next(rec);
+			rec = next_rec;
+			/* Initialize offsets_rec for the next round
+			and assign the old offsets_rec buffer to
+			offsets_next_rec. */
+			{
+				ulint*	offsets_tmp = offsets_rec;
+				offsets_rec = offsets_next_rec;
+				offsets_next_rec = offsets_tmp;
+			}
 		}
 		
 
@@ -2898,11 +2919,11 @@ btr_estimate_number_of_different_key_vals(
 			}
 		}
 
-		offsets1 = rec_get_offsets(rec, index, offsets1,
+		offsets_rec = rec_get_offsets(rec, index, offsets_rec,
 						ULINT_UNDEFINED, &heap);
 		total_external_size +=
 				btr_rec_get_externally_stored_len(rec,
-								offsets1);
+								offsets_rec);
 		mtr_commit(&mtr);
 	}
 
@@ -3598,7 +3619,7 @@ btr_rec_free_externally_stored_fields(
 							MTR_MEMO_PAGE_X_FIX));
 	/* Free possible externally stored fields in the record */
 
-	ut_ad(index->table->comp == rec_offs_comp(offsets));
+	ut_ad(index->table->comp == !!rec_offs_comp(offsets));
 	n_fields = rec_offs_n_fields(offsets);
 
 	for (i = 0; i < n_fields; i++) {
diff --git a/innobase/btr/btr0pcur.c b/innobase/btr/btr0pcur.c
index 74feff8653c..7cc883a6696 100644
--- a/innobase/btr/btr0pcur.c
+++ b/innobase/btr/btr0pcur.c
@@ -78,6 +78,7 @@ btr_pcur_store_position(
 	rec_t*		rec;
 	dict_tree_t*	tree;
 	page_t*		page;
+	ulint		offs;
 	
 	ut_a(cursor->pos_state == BTR_PCUR_IS_POSITIONED);
 	ut_ad(cursor->latch_mode != BTR_NO_LATCHES);
@@ -87,7 +88,8 @@ btr_pcur_store_position(
 	page_cursor = btr_pcur_get_page_cur(cursor);
 
 	rec = page_cur_get_rec(page_cursor);
-	page = buf_frame_align(rec);
+	page = ut_align_down(rec, UNIV_PAGE_SIZE);
+	offs = ut_align_offset(rec, UNIV_PAGE_SIZE);
 
 	ut_ad(mtr_memo_contains(mtr, buf_block_align(page),
 							MTR_MEMO_PAGE_S_FIX)
@@ -95,35 +97,33 @@ btr_pcur_store_position(
 							MTR_MEMO_PAGE_X_FIX));
 	ut_a(cursor->latch_mode != BTR_NO_LATCHES);
 
-	if (page_get_n_recs(page) == 0) {
+	if (UNIV_UNLIKELY(page_get_n_recs(page) == 0)) {
 		/* It must be an empty index tree; NOTE that in this case
 		we do not store the modify_clock, but always do a search
 		if we restore the cursor position */
 
-		ut_a(btr_page_get_next(page, mtr) == FIL_NULL
-		     && btr_page_get_prev(page, mtr) == FIL_NULL);
+		ut_a(btr_page_get_next(page, mtr) == FIL_NULL);
+		ut_a(btr_page_get_prev(page, mtr) == FIL_NULL);
 
-		if (rec == page_get_supremum_rec(page)) {
+		cursor->old_stored = BTR_PCUR_OLD_STORED;
 
-			cursor->rel_pos = BTR_PCUR_AFTER_LAST_IN_TREE;
-			cursor->old_stored = BTR_PCUR_OLD_STORED;
+		if (page_rec_is_supremum_low(offs)) {
 
-			return;
+			cursor->rel_pos = BTR_PCUR_AFTER_LAST_IN_TREE;
+		} else {
+			cursor->rel_pos = BTR_PCUR_BEFORE_FIRST_IN_TREE;
 		}
 
-		cursor->rel_pos = BTR_PCUR_BEFORE_FIRST_IN_TREE;
-		cursor->old_stored = BTR_PCUR_OLD_STORED;
-
 		return;
 	} 
 
-	if (rec == page_get_supremum_rec(page)) {
+	if (page_rec_is_supremum_low(offs)) {
 
 		rec = page_rec_get_prev(rec);
 
 		cursor->rel_pos = BTR_PCUR_AFTER;
 
-	} else if (rec == page_get_infimum_rec(page)) {
+	} else if (page_rec_is_infimum_low(offs)) {
 
 		rec = page_rec_get_next(rec);
 
@@ -139,7 +139,8 @@ btr_pcur_store_position(
 						&cursor->buf_size);
 
 	cursor->block_when_stored = buf_block_align(page);	
-	cursor->modify_clock = buf_frame_get_modify_clock(page);
+	cursor->modify_clock = buf_block_get_modify_clock(
+				cursor->block_when_stored);
 }
 
 /******************************************************************
@@ -202,12 +203,11 @@ btr_pcur_restore_position(
 	dtuple_t*	tuple;
 	ulint		mode;
 	ulint		old_mode;
-	ibool		from_left;
 	mem_heap_t*	heap;
 
-	ut_a(cursor->pos_state == BTR_PCUR_WAS_POSITIONED
+	ut_ad(cursor->pos_state == BTR_PCUR_WAS_POSITIONED
 			|| cursor->pos_state == BTR_PCUR_IS_POSITIONED);
-	if (cursor->old_stored != BTR_PCUR_OLD_STORED) {
+	if (UNIV_UNLIKELY(cursor->old_stored != BTR_PCUR_OLD_STORED)) {
 		ut_print_buf(stderr, (const byte*)cursor, sizeof(btr_pcur_t));
 		if (cursor->trx_if_known) {
 			trx_print(stderr, cursor->trx_if_known);
@@ -216,19 +216,14 @@ btr_pcur_restore_position(
 		ut_a(0);
 	}
 
-	if (cursor->rel_pos == BTR_PCUR_AFTER_LAST_IN_TREE
-	    || cursor->rel_pos == BTR_PCUR_BEFORE_FIRST_IN_TREE) {
+	if (UNIV_UNLIKELY(cursor->rel_pos == BTR_PCUR_AFTER_LAST_IN_TREE
+			|| cursor->rel_pos == BTR_PCUR_BEFORE_FIRST_IN_TREE)) {
 
 	    	/* In these cases we do not try an optimistic restoration,
 	    	but always do a search */
 
-	    	if (cursor->rel_pos == BTR_PCUR_BEFORE_FIRST_IN_TREE) {
-	    		from_left = TRUE;
-	    	} else {
-	    		from_left = FALSE;
-	    	}
-
-		btr_cur_open_at_index_side(from_left,
+		btr_cur_open_at_index_side(
+			cursor->rel_pos == BTR_PCUR_BEFORE_FIRST_IN_TREE,
 			btr_pcur_get_btr_cur(cursor)->index, latch_mode,
 					btr_pcur_get_btr_cur(cursor), mtr);
 
@@ -238,17 +233,18 @@ btr_pcur_restore_position(
 		return(FALSE);
 	}
 	
-	ut_a(cursor->old_rec);
-	ut_a(cursor->old_n_fields);
+	ut_ad(cursor->old_rec);
+	ut_ad(cursor->old_n_fields);
 
 	page = btr_cur_get_page(btr_pcur_get_btr_cur(cursor));
 
-	if (latch_mode == BTR_SEARCH_LEAF || latch_mode == BTR_MODIFY_LEAF) {
+	if (UNIV_LIKELY(latch_mode == BTR_SEARCH_LEAF)
+			|| UNIV_LIKELY(latch_mode == BTR_MODIFY_LEAF)) {
 		/* Try optimistic restoration */
 	    
-		if (buf_page_optimistic_get(latch_mode,
+		if (UNIV_LIKELY(buf_page_optimistic_get(latch_mode,
 					    cursor->block_when_stored, page,
-					    cursor->modify_clock, mtr)) {
+					    cursor->modify_clock, mtr))) {
 			cursor->pos_state = BTR_PCUR_IS_POSITIONED;
 #ifdef UNIV_SYNC_DEBUG
 			buf_page_dbg_add_level(page, SYNC_TREE_NODE);
@@ -297,7 +293,7 @@ btr_pcur_restore_position(
 	/* Save the old search mode of the cursor */
 	old_mode = cursor->search_mode;
 	
-	if (cursor->rel_pos == BTR_PCUR_ON) {
+	if (UNIV_LIKELY(cursor->rel_pos == BTR_PCUR_ON)) {
 		mode = PAGE_CUR_LE;
 	} else if (cursor->rel_pos == BTR_PCUR_AFTER) {
 		mode = PAGE_CUR_G;
@@ -323,12 +319,10 @@ btr_pcur_restore_position(
 		the cursor can now be on a different page! But we can retain
 		the value of old_rec */
 
-		cursor->modify_clock =
-			buf_frame_get_modify_clock(btr_pcur_get_page(cursor));
-
 		cursor->block_when_stored =
 			buf_block_align(btr_pcur_get_page(cursor));
-
+		cursor->modify_clock =
+			buf_block_get_modify_clock(cursor->block_when_stored);
 		cursor->old_stored = BTR_PCUR_OLD_STORED;
 
 		mem_heap_free(heap);
diff --git a/innobase/btr/btr0sea.c b/innobase/btr/btr0sea.c
index cb43876e303..742a02b1de4 100644
--- a/innobase/btr/btr0sea.c
+++ b/innobase/btr/btr0sea.c
@@ -544,7 +544,6 @@ btr_search_check_guess(
 				or PAGE_CUR_GE */
 	mtr_t*		mtr)	/* in: mtr */
 {
-	page_t*		page;
 	rec_t*		rec;
 	rec_t*		prev_rec;
 	rec_t*		next_rec;
@@ -561,7 +560,6 @@ btr_search_check_guess(
 	n_unique = dict_index_get_n_unique_in_tree(cursor->index);
 	
 	rec = btr_cur_get_rec(cursor);
-	page = buf_frame_align(rec);
 
 	ut_ad(page_rec_is_user_rec(rec));
 
@@ -612,12 +610,13 @@ btr_search_check_guess(
 
 	if ((mode == PAGE_CUR_G) || (mode == PAGE_CUR_GE)) {
 
-		ut_ad(rec != page_get_infimum_rec(page));
+		ut_ad(!page_rec_is_infimum(rec));
 		
 		prev_rec = page_rec_get_prev(rec);
 
-		if (prev_rec == page_get_infimum_rec(page)) {
-			success = btr_page_get_prev(page, mtr) == FIL_NULL;
+		if (page_rec_is_infimum(prev_rec)) {
+			success = btr_page_get_prev(
+				buf_frame_align(prev_rec), mtr) == FIL_NULL;
 			goto exit_func;
 		}
 
@@ -634,12 +633,13 @@ btr_search_check_guess(
 		goto exit_func;
 	}
 		
-	ut_ad(rec != page_get_supremum_rec(page));
+	ut_ad(!page_rec_is_supremum(rec));
 	
 	next_rec = page_rec_get_next(rec);
 
-	if (next_rec == page_get_supremum_rec(page)) {
-    		if (btr_page_get_next(page, mtr) == FIL_NULL) {
+	if (page_rec_is_supremum(next_rec)) {
+		if (btr_page_get_next(buf_frame_align(next_rec), mtr)
+							== FIL_NULL) {
 
 			cursor->up_match = 0;
 			success = TRUE;
@@ -694,7 +694,6 @@ btr_search_guess_on_hash(
 	buf_block_t*	block;
 	rec_t*		rec;
 	page_t*		page;
-	ibool		success;
 	ulint		fold;
 	ulint		tuple_n_fields;
 	dulint		tree_id;
@@ -710,7 +709,7 @@ btr_search_guess_on_hash(
 	/* Note that, for efficiency, the struct info may not be protected by
 	any latch here! */
 
-	if (info->n_hash_potential == 0) {
+	if (UNIV_UNLIKELY(info->n_hash_potential == 0)) {
 
 		return(FALSE);
 	}
@@ -720,12 +719,13 @@ btr_search_guess_on_hash(
 
 	tuple_n_fields = dtuple_get_n_fields(tuple);
 
-	if (tuple_n_fields < cursor->n_fields) {
+	if (UNIV_UNLIKELY(tuple_n_fields < cursor->n_fields)) {
 
 		return(FALSE);
 	}
 
-	if ((cursor->n_bytes > 0) && (tuple_n_fields <= cursor->n_fields)) {
+	if (UNIV_UNLIKELY(tuple_n_fields == cursor->n_fields)
+			&& (cursor->n_bytes > 0)) {
 
 	    	return(FALSE);
 	}
@@ -740,39 +740,31 @@ btr_search_guess_on_hash(
 	cursor->fold = fold;
 	cursor->flag = BTR_CUR_HASH;
 	
-	if (!has_search_latch) {
+	if (UNIV_LIKELY(!has_search_latch)) {
 		rw_lock_s_lock(&btr_search_latch);
 	}
 
-	ut_a(btr_search_latch.writer != RW_LOCK_EX);
-	ut_a(btr_search_latch.reader_count > 0);
+	ut_ad(btr_search_latch.writer != RW_LOCK_EX);
+	ut_ad(btr_search_latch.reader_count > 0);
 
 	rec = ha_search_and_get_data(btr_search_sys->hash_index, fold);
 
-	if (!rec) {
-		if (!has_search_latch) {
-			rw_lock_s_unlock(&btr_search_latch);
-		}
-		
-		goto failure;
+	if (UNIV_UNLIKELY(!rec)) {
+		goto failure_unlock;
 	}
 
 	page = buf_frame_align(rec);
 
-	if (!has_search_latch) {
+	if (UNIV_LIKELY(!has_search_latch)) {
 
-		success = buf_page_get_known_nowait(latch_mode, page,
+		if (UNIV_UNLIKELY(!buf_page_get_known_nowait(latch_mode, page,
 						BUF_MAKE_YOUNG,
 						__FILE__, __LINE__,
-						mtr);
-
-		rw_lock_s_unlock(&btr_search_latch);
-
-		if (!success) {
-
-			goto failure;
+						mtr))) {
+			goto failure_unlock;
 		}
 
+		rw_lock_s_unlock(&btr_search_latch);
 		can_only_compare_to_cursor_rec = FALSE;
 
 #ifdef UNIV_SYNC_DEBUG
@@ -782,8 +774,8 @@ btr_search_guess_on_hash(
 
 	block = buf_block_align(page);
 
-	if (block->state == BUF_BLOCK_REMOVE_HASH) {
-		if (!has_search_latch) {
+	if (UNIV_UNLIKELY(block->state == BUF_BLOCK_REMOVE_HASH)) {
+		if (UNIV_LIKELY(!has_search_latch)) {
 	
 			btr_leaf_page_release(page, latch_mode, mtr);
 		}
@@ -791,51 +783,33 @@ btr_search_guess_on_hash(
 		goto failure;
 	}
 
-	ut_a(block->state == BUF_BLOCK_FILE_PAGE);
-	ut_a(page_rec_is_user_rec(rec));	
+	ut_ad(block->state == BUF_BLOCK_FILE_PAGE);
+	ut_ad(page_rec_is_user_rec(rec));
 
 	btr_cur_position(index, rec, cursor);
 
 	/* Check the validity of the guess within the page */
 
-	if (0 != ut_dulint_cmp(tree_id, btr_page_get_index_id(page))) {
-
-		success = FALSE;
-/*
-		fprintf(stderr, "Tree id %lu, page index id %lu fold %lu\n",
-				ut_dulint_get_low(tree_id),
-				ut_dulint_get_low(btr_page_get_index_id(page)),
-				fold);
-*/				
-	} else {
-	        /* If we only have the latch on btr_search_latch, not on the
-		page, it only protects the columns of the record the cursor
-		is positioned on. We cannot look at the next of the previous
-		record to determine if our guess for the cursor position is
-		right. */
-
-		success = btr_search_check_guess(cursor,
-				               can_only_compare_to_cursor_rec,
-					       tuple, mode, mtr);
-	}
-	
-	if (!success) {
-		if (!has_search_latch) {
+	/* If we only have the latch on btr_search_latch, not on the
+	page, it only protects the columns of the record the cursor
+	is positioned on. We cannot look at the next of the previous
+	record to determine if our guess for the cursor position is
+	right. */
+	if (UNIV_EXPECT(ut_dulint_cmp(tree_id, btr_page_get_index_id(page)), 0)
+	    || !btr_search_check_guess(cursor, can_only_compare_to_cursor_rec,
+					       tuple, mode, mtr)) {
+		if (UNIV_LIKELY(!has_search_latch)) {
 		          btr_leaf_page_release(page, latch_mode, mtr);
 		}
 
 		goto failure;
 	}
 
-	if (info->n_hash_potential < BTR_SEARCH_BUILD_LIMIT + 5) {
+	if (UNIV_LIKELY(info->n_hash_potential < BTR_SEARCH_BUILD_LIMIT + 5)) {
 	
 		info->n_hash_potential++;
 	}
 
-	if (info->last_hash_succ != TRUE) {
-		info->last_hash_succ = TRUE;
-	}
-	
 #ifdef notdefined
 	/* These lines of code can be used in a debug version to check
 	the correctness of the searched cursor position: */
@@ -843,15 +817,14 @@ btr_search_guess_on_hash(
 	info->last_hash_succ = FALSE;
 
 	/* Currently, does not work if the following fails: */
-	ut_a(!has_search_latch);
+	ut_ad(!has_search_latch);
 	
 	btr_leaf_page_release(page, latch_mode, mtr);
 
 	btr_cur_search_to_nth_level(index, 0, tuple, mode, latch_mode,
 							&cursor2, 0, mtr);
 	if (mode == PAGE_CUR_GE
-		&& btr_cur_get_rec(&cursor2) == page_get_supremum_rec(
-			buf_frame_align(btr_cur_get_rec(&cursor2)))) {
+		&& page_rec_is_supremum(btr_cur_get_rec(&cursor2))) {
 
 		/* If mode is PAGE_CUR_GE, then the binary search
 		in the index tree may actually take us to the supremum
@@ -861,22 +834,22 @@ btr_search_guess_on_hash(
 
 		btr_pcur_open_on_user_rec(index, tuple, mode, latch_mode,
 				&pcur, mtr);
-		ut_a(btr_pcur_get_rec(&pcur) == btr_cur_get_rec(cursor));
+		ut_ad(btr_pcur_get_rec(&pcur) == btr_cur_get_rec(cursor));
 	} else {
-		ut_a(btr_cur_get_rec(&cursor2) == btr_cur_get_rec(cursor));
+		ut_ad(btr_cur_get_rec(&cursor2) == btr_cur_get_rec(cursor));
 	}
 
 	/* NOTE that it is theoretically possible that the above assertions
 	fail if the page of the cursor gets removed from the buffer pool
 	meanwhile! Thus it might not be a bug. */
-
-	info->last_hash_succ = TRUE;
 #endif
+	info->last_hash_succ = TRUE;
 
 #ifdef UNIV_SEARCH_PERF_STAT
 	btr_search_n_succ++;
 #endif
-	if (!has_search_latch && buf_block_peek_if_too_old(block)) {
+	if (UNIV_LIKELY(!has_search_latch)
+			&& buf_block_peek_if_too_old(block)) {
 
 		buf_page_make_young(page);
 	}	
@@ -889,6 +862,10 @@ btr_search_guess_on_hash(
 	return(TRUE);	
 
 	/*-------------------------------------------*/
+failure_unlock:
+	if (UNIV_LIKELY(!has_search_latch)) {
+		rw_lock_s_unlock(&btr_search_latch);
+	}
 failure:
 	info->n_hash_fail++;
 
@@ -917,7 +894,6 @@ btr_search_drop_page_hash_index(
 	ulint		n_fields;
 	ulint		n_bytes;
 	rec_t*		rec;
-	rec_t*		sup;
 	ulint		fold;
 	ulint		prev_fold;
 	dulint		tree_id;
@@ -968,12 +944,10 @@ btr_search_drop_page_hash_index(
 
 	n_cached = 0;
 
-	sup = page_get_supremum_rec(page);
-
 	rec = page_get_infimum_rec(page);
 	rec = page_rec_get_next(rec);
 
-	if (rec != sup) {
+	if (!page_rec_is_supremum(rec)) {
 		ut_a(n_fields <= rec_get_n_fields(rec, block->index));
 
 		if (n_bytes > 0) {
@@ -988,7 +962,7 @@ btr_search_drop_page_hash_index(
 	heap = NULL;
 	offsets = NULL;
 
-	while (rec != sup) {
+	while (!page_rec_is_supremum(rec)) {
 		/* FIXME: in a mixed tree, not all records may have enough
 		ordering fields: */
 		offsets = rec_get_offsets(rec, block->index,
@@ -1090,7 +1064,6 @@ btr_search_build_page_hash_index(
 	buf_block_t*	block;
 	rec_t*		rec;
 	rec_t*		next_rec;
-	rec_t*		sup;
 	ulint		fold;
 	ulint		next_fold;
 	dulint		tree_id;
@@ -1158,15 +1131,13 @@ btr_search_build_page_hash_index(
 
 	tree_id = btr_page_get_index_id(page);
 
-	sup = page_get_supremum_rec(page);
-
 	rec = page_get_infimum_rec(page);
 	rec = page_rec_get_next(rec);
 
 	offsets = rec_get_offsets(rec, index, offsets,
 					n_fields + (n_bytes > 0), &heap);
 
-	if (rec != sup) {
+	if (!page_rec_is_supremum(rec)) {
 		ut_a(n_fields <= rec_offs_n_fields(offsets));
 
 		if (n_bytes > 0) {
@@ -1188,7 +1159,7 @@ btr_search_build_page_hash_index(
 	for (;;) {
 		next_rec = page_rec_get_next(rec);
 
-		if (next_rec == sup) {
+		if (page_rec_is_supremum(next_rec)) {
 
 			if (side == BTR_SEARCH_RIGHT_SIDE) {
 	
@@ -1443,7 +1414,6 @@ btr_search_update_hash_on_insert(
 {
 	hash_table_t*	table; 
 	buf_block_t*	block;
-	page_t*		page;
 	rec_t*		rec;
 	rec_t*		ins_rec;
 	rec_t*		next_rec;
@@ -1488,19 +1458,18 @@ btr_search_update_hash_on_insert(
 	ins_rec = page_rec_get_next(rec);
 	next_rec = page_rec_get_next(ins_rec);
 
-	page = buf_frame_align(rec);
 	offsets = rec_get_offsets(ins_rec, cursor->index, offsets,
 					ULINT_UNDEFINED, &heap);
 	ins_fold = rec_fold(ins_rec, offsets, n_fields, n_bytes, tree_id);
 
-	if (next_rec != page_get_supremum_rec(page)) {
+	if (!page_rec_is_supremum(next_rec)) {
 		offsets = rec_get_offsets(next_rec, cursor->index, offsets,
 					n_fields + (n_bytes > 0), &heap);
 		next_fold = rec_fold(next_rec, offsets, n_fields,
 							n_bytes, tree_id);
 	}
 
-	if (rec != page_get_infimum_rec(page)) {
+	if (!page_rec_is_infimum(rec)) {
 		offsets = rec_get_offsets(rec, cursor->index, offsets,
 					n_fields + (n_bytes > 0), &heap);
 		fold = rec_fold(rec, offsets, n_fields, n_bytes, tree_id);
@@ -1534,7 +1503,7 @@ btr_search_update_hash_on_insert(
 	}
 
 check_next_rec:
-	if (next_rec == page_get_supremum_rec(page)) {
+	if (page_rec_is_supremum(next_rec)) {
 
 		if (side == BTR_SEARCH_RIGHT_SIDE) {