Added versioning of Maria index

Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
Changed info->lastkey to type MARIA_KEY. Removed info->lastkey_length as this is now part of info->lastkey
Renamed old info->lastkey to info->lastkey_buff
Use exact key lenghts for keys, not USE_WHOLE_KEY
For partial key searches, use SEARCH_PART_KEY
When searching to insert new key on page, use SEARCH_INSERT to mark that key has rowid

Changes done in a lot of files:
- Modified functions to use MARIA_KEY instead of key pointer and key length
- Use keyinfo->root_lock instead of share->key_root_lock[keynr]
- Simplify code by using local variable keyinfo instead if share->keyinfo[i]
- Added #fdef EXTERNAL_LOCKING around removed state elements
- HA_MAX_KEY_BUFF -> MARIA_MAX_KEY_BUFF (to reserve space for transid)
- Changed type of 'nextflag' to uint32 to ensure all SEARCH_xxx flags fits into it

.bzrignore:
  Added missing temporary directory
extra/Makefile.am:
  comp_err is now deleted on make distclean
include/maria.h:
  Added structure MARIA_KEY, which is used for intern key objects in Maria.
  Changed functions to take MARIA_KEY as an argument instead of pointer to packed key.
  Changed some functions that always return true or false to my_bool.
  Added virtual function make_key() to avoid if in _ma_make_key()
  Moved rw_lock_t for locking trees from share->key_root_lock to MARIA_KEYDEF. This makes usage of the locks simpler and faster
include/my_base.h:
  Added HA_RTREE_INDEX flag to mark rtree index. Used for easier checks in ma_check()
  Added SEARCH_INSERT to be used when inserting new keys
  Added SEARCH_PART_KEY for partial searches
  Added SEARCH_USER_KEY_HAS_TRANSID to be used when key we use for searching in btree has a TRANSID
  Added SEARCH_PAGE_KEY_HAS_TRANSID to be used when key we found in btree has a transid
include/my_handler.h:
  Make next_flag 32 bit to make sure we can handle all SEARCH_ bits
mysql-test/include/maria_empty_logs.inc:
  Read and restore current database; Don't assume we are using mysqltest.
  Don't log use databasename to log. Using this include should not cause any result changes.
mysql-test/r/maria-gis-rtree-dynamic.result:
  Updated results after adding some check table commands to help pinpoint errors
mysql-test/r/maria-mvcc.result:
  New tests
mysql-test/r/maria-purge.result:
  New result after adding removal of logs
mysql-test/r/maria-recovery-big.result:
  maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-bitmap.result:
  maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery-rtree-ft.result:
  maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria-recovery.result:
  maria_empty_logs doesn't log 'use mysqltest' anymore
mysql-test/r/maria.result:
  New tests
mysql-test/r/variables-big.result:
  Don't log id as it's not predictable
mysql-test/suite/rpl_ndb/r/rpl_truncate_7ndb_2.result:
  Updated results to new binlog results. (Test has not been run in a long time as it requires --big)
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2-master.opt:
  Moved file to ndb replication test directory
mysql-test/suite/rpl_ndb/t/rpl_truncate_7ndb_2.test:
  Fixed wrong path to included tests
mysql-test/t/maria-gis-rtree-dynamic.test:
  Added some check table commands to help pinpoint errors
mysql-test/t/maria-mvcc.test:
  New tests
mysql-test/t/maria-purge.test:
  Remove logs to make test results predictable
mysql-test/t/maria.test:
  New tests for some possible problems
mysql-test/t/variables-big.test:
  Don't log id as it's not predictable
mysys/my_handler.c:
  Updated function comment to reflect old code
  Changed nextflag to be uint32 to ensure we can have flags > 16 bit
  Changed checking if we are in insert with NULL keys as next_flag can now include additional bits that have to be ignored.
  Added SEARCH_INSERT flag to be used when inserting new keys in btree. This flag tells us the that the keys includes row position and it's thus safe to remove SEARCH_FIND
  Added comparision of transid. This is only done if the keys actually have a transid, which is indicated by nextflag
mysys/my_lock.c:
  Fixed wrong test (Found by Guilhem)
scripts/Makefile.am:
  Ensure that test programs are deleted by make clean
sql/rpl_rli.cc:
  Moved assignment order to fix compiler warning
storage/heap/hp_write.c:
  Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
storage/maria/Makefile.am:
  Remove also maria log files when doing make distclean
storage/maria/ha_maria.cc:
  Use 'file->start_state' as default state for transactional tables without versioning
  At table unlock, set file->state to point to live state. (Needed for information schema to pick up right number of rows)
  In ha_maria::implicit_commit() move all locked (ie open) tables to new transaction. This is needed to ensure ha_maria->info doesn't point to a deleted history event.
  Disable concurrent inserts for insert ... select and table changes with subqueries if statement based replication as this would cause wrong results on slave
storage/maria/ma_blockrec.c:
  Updated comment
storage/maria/ma_check.c:
  Compact key pages (removes transid) when doing --zerofill
  Check that 'page_flag' on key pages contains KEYPAGE_FLAG_HAS_TRANSID if there is a single key on the page with a transid
  Modified functions to use MARIA_KEY instead of key pointer and key length
  Use new interface to _ma_rec_pos(), _ma_dpointer(), _ma_ft_del(), ma_update_state_lsn()
  Removed not needed argument from get_record_for_key()
  Fixed that we check doesn't give errors for RTREE; We now treath these like SPATIAL
  Remove some SPATIAL specific code where the virtual functions can handle this in a general manner
  Use info->lastkey_buff instead of info->lastkey
  _ma_dpos() -> _ma_row_pos_from_key()
  _ma_make_key() -> keyinfo->make_key()
  _ma_print_key() -> _ma_print_keydata()
  _ma_move_key() -> ma_copy_copy()
  Add SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
  Ensure that data on page doesn't overwrite page checksum position
  Use DBUG_DUMP_KEY instead of DBUG_DUMP
  Use exact key lengths instead of USE_WHOLE_KEY to ha_key_cmp()
  Fixed check if rowid points outside of BLOCK_RECORD data file
  Use info->lastkey_buff instead of key on stack in some safe places
  Added #fdef EXTERNAL_LOCKING around removed state elements
storage/maria/ma_close.c:
  Use keyinfo->root_lock instead of share->key_root_lock[keynr]
storage/maria/ma_create.c:
  Removed assert that is already checked in maria_init()
  Force transactinal tables to be of type BLOCK_RECORD
  Fixed wrong usage of HA_PACK_RECORD (should be HA_OPTION_PACK_RECORD)
  Mark keys that uses HA_KEY_ALG_RTREE with HA_RTREE_INDEX for easier handling of these in ma_check
  Store max_trid in index file as state.create_trid. This is used to pack all transids in the index pages relative to max possible transid for file.
storage/maria/ma_dbug.c:
  Changed _ma_print_key() to use MARIA_KEY
storage/maria/ma_delete.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
  info->lastkey2-> info->lastkey_buff2
  Added SEARCH_INSERT to signal ha_key_cmp that we we should also compare rowid for keys
  Use new interface for get_key(), _ma_get_last_key() and others
  _ma_dpos() -> ma_row_pos_from_key()
  Simplify setting of prev_key in del()
  Ensure that KEYPAGE_FLAG_HAS_TRANSID is set in page_flag if key page has transid
  Treath key pages that may have a transid as if keys would be of variable length
storage/maria/ma_delete_all.c:
  Reset history state if maria_delete_all_rows() are called
  Update parameters to _ma_update_state_lsns() call
storage/maria/ma_extra.c:
  Store and restore info->lastkey
storage/maria/ma_ft_boolean_search.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ft_nlq_search.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
  Use lastkey_buff2 instead of info->lastkey+info->s->base.max_key_length (same thing)
storage/maria/ma_ft_update.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_ftdefs.h:
  Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_fulltext.h:
  Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_init.c:
  Check if blocksize is legal
  (Moved test here from ma_open())
storage/maria/ma_key.c:
  Added functions for storing/reading of transid 
  Modified functions to use MARIA_KEY instead of key pointer and key length
  Moved _ma_sp_make_key() out of _ma_make_key() as we now use keyinfo->make_key to create keys
  Add transid to keys if table is versioned
  Added _ma_copy_key()
storage/maria/ma_key_recover.c:
  Add logging of page_flag (holds information if there are keys with transid on page)
  Changed DBUG_PRINT("info" -> DBUG_PRINT("redo" as the redo logging can be quite extensive
  Added lots of DBUG_PRINT()
  Added support for index page operations: KEY_OP_SET_PAGEFLAG and KEY_OP_COMPACT_PAGE
storage/maria/ma_key_recover.h:
  Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_locking.c:
  Added new arguments to _ma_update_state_lsns_sub()
storage/maria/ma_loghandler.c:
  Fixed all logging of LSN to look similar in DBUG log
  Changed if (left != 0) to if (left) as the later is used also later in the code
storage/maria/ma_loghandler.h:
  Added new index page operations
storage/maria/ma_open.c:
  Removed allocated "state_dummy" and instead use share->state.common for transactional tables that are not versioned
  This is needed to not get double increments of state.records (one in ma_write.c and on when log is written)
  Changed info->lastkey to MARIA_KEY type
  Removed resetting of MARIA_HA variables that have 0 as default value (as info is zerofilled)
  Enable versioning for transactional tables with index. Tables with an auto-increment key, rtree or fulltext keys are not versioned.
  Check on open that state.create_trid is correct
  Extend share->base.max_key_length in case of transactional table so that it can hold transid
  Removed 4.0 compatible fulltext key mode as this is not relevant for Maria
  Removed old and wrong #ifdef ENABLE_WHEN_WE_HAVE_TRANS_ROW_ID code block
  Initialize all new virtual function pointers
  Removed storing of state->unique, state->process and store state->create_trid instead
storage/maria/ma_page.c:
  Added comment to describe key page structure
  Added functions to compact key page and log the compact operation
storage/maria/ma_range.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
  Use SEARCH_PART_KEY indicator instead of USE_WHOLE_KEY to detect if we are doing a part key search
  Added handling of pages with transid
storage/maria/ma_recovery.c:
  Don't assert if table we opened are not transactional. This may be a table which has been changed from transactional to not transactinal
  Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rename.c:
  Added new arguments to _ma_update_state_lsns()
storage/maria/ma_rkey.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
  Don't use USE_WHOLE_KEY, use real length of key
  Use share->row_is_visible() to test if row is visible
  Moved search_flag == HA_READ_KEY_EXACT out of 'read-next-row' loop as this only need to be tested once
  Removed test if last_used_keyseg != 0 as this is always true
storage/maria/ma_rnext.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
  Simplify code by using local variable keyinfo instead if share->keyinfo[i]
  Use share->row_is_visible() to test if row is visible
storage/maria/ma_rnext_same.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
  lastkey2 -> lastkey_buff2
storage/maria/ma_rprev.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
  Simplify code by using local variable keyinfo instead if share->keyinfo[i]
  Use share->row_is_visible() to test if row is visible
storage/maria/ma_rsame.c:
  Updated comment
  Simplify code by using local variable keyinfo instead if share->keyinfo[i]
  Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rsamepos.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_index.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
  Use better variable names
  Removed not needed casts
  _ma_dpos() -> _ma_row_pos_from_key()
  Use info->last_rtree_keypos to save position to key instead of info->int_keypos
  Simplify err: condition
  Changed return type for maria_rtree_insert() to my_bool as we are only intressed in ok/fail from this function
storage/maria/ma_rt_index.h:
  Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_key.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
  Simplify maria_rtree_add_key by combining idenitcal code and removing added_len
storage/maria/ma_rt_key.h:
  Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_rt_mbr.c:
  Changed type of 'nextflag' to uint32
  Added 'to' argument to RT_PAGE_MBR_XXX functions to more clearly see which variables changes value
storage/maria/ma_rt_mbr.h:
  Changed type of 'nextflag' to uint32
storage/maria/ma_rt_split.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
  key_length -> key_data_length to catch possible errors
storage/maria/ma_rt_test.c:
  Fixed wrong comment
  Reset recinfo to avoid valgrind varnings
  Fixed wrong argument to create_record() that caused test to fail
storage/maria/ma_search.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
  Added support of keys with optional trid
  Test for SEARCH_PART_KEY instead of USE_WHOLE_KEY to detect part key reads
  _ma_dpos() -> _ma_row_pos_from_key()
  If there may be keys with transid on the page, have _ma_bin_search() call _ma_seq_search()
  Add _ma_skip_xxx() functions to quickly step over keys (faster than calling get_key() in most cases as we don't have to copy key data)
  Combine similar code at end of _ma_get_binary_pack_key()
  Removed not used function _ma_move_key()
  In _ma_search_next() don't call _ma_search() if we aren't on a nod page.
  Update info->cur_row.trid with trid for found key
  
  
  
  Removed some not needed casts
  Added _ma_trid_from_key()
  Use MARIA_SHARE instead of MARIA_HA as arguments to _ma_rec_pos(), _ma_dpointer() and _ma_xxx_keypos_to_recpos() to make functions faster and smaller
storage/maria/ma_sort.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_sp_defs.h:
  _ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
storage/maria/ma_sp_key.c:
  _ma_sp_make_key() now fills in and returns (MARIA_KEY *) value
  Don't test sizeof(double), test against 8 as we are using float8store()
  Use mi_float8store() instead of doing swap of value (same thing but faster)
storage/maria/ma_state.c:
  maria_versioning() now only calls _ma_block_get_status() if table supports versioning
  Added _ma_row_visible_xxx() functions for different occasions
  When emptying history, set info->state to point to the first history event.
storage/maria/ma_state.h:
  Added _ma_row_visible_xxx() prototypes
storage/maria/ma_static.c:
  Indentation changes
storage/maria/ma_statrec.c:
  Fixed arguments to _ma_dpointer() and _ma_rec_pos()
storage/maria/ma_test1.c:
  Call init_thr_lock() if we have versioning
storage/maria/ma_test2.c:
  Call init_thr_lock() if we have versioning
storage/maria/ma_unique.c:
  Modified functions to use MARIA_KEY
storage/maria/ma_update.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
storage/maria/ma_write.c:
  Modified functions to use MARIA_KEY instead of key pointer and key length
  Simplify code by using local variable keyinfo instead if share->keyinfo[i]
  In _ma_enlarge_root(), mark in page_flag if new key has transid
  _ma_dpos() -> _ma_row_pos_from_key()
  Changed return type of _ma_ck_write_tree() to my_bool as we are only testing if result is true or not
  Moved 'reversed' to outside block as area was used later
storage/maria/maria_chk.c:
  Added error if trying to sort with HA_BINARY_PACK_KEY
  Use new interface to get_key() and _ma_dpointer()
  _ma_dpos() -> _ma_row_pos_from_key()
storage/maria/maria_def.h:
  Modified functions to use MARIA_KEY instead of key pointer and key length
  Added 'common' to MARIA_SHARE->state for storing state for transactional tables without versioning
  Added create_trid to MARIA_SHARE
  Removed not used state variables 'process' and 'unique'
  Added defines for handling TRID's in index pages
  Changed to use MARIA_SHARE instead of MARIA_HA for some functions
  Added 'have_versioning' flag if table supports versioning
  Moved key_root_lock from MARIA_SHARE to MARIA_KEYDEF
  Changed last_key to be of type MARIA_KEY. Removed lastkey_length
  lastkey -> lastkey_buff, lastkey2 -> lastkey_buff2
  Added _ma_get_used_and_nod_with_flag() for faster access to page data when page_flag is read
  Added DBUG_DUMP_KEY for easier DBUG_DUMP of a key
  Changed 'nextflag' and assocaited variables to uint32
storage/maria/maria_ftdump.c:
  lastkey -> lastkey_buff
storage/maria/trnman.c:
  Fixed wrong initialization of min_read_from and max_commit_trid
  Added trnman_get_min_safe_trid()
storage/maria/unittest/ma_test_all-t:
  Added --start-from
storage/myisam/mi_check.c:
  Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_delete.c:
  Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/mi_range.c:
  Updated comment
storage/myisam/mi_write.c:
  Added SEARCH_INSERT, as ha_key_cmp() needs it when doing key comparision for inserting key on page in rowid order
storage/myisam/rt_index.c:
  Fixed wrong parameter to rtree_get_req() which could cause crash

1 files changed, 179 insertions, 43 deletions

diff --git a/storage/maria/ma_key.c b/storage/maria/ma_key.c
index ac384821caf..9dfc0128b70 100644
--- a/storage/maria/ma_key.c
+++ b/storage/maria/ma_key.c
@@ -18,12 +18,16 @@
 #include "maria_def.h"
 #include "m_ctype.h"
 #include "ma_sp_defs.h"
+#include "ma_blockrec.h"                        /* For ROW_FLAG_TRANSID */
+#include <trnman.h>
 #ifdef HAVE_IEEEFP_H
 #include <ieeefp.h>
 #endif
 
 #define CHECK_KEYS                              /* Enable safety checks */
 
+static int _ma_put_key_in_record(MARIA_HA *info,uint keynr,uchar *record);
+
 #define FIX_LENGTH(cs, pos, length, char_length)                            \
             do {                                                            \
               if (length > char_length)                                     \
@@ -31,46 +35,146 @@
               set_if_smaller(char_length,length);                           \
             } while(0)
 
-static int _ma_put_key_in_record(MARIA_HA *info,uint keynr,uchar *record);
+
+/**
+  Store trid in a packed format as part of a key
+
+  @fn    transid_store_packed
+  @param info   Maria handler
+  @param to     End of key to which we should store a packed transid
+  @param trid   Trid to be stored
+
+  @notes
+
+  Keys that have a transid has the lowest bit set for the last byte of the key
+  This function sets this bit for the key.
+
+  Trid is max 6 bytes long
+
+  First Trid it's converted to a smaller number by using
+  trid= trid - create_trid.
+  Then trid is then shifted up one bit so that we can use the
+  lowest bit as a marker if it's followed by another trid.
+
+  Trid is then stored as follows:
+
+  if trid < 256-12
+    one byte
+  else
+    one byte prefix (256-length_of_trid_in_bytes) followed by data
+    in high-byte-first order
+
+  Prefix bytes 244 to 249 are reserved for negative transid, that can be used
+  when we pack transid relative to each other on a key block.
+
+  We have to store transid in high-byte-first order to be able to do a
+  fast byte-per-byte comparision of them without packing them up.
+*/
+
+uint transid_store_packed(MARIA_HA *info, uchar *to, ulonglong trid)
+{
+  uchar *start;
+  uint length;
+  uchar buff[8];
+  DBUG_ASSERT(trid < (LL(1) << (MAX_PACK_TRANSID_SIZE*8)));
+  DBUG_ASSERT(trid >= info->s->state.create_trid);
+
+  trid= (trid - info->s->state.create_trid) << 1;
+
+  /* Mark that key contains transid */
+  to[-1]|= 1;
+
+  if (trid < MIN_TRANSID_PACK_PREFIX)
+  {
+    to[0]= (uchar) trid;
+    return 1;
+  }
+  start= to;
+
+  /* store things in low-byte-first-order in buff */
+  to= buff;
+  do
+  {
+    *to++= (uchar) trid;
+    trid= trid>>8;
+  } while (trid);
+
+  length= (uint) (to - buff);
+  start[0]= (uchar) (256 - length);             /* Store length prefix */
+  start++;
+  /* Copy things in high-byte-first order to output buffer */
+  do
+  {
+    *start++= *--to;
+  } while (to != buff);
+  return length+1;
+}
+
+
+/**
+   Read packed transid
+
+   @fn    transid_get_packed
+   @param info   Maria handler
+   @param from	 Transid is stored here
+
+   See transid_store_packed() for how transid is packed
+
+*/
+
+ulonglong transid_get_packed(MARIA_SHARE *share, const uchar *from)
+{
+  ulonglong value;
+  uint length;
+
+  if (from[0] < MIN_TRANSID_PACK_PREFIX)
+    value= (ulonglong) from[0];
+  else
+  {
+    value= 0;
+    for (length= (uint) (256 - from[0]), value= (ulonglong) from[1], from+=2;
+         --length ;
+         from++)
+      value= (value << 8) + ((ulonglong) *from);
+  }
+  return (value >> 1) + share->state.create_trid;
+}
+
 
 /*
-  Make a intern key from a record
+  Make a normal (not spatial or fulltext) intern key from a record
 
   SYNOPSIS
     _ma_make_key()
     info		MyiSAM handler
+    int_key		Store created key here
     keynr		key number
-    key			Store created key here
+    key			Buffer used to store key data
     record		Record
     filepos		Position to record in the data file
 
+  NOTES
+    This is used to generate keys from the record on insert, update and delete
+
   RETURN
-    Length of key
+    key
 */
 
-uint _ma_make_key(register MARIA_HA *info, uint keynr, uchar *key,
-		  const uchar *record, MARIA_RECORD_POS filepos)
+MARIA_KEY *_ma_make_key(MARIA_HA *info, MARIA_KEY *int_key, uint keynr,
+                        uchar *key, const uchar *record,
+                        MARIA_RECORD_POS filepos, ulonglong trid)
 {
   const uchar *pos;
-  uchar *start;
   reg1 HA_KEYSEG *keyseg;
-  my_bool is_ft= info->s->keyinfo[keynr].flag & HA_FULLTEXT;
+  my_bool is_ft;
   DBUG_ENTER("_ma_make_key");
 
-  if (info->s->keyinfo[keynr].flag & HA_SPATIAL)
-  {
-    /*
-      TODO: nulls processing
-    */
-#ifdef HAVE_SPATIAL
-    DBUG_RETURN(_ma_sp_make_key(info,keynr, key,record,filepos));
-#else
-    DBUG_ASSERT(0); /* maria_open should check that this never happens*/
-#endif
-  }
+  int_key->data= key;
+  int_key->flag= 0;                             /* Always return full key */
+  int_key->keyinfo= info->s->keyinfo + keynr;
 
-  start=key;
-  for (keyseg=info->s->keyinfo[keynr].seg ; keyseg->type ;keyseg++)
+  is_ft= int_key->keyinfo->flag & HA_FULLTEXT;
+  for (keyseg= int_key->keyinfo->seg ; keyseg->type ;keyseg++)
   {
     enum ha_base_keytype type=(enum ha_base_keytype) keyseg->type;
     uint length=keyseg->length;
@@ -188,13 +292,22 @@ uint _ma_make_key(register MARIA_HA *info, uint keynr, uchar *key,
       cs->cset->fill(cs, (char*) key+char_length, length-char_length, ' ');
     key+= length;
   }
-  _ma_dpointer(info,key,filepos);
+  _ma_dpointer(info->s, key, filepos);
+  int_key->data_length= (key - int_key->data);
+  int_key->ref_length= info->s->rec_reflength;
+  int_key->flag= 0;
+  if (_ma_have_versioning(info) && trid)
+  {
+    int_key->ref_length+= transid_store_packed(info,
+                                               key + int_key->ref_length,
+                                               (TrID) trid);
+    int_key->flag|= SEARCH_USER_KEY_HAS_TRANSID;
+  }
+
   DBUG_PRINT("exit",("keynr: %d",keynr));
-  DBUG_DUMP("key",start,(uint) (key-start)+keyseg->length);
-  DBUG_EXECUTE("key",
-	       _ma_print_key(DBUG_FILE,info->s->keyinfo[keynr].seg,start,
-			     (uint) (key-start)););
-  DBUG_RETURN((uint) (key-start));		/* Return keylength */
+  DBUG_DUMP_KEY("key", int_key);
+  DBUG_EXECUTE("key", _ma_print_key(DBUG_FILE, int_key););
+  DBUG_RETURN(int_key);
 } /* _ma_make_key */
 
 
@@ -204,35 +317,40 @@ uint _ma_make_key(register MARIA_HA *info, uint keynr, uchar *key,
   SYNOPSIS
     _ma_pack_key()
     info		MARIA handler
-    uint keynr		key number
-    key			Store packed key here
-    old			Not packed key
+    int_key		Store key here
+    keynr		key number
+    key			Buffer for key data
+    old			Original not packed key
     keypart_map         bitmap of used keyparts
     last_used_keyseg	out parameter.  May be NULL
 
    RETURN
-     length of packed key
+   int_key
 
      last_use_keyseg    Store pointer to the keyseg after the last used one
 */
 
-uint _ma_pack_key(register MARIA_HA *info, uint keynr, uchar *key,
-                  const uchar *old, key_part_map keypart_map,
-                  HA_KEYSEG **last_used_keyseg)
+MARIA_KEY *_ma_pack_key(register MARIA_HA *info, MARIA_KEY *int_key,
+                        uint keynr, uchar *key,
+                        const uchar *old, key_part_map keypart_map,
+                        HA_KEYSEG **last_used_keyseg)
 {
-  uchar *start_key=key;
   HA_KEYSEG *keyseg;
-  my_bool is_ft= info->s->keyinfo[keynr].flag & HA_FULLTEXT;
+  my_bool is_ft;
   DBUG_ENTER("_ma_pack_key");
 
+  int_key->data= key;
+  int_key->keyinfo= info->s->keyinfo + keynr;
+
   /* "one part" rtree key is 2*SPDIMS part key in Maria */
-  if (info->s->keyinfo[keynr].key_alg == HA_KEY_ALG_RTREE)
+  if (int_key->keyinfo->key_alg == HA_KEY_ALG_RTREE)
     keypart_map= (((key_part_map)1) << (2*SPDIMS)) - 1;
 
   /* only key prefixes are supported */
   DBUG_ASSERT(((keypart_map+1) & keypart_map) == 0);
 
-  for (keyseg=info->s->keyinfo[keynr].seg ; keyseg->type && keypart_map;
+  is_ft= int_key->keyinfo->flag & HA_FULLTEXT;
+  for (keyseg=int_key->keyinfo->seg ; keyseg->type && keypart_map;
        old+= keyseg->length, keyseg++)
   {
     enum ha_base_keytype type= (enum ha_base_keytype) keyseg->type;
@@ -303,11 +421,29 @@ uint _ma_pack_key(register MARIA_HA *info, uint keynr, uchar *key,
   if (last_used_keyseg)
     *last_used_keyseg= keyseg;
 
-  DBUG_PRINT("exit", ("length: %u", (uint) (key-start_key)));
-  DBUG_RETURN((uint) (key-start_key));
+  /* set flag to SEARCH_PART_KEY if we are not using all key parts */
+  int_key->flag= keyseg->type ? SEARCH_PART_KEY : 0;
+  int_key->ref_length= 0;
+  int_key->data_length= (key - int_key->data);
+
+  DBUG_PRINT("exit", ("length: %u", int_key->data_length));
+  DBUG_RETURN(int_key);
 } /* _ma_pack_key */
 
 
+/**
+   Copy a key
+*/
+
+void _ma_copy_key(MARIA_KEY *to, const MARIA_KEY *from)
+{
+  memcpy(to->data, from->data, from->data_length + from->ref_length);
+  to->keyinfo=     from->keyinfo;
+  to->data_length= from->data_length;
+  to->ref_length=  from->ref_length;
+  to->flag=        from->flag;
+}
+
 
 /*
   Store found key in record
@@ -337,9 +473,9 @@ static int _ma_put_key_in_record(register MARIA_HA *info, uint keynr,
   uchar *blob_ptr;
   DBUG_ENTER("_ma_put_key_in_record");
 
-  blob_ptr= info->lastkey2;             /* Place to put blob parts */
-  key=info->lastkey;                    /* KEy that was read */
-  key_end=key+info->lastkey_length;
+  blob_ptr= info->lastkey_buff2;         /* Place to put blob parts */
+  key= info->last_key.data;               /* Key that was read */
+  key_end= key + info->last_key.data_length;
   for (keyseg=info->s->keyinfo[keynr].seg ; keyseg->type ;keyseg++)
   {
     if (keyseg->null_bit)