Fixed bug #28076: inconsistent binary/varbinary comparison.

After adding an index the <VARBINARY> IN (SELECT <BINARY> ...)
clause returned a wrong result: the VARBINARY value was illegally padded
with zero bytes to the length of the BINARY column for the index search.
(<VARBINARY>, ...) IN (SELECT <BINARY>, ... ) clauses are affected too.


sql/item.cc:
  Fixed bug #28076.
  The Item_cache_str::save_in_field method has been overloaded
  to check cached values for an illegal padding before the saving
  into a field.
sql/item.h:
  Fixed bug #28076.
  The Item_cache_str::is_varbinary flag has been added and the
  Item_cache_str::save_in_field method has been overloaded to prevent
  cached values from an illegal padding when saving in fields.
  The signature of the Item_cache::get_cache method has been
  changed to accept pointers to Item instead of Item_result
  values.
sql/item_cmpfunc.cc:
  Fixed bug #28076.
  The Item_in_optimizer::fix_left method has been modified to
  to call Item_cache::get_cache in a new manner.
sql/item_subselect.cc:
  Fixed bug #28076.
  The subselect_indexsubquery_engine::exec method has been
  modified to take into account field conversion errors
  (copy&paste from subselect_uniquesubquery_engine::exec).
sql/sp_rcontext.cc:
  Fixed bug #28076.
  The sp_rcontext::create_case_expr_holder method has been
  modified to call Item_cache::get_cache in a new manner.
sql/sp_rcontext.h:
  Fixed bug #28076.
  The sp_rcontext::create_case_expr_holder method signature
  has been modified to pass Item pointers to the
  Item_cache::get_cache method.
sql/sql_class.cc:
  Fixed bug #28076.
  The select_max_min_finder_subselect::send_data method has been
  modified to call Item_cache::get_cache in a new manner.
mysql-test/t/subselect.test:
  Added test case for bug #28076.
mysql-test/r/subselect.result:
  Added test case for bug #28076.

1 files changed, 3 insertions, 3 deletions

diff --git a/sql/sp_rcontext.cc b/sql/sp_rcontext.cc
index ac7c46c9fe5..54e016f6099 100644
--- a/sql/sp_rcontext.cc
+++ b/sql/sp_rcontext.cc
@@ -503,14 +503,14 @@ sp_cursor::fetch(THD *thd, List<struct sp_variable> *vars)
 */
 
 Item_cache *
-sp_rcontext::create_case_expr_holder(THD *thd, Item_result result_type)
+sp_rcontext::create_case_expr_holder(THD *thd, const Item *item)
 {
   Item_cache *holder;
   Query_arena current_arena;
 
   thd->set_n_backup_active_arena(thd->spcont->callers_arena, &current_arena);
 
-  holder= Item_cache::get_cache(result_type);
+  holder= Item_cache::get_cache(item);
 
   thd->restore_active_arena(thd->spcont->callers_arena, &current_arena);
 
@@ -559,7 +559,7 @@ sp_rcontext::set_case_expr(THD *thd, int case_expr_id, Item **case_expr_item_ptr
         case_expr_item->result_type())
   {
     m_case_expr_holders[case_expr_id]=
-      create_case_expr_holder(thd, case_expr_item->result_type());
+      create_case_expr_holder(thd, case_expr_item);
   }
 
   m_case_expr_holders[case_expr_id]->store(case_expr_item);