MDEV-9750: Quick memory exhaustion with 'extended_keys=on' ...10.5-mdev9750

(Variant #4, full patch)

Do not produce SEL_ARG graphs that would yield huge numbers of ranges.
Introduce a concept of SEL_ARG graph's "weight". If we are about to
produce a graph whose "weight" exceeds the limit, remove the parts
of SEL_ARG graph that represent the biggest key parts. Do so until
the graph's is within the limit.

Includes
- debug code to verify SEL_ARG graph weight
- A user-visible @@optimizer_max_sel_arg_weight to control the optimization
- Logging the optimization into the optimizer trace.

1 files changed, 1 insertions, 0 deletions

diff --git a/sql/sql_class.h b/sql/sql_class.h
index 79dcd47bbc2..eaa97c2778d 100644
--- a/sql/sql_class.h
+++ b/sql/sql_class.h
@@ -815,6 +815,7 @@ typedef struct system_variables
   uint column_compression_threshold;
   uint column_compression_zlib_level;
   uint in_subquery_conversion_threshold;
+  ulong optimizer_max_sel_arg_weight;
   ulonglong max_rowid_filter_size;
 
   vers_asof_timestamp_t vers_asof_timestamp;