generalization of mtr to support suite.pm extensions:

* no automatic --loose-skip-innodb added by mtr based on the test name.
  instead loose-skip-innodb is now in the default_mysqld.cnf
* have_innodb_plugin.inc is changed to give a verbose "skip" message
  (instead of "require: true")
* My::Suite class. It's support in mtr, and everywhere
* support for suite.pm
* when sorting tests, take combinations into account
* support for SUITENAME_COMBINATIONS
* no special treatment for innodb_plugin in mtr_cases.pm
* two special pre-created config groups: ENV and OPT
* allow option names to start from #
* allow magic option to have an argument
* remove dead code
* fix @-substitution to works as expected
* new processes take the value of $opt_verbose automatically, no need to pass it to a constructor
* innodb_plugin suite uses suite.pm and combinations file to test as much as possible
  (innodb plugin, xtradb plugin, xtradb static - whatever available)
* besides test-master.opt and test-slave.opt a test.opt file is also
  loaded, both for master and slave
* .opt files for all included files are loaded too
* progress report in the xterm titlebar

1 files changed, 136 insertions, 0 deletions

diff --git a/mysql-test/README.suites b/mysql-test/README.suites
new file mode 100644
index 00000000000..d8f39cbc39f
--- /dev/null
+++ b/mysql-test/README.suites
@@ -0,0 +1,136 @@
+These are the assorted notes that will be turned into a manual eventually.
+
+==========================
+Tests are organized in suites.
+A "suite" is a subdirectory inside, one of,
+
+  <basedir>/mysql-test/suite
+  <basedir>/mysql-test
+  <basedir>/share/mysql-test/suite
+  <basedir>/share/mysql-test
+  <basedir>/share/mysql/mysql-test/suite
+  <basedir>/share/mysql/mysql-test
+  <basedir>/storage/*/mysql-test-suites
+
+This is supposed to cover running mtr from a source directory and installed.
+
+==========================
+A suite contains *.test and *.result files. They can be in the t/ and r/
+subdirectories under the suitedir or directly in the suitedir
+(that is suitedir/t/*.test or suitedir/*.test, same for *.result))
+
+==========================
+A suite can contain a suite.opt file - at the same location where .test
+files are. As usual, the .opt file can use $-substitutions for the
+environment variables.
+
+Usually, using my.cnf template (see below) is preferrable.
+==========================
+A suite can have suite.pm file in the suitedir. It must declare a
+package that inherits from My::Suite.
+
+The suite.pm needs to have @ISA=qw(My::Suite) and it must end
+with bless {}; - that is it must return an object of that class.
+
+A suite class can define config_files() and servers() methods.
+
+A config_files method returns a list of additional config files (besides
+my.cnf), that this suite needs to be created. For every file it specifies
+a function that will create it, when given a My::Config object. For example:
+
+   sub config_files { ( 'config.ini' => \&write_ini,
+                         'new.conf' => \&do_new_conf ) }
+
+A servers method returns a list of processes that needs to be started for
+this suite. A process is specified as a pair (regex, hash). A regex must
+match a section in the my.cnf template (for example, qr/mysqld\./ corresponds
+to all mysqld processes), a hash contains these options:
+
+   SORT => a number, processes are started in the order of increasing SORT
+           values (and stopped in the reverse order). mysqld has number 300.
+   START => a function to start a process. It takes two arguments,
+            My::Config::Group and My::Test. If START is undefined the process
+            will not be started.
+   WAIT => a function waits for the process to be started. It takes
+           My::Config::Group as an argument. Internallys mtr first invokes
+           START for all processes, then WAIT for all started processes.
+
+example: sub servers { ( qr/^foo$/ => { SORT => 200,
+                                        START => \&start_foo,
+                                        WAIT => \&wait_foo } ) }
+
+See sphinx suite for an example.
+
+==========================
+A suite can have my.cnf template file in the suitedir.
+A my.cnf template uses a normal my.cnf syntax - groups, options,
+and values - with templating extensions. They are
+
+* There can be groups with non-standard names, not used by mysqld.
+  These groups may be used by the suite.pm file somehow.
+  For example, they can be written to the additional config files.
+  See sphinx suite for an example.
+
+* There can be ENV group. It sets values for the environment variables.
+
+* Values can refer to each other - they will be expanded as needed.
+  A reference to a value of an option looks like @groupname.optionname.
+  For example
+
+  [mysqld.2]
+  master-port= @mysqld.1.port
+
+  it sets the master-port in the mysqld.2 group to the value of
+  port in the mysqld.1 group.
+
+* An option name may start from '#'. In the resulting my.cnf it will look
+  like a comment, but it still can be referred to. For example:
+
+  [example]
+  #foo = localhost:@mysqld.1.port
+  bar = http://@example.#foo/index.html
+
+* There are two special - in this regard - groups.
+
+  Via the ENV group one can refer to any environment variable, not only
+    to values in the [ENV] group of my.cnf file.
+
+  Via the OPT group one can refer to special values:
+    @OPT.vardir - a path to vardir
+    @OPT.port - a new port number is reserved out of the pool. It will not
+                match any other port number used by this test run.
+  See sphinx suite for an example.
+
+Most probably a suite my.cnf will need to start from
+
+  !include include/default_my.cnf
+
+and then modify the configuration as necessary.
+==========================
+
+A suite can have combinations file in the suitedir. It uses my.cnf syntax
+but it cannot use @-substitutions. Instead, it can use $-substitutions for
+the environment variables. Because the combination options will not be
+merged to a my.cnf, but will be added to the command line. Example:
+
+  [conf1]
+  opt1=val1
+
+  [conf2]
+  opt1=val2
+  opt2=$HAVE_SOMETHING
+
+Such a file will cause every test from the suite to be run twice - once
+with mysqld using --opt1=val1 and the other one with mysqld using
+--opt1=val2 --opt2=$HAVE_SOMETHING
+
+One can limit mtr run to a subset of combinations by setting environment
+variable SUITENAME_COMBINATIONS to the ':'-separated set of combination
+names. E.g.
+
+  RPL_COMBINATIONS=mix:row ./mtr --suite rpl
+
+See innodb_plugin suite for an example of how suite.pm may set this variable
+to exclude unsupported configurations.
+==========================
+