MDEV-12263 Feature: skipped test file

A note about unstable-tests lists in mysql-test/README, RedHat
version of it, and also various changes to bring the file
up-to-date

1 files changed, 61 insertions, 41 deletions

diff --git a/mysql-test/README b/mysql-test/README
index 0fba1cc07e3..707820b47ee 100644
--- a/mysql-test/README
+++ b/mysql-test/README
@@ -1,74 +1,94 @@
-This directory contains a test suite for the MySQL daemon. To run
-the currently existing test cases, simply execute ./mysql-test-run in
-this directory. It will fire up the newly built mysqld and test it.
+This directory contains test suites for the MariaDB server. To run
+currently existing test cases, execute ./mysql-test-run in this directory.
 
-Note that you do not have to have to do "make install", and you could
-actually have a co-existing MySQL installation. The tests will not
-conflict with it. To run the test suite in a source directory, you
-must do make first.
+Some tests are known to fail on some platforms or be otherwise unreliable.
+The file "unstable-tests" contains the list of such tests along with
+a comment for every test.
+To exclude them from the test run, execute
+  # ./mysql-test-run --skip-test-list=unstable-tests
 
-All tests must pass. If one or more of them fail on your system, please
-read the following manual section for instructions on how to report the
-problem:
+In general you do not have to have to do "make install", and you can have
+a co-existing MariaDB installation, the tests will not conflict with it.
+To run the tests in a source directory, you must do "make" first.
+
+In Red Hat distributions, you should run the script as user "mysql".
+The user is created with nologin shell, so the best bet is something like
+  # su -
+  # cd /usr/share/mysql-test
+  # su -s /bin/bash mysql -c "./mysql-test-run --skip-test-list=rh-skipped-tests.list"
+
+This will use the installed MariaDB executables, but will run a private copy
+of the server process (using data files within /usr/share/mysql-test),
+so you need not start the mysqld service beforehand.
+
+"rh-skipped-tests.list" is Red Hat version of unstable-tests list, it
+additionally includes tests known to fail specifically on Red Hat builds.
+You can omit it if you want to check whether such failures occur for you.
+
+To clean up afterwards, remove the created "var" subdirectory, e.g.
+  # su -s /bin/bash - mysql -c "rm -rf /usr/share/mysql-test/var"
+
+If one or more tests fail on your system on reasons other than listed
+in lists of unstable tests, please read the following manual section
+for instructions on how to report the problem:
 
 https://mariadb.com/kb/en/reporting-bugs
 
 If you want to use an already running MySQL server for specific tests,
 use the --extern option to mysql-test-run. Please note that in this mode,
-the test suite expects you to provide the names of the tests to run.
+you are expected to provide names of the tests to run.
+
 For example, here is the command to run the "alias" and "analyze" tests
 with an external server:
 
-mysql-test-run --extern socket=/tmp/mysql.sock alias analyze
+  # mysql-test-run --extern socket=/tmp/mysql.sock alias analyze
 
-To match your setup, you might also need to provide --socket, --user, and
-other relevant options.
+To match your setup, you might need to provide other relevant options.
 
-With no test cases named on the command line, mysql-test-run falls back
-to the normal "non-extern" behavior. The reason for this is that some
-tests cannot run with an external server.
+With no test names on the command line, mysql-test-run will attempt
+to execute the default set of tests, which will certainly fail, because
+many tests cannot run with an external server (they need to control the
+options with which the server is started, restart the server during
+execution, etc.)
 
 You can create your own test cases. To create a test case, create a new
 file in the t subdirectory using a text editor. The file should have a .test
 extension. For example:
 
- xemacs t/test_case_name.test
+  # xemacs t/test_case_name.test
 
- In the file, put a set of SQL statements that create some tables,
- load test data, and run some queries to manipulate it.
+In the file, put a set of SQL statements that create some tables,
+load test data, and run some queries to manipulate it.
 
- We would appreciate it if you name your test tables t1, t2, t3 ... (to not
- conflict too much with existing tables).
+Your test should begin by dropping the tables you are going to create and
+end by dropping them again.  This ensures that you can run the test over
+and over again.
 
- Your test should begin by dropping the tables you are going to create and
- end by dropping them again.  This ensures that you can run the test over
- and over again.
- 
- If you are using mysqltest commands (like result file names) in your
- test case, you should create the result file as follows:
+If you are using mysqltest commands in your test case, you should create
+the result file as follows:
 
- mysql-test-run --record test_case_name
+  # mysql-test-run --record test_case_name
 
- or
+  or
 
- mysqltest --record < t/test_case_name.test
+  # mysqltest --record < t/test_case_name.test
 
- If you only have a simple test cases consisting of SQL statements and
- comments, you can create the test case in one of the following ways:
+If you only have a simple test case consisting of SQL statements and
+comments, you can create the result file in one of the following ways:
 
- mysql-test-run --record test_case_name
+  # mysql-test-run --record test_case_name
 
- mysql test < t/test_case_name.test > r/test_case_name.result
+  # mysql test < t/test_case_name.test > r/test_case_name.result
 
- mysqltest --record --database test --result-file=r/test_case_name.result < t/test_case_name.test
+  # mysqltest --record --database test --result-file=r/test_case_name.result < t/test_case_name.test
 
- When this is done, take a look at r/test_case_name.result
- - If the result is incorrect, you have found a bug. In this case, you should
-   edit the test result to the correct results so that we can verify
-   that the bug is corrected in future releases.
+When this is done, take a look at r/test_case_name.result .
+If the result is incorrect, you have found a bug. In this case, you should
+edit the test result to the correct results so that we can verify that
+the bug is corrected in future releases.
 
 If you want to submit your test case you can send it 
-to maria-developers@lists.launchpad.com or attach it to a bug report on
+to maria-developers@lists.launchpad.net or attach it to a bug report on
 http://mariadb.org/jira/.
 
 If the test case is really big or if it contains 'not public' data,