diff options
| author | Yuchen Pei <yuchen.pei@mariadb.com> | 2023-01-25 13:11:47 +1100 |
|---|---|---|
| committer | Yuchen Pei <yuchen.pei@mariadb.com> | 2023-01-25 13:11:47 +1100 |
| commit | a22791ab96d56b792ea658c443d9be60e81e485b (patch) | |
| tree | 3bbba8a7cfdc5f4ade1e4b944bd659ae2f182586 | |
| parent | 2279dddad6bea00c5774a76f410c11fd56ad168f (diff) | |
| download | mariadb-git-bb-10.3-mdev-30464.tar.gz | |
MDEV-30464 Adding a testsuite spider/unfixed with a readme.bb-10.3-mdev-30464
| -rw-r--r-- | storage/spider/mysql-test/spider/unfixed/README.org | 35 |
1 files changed, 35 insertions, 0 deletions
diff --git a/storage/spider/mysql-test/spider/unfixed/README.org b/storage/spider/mysql-test/spider/unfixed/README.org new file mode 100644 index 00000000000..93e996f14c5 --- /dev/null +++ b/storage/spider/mysql-test/spider/unfixed/README.org @@ -0,0 +1,35 @@ +* Unfixed spider tests on known bugs + +Tests under this directory are used to document known bugs. They +should pass on expected /failure/, thus if they fail it could mean +good news :) Ideally every reported bug should start with a test here, +and when the bug is fixed, the test should be updated with its logic +reversed (passing on expected /behaviour/) and moved to the bugfix +suite. However, if we actively work a bug to begin with, we can still +follow the old workflow of starting with a test in bugfix. Possible +scenarios: + +1. A bug is reported, and is worked on. We create a testcase under the + bugfix suite as usual, that should pass on expected behaviour. We + work on it, fix the bug, and push. In this case there's no need to + create a test in the unfixed suite. + +2. A bug is reported, but we do not have time to work on it yet. We + create a testcase for this bug in the unfixed suite, that should + pass on expected failure. We push the testcase. + + While working on other bugs we continue running tests on the + unfixed suite from time to time. + + 1. They all pass, nothing needs to be done + + 2. Some tests fail. Check whether the failure means the bugs are + fixed, and update the issues associated with the bugs. These + actions should not block the work on whatever bug we are + actually working on. Because of this, failures in this suite + should not block moving forward in development including + pushing. + +3. We finally have time to work on a bug with a testcase in the + unfixed suite. This is similar to 1: we reverse the test logic, + move it to bugfix, fix the bug, push. |