Allow to quarantine tests

Signed-off-by: Rémy Coutable <remy@rymai.me>

2 files changed, 17 insertions, 0 deletions

diff --git a/doc/development/testing_guide/flaky_tests.md b/doc/development/testing_guide/flaky_tests.md
index bbb2313ea7b..f6bc9fb0979 100644
--- a/doc/development/testing_guide/flaky_tests.md
+++ b/doc/development/testing_guide/flaky_tests.md
@@ -5,6 +5,18 @@
 It's a test that sometimes fails, but if you retry it enough times, it passes,
 eventually.
 
+## Quarantined tests
+
+Tests can be put in quarantine by assigning `:quarantine` metadata. This means
+they will be skipped unless run with `--tag quarantine`. This can be used for
+tests that are expected to fail while a fix is in progress (similar to how
+[`skip` or `pending`](https://relishapp.com/rspec/rspec-core/v/3-8/docs/pending-and-skipped-examples)
+ can be used).
+
+```
+bin/rspec --tag quarantine
+```
+
 ## Automatic retries and flaky tests detection
 
 On our CI, we use [rspec-retry] to automatically retry a failing example a few
diff --git a/spec/spec_helper.rb b/spec/spec_helper.rb
index ca977effcb6..dfdb23492bf 100644
--- a/spec/spec_helper.rb
+++ b/spec/spec_helper.rb
@@ -132,6 +132,11 @@ RSpec.configure do |config|
     Gitlab::ReleaseBlogPost.instance.instance_variable_set(:@url, 'https://about.gitlab.com')
   end
 
+  config.before(:example, :quarantine) do
+    # Skip tests in quarantine unless we explicitly focus on them.
+    skip('In quarantine') unless config.inclusion_filter[:quarantine]
+  end
+
   config.before(:example, :request_store) do
     RequestStore.begin!
   end