summaryrefslogtreecommitdiff
path: root/doc/development/testing_guide/best_practices.md
diff options
context:
space:
mode:
authorGitLab Bot <gitlab-bot@gitlab.com>2020-05-20 14:34:42 +0000
committerGitLab Bot <gitlab-bot@gitlab.com>2020-05-20 14:34:42 +0000
commit9f46488805e86b1bc341ea1620b866016c2ce5ed (patch)
treef9748c7e287041e37d6da49e0a29c9511dc34768 /doc/development/testing_guide/best_practices.md
parentdfc92d081ea0332d69c8aca2f0e745cb48ae5e6d (diff)
downloadgitlab-ce-9f46488805e86b1bc341ea1620b866016c2ce5ed.tar.gz
Add latest changes from gitlab-org/gitlab@13-0-stable-ee
Diffstat (limited to 'doc/development/testing_guide/best_practices.md')
-rw-r--r--doc/development/testing_guide/best_practices.md60
1 files changed, 40 insertions, 20 deletions
diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md
index 62767180077..f0137e542cc 100644
--- a/doc/development/testing_guide/best_practices.md
+++ b/doc/development/testing_guide/best_practices.md
@@ -17,24 +17,22 @@ our test design. We can find some helpful heuristics documented in the Handbook
## Test speed
-GitLab has a massive test suite that, without [parallelization], can take hours
+GitLab has a massive test suite that, without [parallelization](ci.md#test-suite-parallelization-on-the-ci), can take hours
to run. It's important that we make an effort to write tests that are accurate
and effective _as well as_ fast.
Here are some things to keep in mind regarding test performance:
-- `double` and `spy` are faster than `FactoryBot.build(...)`
+- `instance_double` and `spy` are faster than `FactoryBot.build(...)`
- `FactoryBot.build(...)` and `.build_stubbed` are faster than `.create`.
- Don't `create` an object when `build`, `build_stubbed`, `attributes_for`,
- `spy`, or `double` will do. Database persistence is slow!
+ `spy`, or `instance_double` will do. Database persistence is slow!
- Don't mark a feature as requiring JavaScript (through `:js` in RSpec) unless it's _actually_ required for the test
to be valid. Headless browser testing is slow!
-[parallelization]: ci.md#test-suite-parallelization-on-the-ci
-
## RSpec
-To run rspec tests:
+To run RSpec tests:
```shell
# run all tests
@@ -71,6 +69,8 @@ FDOC=1 bin/rspec spec/[path]/[to]/[spec].rb
- Use `Gitlab.config.gitlab.host` rather than hard coding `'localhost'`
- Don't assert against the absolute value of a sequence-generated attribute (see
[Gotchas](../gotchas.md#do-not-assert-against-the-absolute-value-of-a-sequence-generated-attribute)).
+- Avoid using `expect_any_instance_of` or `allow_any_instance_of` (see
+ [Gotchas](../gotchas.md#do-not-assert-against-the-absolute-value-of-a-sequence-generated-attribute)).
- Don't supply the `:each` argument to hooks since it's the default.
- On `before` and `after` hooks, prefer it scoped to `:context` over `:all`
- When using `evaluate_script("$('.js-foo').testSomething()")` (or `execute_script`) which acts on a given element,
@@ -320,26 +320,48 @@ stub_feature_flags(ci_live_trace: false)
Feature.enabled?(:ci_live_trace) # => false
```
-If you wish to set up a test where a feature flag is disabled for some
-actors and not others, you can specify this in options passed to the
-helper. For example, to disable the `ci_live_trace` feature flag for a
-specifc project:
+If you wish to set up a test where a feature flag is enabled only
+for some actors and not others, you can specify this in options
+passed to the helper. For example, to enable the `ci_live_trace`
+feature flag for a specifc project:
```ruby
project1, project2 = build_list(:project, 2)
-# Feature will only be disabled for project1
-stub_feature_flags(ci_live_trace: { enabled: false, thing: project1 })
+# Feature will only be enabled for project1
+stub_feature_flags(ci_live_trace: project1)
+
+Feature.enabled?(:ci_live_trace) # => false
+Feature.enabled?(:ci_live_trace, project1) # => true
+Feature.enabled?(:ci_live_trace, project2) # => false
+```
+
+This represents an actual behavior of FlipperGate:
-Feature.enabled?(:ci_live_trace, project1) # => false
-Feature.enabled?(:ci_live_trace, project2) # => true
+1. You can enable an override for a specified actor to be enabled
+1. You can disable (remove) an override for a specified actor,
+ fallbacking to default state
+1. There's no way to model that you explicitly disable a specified actor
+
+```ruby
+Feature.enable(:my_feature)
+Feature.disable(:my_feature, project1)
+Feature.enabled?(:my_feature) # => true
+Feature.enabled?(:my_feature, project1) # => true
+```
+
+```ruby
+Feature.disable(:my_feature2)
+Feature.enable(:my_feature2, project1)
+Feature.enabled?(:my_feature2) # => false
+Feature.enabled?(:my_feature2, project1) # => true
```
### Pristine test environments
The code exercised by a single GitLab test may access and modify many items of
data. Without careful preparation before a test runs, and cleanup afterward,
-data can be changed by a test in such a way that it affects the behaviour of
+data can be changed by a test in such a way that it affects the behavior of
following tests. This should be avoided at all costs! Fortunately, the existing
test framework handles most cases already.
@@ -493,7 +515,7 @@ range of inputs. By specifying the test case once, alongside a table of inputs
and the expected output for each, your tests can be made easier to read and more
compact.
-We use the [rspec-parameterized](https://github.com/tomykaira/rspec-parameterized)
+We use the [RSpec::Parameterized](https://github.com/tomykaira/rspec-parameterized)
gem. A short example, using the table syntax and checking Ruby equality for a
range of inputs, might look like this:
@@ -526,7 +548,7 @@ objects, FactoryBot-created objects etc. can lead to
### Prometheus tests
Prometheus metrics may be preserved from one test run to another. To ensure that metrics are
-reset before each example, add the `:prometheus` tag to the Rspec test.
+reset before each example, add the `:prometheus` tag to the RSpec test.
### Matchers
@@ -651,7 +673,7 @@ end
### Factories
-GitLab uses [factory_bot] as a test fixture replacement.
+GitLab uses [factory_bot](https://github.com/thoughtbot/factory_bot) as a test fixture replacement.
- Factory definitions live in `spec/factories/`, named using the pluralization
of their corresponding model (`User` factories are defined in `users.rb`).
@@ -666,8 +688,6 @@ GitLab uses [factory_bot] as a test fixture replacement.
- Factories don't have to be limited to `ActiveRecord` objects.
[See example](https://gitlab.com/gitlab-org/gitlab-foss/commit/0b8cefd3b2385a21cfed779bd659978c0402766d).
-[factory_bot]: https://github.com/thoughtbot/factory_bot
-
### Fixtures
All fixtures should be placed under `spec/fixtures/`.