diff options
Diffstat (limited to 'doc/development/testing_guide/end_to_end/resources.md')
| -rw-r--r-- | doc/development/testing_guide/end_to_end/resources.md | 180 |
1 files changed, 1 insertions, 179 deletions
diff --git a/doc/development/testing_guide/end_to_end/resources.md b/doc/development/testing_guide/end_to_end/resources.md index 6e29e9b9dff..37d354bf60c 100644 --- a/doc/development/testing_guide/end_to_end/resources.md +++ b/doc/development/testing_guide/end_to_end/resources.md @@ -8,17 +8,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w Resources are primarily created using Browser UI steps, but can also be created via the API or the CLI. -A typical resource class is used to create a new resource that can be used in a single test. However, several tests can -end up creating the same kind of resource and use it in ways that mean it could have been -used by more than one test. Creating a new resource each time is not efficient. Therefore, we can also create reusable -resources that are created once and can then be used by many tests. - -In the following section the content focuses on single-use resources, however it also applies to reusable resources. -Information specific to [reusable resources is detailed below](#reusable-resources). - ## How to properly implement a resource class? -All non-reusable resource classes should inherit from `Resource::Base`. +All resource classes should inherit from `Resource::Base`. There is only one mandatory method to implement to define a resource class. This is the `#fabricate!` method, which is used to build the resource via the @@ -398,176 +390,6 @@ end In this case, the result is similar to calling `Resource::Shirt.fabricate!`. -## Reusable resources - -Reusable resources are created by the first test that needs a particular kind of resource, and then any test that needs -the same kind of resource can reuse it instead of creating a new one. - -The `ReusableProject` resource is an example of this class: - -```ruby -module QA - module Resource - class ReusableProject < Project # A reusable resource inherits from the resource class that we want to be able to reuse. - prepend Reusable # The Reusable module mixes in some methods that help implement reuse. - - def initialize - super # A ReusableProject is a Project so it should be initialized as one. - - # Some Project attributes aren't valid and need to be overridden. For example, a ReusableProject keeps its name once it's created, - # so we don't add a random string to the name specified. - @add_name_uuid = false - - # It has a default name, and a different name can be specified when a resource is first created. However, the same name must be - # provided any time that instance of the resource is used. - @name = "reusable_project" - - # Several instances of a ReusableProject can exists as long as each is identified via a unique value for `reuse_as`. - @reuse_as = :default_project - end - - # All reusable resource classes must validate that an instance meets the conditions that allow reuse. For example, - # by confirming that the name specified for the instance is valid and doesn't conflict with other instances. - def validate_reuse_preconditions - raise ResourceReuseError unless reused_name_valid? - end - - # Internally we identify an instance of a reusable resource by a unique value of `@reuse_as`, but in GitLab the - # resource has one or more attributes that must also be unique. This method lists those attributes and allows the - # test framework to check that each instance of a reusable resource has values that match the associated values - # in Gitlab. - def unique_identifiers - [:name, :path] - end - end - end -end -``` - -Reusable resources aren't removed immediately when `remove_via_api!` is called. Instead, they're removed after the test -suite completes. To do so each class must be registered with `QA::Resource::ReusableCollection` in `qa/spec/spec_helper.rb` -as in the example below. Registration allows `QA::Resource::ReusableCollection` to keep track of each instance of each -registered class, and to delete them all in the `config.after(:suite)` hook. - -```ruby -config.before(:suite) do |suite| - QA::Resource::ReusableCollection.register_resource_classes do |collection| - QA::Resource::ReusableProject.register(collection) - end -end -``` - -Consider some examples of how a reusable resource is used: - -```ruby -# This will create a project. -default_project = Resource::ReusableProject.fabricate_via_api! -default_project.name # => "reusable_project" -default_project.reuse_as # => :default_project -``` - -Then in another test we could reuse the project: - -```ruby -# This will fetch the project created above rather than creating a new one. -default_project_again = Resource::ReusableProject.fabricate_via_api! -default_project_again.name # => "reusable_project" -default_project_again.reuse_as # => :default_project -``` - -We can also create another project that we want to change in a way that might not be suitable for tests using the -default project: - -```ruby -project_with_member = Resource::ReusableProject.fabricate_via_api! do |project| - project.name = "project-with-member" - project.reuse_as = :project_with_member -end - -project_with_member.add_member(user) -``` - -Another test can reuse that project: - -```ruby -project_still_has_member = Resource::ReusableProject.fabricate_via_api! do |project| - project.name = "project-with-member" - project.reuse_as = :project_with_member -end - -expect(project_still_has_member).to have_member(user) -``` - -However, if we don't provide the name again an error will be raised: - -```ruby -Resource::ReusableProject.fabricate_via_api! do |project| - project.reuse_as = :project_with_member -end - -# => ResourceReuseError will be raised because it will try to use the default name, "reusable_project", which doesn't -# match the name specified when the project was first fabricated. -``` - -### Validating reusable resources - -Reusable resources can speed up test suites by avoiding the cost of creating the same resource again and again. However, -that can cause problems if a test makes changes to a resource that prevent it from being reused as expected by later -tests. That can lead to order-dependent test failures that can be difficult to troubleshoot. - -For example, the default project created by `QA::Resource::ReusableProject` has `auto_devops_enabled` set to `false` -(inherited from `QA::Resource::Project`). If a test reuses that project and enables Auto DevOps, subsequent tests that reuse -the project will fail if they expect Auto DevOps to be disabled. - -We try to avoid that kind of trouble by validating reusable resources after a test suite. If the environment variable -`QA_VALIDATE_RESOURCE_REUSE` is set to `true` the test framework will check each reusable resource to verify that none -of the attributes they were created with have been changed. It does that by creating a new resource using the same -attributes that were used to create the original resource. It then compares the new resource to the original and raises -an error if any attributes don't match. - -#### Implementation - -When you implement a new type of reusable resource there are two `private` methods you must implement so the resource -can be validated. They are: - -- `reference_resource`: creates a new instance of the resource that can be compared with the one that was used during the tests. -- `unique_identifiers`: returns an array of attributes that allow the resource to be identified (for example, name) and that are therefore -expected to differ when comparing the reference resource with the resource reused in the tests. - -The following example shows the implementation of those two methods in `QA::Resource::ReusableProject`. - -```ruby -# Creates a new project that can be compared to a reused project, using the attributes of the original. -# -# @return [QA::Resource] a new instance of Resource::ReusableProject that should be a copy of the original resource -def reference_resource - # These are the attributes that the reused resource was created with - attributes = self.class.resources[reuse_as][:attributes] - - # Two projects can't have the same path, and since we typically use the same value for the name and path, we assign - # a unique name and path to the reference resource. - name = "reference_resource_#{SecureRandom.hex(8)}_for_#{attributes.delete(:name)}" - - Project.fabricate_via_api! do |project| - self.class.resources[reuse_as][:attributes].each do |attribute_name, attribute_value| - project.instance_variable_set("@#{attribute_name}", attribute_value) if attribute_value - end - project.name = name - project.path = name - project.path_with_namespace = "#{project.group.full_path}/#{project.name}" - end -end - -# The attributes of the resource that should be the same whenever a test wants to reuse a project. -# -# @return [Array<Symbol>] the attribute names. -def unique_identifiers - # As noted above, path must be unique, and since we typically use the same value for both, we treat name and path - # as unique. These attributes are ignored when we compare the reference and reused resources. - [:name, :path] -end -``` - ### Resources cleanup We have a mechanism to [collect](https://gitlab.com/gitlab-org/gitlab/-/blob/44345381e89d6bbd440f7b4c680d03e8b75b86de/qa/qa/tools/test_resource_data_processor.rb#L32) |