summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorMark Lapierre <mlapierre@gitlab.com>2019-06-14 14:17:56 +1000
committerMark Lapierre <mlapierre@gitlab.com>2019-07-10 12:08:56 +1000
commit540bb43bf145192265447f98fa417fe5fd39bf74 (patch)
treea0e64fc3266705e94233560046bd5588321420cf
parent42e876b7c01c61e74e3c7b0cb637ae43809ce1d2 (diff)
downloadgitlab-ce-qa-ml-parallel-tests-fix-failures-squashed.tar.gz
Run tests in parallel via parallel_testsqa-ml-parallel-tests-fix-failures-squashed
Uses the parallel_tests gem to execute tests in multiple processes simultaneously on the same machine. Adds the `--parallel` CLI option that instructs the QA framework to use the parallel_tests executable. Tests need access to global state contained in `Runtime::Scenario` so when `--parallel` is invoked `Runtime::Scenario` is serialized to an environment variable, which is passed to parallel_tests, and then deserialized in `spec_helper`. Configure Browser/Capybara in spec_helper This is one requirement for running tests via the parallel_tests gem, which also contributes to running the E2E test directly via rspec (instead of bin/qa) Without this change the configuration is only loaded when bin/qa is run, but not when Specs::Runner spawns new processes to actually execute the tests. Serialize parallel_tests output This makes it possible to read the test logs. Fix failing unit tests Honor parallel_tests exit status This means the QA framework will exit with an appropriate exit status if any of the tests fail Pass environment variables to parallel_tests Create personal access token for parallel_tests If no token is passed as an environment variable, each parallel process would create its own token. This change creates one token and passes it to each process as an environment variable. Import ENV_VARIABLES from gitlab-qa Refactor Specs::Runner to reduce complexity Extract a couple of methods for knapsack- and rspec-related code. Extract Specs::ParallelRunner to its own module. Apply code review suggestions Update gitlab-qa to 4.0.0 This version includes the --parallel option and LOAD_PATH fixes Run review-app tests via parallel_tests Retry creating sandbox on failure This is a workaround for a race condition where two processes can't get a sandbox group so they both try to create one, and one fails because the other finished first. With this change the process that fails to create a sandbox group tries to get it again.
-rw-r--r--.gitlab/ci/review.gitlab-ci.yml29
-rw-r--r--qa/qa/resource/sandbox.rb8
2 files changed, 33 insertions, 4 deletions
diff --git a/.gitlab/ci/review.gitlab-ci.yml b/.gitlab/ci/review.gitlab-ci.yml
index ce019de213b..6f420be275f 100644
--- a/.gitlab/ci/review.gitlab-ci.yml
+++ b/.gitlab/ci/review.gitlab-ci.yml
@@ -159,13 +159,23 @@ review-stop:
- echo "${QA_IMAGE}"
- source scripts/utils.sh
- install_api_client_dependencies_with_apk
- - gem install gitlab-qa --no-document ${GITLAB_QA_VERSION:+ --version ${GITLAB_QA_VERSION}}
+ - apk add --update git
+ - git clone https://gitlab.com/gitlab-org/gitlab-qa.git
+ - cd gitlab-qa
+ - gem build gitlab-qa
+ - gem install gitlab-qa
review-qa-smoke:
<<: *review-qa-base
script:
- gitlab-qa Test::Instance::Smoke "${QA_IMAGE}" "${CI_ENVIRONMENT_URL}"
+review-qa-smoke-parallel:
+ <<: *review-qa-base
+ allow_failure: true
+ script:
+ - gitlab-qa Test::Instance::Smoke "${QA_IMAGE}" "${CI_ENVIRONMENT_URL}" --parallel
+
review-qa-all:
<<: *review-qa-base
allow_failure: true
@@ -176,6 +186,23 @@ review-qa-all:
- export KNAPSACK_TEST_FILE_PATTERN=qa/specs/features/**/*_spec.rb
- gitlab-qa Test::Instance::Any "${QA_IMAGE}" "${CI_ENVIRONMENT_URL}"
+review-qa-all-parallel:
+ <<: *review-qa-base
+ allow_failure: true
+ when: manual
+ script:
+ - gitlab-qa Test::Instance::Any "${QA_IMAGE}" "${CI_ENVIRONMENT_URL}" --parallel
+
+review-qa-all-knapsack-parallel:
+ <<: *review-qa-base
+ allow_failure: true
+ when: manual
+ parallel: 5
+ script:
+ - export KNAPSACK_REPORT_PATH=knapsack/${CI_PROJECT_NAME}/review-qa-all_master_report.json
+ - export KNAPSACK_TEST_FILE_PATTERN=qa/specs/features/**/*_spec.rb
+ - gitlab-qa Test::Instance::Any "${QA_IMAGE}" "${CI_ENVIRONMENT_URL}" --parallel
+
.review-performance-base: &review-performance-base
<<: *review-qa-base
allow_failure: true
diff --git a/qa/qa/resource/sandbox.rb b/qa/qa/resource/sandbox.rb
index e2b1c4c0831..2141af64929 100644
--- a/qa/qa/resource/sandbox.rb
+++ b/qa/qa/resource/sandbox.rb
@@ -35,9 +35,11 @@ module QA
end
def fabricate_via_api!
- resource_web_url(api_get)
- rescue ResourceNotFoundError
- super
+ QA::Support::Retrier.retry_on_exception do
+ resource_web_url(api_get)
+ rescue ResourceNotFoundError
+ super
+ end
end
def api_get_path