diff options
Diffstat (limited to 'doc/ci')
| -rw-r--r-- | doc/ci/docker/using_docker_build.md | 24 | ||||
| -rw-r--r-- | doc/ci/examples/README.md | 37 | ||||
| -rw-r--r-- | doc/ci/examples/authenticating-with-hashicorp-vault/img/vault-read-secrets-production.png | bin | 0 -> 326852 bytes | |||
| -rw-r--r-- | doc/ci/examples/authenticating-with-hashicorp-vault/img/vault-read-secrets-staging.png | bin | 0 -> 400945 bytes | |||
| -rw-r--r-- | doc/ci/examples/authenticating-with-hashicorp-vault/index.md | 215 | ||||
| -rw-r--r-- | doc/ci/junit_test_reports.md | 2 | ||||
| -rw-r--r-- | doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_immediate_merge_v12_6.png | bin | 63888 -> 18607 bytes | |||
| -rw-r--r-- | doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md | 21 | ||||
| -rw-r--r-- | doc/ci/pipelines/img/job_group_v12_10.png | bin | 21609 -> 5436 bytes | |||
| -rw-r--r-- | doc/ci/pipelines/settings.md | 2 | ||||
| -rw-r--r-- | doc/ci/review_apps/index.md | 16 | ||||
| -rw-r--r-- | doc/ci/variables/README.md | 78 | ||||
| -rw-r--r-- | doc/ci/variables/predefined_variables.md | 1 | ||||
| -rw-r--r-- | doc/ci/yaml/README.md | 14 |
14 files changed, 362 insertions, 48 deletions
diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 0ae7a10a290..427f61deb29 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -118,7 +118,7 @@ not without its own challenges: instance of Docker engine so they won't conflict with each other. But this also means that jobs can be slower because there's no caching of layers. - By default, Docker 17.09 and higher uses `--storage-driver overlay2` which is - the recommended storage driver. See [Using the overlayfs driver](#using-the-overlayfs-driver) + the recommended storage driver. See [Using the overlayfs driver](#use-the-overlayfs-driver) for details. - Since the `docker:19.03.8-dind` container and the Runner container don't share their root filesystem, the job's working directory can be used as a mount point for @@ -448,7 +448,7 @@ The steps in the `script` section for the `build` stage can be summed up to: 1. The last two commands push the tagged Docker images to the container registry so that they may also be used as cache for subsequent builds. -## Using the OverlayFS driver +## Use the OverlayFS driver NOTE: **Note:** The shared Runners on GitLab.com use the `overlay2` driver by default. @@ -480,18 +480,22 @@ which can be avoided if a different driver is used, for example `overlay2`. overlay ``` -### Use driver per project +### Use the OverlayFS driver per project -You can enable the driver for each project individually by editing the project's `.gitlab-ci.yml`: +You can enable the driver for each project individually by using the `DOCKER_DRIVER` +environment [variable](../yaml/README.md#variables) in `.gitlab-ci.yml`: ```yaml variables: DOCKER_DRIVER: overlay2 ``` -### Use driver for every project +### Use the OverlayFS driver for every project -To enable the driver for every project, you can set the environment variable for every build by adding `environment` in the `[[runners]]` section of `config.toml`: +If you use your own [GitLab Runners](https://docs.gitlab.com/runner/), you +can enable the driver for every project by setting the `DOCKER_DRIVER` +environment variable in the +[`[[runners]]` section of `config.toml`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section): ```toml environment = ["DOCKER_DRIVER=overlay2"] @@ -499,11 +503,9 @@ environment = ["DOCKER_DRIVER=overlay2"] If you're running multiple Runners you will have to modify all configuration files. -> **Notes:** -> -> - More information about the Runner configuration is available in the [Runner documentation](https://docs.gitlab.com/runner/configuration/). -> - For more information about using OverlayFS with Docker, you can read -> [Use the OverlayFS storage driver](https://docs.docker.com/engine/userguide/storagedriver/overlayfs-driver/). +NOTE: **Note:** +Read more about the [Runner configuration](https://docs.gitlab.com/runner/configuration/) +and [using the OverlayFS storage driver](https://docs.docker.com/engine/userguide/storagedriver/overlayfs-driver/). ## Using the GitLab Container Registry diff --git a/doc/ci/examples/README.md b/doc/ci/examples/README.md index 64367727371..1b3c4c887f4 100644 --- a/doc/ci/examples/README.md +++ b/doc/ci/examples/README.md @@ -20,24 +20,25 @@ Examples are available in several forms. As a collection of: The following table lists examples with step-by-step tutorials that are contained in this section. -| Use case | Resource | -|:----------------------------|:---------------------------------------------------------------------------------------------------------------------------| -| Browser performance testing | [Browser Performance Testing with the Sitespeed.io container](../../user/project/merge_requests/browser_performance_testing.md). | -| Clojure | [Test a Clojure application with GitLab CI/CD](test-clojure-application.md). | -| Deployment with Dpl | [Using `dpl` as deployment tool](deployment/README.md). | -| Elixir | [Testing a Phoenix application with GitLab CI/CD](test_phoenix_app_with_gitlab_ci_cd/index.md). | -| End-to-end testing | [End-to-end testing with GitLab CI/CD and WebdriverIO](end_to_end_testing_webdriverio/index.md). | -| Game development | [DevOps and Game Dev with GitLab CI/CD](devops_and_game_dev_with_gitlab_ci_cd/index.md). | -| GitLab Pages | See the [GitLab Pages](../../user/project/pages/index.md) documentation for a complete example of deploying a static site. | -| Java with Spring Boot | [Deploy a Spring Boot application to Cloud Foundry with GitLab CI/CD](deploy_spring_boot_to_cloud_foundry/index.md). | -| Java with Maven | [How to deploy Maven projects to Artifactory with GitLab CI/CD](artifactory_and_gitlab/index.md). | -| PHP with PHPunit, atoum | [Testing PHP projects](php.md). | -| PHP with NPM, SCP | [Running Composer and NPM scripts with deployment via SCP in GitLab CI/CD](deployment/composer-npm-deploy.md). | -| PHP with Laravel, Envoy | [Test and deploy Laravel applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md). | -| Python on Heroku | [Test and deploy a Python application with GitLab CI/CD](test-and-deploy-python-application-to-heroku.md). | -| Ruby on Heroku | [Test and deploy a Ruby application with GitLab CI/CD](test-and-deploy-ruby-application-to-heroku.md). | -| Scala on Heroku | [Test and deploy a Scala application to Heroku](test-scala-application.md). | -| Parallel testing Ruby & JS | [GitLab CI/CD parallel jobs testing for Ruby & JavaScript projects](https://docs.knapsackpro.com/2019/how-to-run-parallel-jobs-for-rspec-tests-on-gitlab-ci-pipeline-and-speed-up-ruby-javascript-testing). | +| Use case | Resource | +|:------------------------------|:---------------------------------------------------------------------------------------------------------------------------| +| Browser performance testing | [Browser Performance Testing with the Sitespeed.io container](../../user/project/merge_requests/browser_performance_testing.md). | +| Clojure | [Test a Clojure application with GitLab CI/CD](test-clojure-application.md). | +| Deployment with Dpl | [Using `dpl` as deployment tool](deployment/README.md). | +| Elixir | [Testing a Phoenix application with GitLab CI/CD](test_phoenix_app_with_gitlab_ci_cd/index.md). | +| End-to-end testing | [End-to-end testing with GitLab CI/CD and WebdriverIO](end_to_end_testing_webdriverio/index.md). | +| Game development | [DevOps and Game Dev with GitLab CI/CD](devops_and_game_dev_with_gitlab_ci_cd/index.md). | +| GitLab Pages | See the [GitLab Pages](../../user/project/pages/index.md) documentation for a complete example of deploying a static site. | +| Java with Spring Boot | [Deploy a Spring Boot application to Cloud Foundry with GitLab CI/CD](deploy_spring_boot_to_cloud_foundry/index.md). | +| Java with Maven | [How to deploy Maven projects to Artifactory with GitLab CI/CD](artifactory_and_gitlab/index.md). | +| PHP with PHPunit, atoum | [Testing PHP projects](php.md). | +| PHP with NPM, SCP | [Running Composer and NPM scripts with deployment via SCP in GitLab CI/CD](deployment/composer-npm-deploy.md). | +| PHP with Laravel, Envoy | [Test and deploy Laravel applications with GitLab CI/CD and Envoy](laravel_with_gitlab_and_envoy/index.md). | +| Python on Heroku | [Test and deploy a Python application with GitLab CI/CD](test-and-deploy-python-application-to-heroku.md). | +| Ruby on Heroku | [Test and deploy a Ruby application with GitLab CI/CD](test-and-deploy-ruby-application-to-heroku.md). | +| Scala on Heroku | [Test and deploy a Scala application to Heroku](test-scala-application.md). | +| Parallel testing Ruby & JS | [GitLab CI/CD parallel jobs testing for Ruby & JavaScript projects](https://docs.knapsackpro.com/2019/how-to-run-parallel-jobs-for-rspec-tests-on-gitlab-ci-pipeline-and-speed-up-ruby-javascript-testing). | +| Secrets management with Vault | [Authenticating and Reading Secrets With Hashicorp Vault](authenticating-with-hashicorp-vault/index.md). | ### Contributing examples diff --git a/doc/ci/examples/authenticating-with-hashicorp-vault/img/vault-read-secrets-production.png b/doc/ci/examples/authenticating-with-hashicorp-vault/img/vault-read-secrets-production.png Binary files differnew file mode 100644 index 00000000000..65c8546deb2 --- /dev/null +++ b/doc/ci/examples/authenticating-with-hashicorp-vault/img/vault-read-secrets-production.png diff --git a/doc/ci/examples/authenticating-with-hashicorp-vault/img/vault-read-secrets-staging.png b/doc/ci/examples/authenticating-with-hashicorp-vault/img/vault-read-secrets-staging.png Binary files differnew file mode 100644 index 00000000000..2399d8f8879 --- /dev/null +++ b/doc/ci/examples/authenticating-with-hashicorp-vault/img/vault-read-secrets-staging.png diff --git a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md new file mode 100644 index 00000000000..2986afe8284 --- /dev/null +++ b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md @@ -0,0 +1,215 @@ +--- +type: tutorial +--- + +# Authenticating and Reading Secrets With Hashicorp Vault + +This tutorial demonstrates how to authenticate, configure, and read secrets with HashiCorp's Vault from GitLab CI/CD. + +## Requirements + +This tutorial assumes you are familiar with GitLab CI/CD and Vault. + +To follow along, you will need: + +- An account on GitLab. +- A running Vault server and the access required to configure authentication and create roles and policies. + +NOTE: **Note:** +You will need to replace the `vault.example.com` URL below with the URL of your Vault server and `gitlab.example.com` with the URL of your GitLab instance. + +## How it works + +Each job has JSON Web Token (JWT) provided as environment variable named `CI_JOB_JWT`. This JWT can be used to authenticate with Vault using the [JWT Auth](https://www.vaultproject.io/docs/auth/jwt/#jwt-authentication) method. + +The JWT's payload looks like this: + +```json +{ + "jti": "c82eeb0c-5c6f-4a33-abf5-4c474b92b558", # Unique identifier for this token + "iss": "gitlab.example.com", # Issuer, the domain of your GitLab instance + "iat": 1585710286, # Issued at + "nbf": 1585798372, # Not valid before + "exp": 1585713886, # Expire at + "sub": "job_1212", # Subject (job id) + "namespace_id": "1", + "namespace_path": "mygroup", + "project_id": "22", + "project_path": "mygroup/myproject", + "user_id": "42", + "user_login": "myuser", + "user_email": "myuser@example.com" + "pipeline_id": "1212", + "job_id": "1212", + "ref": "auto-deploy-2020-04-01", # Git ref for this job + "ref_type": "branch", # Git ref type, branch or tag + "ref_protected": "true" # true if this git ref is protected, false otherwise +} +``` + +The JWT is encoded by using RS256 and signed with your GitLab instance's OpenID Connect private key. The expire time for the token will be set to job's timeout, if specifed, or 5 minutes if it is not. The key used to sign this token may change without any notice. In such case retrying the job will generate new JWT using the current signing key. + +You can use this JWT and your instance's JWKS endpoint (`https://gitlab.example.com/-/jwks`) to authenticate with a Vault server that is configured to allow the JWT Authentication method for authentication. + +When configuring roles in Vault, you can use [bound_claims](https://www.vaultproject.io/docs/auth/jwt/#bound-claims) to match against the JWT's claims and restrict which secrets each CI job has access to. + +To communicate with Vault, you can use either its CLI client or perform API requests (using `curl` or another client). + +## Example + +CAUTION: **Caution**: +JWTs are credentials, which can grant access to resources. Be careful where you paste them! + +Let's say you have the passwords for your staging and production databases stored in a Vault server that is running on `http://vault.example.com:8200`. Your staging password is `pa$$w0rd` and your production password is `real-pa$$w0rd`. + +```shell +$ vault kv get -field=password secret/myproject/staging/db +pa$$w0rd + +$ vault kv get -field=password secret/myproject/production/db +real-pa$$w0rd +``` + +To configure your Vault server, start by enabling the [JWT Auth](https://www.vaultproject.io/docs/auth/jwt/) method: + +```shell +$ vault auth enable jwt +Success! Enabled jwt auth method at: jwt/ +``` + +Then create policies that allow you to read these secrets (one for each secret): + +```shell +$ vault policy write myproject-staging - <<EOF +# Policy name: myproject-staging +# +# Read-only permission on 'secret/data/myproject/staging/*' path +path "secret/data/myproject/staging/*" { + capabilities = [ "read" ] +} +EOF +Success! Uploaded policy: myproject-staging + +$ vault policy write myproject-production - <<EOF +# Policy name: myproject-production +# +# Read-only permission on 'secret/data/myproject/production/*' path +path "secret/data/myproject/production/*" { + capabilities = [ "read" ] +} +EOF +Success! Uploaded policy: myproject-production +``` + +You'll also need roles that will link the JWT with these policies. + +One for staging named `myproject-staging`: + +```shell +$ vault write auth/jwt/role/myproject-staging - <<EOF +{ + "role_type": "jwt", + "policies": ["myproject-staging"], + "token_explicit_max_ttl": 60, + "user_claim": "user_email", + "bound_claims": { + "project_id": "22", + "ref": "master", + "ref_type": "branch" + } +} +EOF +``` + +And one for production named `myproject-production`: + +```shell +$ vault write auth/jwt/role/myproject-production - <<EOF +{ + "role_type": "jwt", + "policies": ["myproject-production"], + "token_explicit_max_ttl": 60, + "user_claim": "user_email", + "bound_claims_type": "glob", + "bound_claims": { + "project_id": "22", + "ref_protected": "true", + "ref_type": "branch", + "ref": "auto-deploy-*" + } +} +EOF +``` + +This example uses [bound_claims](https://www.vaultproject.io/api/auth/jwt#bound_claims) to specify that only a JWT with matching values for the specified claims will be allowed to authenticate. + +Combined with GitLab's [protected branches](../../../user/project/protected_branches.md), you can restrict who is able to authenticate and read the secrets. + +[token_explicit_max_ttl](https://www.vaultproject.io/api/auth/jwt#token_explicit_max_ttl) specifies that the token issued by Vault, upon successful authentication, has a hard lifetime limit of 60 seconds. + +[user_claim](https://www.vaultproject.io/api/auth/jwt#user_claim) specifies the name for the Identity alias created by Vault upon a successful login. + +[bound_claims_type](https://www.vaultproject.io/api-docs/auth/jwt#bound_claims_type) configures the interpretation of the `bound_claims` values. If set to `glob`, the values will be interpreted as globs, with `*` matching any number of characters. + +For the full list of options, see Vault's [Create Role documentation](https://www.vaultproject.io/api/auth/jwt#create-role). + +CAUTION: **Caution**: +Always restrict your roles to project or namespace by using one of the provided claims (e.g. `project_id` or `namespace_id`). Otherwise any JWT generated by this instance may be allowed to authenticate using this role. + +Now, configure the JWT Authentication method: + +```shell +$ vault write auth/jwt/config \ + jwks_url="https://gitlab.example.com/-/jwks" \ + bound_issuer="gitlab.example.com" +``` + +[bound_issuer](https://www.vaultproject.io/api/auth/jwt#inlinecode-bound_issuer) specifies that only a JWT with the issuer (that is, the `iss` claim) set to `gitlab.example.com` can use this method to authenticate, and that the JWKS endpoint (`https://gitlab.example.com/-/jwks`) should be used to validate the token. + +For the full list of available configuration options, see Vault's [API documentation](https://www.vaultproject.io/api/auth/jwt#configure). + +The following job, when run for the `master` branch, will be able to read secrets under `secret/myproject/staging/`, but not the secrets under `secret/myproject/production/`: + +```yaml +read_secrets: + script: + # Check job's ref name + - echo $CI_COMMIT_REF_NAME + # and is this ref protected + - echo $CI_COMMIT_REF_PROTECTED + # Vault's address can be provided here or as CI variable + - export VAULT_ADDR=http://vault.example.com:8200 + # Authenticate and get token. Token expiry time and other properties can be configured + # when configuring JWT Auth - https://www.vaultproject.io/api/auth/jwt#parameters-1 + - export VAULT_TOKEN="$(vault write -field=token auth/jwt/login role=myproject-staging jwt=$CI_JOB_JWT)" + # Now use the VAULT_TOKEN to read the secret and store it in an environment variable + - export PASSWORD="$(vault kv get -field=password secret/myproject/staging/db)" + # Use the secret + - echo $PASSWORD + # This will fail because the role myproject-staging can not read secrets from secret/myproject/production/* + - export PASSWORD="$(vault kv get -field=password secret/myproject/production/db)" +``` + + + +The following job will be able to authenticate using the `myproject-production` role and read secrets under `/secret/myproject/production/`: + +```yaml +read_secrets: + script: + # Check job's ref name + - echo $CI_COMMIT_REF_NAME + # and is this ref protected + - echo $CI_COMMIT_REF_PROTECTED + # Vault's address can be provided here or as CI variable + - export VAULT_ADDR=http://vault.example.com:8200 + # Authenticate and get token. Token expiry time and other properties can be configured + # when configuring JWT Auth - https://www.vaultproject.io/api/auth/jwt#parameters-1 + - export VAULT_TOKEN="$(vault write -field=token auth/jwt/login role=myproject-production jwt=$CI_JOB_JWT)" + # Now use the VAULT_TOKEN to read the secret and store it in environment variable + - export PASSWORD="$(vault kv get -field=password secret/myproject/production/db)" + # Use the secret + - echo $PASSWORD +``` + + diff --git a/doc/ci/junit_test_reports.md b/doc/ci/junit_test_reports.md index a97d4b865c8..c79db6dfbea 100644 --- a/doc/ci/junit_test_reports.md +++ b/doc/ci/junit_test_reports.md @@ -88,7 +88,7 @@ ruby: stage: test script: - bundle install - - rspec spec/lib/ --format RspecJunitFormatter --out rspec.xml + - bundle exec rspec --format progress --format RspecJunitFormatter --out rspec.xml artifacts: paths: - rspec.xml diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_immediate_merge_v12_6.png b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_immediate_merge_v12_6.png Binary files differindex de5897c271b..7b903716a3d 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_immediate_merge_v12_6.png +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/img/merge_train_immediate_merge_v12_6.png diff --git a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md index b6706c2a272..641192afea1 100644 --- a/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md +++ b/doc/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/index.md @@ -168,6 +168,27 @@ A Merge Train pipeline cannot be retried because the merge request is dropped fr In the case of pipeline failure, you should [re-enqueue](#add-a-merge-request-to-a-merge-train) the merge request to the merge train, which will then initiate a new pipeline. +### Unable to add to merge train with message "The pipeline for this merge request failed." + +Sometimes the **Start/Add to Merge Train** button is not available and the merge request says, +"The pipeline for this merge request failed. Please retry the job or push a new commit to fix the failure." + +This issue occurs when [**Pipelines must succeed**](../../../../user/project/merge_requests/merge_when_pipeline_succeeds.md#only-allow-merge-requests-to-be-merged-if-the-pipeline-succeeds) +is enabled in **Settings > General > Merge requests**. This option requires that you +run a new successful pipeline before you can re-add a merge request to a merge train. + +Merge trains ensure that each pipeline has succeeded before a merge happens, so +you can clear the **Pipelines must succeed** check box and keep +**Merge pipelines will try to validate the post-merge result prior to merging** (merge trains) enabled. + +If you want to keep the **Pipelines must succeed** option enabled along with Merge +Trains, you can create a new pipeline for merged results when this error occurs by +going to the **Pipelines** tab and clicking **Run pipeline**. Then click +**Start/Add to merge train when pipeline succeeds**. + +See [the related issue](https://gitlab.com/gitlab-org/gitlab/issues/35135) +for more information. + ### Merge Trains feature flag **(PREMIUM ONLY)** To enable and disable the Merge Trains feature, use the `:disable_merge_trains` feature flag. diff --git a/doc/ci/pipelines/img/job_group_v12_10.png b/doc/ci/pipelines/img/job_group_v12_10.png Binary files differindex 0ce121ab572..27e6bfbfc0f 100644 --- a/doc/ci/pipelines/img/job_group_v12_10.png +++ b/doc/ci/pipelines/img/job_group_v12_10.png diff --git a/doc/ci/pipelines/settings.md b/doc/ci/pipelines/settings.md index bddf64f397e..7ac9ba6a7dd 100644 --- a/doc/ci/pipelines/settings.md +++ b/doc/ci/pipelines/settings.md @@ -160,7 +160,7 @@ This also determines the visibility of these related features: - Job output logs - Job artifacts -- The [pipeline security dashboard](../../user/application_security/security_dashboard/index.md#pipeline-security-dashboard) **(ULTIMATE)** +- The [pipeline security dashboard](../../user/application_security/security_dashboard/index.md#pipeline-security) **(ULTIMATE)** If **Public pipelines** is enabled (default): diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md index 860eab469dd..c3bdd524bff 100644 --- a/doc/ci/review_apps/index.md +++ b/doc/ci/review_apps/index.md @@ -188,9 +188,6 @@ With Visual Reviews, you can provide a feedback form to your Review Apps so that reviewers can post comments directly from the app back to the merge request that spawned the Review App. -NOTE: **Note:** Visual Reviews currently only work for public projects. Support for private -and internal projects [is planned](https://gitlab.com/gitlab-org/gitlab/-/issues/42750). - ### Configuring Visual Reviews Ensure that the `anonymous_visual_review_feedback` feature flag is enabled. @@ -218,6 +215,7 @@ looks like: data-merge-request-id='1' data-mr-url='https://gitlab.example.com' data-project-path='sarah/review-app-tester' + data-require-auth='true' id='review-app-toolbar-script' src='https://gitlab.example.com/assets/webpack/visual_review_toolbar.js'> </script> @@ -235,6 +233,7 @@ to replace those values at runtime when each review app is created: - `data-mr-url` is the URL of the GitLab instance and will be the same for all review apps. - `data-project-path` is the project's path, which can be found by `CI_PROJECT_PATH`. +- `data-require-auth` is optional for public projects but required for [private and internal ones](#visual-reviews-in-private-or-internal-projects). If this is set to `true`, the user will be required to enter their [personal access token](../../user/profile/personal_access_tokens.md) instead of their name and email. - `id` is always `review-app-toolbar-script`, you don't need to change that. - `src` is the source of the review toolbar script, which resides in the respective GitLab instance and will be the same for all review apps. @@ -272,6 +271,15 @@ can supply the ID by either: - Dynamically adding the `data-merge-request-id` value during the build of the app. - Supplying it manually through the visual review form in the app. +### Visual Reviews in private or internal projects + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/42750#note_317271120) in GitLab 12.10. + +To enable visual reviews for private and internal projects, set the +[`data-require-auth` variable](#configuring-visual-reviews) to `true`. When enabled, +the user must enter a [personal access token](../../user/profile/personal_access_tokens.md) +with `read_api` scope before submitting feedback. + ### Using Visual Reviews After Visual Reviews has been [enabled](#configuring-visual-reviews) for the @@ -285,7 +293,7 @@ To use the feedback form: 1. Make a comment on the visual review. You can make use of all the [Markdown annotations](../../user/markdown.md) that are also available in merge request comments. -1. Submit your feedback anonymously or add your name. +1. If `data-require-auth` is `true`, you must enter your [personal access token](../../user/profile/personal_access_tokens.md). Otherwise, you must enter your name, and optionally, your email. 1. Finally, click **Send feedback**. After you make and submit a comment in the visual review box, it will appear diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index e619a32b90f..99fbc2134a4 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -81,6 +81,8 @@ echo "$KUBE_CA_PEM" > "$(pwd)/kube.ca.pem" kubectl config set-cluster e2e --server="$KUBE_URL" --certificate-authority="$(pwd)/kube.ca.pem" ``` +There are [some predefined variables](#custom-variables-validated-by-gitlab) of this type, which may be further validated. They will appear when you add or update a variable. + ##### File type The example above can now be simplified by creating a "File" type variable, and using @@ -116,6 +118,21 @@ If the value does not meet the requirements above, then the CI variable will fai In order to save, either alter the value to meet the masking requirements or disable **Masked** for the variable. +#### Custom variables validated by GitLab + +Some variables are listed in the UI so you can choose them more quickly. +GitLab validates the values of these variables to ensure they are in the correct format. + +| Variable | Allowed Values | Introduced in | +|-------------------------|----------------------------------------------------|---------------| +| `AWS_ACCESS_KEY_ID` | 20 characters: letters, digits | 12.10 | +| `AWS_DEFAULT_REGION` | Any | 12.10 | +| `AWS_SECRET_ACCESS_KEY` | 40 characters: letters, digits, special characters | 12.10 | + +NOTE: **Note:** +When you store credentials, there are security implications. If you are using AWS keys, +for example, follow their [best practices](https://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html). + ## Getting started To get started with environment variables in the scope of GitLab @@ -482,11 +499,16 @@ value you set for this specific pipeline: ## Environment variables expressions -> Introduced in GitLab 10.7. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/37397) in GitLab 10.7 for [the `only` and `except` CI keywords](../yaml/README.md#onlyexcept-advanced) +> - [Expanded](https://gitlab.com/gitlab-org/gitlab/issues/27863) in GitLab 12.3 with [the `rules` keyword](../yaml/README.md#rules) -It is possible to use variables expressions with only / except policies in -`.gitlab-ci.yml`. By using this approach you can limit what jobs are going to -be created within a pipeline after pushing a code to GitLab. +Variable expressions can be used to limit what jobs are going to be created +within a pipeline after pushing changes to GitLab. + +In `.gitlab-ci.yml`, they work with both + +- [`rules`](../yaml/README.md#rules), which is the recommended approach, and +- [`only` and `except`](../yaml/README.md#onlyexcept-basic), which are candidates for deprecation. This is particularly useful in combination with variables and triggered pipeline variables. @@ -573,8 +595,8 @@ Below you can find supported syntax reference: Examples: - - `$VARIABLE =~ /^content.*/` - - `$VARIABLE_1 !~ /^content.*/` (introduced in GitLab 11.11) + - `=~`: True if pattern is matched. Ex: `$VARIABLE =~ /^content.*/` + - `!~`: True if pattern is not matched. Ex: `$VARIABLE_1 !~ /^content.*/` ([Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/61900) in GitLab 11.11) Variable pattern matching with regular expressions uses the [RE2 regular expression syntax](https://github.com/google/re2/wiki/Syntax). @@ -596,8 +618,48 @@ Below you can find supported syntax reference: It is possible to join multiple conditions using `&&` or `||`. Any of the otherwise supported syntax may be used in a conjunctive or disjunctive statement. - Precedence of operators follows standard Ruby 2.5 operation - [precedence](https://ruby-doc.org/core-2.5.0/doc/syntax/precedence_rdoc.html). + Precedence of operators follows the + [Ruby 2.5 standard](https://ruby-doc.org/core-2.5.0/doc/syntax/precedence_rdoc.html), + so `&&` is evaluated before `||`. + +### Storing regular expressions in variables + +It is possible to store a regular expression in a variable, to be used for pattern matching: + +```yaml +variables: + STAGINGRELS: '/staging0|staging1/' + +deploy_staging: + script: do.sh deploy staging + environment: staging + rules: + - if: '$RELEASE =~ $STAGINGRELS' +``` + +NOTE: **Note:** +The available regular expression syntax is limited. See [related issue](https://gitlab.com/gitlab-org/gitlab/issues/35438) +for more details. + +If needed, you can use a test pipeline to determine whether a regular expression will +work in a variable. The example below tests the `^mast.*` regular expression directly, +as well as from within a variable: + +```yaml +variables: + MYSTRING: 'master' + MYREGEX: '/^mast.*/' + +testdirect: + script: /bin/true + rules: + - if: '$MYSTRING =~ /^mast.*/' + +testvariable: + script: /bin/true + rules: + - if: '$MYSTRING =~ $MYREGEX' +``` ## Debug logging diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md index 8b8eeb83d16..f53fd371c10 100644 --- a/doc/ci/variables/predefined_variables.md +++ b/doc/ci/variables/predefined_variables.md @@ -62,6 +62,7 @@ future GitLab releases.** | `CI_JOB_NAME` | 9.0 | 0.5 | The name of the job as defined in `.gitlab-ci.yml` | | `CI_JOB_STAGE` | 9.0 | 0.5 | The name of the stage as defined in `.gitlab-ci.yml` | | `CI_JOB_TOKEN` | 9.0 | 1.2 | Token used for authenticating with the [GitLab Container Registry](../../user/packages/container_registry/index.md) and downloading [dependent repositories](../../user/project/new_ci_build_permissions_model.md#dependent-repositories) | +| `CI_JOB_JWT` | 12.10 | all | RS256 JSON web token that can be used for authenticating with third party systems that support JWT authentication, for example [HashiCorp's Vault](../examples/authenticating-with-hashicorp-vault). | | `CI_JOB_URL` | 11.1 | 0.5 | Job details URL | | `CI_MERGE_REQUEST_ASSIGNEES` | 11.9 | all | Comma-separated list of username(s) of assignee(s) for the merge request if [the pipelines are for merge requests](../merge_request_pipelines/index.md). Available only if `only: [merge_requests]` or [`rules`](../yaml/README.md#rules) syntax is used and the merge request is created. | | `CI_MERGE_REQUEST_CHANGED_PAGE_PATHS` | 12.9 | all | Comma-separated list of paths of changed pages in a deployed [Review App](../review_apps/index.md) for a [Merge Request](../merge_request_pipelines/index.md). A [Route Map](../review_apps/index.md#route-maps) must be configured. | diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index e79d44cb057..31459735101 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -3056,6 +3056,9 @@ There can be multiple `resource_group`s defined per environment. A good use case is when deploying to physical devices. You may have more than one physical device, and each one can be deployed to, but there can be only one deployment per device at any given time. +NOTE: **Note:** +This key can only contain letters, digits, `-`, `_`, `/`, `$`, `{`, `}`, `.`, and spaces, but it cannot start or end with `/`. + ### `include` > - Introduced in [GitLab Premium](https://about.gitlab.com/pricing/) 10.5. @@ -3777,11 +3780,12 @@ script: You can set the number for attempts the running job will try to execute each of the following stages: -| Variable | Description | -|-------------------------------- |-------------| -| **GET_SOURCES_ATTEMPTS** | Number of attempts to fetch sources running a job | -| **ARTIFACT_DOWNLOAD_ATTEMPTS** | Number of attempts to download artifacts running a job | -| **RESTORE_CACHE_ATTEMPTS** | Number of attempts to restore the cache running a job | +| Variable | Description | +|-----------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| **GET_SOURCES_ATTEMPTS** | Number of attempts to fetch sources running a job | +| **ARTIFACT_DOWNLOAD_ATTEMPTS** | Number of attempts to download artifacts running a job | +| **RESTORE_CACHE_ATTEMPTS** | Number of attempts to restore the cache running a job | +| **EXECUTOR_JOB_SECTION_ATTEMPTS** | [Since GitLab 12.10](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4450), the number of attempts to run a section in a job after a [`No Such Container`](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/4450) error ([Docker executor](https://docs.gitlab.com/runner/executors/docker.html) only). | The default is one single attempt. |
