diff options
Diffstat (limited to 'doc/topics/autodevops/index.md')
| -rw-r--r-- | doc/topics/autodevops/index.md | 172 |
1 files changed, 137 insertions, 35 deletions
diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index fec575f263f..fe6a88b1df7 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -41,6 +41,7 @@ project in an easy and automatic way: 1. [Auto Code Quality](#auto-code-quality) 1. [Auto SAST (Static Application Security Testing)](#auto-sast) 1. [Auto Dependency Scanning](#auto-dependency-scanning) +1. [Auto License Management](#auto-license-management) 1. [Auto Container Scanning](#auto-container-scanning) 1. [Auto Review Apps](#auto-review-apps) 1. [Auto DAST (Dynamic Application Security Testing)](#auto-dast) @@ -62,7 +63,7 @@ Auto DevOps provides great defaults for all the stages; you can, however, For an overview on the creation of Auto DevOps, read the blog post [From 2/3 of the Self-Hosted Git Market, to the Next-Generation CI System, to Auto DevOps](https://about.gitlab.com/2017/06/29/whats-next-for-gitlab-ci/). -## Prerequisites +## Requirements TIP: **Tip:** For self-hosted installations, the easiest way to make use of Auto DevOps is to @@ -112,25 +113,26 @@ NOTE: **Note:** If you do not have Kubernetes or Prometheus installed, then Auto Review Apps, Auto Deploy, and Auto Monitoring will be silently skipped. -### Auto DevOps base domain +## Auto DevOps base domain The Auto DevOps base domain is required if you want to make use of [Auto -Review Apps](#auto-review-apps) and [Auto Deploy](#auto-deploy). It is defined -either under the project's CI/CD settings while -[enabling Auto DevOps](#enabling-auto-devops) or in instance-wide settings in -the CI/CD section. -It can also be set at the project or group level as a variable, `AUTO_DEVOPS_DOMAIN`. +Review Apps](#auto-review-apps) and [Auto Deploy](#auto-deploy). It can be defined +in three places: -A wildcard DNS A record matching the base domain is required, for example, +- either under the project's CI/CD settings while [enabling Auto DevOps](#enabling-auto-devops) +- or in instance-wide settings in the **admin area > Settings** under the "Continuous Integration and Delivery" section +- or at the project or group level as a variable: `AUTO_DEVOPS_DOMAIN` (required if you want to use [multiple clusters](#using-multiple-kubernetes-clusters)) + +A wildcard DNS A record matching the base domain(s) is required, for example, given a base domain of `example.com`, you'd need a DNS entry like: ``` *.example.com 3600 A 1.2.3.4 ``` -where `example.com` is the domain name under which the deployed apps will be served, +In this case, `example.com` is the domain name under which the deployed apps will be served, and `1.2.3.4` is the IP address of your load balancer; generally NGINX -([see prerequisites](#prerequisites)). How to set up the DNS record is beyond +([see requirements](#requirements)). How to set up the DNS record is beyond the scope of this document; you should check with your DNS provider. Alternatively you can use free public services like [xip.io](http://xip.io) or @@ -146,6 +148,56 @@ If GitLab is installed using the [GitLab Omnibus Helm Chart], there are two options: provide a static IP, or have one assigned. For more information see the relevant docs on the [network prerequisites](../../install/kubernetes/gitlab_omnibus.md#networking-prerequisites). +## Using multiple Kubernetes clusters **[PREMIUM]** + +When using Auto DevOps, you may want to deploy different environments to +different Kubernetes clusters. This is possible due to the 1:1 connection that +[exists between them](../../user/project/clusters/index.md#multiple-kubernetes-clusters). + +In the [Auto DevOps template](https://gitlab.com/gitlab-org/gitlab-ci-yml/blob/master/Auto-DevOps.gitlab-ci.yml) +(used behind the scenes by Auto DevOps), there are currently 3 defined environment names that you need to know: + +- `review/` (every environment starting with `review/`) +- `staging` +- `production` + +Those environments are tied to jobs that use [Auto Deploy](#auto-deploy), so +except for the environment scope, they would also need to have a different +domain they would be deployed to. This is why you need to define a separate +`AUTO_DEVOPS_DOMAIN` variable for all the above +[based on the environment](../../ci/variables/README.md#limiting-environment-scopes-of-variables). + +The following table is an example of how the three different clusters would +be configured. + +| Cluster name | Cluster environment scope | `AUTO_DEVOPS_DOMAIN` variable value | Variable environment scope | Notes | +| ------------ | -------------- | ----------------------------- | ------------- | ------ | +| review | `review/*` | `review.example.com` | `review/*` | The review cluster which will run all [Review Apps](../../ci/review_apps/index.md). `*` is a wildcard, which means it will be used by every environment name starting with `review/`. | +| staging | `staging` | `staging.example.com` | `staging` | (Optional) The staging cluster which will run the deployments of the staging environments. You need to [enable it first](#deploy-policy-for-staging-and-production-environments). | +| production | `production` | `example.com` | `production` | The production cluster which will run the deployments of the production environment. You can use [incremental rollouts](#incremental-rollout-to-production). | + +To add a different cluster for each environment: + +1. Navigate to your project's **Operations > Kubernetes** and create the Kubernetes clusters + with their respective environment scope as described from the table above. + +  + +1. After the clusters are created, navigate to each one and install Helm Tiller + and Ingress. +1. Make sure you have [configured your DNS](#auto-devops-base-domain) with the + specified Auto DevOps domains. +1. Navigate to your project's **Settings > CI/CD > Variables** and add + the `AUTO_DEVOPS_DOMAIN` variables with their respective environment + scope. + +  + +Now that all is configured, you can test your setup by creating a merge request +and verifying that your app is deployed as a review app in the Kubernetes +cluster with the `review/*` environment scope. Similarly, you can check the +other environments. + ## Quick start If you are using GitLab.com, see our [quick start guide](quick_start_guide.md) @@ -154,18 +206,18 @@ Google Cloud. ## Enabling Auto DevOps -If you haven't done already, read the [prerequisites](#prerequisites) to make +If you haven't done already, read the [requirements](#requirements) to make full use of Auto DevOps. If this is your fist time, we recommend you follow the -[quick start guide](#quick-start). +[quick start guide](quick_start_guide.md). To enable Auto DevOps to your project: -1. Check that your project doesn't have a `.gitlab-ci.yml`, and remove it otherwise -1. Go to your project's **Settings > CI/CD > General pipelines settings** and - find the Auto DevOps section +1. Check that your project doesn't have a `.gitlab-ci.yml`, or remove it otherwise +1. Go to your project's **Settings > CI/CD > Auto DevOps** 1. Select "Enable Auto DevOps" 1. Optionally, but recommended, add in the [base domain](#auto-devops-base-domain) - that will be used by Kubernetes to deploy your application + that will be used by Kubernetes to [deploy your application](#auto-deploy) + and choose the [deployment strategy](#deployment-strategy) 1. Hit **Save changes** for the changes to take effect Once saved, an Auto DevOps pipeline will be triggered on the default branch. @@ -182,6 +234,24 @@ in **Admin Area > Settings > Continuous Integration and Deployment**. Doing that all the projects that haven't explicitly set an option will have Auto DevOps enabled by default. +### Deployment strategy + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/38542) in GitLab 11.0. + +You can change the deployment strategy used by Auto DevOps by going to your +project's **Settings > CI/CD > Auto DevOps**. + +The available options are: + +- **Continuous deployment to production** - enables [Auto Deploy](#auto-deploy) + by setting the [`STAGING_ENABLED`](#deploy-policy-for-staging-and-production-environments) and + [`INCREMENTAL_ROLLOUT_ENABLED`](#incremental-rollout-to-production) variables + to false. +- **Automatic deployment to staging, manual deployment to production** - sets the + [`STAGING_ENABLED`](#deploy-policy-for-staging-and-production-environments) and + [`INCREMENTAL_ROLLOUT_ENABLED`](#incremental-rollout-to-production) variables + to true, and the user is responsible for manually deploying to staging and production. + ## Stages of Auto DevOps The following sections describe the stages of Auto DevOps. Read them carefully @@ -230,7 +300,7 @@ In GitLab Starter, differences between the source and target branches are also [shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality_diff.html). -### Auto SAST +### Auto SAST **[ULTIMATE]** > Introduced in [GitLab Ultimate][ee] 10.3. @@ -241,9 +311,9 @@ report is created, it's uploaded as an artifact which you can later download and check out. In GitLab Ultimate, any security warnings are also -[shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/sast.html). +[shown in the merge request widget](https://docs.gitlab.com/ee//user/project/merge_requests/sast.html). -### Auto Dependency Scanning +### Auto Dependency Scanning **[ULTIMATE]** > Introduced in [GitLab Ultimate][ee] 10.7. @@ -254,7 +324,20 @@ report is created, it's uploaded as an artifact which you can later download and check out. In GitLab Ultimate, any security warnings are also -[shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/dependency_scanning.html). +[shown in the merge request widget](https://docs.gitlab.com/ee//user/project/merge_requests/dependency_scanning.html). + +### Auto License Management **[ULTIMATE]** + +> Introduced in [GitLab Ultimate][ee] 11.0. + +License Management uses the +[License Management Docker image](https://gitlab.com/gitlab-org/security-products/license_management) +to search the project dependencies for their license. Once the +report is created, it's uploaded as an artifact which you can later download and +check out. + +In GitLab Ultimate, any licenses are also +[shown in the merge request widget](https://docs.gitlab.com/ee//user/project/merge_requests/license_management.html). ### Auto Container Scanning @@ -267,13 +350,13 @@ created, it's uploaded as an artifact which you can later download and check out. In GitLab Ultimate, any security warnings are also -[shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/container_scanning.html). +[shown in the merge request widget](https://docs.gitlab.com/ee//user/project/merge_requests/container_scanning.html). ### Auto Review Apps NOTE: **Note:** This is an optional step, since many projects do not have a Kubernetes cluster -available. If the [prerequisites](#prerequisites) are not met, the job will +available. If the [requirements](#requirements) are not met, the job will silently be skipped. CAUTION: **Caution:** @@ -295,7 +378,7 @@ up in the merge request widget for easy discovery. When the branch is deleted, for example after the merge request is merged, the Review App will automatically be deleted. -### Auto DAST +### Auto DAST **[ULTIMATE]** > Introduced in [GitLab Ultimate][ee] 10.4. @@ -306,9 +389,9 @@ issues. Once the report is created, it's uploaded as an artifact which you can later download and check out. In GitLab Ultimate, any security warnings are also -[shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/dast.html). +[shown in the merge request widget](https://docs.gitlab.com/ee//user/project/merge_requests/dast.html). -### Auto Browser Performance Testing +### Auto Browser Performance Testing **[PREMIUM]** > Introduced in [GitLab Premium][ee] 10.4. @@ -320,13 +403,14 @@ Auto Browser Performance Testing utilizes the [Sitespeed.io container](https://h /direction ``` -In GitLab Premium, performance differences between the source and target branches are [shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/browser_performance_testing.html). +In GitLab Premium, performance differences between the source +and target branches are [shown in the merge request widget](https://docs.gitlab.com/ee//user/project/merge_requests/browser_performance_testing.html). ### Auto Deploy NOTE: **Note:** This is an optional step, since many projects do not have a Kubernetes cluster -available. If the [prerequisites](#prerequisites) are not met, the job will +available. If the [requirements](#requirements) are not met, the job will silently be skipped. CAUTION: **Caution:** @@ -360,10 +444,19 @@ no longer be valid as soon as the deployment job finishes. This means that Kubernetes can run the application, but in case it should be restarted or executed somewhere else, it cannot be accessed again. +> [Introduced][ce-19507] in GitLab 11.0. + +For internal and private projects a [GitLab Deploy Token](../../user/project/deploy_tokens/index.md###gitlab-deploy-token) +will be automatically created, when Auto DevOps is enabled and the Auto DevOps settings are saved. This Deploy Token +can be used for permanent access to the registry. + +Note: **Note** +When the GitLab Deploy Token has been manually revoked, it won't be automatically created. + ### Auto Monitoring NOTE: **Note:** -Check the [prerequisites](#prerequisites) for Auto Monitoring to make this stage +Check the [requirements](#requirements) for Auto Monitoring to make this stage work. Once your application is deployed, Auto Monitoring makes it possible to monitor @@ -437,7 +530,7 @@ repo or by specifying a project variable: file in it, Auto DevOps will detect the chart and use it instead of the [default one](https://gitlab.com/charts/charts.gitlab.io/tree/master/charts/auto-deploy-app). This can be a great way to control exactly how your application is deployed. -- **Project variable** - Create a [variable](../../ci/variables/README.md#variables) +- **Project variable** - Create a [project variable](../../ci/variables/README.md#secret-variables) `AUTO_DEVOPS_CHART` with the URL of a custom chart to use. ### Customizing `.gitlab-ci.yml` @@ -493,21 +586,23 @@ also be customized, and you can easily use a [custom buildpack](#custom-buildpac | `POSTGRES_PASSWORD` | The PostgreSQL password; defaults to `testing-password`. Set it to use a custom password. | | `POSTGRES_DB` | The PostgreSQL database name; defaults to the value of [`$CI_ENVIRONMENT_SLUG`](../../ci/variables/README.md#predefined-variables-environment-variables). Set it to use a custom database name. | | `BUILDPACK_URL` | The buildpack's full URL. It can point to either Git repositories or a tarball URL. For Git repositories, it is possible to point to a specific `ref`, for example `https://github.com/heroku/heroku-buildpack-ruby.git#v142` | +| `SAST_CONFIDENCE_LEVEL` | The minimum confidence level of security issues you want to be reported; `1` for Low, `2` for Medium, `3` for High; defaults to `3`.| +| `DEP_SCAN_DISABLE_REMOTE_CHECKS` | Whether remote Dependency Scanning checks are disabled; defaults to `"false"`. Set to `"true"` to disable checks that send data to GitLab central servers. [Read more about remote checks](https://gitlab.com/gitlab-org/security-products/dependency-scanning#remote-checks).| | `STAGING_ENABLED` | From GitLab 10.8, this variable can be used to define a [deploy policy for staging and production environments](#deploy-policy-for-staging-and-production-environments). | | `CANARY_ENABLED` | From GitLab 11.0, this variable can be used to define a [deploy policy for canary environments](#deploy-policy-for-canary-environments). | | `INCREMENTAL_ROLLOUT_ENABLED`| From GitLab 10.8, this variable can be used to enable an [incremental rollout](#incremental-rollout-to-production) of your application for the production environment. | | `TEST_DISABLED` | From GitLab 11.0, this variable can be used to disable the `test` job. If the variable is present, the job will not be created. | -| `CODE_QUALITY_DISABLED` | From GitLab 11.0, this variable can be used to disable the `code_quality` job. If the variable is present, the job will not be created. | +| `CODEQUALITY_DISABLED` | From GitLab 11.0, this variable can be used to disable the `codequality` job. If the variable is present, the job will not be created. | | `SAST_DISABLED` | From GitLab 11.0, this variable can be used to disable the `sast` job. If the variable is present, the job will not be created. | | `DEPENDENCY_SCANNING_DISABLED` | From GitLab 11.0, this variable can be used to disable the `dependency_scanning` job. If the variable is present, the job will not be created. | -| `CONTAINER_SCANNING_DISABLED` | From GitLab 11.0, this variable can be used to disable the `container_scanning` job. If the variable is present, the job will not be created. | +| `CONTAINER_SCANNING_DISABLED` | From GitLab 11.0, this variable can be used to disable the `sast:container` job. If the variable is present, the job will not be created. | | `REVIEW_DISABLED` | From GitLab 11.0, this variable can be used to disable the `review` and the manual `review:stop` job. If the variable is present, these jobs will not be created. | | `DAST_DISABLED` | From GitLab 11.0, this variable can be used to disable the `dast` job. If the variable is present, the job will not be created. | | `PERFORMANCE_DISABLED` | From GitLab 11.0, this variable can be used to disable the `performance` job. If the variable is present, the job will not be created. | TIP: **Tip:** Set up the replica variables using a -[project variable](../../ci/variables/README.md#variables) +[project variable](../../ci/variables/README.md#secret-variables) and scale your application by just redeploying it! CAUTION: **Caution:** @@ -575,6 +670,9 @@ service: > [Introduced](https://gitlab.com/gitlab-org/gitlab-ci-yml/merge_requests/160) in GitLab 10.8. +TIP: **Tip:** +You can also set this inside your [project's settings](#deployment-strategy). + The normal behavior of Auto DevOps is to use Continuous Deployment, pushing automatically to the `production` environment every time a new pipeline is run on the default branch. However, there are cases where you might want to use a @@ -582,7 +680,7 @@ staging environment and deploy to production manually. For this scenario, the `STAGING_ENABLED` environment variable was introduced. If `STAGING_ENABLED` is defined in your project (e.g., set `STAGING_ENABLED` to -`1` as a variable), then the application will be automatically deployed +`1` as a secret variable), then the application will be automatically deployed to a `staging` environment, and a `production_manual` job will be created for you when you're ready to manually deploy to production. @@ -595,7 +693,7 @@ A [canary environment](https://docs.gitlab.com/ee/user/project/canary_deployment before any changes are deployed to production. If `CANARY_ENABLED` is defined in your project (e.g., set `CANARY_ENABLED` to -`1` as a variable) then two manual jobs will be created: +`1` as a secret variable) then two manual jobs will be created: - `canary` which will deploy the application to the canary environment - `production_manual` which is to be used by you when you're ready to manually @@ -605,13 +703,16 @@ If `CANARY_ENABLED` is defined in your project (e.g., set `CANARY_ENABLED` to > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/5415) in GitLab 10.8. +TIP: **Tip:** +You can also set this inside your [project's settings](#deployment-strategy). + When you have a new version of your app to deploy in production, you may want to use an incremental rollout to replace just a few pods with the latest code. This will allow you to first check how the app is behaving, and later manually increasing the rollout up to 100%. If `INCREMENTAL_ROLLOUT_ENABLED` is defined in your project (e.g., set -`INCREMENTAL_ROLLOUT_ENABLED` to `1` as a variable), then instead of the +`INCREMENTAL_ROLLOUT_ENABLED` to `1` as a secret variable), then instead of the standard `production` job, 4 different [manual jobs](../../ci/pipelines.md#manual-actions-from-the-pipeline-graph) will be created: @@ -741,3 +842,4 @@ curl --data "value=true" --header "PRIVATE-TOKEN: personal_access_token" https:/ [Auto DevOps template]: https://gitlab.com/gitlab-org/gitlab-ci-yml/blob/master/Auto-DevOps.gitlab-ci.yml [GitLab Omnibus Helm Chart]: ../../install/kubernetes/gitlab_omnibus.md [ee]: https://about.gitlab.com/products/ +[ce-19507]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/19507 |