diff options
Diffstat (limited to 'doc/topics/autodevops')
| -rw-r--r-- | doc/topics/autodevops/customize.md | 43 | ||||
| -rw-r--r-- | doc/topics/autodevops/img/guide_pipeline_stages_v12_3.png | bin | 40329 -> 0 bytes | |||
| -rw-r--r-- | doc/topics/autodevops/img/guide_pipeline_stages_v13_0.png | bin | 0 -> 65686 bytes | |||
| -rw-r--r-- | doc/topics/autodevops/index.md | 205 | ||||
| -rw-r--r-- | doc/topics/autodevops/quick_start_guide.md | 14 | ||||
| -rw-r--r-- | doc/topics/autodevops/requirements.md | 134 | ||||
| -rw-r--r-- | doc/topics/autodevops/stages.md | 60 |
7 files changed, 239 insertions, 217 deletions
diff --git a/doc/topics/autodevops/customize.md b/doc/topics/autodevops/customize.md index 056b4c1caf4..253d5e56463 100644 --- a/doc/topics/autodevops/customize.md +++ b/doc/topics/autodevops/customize.md @@ -72,25 +72,20 @@ Avoid passing secrets as Docker build arguments if possible, as they may be persisted in your image. See [this discussion of best practices with secrets](https://github.com/moby/moby/issues/13490) for details. -## Passing secrets to `docker build` +## Forward CI variables to the build environment -> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/25514) in GitLab 12.3, but available in versions 11.9 and above. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25514) in GitLab 12.3, but available in versions 11.9 and above. -CI environment variables can be passed as -[build secrets](https://docs.docker.com/develop/develop-images/build_enhancements/#new-docker-build-secret-information) to the `docker build` command by listing them -by name, comma-separated, in the `AUTO_DEVOPS_BUILD_IMAGE_FORWARDED_CI_VARIABLES` -variable. For example, to forward the variables `CI_COMMIT_SHA` and `CI_ENVIRONMENT_NAME`, -set `AUTO_DEVOPS_BUILD_IMAGE_FORWARDED_CI_VARIABLES` to `CI_COMMIT_SHA,CI_ENVIRONMENT_NAME`. +CI variables can be forwarded into the build environment using the +`AUTO_DEVOPS_BUILD_IMAGE_FORWARDED_CI_VARIABLES` CI variable. +The forwarded variables should be specified by name in a comma-separated +list. For example, to forward the variables `CI_COMMIT_SHA` and +`CI_ENVIRONMENT_NAME`, set `AUTO_DEVOPS_BUILD_IMAGE_FORWARDED_CI_VARIABLES` +to `CI_COMMIT_SHA,CI_ENVIRONMENT_NAME`. -CAUTION: **Caution:** -Unlike build arguments, these variables are not persisted by Docker in the final image, -though you can still persist them yourself. - -In projects: - -- Without a `Dockerfile`, these are available automatically as environment - variables. -- With a `Dockerfile`, the following is required: +- When using Buildpacks, the forwarded variables are available automatically + as environment variables. +- When using a `Dockerfile`, the following additional steps are required: 1. Activate the experimental `Dockerfile` syntax by adding the following code to the top of the file: @@ -128,7 +123,7 @@ repository or by specifying a project variable: ## Customize values for Helm Chart -> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/30628) in GitLab 12.6, `.gitlab/auto-deploy-values.yaml` will be used by default for Helm upgrades. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30628) in GitLab 12.6, `.gitlab/auto-deploy-values.yaml` will be used by default for Helm upgrades. You can override the default values in the `values.yaml` file in the [default Helm chart](https://gitlab.com/gitlab-org/charts/auto-deploy-app) by either: @@ -175,7 +170,7 @@ into your project and edit it as needed. ## Customizing the Kubernetes namespace -> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/27630) in GitLab 12.6. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27630) in GitLab 12.6. For clusters not managed by GitLab, you can customize the namespace in `.gitlab-ci.yml` by specifying @@ -302,13 +297,15 @@ applications. | `<ENVIRONMENT>_ADDITIONAL_HOSTS` | For a specific environment, the fully qualified domain names specified as a comma-separated list that are added to the Ingress hosts. This takes precedence over `ADDITIONAL_HOSTS`. | | `AUTO_DEVOPS_ATOMIC_RELEASE` | As of GitLab 13.0, Auto DevOps uses [`--atomic`](https://v2.helm.sh/docs/helm/#options-43) for Helm deployments by default. Set this variable to `false` to disable the use of `--atomic` | | `AUTO_DEVOPS_BUILD_IMAGE_CNB_ENABLED` | When set to a non-empty value and no `Dockerfile` is present, Auto Build builds your application using Cloud Native Buildpacks instead of Herokuish. [More details](stages.md#auto-build-using-cloud-native-buildpacks-beta). | +| `AUTO_DEVOPS_BUILD_IMAGE_CNB_BUILDER` | The builder used when building with Cloud Native Buildpacks. The default builder is `heroku/buildpacks:18`. [More details](stages.md#auto-build-using-cloud-native-buildpacks-beta). | | `AUTO_DEVOPS_BUILD_IMAGE_EXTRA_ARGS` | Extra arguments to be passed to the `docker build` command. Note that using quotes won't prevent word splitting. [More details](#passing-arguments-to-docker-build). | -| `AUTO_DEVOPS_BUILD_IMAGE_FORWARDED_CI_VARIABLES` | A [comma-separated list of CI variable names](#passing-secrets-to-docker-build) to be passed to the `docker build` command as secrets. | +| `AUTO_DEVOPS_BUILD_IMAGE_FORWARDED_CI_VARIABLES` | A [comma-separated list of CI variable names](#forward-ci-variables-to-the-build-environment) to be forwarded to the build environment (the buildpack builder or `docker build`). | | `AUTO_DEVOPS_CHART` | Helm Chart used to deploy your apps. Defaults to the one [provided by GitLab](https://gitlab.com/gitlab-org/charts/auto-deploy-app). | | `AUTO_DEVOPS_CHART_REPOSITORY` | Helm Chart repository used to search for charts. Defaults to `https://charts.gitlab.io`. | | `AUTO_DEVOPS_CHART_REPOSITORY_NAME` | From GitLab 11.11, used to set the name of the Helm repository. Defaults to `gitlab`. | | `AUTO_DEVOPS_CHART_REPOSITORY_USERNAME` | From GitLab 11.11, used to set a username to connect to the Helm repository. Defaults to no credentials. Also set `AUTO_DEVOPS_CHART_REPOSITORY_PASSWORD`. | | `AUTO_DEVOPS_CHART_REPOSITORY_PASSWORD` | From GitLab 11.11, used to set a password to connect to the Helm repository. Defaults to no credentials. Also set `AUTO_DEVOPS_CHART_REPOSITORY_USERNAME`. | +| `AUTO_DEVOPS_DEPLOY_DEBUG` | From GitLab 13.1, if this variable is present, Helm will output debug logs. | | `AUTO_DEVOPS_MODSECURITY_SEC_RULE_ENGINE` | From GitLab 12.5, used in combination with [ModSecurity feature flag](../../user/clusters/applications.md#web-application-firewall-modsecurity) to toggle [ModSecurity's `SecRuleEngine`](https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual-(v2.x)#SecRuleEngine) behavior. Defaults to `DetectionOnly`. | | `BUILDPACK_URL` | Buildpack's full URL. Can point to either [a Git repository URL or a tarball URL](#custom-buildpacks). | | `CANARY_ENABLED` | From GitLab 11.0, used to define a [deploy policy for canary environments](#deploy-policy-for-canary-environments-premium). | @@ -368,7 +365,7 @@ The following table lists variables used to disable jobs. ### Application secret variables -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/49056) in GitLab 11.7. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/49056) in GitLab 11.7. Some applications need to define secret variables that are accessible by the deployed application. Auto DevOps detects variables starting with `K8S_SECRET_`, and makes @@ -506,7 +503,7 @@ If you define `CANARY_ENABLED` in your project, such as setting `CANARY_ENABLED` ### Incremental rollout to production **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/5415) in GitLab 10.8. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5415) in GitLab 10.8. TIP: **Tip:** You can also set this inside your [project's settings](index.md#deployment-strategy). @@ -563,7 +560,7 @@ removed in the future. ### Timed incremental rollout to production **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/issues/7545) in GitLab 11.4. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7545) in GitLab 11.4. TIP: **Tip:** You can also set this inside your [project's settings](index.md#deployment-strategy). @@ -597,7 +594,7 @@ The banner can be disabled for: - By an administrator running the following in a Rails console: ```ruby - Feature.get(:auto_devops_banner_disabled).enable + Feature.enable(:auto_devops_banner_disabled) ``` - Through the REST API with an admin access token: diff --git a/doc/topics/autodevops/img/guide_pipeline_stages_v12_3.png b/doc/topics/autodevops/img/guide_pipeline_stages_v12_3.png Binary files differdeleted file mode 100644 index b9bab112a9f..00000000000 --- a/doc/topics/autodevops/img/guide_pipeline_stages_v12_3.png +++ /dev/null diff --git a/doc/topics/autodevops/img/guide_pipeline_stages_v13_0.png b/doc/topics/autodevops/img/guide_pipeline_stages_v13_0.png Binary files differnew file mode 100644 index 00000000000..fb102879556 --- /dev/null +++ b/doc/topics/autodevops/img/guide_pipeline_stages_v13_0.png diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index e7165136cf0..767ea5ee7b7 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -1,6 +1,6 @@ # Auto DevOps -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/37115) in GitLab 10.0. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/37115) in GitLab 10.0. > - Generally available on GitLab 11.0. Auto DevOps provides pre-defined CI/CD configuration allowing you to automatically @@ -25,9 +25,11 @@ and GitLab does the rest, improving your productivity and efficiency. For an introduction to Auto DevOps, watch [AutoDevOps in GitLab 11.0](https://youtu.be/0Tc0YYBxqi4). +For requirements, see [Requirements for Auto DevOps](requirements.md) for more information. + ## Enabled by default -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/41729) in GitLab 11.3. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/41729) in GitLab 11.3. Auto DevOps is enabled by default for all projects and attempts to run on all pipelines in each project. An instance administrator can enable or disable this default in the @@ -35,7 +37,7 @@ in each project. An instance administrator can enable or disable this default in Auto DevOps automatically disables in individual projects on their first pipeline failure, if it has not been explicitly enabled for the project. -Since [GitLab 12.7](https://gitlab.com/gitlab-org/gitlab/issues/26655), Auto DevOps +Since [GitLab 12.7](https://gitlab.com/gitlab-org/gitlab/-/issues/26655), Auto DevOps runs on pipelines automatically only if a [`Dockerfile` or matching buildpack](stages.md#auto-build) exists. @@ -54,7 +56,7 @@ configuring a cluster on GKE. After configuring the provider, you can follow the steps in the [quick start guide](quick_start_guide.md) to get started. In [GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/issues/208132) and later, it is -possible to leverage Auto DevOps to deploy to [AWS ECS](#aws-ecs). +possible to leverage Auto DevOps to deploy to [AWS ECS](requirements.md#auto-devops-requirements-for-amazon-ecs). ## Comparison to application platforms and PaaS @@ -76,13 +78,14 @@ in multiple ways: ## Features -Comprised of a set of stages, Auto DevOps brings these best practices to your +Comprised of a set of [stages](stages.md), Auto DevOps brings these best practices to your project in a simple and automatic way: 1. [Auto Build](stages.md#auto-build) 1. [Auto Test](stages.md#auto-test) 1. [Auto Code Quality](stages.md#auto-code-quality-starter) **(STARTER)** 1. [Auto SAST (Static Application Security Testing)](stages.md#auto-sast-ultimate) **(ULTIMATE)** +1. [Auto Secret Detection](stages.md#auto-secret-detection-ultimate) **(ULTIMATE)** 1. [Auto Dependency Scanning](stages.md#auto-dependency-scanning-ultimate) **(ULTIMATE)** 1. [Auto License Compliance](stages.md#auto-license-compliance-ultimate) **(ULTIMATE)** 1. [Auto Container Scanning](stages.md#auto-container-scanning-ultimate) **(ULTIMATE)** @@ -111,127 +114,6 @@ NOTE: **Note** Kubernetes clusters can [be used without](../../user/project/clusters/index.md) Auto DevOps. -## Requirements - -### Kubernetes - -To make full use of Auto DevOps with Kubernetes, you need: - -- **Kubernetes** (for [Auto Review Apps](stages.md#auto-review-apps), - [Auto Deploy](stages.md#auto-deploy), and [Auto Monitoring](stages.md#auto-monitoring)) - - To enable deployments, you need: - - 1. A [Kubernetes 1.12+ cluster](../../user/project/clusters/index.md) for your - project. The easiest way is to create a - [new cluster using the GitLab UI](../../user/project/clusters/add_remove_clusters.md#create-new-cluster). - For Kubernetes 1.16+ clusters, you must perform additional configuration for - [Auto Deploy for Kubernetes 1.16+](stages.md#kubernetes-116). - 1. NGINX Ingress. You can deploy it to your Kubernetes cluster by installing - the [GitLab-managed app for Ingress](../../user/clusters/applications.md#ingress), - after configuring GitLab's Kubernetes integration in the previous step. - - Alternatively, you can use the - [`nginx-ingress`](https://github.com/helm/charts/tree/master/stable/nginx-ingress) - Helm chart to install Ingress manually. - - NOTE: **Note:** - If you use your own Ingress instead of the one provided by GitLab's managed - apps, ensure you're running at least version 0.9.0 of NGINX Ingress and - [enable Prometheus metrics](https://github.com/helm/charts/tree/master/stable/nginx-ingress#prometheus-metrics) - for the response metrics to appear. You must also - [annotate](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) - the NGINX Ingress deployment to be scraped by Prometheus using - `prometheus.io/scrape: "true"` and `prometheus.io/port: "10254"`. - -- **Base domain** (for [Auto Review Apps](stages.md#auto-review-apps), - [Auto Deploy](stages.md#auto-deploy), and [Auto Monitoring](stages.md#auto-monitoring)) - - You need a domain configured with wildcard DNS, which all of your Auto DevOps - applications will use. If you're using the - [GitLab-managed app for Ingress](../../user/clusters/applications.md#ingress), - the URL endpoint is automatically configured for you. - - You must also [specify the Auto DevOps base domain](#auto-devops-base-domain). - -- **GitLab Runner** (for all stages) - - Your Runner must be configured to run Docker, usually with either the - [Docker](https://docs.gitlab.com/runner/executors/docker.html) - or [Kubernetes](https://docs.gitlab.com/runner/executors/kubernetes.html) executors, with - [privileged mode enabled](https://docs.gitlab.com/runner/executors/docker.html#use-docker-in-docker-with-privileged-mode). - The Runners don't need to be installed in the Kubernetes cluster, but the - Kubernetes executor is easy to use and automatically autoscales. - You can configure Docker-based Runners to autoscale as well, using - [Docker Machine](https://docs.gitlab.com/runner/install/autoscaling.html). - - If you've configured GitLab's Kubernetes integration in the first step, you - can deploy it to your cluster by installing the - [GitLab-managed app for GitLab Runner](../../user/clusters/applications.md#gitlab-runner). - - Runners should be registered as [shared Runners](../../ci/runners/README.md#registering-a-shared-runner) - for the entire GitLab instance, or [specific Runners](../../ci/runners/README.md#registering-a-specific-runner) - that are assigned to specific projects (the default if you've installed the - GitLab Runner managed application). - -- **Prometheus** (for [Auto Monitoring](stages.md#auto-monitoring)) - - To enable Auto Monitoring, you need Prometheus installed either inside or - outside your cluster, and configured to scrape your Kubernetes cluster. - If you've configured GitLab's Kubernetes integration, you can deploy it to - your cluster by installing the - [GitLab-managed app for Prometheus](../../user/clusters/applications.md#prometheus). - - The [Prometheus service](../../user/project/integrations/prometheus.md) - integration must be enabled for the project, or enabled as a - [default service template](../../user/project/integrations/services_templates.md) - for the entire GitLab installation. - - To get response metrics (in addition to system metrics), you must - [configure Prometheus to monitor NGINX](../../user/project/integrations/prometheus_library/nginx_ingress.md#configuring-nginx-ingress-monitoring). - -- **cert-manager** (optional, for TLS/HTTPS) - - To enable HTTPS endpoints for your application, you must install cert-manager, - a native Kubernetes certificate management controller that helps with issuing - certificates. Installing cert-manager on your cluster issues a - [Let’s Encrypt](https://letsencrypt.org/) certificate and ensures the - certificates are valid and up-to-date. If you've configured GitLab's Kubernetes - integration, you can deploy it to your cluster by installing the - [GitLab-managed app for cert-manager](../../user/clusters/applications.md#cert-manager). - -If you don't have Kubernetes or Prometheus installed, then -[Auto Review Apps](stages.md#auto-review-apps), -[Auto Deploy](stages.md#auto-deploy), and [Auto Monitoring](stages.md#auto-monitoring) -are skipped. - -After all requirements are met, you can [enable Auto DevOps](#enablingdisabling-auto-devops). - -### AWS ECS - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208132) in GitLab 13.0. - -You can choose to target [AWS ECS](../../ci/cloud_deployment/index.md) as a deployment platform instead of using Kubernetes. - -To get started on Auto DevOps to ECS, you'll have to add a specific Environment -Variable. To do so, follow these steps: - -1. In your project, go to **Settings > CI / CD** and expand the **Variables** - section. - -1. Specify which AWS platform to target during the Auto DevOps deployment - by adding the `AUTO_DEVOPS_PLATFORM_TARGET` variable. - -1. Give this variable the value `ECS` before saving it. - -When you trigger a pipeline, if you have AutoDev Ops enabled and if you have correctly -[entered AWS credentials as environment variables](../../ci/cloud_deployment/index.md#deploy-your-application-to-aws-elastic-container-service-ecs), -your application will be deployed to AWS ECS. - -NOTE: **Note:** -If you have both a valid `AUTO_DEVOPS_PLATFORM_TARGET` variable and a Kubernetes cluster tied to your project, -only the deployment to Kubernetes will run. - ## Auto DevOps base domain The Auto DevOps base domain is required to use @@ -239,16 +121,18 @@ The Auto DevOps base domain is required to use [Auto Monitoring](stages.md#auto-monitoring). You can define the base domain in any of the following places: -- either under the cluster's settings, whether for +- either under the cluster's settings, whether for an instance, [projects](../../user/project/clusters/index.md#base-domain) or [groups](../../user/group/clusters/index.md#base-domain) -- or in instance-wide settings in **{admin}** **Admin Area > Settings** under the - **Continuous Integration and Delivery** section - or at the project level as a variable: `KUBE_INGRESS_BASE_DOMAIN` -- or at the group level as a variable: `KUBE_INGRESS_BASE_DOMAIN`. +- or at the group level as a variable: `KUBE_INGRESS_BASE_DOMAIN` +- or as an instance-wide fallback in **{admin}** **Admin Area > Settings** under the + **Continuous Integration and Delivery** section The base domain variable `KUBE_INGRESS_BASE_DOMAIN` follows the same order of precedence as other environment [variables](../../ci/variables/README.md#priority-of-environment-variables). +If the CI/CD variable is not set and the cluster setting is left blank, the instance-wide **Auto DevOps domain** +setting will be used if set. TIP: **Tip:** If you use the [GitLab managed app for Ingress](../../user/clusters/applications.md#ingress), @@ -256,9 +140,9 @@ the URL endpoint should be automatically configured for you. All you must do is use its value for the `KUBE_INGRESS_BASE_DOMAIN` variable. NOTE: **Note:** -`AUTO_DEVOPS_DOMAIN` was [deprecated in GitLab 11.8](https://gitlab.com/gitlab-org/gitlab-foss/issues/52363) +`AUTO_DEVOPS_DOMAIN` was [deprecated in GitLab 11.8](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/52363) and replaced with `KUBE_INGRESS_BASE_DOMAIN`, and removed in -[GitLab 12.0](https://gitlab.com/gitlab-org/gitlab-foss/issues/56959). +[GitLab 12.0](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/56959). Auto DevOps requires a wildcard DNS A record matching the base domain(s). For a base domain of `example.com`, you'd need a DNS entry like: @@ -305,7 +189,7 @@ After enabling the feature, an Auto DevOps pipeline is triggered on the default ### At the group level -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/52447) in GitLab 11.10. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/52447) in GitLab 11.10. Only administrators and group owners can enable or disable Auto DevOps at the group level. @@ -330,19 +214,9 @@ Auto DevOps at the group and project level, respectively. for Auto Deploy and Auto Review Apps to use. 1. Click **Save changes** for the changes to take effect. -### Enable for a percentage of projects - -You can use a feature flag to enable Auto DevOps by default to your desired percentage -of projects. From the console, enter the following command, replacing `10` with -your desired percentage: - -```ruby -Feature.get(:force_autodevops_on_by_default).enable_percentage_of_actors(10) -``` - ### Deployment strategy -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/issues/38542) in GitLab 11.0. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/38542) in GitLab 11.0. You can change the deployment strategy used by Auto DevOps by going to your project's **{settings}** **Settings > CI/CD > Auto DevOps**. The following options @@ -362,13 +236,17 @@ are available: - `master` branch is directly deployed to staging. - Manual actions are provided for incremental rollout to production. +TIP: **Tip:** +Use the [blue-green deployment](../../ci/environments/incremental_rollouts.md#blue-green-deployment) technique +to minimize downtime and risk. + ## Using multiple Kubernetes clusters **(PREMIUM)** When using Auto DevOps, you can deploy different environments to different Kubernetes clusters, due to the 1:1 connection [existing between them](../../user/project/clusters/index.md#multiple-kubernetes-clusters-premium). -The [template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml) +The [Deploy Job template](https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml) used by Auto DevOps currently defines 3 environment names: - `review/` (every environment starting with `review/`) @@ -393,9 +271,6 @@ To add a different cluster for each environment: 1. Navigate to your project's **{cloud-gear}** **Operations > Kubernetes**. 1. Create the Kubernetes clusters with their respective environment scope, as described from the table above. - -  - 1. After creating the clusters, navigate to each cluster and install Helm Tiller and Ingress. Wait for the Ingress IP address to be assigned. 1. Make sure you've [configured your DNS](#auto-devops-base-domain) with the @@ -408,35 +283,6 @@ and verifying your application is deployed as a Review App in the Kubernetes cluster with the `review/*` environment scope. Similarly, you can check the other environments. -## Currently supported languages - -Note that not all buildpacks support Auto Test yet, as it's a relatively new -enhancement. All of Heroku's -[officially supported languages](https://devcenter.heroku.com/articles/heroku-ci#supported-languages) -support Auto Test. The languages supported by Heroku's Herokuish buildpacks all -support Auto Test, but notably the multi-buildpack does not. - -As of GitLab 10.0, the supported buildpacks are: - -```plaintext -- heroku-buildpack-multi v1.0.0 -- heroku-buildpack-ruby v168 -- heroku-buildpack-nodejs v99 -- heroku-buildpack-clojure v77 -- heroku-buildpack-python v99 -- heroku-buildpack-java v53 -- heroku-buildpack-gradle v23 -- heroku-buildpack-scala v78 -- heroku-buildpack-play v26 -- heroku-buildpack-php v122 -- heroku-buildpack-go v72 -- heroku-buildpack-erlang fa17af9 -- buildpack-nginx v8 -``` - -If your application needs a buildpack that is not in the above list, you -might want to use a [custom buildpack](customize.md#custom-buildpacks). - ## Limitations The following restrictions apply. @@ -490,11 +336,6 @@ The following are possible reasons: - No buildpack may exist for your application. Try specifying a [custom buildpack](customize.md#custom-buildpacks). -### Mismatch between testing frameworks - -Auto Test may fail because of a mismatch between testing frameworks. In this -case, you may need to customize your `.gitlab-ci.yml` with your test commands. - ### Pipeline that extends Auto DevOps with only / except fails If your pipeline fails with the following message: diff --git a/doc/topics/autodevops/quick_start_guide.md b/doc/topics/autodevops/quick_start_guide.md index 859219689f9..ec5191dd4ac 100644 --- a/doc/topics/autodevops/quick_start_guide.md +++ b/doc/topics/autodevops/quick_start_guide.md @@ -152,8 +152,6 @@ these steps to enable Auto DevOps if it's disabled: After you save your changes, GitLab creates a new pipeline. To view it, go to **{rocket}** **CI/CD > Pipelines**. - - In the next section, we explain what each job does in the pipeline. ## Deploy the application @@ -167,7 +165,7 @@ without refreshing the page to **{status_success}** (for success) or The jobs are separated into stages: - + - **Build** - The application builds a Docker image and uploads it to your project's [Container Registry](../../user/packages/container_registry/index.md) ([Auto Build](stages.md#auto-build)). @@ -182,8 +180,9 @@ The jobs are separated into stages: - The `dependency_scanning` job checks if the application has any dependencies susceptible to vulnerabilities and is allowed to fail ([Auto Dependency Scanning](stages.md#auto-dependency-scanning-ultimate)) **(ULTIMATE)** - - The `sast` job runs static analysis on the current code to check for potential - security issues and is allowed to fail ([Auto SAST](stages.md#auto-sast-ultimate)) **(ULTIMATE)** + - Jobs suffixed with `-sast` run static analysis on the current code to check for potential + security issues, and are allowed to fail ([Auto SAST](stages.md#auto-sast-ultimate)) **(ULTIMATE)** + - The `secret-detection` job checks for leaked secrets and is allowed to fail ([Auto Secret Detection](stages.md#auto-secret-detection-ultimate)) **(ULTIMATE)** - The `license_management` job searches the application's dependencies to determine each of their licenses and is allowed to fail ([Auto License Compliance](stages.md#auto-license-compliance-ultimate)) **(ULTIMATE)** @@ -191,12 +190,17 @@ The jobs are separated into stages: NOTE: **Note:** All jobs except `test` are allowed to fail in the test stage. +- **Review** - Pipelines on `master` include this stage with a `dast_environment_deploy` job. + To learn more, see [Dynamic Application Security Testing (DAST)](../../user/application_security/dast/index.md). + - **Production** - After the tests and checks finish, the application deploys in Kubernetes ([Auto Deploy](stages.md#auto-deploy)). - **Performance** - Performance tests are run on the deployed application ([Auto Browser Performance Testing](stages.md#auto-browser-performance-testing-premium)). **(PREMIUM)** +- **Cleanup** - Pipelines on `master` include this stage with a `stop_dast_environment` job. + After running a pipeline, you should view your deployed website and learn how to monitor it. diff --git a/doc/topics/autodevops/requirements.md b/doc/topics/autodevops/requirements.md new file mode 100644 index 00000000000..b09a571fd16 --- /dev/null +++ b/doc/topics/autodevops/requirements.md @@ -0,0 +1,134 @@ +# Requirements for Auto DevOps + +You can set up Auto DevOps for [Kubernetes](#auto-devops-requirements-for-kubernetes) +or [Amazon Elastic Container Service (ECS)](#auto-devops-requirements-for-amazon-ecs). +For more information about Auto DevOps, see [the main Auto DevOps page](index.md) +or the [quick start guide](quick_start_guide.md). + +## Auto DevOps requirements for Kubernetes + +To make full use of Auto DevOps with Kubernetes, you need: + +- **Kubernetes** (for [Auto Review Apps](stages.md#auto-review-apps), + [Auto Deploy](stages.md#auto-deploy), and [Auto Monitoring](stages.md#auto-monitoring)) + + To enable deployments, you need: + + 1. A [Kubernetes 1.12+ cluster](../../user/project/clusters/index.md) for your + project. The easiest way is to create a + [new cluster using the GitLab UI](../../user/project/clusters/add_remove_clusters.md#create-new-cluster). + For Kubernetes 1.16+ clusters, you must perform additional configuration for + [Auto Deploy for Kubernetes 1.16+](stages.md#kubernetes-116). + 1. NGINX Ingress. You can deploy it to your Kubernetes cluster by installing + the [GitLab-managed app for Ingress](../../user/clusters/applications.md#ingress), + after configuring GitLab's Kubernetes integration in the previous step. + + Alternatively, you can use the + [`nginx-ingress`](https://github.com/helm/charts/tree/master/stable/nginx-ingress) + Helm chart to install Ingress manually. + + NOTE: **Note:** + If you use your own Ingress instead of the one provided by GitLab's managed + apps, ensure you're running at least version 0.9.0 of NGINX Ingress and + [enable Prometheus metrics](https://github.com/helm/charts/tree/master/stable/nginx-ingress#prometheus-metrics) + for the response metrics to appear. You must also + [annotate](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) + the NGINX Ingress deployment to be scraped by Prometheus using + `prometheus.io/scrape: "true"` and `prometheus.io/port: "10254"`. + +- **Base domain** (for [Auto Review Apps](stages.md#auto-review-apps), + [Auto Deploy](stages.md#auto-deploy), and [Auto Monitoring](stages.md#auto-monitoring)) + + You need a domain configured with wildcard DNS, which all of your Auto DevOps + applications will use. If you're using the + [GitLab-managed app for Ingress](../../user/clusters/applications.md#ingress), + the URL endpoint is automatically configured for you. + + You must also [specify the Auto DevOps base domain](index.md#auto-devops-base-domain). + +- **GitLab Runner** (for all stages) + + Your Runner must be configured to run Docker, usually with either the + [Docker](https://docs.gitlab.com/runner/executors/docker.html) + or [Kubernetes](https://docs.gitlab.com/runner/executors/kubernetes.html) executors, with + [privileged mode enabled](https://docs.gitlab.com/runner/executors/docker.html#use-docker-in-docker-with-privileged-mode). + The Runners don't need to be installed in the Kubernetes cluster, but the + Kubernetes executor is easy to use and automatically autoscales. + You can configure Docker-based Runners to autoscale as well, using + [Docker Machine](https://docs.gitlab.com/runner/install/autoscaling.html). + + If you've configured GitLab's Kubernetes integration in the first step, you + can deploy it to your cluster by installing the + [GitLab-managed app for GitLab Runner](../../user/clusters/applications.md#gitlab-runner). + + Runners should be registered as [shared Runners](../../ci/runners/README.md#shared-runners) + for the entire GitLab instance, or [specific Runners](../../ci/runners/README.md#specific-runners) + that are assigned to specific projects (the default if you've installed the + GitLab Runner managed application). + +- **Prometheus** (for [Auto Monitoring](stages.md#auto-monitoring)) + + To enable Auto Monitoring, you need Prometheus installed either inside or + outside your cluster, and configured to scrape your Kubernetes cluster. + If you've configured GitLab's Kubernetes integration, you can deploy it to + your cluster by installing the + [GitLab-managed app for Prometheus](../../user/clusters/applications.md#prometheus). + + The [Prometheus service](../../user/project/integrations/prometheus.md) + integration must be enabled for the project, or enabled as a + [default service template](../../user/project/integrations/services_templates.md) + for the entire GitLab installation. + + To get response metrics (in addition to system metrics), you must + [configure Prometheus to monitor NGINX](../../user/project/integrations/prometheus_library/nginx_ingress.md#configuring-nginx-ingress-monitoring). + +- **cert-manager** (optional, for TLS/HTTPS) + + To enable HTTPS endpoints for your application, you must install cert-manager, + a native Kubernetes certificate management controller that helps with issuing + certificates. Installing cert-manager on your cluster issues a + [Let’s Encrypt](https://letsencrypt.org/) certificate and ensures the + certificates are valid and up-to-date. If you've configured GitLab's Kubernetes + integration, you can deploy it to your cluster by installing the + [GitLab-managed app for cert-manager](../../user/clusters/applications.md#cert-manager). + +If you don't have Kubernetes or Prometheus installed, then +[Auto Review Apps](stages.md#auto-review-apps), +[Auto Deploy](stages.md#auto-deploy), and [Auto Monitoring](stages.md#auto-monitoring) +are skipped. + +After all requirements are met, you can [enable Auto DevOps](index.md#enablingdisabling-auto-devops). + +## Auto DevOps requirements for Amazon ECS + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/208132) in GitLab 13.0. + +You can choose to target [Amazon Elastic Container Service (ECS)](../../ci/cloud_deployment/index.md) as a deployment platform instead of using Kubernetes. + +To get started on Auto DevOps to Amazon ECS, you'll have to add a specific Environment +Variable. To do so, follow these steps: + +1. In your project, go to **Settings > CI / CD** and expand the **Variables** + section. + +1. Specify which AWS platform to target during the Auto DevOps deployment + by adding the `AUTO_DEVOPS_PLATFORM_TARGET` variable. + +1. Give this variable the value `ECS` before saving it. + +When you trigger a pipeline, if Auto DevOps is enabled and if you've correctly +[entered AWS credentials as environment variables](../../ci/cloud_deployment/index.md#deploy-your-application-to-the-aws-elastic-container-service-ecs), +your application will be deployed to Amazon ECS. + +NOTE: **Note:** +If you have both a valid `AUTO_DEVOPS_PLATFORM_TARGET` variable and a Kubernetes cluster tied to your project, +only the deployment to Kubernetes will run. + +CAUTION: **Warning:** +Setting the `AUTO_DEVOPS_PLATFORM_TARGET` variable to `ECS` will trigger jobs +defined in the [`Jobs/Deploy/ECS.gitlab-ci.yml` template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy/ECS.gitlab-ci.yml). +However, it's not recommended to [include](../../ci/yaml/README.md#includetemplate) +it on its own. This template is designed to be used with Auto DevOps only. It may change +unexpectedly causing your pipeline to fail if included on its own. Also, the job +names within this template may also change. Don't override these jobs' names in your +own pipeline, as the override will stop working when the name changes. diff --git a/doc/topics/autodevops/stages.md b/doc/topics/autodevops/stages.md index 8c56a87ba30..0c7c4919431 100644 --- a/doc/topics/autodevops/stages.md +++ b/doc/topics/autodevops/stages.md @@ -53,7 +53,9 @@ troubleshoot. Auto Build supports building your application using [Cloud Native Buildpacks](https://buildpacks.io) through the [`pack` command](https://github.com/buildpacks/pack). To use Cloud Native Buildpacks, -set the CI variable `AUTO_DEVOPS_BUILD_IMAGE_CNB_ENABLED` to a non-empty value. +set the CI variable `AUTO_DEVOPS_BUILD_IMAGE_CNB_ENABLED` to a non-empty +value. The default builder is `heroku/buildpacks:18` but a different builder +can be selected using the CI variable `AUTO_DEVOPS_BUILD_IMAGE_CNB_BUILDER`. Cloud Native Buildpacks (CNBs) are an evolution of Heroku buildpacks, and will eventually supersede Herokuish-based builds within Auto DevOps. For more @@ -82,11 +84,39 @@ Auto Test runs the appropriate tests for your application using your project to detect the language and framework. Several languages and frameworks are detected automatically, but if your language is not detected, you may be able to create a [custom buildpack](customize.md#custom-buildpacks). -Check the [currently supported languages](index.md#currently-supported-languages). +Check the [currently supported languages](#currently-supported-languages). Auto Test uses tests you already have in your application. If there are no tests, it's up to you to add them. +### Currently supported languages + +Note that not all buildpacks support Auto Test yet, as it's a relatively new +enhancement. All of Heroku's +[officially supported languages](https://devcenter.heroku.com/articles/heroku-ci#supported-languages) +support Auto Test. The languages supported by Heroku's Herokuish buildpacks all +support Auto Test, but notably the multi-buildpack does not. + +The supported buildpacks are: + +```plaintext +- heroku-buildpack-multi +- heroku-buildpack-ruby +- heroku-buildpack-nodejs +- heroku-buildpack-clojure +- heroku-buildpack-python +- heroku-buildpack-java +- heroku-buildpack-gradle +- heroku-buildpack-scala +- heroku-buildpack-play +- heroku-buildpack-php +- heroku-buildpack-go +- buildpack-nginx +``` + +If your application needs a buildpack that is not in the above list, you +might want to use a [custom buildpack](customize.md#custom-buildpacks). + ## Auto Code Quality **(STARTER)** Auto Code Quality uses the @@ -114,6 +144,22 @@ warnings. To learn more about [how SAST works](../../user/application_security/sast/index.md), see the documentation. +## Auto Secret Detection **(ULTIMATE)** + +> Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.1. + +Secret Detection uses the +[Secret Detection Docker image](https://gitlab.com/gitlab-org/security-products/analyzers/secrets) to run Secret Detection on the current code, and checks for leaked secrets. The +Auto Secret Detection stage runs only on the +[Ultimate](https://about.gitlab.com/pricing/) tier, and requires +[GitLab Runner](https://docs.gitlab.com/runner/) 11.5 or above. + +After creating the report, it's uploaded as an artifact which you can later +download and evaluate. The merge request widget also displays any security +warnings. + +To learn more, see [Secret Detection](../../user/application_security/secret_detection/index.md). + ## Auto Dependency Scanning **(ULTIMATE)** > Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.7. @@ -166,7 +212,7 @@ see the documentation. ## Auto Review Apps This is an optional step, since many projects don't have a Kubernetes cluster -available. If the [requirements](index.md#requirements) are not met, the job is +available. If the [requirements](requirements.md) are not met, the job is silently skipped. [Review Apps](../../ci/review_apps/index.md) are temporary application environments based on the @@ -267,7 +313,7 @@ Any performance differences between the source and target branches are also ## Auto Deploy This is an optional step, since many projects don't have a Kubernetes cluster -available. If the [requirements](index.md#requirements) are not met, the job is skipped. +available. If the [requirements](requirements.md) are not met, the job is skipped. After a branch or merge request is merged into the project's default branch (usually `master`), Auto Deploy deploys the application to a `production` environment in @@ -328,7 +374,7 @@ as it attempts to fetch the image using `CI_REGISTRY_PASSWORD`. CAUTION: **Deprecation** The default value for the `deploymentApiVersion` setting was changed from -`extensions/v1beta` to `apps/v1` in [GitLab 13.0](https://gitlab.com/gitlab-org/charts/auto-deploy-app/issues/47). +`extensions/v1beta` to `apps/v1` in [GitLab 13.0](https://gitlab.com/gitlab-org/charts/auto-deploy-app/-/issues/47). In Kubernetes 1.16 and later, a number of [APIs were removed](https://kubernetes.io/blog/2019/07/18/api-deprecations-in-1-16/), @@ -467,7 +513,7 @@ traffic within a local namespace, and from the `gitlab-managed-apps` namespace. All other inbound connections are blocked. Outbound traffic (for example, to the Internet) is not affected by the default policy. -You can also provide a custom [policy specification](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.16/#networkpolicyspec-v1-networking-k8s-io) +You can also provide a custom [policy specification](https://kubernetes.io/docs/concepts/services-networking/network-policies/) in the `.gitlab/auto-deploy-values.yaml` file, for example: ```yaml @@ -568,7 +614,7 @@ GitLab provides some initial alerts for you after you install Prometheus: To use Auto Monitoring: -1. [Install and configure the requirements](index.md#requirements). +1. [Install and configure the Auto DevOps requirements](requirements.md). 1. [Enable Auto DevOps](index.md#enablingdisabling-auto-devops), if you haven't done already. 1. Navigate to your project's **{rocket}** **CI/CD > Pipelines** and click **Run Pipeline**. 1. After the pipeline finishes successfully, open the |
