diff options
Diffstat (limited to 'doc/topics/autodevops')
| -rw-r--r-- | doc/topics/autodevops/cicd_variables.md | 31 | ||||
| -rw-r--r-- | doc/topics/autodevops/customize.md | 399 | ||||
| -rw-r--r-- | doc/topics/autodevops/multiple_clusters_auto_devops.md | 2 | ||||
| -rw-r--r-- | doc/topics/autodevops/stages.md | 4 | ||||
| -rw-r--r-- | doc/topics/autodevops/troubleshooting.md | 2 | ||||
| -rw-r--r-- | doc/topics/autodevops/upgrading_auto_deploy_dependencies.md | 2 | ||||
| -rw-r--r-- | doc/topics/autodevops/upgrading_postgresql.md | 4 |
7 files changed, 226 insertions, 218 deletions
diff --git a/doc/topics/autodevops/cicd_variables.md b/doc/topics/autodevops/cicd_variables.md index db2b052784d..169d34aad77 100644 --- a/doc/topics/autodevops/cicd_variables.md +++ b/doc/topics/autodevops/cicd_variables.md @@ -22,7 +22,7 @@ Use these variables to customize and deploy your build. | `AUTO_DEVOPS_ATOMIC_RELEASE` | As of GitLab 13.0, Auto DevOps uses [`--atomic`](https://v2.helm.sh/docs/helm/#options-43) for Helm deployments by default. Set this variable to `false` to disable the use of `--atomic` | | `AUTO_DEVOPS_BUILD_IMAGE_CNB_ENABLED` | Set to `false` to use Herokuish instead of Cloud Native Buildpacks with Auto Build. [More details](stages.md#auto-build-using-cloud-native-buildpacks). | | `AUTO_DEVOPS_BUILD_IMAGE_CNB_BUILDER` | The builder used when building with Cloud Native Buildpacks. The default builder is `heroku/buildpacks:18`. [More details](stages.md#auto-build-using-cloud-native-buildpacks). | -| `AUTO_DEVOPS_BUILD_IMAGE_EXTRA_ARGS` | Extra arguments to be passed to the `docker build` command. Using quotes doesn't prevent word splitting. [More details](customize.md#passing-arguments-to-docker-build). | +| `AUTO_DEVOPS_BUILD_IMAGE_EXTRA_ARGS` | Extra arguments to be passed to the `docker build` command. Using quotes doesn't prevent word splitting. [More details](customize.md#pass-arguments-to-docker-build). | | `AUTO_DEVOPS_BUILD_IMAGE_FORWARDED_CI_VARIABLES` | A [comma-separated list of CI/CD variable names](customize.md#forward-cicd-variables-to-the-build-environment) to be forwarded to the build environment (the buildpack builder or `docker build`). | | `AUTO_DEVOPS_BUILD_IMAGE_CNB_PORT` | In GitLab 15.0 and later, port exposed by the generated Docker image. Set to `false` to prevent exposing any ports. Defaults to `5000`. | | `AUTO_DEVOPS_CHART` | Helm Chart used to deploy your apps. Defaults to the one [provided by GitLab](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/tree/master/assets/auto-deploy-app). | @@ -42,9 +42,9 @@ Use these variables to customize and deploy your build. | `CI_APPLICATION_REPOSITORY` | The repository of container image being built or deployed, `$CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG`. For more details, read [Custom container image](customize.md#custom-container-image). | | `CI_APPLICATION_TAG` | The tag of the container image being built or deployed, `$CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG`. For more details, read [Custom container image](customize.md#custom-container-image). | | `DAST_AUTO_DEPLOY_IMAGE_VERSION` | Customize the image version used for DAST deployments on the default branch. Should usually be the same as `AUTO_DEPLOY_IMAGE_VERSION`. See [list of versions](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/releases). | -| `DOCKERFILE_PATH` | From GitLab 13.2, allows overriding the [default Dockerfile path for the build stage](customize.md#custom-dockerfile) | -| `HELM_RELEASE_NAME` | From GitLab 12.1, allows the `helm` release name to be overridden. Can be used to assign unique release names when deploying multiple projects to a single namespace. | -| `HELM_UPGRADE_VALUES_FILE` | From GitLab 12.6, allows the `helm upgrade` values file to be overridden. Defaults to `.gitlab/auto-deploy-values.yaml`. | +| `DOCKERFILE_PATH` | From GitLab 13.2, allows overriding the [default Dockerfile path for the build stage](customize.md#custom-dockerfiles) | +| `HELM_RELEASE_NAME` | Allows the `helm` release name to be overridden. Can be used to assign unique release names when deploying multiple projects to a single namespace. | +| `HELM_UPGRADE_VALUES_FILE` | Allows the `helm upgrade` values file to be overridden. Defaults to `.gitlab/auto-deploy-values.yaml`. | | `HELM_UPGRADE_EXTRA_ARGS` | Allows extra options in `helm upgrade` commands when deploying the application. Using quotes doesn't prevent word splitting. | | `INCREMENTAL_ROLLOUT_MODE` | If present, can be used to enable an [incremental rollout](#incremental-rollout-to-production) of your application for the production environment. Set to `manual` for manual deployment jobs or `timed` for automatic rollout deployments with a 5 minute delay each one. | | `K8S_SECRET_*` | Any variable prefixed with [`K8S_SECRET_`](#configure-application-secret-variables) is made available by Auto DevOps as environment variables to the deployed application. | @@ -53,27 +53,31 @@ Use these variables to customize and deploy your build. | `KUBE_NAMESPACE` | The namespace used for deployments. When using certificate-based clusters, [this value should not be overwritten directly](../../user/project/clusters/deploy_to_cluster.md#custom-namespace). | | `KUBECONFIG` | The kubeconfig to use for deployments. User-provided values take priority over GitLab-provided values. | | `PRODUCTION_REPLICAS` | Number of replicas to deploy in the production environment. Takes precedence over `REPLICAS` and defaults to 1. For zero downtime upgrades, set to 2 or greater. | -| `REPLICAS` | Number of replicas to deploy. Defaults to 1. Change this variable instead of [modifying](customize.md#customize-values-for-helm-chart) `replicaCount`. | +| `REPLICAS` | Number of replicas to deploy. Defaults to 1. Change this variable instead of [modifying](customize.md#customize-helm-chart-values) `replicaCount`. | | `ROLLOUT_RESOURCE_TYPE` | Allows specification of the resource type being deployed when using a custom Helm chart. Default value is `deployment`. | -| `ROLLOUT_STATUS_DISABLED` | From GitLab 12.0, used to disable rollout status check because it does not support all resource types, for example, `cronjob`. | +| `ROLLOUT_STATUS_DISABLED` | Used to disable rollout status check because it does not support all resource types, for example, `cronjob`. | | `STAGING_ENABLED` | Used to define a [deploy policy for staging and production environments](#deploy-policy-for-staging-and-production-environments). | | `TRACE` | Set to any value to make Helm commands produce verbose output. You can use this setting to help diagnose Auto DevOps deployment problems. | ## Database variables +WARNING: +The default value of `true` for `POSTGRES_ENABLED` was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/387766) +in GitLab 15.8. In [GitLab 16.0](https://gitlab.com/gitlab-org/gitlab/-/issues/343988), the default value will change to `false`. + Use these variables to integrate CI/CD with PostgreSQL databases. | **CI/CD variable** | **Description** | |-----------------------------------------|------------------------------------| | `DB_INITIALIZE` | Used to specify the command to run to initialize the application's PostgreSQL database. Runs inside the application pod. | | `DB_MIGRATE` | Used to specify the command to run to migrate the application's PostgreSQL database. Runs inside the application pod. | -| `POSTGRES_ENABLED` | Whether PostgreSQL is enabled. Defaults to `true`. Set to `false` to disable the automatic deployment of PostgreSQL. | +| `POSTGRES_ENABLED` | Whether PostgreSQL is enabled. Defaults to `true` until GitLab 16.0, when the default will change to `false`. Set to `false` to disable the automatic deployment of PostgreSQL. | | `POSTGRES_USER` | The PostgreSQL user. Defaults to `user`. Set it to use a custom username. | | `POSTGRES_PASSWORD` | The PostgreSQL password. Defaults to `testing-password`. Set it to use a custom password. | | `POSTGRES_DB` | The PostgreSQL database name. Defaults to the value of [`$CI_ENVIRONMENT_SLUG`](../../ci/variables/index.md#predefined-cicd-variables). Set it to use a custom database name. | | `POSTGRES_VERSION` | Tag for the [`postgres` Docker image](https://hub.docker.com/_/postgres) to use. Defaults to `9.6.16` for tests and deployments as of GitLab 13.0 (previously `9.6.2`). If `AUTO_DEVOPS_POSTGRES_CHANNEL` is set to `1`, deployments uses the default version `9.6.2`. | | `POSTGRES_HELM_UPGRADE_VALUES_FILE` | In GitLab 13.8 and later, and when using [auto-deploy-image v2](upgrading_auto_deploy_dependencies.md), this variable allows the `helm upgrade` values file for PostgreSQL to be overridden. Defaults to `.gitlab/auto-deploy-postgres-values.yaml`. | -| `POSTGRES_HELM_UPGRADE_EXTRA_ARGS` | In GitLab 13.8 and later, and when using [auto-deploy-image v2](upgrading_auto_deploy_dependencies.md), this variable allows extra PostgreSQL options in `helm upgrade` commands when deploying the application. Note that using quotes doesn't prevent word splitting. | +| `POSTGRES_HELM_UPGRADE_EXTRA_ARGS` | In GitLab 13.8 and later, and when using [auto-deploy-image v2](upgrading_auto_deploy_dependencies.md), this variable allows extra PostgreSQL options in `helm upgrade` commands when deploying the application. Using quotes doesn't prevent word splitting. | | `POSTGRES_CHART_REPOSITORY` | Helm Chart repository used to search for PostgreSQL chart. Defaults to `https://raw.githubusercontent.com/bitnami/charts/eb5f9a9513d987b519f0ecd732e7031241c50328/bitnami`. | | `POSTGRES_CHART_VERSION` | Helm Chart version used for PostgreSQL chart. Defaults to `8.2.1`. | @@ -134,6 +138,10 @@ Auto DevOps detects CI/CD variables starting with `K8S_SECRET_`, and makes them available to the deployed application as environment variables. +Prerequisite: + +- The variable value must be a single line. + To configure secret variables: 1. On the top bar, select **Main menu > Projects** and find your project. @@ -190,7 +198,7 @@ limitations with the Auto DevOps scripting environment. Add replica variables when you want to scale your deployments: -1. Add a replica variable as a [project CI/CD variable](../../ci/variables/index.md#add-a-cicd-variable-to-a-project). +1. Add a replica variable as a [project CI/CD variable](../../ci/variables/index.md#for-a-project). 1. To scale your application, redeploy it. WARNING: @@ -290,7 +298,7 @@ You can run the rollout jobs in any order. To scale down, rerun a lower percentage job. After you run the `rollout 100%` job, you cannot scale down, and must -[rollback your deployment](../../ci/environments/index.md#retry-or-roll-back-a-deployment). +[roll back your deployment](../../ci/environments/index.md#retry-or-roll-back-a-deployment). ### Example incremental rollout configurations @@ -310,9 +318,6 @@ With `INCREMENTAL_ROLLOUT_MODE` set to `manual` and with `STAGING_ENABLED`:  -WARNING: -This configuration is deprecated, and is scheduled to be removed in the future. - ## Timed incremental rollout to production **(PREMIUM)** Use a timed incremental rollout to continuously deploy your application, starting with diff --git a/doc/topics/autodevops/customize.md b/doc/topics/autodevops/customize.md index c42f5825b6a..776112d6303 100644 --- a/doc/topics/autodevops/customize.md +++ b/doc/topics/autodevops/customize.md @@ -4,23 +4,46 @@ group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Customizing Auto DevOps **(FREE)** +# Customize Auto DevOps **(FREE)** -While [Auto DevOps](index.md) provides great defaults to get you started, you can customize -almost everything to fit your needs. Auto DevOps offers everything from custom -[buildpacks](#custom-buildpacks), to [Dockerfiles](#custom-dockerfile), and -[Helm charts](#custom-helm-chart). You can even copy the complete -[CI/CD configuration](#customizing-gitlab-ciyml) into your project to enable -staging and canary deployments, -[manage Auto DevOps with GitLab APIs](customize.md#extend-auto-devops-with-the-api), and more. +You can customize components of Auto DevOps to fit your needs. For example, you can: + +- Add custom [buildpacks](#custom-buildpacks), [Dockerfiles](#custom-dockerfiles), and [Helm charts](#custom-helm-chart). +- Enable staging and canary deployments with a custom [CI/CD configuration](#customize-gitlab-ciyml). +- Extend Auto DevOps with the [GitLab API](#extend-auto-devops-with-the-api). + +## Auto DevOps banner + +When Auto DevOps is not enabled, a banner displays for users with at +least the Maintainer role: + + + +The banner can be disabled for: + +- A user, when they dismiss it themselves. +- A project, by explicitly [disabling Auto DevOps](index.md#enable-or-disable-auto-devops). +- An entire GitLab instance: + - By an administrator running the following in a Rails console: + + ```ruby + Feature.enable(:auto_devops_banner_disabled) + ``` + + - Through the REST API with an administrator access token: + + ```shell + curl --data "value=true" --header "PRIVATE-TOKEN: <personal_access_token>" "https://gitlab.example.com/api/v4/features/auto_devops_banner_disabled" + ``` ## Custom buildpacks -If the automatic buildpack detection fails for your project, or if you -need more control over your build, you can customize the buildpacks -used for the build. +You can customize your buildpacks when either: -### Custom buildpacks with Cloud Native Buildpacks +- The automatic buildpack detection fails for your project. +- You need more control over your build. + +### Customize buildpacks with Cloud Native Buildpacks > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28165) in GitLab 12.10. @@ -29,17 +52,18 @@ Specify either: - The CI/CD variable `BUILDPACK_URL` with any of [`pack`'s URI specification formats](https://buildpacks.io/docs/app-developer-guide/specify-buildpacks/). - A [`project.toml` project descriptor](https://buildpacks.io/docs/app-developer-guide/using-project-descriptor/) with the buildpacks you would like to include. -### Custom buildpacks with Herokuish +### Customize buildpacks with Herokuish Specify either: - The CI/CD variable `BUILDPACK_URL`. -- A `.buildpacks` file at the root of your project, containing one buildpack URL per line. +- A `.buildpacks` file at the root of your project that contains one buildpack URL per line. The buildpack URL can point to either a Git repository URL or a tarball URL. -For Git repositories, you can point to a specific Git reference (such as -commit SHA, tag name, or branch name) by appending `#<ref>` to the Git repository URL. -For example: + +For Git repositories, you can point to a specific Git reference by +appending `#<ref>` to the Git repository URL. For example, you can +reference: - The tag `v142`: `https://github.com/heroku/heroku-buildpack-ruby.git#v142`. - The branch `mybranch`: `https://github.com/heroku/heroku-buildpack-ruby.git#mybranch`. @@ -47,35 +71,38 @@ For example: ### Multiple buildpacks -Using multiple buildpacks is not fully supported by Auto DevOps, because Auto Test -can't use the `.buildpacks` file. The buildpack -[heroku-buildpack-multi](https://github.com/heroku/heroku-buildpack-multi/), used -in the backend to parse the `.buildpacks` file, does not provide the necessary commands -`bin/test-compile` and `bin/test`. +Because Auto Test cannot use the `.buildpacks` file, Auto DevOps does +not support multiple buildpacks. The buildpack +[heroku-buildpack-multi](https://github.com/heroku/heroku-buildpack-multi/), +used in the backend to parse the `.buildpacks` file, does not provide +the necessary commands `bin/test-compile` and `bin/test`. -If your goal is to use only a single custom buildpack, you should provide the project CI/CD variable +To use only a single custom buildpack, you should provide the project CI/CD variable `BUILDPACK_URL` instead. -## Custom `Dockerfile` +## Custom Dockerfiles -> Support for `DOCKERFILE_PATH` was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/35662) in GitLab 13.2 +> `DOCKERFILE_PATH` [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/35662) in GitLab 13.2. -If your project has a `Dockerfile` in the root of the project repository, Auto DevOps -builds a Docker image based on the Dockerfile, rather than using buildpacks. -This can be much faster and result in smaller images, especially if your -Dockerfile is based on [Alpine](https://hub.docker.com/_/alpine/). +If you have a Dockerfile in the root of your project repository, Auto +DevOps builds a Docker image based on the Dockerfile. This can be +faster than using a buildpack. It can also result in smaller images, +especially if your Dockerfile is based on +[Alpine](https://hub.docker.com/_/alpine/). If you set the `DOCKERFILE_PATH` CI/CD variable, Auto Build looks for a Dockerfile there instead. -## Passing arguments to `docker build` +### Pass arguments to `docker build` + +You can pass arguments to `docker build` with the +`AUTO_DEVOPS_BUILD_IMAGE_EXTRA_ARGS` project CI/CD variable. -Arguments can be passed to the `docker build` command using the -`AUTO_DEVOPS_BUILD_IMAGE_EXTRA_ARGS` project CI/CD variable. For example, to build a -Docker image based on based on the `ruby:alpine` instead of the default `ruby:latest`: +For example, to build a Docker image based on based on the +`ruby:alpine` instead of the default `ruby:latest`: 1. Set `AUTO_DEVOPS_BUILD_IMAGE_EXTRA_ARGS` to `--build-arg=RUBY_VERSION=alpine`. -1. Add the following to a custom `Dockerfile`: +1. Add the following to a custom Dockerfile: ```dockerfile ARG RUBY_VERSION=latest @@ -84,14 +111,12 @@ Docker image based on based on the `ruby:alpine` instead of the default `ruby:la # ... put your stuff here ``` -Use Base64 encoding if you need to pass complex values, such as newlines and -spaces. Left unencoded, complex values like these can cause escaping issues -due to how Auto DevOps uses the arguments. +To pass complex values like spaces and newlines, use Base64 encoding. +Complex, unencoded values can cause issues with character escaping. WARNING: -Avoid passing secrets as Docker build arguments if possible, as they may be -persisted in your image. See -[this discussion of best practices with secrets](https://github.com/moby/moby/issues/13490) for details. +Do not pass secrets as Docker build arguments. Secrets might persist in your image. For more information, see +[this discussion of best practices with secrets](https://github.com/moby/moby/issues/13490). ## Custom container image @@ -104,12 +129,12 @@ You can override this behavior by defining specific variables: | Image Tag | `$CI_COMMIT_SHA` for branch pipelines. `$CI_COMMIT_TAG` for tag pipelines. | `$CI_APPLICATION_TAG` | These variables also affect Auto Build and Auto Container Scanning. If you don't want to build and push an image to -`$CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG`, consider -including only `Jobs/Deploy.gitlab-ci.yml`, or [disabling the `build` jobs](cicd_variables.md#job-disabling-variables). +`$CI_APPLICATION_REPOSITORY:$CI_APPLICATION_TAG`, include only `Jobs/Deploy.gitlab-ci.yml`, or +[disable the `build` jobs](cicd_variables.md#job-disabling-variables). If you use Auto Container Scanning and set a value for `$CI_APPLICATION_REPOSITORY`, then you should -also update `$CS_DEFAULT_BRANCH_IMAGE`. See [Setting the default branch image](../../user/application_security/container_scanning/index.md#setting-the-default-branch-image) -for more details. +also update `$CS_DEFAULT_BRANCH_IMAGE`. For more information, see +[Setting the default branch image](../../user/application_security/container_scanning/index.md#setting-the-default-branch-image). Here is an example setup in your `.gitlab-ci.yml`: @@ -123,141 +148,129 @@ variables: You can extend and manage your Auto DevOps configuration with GitLab APIs: -- [Settings that can be accessed with API calls](../../api/settings.md#list-of-settings-that-can-be-accessed-via-api-calls), +- [Use API calls to access settings](../../api/settings.md#list-of-settings-that-can-be-accessed-via-api-calls), which include `auto_devops_enabled`, to enable Auto DevOps on projects by default. -- [Creating a new project](../../api/projects.md#create-project). -- [Editing groups](../../api/groups.md#update-group). -- [Editing projects](../../api/projects.md#edit-project). +- [Create a new project](../../api/projects.md#create-project). +- [Edit groups](../../api/groups.md#update-group). +- [Edit projects](../../api/projects.md#edit-project). ## Forward CI/CD variables to the build environment -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25514) in GitLab 12.3, but available in GitLab 12.0 and later. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25514) in GitLab 12.3. -CI/CD variables can be forwarded into the build environment using the -`AUTO_DEVOPS_BUILD_IMAGE_FORWARDED_CI_VARIABLES` CI/CD variable. -The forwarded variables should be specified by name in a comma-separated -list. For example, to forward the variables `CI_COMMIT_SHA` and -`CI_ENVIRONMENT_NAME`, set `AUTO_DEVOPS_BUILD_IMAGE_FORWARDED_CI_VARIABLES` -to `CI_COMMIT_SHA,CI_ENVIRONMENT_NAME`. +To forward CI/CD variables to the build environment, add the names of the variables +you want to forward to the `AUTO_DEVOPS_BUILD_IMAGE_FORWARDED_CI_VARIABLES` CI/CD variable. +Separate multiple variables with commas. -- When using Buildpacks, the forwarded variables are available automatically - as environment variables. -- When using a `Dockerfile`, the following additional steps are required: +For example, to forward the variables `CI_COMMIT_SHA` and `CI_ENVIRONMENT_NAME`: - 1. Activate the experimental `Dockerfile` syntax by adding the following code - to the top of the file: +```yaml +variables: + AUTO_DEVOPS_BUILD_IMAGE_FORWARDED_CI_VARIABLES: CI_COMMIT_SHA,CI_ENVIRONMENT_NAME +``` + +If you use buildpacks, the forwarded variables are available automatically as environment variables. - ```dockerfile - # syntax = docker/dockerfile:experimental - ``` +If you use a Dockerfile: - 1. To make secrets available in any `RUN $COMMAND` in the `Dockerfile`, mount - the secret file and source it prior to running `$COMMAND`: +1. To activate the experimental Dockerfile syntax, add the following to your Dockerfile: + + ```dockerfile + # syntax = docker/dockerfile:experimental + ``` - ```dockerfile - RUN --mount=type=secret,id=auto-devops-build-secrets . /run/secrets/auto-devops-build-secrets && $COMMAND - ``` +1. To make secrets available in any `RUN $COMMAND` in the `Dockerfile`, mount + the secret file and source it before you run `$COMMAND`: + + ```dockerfile + RUN --mount=type=secret,id=auto-devops-build-secrets . /run/secrets/auto-devops-build-secrets && $COMMAND + ``` When `AUTO_DEVOPS_BUILD_IMAGE_FORWARDED_CI_VARIABLES` is set, Auto DevOps enables the experimental [Docker BuildKit](https://docs.docker.com/build/buildkit/) feature to use the `--secret` flag. -## Custom Helm Chart +## Custom Helm chart Auto DevOps uses [Helm](https://helm.sh/) to deploy your application to Kubernetes. -You can override the Helm chart used by bundling up a chart into your project +You can override the Helm chart used by bundling a chart in your project repository or by specifying a project CI/CD variable: - **Bundled chart** - If your project has a `./chart` directory with a `Chart.yaml` file in it, Auto DevOps detects the chart and uses it instead of the - [default chart](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/tree/master/assets/auto-deploy-app), enabling - you to control exactly how your application is deployed. + [default chart](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/tree/master/assets/auto-deploy-app). - **Project variable** - Create a [project CI/CD variable](../../ci/variables/index.md) - `AUTO_DEVOPS_CHART` with the URL of a custom chart to use, or create two project - variables: `AUTO_DEVOPS_CHART_REPOSITORY` with the URL of a custom chart repository, - and `AUTO_DEVOPS_CHART` with the path to the chart. + `AUTO_DEVOPS_CHART` with the URL of a custom chart. You can also create two project + variables: -## Customize values for Helm Chart + - `AUTO_DEVOPS_CHART_REPOSITORY` - The URL of a custom chart repository. + - `AUTO_DEVOPS_CHART` - The path to the chart. + +### Customize Helm chart values > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30628) in GitLab 12.6, `.gitlab/auto-deploy-values.yaml` is used by default for Helm upgrades. -You can override the default values in the `values.yaml` file in the -[default Helm chart](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/tree/master/assets/auto-deploy-app) by either: +To override the default values in the `values.yaml` file in the +[default Helm chart](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/tree/master/assets/auto-deploy-app), either: -- Adding a file named `.gitlab/auto-deploy-values.yaml` to your repository, which is - automatically used, if found. -- Adding a file with a different name or path to the repository, and setting the - `HELM_UPGRADE_VALUES_FILE` [CI/CD variable](cicd_variables.md) with - the path and name. +- Add a file named `.gitlab/auto-deploy-values.yaml` to your repository. This file is automatically used. +- Add a file with a different name or path to the repository. Set the + `HELM_UPGRADE_VALUES_FILE` [CI/CD variable](cicd_variables.md) with the path and name of the file. -Some values cannot be overridden with the options above. Settings like `replicaCount` should instead be overridden with the `REPLICAS` -[build and deployment](cicd_variables.md#build-and-deployment-variables) CI/CD variable. Follow [this issue](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/issues/31) for more information. +Some values cannot be overridden with the options above, but [this issue](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/issues/31) proposes to change this behavior. +To override settings like `replicaCount`, use the `REPLICAS` [build and deployment](cicd_variables.md#build-and-deployment-variables) CI/CD variable. -NOTE: -For GitLab 12.5 and earlier, use the `HELM_UPGRADE_EXTRA_ARGS` variable -to override the default chart values by setting `HELM_UPGRADE_EXTRA_ARGS` to `--values <my-values.yaml>`. +### Customize `helm upgrade` -## Customize the `helm upgrade` command +The [auto-deploy-image](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image) uses the `helm upgrade` command. +To customize this command, pass it options with the `HELM_UPGRADE_EXTRA_ARGS` CI/CD variable. -You can customize the `helm upgrade` command used in the [auto-deploy-image](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image) -by passing options to the command with the `HELM_UPGRADE_EXTRA_ARGS` CI/CD variable. -For example, set the value of `HELM_UPGRADE_EXTRA_ARGS` to `--no-hooks` to disable -pre-upgrade and post-upgrade hooks when the command is executed. +For example, to disable pre- and post-upgrade hooks when `helm upgrade` runs: -See [the official documentation](https://helm.sh/docs/helm/helm_upgrade/) for the full -list of options. +```yaml +variables: + HELM_UPGRADE_EXTRA_ARGS: --no-hooks +``` -## Custom Helm chart per environment +For a full list of options, see [the official `helm upgrade` documentation](https://helm.sh/docs/helm/helm_upgrade/). -You can specify the use of a custom Helm chart per environment by scoping the CI/CD variable -to the desired environment. See [Limit environment scope of CI/CD variables](../../ci/variables/index.md#limit-the-environment-scope-of-a-cicd-variable). +### Limit a Helm chart to one environment -## Customizing `.gitlab-ci.yml` +To limit a custom chart to one environment, add the environment scope to your CI/CD variables. +For more information, see [Limit the environment scope of CI/CD variables](../../ci/environments/index.md#limit-the-environment-scope-of-a-cicd-variable). -Auto DevOps is completely customizable because the -[Auto DevOps template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml) -is just an implementation of a [`.gitlab-ci.yml`](../../ci/yaml/index.md) file, -and uses only features available to any implementation of `.gitlab-ci.yml`. +## Customize `.gitlab-ci.yml` -To modify the CI/CD pipeline used by Auto DevOps, -[`include` the template](../../ci/yaml/index.md#includetemplate), and customize -it as needed by adding a `.gitlab-ci.yml` file to the root of your repository -containing the following: +Auto DevOps is highly customizable because the [Auto DevOps template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml) +is an implementation of a [`.gitlab-ci.yml`](../../ci/yaml/index.md) file. +The template uses only features available to any implementation of `.gitlab-ci.yml`. -```yaml -include: - - template: Auto-DevOps.gitlab-ci.yml -``` +To add custom behaviors to the CI/CD pipeline used by Auto DevOps: -Add your changes, and your additions are merged with the -[Auto DevOps template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml) -using the behavior described for [`include`](../../ci/yaml/index.md#include). +1. To the root of your repository, add a `.gitlab-ci.yml` file with the following contents: -If you need to specifically remove a part of the file, you can also copy and paste the contents of the -[Auto DevOps template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml) -into your project and edit it as needed. + ```yaml + include: + - template: Auto-DevOps.gitlab-ci.yml + ``` -## Use multiple Kubernetes clusters +1. Add your changes to the `.gitlab-ci.yml` file. Your changes are merged with the Auto DevOps template. For more information about +how `include` merges your changes, see [the `include` documentation](../../ci/yaml/index.md#include). -See [Multiple Kubernetes clusters for Auto DevOps](multiple_clusters_auto_devops.md). +To remove behaviors from the Auto DevOps pipeline: -## Customizing the Kubernetes namespace +1. Copy the [Auto DevOps template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml) +into your project. +1. Edit your copy of the template as needed. -In GitLab 14.5 and earlier, you could use `environment:kubernetes:namespace` -to specify a namespace for the environment. -However, this feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8), -along with certificate-based integration. - -You should now use the `KUBE_NAMESPACE` environment variable and -[limit the environments it is available for](../../ci/environments/index.md#scope-environments-with-specs). +### Use individual components of Auto DevOps -## Using components of Auto DevOps +If you only require a subset of the features offered by Auto DevOps, +you can include individual Auto DevOps jobs in your own +`.gitlab-ci.yml`. Be sure to also define the stage required by each +job in your `.gitlab-ci.yml` file. -If you only require a subset of the features offered by Auto DevOps, you can include -individual Auto DevOps jobs into your own `.gitlab-ci.yml`. Each component job relies -on a stage that should be defined in the `.gitlab-ci.yml` that includes the template. - -For example, to make use of [Auto Build](stages.md#auto-build), you can add the following to +For example, to use [Auto Build](stages.md#auto-build), you can add the following to your `.gitlab-ci.yml`: ```yaml @@ -268,26 +281,39 @@ include: - template: Jobs/Build.gitlab-ci.yml ``` -See the [Auto DevOps template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml) for information on available jobs. +For a list of available jobs, see the [Auto DevOps template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml). WARNING: -Auto DevOps templates using the [`only`](../../ci/yaml/index.md#only--except) or +From [GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/issues/213336), +Auto DevOps templates that use the [`only`](../../ci/yaml/index.md#only--except) or [`except`](../../ci/yaml/index.md#only--except) syntax have switched -to the [`rules`](../../ci/yaml/index.md#rules) syntax, starting in -[GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/issues/213336). -If your `.gitlab-ci.yml` extends these Auto DevOps templates and override the `only` or -`except` keywords, you must migrate your templates to use the -[`rules`](../../ci/yaml/index.md#rules) syntax after the -base template is migrated to use the `rules` syntax. -For users who cannot migrate just yet, you can alternatively pin your templates to +to the [`rules`](../../ci/yaml/index.md#rules) syntax. +If your `.gitlab-ci.yml` extends these Auto DevOps templates and overrides `only` or +`except`, migrate your templates to the +[`rules`](../../ci/yaml/index.md#rules) syntax. +If you cannot migrate, you can pin your templates to the [GitLab 12.10 based templates](https://gitlab.com/gitlab-org/auto-devops-v12-10). +## Use multiple Kubernetes clusters + +See [Multiple Kubernetes clusters for Auto DevOps](multiple_clusters_auto_devops.md). + +## Customizing the Kubernetes namespace + +In GitLab 14.5 and earlier, you could use `environment:kubernetes:namespace` +to specify a namespace for the environment. +However, this feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8), +along with certificate-based integration. + +You should now use the `KUBE_NAMESPACE` environment variable and +[limit its environment scope](../../ci/environments/index.md#limit-the-environment-scope-of-a-cicd-variable). + ## Use images hosted in a local Docker registry You can configure many Auto DevOps jobs to run in an [offline environment](../../user/application_security/offline_deployments/index.md): 1. Copy the required Auto DevOps Docker images from Docker Hub and `registry.gitlab.com` to their local GitLab container registry. -1. After the images are hosted and available in a local registry, edit `.gitlab-ci.yml` to point to the locally-hosted images. For example: +1. After the images are hosted and available in a local registry, edit `.gitlab-ci.yml` to point to the locally hosted images. For example: ```yaml include: @@ -305,10 +331,18 @@ You can configure many Auto DevOps jobs to run in an [offline environment](../.. ## PostgreSQL database support -To support applications requiring a database, -[PostgreSQL](https://www.postgresql.org/) is provisioned by default. The credentials to access -the database are preconfigured, but can be customized by setting the associated -[CI/CD variables](cicd_variables.md). You can use these credentials to define a `DATABASE_URL`: +WARNING: +Provisioning a PostgreSQL database by default was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/387766) +in GitLab 15.8 and will no longer be the default from 16.0. To enable database provisioning, set +the associated [CI/CD variable](cicd_variables.md#database-variables). + +To support applications that require a database, +[PostgreSQL](https://www.postgresql.org/) is provisioned by default. +The credentials to access the database are preconfigured. + +To customize the credentials, set the associated +[CI/CD variables](cicd_variables.md). You can also +define a custom `DATABASE_URL`: ```yaml postgres://user:password@postgres-host:postgres-port/postgres-database @@ -316,24 +350,20 @@ postgres://user:password@postgres-host:postgres-port/postgres-database ### Upgrading PostgreSQL -WARNING: -The CI/CD variable `AUTO_DEVOPS_POSTGRES_CHANNEL` that controls default provisioned -PostgreSQL was changed to `2` in [GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/issues/210499). -To keep using the old PostgreSQL, set the `AUTO_DEVOPS_POSTGRES_CHANNEL` variable to -`1`. - -The version of the chart used to provision PostgreSQL: - -- Is 8.2.1 in GitLab 13.0 and later, but can be set back to 0.7.1 if needed. -- Can be set to from 0.7.1 to 8.2.1 in GitLab 12.9 and 12.10. -- Is 0.7.1 in GitLab 12.8 and earlier. +GitLab uses chart version 8.2.1 to provision PostgreSQL by default. +You can set the version from 0.7.1 to 8.2.1. -GitLab encourages users to [migrate their database](upgrading_postgresql.md) +If you use an older chart version, you should [migrate your database](upgrading_postgresql.md) to the newer PostgreSQL. +The CI/CD variable `AUTO_DEVOPS_POSTGRES_CHANNEL` that controls default provisioned +PostgreSQL changed to `2` in [GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/issues/210499). +To use the old PostgreSQL, set the `AUTO_DEVOPS_POSTGRES_CHANNEL` variable to +`1`. + ### Customize values for PostgreSQL Helm Chart -> [Introduced](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/issues/113) in auto-deploy-image v2, in GitLab 13.8. +> [Introduced](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/issues/113) in GitLab 13.8 with auto-deploy-image v2. To set custom values, do one of the following: @@ -344,53 +374,26 @@ To set custom values, do one of the following: and name. - Set the `POSTGRES_HELM_UPGRADE_EXTRA_ARGS` [environment variable](cicd_variables.md#database-variables). -### Using external PostgreSQL database providers +### Use external PostgreSQL database providers -While Auto DevOps provides out-of-the-box support for a PostgreSQL container for -production environments, for some use cases, it may not be sufficiently secure or -resilient, and you may want to use an external managed provider (such as -AWS Relational Database Service) for PostgreSQL. +Auto DevOps provides out-of-the-box support for a PostgreSQL container +for production environments. However, you might want to use an +external managed provider like AWS Relational Database Service. -You must define environment-scoped CI/CD variables for `POSTGRES_ENABLED` and -`DATABASE_URL` in your project's CI/CD settings: +To use an external managed provider: -1. Disable the built-in PostgreSQL installation for the required environments using - environment-scoped [CI/CD variables](../../ci/environments/index.md#scope-environments-with-specs). - For this use case, it's likely that only `production` must be added to this - list. The built-in PostgreSQL setup for Review Apps and staging is sufficient. +1. Disable the built-in PostgreSQL installation for the required environments with + environment-scoped [CI/CD variables](../../ci/environments/index.md#limit-the-environment-scope-of-a-cicd-variable). + Because the built-in PostgreSQL setup for Review Apps and staging is sufficient, you might only need to + disable the installation for `production`.  -1. Define the `DATABASE_URL` variable as an environment-scoped variable that is +1. Define the `DATABASE_URL` variable as an environment-scoped variable available to your application. This should be a URL in the following format: ```yaml postgres://user:password@postgres-host:postgres-port/postgres-database ``` -You must ensure that your Kubernetes cluster has network access to wherever -PostgreSQL is hosted. - -## Auto DevOps banner - -The following Auto DevOps banner displays for users with Maintainer or greater -permissions on new projects when Auto DevOps is not enabled: - - - -The banner can be disabled for: - -- A user, when they dismiss it themselves. -- A project, by explicitly [disabling Auto DevOps](index.md#enable-or-disable-auto-devops). -- An entire GitLab instance: - - By an administrator running the following in a Rails console: - - ```ruby - Feature.enable(:auto_devops_banner_disabled) - ``` - - - Through the REST API with an administrator access token: - - ```shell - curl --data "value=true" --header "PRIVATE-TOKEN: <personal_access_token>" "https://gitlab.example.com/api/v4/features/auto_devops_banner_disabled" - ``` +1. Ensure your Kubernetes cluster has network access to where PostgreSQL is hosted. diff --git a/doc/topics/autodevops/multiple_clusters_auto_devops.md b/doc/topics/autodevops/multiple_clusters_auto_devops.md index cf775a35eb7..3411beedefb 100644 --- a/doc/topics/autodevops/multiple_clusters_auto_devops.md +++ b/doc/topics/autodevops/multiple_clusters_auto_devops.md @@ -26,7 +26,7 @@ To deploy your environments to different Kubernetes clusters: 1. [Configure each agent to access your project](../../user/clusters/agent/install/index.md#configure-your-agent). 1. [Install NGINX Ingress Controller](cloud_deployments/auto_devops_with_gke.md#install-ingress) in each cluster. Save the IP address and Kubernetes namespace for the next step. 1. [Configure the Auto DevOps CI/CD Pipeline variables](cicd_variables.md#build-and-deployment-variables) - - Set up a `KUBE_CONTEXT` variable [for each environment](../../ci/variables/index.md#limit-the-environment-scope-of-a-cicd-variable). The value must point to the agent of the relevant cluster. + - Set up a `KUBE_CONTEXT` variable [for each environment](../../ci/environments/index.md#limit-the-environment-scope-of-a-cicd-variable). The value must point to the agent of the relevant cluster. - Set up a `KUBE_INGRESS_BASE_DOMAIN`. You must [configure the base domain](requirements.md#auto-devops-base-domain) for each environment to point to the Ingress of the relevant cluster. - Add a `KUBE_NAMESPACE` variable with a value of the Kubernetes namespace you want your deployments to target. You can scope the variable to multiple environments. diff --git a/doc/topics/autodevops/stages.md b/doc/topics/autodevops/stages.md index 0ca9c6fa3de..aba0b2d7fae 100644 --- a/doc/topics/autodevops/stages.md +++ b/doc/topics/autodevops/stages.md @@ -451,7 +451,7 @@ To use Auto Deploy on a Kubernetes 1.16+ cluster: 1. If you are deploying your application for the first time in GitLab 13.0 or later, no configuration should be required. -1. In GitLab 12.10 and earlier, set the following in the [`.gitlab/auto-deploy-values.yaml` file](customize.md#customize-values-for-helm-chart): +1. In GitLab 12.10 and earlier, set the following in the [`.gitlab/auto-deploy-values.yaml` file](customize.md#customize-helm-chart-values): ```yaml deploymentApiVersion: apps/v1 @@ -535,7 +535,7 @@ you must: After configuring your worker to respond to health checks, run a Sidekiq worker for your Rails application. You can enable workers by setting the -following in the [`.gitlab/auto-deploy-values.yaml` file](customize.md#customize-values-for-helm-chart): +following in the [`.gitlab/auto-deploy-values.yaml` file](customize.md#customize-helm-chart-values): ```yaml workers: diff --git a/doc/topics/autodevops/troubleshooting.md b/doc/topics/autodevops/troubleshooting.md index ef420323b32..dd715e95512 100644 --- a/doc/topics/autodevops/troubleshooting.md +++ b/doc/topics/autodevops/troubleshooting.md @@ -230,7 +230,7 @@ LAST SEEN TYPE REASON OBJECT ``` To change the port used for the liveness checks, pass -[custom values to the Helm chart](customize.md#customize-values-for-helm-chart) +[custom values to the Helm chart](customize.md#customize-helm-chart-values) used by Auto DevOps: 1. Create a directory and file at the root of your repository named `.gitlab/auto-deploy-values.yaml`. diff --git a/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md b/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md index 4fafc89cac1..858562eef48 100644 --- a/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md +++ b/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md @@ -116,7 +116,7 @@ If your Auto DevOps project has an active environment that was deployed with the 1. Deploy your environment as usual. This deployment uses Helm v3. 1. If the deployment succeeds, you can safely run `<environment-name>:helm-2to3:cleanup`. This deletes all Helm v2 release data from the namespace. -1. Remove the `MIGRATE_HELM_2TO3` CI/CD variable or set it to `false`. You can do this one environment at a time using [environment scopes](../../ci/environments/index.md#scope-environments-with-specs). +1. Remove the `MIGRATE_HELM_2TO3` CI/CD variable or set it to `false`. You can do this one environment at a time using [environment scopes](../../ci/environments/index.md#limit-the-environment-scope-of-a-cicd-variable). #### In-Cluster PostgreSQL Channel 2 diff --git a/doc/topics/autodevops/upgrading_postgresql.md b/doc/topics/autodevops/upgrading_postgresql.md index d43b1cae9e9..f18d5c49be5 100644 --- a/doc/topics/autodevops/upgrading_postgresql.md +++ b/doc/topics/autodevops/upgrading_postgresql.md @@ -174,7 +174,7 @@ deleted, you can choose to retain the [persistent volume](#retain-persistent-vol NOTE: You can also -[scope](../../ci/environments/index.md#scope-environments-with-specs) the +[scope](../../ci/environments/index.md#limit-the-environment-scope-of-a-cicd-variable) the `AUTO_DEVOPS_POSTGRES_CHANNEL`, `AUTO_DEVOPS_POSTGRES_DELETE_V1` and `POSTGRES_VERSION` variables to specific environments, for example, `staging`. @@ -189,7 +189,7 @@ higher*. This is the minimum PostgreSQL version supported by Auto DevOps. See also the list of [tags available](https://hub.docker.com/r/bitnami/postgresql/tags). 1. Set `PRODUCTION_REPLICAS` to `0`. For other environments, use - `REPLICAS` with an [environment scope](../../ci/environments/index.md#scope-environments-with-specs). + `REPLICAS` with an [environment scope](../../ci/environments/index.md#limit-the-environment-scope-of-a-cicd-variable). 1. If you have set the `DB_INITIALIZE` or `DB_MIGRATE` variables, either remove the variables, or rename the variables temporarily to `XDB_INITIALIZE` or the `XDB_MIGRATE` to effectively disable them. |