diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2020-09-29 18:09:52 +0000 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2020-09-29 18:09:52 +0000 |
commit | 5c2377d19572bdaaa6026fa6be131c3649074b36 (patch) | |
tree | 04acff23fd41360701c29c8694d24e587bc7ac5a /doc | |
parent | 20fda899a62cc27a4d40a168640e7e926c69eb62 (diff) | |
download | gitlab-ce-5c2377d19572bdaaa6026fa6be131c3649074b36.tar.gz |
Add latest changes from gitlab-org/gitlab@master
Diffstat (limited to 'doc')
-rw-r--r-- | doc/api/container_registry.md | 2 | ||||
-rw-r--r-- | doc/ci/docker/using_docker_build.md | 3 | ||||
-rw-r--r-- | doc/ci/docker/using_docker_images.md | 3 | ||||
-rw-r--r-- | doc/ci/pipelines/job_artifacts.md | 11 | ||||
-rw-r--r-- | doc/development/documentation/index.md | 42 | ||||
-rw-r--r-- | doc/topics/autodevops/customize.md | 2 | ||||
-rw-r--r-- | doc/topics/autodevops/stages.md | 2 | ||||
-rw-r--r-- | doc/topics/autodevops/upgrading_auto_deploy_dependencies.md | 262 | ||||
-rw-r--r-- | doc/topics/autodevops/upgrading_chart.md | 75 | ||||
-rw-r--r-- | doc/user/application_security/container_scanning/index.md | 4 | ||||
-rw-r--r-- | doc/user/application_security/dast/index.md | 4 | ||||
-rw-r--r-- | doc/user/application_security/dependency_scanning/index.md | 4 | ||||
-rw-r--r-- | doc/user/application_security/index.md | 9 | ||||
-rw-r--r-- | doc/user/application_security/sast/index.md | 4 | ||||
-rw-r--r-- | doc/user/application_security/secret_detection/index.md | 6 | ||||
-rw-r--r-- | doc/user/project/clusters/index.md | 47 |
16 files changed, 370 insertions, 110 deletions
diff --git a/doc/api/container_registry.md b/doc/api/container_registry.md index 71a5cf56684..366c0b3ddc0 100644 --- a/doc/api/container_registry.md +++ b/doc/api/container_registry.md @@ -24,7 +24,6 @@ GET /projects/:id/registry/repositories | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) accessible by the authenticated user. | | `tags` | boolean | no | If the parameter is included as true, each repository will include an array of `"tags"` in the response. | -| `name` | string | no | Returns a list of repositories with a name that matches the value. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/29763) in GitLab 13.0). | | `tags_count` | boolean | no | If the parameter is included as true, each repository will include `"tags_count"` in the response ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/32141) in GitLab 13.1). | ```shell @@ -68,7 +67,6 @@ GET /groups/:id/registry/repositories | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) accessible by the authenticated user. | | `tags` | boolean | no | If the parameter is included as true, each repository will include an array of `"tags"` in the response. | -| `name` | string | no | Returns a list of repositories with a name that matches the value. ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/29763) in GitLab 13.0). | | `tags_count` | boolean | no | If the parameter is included as true, each repository will include `"tags_count"` in the response ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/32141) in GitLab 13.1). | ```shell diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 34ca7f8ea7c..a319c5f09ab 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -48,7 +48,6 @@ The simplest approach is to install GitLab Runner in `shell` execution mode. GitLab Runner then executes job scripts as the `gitlab-runner` user. 1. Install [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-runner/#installation). - 1. During GitLab Runner installation select `shell` as method of executing job scripts or use command: ```shell @@ -158,7 +157,6 @@ Runner is installed using the [Helm chart](https://docs.gitlab.com/runner/instal See the [related issue](https://gitlab.com/gitlab-org/charts/gitlab-runner/-/issues/83) for details. 1. Install [GitLab Runner](https://docs.gitlab.com/runner/install/). - 1. Register GitLab Runner from the command line to use `docker` and `privileged` mode: @@ -318,7 +316,6 @@ are done to the services as well, making these incompatible. In order to do that, follow the steps: 1. Install [GitLab Runner](https://docs.gitlab.com/runner/install/). - 1. Register GitLab Runner from the command line to use `docker` and share `/var/run/docker.sock`: ```shell diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 62b87c2e858..185de159416 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -678,7 +678,6 @@ To add `DOCKER_AUTH_CONFIG` to a runner: - The double quotes included in the `DOCKER_AUTH_CONFIG` data must be escaped with backslashes. This prevents them from being interpreted as TOML. - - The `environment` option is a list. Your runner may have existing entries and you should add this to the list, not replace it. @@ -725,10 +724,8 @@ image which is private and requires you to log in into a private container regis To configure access for `aws_account_id.dkr.ecr.region.amazonaws.com`, follow these steps: 1. Make sure `docker-credential-ecr-login` is available in GitLab Runner's `$PATH`. - 1. Have any of the following [AWS credentials setup](https://github.com/awslabs/amazon-ecr-credential-helper#aws-credentials). Make sure that GitLab Runner can access the credentials. - 1. Make GitLab Runner use it. There are two ways to accomplish this. Either: - Create a [variable](../variables/README.md#gitlab-cicd-environment-variables) diff --git a/doc/ci/pipelines/job_artifacts.md b/doc/ci/pipelines/job_artifacts.md index 750a76bfaa0..39633163ac7 100644 --- a/doc/ci/pipelines/job_artifacts.md +++ b/doc/ci/pipelines/job_artifacts.md @@ -431,6 +431,17 @@ To erase a job: In order to retrieve a job artifact of a different project, you might need to use a private token in order to [authenticate and download](../../api/job_artifacts.md#get-job-artifacts) the artifacts. +## Troubleshooting + +### Error message `No files to upload` + +This is often preceded by other errors or warnings that specify the filename and why it wasn't +generated in the first place. Please check the entire job log for such messages. + +If you find no helpful messages, please retry the failed job after activating +[CI debug logging](../variables/README.md#debug-logging). +This provides useful information to investigate further. + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/development/documentation/index.md b/doc/development/documentation/index.md index dc1cf3ec466..3a02ea5aa83 100644 --- a/doc/development/documentation/index.md +++ b/doc/development/documentation/index.md @@ -1,4 +1,7 @@ --- +stage: none +group: Documentation Guidelines +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers description: Learn how to contribute to GitLab Documentation. --- @@ -128,7 +131,7 @@ The following metadata should be added when a page is moved to another location: - `redirect_to`: The relative path and filename (with an `.md` extension) of the location to which visitors should be redirected for a moved page. - [Learn more](#changing-document-location). + [Learn more](#move-or-rename-a-page). - `disqus_identifier`: Identifier for Disqus commenting system. Used to keep comments with a page that's been moved to a new URL. [Learn more](#redirections-for-pages-with-disqus-comments). @@ -156,17 +159,18 @@ Nanoc layout), which will be displayed at the top of the page if defined: [algorithm](https://gitlab.com/gitlab-org/gitlab-docs/-/blob/master/lib/helpers/reading_time.rb) to calculate the reading time based on the number of words. -## Changing document location +## Move or rename a page + +Moving or renaming a document is the same as changing its location. This process +requires specific steps to ensure that visitors can find the new +documentation page, whether they're using `/help` from a GitLab instance or by +visiting <https://docs.gitlab.com>. -Changing a document's location requires specific steps to ensure that -users can seamlessly access the new doc page, whether they are accessing content -on a GitLab instance domain at `/help` or at <https://docs.gitlab.com>. Be sure to assign a -technical writer if you have any questions during the process (such as -whether the move is necessary), and ensure that a technical writer reviews this -change prior to merging. +Be sure to assign a technical writer to a page move or rename MR. Technical +Writers can help with any questions and can review your change. -If you indeed need to change a document's location, do not remove the old -document, but instead replace all of its content with the following: +To change a document's location, don't remove the old document, but instead +replace all of its content with the following: ```markdown --- @@ -176,14 +180,18 @@ redirect_to: '../path/to/file/index.md' This document was moved to [another location](../path/to/file/index.md). ``` -Where `../path/to/file/index.md` is usually the relative path to the old document. +Replace `../path/to/file/index.md` with the relative path to the old document. + +The `redirect_to` variable supports both full and relative URLs; for example: + +- `https://docs.gitlab.com/ee/path/to/file.html` +- `../path/to/file.html` +- `path/to/file.md` -The `redirect_to` variable supports both full and relative URLs, for example -`https://docs.gitlab.com/ee/path/to/file.html`, `../path/to/file.html`, `path/to/file.md`. -It ensures that the redirect will work for <https://docs.gitlab.com> and any `*.md` paths -will be compiled to `*.html`. -The new line underneath the front matter informs the user that the document -changed location and is useful for someone that browses that file from the repository. +The redirect works for <https://docs.gitlab.com>, and any `*.md` paths are +changed to `*.html`. The description line following the `redirect_to` code +informs the visitor that the document changed location if the redirect process +doesn't complete successfully. For example, if you move `doc/workflow/lfs/index.md` to `doc/administration/lfs.md`, then the steps would be: diff --git a/doc/topics/autodevops/customize.md b/doc/topics/autodevops/customize.md index 13aa8f7e035..45083dfeb64 100644 --- a/doc/topics/autodevops/customize.md +++ b/doc/topics/autodevops/customize.md @@ -332,7 +332,7 @@ applications. | `AUTO_DEVOPS_CHART_REPOSITORY_USERNAME` | From GitLab 11.11, used to set a username to connect to the Helm repository. Defaults to no credentials. Also set `AUTO_DEVOPS_CHART_REPOSITORY_PASSWORD`. | | `AUTO_DEVOPS_CHART_REPOSITORY_PASSWORD` | From GitLab 11.11, used to set a password to connect to the Helm repository. Defaults to no credentials. Also set `AUTO_DEVOPS_CHART_REPOSITORY_USERNAME`. | | `AUTO_DEVOPS_DEPLOY_DEBUG` | From GitLab 13.1, if this variable is present, Helm will output debug logs. | -| `AUTO_DEVOPS_ALLOW_TO_FORCE_DEPLOY_V<N>` | From [auto-deploy-image](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image) v1.0.0, if this variable is present, a new major version of chart is forcibly deployed. [More details](upgrading_chart.md#ignore-warning-and-continue-deploying) | +| `AUTO_DEVOPS_ALLOW_TO_FORCE_DEPLOY_V<N>` | From [auto-deploy-image](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image) v1.0.0, if this variable is present, a new major version of chart is forcibly deployed. For more information, see [Ignore warnings and continue deploying](upgrading_auto_deploy_dependencies.md#ignore-warnings-and-continue-deploying). | | `AUTO_DEVOPS_MODSECURITY_SEC_RULE_ENGINE` | From GitLab 12.5, used in combination with [ModSecurity feature flag](../../user/clusters/applications.md#web-application-firewall-modsecurity) to toggle [ModSecurity's `SecRuleEngine`](https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual-(v2.x)#SecRuleEngine) behavior. Defaults to `DetectionOnly`. | | `BUILDPACK_URL` | Buildpack's full URL. Can point to either [a Git repository URL or a tarball URL](#custom-buildpacks). | | `CANARY_ENABLED` | From GitLab 11.0, used to define a [deploy policy for canary environments](#deploy-policy-for-canary-environments). | diff --git a/doc/topics/autodevops/stages.md b/doc/topics/autodevops/stages.md index b58c369714e..0e9f0812a9a 100644 --- a/doc/topics/autodevops/stages.md +++ b/doc/topics/autodevops/stages.md @@ -466,7 +466,7 @@ application runs. ### Upgrade auto-deploy-app Chart -You can upgrade auto-deploy-app chart by following the [upgrade guide](upgrading_chart.md). +You can upgrade the auto-deploy-app chart by following the [upgrade guide](upgrading_auto_deploy_dependencies.md). ### Workers diff --git a/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md b/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md new file mode 100644 index 00000000000..1aefb6b34df --- /dev/null +++ b/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md @@ -0,0 +1,262 @@ +--- +stage: Release +group: Progressive Delivery +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +type: reference +--- + +# Upgrading deployments for newer Auto Deploy dependencies (Auto Deploy template, auto-deploy-image and auto-deploy-app chart) + +[Auto Deploy](stages.md#auto-deploy) is a feature that deploys your application to a Kubernetes cluster. +It consists of several dependencies: + +- [Auto Deploy template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml) is a set of pipeline jobs and scripts that makes use of `auto-deploy-image`. +- [`auto-deploy-image`](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image) is the executable image that communicates with the Kubernetes cluster. +- [`auto-deploy-app chart`](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/tree/master/assets/auto-deploy-app) is the Helm chart for deploying your application. + +The `auto-deploy-image` and `auto-deploy-app` charts use [Semantic Versioning](https://semver.org/). +By default, your Auto DevOps project keeps using the stable and non-breaking version. +However, these dependencies could be upgraded in a major version release of GitLab +with breaking changes requiring you to upgrade your deployments. + +This guide explains how to upgrade your deployments with newer or different major versions of Auto Deploy dependencies. + +## Verify dependency versions + +The process to check the current versions differs depending on which template you +are using. First verify which template is in use: + +- For self-managed instances, the [stable Auto Deploy template bundled with the GitLab package](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml) + is being used. +- [The GitLab.com stable Auto Deploy template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml) + is being used if **one** of the following is true: + - Your Auto DevOps project doesn't have a `.gitlab-ci.yml` file. + - Your Auto DevOps project has a `.gitlab-ci.yml` and [includes](../../ci/yaml/README.md#includetemplate) + the `Auto-DevOps.gitlab-ci.yml` template. +- [The latest Auto Deploy template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.latest.gitlab-ci.yml) + is being used if **both** of the following is true: + - Your Auto DevOps project has a `.gitlab-ci.yml` file and [includes](../../ci/yaml/README.md#includetemplate) + the `Auto-DevOps.gitlab-ci.yml` template. + - It also includes [the latest Auto Deploy template](#early-adopters) + +If you know what template is being used: + +- The `auto-deploy-image` version is in the template (for example `auto-deploy-image:v1.0.3`). +- The `auto-deploy-app` chart version is [in the auto-deploy-image repository](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/blob/v1.0.3/assets/auto-deploy-app/Chart.yaml) + (for example `version: 1.0.3`). + +## Compatibility + +The following table explains the version compatibility between GitLab and Auto Deploy dependencies: + +| GitLab version | `auto-deploy-image` version | Notes | +|------------------|-----------------------------|-------| +| v10.0 to v14.0 | v0.1.0 to v2.0.0 | v0 and v1 auto-deploy-image are backwards compatible. | +| v13.4 and higher | v2.0.0 and higher | v2 auto-deploy-image contains breaking changes, as explained in the [upgrade guide](#upgrade-deployments-to-the-v2-auto-deploy-image). | + +You can find the current stable version of auto-deploy-image in the [Auto Deploy stable template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml). + +## Upgrade Guide + +Projects using Auto DevOps must use the unmodified chart managed by GitLab. +[Customized charts](customize.md#custom-helm-chart) are unsupported. + +### Upgrade deployments to the v1 `auto-deploy-image` + +The v1 chart is backward compatible with the v0 chart, so no configuration changes are needed. + +### Upgrade deployments to the v2 `auto-deploy-image` + +The v2 auto-deploy-image contains multiple dependency and architectural changes. +If your Auto DevOps project has an active environment deployed with the v1 `auto-deploy-image`, +please proceed with the following upgrade guide. Otherwise, you can skip this process. + +#### Helm 3 + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/228609) in GitLab 13.4. + +The `auto-deploy-image` uses the Helm binary to manipulate the releases. +Previously, `auto-deploy-image` used Helm v2, which used Tiller in a cluster. +In the v2 `auto-deploy-image`, it uses Helm v3 that doesn't require Tiller anymore. + +If your Auto DevOps project has an active environment that was deployed with the v1 +`auto-deploy-image`, use the following steps to upgrade to v2, which uses Helm 3: + +1. Modify your `.gitlab-ci.yml` with: + + ```yaml + include: + - template: Auto-DevOps.gitlab-ci.yml + - remote: https://gitlab.com/hfyngvason/ci-templates/-/raw/master/Helm-2to3.gitlab-ci.yml + + variables: + # If this variable is not present, the migration jobs will not show up + MIGRATE_HELM_2TO3: "true" + + .auto-deploy: + image: registry.gitlab.com/gitlab-org/cluster-integration/auto-deploy-image:v2.0.0-beta.1 + variables: + AUTO_DEVOPS_FORCE_DEPLOY_V2: 1 + ``` + +1. Run the `<environment-name>:helm-2to3:migrate` job. +1. Deploy your environment as usual. This deployment uses Helm 3. +1. If the deployment succeeds, you can safely run `environment:helm-2to3:cleanup`. + This deletes all Helm 2 release data from the namespace. + + If you accidentally delete the Helm 2 releases before you are ready, the `<environment-name>:helm2to3:migrate` + job saves a backup for 1 week in a job artifact called `helm-2-release-backups`. + The backup is in a Kubernetes manifest file that can be restored using + `kubectl apply -f $backup`. +1. Remove the `MIGRATE_HELM_2TO3` variable. + +#### Traffic routing change for canary deployments and incremental rollouts + +> [Introduced](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/merge_requests/109) in GitLab 13.4. + +Auto Deploy supports advanced deployment strategies such as [canary deployments](customize.md#deploy-policy-for-canary-environments) +and [incremental rollouts](../../ci/environments/incremental_rollouts.md). + +Previously, `auto-deploy-image` created one service to balance the traffic between +unstable and stable tracks by changing the replica ratio. In the v2 `auto-deploy-image`, +it controls the traffic with [Canary Ingress](https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#canary). + +For more details, see the [v2 `auto-deploy-app` chart resource architecture](#v2-chart-resource-architecture). + +If your Auto DevOps project has active `canary` or `rollout` track releases in the +`production` environment deployed with the v1 `auto-deploy-image`, use the following +steps to upgrade to v2: + +1. Verify your project is [using the v1 `auto-deploy-image`](#verify-dependency-versions). + If not, [specify the version](#use-a-specific-version-of-auto-deploy-dependencies). +1. If you're in the process of deploying `canary` or `rollout` deployments, promote + them to `production` first to delete the unstable tracks. +1. Verify your project is [using the v2 `auto-deploy-image`](#verify-dependency-versions). + If not, [specify the version](#use-a-specific-version-of-auto-deploy-dependencies). +1. Add an `AUTO_DEVOPS_FORCE_DEPLOY_V2` environment variable with a value of `true` + in the GitLab CI/CD settings. +1. Create a new pipeline and run the `production` job to renew the resource architecture + with the v2 `auto-deploy-app chart`. +1. Remove the `AUTO_DEVOPS_FORCE_DEPLOY_V2` environment variable. + +### Use a specific version of Auto Deploy dependencies + +To use a specifc version of Auto Deploy dependencies, specify the previous Auto Deploy +stable template that contains the [desired version of `auto-deploy-image` and `auto-deploy-app`](#verify-dependency-versions). + +For example, if the template is bundled in GitLab v13.3, change your `.gitlab-ci.yml` to: + +```yaml +include: + - template: Auto-DevOps.gitlab-ci.yml + - remote: https://gitlab.com/gitlab-org/gitlab/-/blob/v13.3.0-ee/lib/gitlab/ci/templates/Jobs/Deploy.gitlab-ci.yml +``` + +### Ignore warnings and continue deploying + +If you are certain that the new chart version is safe to be deployed, you can add +the `AUTO_DEVOPS_FORCE_DEPLOY_V<major-version-number>` [environment variable](customize.md#build-and-deployment) +to force the deployment to continue. + +For example, if you want to deploy the `v2.0.0` chart on a deployment that previously +used the `v0.17.0` chart, add `AUTO_DEVOPS_FORCE_DEPLOY_V2`. + +## Early adopters + +If you want to use the latest beta or unstable version of `auto-deploy-image`, include +the latest Auto Deploy template into your `.gitlab-ci.yml`: + +```yaml +include: + - template: Auto-DevOps.gitlab-ci.yml + - remote: https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Jobs/Deploy.latest.gitlab-ci.yml +``` + +CAUTION: **Warning:** +Using a beta or unstable `auto-deploy-image` could cause unrecoverable damage to +your environments. Do not test it with important projects or environments. + +The next stable template update is [planned for GitLab v14.0](https://gitlab.com/gitlab-org/gitlab/-/issues/232788). + +## Resource Architectures of the `auto-deploy-app` chart + +### v0 and v1 chart resource architecture + +```mermaid +graph TD; +subgraph gl-managed-app +Z[Nginx Ingress] +end +Z[Nginx Ingress] --> A(Ingress); +Z[Nginx Ingress] --> B(Ingress); +subgraph stg namespace +B[Ingress] --> H(...); +end +subgraph prd namespace +A[Ingress] --> D(Service); +D[Service] --> E(Deployment:Pods:app:stable); +D[Service] --> F(Deployment:Pods:app:canary); +D[Service] --> I(Deployment:Pods:app:rollout); +E(Deployment:Pods:app:stable)---id1[(Pods:Postgres)] +F(Deployment:Pods:app:canary)---id1[(Pods:Postgres)] +I(Deployment:Pods:app:rollout)---id1[(Pods:Postgres)] +end +``` + +### v2 chart resource architecture + +```mermaid +graph TD; +subgraph gl-managed-app +Z[Nginx Ingress] +end +Z[Nginx Ingress] --> A(Ingress); +Z[Nginx Ingress] --> B(Ingress); +Z[Nginx Ingress] --> |If canary is present or incremental rollout/|J(Canary Ingress); +subgraph stg namespace +B[Ingress] --> H(...); +end +subgraph prd namespace +subgraph stable track +A[Ingress] --> D[Service]; +D[Service] --> E(Deployment:Pods:app:stable); +end +subgraph canary track +J(Canary Ingress) --> K[Service] +K[Service] --> F(Deployment:Pods:app:canary); +end +E(Deployment:Pods:app:stable)---id1[(Pods:Postgres)] +F(Deployment:Pods:app:canary)---id1[(Pods:Postgres)] +end +``` + +## Troubleshooting + +### Major version mismatch warning + +If deploying a chart that has a major version that is different from the previous one, +the new chart might not be correctly deployed. This could be due to an architectural +change. If that happens, the deployment job fails with a message similar to: + +```plaintext +************************************************************************************* + [WARNING] +Detected a major version difference between the the chart that is currently deploying (auto-deploy-app-v0.7.0), and the previously deployed chart (auto-deploy-app-v1.0.0). +A new major version might not be backward compatible with the current release (production). The deployment could fail or be stuck in an unrecoverable status. +... +``` + +To clear this error message and resume deployments, you must do one of the following: + +- Manually [upgrade the chart version](#upgrade-guide). +- [Use a specific chart version](#use-a-specific-version-of-auto-deploy-dependencies). + +### Error: `missing key "app.kubernetes.io/managed-by": must be set to "Helm"` + +If your cluster has a deployment that was deployed with the v1 `auto-deploy-image`, +you might encounter the following error: + +- `Error: rendered manifests contain a resource that already exists. Unable to continue with install: Secret "production-postgresql" in namespace "<project-name>-production" exists and cannot be imported into the current release: invalid ownership metadata; label validation error: missing key "app.kubernetes.io/managed-by": must be set to "Helm"; annotation validation error: missing key "meta.helm.sh/release-name": must be set to "production-postgresql"; annotation validation error: missing key "meta.helm.sh/release-namespace": must be set to "<project-name>-production"` + +This is because the previous deployment was deployed with Helm2, which is not compatible with Helm3. +To resolve the problem, please follow the [upgrade guide](#upgrade-deployments-to-the-v2-auto-deploy-image). diff --git a/doc/topics/autodevops/upgrading_chart.md b/doc/topics/autodevops/upgrading_chart.md index ffa485f6d2c..e4fb84d4509 100644 --- a/doc/topics/autodevops/upgrading_chart.md +++ b/doc/topics/autodevops/upgrading_chart.md @@ -1,72 +1,5 @@ -# Upgrading auto-deploy-app chart for Auto DevOps +--- +redirect_to: 'upgrading_auto_deploy_dependencies.md' +--- -Auto DevOps provides the auto-deploy-app chart for deploying your application to the -Kubernetes cluster with Helm/Tiller. Major version changes of this chart could have -a significantly different resource architecture, and may not be backwards compatible. - -This guide provides instructions on how to upgrade your deployments to use the latest -chart and resource architecture. - -## Compatibility - -The following table lists the version compatibility between GitLab and [auto-deploy-image](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image) (with the [auto-deploy-app chart](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/tree/master/assets/auto-deploy-app)). - -| GitLab version | auto-deploy-image version | Notes | -|------------------|---------------------------|--------------------------------------------| -| v10.0 and higher | v0.1.0 and higher | v0 and v1 charts are backwards compatible. | - -## Upgrade Guide - -The Auto DevOps project must use the unmodified chart managed by GitLab. -[Customized charts](customize.md#custom-helm-chart) are unsupported. - -### v1 chart - -The v1 chart is backward compatible with the v0 chart, so no configuration changes are needed. - -## Troubleshooting - -### Major version mismatch warning - -If deploying a chart that has a major version that is different from the previous one, -the new chart might not be correctly deployed. This could be due to an architectural -change. If that happens, the deployment job fails with a message similar to: - -```plaintext -************************************************************************************* - [WARNING] -Detected a major version difference between the the chart that is currently deploying (auto-deploy-app-v0.7.0), and the previously deployed chart (auto-deploy-app-v1.0.0). -A new major version might not be backward compatible with the current release (production). The deployment could fail or be stuck in an unrecoverable status. -... -``` - -To clear this error message and resume deployments, you must do one of the following: - -- Manually [upgrade the chart version](#upgrade-guide). -- [Use a specific chart version](#use-a-specific-chart-version). - -#### Use a specific chart version - -To use a specific chart version, you must specify a corresponding version of [auto-deploy-image](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image). -Do this by [customizing the image in your `.gitlab-ci.yml`](customize.md#customizing-gitlab-ciyml). - -For example, create the following `.gitlab-ci.yml` file in the project. It configures Auto DevOps -to use [auto-deploy-image](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image) version `v0.17.0` -for deployment jobs. It will download the chart from [chart repository](https://charts.gitlab.io/): - -```yaml -include: - - template: Auto-DevOps.gitlab-ci.yml - -.auto-deploy: - image: "registry.gitlab.com/gitlab-org/cluster-integration/auto-deploy-image:v0.17.0" -``` - -#### Ignore warning and continue deploying - -If you are certain that the new chart version is safe to be deployed, -you can add the `AUTO_DEVOPS_FORCE_DEPLOY_V<N>` [environment variable](customize.md#build-and-deployment) -to force the deployment to continue, where `<N>` is the major version. - -For example, if you want to deploy the v2.0.0 chart on a deployment that previously -used the v0.17.0 chart, add `AUTO_DEVOPS_FORCE_DEPLOY_V2`. +This document was moved to [another location](upgrading_auto_deploy_dependencies.md). diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md index d207edf3656..fec90a2bdf3 100644 --- a/doc/user/application_security/container_scanning/index.md +++ b/doc/user/application_security/container_scanning/index.md @@ -433,3 +433,7 @@ This is a result of a bug in Docker which is now [fixed](https://github.com/cont To prevent the error, ensure the Docker version that the runner is using is `18.09.03` or higher. For more information, see [issue #10241](https://gitlab.com/gitlab-org/gitlab/-/issues/10241 "Investigate why Container Scanning is not working with NFS mounts"). + +### Getting warning message `gl-container-scanning-report.json: no matching files` + +For information on this, see the [general Application Security troubleshooting section](../../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload). diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md index dea4665f5f1..1eeba0ceb0e 100644 --- a/doc/user/application_security/dast/index.md +++ b/doc/user/application_security/dast/index.md @@ -897,6 +897,10 @@ Change the number after `-Xmx` to the required memory amount. If your DAST job exceeds the job timeout and you need to reduce the scan duration, we shared some tips for optimizing DAST scans in a [blog post](https://about.gitlab.com/blog/2020/08/31/how-to-configure-dast-full-scans-for-complex-web-applications/). +### Getting warning message `gl-dast-report.json: no matching files` + +For information on this, see the [general Application Security troubleshooting section](../../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload). + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md index 88a498641f6..67803f2f16d 100644 --- a/doc/user/application_security/dependency_scanning/index.md +++ b/doc/user/application_security/dependency_scanning/index.md @@ -485,6 +485,10 @@ Consider updating to Docker `19.03.1` or greater. Older versions are not affected. Read more in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/13830#note_211354992 "Current SAST container fails"). +### Getting warning message `gl-dependency-scanning-report.json: no matching files` + +For information on this, see the [general Application Security troubleshooting section](../../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload). + ### Limitation when using rules:exists The [dependency scanning CI template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/Dependency-Scanning.gitlab-ci.yml) diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md index c5b729aa9e7..413a9f894e2 100644 --- a/doc/user/application_security/index.md +++ b/doc/user/application_security/index.md @@ -458,6 +458,15 @@ To fix this issue, you can either: [Learn more on overriding SAST jobs](sast/index.md#overriding-sast-jobs). All the security scanning tools define their stage, so this error can occur with all of them. +### Getting warning messages `… report.json: no matching files` + +This is often followed by the [error `No files to upload`](../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload), +and preceded by other errors or warnings that indicate why the JSON report wasn't generated. Please +check the entire job log for such messages. If you don't find these messages, retry the failed job +after setting `SECURE_LOG_LEVEL: "debug"` as a +[custom environment variable](../../ci/variables/README.md#custom-environment-variables). +This provides useful information to investigate further. + ### Getting error message `sast job: config key may not be used with 'rules': only/except` When [including](../../ci/yaml/README.md#includetemplate) a `.gitlab-ci.yml` template diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index be01dca8d9a..d1e3c2b5b15 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -547,3 +547,7 @@ This error occurs when the Docker version that runs the SAST job is `19.03.0`. Consider updating to Docker `19.03.1` or greater. Older versions are not affected. Read more in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/13830#note_211354992 "Current SAST container fails"). + +### Getting warning message `gl-sast-report.json: no matching files` + +For information on this, see the [general Application Security troubleshooting section](../../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload). diff --git a/doc/user/application_security/secret_detection/index.md b/doc/user/application_security/secret_detection/index.md index f3e411cdc16..3c1e9e0f3cd 100644 --- a/doc/user/application_security/secret_detection/index.md +++ b/doc/user/application_security/secret_detection/index.md @@ -197,3 +197,9 @@ We have created a [short video walkthrough](https://youtu.be/wDtc_K00Y0A) showca <figure class="video-container"> <iframe src="https://www.youtube.com/embed/wDtc_K00Y0A" frameborder="0" allowfullscreen="true"> </iframe> </figure> + +## Troubleshooting + +### Getting warning message `gl-secret-detection-report.json: no matching files` + +For information on this, see the [general Application Security troubleshooting section](../../../ci/pipelines/job_artifacts.md#error-message-no-files-to-upload). diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index 0b755a81616..b647c162152 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -270,20 +270,43 @@ If your cluster was created before GitLab 12.2, default `KUBE_NAMESPACE` will be ### Custom namespace -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27630) in GitLab 12.6. - -The Kubernetes integration defaults to project-environment-specific namespaces -of the form `<project_name>-<project_id>-<environment>` (see [Deployment -variables](#deployment-variables)). - -For **non**-GitLab-managed clusters, the namespace can be customized using -[`environment:kubernetes:namespace`](../../../ci/environments/index.md#configuring-kubernetes-deployments) -in `.gitlab-ci.yml`. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27630) in GitLab 12.6. +> - An option to use project-wide namespaces [was added](https://gitlab.com/gitlab-org/gitlab/-/issues/38054) in GitLab 13.5. + +The Kubernetes integration provides a `KUBECONFIG` with an auto-generated namespace +to deployment jobs. It defaults to using project-environment specific namespaces +of the form `<prefix>-<environment>`, where `<prefix>` is of the form +`<project_name>-<project_id>`. To learn more, read [Deployment variables](#deployment-variables). + +You can customize the deployment namespace in a few ways: + +- You can choose between a **namespace per [environment](../../../ci/environments/index.md)** + or a **namespace per project**. A namespace per environment is the default and recommended + setting, as it prevents the mixing of resources between production and non-production environments. +- When using a project-level cluster, you can additionally customize the namespace prefix. + When using namespace-per-environment, the deployment namespace is `<prefix>-<environment>`, + but otherwise just `<prefix>`. +- For **non-managed** clusters, the auto-generated namespace is set in the `KUBECONFIG`, + but the user is responsible for ensuring its existence. You can fully customize + this value using + [`environment:kubernetes:namespace`](../../../ci/environments/index.md#configuring-kubernetes-deployments) + in `.gitlab-ci.yml`. NOTE: **Note:** -When using a [GitLab-managed cluster](#gitlab-managed-clusters), the -namespaces are created automatically prior to deployment and [can not be -customized](https://gitlab.com/gitlab-org/gitlab/-/issues/38054). +When you customize the namespace, existing environments remain linked to their current +namespaces until you [clear the cluster cache](#clearing-the-cluster-cache). + +CAUTION: **Warning:** +By default, anyone who can create a deployment job can access any CI variable within +an environment's deployment job. This includes `KUBECONFIG`, which gives access to +any secret available to the associated service account in your cluster. +To keep your production credentials safe, consider using +[Protected Environments](../../../ci/environments/protected_environments.md), +combined with either + +- a GitLab-managed cluster and namespace per environment, +- *or*, an environment-scoped cluster per protected environment. The same cluster + can be added multiple times with multiple restricted service accounts. ### Integrations |