diff options
Diffstat (limited to 'doc/topics')
22 files changed, 218 insertions, 156 deletions
diff --git a/doc/topics/authentication/index.md b/doc/topics/authentication/index.md index 6ce5e203bbf..9e37ebb699b 100644 --- a/doc/topics/authentication/index.md +++ b/doc/topics/authentication/index.md @@ -44,5 +44,4 @@ This page gathers all the resources for the topic **Authentication** within GitL - [Kanboard Plugin GitLab Authentication](https://github.com/kanboard/plugin-gitlab-auth) - [Jenkins GitLab OAuth Plugin](https://wiki.jenkins.io/display/JENKINS/GitLab+OAuth+Plugin) -- [How to customize GitLab to support OpenID authentication](https://blog.eric.van-der-vlist.com/2013/11/23/how-to-customize-gitlab-to-support-openid-authentication/) - [OKD - Configuring Authentication and User Agent](https://docs.okd.io/3.11/install_config/configuring_authentication.html#GitLab) diff --git a/doc/topics/autodevops/customize.md b/doc/topics/autodevops/customize.md index 2d6da4d322b..b5952494201 100644 --- a/doc/topics/autodevops/customize.md +++ b/doc/topics/autodevops/customize.md @@ -5,7 +5,8 @@ almost everything to fit your needs. Auto DevOps offers everything from custom [buildpacks](#custom-buildpacks), to [Dockerfiles](#custom-dockerfile), and [Helm charts](#custom-helm-chart). You can even copy the complete [CI/CD configuration](#customizing-gitlab-ciyml) into your project to enable -staging and canary deployments, and more. +staging and canary deployments, +[manage Auto DevOps with GitLab APIs](customize.md#extend-auto-devops-with-the-api), and more. ## Custom buildpacks @@ -77,6 +78,16 @@ Avoid passing secrets as Docker build arguments if possible, as they may be persisted in your image. See [this discussion of best practices with secrets](https://github.com/moby/moby/issues/13490) for details. +## Extend Auto DevOps with the API + +You can extend and manage your Auto DevOps configuration with GitLab APIs: + +- [Settings that can be accessed with API calls](../../api/settings.md#list-of-settings-that-can-be-accessed-via-api-calls), + which include `auto_devops_enabled`, to enable Auto DevOps on projects by default. +- [Creating a new project](../../api/projects.md#create-project). +- [Editing groups](../../api/groups.md#update-group). +- [Editing projects](../../api/projects.md#edit-project). + ## Forward CI variables to the build environment > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25514) in GitLab 12.3, but available in versions 11.9 and above. @@ -381,7 +392,7 @@ these prefixed variables available to the deployed application as environment va To configure your application variables: -1. Go to your project's **{settings}** **Settings > CI/CD**, then expand the +1. Go to your project's **Settings > CI/CD**, then expand the **Variables** section. 1. Create a CI/CD variable, ensuring the key is prefixed with diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index 01e61095fe2..e8a344a41d7 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -104,8 +104,10 @@ knowledge of the following: - [GitLab Runner](https://docs.gitlab.com/runner/) - [Prometheus](https://prometheus.io/docs/introduction/overview/) -Auto DevOps provides great defaults for all the stages and makes use of [CI templates](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates); you can, however, -[customize](customize.md) almost everything to your needs. +Auto DevOps provides great defaults for all the stages and makes use of +[CI templates](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates). You can, however, +[customize](customize.md) almost everything to your needs, and +[manage Auto DevOps with GitLab APIs](customize.md#extend-auto-devops-with-the-api). For an overview on the creation of Auto DevOps, read more [in this blog post](https://about.gitlab.com/blog/2017/06/29/whats-next-for-gitlab-ci/). @@ -130,7 +132,7 @@ any of the following places: [groups](../../user/group/clusters/index.md#base-domain) - or at the project level as a variable: `KUBE_INGRESS_BASE_DOMAIN` - or at the group level as a variable: `KUBE_INGRESS_BASE_DOMAIN` -- or as an instance-wide fallback in **{admin}** **Admin Area > Settings** under the +- or as an instance-wide fallback in **Admin Area > Settings** under the **Continuous Integration and Delivery** section The base domain variable `KUBE_INGRESS_BASE_DOMAIN` follows the same order of precedence @@ -185,7 +187,7 @@ instance level. If enabling, check that your project does not have a `.gitlab-ci.yml`, or if one exists, remove it. -1. Go to your project's **{settings}** **Settings > CI/CD > Auto DevOps**. +1. Go to your project's **Settings > CI/CD > Auto DevOps**. 1. Select the **Default to Auto DevOps pipeline** checkbox to enable it. 1. (Optional, but recommended) When enabling, you can add in the [base domain](#auto-devops-base-domain) Auto DevOps uses to @@ -207,7 +209,7 @@ is specifically enabled or disabled on the subgroup or project. To enable or disable Auto DevOps at the group level: -1. Go to your group's **{settings}** **Settings > CI/CD > Auto DevOps** page. +1. Go to your group's **Settings > CI/CD > Auto DevOps** page. 1. Select the **Default to Auto DevOps pipeline** checkbox to enable it. 1. Click **Save changes** for the changes to take effect. @@ -216,7 +218,7 @@ To enable or disable Auto DevOps at the group level: Even when disabled at the instance level, group owners and project maintainers can still enable Auto DevOps at the group and project level, respectively. -1. Go to **{admin}** **Admin Area > Settings > Continuous Integration and Deployment**. +1. Go to **Admin Area > Settings > Continuous Integration and Deployment**. 1. Select **Default to Auto DevOps pipeline for all projects** to enable it. 1. (Optional) You can set up the Auto DevOps [base domain](#auto-devops-base-domain), for Auto Deploy and Auto Review Apps to use. @@ -227,7 +229,7 @@ Auto DevOps at the group and project level, respectively. > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/38542) in GitLab 11.0. You can change the deployment strategy used by Auto DevOps by going to your -project's **{settings}** **Settings > CI/CD > Auto DevOps**. The following options +project's **Settings > CI/CD > Auto DevOps**. The following options are available: - **Continuous deployment to production**: Enables [Auto Deploy](stages.md#auto-deploy) @@ -276,14 +278,14 @@ The following table is an example of how to configure the three different cluste To add a different cluster for each environment: -1. Navigate to your project's **{cloud-gear}** **Operations > Kubernetes**. +1. Navigate to your project's **Operations > Kubernetes**. 1. Create the Kubernetes clusters with their respective environment scope, as described from the table above. 1. After creating the clusters, navigate to each cluster and install Ingress. Wait for the Ingress IP address to be assigned. 1. Make sure you've [configured your DNS](#auto-devops-base-domain) with the specified Auto DevOps domains. -1. Navigate to each cluster's page, through **{cloud-gear}** **Operations > Kubernetes**, +1. Navigate to each cluster's page, through **Operations > Kubernetes**, and add the domain based on its Ingress IP address. After completing configuration, you can test your setup by creating a merge request diff --git a/doc/topics/autodevops/quick_start_guide.md b/doc/topics/autodevops/quick_start_guide.md index 4f8074f047e..6b9b461e76e 100644 --- a/doc/topics/autodevops/quick_start_guide.md +++ b/doc/topics/autodevops/quick_start_guide.md @@ -57,7 +57,7 @@ to deploy this project to. ## Create a Kubernetes cluster from within GitLab 1. On your project's landing page, click **Add Kubernetes cluster** - (note that this option is also available when you navigate to **{cloud-gear}** **Operations > Kubernetes**). + (note that this option is also available when you navigate to **Operations > Kubernetes**).  @@ -127,7 +127,7 @@ While Auto DevOps is enabled by default, Auto DevOps can be disabled at both the instance level (for self-managed instances) and the group level. Complete these steps to enable Auto DevOps if it's disabled: -1. Navigate to **{settings}** **Settings > CI/CD > Auto DevOps**, and click **Expand**. +1. Navigate to **Settings > CI/CD > Auto DevOps**, and click **Expand**. 1. Select **Default to Auto DevOps pipeline** to display more options. 1. In **Deployment strategy**, select your desired [continuous deployment strategy](index.md#deployment-strategy) to deploy the application to production after the pipeline successfully runs on the `master` branch. @@ -194,7 +194,7 @@ to monitor it. After successfully deploying your application, you can view its website and check on its health on the **Environments** page by navigating to -**{cloud-gear}** **Operations > Environments**. This page displays details about +**Operations > Environments**. This page displays details about the deployed applications, and the right-hand column displays icons that link you to common environment tasks: @@ -221,7 +221,7 @@ takes you to the pod's logs page. TIP: **Tip:** The example shows only one pod hosting the application at the moment, but you can add more pods by defining the [`REPLICAS` variable](customize.md#environment-variables) -in **{settings}** **Settings > CI/CD > Environment variables**. +in **Settings > CI/CD > Environment variables**. ### Work with branches diff --git a/doc/topics/autodevops/stages.md b/doc/topics/autodevops/stages.md index bf1594130f4..7c6cf043820 100644 --- a/doc/topics/autodevops/stages.md +++ b/doc/topics/autodevops/stages.md @@ -90,6 +90,12 @@ Check the [currently supported languages](#currently-supported-languages). Auto Test uses tests you already have in your application. If there are no tests, it's up to you to add them. +NOTE: **Note:** +Not all buildpacks supported by [Auto Build](#auto-build) are supported by Auto Test. +Auto Test uses [Herokuish](https://gitlab.com/gitlab-org/gitlab/-/issues/212689), *not* +Cloud Native Buildpacks, and only buildpacks that implement the +[Testpack API](https://devcenter.heroku.com/articles/testpack-api) are supported. + ### Currently supported languages Note that not all buildpacks support Auto Test yet, as it's a relatively new @@ -454,8 +460,9 @@ For example, in a Rails application in an image built with Unless your repository contains a `Dockerfile`, your image is built with Herokuish, and you must prefix commands run in these images with -`/bin/herokuish procfile exec` to replicate the environment where your application -will run. +`/bin/herokuish procfile exec` (for Herokuish) or `/cnb/lifecycle/launcher` +(for Cloud Native Buildpacks) to replicate the environment where your +application runs. ### Upgrade auto-deploy-app Chart @@ -616,6 +623,12 @@ For example, to start a Rails console from the application root directory, run: /bin/herokuish procfile exec bin/rails c ``` +When using Cloud Native Buildpacks, instead of `/bin/herokuish procfile exec`, use + +```shell +/cnb/lifecycle/launcher $COMMAND +``` + ## Auto Monitoring After your application deploys, Auto Monitoring helps you monitor @@ -644,6 +657,6 @@ To use Auto Monitoring: 1. After the pipeline finishes successfully, open the [monitoring dashboard for a deployed environment](../../ci/environments/index.md#monitoring-environments) to view the metrics of your deployed application. To view the metrics of the - whole Kubernetes cluster, navigate to **{cloud-gear}** **Operations > Metrics**. + whole Kubernetes cluster, navigate to **Operations > Metrics**.  diff --git a/doc/topics/cron/index.md b/doc/topics/cron/index.md new file mode 100644 index 00000000000..a3dd3b77c22 --- /dev/null +++ b/doc/topics/cron/index.md @@ -0,0 +1,63 @@ +# Cron + +Cron syntax is used to schedule when jobs should run. + +You may need to use a cron syntax string to +[trigger nightly pipelines](../../ci/triggers/README.md#using-cron-to-trigger-nightly-pipelines), +create a [pipeline schedule](../../api/pipeline_schedules.md#create-a-new-pipeline-schedule), +or to prevent unintentional releases by setting a +[deploy freeze](../../user/project/releases/index.md#prevent-unintentional-releases-by-setting-a-deploy-freeze). + +## Cron syntax + +Cron scheduling uses a series of five numbers, separated by spaces: + +```plaintext +# ┌───────────── minute (0 - 59) +# │ ┌───────────── hour (0 - 23) +# │ │ ┌───────────── day of the month (1 - 31) +# │ │ │ ┌───────────── month (1 - 12) +# │ │ │ │ ┌───────────── day of the week (0 - 6) (Sunday to Saturday) +# │ │ │ │ │ +# │ │ │ │ │ +# │ │ │ │ │ +# * * * * * <command to execute> +``` + +[Source: [Wikipedia](https://en.wikipedia.org/wiki/Cron)] + +In cron syntax, the asterisk (`*`) means 'every,' so the following cron strings +are valid: + +- Run once an hour at the beginning of the hour: `0 * * * *` +- Run once a day at midnight: `0 0 * * *` +- Run once a week at midnight on Sunday morning: `0 0 * * 0` +- Run once a month at midnight of the first day of the month: `0 0 1 * *` +- Run once a year at midnight of 1 January: `0 0 1 1 *` + +For complete cron documentation, refer to the +[crontab(5) — Linux manual page](https://man7.org/linux/man-pages/man5/crontab.5.html). +This documentation is accessible offline by entering `man 5 crontab` in a Linux or MacOS +terminal. + +## Cron examples + +```plaintext +# Run at 7:00pm every day: +0 19 * * * + +# Run every minute on the 10th of June: +* * 3 6 * + +# Run at 06:30 every Friday: +30 6 * * 5 +``` + +More examples of how to write a cron schedule can be found at +[crontab.guru](https://crontab.guru/examples.html). + +## How GitLab parses cron syntax strings + +GitLab uses [fugit](https://github.com/floraison/fugit) to parse cron syntax +strings on the server and [cron-validate](https://github.com/Airfooox/cron-validate) +to validate cron syntax in the browser. diff --git a/doc/topics/git/feature_branch_development.md b/doc/topics/git/feature_branch_development.md index ab3adf54dd7..762eddbc130 100644 --- a/doc/topics/git/feature_branch_development.md +++ b/doc/topics/git/feature_branch_development.md @@ -1,4 +1,7 @@ --- +stage: Create +group: Source Code +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers" type: how-tos --- diff --git a/doc/topics/git/how_to_install_git/index.md b/doc/topics/git/how_to_install_git/index.md index 75ea6183a32..7b842b6a409 100644 --- a/doc/topics/git/how_to_install_git/index.md +++ b/doc/topics/git/how_to_install_git/index.md @@ -1,4 +1,7 @@ --- +stage: Create +group: Source Code +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers" author: Sean Packham author_gitlab: SeanPackham level: beginner diff --git a/doc/topics/git/img/create_merge_request_v13_1.png b/doc/topics/git/img/create_merge_request_v13_1.png Binary files differindex a725149f6a2..d59cfc74290 100644 --- a/doc/topics/git/img/create_merge_request_v13_1.png +++ b/doc/topics/git/img/create_merge_request_v13_1.png diff --git a/doc/topics/git/img/modify_branches_v13_1.png b/doc/topics/git/img/modify_branches_v13_1.png Binary files differindex dc517dd249f..781f54fc3c0 100644 --- a/doc/topics/git/img/modify_branches_v13_1.png +++ b/doc/topics/git/img/modify_branches_v13_1.png diff --git a/doc/topics/git/index.md b/doc/topics/git/index.md index 89da3dfdbd0..92181fb7bb0 100644 --- a/doc/topics/git/index.md +++ b/doc/topics/git/index.md @@ -1,4 +1,7 @@ --- +stage: Create +group: Source Code +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers" type: index --- diff --git a/doc/topics/git/lfs/index.md b/doc/topics/git/lfs/index.md index 1e2f45fd67b..5875cdd4ca1 100644 --- a/doc/topics/git/lfs/index.md +++ b/doc/topics/git/lfs/index.md @@ -1,4 +1,8 @@ --- +stage: Create +group: Source Code +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers" +type: reference, howto disqus_identifier: 'https://docs.gitlab.com/ee/workflow/lfs/lfs/index.html' --- diff --git a/doc/topics/git/lfs/migrate_from_git_annex_to_git_lfs.md b/doc/topics/git/lfs/migrate_from_git_annex_to_git_lfs.md index 09087fcae13..c62b7e1cc12 100644 --- a/doc/topics/git/lfs/migrate_from_git_annex_to_git_lfs.md +++ b/doc/topics/git/lfs/migrate_from_git_annex_to_git_lfs.md @@ -1,6 +1,13 @@ +--- +stage: Create +group: Source Code +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers" +type: reference, howto +--- + # Migration guide from Git Annex to Git LFS ->**Note:** +NOTE: **Note:** Git Annex support [has been removed](https://gitlab.com/gitlab-org/gitlab/-/issues/1648) in GitLab Enterprise Edition 9.0 (2017/03/22). @@ -30,7 +37,7 @@ ones that GitLab developed. ## Migration steps ->**Note:** +NOTE: **Note:** Since Git Annex files are stored in a sub-directory of the normal repositories (`.git/annex/objects`) and LFS files are stored outside of the repositories, they are not compatible as they are using a different scheme. Therefore, the diff --git a/doc/topics/git/lfs/migrate_to_git_lfs.md b/doc/topics/git/lfs/migrate_to_git_lfs.md index 3e287c0816d..944f4d8f78d 100644 --- a/doc/topics/git/lfs/migrate_to_git_lfs.md +++ b/doc/topics/git/lfs/migrate_to_git_lfs.md @@ -96,9 +96,10 @@ Consider an example upstream project, `git@gitlab.com:gitlab-tests/test-git-lfs- 1. Clean up the repository: ```shell - # cd path/to/mirror/repo: + # Change into the mirror repo directory: cd test-git-lfs-repo-migration.git - # clean up the repo: + + # Clean up the repo: git reflog expire --expire=now --all && git gc --prune=now --aggressive ``` @@ -128,12 +129,23 @@ Consider an example upstream project, `git@gitlab.com:gitlab-tests/test-git-lfs- 1. Track the files you want with LFS: ```shell - # cd path/to/upstream/repo: + # Change into the /tmp directory + cd /tmp + + # Clone the repo + git clone git@gitlab.com:gitlab-tests/test-git-lfs-repo-migration.git + + # Change into the upstream repo directory: cd test-git-lfs-repo-migration + # You may need to reset your local copy with upstream's `master` after force-pushing from the mirror: git reset --hard origin/master + # Track the files with LFS: git lfs track "*.gif" "*.png" "*.jpg" "*.psd" "*.mp4" "img/" + + # Push up changes to .gitattributes + git add .gitattributes && git commit -m 'Track .gif,.png,.jpg,.psd,.mp4 and img/' && git push ``` Now all existing the files you converted, as well as the new diff --git a/doc/topics/git/numerous_undo_possibilities_in_git/index.md b/doc/topics/git/numerous_undo_possibilities_in_git/index.md index fdf86d8f646..285ab133196 100644 --- a/doc/topics/git/numerous_undo_possibilities_in_git/index.md +++ b/doc/topics/git/numerous_undo_possibilities_in_git/index.md @@ -1,4 +1,7 @@ --- +stage: Create +group: Source Code +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers" author: Crt Mori author_gitlab: Letme level: intermediary diff --git a/doc/topics/git/partial_clone.md b/doc/topics/git/partial_clone.md index de25c8a3283..c976eda688a 100644 --- a/doc/topics/git/partial_clone.md +++ b/doc/topics/git/partial_clone.md @@ -1,3 +1,10 @@ +--- +stage: Create +group: Source Code +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers" +type: reference, howto +--- + # Partial Clone As Git repositories grow in size, they can become cumbersome to work with @@ -182,3 +189,47 @@ For more details, see the Git documentation for # Checkout master git checkout master ``` + +## Remove partial clone filtering + +Git repositories with partial clone filtering can have the filtering removed. To +remove filtering: + +1. Fetch everything that has been excluded by the filters, to make sure that the + repository is complete. If `git sparse-checkout` was used, use + `git sparse-checkout disable` to disable it. See the + [`disable` documentation](https://git-scm.com/docs/git-sparse-checkout#Documentation/git-sparse-checkout.txt-emdisableem) + for more information. + + Then do a regular `fetch` to ensure that the repository is complete. To check if + there are missing objects to fetch, and then fetch them, especially when not using + `git sparse-checkout`, the following commands can be used: + + ```shell + # Show missing objects + git rev-list --objects --all --missing=print | grep -e '^\?' + + # Show missing objects without a '?' character before them (needs GNU grep) + git rev-list --objects --all --missing=print | grep -oP '^\?\K\w+' + + # Fetch missing objects + git fetch origin $(git rev-list --objects --all --missing=print | grep -oP '^\?\K\w+') + + # Show number of missing objects + git rev-list --objects --all --missing=print | grep -e '^\?' | wc -l + ``` + +1. Repack everything. This can be done using `git repack -a -d`, for example. This + should leave only three files in `.git/objects/pack/`: + - A `pack-<SHA1>.pack` file. + - Its corresponding `pack-<SHA1>.idx` file. + - A `pack-<SHA1>.promisor` file. + +1. Delete the `.promisor` file. The above step should have left only one + `pack-<SHA1>.promisor` file, which should be empty and should be deleted. + +1. Remove partial clone configuration. The partial clone-related configuration + variables should be removed from Git config files. Usually only the following + configuration must be removed: + - `remote.origin.promisor`. + - `remote.origin.partialclonefilter`. diff --git a/doc/topics/git/troubleshooting_git.md b/doc/topics/git/troubleshooting_git.md index 010a811bd33..523718e4133 100644 --- a/doc/topics/git/troubleshooting_git.md +++ b/doc/topics/git/troubleshooting_git.md @@ -1,4 +1,7 @@ --- +stage: Create +group: Source Code +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers" type: howto --- diff --git a/doc/topics/git/useful_git_commands.md b/doc/topics/git/useful_git_commands.md index 711567dbd06..0bf6075a1ea 100644 --- a/doc/topics/git/useful_git_commands.md +++ b/doc/topics/git/useful_git_commands.md @@ -1,4 +1,7 @@ --- +stage: Create +group: Source Code +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers" type: reference --- diff --git a/doc/topics/gitlab_flow.md b/doc/topics/gitlab_flow.md index 04c942ab532..339da40b29d 100644 --- a/doc/topics/gitlab_flow.md +++ b/doc/topics/gitlab_flow.md @@ -1,4 +1,7 @@ --- +stage: Create +group: Source Code +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers" disqus_identifier: 'https://docs.gitlab.com/ee/workflow/gitlab_flow.html' --- @@ -266,11 +269,11 @@ One option is to use continuous integration (CI) to merge in `master` at the sta Another option is to only merge in from well-defined points in time, for example, a tagged release. You could also use [feature toggles](https://martinfowler.com/bliki/FeatureToggle.html) to hide incomplete features so you can still merge back into `master` every day. -> **Note:** Don't confuse automatic branch testing with continuous integration. -> Martin Fowler makes this distinction in [his article about feature branches](https://martinfowler.com/bliki/FeatureBranch.html): -> -> "I've heard people say they are doing CI because they are running builds, perhaps using a CI server, on every branch with every commit. -> That's continuous building, and a Good Thing, but there's no *integration*, so it's not CI." +NOTE: **Note:** +Don't confuse automatic branch testing with continuous integration. +Martin Fowler makes this distinction in [his article about feature branches](https://martinfowler.com/bliki/FeatureBranch.html): +"I've heard people say they are doing CI because they are running builds, perhaps using a CI server, on every branch with every commit. +That's continuous building, and a Good Thing, but there's no *integration*, so it's not CI." In conclusion, you should try to prevent merge commits, but not eliminate them. Your codebase should be clean, but your history should represent what actually happened. diff --git a/doc/topics/index.md b/doc/topics/index.md index 634dd70613a..3b92bbb55cb 100644 --- a/doc/topics/index.md +++ b/doc/topics/index.md @@ -10,10 +10,12 @@ tutorials, technical overviews, blog posts) and videos. - [Auto DevOps](autodevops/index.md) - [Authentication](authentication/index.md) - [Continuous Integration (GitLab CI/CD)](../ci/README.md) +- [Cron](cron/index.md) - [Git](git/index.md) - [GitLab Flow](gitlab_flow.md) - [GitLab Installation](../install/README.md) - [GitLab Pages](../user/project/pages/index.md) - [Offline GitLab](offline/index.md) ->**Note:** More topics will be available soon. +NOTE: **Note:** +More topics will be available soon. diff --git a/doc/topics/offline/index.md b/doc/topics/offline/index.md index 6d4c486d350..a8366659c22 100644 --- a/doc/topics/offline/index.md +++ b/doc/topics/offline/index.md @@ -13,126 +13,3 @@ If you plan to deploy a GitLab instance on a physically-isolated and offline net Follow these best practices to use GitLab's features in an offline environment: - [Operating the GitLab Secure scanners in an offline environment](../../user/application_security/offline_deployments/index.md). - -## Loading Docker images onto your offline host - -To use many GitLab features, including -[security scans](../../user/application_security/index.md#working-in-an-offline-environment) -and [Auto DevOps](../autodevops/), the GitLab Runner must be able to fetch the -relevant Docker images. - -The process for making these images available without direct access to the public internet -involves downloading the images then packaging and transferring them to the offline host. Here's an -example of such a transfer: - -1. Download Docker images from public internet. -1. Package Docker images as tar archives. -1. Transfer images to offline environment. -1. Load transferred images into offline Docker registry. - -### Using the official GitLab template - -GitLab provides a [vendored template](../../ci/yaml/README.md#includetemplate) -to ease this process. - -This template should be used in a new, empty project, with a `gitlab-ci.yml` file containing: - -```yaml -include: - - template: Secure-Binaries.gitlab-ci.yml -``` - -The pipeline downloads the Docker images needed for the Security Scanners and saves them as -[job artifacts](../../ci/pipelines/job_artifacts.md) or pushes them to the [Container Registry](../../user/packages/container_registry/index.md) -of the project where the pipeline is executed. These archives can be transferred to another location -and [loaded](https://docs.docker.com/engine/reference/commandline/load/) in a Docker daemon. -This method requires a GitLab Runner with access to both `gitlab.com` (including -`registry.gitlab.com`) and the local offline instance. This runner must run in -[privileged mode](https://docs.gitlab.com/runner/executors/docker.html#use-docker-in-docker-with-privileged-mode) -to be able to use the `docker` command inside the jobs. This runner can be installed in a DMZ or on -a bastion, and used only for this specific project. - -#### Scheduling the updates - -By default, this project's pipeline will run only once, when the `.gitlab-ci.yml` is added to the -repo. To update the GitLab security scanners and signatures, it's necessary to run this pipeline -regularly. GitLab provides a way to [schedule pipelines](../../ci/pipelines/schedules.md). For -example, you can set this up to download and store the Docker images every week. - -Some images can be updated more frequently than others. For example, the [vulnerability database](https://hub.docker.com/r/arminc/clair-db/tags) -for Container Scanning is updated daily. To update this single image, create a new Scheduled -Pipeline that runs daily and set `SECURE_BINARIES_ANALYZERS` to `clair-vulnerabilities-db`. Only -this job will be triggered, and the image will be updated daily and made available in the project -registry. - -#### Using the secure bundle created - -The project using the `Secure-Binaries.gitlab-ci.yml` template should now host all the required -images and resources needed to run GitLab Security features. - -Next, you must tell the offline instance to use these resources instead of the default ones on -GitLab.com. To do so, set the environment variable `SECURE_ANALYZERS_PREFIX` with the URL of the -project [container registry](../../user/packages/container_registry/index.md). - -You can set this variable in the projects' `.gitlab-ci.yml`, or -in the GitLab UI at the project or group level. See the [GitLab CI/CD environment variables page](../../ci/variables/README.md#custom-environment-variables) -for more information. - -#### Variables - -The following table shows which variables you can use with the `Secure-Binaries.gitlab-ci.yml` -template: - -| VARIABLE | Description | Default value | -|-------------------------------------------|-----------------------------------------------|-----------------------------------| -| `SECURE_BINARIES_ANALYZERS` | Comma-separated list of analyzers to download | `"bandit, brakeman, gosec, and so on..."` | -| `SECURE_BINARIES_DOWNLOAD_IMAGES` | Used to disable jobs | `"true"` | -| `SECURE_BINARIES_PUSH_IMAGES` | Push files to the project registry | `"true"` | -| `SECURE_BINARIES_SAVE_ARTIFACTS` | Also save image archives as artifacts | `"false"` | -| `SECURE_BINARIES_ANALYZER_VERSION` | Default analyzer version (Docker tag) | `"2"` | - -### Alternate way without the official template - -If it's not possible to follow the above method, the images can be transferred manually instead: - -#### Example image packager script - -```shell -#!/bin/bash -set -ux - -# Specify needed analyzer images -analyzers=${SAST_ANALYZERS:-"bandit eslint gosec"} -gitlab=registry.gitlab.com/gitlab-org/security-products/analyzers/ - -for i in "${analyzers[@]}" -do - tarname="${i}_2.tar" - docker pull $gitlab$i:2 - docker save $gitlab$i:2 -o ./analyzers/${tarname} - chmod +r ./analyzers/${tarname} -done -``` - -#### Example image loader script - -This example loads the images from a bastion host to an offline host. In certain configurations, -physical media may be needed for such a transfer: - -```shell -#!/bin/bash -set -ux - -# Specify needed analyzer images -analyzers=${SAST_ANALYZERS:-"bandit eslint gosec"} -registry=$GITLAB_HOST:4567 - -for i in "${analyzers[@]}" -do - tarname="${i}_2.tar" - scp ./analyzers/${tarname} ${GITLAB_HOST}:~/${tarname} - ssh $GITLAB_HOST "sudo docker load -i ${tarname}" - ssh $GITLAB_HOST "sudo docker tag $(sudo docker images | grep $i | awk '{print $3}') ${registry}/analyzers/${i}:2" - ssh $GITLAB_HOST "sudo docker push ${registry}/analyzers/${i}:2" -done -``` diff --git a/doc/topics/offline/quick_start_guide.md b/doc/topics/offline/quick_start_guide.md index 0abdd08ffcf..817d7d31180 100644 --- a/doc/topics/offline/quick_start_guide.md +++ b/doc/topics/offline/quick_start_guide.md @@ -34,7 +34,7 @@ Follow these steps to enable SSL for your fresh instance. Note that these steps ```ruby # Update external_url from "http" to "https" - external_url "https://example.gitlab.com" + external_url "https://gitlab.example.com" # Set Let's Encrypt to false letsencrypt['enable'] = false @@ -64,8 +64,8 @@ Follow these steps to enable the container registry. Note that these steps refle ```ruby # Change external_registry_url to match external_url, but append the port 4567 - external_url "https://example.gitlab.com" - registry_external_url "https://example.gitlab.com:4567" + external_url "https://gitlab.example.com" + registry_external_url "https://gitlab.example.com:4567" ``` 1. Reconfigure your instance to apply the changes: |
