diff options
Diffstat (limited to 'doc/topics')
-rw-r--r-- | doc/topics/authentication/index.md | 1 | ||||
-rw-r--r-- | doc/topics/autodevops/customize.md | 2 | ||||
-rw-r--r-- | doc/topics/autodevops/index.md | 4 | ||||
-rw-r--r-- | doc/topics/autodevops/requirements.md | 2 | ||||
-rw-r--r-- | doc/topics/autodevops/stages.md | 21 | ||||
-rw-r--r-- | doc/topics/autodevops/troubleshooting.md | 2 | ||||
-rw-r--r-- | doc/topics/git/feature_branch_development.md | 23 | ||||
-rw-r--r-- | doc/topics/git/lfs/index.md | 8 | ||||
-rw-r--r-- | doc/topics/git/numerous_undo_possibilities_in_git/index.md | 2 | ||||
-rw-r--r-- | doc/topics/git/partial_clone.md | 83 | ||||
-rw-r--r-- | doc/topics/git/tags.md | 2 | ||||
-rw-r--r-- | doc/topics/git/troubleshooting_git.md | 68 | ||||
-rw-r--r-- | doc/topics/git/useful_git_commands.md | 2 |
13 files changed, 127 insertions, 93 deletions
diff --git a/doc/topics/authentication/index.md b/doc/topics/authentication/index.md index 1988e3e2890..25640671e78 100644 --- a/doc/topics/authentication/index.md +++ b/doc/topics/authentication/index.md @@ -35,7 +35,6 @@ This page gathers all the resources for the topic **Authentication** within GitL - [SAML OmniAuth Provider](../../integration/saml.md) - [SAML for GitLab.com Groups](../../user/group/saml_sso/index.md) **(PREMIUM SAAS)** - [SCIM user provisioning for GitLab.com Groups](../../user/group/saml_sso/scim_setup.md) **(PREMIUM SAAS)** - - [Okta SSO provider](../../administration/auth/okta.md) - [Kerberos integration (GitLab EE)](../../integration/kerberos.md) **(STARTER)** ## API diff --git a/doc/topics/autodevops/customize.md b/doc/topics/autodevops/customize.md index 7dcecb9af1e..ae6e57cd38e 100644 --- a/doc/topics/autodevops/customize.md +++ b/doc/topics/autodevops/customize.md @@ -171,7 +171,7 @@ list of options. ## Custom Helm chart per environment You can specify the use of a custom Helm chart per environment by scoping the CI/CD variable -to the desired environment. See [Limiting environment scopes of variables](../../ci/variables/README.md#limit-the-environment-scopes-of-cicd-variables). +to the desired environment. See [Limit environment scope of CI/CD variables](../../ci/variables/README.md#limit-the-environment-scope-of-a-cicd-variable). ## Customizing `.gitlab-ci.yml` diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index dbb63275a06..03454649c7e 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -231,7 +231,7 @@ any of the following places: **Continuous Integration and Delivery** section The base domain variable `KUBE_INGRESS_BASE_DOMAIN` follows the same order of precedence -as other environment [variables](../../ci/variables/README.md#priority-of-cicd-variables). +as other environment [variables](../../ci/variables/README.md#cicd-variable-precedence). If the CI/CD variable is not set and the cluster setting is left blank, the instance-wide **Auto DevOps domain** setting is used if set. @@ -274,7 +274,7 @@ used by Auto DevOps currently defines 3 environment names: Those environments are tied to jobs using [Auto Deploy](stages.md#auto-deploy), so except for the environment scope, they must have a different deployment domain. You must define a separate `KUBE_INGRESS_BASE_DOMAIN` variable for each of the above -[based on the environment](../../ci/variables/README.md#limit-the-environment-scopes-of-cicd-variables). +[based on the environment](../../ci/variables/README.md#limit-the-environment-scope-of-a-cicd-variable). The following table is an example of how to configure the three different clusters: diff --git a/doc/topics/autodevops/requirements.md b/doc/topics/autodevops/requirements.md index d536282cca4..8fb16511e34 100644 --- a/doc/topics/autodevops/requirements.md +++ b/doc/topics/autodevops/requirements.md @@ -91,7 +91,7 @@ To make full use of Auto DevOps with Kubernetes, you need: To enable HTTPS endpoints for your application, you must install cert-manager, a native Kubernetes certificate management controller that helps with issuing certificates. Installing cert-manager on your cluster issues a - [Let’s Encrypt](https://letsencrypt.org/) certificate and ensures the + [Let's Encrypt](https://letsencrypt.org/) certificate and ensures the certificates are valid and up-to-date. If you've configured the GitLab integration with Kubernetes, you can deploy it to your cluster by installing the [GitLab-managed app for cert-manager](../../user/clusters/applications.md#cert-manager). diff --git a/doc/topics/autodevops/stages.md b/doc/topics/autodevops/stages.md index 913c221d8ec..97edc9adc06 100644 --- a/doc/topics/autodevops/stages.md +++ b/doc/topics/autodevops/stages.md @@ -137,8 +137,7 @@ might want to use a [custom buildpack](customize.md#custom-buildpacks). ## Auto Code Quality -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1984) in [GitLab](https://about.gitlab.com/pricing/) 9.3. -> - Made [available in all tiers](https://gitlab.com/gitlab-org/gitlab/-/issues/212499) in GitLab 13.2. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/212499) to GitLab Free in 13.2. Auto Code Quality uses the [Code Quality image](https://gitlab.com/gitlab-org/ci-cd/codequality) to run @@ -165,7 +164,7 @@ see the documentation. ## Auto Secret Detection -> - Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.1. +> - Introduced in GitLab Ultimate 13.1. > - [Select functionality made available in all tiers](../../user/application_security/secret_detection/#making-secret-detection-available-to-all-gitlab-tiers) in 13.3 Secret Detection uses the @@ -179,8 +178,6 @@ To learn more, see [Secret Detection](../../user/application_security/secret_det ## Auto Dependency Scanning **(ULTIMATE)** -> Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.7. - Dependency Scanning runs analysis on the project's dependencies and checks for potential security issues. The Auto Dependency Scanning stage is skipped on licenses other than [Ultimate](https://about.gitlab.com/pricing/) and requires @@ -195,7 +192,7 @@ see the documentation. ## Auto License Compliance **(ULTIMATE)** -> Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.0. +> Introduced in GitLab Ultimate 11.0. License Compliance uses the [License Compliance Docker image](https://gitlab.com/gitlab-org/security-products/analyzers/license-finder) @@ -211,8 +208,6 @@ documentation. ## Auto Container Scanning **(ULTIMATE)** -> Introduced in GitLab 10.4. - Vulnerability Static Analysis for containers uses [Clair](https://github.com/quay/clair) to check for potential security issues on Docker images. The Auto Container Scanning stage is skipped on licenses other than [Ultimate](https://about.gitlab.com/pricing/). @@ -263,8 +258,6 @@ in the first place, and thus not realize that it needs to re-apply the old confi ## Auto DAST **(ULTIMATE)** -> Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 10.4. - Dynamic Application Security Testing (DAST) uses the popular open source tool [OWASP ZAProxy](https://github.com/zaproxy/zaproxy) to analyze the current code and check for potential security issues. The Auto DAST stage is skipped on @@ -422,10 +415,10 @@ including support for `Deployment` in the `extensions/v1beta1` version. To use Auto Deploy on a Kubernetes 1.16+ cluster: -1. If you are deploying your application for the first time on GitLab 13.0 or +1. If you are deploying your application for the first time in GitLab 13.0 or newer, no configuration should be required. -1. On GitLab 12.10 or older, set the following in the [`.gitlab/auto-deploy-values.yaml` file](customize.md#customize-values-for-helm-chart): +1. In GitLab 12.10 or older, set the following in the [`.gitlab/auto-deploy-values.yaml` file](customize.md#customize-values-for-helm-chart): ```yaml deploymentApiVersion: apps/v1 @@ -439,7 +432,7 @@ To use Auto Deploy on a Kubernetes 1.16+ cluster: GitLab 12.9 or 12.10, set `AUTO_DEVOPS_POSTGRES_CHANNEL` to `2`. WARNING: -On GitLab 12.9 and 12.10, opting into +In GitLab 12.9 and 12.10, opting into `AUTO_DEVOPS_POSTGRES_CHANNEL` version `2` deletes the version `1` PostgreSQL database. Follow the [guide to upgrading PostgreSQL](upgrading_postgresql.md) to back up and restore your database before opting into version `2` (On @@ -739,7 +732,7 @@ To use Auto Monitoring: 1. [Install and configure the Auto DevOps requirements](requirements.md). 1. [Enable Auto DevOps](index.md#enable-or-disable-auto-devops), if you haven't done already. -1. Navigate to your project's **{rocket}** **CI/CD > Pipelines** and click **Run Pipeline**. +1. Navigate to your project's **{rocket}** **CI/CD > Pipelines** and click **Run pipeline**. 1. After the pipeline finishes successfully, open the [monitoring dashboard for a deployed environment](../../ci/environments/index.md#monitoring-environments) to view the metrics of your deployed application. To view the metrics of the diff --git a/doc/topics/autodevops/troubleshooting.md b/doc/topics/autodevops/troubleshooting.md index ed75dd4e262..cf2a2133fa3 100644 --- a/doc/topics/autodevops/troubleshooting.md +++ b/doc/topics/autodevops/troubleshooting.md @@ -39,7 +39,7 @@ Found errors in your .gitlab-ci.yml: jobs:test config key may not be used with `rules`: only ``` -This error appears when the included job’s rules configuration has been overridden with the `only` or `except` syntax. +This error appears when the included job's rules configuration has been overridden with the `only` or `except` syntax. To fix this issue, you must either: - Transition your `only/except` syntax to rules. diff --git a/doc/topics/git/feature_branch_development.md b/doc/topics/git/feature_branch_development.md index 5f4b5632f33..8d18441aadd 100644 --- a/doc/topics/git/feature_branch_development.md +++ b/doc/topics/git/feature_branch_development.md @@ -23,10 +23,29 @@ In such cases, it may be more efficient to submit an MR on the release post feat In this case, the feature branch would be `release-X-Y`. Assuming the `release-X-Y` branch already exists, you can set up an MR against that branch, with the following steps: -1. Create a new branch (`test-branch`) against the feature branch (`release-X-Y`): +1. Navigate to the main (master) branch: ```shell - git checkout -b test-branch release-X-Y + git checkout master + ``` + +1. Make sure you have the latest version of your repository: + + ```shell + git fetch + git pull + ``` + +1. Check out the feature branch: + + ```shell + git checkout release-x-y + ``` + +1. Create a new branch (`test-branch`) against the feature branch (`release-x-y`): + + ```shell + git checkout -b test-branch release-x-y ``` You should now be on a branch named `test-branch`. diff --git a/doc/topics/git/lfs/index.md b/doc/topics/git/lfs/index.md index 11c0fcc2373..0851d3f6b50 100644 --- a/doc/topics/git/lfs/index.md +++ b/doc/topics/git/lfs/index.md @@ -114,10 +114,10 @@ See the documentation on [File Locking](../../../user/project/file_lock.md). ## LFS objects in project archives > - Support for including Git LFS blobs inside [project source downloads](../../../user/project/repository/index.md) was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15079) in GitLab 13.5. -> - It was [deployed behind a feature flag](../../../user/feature_flags.md), disabled by default. -> - [Became enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/268409) on GitLab 13.6. -> - It's enabled on GitLab.com. -> - It's recommended for production use. +> - [Deployed behind a feature flag](../../../user/feature_flags.md), disabled by default. +> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/268409) in GitLab 13.6. +> - Enabled on GitLab.com. +> - Recommended for production use. > - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-lfs-objects-in-project-archives). WARNING: diff --git a/doc/topics/git/numerous_undo_possibilities_in_git/index.md b/doc/topics/git/numerous_undo_possibilities_in_git/index.md index 76fc9bc92b0..eba471882f1 100644 --- a/doc/topics/git/numerous_undo_possibilities_in_git/index.md +++ b/doc/topics/git/numerous_undo_possibilities_in_git/index.md @@ -379,7 +379,7 @@ it also provides a clear timeline and development structure.  -If you want to revert changes introduced in certain `commit-id` you can simply +If you want to revert changes introduced in certain `commit-id`, you can revert that `commit-id` (swap additions and deletions) in newly created commit: You can do this with diff --git a/doc/topics/git/partial_clone.md b/doc/topics/git/partial_clone.md index 7f2543f040a..cad29d30af4 100644 --- a/doc/topics/git/partial_clone.md +++ b/doc/topics/git/partial_clone.md @@ -5,11 +5,13 @@ info: "To determine the technical writer assigned to the Stage/Group associated type: reference, howto --- -# Partial Clone **(FREE)** +# Partial clone **(FREE)** As Git repositories grow in size, they can become cumbersome to work with -because of the large amount of history that must be downloaded, and the large -amount of disk space they require. +because of: + +- The large amount of history that must be downloaded. +- The large amount of disk space they require. [Partial clone](https://github.com/git/git/blob/master/Documentation/technical/partial-clone.txt) is a performance optimization that "allows Git to function without having a @@ -58,9 +60,10 @@ Updating files: 100% (13008/13008), done. Filtering content: 100% (3/3), 131.24 MiB | 4.65 MiB/s, done. ``` -The output is longer because Git first clones the repository excluding -files larger than 1 megabyte, and second download any missing large files needed -to checkout the `master` branch. +The output is longer because Git: + +1. Clones the repository excluding files larger than 1 megabyte. +1. Downloads any missing large files needed to check out the default branch. When changing branches, Git may need to download more missing files. @@ -68,9 +71,9 @@ When changing branches, Git may need to download more missing files. > [Introduced](https://gitlab.com/gitlab-org/gitaly/-/issues/2553) in GitLab 12.10. -For enormous repositories with millions of files, and long history, it may be -helpful to exclude all files and use in combination with `sparse-checkout` to -reduce the size of your working copy. +For repositories with millions of files and a long history, you can exclude all files and use +[`git sparse-checkout`](https://git-scm.com/docs/git-sparse-checkout) to reduce the size of +your working copy. ```plaintext # Clone the repo excluding all files @@ -108,21 +111,22 @@ For more details, see the Git documentation for ## Filter by file path -WARNING: -Partial Clone using `sparse` filters is experimental, slow, and -significantly increases Gitaly resource utilization when cloning and fetching. +Deeper integration between partial clone and sparse checkout is possible through the +`--filter=sparse:oid=<blob-ish>` filter spec. This mode of filtering uses a format similar to a +`.gitignore` file to specify which files to include when cloning and fetching. -Deeper integration between Partial Clone and Sparse Checkout is being explored -through the `--filter=sparse:oid=<blob-ish>` filter spec, but this is highly -experimental. This mode of filtering uses a format similar to a `.gitignore` -file to specify which files should be included when cloning and fetching. +WARNING: +Partial clone using `sparse` filters is still experimental. It might be slow and significantly increase +[Gitaly](../../administration/gitaly/index.md) resource utilization when cloning and fetching. +[Filter all blobs and use sparse-checkout](#filter-by-object-type) instead, because +[`git-sparse-checkout`](https://git-scm.com/docs/git-sparse-checkout) simplifies +this type of partial clone use and overcomes its limitations. For more details, see the Git documentation for -[`rev-list-options`](https://gitlab.com/gitlab-org/git/-/blob/9fadedd637b312089337d73c3ed8447e9f0aa775/Documentation/rev-list-options.txt#L735-780). +[`rev-list-options`](https://git-scm.com/docs/git-rev-list#Documentation/git-rev-list.txt---filterltfilter-specgt). -1. **Create a filter spec.** For example, consider a monolithic repository with - many applications, each in a different subdirectory in the root. Create a file - `shiny-app/.filterspec` using the GitLab web interface: +1. Create a filter spec. For example, consider a monolithic repository with many applications, + each in a different subdirectory in the root. Create a file `shiny-app/.filterspec`: ```plaintext # Only the paths listed in the file will be downloaded when performing a @@ -142,28 +146,14 @@ For more details, see the Git documentation for shared-component-b/ ``` -1. **Create a new Git repository and fetch.** Support for `--filter=sparse:oid` - using the clone command is incomplete, so we emulate the clone command - by hand, using `git init` and `git fetch`. Follow - [issue tracking support for `--filter=sparse:oid`](https://gitlab.com/gitlab-org/git/-/issues/4) - for updates. +1. Clone and filter by path. Support for `--filter=sparse:oid` using the + clone command is not yet fully integrated with sparse checkout. ```shell - # Create a new directory for the Git repository - mkdir jumbo-repo && cd jumbo-repo - - # Initialize a new Git repository - git init - - # Add the remote - git remote add origin <url> - - # Enable partial clone support for the remote - git config --local extensions.partialClone origin - # Fetch the filtered set of objects using the filterspec stored on the - # server. WARNING: this step is slow! - git fetch --filter=sparse:oid=master:shiny-app/.gitfilterspec origin + # Clone the filtered set of objects using the filterspec stored on the + # server. WARNING: this step may be very slow! + git clone --sparse --filter=sparse:oid=master:shiny-app/.gitfilterspec <url> # Optional: observe there are missing objects that we have not fetched git rev-list --all --quiet --objects --missing=print | wc -l @@ -175,21 +165,6 @@ For more details, see the Git documentation for entire repository. You many need to disable or reconfigure these integrations. -1. **Sparse checkout** must be enabled and configured to prevent objects from - other paths being downloaded automatically when checking out branches. Follow - [issue proposing automating sparse checkouts](https://gitlab.com/gitlab-org/git/-/issues/5) for updates. - - ```shell - # Enable sparse checkout - git config --local core.sparsecheckout true - - # Configure sparse checkout - git show master:snazzy-app/.gitfilterspec >> .git/info/sparse-checkout - - # Checkout master - git checkout master - ``` - ## Remove partial clone filtering Git repositories with partial clone filtering can have the filtering removed. To diff --git a/doc/topics/git/tags.md b/doc/topics/git/tags.md index 76e3cff3edc..70580ecf778 100644 --- a/doc/topics/git/tags.md +++ b/doc/topics/git/tags.md @@ -28,7 +28,7 @@ git checkout master git tag my_lightweight_tag # Annotated tag -git tag -a v1.0 -m ‘Version 1.0’ +git tag -a v1.0 -m 'Version 1.0' # Show list of the existing tags git tag diff --git a/doc/topics/git/troubleshooting_git.md b/doc/topics/git/troubleshooting_git.md index be4903e2cb9..8db683f6291 100644 --- a/doc/topics/git/troubleshooting_git.md +++ b/doc/topics/git/troubleshooting_git.md @@ -47,15 +47,15 @@ errors can sometimes be caused by underlying issues with SSH (such as authentication). Make sure that SSH is correctly configured by following the instructions in the [SSH troubleshooting](../../ssh/README.md#troubleshooting-ssh-connections) documentation. -If you're a GitLab administrator and have access to the server, you can also prevent -session timeouts by configuring SSH `keep alive` either on the client or on the server. +If you're a GitLab administrator with server access, you can also prevent +session timeouts by configuring SSH `keep-alive` on the client or the server. NOTE: Configuring both the client and the server is unnecessary. **To configure SSH on the client side**: -- On UNIX, edit `~/.ssh/config` (create the file if it doesn’t exist) and +- On UNIX, edit `~/.ssh/config` (create the file if it doesn't exist) and add or edit: ```plaintext @@ -153,7 +153,7 @@ and provide GitLab with more information on how to improve the service. ## `git clone` over HTTP fails with `transfer closed with outstanding read data remaining` error -If the buffer size is lower than what is allowed in the request, the action fails with an error similar to the one below: +Sometimes, when cloning old or large repositories, the following error is thrown: ```plaintext error: RPC failed; curl 18 transfer closed with outstanding read data remaining @@ -162,11 +162,59 @@ fatal: early EOF fatal: index-pack failed ``` -This can be fixed by increasing the existing `http.postBuffer` value to one greater than the repository size. For example, if `git clone` fails when cloning a 500M repository, you should set `http.postBuffer` to `524288000`. That setting ensures the request only starts buffering after the first 524288000 bytes. +This is a common problem with Git itself, due to its inability to handle large files or large quantities of files. +[Git LFS](https://about.gitlab.com/blog/2017/01/30/getting-started-with-git-lfs-tutorial/) was created to work around this problem; however, even it has limitations. It's usually due to one of these reasons: -NOTE: -The default value of `http.postBuffer`, 1 MiB, is applied if the setting is not configured. +- The number of files in the repository. +- The number of revisions in the history. +- The existence of large files in the repository. -```shell -git config http.postBuffer 524288000 -``` +The root causes vary, so multiple potential solutions exist, and you may need to +apply more than one: + +- If this error occurs when cloning a large repository, you can + [decrease the cloning depth](../../ci/large_repositories/index.md#shallow-cloning) + to a value of `1`. For example: + + ```shell + variables: + GIT_DEPTH: 1 + ``` + +- You can increase the + [http.postBuffer](https://git-scm.com/docs/git-config#Documentation/git-config.txt-httppostBuffer) + value in your local Git configuration from the default 1 MB value to a value greater + than the repository size. For example, if `git clone` fails when cloning a 500 MB + repository, you should set `http.postBuffer` to `524288000`: + + ```shell + # Set the http.postBuffer size, in bytes + git config http.postBuffer 524288000 + ``` + +- You can increase the `http.postBuffer` on the server side: + + 1. Modify the GitLab instance's + [`gitlab.rb`](https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/13.5.1+ee.0/files/gitlab-config-template/gitlab.rb.template#L1435-1455) file: + + ```shell + omnibus_gitconfig['system'] = { + # Set the http.postBuffer size, in bytes + "http" => ["postBuffer" => 524288000] + } + ``` + + 1. After applying this change, apply the configuration change: + + ```shell + sudo gitlab-ctl reconfigure + ``` + +For example, if a repository has a very long history and no large files, changing +the depth should fix the problem. However, if a repository has very large files, +even a depth of 1 may be too large, thus requiring the `postBuffer` change. +If you increase your local `postBuffer` but the NGINX value on the backend is still +too small, the error persists. + +Modifying the server is not always an option, and introduces more potential risk. +Attempt local changes first. diff --git a/doc/topics/git/useful_git_commands.md b/doc/topics/git/useful_git_commands.md index 94279e521b6..38b44d97583 100644 --- a/doc/topics/git/useful_git_commands.md +++ b/doc/topics/git/useful_git_commands.md @@ -175,7 +175,7 @@ GIT_TRACE_PACKET=1 GIT_TRACE=2 GIT_CURL_VERBOSE=1 git clone <url> Git includes a complete set of [traces for debugging Git commands](https://git-scm.com/book/en/v2/Git-Internals-Environment-Variables#_debugging), for example: - `GIT_TRACE_PERFORMANCE=1`: enables tracing of performance data, showing how long each particular `git` invocation takes. -- `GIT_TRACE_SETUP=1`: enables tracing of what `git` is discovering about the repository and environment it’s interacting with. +- `GIT_TRACE_SETUP=1`: enables tracing of what `git` is discovering about the repository and environment it's interacting with. - `GIT_TRACE_PACKET=1`: enables packet-level tracing for network operations. ## Rebasing |