diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2021-04-20 23:50:22 +0000 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2021-04-20 23:50:22 +0000 |
commit | 9dc93a4519d9d5d7be48ff274127136236a3adb3 (patch) | |
tree | 70467ae3692a0e35e5ea56bcb803eb512a10bedb /doc/topics/git | |
parent | 4b0f34b6d759d6299322b3a54453e930c6121ff0 (diff) | |
download | gitlab-ce-9dc93a4519d9d5d7be48ff274127136236a3adb3.tar.gz |
Add latest changes from gitlab-org/gitlab@13-11-stable-eev13.11.0-rc43
Diffstat (limited to 'doc/topics/git')
-rw-r--r-- | doc/topics/git/feature_branch_development.md | 23 | ||||
-rw-r--r-- | doc/topics/git/lfs/index.md | 8 | ||||
-rw-r--r-- | doc/topics/git/numerous_undo_possibilities_in_git/index.md | 2 | ||||
-rw-r--r-- | doc/topics/git/partial_clone.md | 83 | ||||
-rw-r--r-- | doc/topics/git/tags.md | 2 | ||||
-rw-r--r-- | doc/topics/git/troubleshooting_git.md | 68 | ||||
-rw-r--r-- | doc/topics/git/useful_git_commands.md | 2 |
7 files changed, 115 insertions, 73 deletions
diff --git a/doc/topics/git/feature_branch_development.md b/doc/topics/git/feature_branch_development.md index 5f4b5632f33..8d18441aadd 100644 --- a/doc/topics/git/feature_branch_development.md +++ b/doc/topics/git/feature_branch_development.md @@ -23,10 +23,29 @@ In such cases, it may be more efficient to submit an MR on the release post feat In this case, the feature branch would be `release-X-Y`. Assuming the `release-X-Y` branch already exists, you can set up an MR against that branch, with the following steps: -1. Create a new branch (`test-branch`) against the feature branch (`release-X-Y`): +1. Navigate to the main (master) branch: ```shell - git checkout -b test-branch release-X-Y + git checkout master + ``` + +1. Make sure you have the latest version of your repository: + + ```shell + git fetch + git pull + ``` + +1. Check out the feature branch: + + ```shell + git checkout release-x-y + ``` + +1. Create a new branch (`test-branch`) against the feature branch (`release-x-y`): + + ```shell + git checkout -b test-branch release-x-y ``` You should now be on a branch named `test-branch`. diff --git a/doc/topics/git/lfs/index.md b/doc/topics/git/lfs/index.md index 11c0fcc2373..0851d3f6b50 100644 --- a/doc/topics/git/lfs/index.md +++ b/doc/topics/git/lfs/index.md @@ -114,10 +114,10 @@ See the documentation on [File Locking](../../../user/project/file_lock.md). ## LFS objects in project archives > - Support for including Git LFS blobs inside [project source downloads](../../../user/project/repository/index.md) was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15079) in GitLab 13.5. -> - It was [deployed behind a feature flag](../../../user/feature_flags.md), disabled by default. -> - [Became enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/268409) on GitLab 13.6. -> - It's enabled on GitLab.com. -> - It's recommended for production use. +> - [Deployed behind a feature flag](../../../user/feature_flags.md), disabled by default. +> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/268409) in GitLab 13.6. +> - Enabled on GitLab.com. +> - Recommended for production use. > - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-lfs-objects-in-project-archives). WARNING: diff --git a/doc/topics/git/numerous_undo_possibilities_in_git/index.md b/doc/topics/git/numerous_undo_possibilities_in_git/index.md index 76fc9bc92b0..eba471882f1 100644 --- a/doc/topics/git/numerous_undo_possibilities_in_git/index.md +++ b/doc/topics/git/numerous_undo_possibilities_in_git/index.md @@ -379,7 +379,7 @@ it also provides a clear timeline and development structure. ![Use revert to keep branch flowing](img/revert.png) -If you want to revert changes introduced in certain `commit-id` you can simply +If you want to revert changes introduced in certain `commit-id`, you can revert that `commit-id` (swap additions and deletions) in newly created commit: You can do this with diff --git a/doc/topics/git/partial_clone.md b/doc/topics/git/partial_clone.md index 7f2543f040a..cad29d30af4 100644 --- a/doc/topics/git/partial_clone.md +++ b/doc/topics/git/partial_clone.md @@ -5,11 +5,13 @@ info: "To determine the technical writer assigned to the Stage/Group associated type: reference, howto --- -# Partial Clone **(FREE)** +# Partial clone **(FREE)** As Git repositories grow in size, they can become cumbersome to work with -because of the large amount of history that must be downloaded, and the large -amount of disk space they require. +because of: + +- The large amount of history that must be downloaded. +- The large amount of disk space they require. [Partial clone](https://github.com/git/git/blob/master/Documentation/technical/partial-clone.txt) is a performance optimization that "allows Git to function without having a @@ -58,9 +60,10 @@ Updating files: 100% (13008/13008), done. Filtering content: 100% (3/3), 131.24 MiB | 4.65 MiB/s, done. ``` -The output is longer because Git first clones the repository excluding -files larger than 1 megabyte, and second download any missing large files needed -to checkout the `master` branch. +The output is longer because Git: + +1. Clones the repository excluding files larger than 1 megabyte. +1. Downloads any missing large files needed to check out the default branch. When changing branches, Git may need to download more missing files. @@ -68,9 +71,9 @@ When changing branches, Git may need to download more missing files. > [Introduced](https://gitlab.com/gitlab-org/gitaly/-/issues/2553) in GitLab 12.10. -For enormous repositories with millions of files, and long history, it may be -helpful to exclude all files and use in combination with `sparse-checkout` to -reduce the size of your working copy. +For repositories with millions of files and a long history, you can exclude all files and use +[`git sparse-checkout`](https://git-scm.com/docs/git-sparse-checkout) to reduce the size of +your working copy. ```plaintext # Clone the repo excluding all files @@ -108,21 +111,22 @@ For more details, see the Git documentation for ## Filter by file path -WARNING: -Partial Clone using `sparse` filters is experimental, slow, and -significantly increases Gitaly resource utilization when cloning and fetching. +Deeper integration between partial clone and sparse checkout is possible through the +`--filter=sparse:oid=<blob-ish>` filter spec. This mode of filtering uses a format similar to a +`.gitignore` file to specify which files to include when cloning and fetching. -Deeper integration between Partial Clone and Sparse Checkout is being explored -through the `--filter=sparse:oid=<blob-ish>` filter spec, but this is highly -experimental. This mode of filtering uses a format similar to a `.gitignore` -file to specify which files should be included when cloning and fetching. +WARNING: +Partial clone using `sparse` filters is still experimental. It might be slow and significantly increase +[Gitaly](../../administration/gitaly/index.md) resource utilization when cloning and fetching. +[Filter all blobs and use sparse-checkout](#filter-by-object-type) instead, because +[`git-sparse-checkout`](https://git-scm.com/docs/git-sparse-checkout) simplifies +this type of partial clone use and overcomes its limitations. For more details, see the Git documentation for -[`rev-list-options`](https://gitlab.com/gitlab-org/git/-/blob/9fadedd637b312089337d73c3ed8447e9f0aa775/Documentation/rev-list-options.txt#L735-780). +[`rev-list-options`](https://git-scm.com/docs/git-rev-list#Documentation/git-rev-list.txt---filterltfilter-specgt). -1. **Create a filter spec.** For example, consider a monolithic repository with - many applications, each in a different subdirectory in the root. Create a file - `shiny-app/.filterspec` using the GitLab web interface: +1. Create a filter spec. For example, consider a monolithic repository with many applications, + each in a different subdirectory in the root. Create a file `shiny-app/.filterspec`: ```plaintext # Only the paths listed in the file will be downloaded when performing a @@ -142,28 +146,14 @@ For more details, see the Git documentation for shared-component-b/ ``` -1. **Create a new Git repository and fetch.** Support for `--filter=sparse:oid` - using the clone command is incomplete, so we emulate the clone command - by hand, using `git init` and `git fetch`. Follow - [issue tracking support for `--filter=sparse:oid`](https://gitlab.com/gitlab-org/git/-/issues/4) - for updates. +1. Clone and filter by path. Support for `--filter=sparse:oid` using the + clone command is not yet fully integrated with sparse checkout. ```shell - # Create a new directory for the Git repository - mkdir jumbo-repo && cd jumbo-repo - - # Initialize a new Git repository - git init - - # Add the remote - git remote add origin <url> - - # Enable partial clone support for the remote - git config --local extensions.partialClone origin - # Fetch the filtered set of objects using the filterspec stored on the - # server. WARNING: this step is slow! - git fetch --filter=sparse:oid=master:shiny-app/.gitfilterspec origin + # Clone the filtered set of objects using the filterspec stored on the + # server. WARNING: this step may be very slow! + git clone --sparse --filter=sparse:oid=master:shiny-app/.gitfilterspec <url> # Optional: observe there are missing objects that we have not fetched git rev-list --all --quiet --objects --missing=print | wc -l @@ -175,21 +165,6 @@ For more details, see the Git documentation for entire repository. You many need to disable or reconfigure these integrations. -1. **Sparse checkout** must be enabled and configured to prevent objects from - other paths being downloaded automatically when checking out branches. Follow - [issue proposing automating sparse checkouts](https://gitlab.com/gitlab-org/git/-/issues/5) for updates. - - ```shell - # Enable sparse checkout - git config --local core.sparsecheckout true - - # Configure sparse checkout - git show master:snazzy-app/.gitfilterspec >> .git/info/sparse-checkout - - # Checkout master - git checkout master - ``` - ## Remove partial clone filtering Git repositories with partial clone filtering can have the filtering removed. To diff --git a/doc/topics/git/tags.md b/doc/topics/git/tags.md index 76e3cff3edc..70580ecf778 100644 --- a/doc/topics/git/tags.md +++ b/doc/topics/git/tags.md @@ -28,7 +28,7 @@ git checkout master git tag my_lightweight_tag # Annotated tag -git tag -a v1.0 -m ‘Version 1.0’ +git tag -a v1.0 -m 'Version 1.0' # Show list of the existing tags git tag diff --git a/doc/topics/git/troubleshooting_git.md b/doc/topics/git/troubleshooting_git.md index be4903e2cb9..8db683f6291 100644 --- a/doc/topics/git/troubleshooting_git.md +++ b/doc/topics/git/troubleshooting_git.md @@ -47,15 +47,15 @@ errors can sometimes be caused by underlying issues with SSH (such as authentication). Make sure that SSH is correctly configured by following the instructions in the [SSH troubleshooting](../../ssh/README.md#troubleshooting-ssh-connections) documentation. -If you're a GitLab administrator and have access to the server, you can also prevent -session timeouts by configuring SSH `keep alive` either on the client or on the server. +If you're a GitLab administrator with server access, you can also prevent +session timeouts by configuring SSH `keep-alive` on the client or the server. NOTE: Configuring both the client and the server is unnecessary. **To configure SSH on the client side**: -- On UNIX, edit `~/.ssh/config` (create the file if it doesn’t exist) and +- On UNIX, edit `~/.ssh/config` (create the file if it doesn't exist) and add or edit: ```plaintext @@ -153,7 +153,7 @@ and provide GitLab with more information on how to improve the service. ## `git clone` over HTTP fails with `transfer closed with outstanding read data remaining` error -If the buffer size is lower than what is allowed in the request, the action fails with an error similar to the one below: +Sometimes, when cloning old or large repositories, the following error is thrown: ```plaintext error: RPC failed; curl 18 transfer closed with outstanding read data remaining @@ -162,11 +162,59 @@ fatal: early EOF fatal: index-pack failed ``` -This can be fixed by increasing the existing `http.postBuffer` value to one greater than the repository size. For example, if `git clone` fails when cloning a 500M repository, you should set `http.postBuffer` to `524288000`. That setting ensures the request only starts buffering after the first 524288000 bytes. +This is a common problem with Git itself, due to its inability to handle large files or large quantities of files. +[Git LFS](https://about.gitlab.com/blog/2017/01/30/getting-started-with-git-lfs-tutorial/) was created to work around this problem; however, even it has limitations. It's usually due to one of these reasons: -NOTE: -The default value of `http.postBuffer`, 1 MiB, is applied if the setting is not configured. +- The number of files in the repository. +- The number of revisions in the history. +- The existence of large files in the repository. -```shell -git config http.postBuffer 524288000 -``` +The root causes vary, so multiple potential solutions exist, and you may need to +apply more than one: + +- If this error occurs when cloning a large repository, you can + [decrease the cloning depth](../../ci/large_repositories/index.md#shallow-cloning) + to a value of `1`. For example: + + ```shell + variables: + GIT_DEPTH: 1 + ``` + +- You can increase the + [http.postBuffer](https://git-scm.com/docs/git-config#Documentation/git-config.txt-httppostBuffer) + value in your local Git configuration from the default 1 MB value to a value greater + than the repository size. For example, if `git clone` fails when cloning a 500 MB + repository, you should set `http.postBuffer` to `524288000`: + + ```shell + # Set the http.postBuffer size, in bytes + git config http.postBuffer 524288000 + ``` + +- You can increase the `http.postBuffer` on the server side: + + 1. Modify the GitLab instance's + [`gitlab.rb`](https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/13.5.1+ee.0/files/gitlab-config-template/gitlab.rb.template#L1435-1455) file: + + ```shell + omnibus_gitconfig['system'] = { + # Set the http.postBuffer size, in bytes + "http" => ["postBuffer" => 524288000] + } + ``` + + 1. After applying this change, apply the configuration change: + + ```shell + sudo gitlab-ctl reconfigure + ``` + +For example, if a repository has a very long history and no large files, changing +the depth should fix the problem. However, if a repository has very large files, +even a depth of 1 may be too large, thus requiring the `postBuffer` change. +If you increase your local `postBuffer` but the NGINX value on the backend is still +too small, the error persists. + +Modifying the server is not always an option, and introduces more potential risk. +Attempt local changes first. diff --git a/doc/topics/git/useful_git_commands.md b/doc/topics/git/useful_git_commands.md index 94279e521b6..38b44d97583 100644 --- a/doc/topics/git/useful_git_commands.md +++ b/doc/topics/git/useful_git_commands.md @@ -175,7 +175,7 @@ GIT_TRACE_PACKET=1 GIT_TRACE=2 GIT_CURL_VERBOSE=1 git clone <url> Git includes a complete set of [traces for debugging Git commands](https://git-scm.com/book/en/v2/Git-Internals-Environment-Variables#_debugging), for example: - `GIT_TRACE_PERFORMANCE=1`: enables tracing of performance data, showing how long each particular `git` invocation takes. -- `GIT_TRACE_SETUP=1`: enables tracing of what `git` is discovering about the repository and environment it’s interacting with. +- `GIT_TRACE_SETUP=1`: enables tracing of what `git` is discovering about the repository and environment it's interacting with. - `GIT_TRACE_PACKET=1`: enables packet-level tracing for network operations. ## Rebasing |