diff options
Diffstat (limited to 'doc/user/project/repository/mirror/bidirectional.md')
-rw-r--r-- | doc/user/project/repository/mirror/bidirectional.md | 171 |
1 files changed, 171 insertions, 0 deletions
diff --git a/doc/user/project/repository/mirror/bidirectional.md b/doc/user/project/repository/mirror/bidirectional.md new file mode 100644 index 00000000000..c4254655fcc --- /dev/null +++ b/doc/user/project/repository/mirror/bidirectional.md @@ -0,0 +1,171 @@ +--- +stage: Create +group: Source Code +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +disqus_identifier: 'https://docs.gitlab.com/ee/workflow/repository_mirroring.html' +--- + +# Bidirectional mirroring **(PREMIUM)** + +> Moved to GitLab Premium in 13.9. + +WARNING: +Bidirectional mirroring may cause conflicts. + +Bidirectional [mirroring](index.md) configures two repositories to both pull from, +and push to, each other. There is no guarantee that either repository can update +without errors. + +## Reduce conflicts in bidirectional mirroring + +If you configure bidirectional mirroring, prepare your repositories for +conflicts. Configure them to reduce conflicts, and how to settle them when they occur: + +- [Mirror only protected branches](index.md#mirror-only-protected-branches). Rewriting + any mirrored commit on either remote causes conflicts and mirroring to fail. +- [Protect the branches](../../protected_branches.md) you want to mirror on both + remotes to prevent conflicts caused by rewriting history. +- Reduce mirroring delay with a [push event webhook](../../integrations/webhook_events.md#push-events). + Bidirectional mirroring creates a race condition where commits made close together + to the same branch cause conflicts. Push event webhooks can help mitigate the race + condition. Push mirroring from GitLab is rate limited to once per minute when only + push mirroring protected branches. +- Prevent conflicts [using a pre-receive hook](#prevent-conflicts-by-using-a-pre-receive-hook). + +## Configure a webhook to trigger an immediate pull to GitLab + +A [push event webhook](../../integrations/webhook_events.md#push-events) in the downstream +instance can help reduce race conditions by syncing changes more frequently. + +Prerequisites: + +- You have configured the [push](push.md#set-up-a-push-mirror-to-another-gitlab-instance-with-2fa-activated) +and [pull](pull.md#pull-from-a-remote-repository) mirrors in the upstream GitLab instance. + +To create the webhook in the downstream instance: + +1. Create a [personal access token](../../../profile/personal_access_tokens.md) with `API` scope. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Webhooks**. +1. Add the webhook **URL**, which (in this case) uses the + [Pull Mirror API](../../../../api/projects.md#start-the-pull-mirroring-process-for-a-project) + request to trigger an immediate pull after a repository update: + + ```plaintext + https://gitlab.example.com/api/v4/projects/:id/mirror/pull?private_token=<your_access_token> + ``` + +1. Select **Push Events**. +1. Select **Add Webhook**. + +To test the integration, select **Test** and confirm GitLab doesn't return an error message. + +## Prevent conflicts by using a pre-receive hook + +WARNING: +This solution negatively affects the performance of Git push operations, because +they are proxied to the upstream Git repository. + +In this configuration, one Git repository acts as the authoritative upstream, and +the other as downstream. This server-side `pre-receive` hook accepts a push only +after first pushing the commit to the upstream repository. Install this hook on +your downstream repository. + +For example: + +```shell +#!/usr/bin/env bash + +# --- Assume only one push mirror target +# Push mirroring remotes are named `remote_mirror_<id>`. +# This line finds the first remote and uses that. +TARGET_REPO=$(git remote | grep -m 1 remote_mirror) + +proxy_push() +{ + # --- Arguments + OLDREV=$(git rev-parse $1) + NEWREV=$(git rev-parse $2) + REFNAME="$3" + + # --- Pattern of branches to proxy pushes + allowlist=$(expr "$branch" : "\(master\)") + + case "$refname" in + refs/heads/*) + branch=$(expr "$refname" : "refs/heads/\(.*\)") + + if [ "$allowlist" = "$branch" ]; then + # handle https://git-scm.com/docs/git-receive-pack#_quarantine_environment + unset GIT_QUARANTINE_PATH + error="$(git push --quiet $TARGET_REPO $NEWREV:$REFNAME 2>&1)" + fail=$? + + if [ "$fail" != "0" ]; then + echo >&2 "" + echo >&2 " Error: updates were rejected by upstream server" + echo >&2 " This is usually caused by another repository pushing changes" + echo >&2 " to the same ref. You may want to first integrate remote changes" + echo >&2 "" + return + fi + fi + ;; + esac +} + +# Allow dual mode: run from the command line just like the update hook, or +# if no arguments are given, then run as a hook script: +if [ -n "$1" -a -n "$2" -a -n "$3" ]; then + # Output to the terminal in command line mode. If someone wanted to + # resend an email, they could redirect the output to sendmail themselves + PAGER= proxy_push $2 $3 $1 +else + # Push is proxied upstream one ref at a time. It is possible for some refs + # to succeed, and others to fail. This results in a failed push. + while read oldrev newrev refname + do + proxy_push $oldrev $newrev $refname + done +fi +``` + +This sample has a few limitations: + +- It may not work for your use case without modification: + - It doesn't regard different types of authentication mechanisms for the mirror. + - It doesn't work with forced updates (rewriting history). + - Only branches that match the `allowlist` patterns are proxy pushed. +- The script circumvents the Git hook quarantine environment because the update of `$TARGET_REPO` + is seen as a ref update, and Git displays warnings about it. + +## Mirror with Perforce Helix via Git Fusion **(PREMIUM)** + +> Moved to GitLab Premium in 13.9. + +WARNING: +Bidirectional mirroring should not be used as a permanent configuration. Refer to +[Migrating from Perforce Helix](../../import/perforce.md) for alternative migration approaches. + +[Git Fusion](https://www.perforce.com/manuals/git-fusion/#Git-Fusion/section_avy_hyc_gl.html) provides a Git interface +to [Perforce Helix](https://www.perforce.com/products). GitLab can use the Perforce Helix +interface to bidirectionally mirror projects. It can help when migrating from Perforce Helix +to GitLab, if overlapping Perforce Helix workspaces cannot be migrated simultaneously. + +If you mirror with Perforce Helix, mirror only protected branches. Perforce Helix +rejects any pushes that rewrite history. Only the fewest number of branches should be mirrored +due to the performance limitations of Git Fusion. + +When you configure mirroring with Perforce Helix by using Git Fusion, we recommend these Git Fusion +settings: + +- Disable `change-pusher`. Otherwise, every commit is rewritten as being committed + by the mirroring account, rather than mapping to existing Perforce Helix users or the `unknown_git` user. +- Use the `unknown_git` user as the commit author, if the GitLab user doesn't exist in + Perforce Helix. + +Read about [Git Fusion settings on Perforce.com](https://www.perforce.com/manuals/git-fusion/Content/Git-Fusion/section_vss_bdw_w3.html#section_zdp_zz1_3l). + +## Related topics + +- [Configure server hooks](../../../../administration/server_hooks.md) on a GitLab server. |