diff options
Diffstat (limited to 'doc/administration/geo/replication')
13 files changed, 136 insertions, 78 deletions
diff --git a/doc/administration/geo/replication/configuration.md b/doc/administration/geo/replication/configuration.md index 5b22741f578..88f1ad5b490 100644 --- a/doc/administration/geo/replication/configuration.md +++ b/doc/administration/geo/replication/configuration.md @@ -199,7 +199,7 @@ keys must be manually replicated to the **secondary** site. gitlab-ctl reconfigure ``` -1. On the top bar, select **Menu >** **{admin}** **Admin**. +1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Geo > Sites**. 1. Select **New site**.  @@ -257,7 +257,7 @@ method to be enabled. This is enabled by default, but if converting an existing On the **primary** site: -1. On the top bar, select **Menu >** **{admin}** **Admin**. +1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Settings > General**. 1. Expand **Visibility and access controls**. 1. Ensure "Enabled Git access protocols" is set to either "Both SSH and HTTP(S)" or "Only HTTP(S)". @@ -267,7 +267,7 @@ On the **primary** site: You can sign in to the **secondary** site with the same credentials you used with the **primary** site. After you sign in: -1. On the top bar, select **Menu >** **{admin}** **Admin**. +1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Geo > Sites**. 1. Verify that it's correctly identified as a **secondary** Geo site, and that Geo is enabled. @@ -338,7 +338,7 @@ when: ## Upgrading Geo -See the [updating the Geo sites document](updating_the_geo_nodes.md). +See the [updating the Geo sites document](updating_the_geo_sites.md). ## Troubleshooting diff --git a/doc/administration/geo/replication/datatypes.md b/doc/administration/geo/replication/datatypes.md index a696e5410e5..3f38436429a 100644 --- a/doc/administration/geo/replication/datatypes.md +++ b/doc/administration/geo/replication/datatypes.md @@ -47,12 +47,16 @@ verification methods: | Blobs | Container registry _(object storage)_ | Geo with API/Managed/Docker API (*2*) | _Not implemented_ | | Blobs | Package registry _(file system)_ | Geo with API | SHA256 checksum | | Blobs | Package registry _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ | +| Blobs | Infrastructure registry _(file system)_ | Geo with API | SHA256 checksum | +| Blobs | Infrastructure registry _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ | | Blobs | Versioned Terraform State _(file system)_ | Geo with API | SHA256 checksum | | Blobs | Versioned Terraform State _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ | | Blobs | External Merge Request Diffs _(file system)_ | Geo with API | _Not implemented_ | | Blobs | External Merge Request Diffs _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ | | Blobs | Pipeline artifacts _(file system)_ | Geo with API | SHA256 checksum | | Blobs | Pipeline artifacts _(object storage)_ | Geo with API/Managed (*2*) | SHA256 checksum | +| Blobs | Pages _(file system)_ | Geo with API | _Not implemented_ | +| Blobs | Pages _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ | - (*1*): Redis replication can be used as part of HA with Redis sentinel. It's not used between Geo sites. - (*2*): Object storage replication can be performed by Geo or by your object storage provider/appliance @@ -189,21 +193,15 @@ successfully, you must replicate their data using some other means. |[Job logs](../../job_logs.md) | **Yes** (10.4) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/8923) | Via Object Storage provider if supported. Native Geo support (Beta). | Verified only on transfer or manually using [Integrity Check Rake Task](../../raketasks/check.md) on both sites and comparing the output between them. | |[Container Registry](../../packages/container_registry.md) | **Yes** (12.3) | No | No | Disabled by default. See [instructions](docker_registry.md) to enable. | |[Content in object storage (beta)](object_storage.md) | **Yes** (12.4) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/13845) | No | | -|[Infrastructure Registry for Terraform Module](../../../user/packages/terraform_module_registry/index.md) | **Yes** (14.0) | [**Yes**](#limitation-of-verification-for-files-in-object-storage) (14.0) | Via Object Storage provider if supported. Native Geo support (Beta). | Behind feature flag `geo_package_file_replication`, enabled by default. | +|[Infrastructure Registry](../../../user/packages/infrastructure_registry/index.md) | **Yes** (14.0) | [**Yes**](#limitation-of-verification-for-files-in-object-storage) (14.0) | Via Object Storage provider if supported. Native Geo support (Beta). | Behind feature flag `geo_package_file_replication`, enabled by default. | |[Project designs repository](../../../user/project/issues/design_management.md) | **Yes** (12.7) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/32467) | No | Designs also require replication of LFS objects and Uploads. | -|[Package Registry for npm](../../../user/packages/npm_registry/index.md) | **Yes** (13.2) | [**Yes**](#limitation-of-verification-for-files-in-object-storage) (13.10) | Via Object Storage provider if supported. Native Geo support (Beta). | Behind feature flag `geo_package_file_replication`, enabled by default. | -|[Package Registry for Maven](../../../user/packages/maven_repository/index.md) | **Yes** (13.2) | [**Yes**](#limitation-of-verification-for-files-in-object-storage) (13.10) | Via Object Storage provider if supported. Native Geo support (Beta). | Behind feature flag `geo_package_file_replication`, enabled by default. | -|[Package Registry for Conan](../../../user/packages/conan_repository/index.md) | **Yes** (13.2) | [**Yes**](#limitation-of-verification-for-files-in-object-storage) (13.10) | Via Object Storage provider if supported. Native Geo support (Beta). | Behind feature flag `geo_package_file_replication`, enabled by default. | -|[Package Registry for NuGet](../../../user/packages/nuget_repository/index.md) | **Yes** (13.2) | [**Yes**](#limitation-of-verification-for-files-in-object-storage) (13.10) | Via Object Storage provider if supported. Native Geo support (Beta). | Behind feature flag `geo_package_file_replication`, enabled by default. | -|[Package Registry for PyPI](../../../user/packages/pypi_repository/index.md) | **Yes** (13.2) | [**Yes**](#limitation-of-verification-for-files-in-object-storage) (13.10) | Via Object Storage provider if supported. Native Geo support (Beta). | Behind feature flag `geo_package_file_replication`, enabled by default. | -|[Package Registry for Composer](../../../user/packages/composer_repository/index.md) | **Yes** (13.2) | [**Yes**](#limitation-of-verification-for-files-in-object-storage) (13.10) | Via Object Storage provider if supported. Native Geo support (Beta). | Behind feature flag `geo_package_file_replication`, enabled by default. | -|[Package Registry for generic packages](../../../user/packages/generic_packages/index.md) | **Yes** (13.5) | [**Yes**](#limitation-of-verification-for-files-in-object-storage) (13.10) | Via Object Storage provider if supported. Native Geo support (Beta). | Behind feature flag `geo_package_file_replication`, enabled by default. | +|[Package Registry](../../../user/packages/package_registry/index.md) | **Yes** (13.2) | [**Yes**](#limitation-of-verification-for-files-in-object-storage) (13.10) | Via Object Storage provider if supported. Native Geo support (Beta). | Behind feature flag `geo_package_file_replication`, enabled by default. | |[Versioned Terraform State](../../terraform_state.md) | **Yes** (13.5) | [**Yes**](#limitation-of-verification-for-files-in-object-storage) (13.12) | Via Object Storage provider if supported. Native Geo support (Beta). | Replication is behind the feature flag `geo_terraform_state_version_replication`, enabled by default. Verification was behind the feature flag `geo_terraform_state_version_verification`, which was removed in 14.0| |[External merge request diffs](../../merge_request_diffs.md) | **Yes** (13.5) | No | Via Object Storage provider if supported. Native Geo support (Beta). | Replication is behind the feature flag `geo_merge_request_diff_replication`, enabled by default. Verification is under development, behind the feature flag `geo_merge_request_diff_verification`, introduced in 14.0.| |[Versioned snippets](../../../user/snippets.md#versioned-snippets) | [**Yes** (13.7)](https://gitlab.com/groups/gitlab-org/-/epics/2809) | [**Yes** (14.2)](https://gitlab.com/groups/gitlab-org/-/epics/2810) | No | Verification was implemented behind the feature flag `geo_snippet_repository_verification` in 13.11, and the feature flag was removed in 14.2. | |[Server-side Git hooks](../../server_hooks.md) | [No](https://gitlab.com/groups/gitlab-org/-/epics/1867) | No | No | | |[Elasticsearch integration](../../../integration/elasticsearch.md) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/1186) | No | No | | -|[GitLab Pages](../../pages/index.md) | [No](https://gitlab.com/groups/gitlab-org/-/epics/589) | No | Via Object Storage provider if supported. **No** native Geo support (Beta). | | +|[GitLab Pages](../../pages/index.md) | [**Yes** (14.3)](https://gitlab.com/groups/gitlab-org/-/epics/589) | No | Via Object Storage provider if supported. Native Geo support (Beta). | Behind feature flag `geo_pages_deployment_replication`, enabled by default. | |[Dependency proxy images](../../../user/packages/dependency_proxy/index.md) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/259694) | No | No | Blocked on [Geo: Secondary Mimicry](https://gitlab.com/groups/gitlab-org/-/epics/1528). Replication of this cache is not needed for Disaster Recovery purposes because it can be recreated from external sources. | |[Vulnerability Export](../../../user/application_security/vulnerability_report/#export-vulnerability-details) | [Not planned](https://gitlab.com/groups/gitlab-org/-/epics/3111) | No | | Not planned because they are ephemeral and sensitive. They can be regenerated on demand. | diff --git a/doc/administration/geo/replication/disable_geo.md b/doc/administration/geo/replication/disable_geo.md index 485a5ee1950..ae2597ad649 100644 --- a/doc/administration/geo/replication/disable_geo.md +++ b/doc/administration/geo/replication/disable_geo.md @@ -36,7 +36,7 @@ to do that. To remove the **primary** site: 1. [Remove all secondary Geo sites](remove_geo_site.md) -1. On the top bar, select **Menu >** **{admin}** **Admin**. +1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Geo > Nodes**. 1. Select **Remove** for the **primary** node. 1. Confirm by selecting **Remove** when the prompt appears. diff --git a/doc/administration/geo/replication/docker_registry.md b/doc/administration/geo/replication/docker_registry.md index 5cc4f66017b..4004ef3c17f 100644 --- a/doc/administration/geo/replication/docker_registry.md +++ b/doc/administration/geo/replication/docker_registry.md @@ -130,7 +130,7 @@ For each application and Sidekiq node on the **secondary** site: To verify Container Registry replication is working, on the **secondary** site: -1. On the top bar, select **Menu >** **{admin}** **Admin**. +1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Geo > Nodes**. The initial replication, or "backfill", is probably still in progress. diff --git a/doc/administration/geo/replication/faq.md b/doc/administration/geo/replication/faq.md index 28030dccb3b..70a6e506c28 100644 --- a/doc/administration/geo/replication/faq.md +++ b/doc/administration/geo/replication/faq.md @@ -52,7 +52,7 @@ query. ## Can I `git push` to a **secondary** site? -Yes! Pushing directly to a **secondary** site (for both HTTP and SSH, including Git LFS) was [introduced](https://about.gitlab.com/releases/2018/09/22/gitlab-11-3-released/) in [GitLab Premium](https://about.gitlab.com/pricing/#self-managed) 11.3. +Yes! Pushing directly to a **secondary** site (for both HTTP and SSH, including Git LFS) was [introduced](https://about.gitlab.com/releases/2018/09/22/gitlab-11-3-released/) in GitLab 11.3. ## How long does it take to have a commit replicated to a **secondary** site? diff --git a/doc/administration/geo/replication/object_storage.md b/doc/administration/geo/replication/object_storage.md index d3931c0ba80..1f799b30125 100644 --- a/doc/administration/geo/replication/object_storage.md +++ b/doc/administration/geo/replication/object_storage.md @@ -9,15 +9,19 @@ type: howto Geo can be used in combination with Object Storage (AWS S3, or other compatible object storage). -The storage method for files is recorded in the database, and the database is replicated -from the **primary** Geo site to the **secondary** Geo site, so the **secondary** Geo site -must match the storage method of the **primary** Geo site. -Therefore, if the **primary** Geo site uses object storage, the **secondary** Geo site must use it too. - Currently, **secondary** sites can use either: - The same storage bucket as the **primary** site. - A replicated storage bucket. +- Local storage, if the primary uses local storage. + +The storage method (local or object storage) for files is recorded in the database, and the database +is replicated from the **primary** Geo site to the **secondary** Geo site. + +When accessing an uploaded object, we get its storage method (local or object storage) from the +database, so the **secondary** Geo site must match the storage method of the **primary** Geo site. + +Therefore, if the **primary** Geo site uses object storage, the **secondary** Geo site must use it too. To have: @@ -38,7 +42,7 @@ whether they are stored on the local file system or in object storage. To enable GitLab replication: -1. On the top bar, select **Menu >** **{admin}** **Admin**. +1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Geo > Nodes**. 1. Select **Edit** on the **secondary** site. 1. In the **Synchronization Settings** section, find the **Allow this secondary node to replicate content on Object Storage** diff --git a/doc/administration/geo/replication/remove_geo_site.md b/doc/administration/geo/replication/remove_geo_site.md index 274eb28dbc9..b69dfe2e723 100644 --- a/doc/administration/geo/replication/remove_geo_site.md +++ b/doc/administration/geo/replication/remove_geo_site.md @@ -9,7 +9,7 @@ type: howto **Secondary** sites can be removed from the Geo cluster using the Geo administration page of the **primary** site. To remove a **secondary** site: -1. On the top bar, select **Menu >** **{admin}** **Admin**. +1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Geo > Nodes**. 1. Select the **Remove** button for the **secondary** site you want to remove. 1. Confirm by selecting **Remove** when the prompt appears. diff --git a/doc/administration/geo/replication/troubleshooting.md b/doc/administration/geo/replication/troubleshooting.md index e072696b37c..7b82d742bd5 100644 --- a/doc/administration/geo/replication/troubleshooting.md +++ b/doc/administration/geo/replication/troubleshooting.md @@ -27,7 +27,7 @@ Before attempting more advanced troubleshooting: On the **primary** node: -1. On the top bar, select **Menu >** **{admin}** **Admin**. +1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Geo > Nodes**. We perform the following health checks on each **secondary** node @@ -610,7 +610,7 @@ to start again from scratch, there are a few steps that can help you: ### Design repository failures on mirrored projects and project imports -On the top bar, under **Menu >** **{admin}** **Admin > Geo > Nodes**, +On the top bar, under **Menu > Admin > Geo > Nodes**, if the Design repositories progress bar shows `Synced` and `Failed` greater than 100%, and negative `Queued`, then the instance is likely affected by @@ -836,7 +836,7 @@ node's URL matches its external URL. On the **primary** node: -1. On the top bar, select **Menu >** **{admin}** **Admin**. +1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Geo > Nodes**. 1. Find the affected **secondary** site and select **Edit**. 1. Ensure the **URL** field matches the value found in `/etc/gitlab/gitlab.rb` diff --git a/doc/administration/geo/replication/tuning.md b/doc/administration/geo/replication/tuning.md index 9807f3e6444..bcff6181296 100644 --- a/doc/administration/geo/replication/tuning.md +++ b/doc/administration/geo/replication/tuning.md @@ -14,7 +14,7 @@ in the background. On the **primary** site: -1. On the top bar, select **Menu >** **{admin}** **Admin**. +1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Geo > Nodes**. 1. Select **Edit** of the secondary node you want to tune. 1. Under **Tuning settings**, there are several variables that can be tuned to diff --git a/doc/administration/geo/replication/updating_the_geo_nodes.md b/doc/administration/geo/replication/updating_the_geo_nodes.md index 03570048071..f07c8d547a4 100644 --- a/doc/administration/geo/replication/updating_the_geo_nodes.md +++ b/doc/administration/geo/replication/updating_the_geo_nodes.md @@ -1,52 +1,9 @@ --- -stage: Enablement -group: Geo -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: howto +redirect_to: 'updating_the_geo_sites.md' +remove_date: '2021-11-23' --- -# Updating the Geo nodes **(PREMIUM SELF)** +This file was moved to [another location](updating_the_geo_sites.md). -WARNING: -Read these sections carefully before updating your Geo nodes. Not following -version-specific update steps may result in unexpected downtime. If you have -any specific questions, [contact Support](https://about.gitlab.com/support/#contact-support). - -Updating Geo nodes involves performing: - -1. [Version-specific update steps](version_specific_updates.md), depending on the - version being updated to or from. -1. [General update steps](#general-update-steps), for all updates. - -## General update steps - -NOTE: -These general update steps are not intended for [high-availability deployments](https://docs.gitlab.com/omnibus/update/README.html#multi-node--ha-deployment), and will cause downtime. If you want to avoid downtime, consider using [zero downtime updates](https://docs.gitlab.com/omnibus/update/README.html#zero-downtime-updates). - -To update the Geo nodes when a new GitLab version is released, update **primary** -and all **secondary** nodes: - -1. **Optional:** [Pause replication on each **secondary** node.](../index.md#pausing-and-resuming-replication) -1. Log into the **primary** node. -1. [Update GitLab on the **primary** node using Omnibus](https://docs.gitlab.com/omnibus/update/#update-using-the-official-repositories). -1. Log into each **secondary** node. -1. [Update GitLab on each **secondary** node using Omnibus](https://docs.gitlab.com/omnibus/update/#update-using-the-official-repositories). -1. If you paused replication in step 1, [resume replication on each **secondary**](../index.md#pausing-and-resuming-replication) -1. [Test](#check-status-after-updating) **primary** and **secondary** nodes, and check version in each. - -### Check status after updating - -Now that the update process is complete, you may want to check whether -everything is working correctly: - -1. Run the Geo Rake task on all nodes, everything should be green: - - ```shell - sudo gitlab-rake gitlab:geo:check - ``` - -1. Check the **primary** node's Geo dashboard for any errors. -1. Test the data replication by pushing code to the **primary** node and see if it - is received by **secondary** nodes. - -If you encounter any issues, see the [Geo troubleshooting guide](troubleshooting.md). +<!-- This redirect file can be deleted after <2021-11-23>. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/geo/replication/updating_the_geo_sites.md b/doc/administration/geo/replication/updating_the_geo_sites.md new file mode 100644 index 00000000000..e82afe5f0d4 --- /dev/null +++ b/doc/administration/geo/replication/updating_the_geo_sites.md @@ -0,0 +1,54 @@ +--- +stage: Enablement +group: Geo +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +type: howto +--- + +# Updating the Geo sites **(PREMIUM SELF)** + +WARNING: +Read these sections carefully before updating your Geo sites. Not following +version-specific update steps may result in unexpected downtime. If you have +any specific questions, [contact Support](https://about.gitlab.com/support/#contact-support). + +Updating Geo sites involves performing: + +1. [Version-specific update steps](version_specific_updates.md), depending on the + version being updated to or from. +1. [General update steps](#general-update-steps), for all updates. + +## General update steps + +NOTE: +These general update steps are not intended for multi-site deployments, +and will cause downtime. If you want to avoid downtime, consider using +[zero downtime upgrades](../../../update/zero_downtime.md#multi-node--ha-deployment-with-geo). + +To update the Geo sites when a new GitLab version is released, update **primary** +and all **secondary** sites: + +1. **Optional:** [Pause replication on each **secondary** sites.](../index.md#pausing-and-resuming-replication) +1. SSH into each node of the **primary** site. +1. [Upgrade GitLab on the **primary** site](../../../update/package/index.md#upgrade-using-the-official-repositories). +1. SSH into each node of **secondary** sites. +1. [Upgrade GitLab on each **secondary** site](../../../update/package/index.md#upgrade-using-the-official-repositories). +1. If you paused replication in step 1, [resume replication on each **secondary**](../index.md#pausing-and-resuming-replication) +1. [Test](#check-status-after-updating) **primary** and **secondary** sites, and check version in each. + +### Check status after updating + +Now that the update process is complete, you may want to check whether +everything is working correctly: + +1. Run the Geo Rake task on an application node for the primary and secondary sites. Everything should be green: + + ```shell + sudo gitlab-rake gitlab:geo:check + ``` + +1. Check the **primary** site's Geo dashboard for any errors. +1. Test the data replication by pushing code to the **primary** site and see if it + is received by **secondary** sites. + +If you encounter any issues, see the [Geo troubleshooting guide](troubleshooting.md). diff --git a/doc/administration/geo/replication/usage.md b/doc/administration/geo/replication/usage.md index 7fe8eec467e..f3c8f6ac759 100644 --- a/doc/administration/geo/replication/usage.md +++ b/doc/administration/geo/replication/usage.md @@ -11,7 +11,7 @@ type: howto After you set up the [database replication and configure the Geo nodes](../index.md#setup-instructions), use your closest GitLab site as you would do with the primary one. -You can push directly to a **secondary** site (for both HTTP, SSH including Git LFS), and the request will be proxied to the primary site instead ([introduced](https://about.gitlab.com/releases/2018/09/22/gitlab-11-3-released/) in [GitLab Premium](https://about.gitlab.com/pricing/#self-managed) 11.3). +You can push directly to a **secondary** site (for both HTTP, SSH including Git LFS), and the request will be proxied to the primary site instead ([introduced](https://about.gitlab.com/releases/2018/09/22/gitlab-11-3-released/) in GitLab 11.3). Example of the output you will see when pushing to a **secondary** site: @@ -33,3 +33,13 @@ you can't store credentials in the URL like `user:password@URL`. Instead, you ca for Unix-like operating systems or `_netrc` for Windows. In that case, the credentials will be stored as a plain text. If you're looking for a more secure way to store credentials, you can use [Git Credential Storage](https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage). + +## Fetch Go modules from Geo secondary sites + +Go modules can be pulled from secondary sites, with a number of limitations: + +- Git configuration (using `insteadOf`) is needed to fetch data from the Geo secondary site. +- For private projects, authentication details need to be specified in `~/.netrc`. + +Read more in the +[working with projects `go get` documentation](../../../user/project/working_with_projects.md#fetch-go-modules-from-geo-secondary-sites). diff --git a/doc/administration/geo/replication/version_specific_updates.md b/doc/administration/geo/replication/version_specific_updates.md index 155c06f90b8..84193e6baac 100644 --- a/doc/administration/geo/replication/version_specific_updates.md +++ b/doc/administration/geo/replication/version_specific_updates.md @@ -8,9 +8,44 @@ type: howto # Version-specific update instructions **(PREMIUM SELF)** Review this page for update instructions for your version. These steps -accompany the [general steps](updating_the_geo_nodes.md#general-update-steps) +accompany the [general steps](updating_the_geo_sites.md#general-update-steps) for updating Geo nodes. +## Updating to 14.1, 14.2, 14.3 + +We found an [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/336013) where the Container Registry replication wasn't fully working if you used multi-arch images. In case of a multi-arch image, only the primary architecture (for example `amd64`) would be replicated to the secondary node. This has been [fixed in GitLab 14.3](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/67624) and was backported to 14.2 and 14.1, but manual steps are required to force a re-sync. + +You can check if you are affected by running: + +```shell +docker manifest inspect <SECONDARY_IMAGE_LOCATION> | jq '.mediaType' +``` + +Where `<SECONDARY_IMAGE_LOCATION>` is a container image on your secondary node. +If the output matches `application/vnd.docker.distribution.manifest.list.v2+json` +(there can be a `mediaType` entry at several levels, we only care about the top level entry), +then you don't need to do anything. + +Otherwise, on all your **secondary** nodes, in a [Rails console](../../operations/rails_console.md), run the following: + + ```ruby + list_type = 'application/vnd.docker.distribution.manifest.list.v2+json' + + Geo::ContainerRepositoryRegistry.synced.each do |gcr| + cr = gcr.container_repository + primary = Geo::ContainerRepositorySync.new(cr) + cr.tags.each do |tag| + primary_manifest = JSON.parse(primary.send(:client).repository_raw_manifest(cr.path, tag.name)) + next unless primary_manifest['mediaType'].eql?(list_type) + + cr.delete_tag_by_name(tag.name) + end + primary.execute + end + ``` + +If you are running a version prior to 14.1 and are using Geo and multi-arch containers in your Container Registry, we recommend [upgrading](updating_the_geo_sites.md) to at least GitLab 14.1. + ## Updating to GitLab 14.0/14.1 We found an issue where [Primary sites can not be removed from the UI](https://gitlab.com/gitlab-org/gitlab/-/issues/338231). @@ -277,7 +312,7 @@ GitLab 12.2 includes the following minor PostgreSQL updates: This update occurs even if major PostgreSQL updates are disabled. -Before [refreshing Foreign Data Wrapper during a Geo upgrade](https://docs.gitlab.com/omnibus/update/README.html#run-post-deployment-migrations-and-checks), +Before [refreshing Foreign Data Wrapper during a Geo upgrade](../../../update/zero_downtime.md#step-4-run-post-deployment-migrations-and-checks), restart the Geo tracking database: ```shell |