diff options
Diffstat (limited to 'doc/administration/geo')
16 files changed, 305 insertions, 92 deletions
diff --git a/doc/administration/geo/replication/database.md b/doc/administration/geo/replication/database.md index 72c3692716b..bddd30dbb2a 100644 --- a/doc/administration/geo/replication/database.md +++ b/doc/administration/geo/replication/database.md @@ -266,13 +266,13 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o 1. SSH into your GitLab **secondary** server and login as root: - ``` + ```sh sudo -i ``` 1. Stop application server and Sidekiq - ``` + ```sh gitlab-ctl stop unicorn gitlab-ctl stop sidekiq ``` @@ -295,7 +295,7 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o 1. Create a file `server.crt` in the **secondary** server, with the content you got on the last step of the **primary** node's setup: - ``` + ```sh editor server.crt ``` diff --git a/doc/administration/geo/replication/datatypes.md b/doc/administration/geo/replication/datatypes.md new file mode 100644 index 00000000000..75ce7503c34 --- /dev/null +++ b/doc/administration/geo/replication/datatypes.md @@ -0,0 +1,168 @@ +# Geo data types support + +A Geo data type is a specific class of data that is required by one or more GitLab features to +store relevant information. + +To replicate data produced by these features with Geo, we use several strategies to access, transfer, and verify them. + +## Data types + +We currently distinguish between three different data types: + +- [Git repositories](#git-repositories) +- [Blobs](#blobs) +- [Database](#database) + +See the list below of each feature or component we replicate, its corresponding data type, replication and +verification methods: + +| Type | Feature / component | Replication method | Verification method | +|----------|-----------------------------------------------|---------------------------------------------|----------------------| +| Database | Application data in PostgreSQL | Native | Native | +| Database | Redis | _N/A_ (*1*) | _N/A_ | +| Database | Elasticsearch | Native | Native | +| Database | Personal snippets | Postgres Replication | Postgres Replication | +| Database | Project snippets | Postgres Replication | Postgres Replication | +| Database | SSH public keys | Postgres Replication | Postgres Replication | +| Git | Project repository | Geo with Gitaly | Gitaly Checksum | +| Git | Project wiki repository | Geo with Gitaly | Gitaly Checksum | +| Git | Project designs repository | Geo with Gitaly | Gitaly Checksum | +| Git | Object pools for forked project deduplication | Geo with Gitaly | _Not implemented_ | +| Blobs | User uploads _(filesystem)_ | Geo with API | _Not implemented_ | +| Blobs | User uploads _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ | +| Blobs | LFS objects _(filesystem)_ | Geo with API | _Not implemented_ | +| Blobs | LFS objects _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ | +| Blobs | CI job artifacts _(filesystem)_ | Geo with API | _Not implemented_ | +| Blobs | CI job artifacts _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ | +| Blobs | Archived CI build traces _(filesystem)_ | Geo with API | _Not implemented_ | +| Blobs | Archived CI build traces _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ | +| Blobs | Container registry _(filesystem)_ | Geo with API/Docker API | _Not implemented_ | +| Blobs | Container registry _(object storage)_ | Geo with API/Managed/Docker API (*2*) | _Not implemented_ | + +- (*1*): Redis replication can be used as part of HA with Redis sentinel. It's not used between Geo nodes. +- (*2*): Object storage replication can be performed by Geo or by your object storage provider/appliance + native replication feature. + +### Git repositories + +A GitLab instance can have one or more repository shards. Each shard has a Gitaly instance that +is responsible for allowing access and operations on the locally stored Git repositories. It can run +on a machine with a single disk, multiple disks mounted as a single mount-point (like with a RAID array), +or using LVM. + +It requires no special filesystem and can work with NFS or a mounted Storage Appliance (there may be +performance limitations when using a remote filesystem). + +Communication is done via Gitaly's own gRPC API. There are three possible ways of synchronization: + +- Using regular Git clone/fetch from one Geo node to another (with special authentication). +- Using repository snapshots (for when the first method fails or repository is corrupt). +- Manual trigger from the Admin UI (a combination of both of the above). + +Each project can have at most 3 different repositories: + +- A project repository, where the source code is stored. +- A wiki repository, where the wiki content is stored. +- A design repository, where design artifacts are indexed (assets are actually in LFS). + +They all live in the same shard and share the same base name with a `-wiki` and `-design` suffix +for Wiki and Design Repository cases. + +### Blobs + +GitLab stores files and blobs such as Issue attachments or LFS objects into either: + +- The filesystem in a specific location. +- An Object Storage solution. Object Storage solutions can be: + - Cloud based like Amazon S3 Google Cloud Storage. + - Self hosted (like MinIO). + - A Storage Appliance that exposes an Object Storage-compatible API. + +When using the filesystem store instead of Object Storage, you need to use network mounted filesystems +to run GitLab when using more than one server (for example with a High Availability setup). + +With respect to replication and verification: + +- We transfer files and blobs using an internal API request. +- With Object Storage, you can either: + - Use a cloud provider replication functionality. + - Have GitLab replicate it for you. + +### Database + +GitLab relies on data stored in multiple databases, for different use-cases. +PostgreSQL is the single point of truth for user-generated content in the Web interface, like issues content, comments +as well as permissions and credentials. + +PostgreSQL can also hold some level of cached data like HTML rendered Markdown, cached merge-requests diff (this can +also be configured to be offloaded to object storage). + +We use PostgreSQL's own replication functionality to replicate data from the primary to secondary nodes. + +We use Redis both as a cache store and to hold persistent data for our background jobs system. Because both +use-cases has data that are exclusive to the same Geo node, we don't replicate it between nodes. + +Elasticsearch is an optional database, that can enable advanced searching capabilities, like improved Global Search +in both source-code level and user generated content in Issues / Merge-Requests and discussions. Currently it's not +supported in Geo. + +## Limitations on replication/verification + +The following table lists the GitLab features along with their replication +and verification status on a **secondary** node. + +You can keep track of the progress to implement the missing items in +these epics/issues: + +- [Unreplicated Data Types](https://gitlab.com/groups/gitlab-org/-/epics/893) +- [Verify all replicated data](https://gitlab.com/groups/gitlab-org/-/epics/1430) + +DANGER: **DANGER** +Features not on this list, or with **No** in the **Replicated** column, +are not replicated on the **secondary** node. Failing over without manually +replicating data from those features will cause the data to be **lost**. +If you wish to use those features on a **secondary** node, or to execute a failover +successfully, you must replicate their data using some other means. + +| Feature | Replicated | Verified | Notes | +|-----------------------------------------------------|--------------------------|-----------------------------|---------------------------------------------| +| Application data in PostgreSQL | **Yes** | **Yes** | | +| Project repository | **Yes** | **Yes** | | +| Project wiki repository | **Yes** | **Yes** | | +| Project designs repository | **Yes** | [No][design-verification] | | +| Uploads | **Yes** | [No][upload-verification] | Verified only on transfer, or manually (*1*)| +| LFS objects | **Yes** | [No][lfs-verification] | Verified only on transfer, or manually (*1*)| +| CI job artifacts (other than traces) | **Yes** | [No][artifact-verification] | Verified only manually (*1*) | +| Archived traces | **Yes** | [No][artifact-verification] | Verified only on transfer, or manually (*1*)| +| Personal snippets | **Yes** | **Yes** | | +| Project snippets | **Yes** | **Yes** | | +| Object pools for forked project deduplication | **Yes** | No | | +| [Server-side Git Hooks][custom-hooks] | No | No | | +| [Elasticsearch integration][elasticsearch] | No | No | | +| [GitLab Pages][gitlab-pages] | [No][pages-replication] | No | | +| [Container Registry][container-registry] | **Yes** | No | | +| [NPM Registry][npm-registry] | No | No | | +| [Maven Repository][maven-repository] | No | No | | +| [Conan Repository][conan-repository] | No | No | | +| [External merge request diffs][merge-request-diffs] | [No][diffs-replication] | No | | +| Content in object storage | **Yes** | No | | + +- (*1*): The integrity can be verified manually using + [Integrity Check Rake Task](../../raketasks/check.md) on both nodes and comparing the output between them. + +[design-replication]: https://gitlab.com/groups/gitlab-org/-/epics/1633 +[design-verification]: https://gitlab.com/gitlab-org/gitlab/issues/32467 +[upload-verification]: https://gitlab.com/groups/gitlab-org/-/epics/1817 +[lfs-verification]: https://gitlab.com/gitlab-org/gitlab/issues/8922 +[artifact-verification]: https://gitlab.com/gitlab-org/gitlab/issues/8923 +[diffs-replication]: https://gitlab.com/gitlab-org/gitlab/issues/33817 +[pages-replication]: https://gitlab.com/groups/gitlab-org/-/epics/589 + +[custom-hooks]: ../../custom_hooks.md +[elasticsearch]: ../../../integration/elasticsearch.md +[gitlab-pages]: ../../pages/index.md +[container-registry]: ../../packages/container_registry.md +[npm-registry]: ../../../user/packages/npm_registry/index.md +[maven-repository]: ../../../user/packages/maven_repository/index.md +[conan-repository]: ../../../user/packages/conan_repository/index.md +[merge-request-diffs]: ../../merge_request_diffs.md diff --git a/doc/administration/geo/replication/external_database.md b/doc/administration/geo/replication/external_database.md index 4451d3c6c08..6948dcc0c68 100644 --- a/doc/administration/geo/replication/external_database.md +++ b/doc/administration/geo/replication/external_database.md @@ -13,13 +13,13 @@ developed and tested. We aim to be compatible with most external 1. SSH into a GitLab **primary** application server and login as root: - ```sh + ```bash sudo -i ``` 1. Execute the command below to define the node as **primary** node: - ```sh + ```bash gitlab-ctl set-geo-primary-node ``` @@ -47,7 +47,7 @@ configures the **primary** node's database to be replicated by making changes to `pg_hba.conf` and `postgresql.conf`. Make the following configuration changes manually to your external database configuration: -``` +```plaintext ## ## Geo Primary Role ## - pg_hba.conf @@ -55,7 +55,7 @@ manually to your external database configuration: host replication gitlab_replicator <trusted secondary IP>/32 md5 ``` -``` +```plaintext ## ## Geo Primary Role ## - postgresql.conf @@ -75,7 +75,7 @@ hot_standby = on Make the following configuration changes manually to your `postgresql.conf` of external replica database: -``` +```plaintext ## ## Geo Secondary Role ## - postgresql.conf diff --git a/doc/administration/geo/replication/high_availability.md b/doc/administration/geo/replication/high_availability.md index faa9d051107..19266a6b358 100644 --- a/doc/administration/geo/replication/high_availability.md +++ b/doc/administration/geo/replication/high_availability.md @@ -123,6 +123,20 @@ a single node only, rather than as a PostgreSQL cluster. Configure the [**secondary** database](database.md) as a read-only replica of the **primary** database. Use the following as a guide. +1. Generate an MD5 hash of the desired password for the database user that the + GitLab application will use to access the read-replica database: + + Note that the username (`gitlab` by default) is incorporated into the hash. + + ```sh + gitlab-ctl pg-password-md5 gitlab + # Enter password: <your_password_here> + # Confirm password: <your_password_here> + # fca0b89a972d69f00eb3ec98a5838484 + ``` + + Use this hash to fill in `<md5_hash_of_your_password>` in the next step. + 1. Edit `/etc/gitlab/gitlab.rb` in the replica database machine, and add the following: @@ -167,6 +181,22 @@ only a single machine, rather than as a PostgreSQL cluster. Configure the tracking database. +1. Generate an MD5 hash of the desired password for the database user that the + GitLab application will use to access the tracking database: + + Note that the username (`gitlab_geo` by default) is incorporated into the + hash. + + ```sh + gitlab-ctl pg-password-md5 gitlab_geo + # Enter password: <your_password_here> + # Confirm password: <your_password_here> + # fca0b89a972d69f00eb3ec98a5838484 + ``` + + Use this hash to fill in `<tracking_database_password_md5_hash>` in the next + step. + 1. Edit `/etc/gitlab/gitlab.rb` in the tracking database machine, and add the following: diff --git a/doc/administration/geo/replication/img/adding_a_secondary_node.png b/doc/administration/geo/replication/img/adding_a_secondary_node.png Binary files differindex 5421b578672..e33b690da18 100644 --- a/doc/administration/geo/replication/img/adding_a_secondary_node.png +++ b/doc/administration/geo/replication/img/adding_a_secondary_node.png diff --git a/doc/administration/geo/replication/img/single_git_add_geolocation_rule.png b/doc/administration/geo/replication/img/single_git_add_geolocation_rule.png Binary files differindex 4b04ba8d1f1..0d1b12e925f 100644 --- a/doc/administration/geo/replication/img/single_git_add_geolocation_rule.png +++ b/doc/administration/geo/replication/img/single_git_add_geolocation_rule.png diff --git a/doc/administration/geo/replication/img/single_git_add_traffic_policy_endpoints.png b/doc/administration/geo/replication/img/single_git_add_traffic_policy_endpoints.png Binary files differindex c19ad57c953..4dfb78986da 100644 --- a/doc/administration/geo/replication/img/single_git_add_traffic_policy_endpoints.png +++ b/doc/administration/geo/replication/img/single_git_add_traffic_policy_endpoints.png diff --git a/doc/administration/geo/replication/img/single_git_clone_panel.png b/doc/administration/geo/replication/img/single_git_clone_panel.png Binary files differindex 8aa0bd2f7d8..427224f5b78 100644 --- a/doc/administration/geo/replication/img/single_git_clone_panel.png +++ b/doc/administration/geo/replication/img/single_git_clone_panel.png diff --git a/doc/administration/geo/replication/img/single_git_create_policy_records_with_traffic_policy.png b/doc/administration/geo/replication/img/single_git_create_policy_records_with_traffic_policy.png Binary files differindex a554532f3b8..ecc4859ca17 100644 --- a/doc/administration/geo/replication/img/single_git_create_policy_records_with_traffic_policy.png +++ b/doc/administration/geo/replication/img/single_git_create_policy_records_with_traffic_policy.png diff --git a/doc/administration/geo/replication/img/single_git_created_policy_record.png b/doc/administration/geo/replication/img/single_git_created_policy_record.png Binary files differindex 74c42395e15..f541c0dd236 100644 --- a/doc/administration/geo/replication/img/single_git_created_policy_record.png +++ b/doc/administration/geo/replication/img/single_git_created_policy_record.png diff --git a/doc/administration/geo/replication/img/single_git_name_policy.png b/doc/administration/geo/replication/img/single_git_name_policy.png Binary files differindex 1a976539e94..5571a41cb3c 100644 --- a/doc/administration/geo/replication/img/single_git_name_policy.png +++ b/doc/administration/geo/replication/img/single_git_name_policy.png diff --git a/doc/administration/geo/replication/img/single_git_policy_diagram.png b/doc/administration/geo/replication/img/single_git_policy_diagram.png Binary files differindex d62952dbbb3..eacd4de0e29 100644 --- a/doc/administration/geo/replication/img/single_git_policy_diagram.png +++ b/doc/administration/geo/replication/img/single_git_policy_diagram.png diff --git a/doc/administration/geo/replication/img/single_git_traffic_policies.png b/doc/administration/geo/replication/img/single_git_traffic_policies.png Binary files differindex b3193c23d99..197b0ac182f 100644 --- a/doc/administration/geo/replication/img/single_git_traffic_policies.png +++ b/doc/administration/geo/replication/img/single_git_traffic_policies.png diff --git a/doc/administration/geo/replication/index.md b/doc/administration/geo/replication/index.md index 0d2ca9ea9d9..04f61775b29 100644 --- a/doc/administration/geo/replication/index.md +++ b/doc/administration/geo/replication/index.md @@ -252,74 +252,13 @@ This list of limitations only reflects the latest version of GitLab. If you are ### Limitations on replication/verification -The following table lists the GitLab features along with their replication -and verification status on a **secondary** node. - You can keep track of the progress to implement the missing items in these epics/issues: - [Unreplicated Data Types](https://gitlab.com/groups/gitlab-org/-/epics/893) - [Verify all replicated data](https://gitlab.com/groups/gitlab-org/-/epics/1430) -| Feature | Replicated | Verified | Notes | -|-----------------------------------------------------|--------------------------|-----------------------------|--------------------------------------------| -| All database content | **Yes** | **Yes** | | -| Project repository | **Yes** | **Yes** | | -| Project wiki repository | **Yes** | **Yes** | | -| Project designs repository | **Yes** | [No][design-verification] | Behind feature flag (2) | -| Uploads | **Yes** | [No][upload-verification] | Verified only on transfer, or manually (1) | -| LFS Objects | **Yes** | [No][lfs-verification] | Verified only on transfer, or manually (1) | -| CI job artifacts (other than traces) | **Yes** | [No][artifact-verification] | Verified only manually (1) | -| Archived traces | **Yes** | [No][artifact-verification] | Verified only on transfer, or manually (1) | -| Personal snippets | **Yes** | **Yes** | | -| Version-controlled personal snippets | No | No | [Not yet supported][unsupported-snippets] | -| Project snippets | **Yes** | **Yes** | | -| Version-controlled project snippets | No | No | [Not yet supported][unsupported-snippets] | -| Object pools for forked project deduplication | **Yes** | No | | -| [Server-side Git Hooks][custom-hooks] | No | No | | -| [Elasticsearch integration][elasticsearch] | No | No | | -| [GitLab Pages][gitlab-pages] | [No][pages-replication] | No | | -| [Container Registry][container-registry] | **Yes** | No | | -| [NPM Registry][npm-registry] | No | No | | -| [Maven Repository][maven-repository] | No | No | | -| [Conan Repository][conan-repository] | No | No | | -| [External merge request diffs][merge-request-diffs] | [No][diffs-replication] | No | | -| Content in object storage | **Yes** | No | | - -[design-replication]: https://gitlab.com/groups/gitlab-org/-/epics/1633 -[design-verification]: https://gitlab.com/gitlab-org/gitlab/issues/32467 -[upload-verification]: https://gitlab.com/groups/gitlab-org/-/epics/1817 -[lfs-verification]: https://gitlab.com/gitlab-org/gitlab/issues/8922 -[artifact-verification]: https://gitlab.com/gitlab-org/gitlab/issues/8923 -[diffs-replication]: https://gitlab.com/gitlab-org/gitlab/issues/33817 -[pages-replication]: https://gitlab.com/groups/gitlab-org/-/epics/589 - -[unsupported-snippets]: https://gitlab.com/gitlab-org/gitlab/issues/14228 -[custom-hooks]: ../../custom_hooks.md -[elasticsearch]: ../../../integration/elasticsearch.md -[gitlab-pages]: ../../pages/index.md -[container-registry]: ../../packages/container_registry.md -[npm-registry]: ../../../user/packages/npm_registry/index.md -[maven-repository]: ../../../user/packages/maven_repository/index.md -[conan-repository]: ../../../user/packages/conan_repository/index.md -[merge-request-diffs]: ../../merge_request_diffs.md - -1. The integrity can be verified manually using -[Integrity Check Rake Task](../../raketasks/check.md) -on both nodes and comparing the output between them. -1. Enable the `enable_geo_design_sync` feature flag by running the -following in a Rails console: - - ```ruby - Feature.disable(:enable_geo_design_sync) - ``` - -DANGER: **DANGER** -Features not on this list, or with **No** in the **Replicated** column, -are not replicated on the **secondary** node. Failing over without manually -replicating data from those features will cause the data to be **lost**. -If you wish to use those features on a **secondary** node, or to execute a failover -successfully, you must replicate their data using some other means. +There is a complete list of all GitLab [data types](datatypes.md) and [existing support for replication and verification](datatypes.md#limitations-on-replicationverification). ## Frequently Asked Questions diff --git a/doc/administration/geo/replication/troubleshooting.md b/doc/administration/geo/replication/troubleshooting.md index 0a2602261d1..7c36d55027a 100644 --- a/doc/administration/geo/replication/troubleshooting.md +++ b/doc/administration/geo/replication/troubleshooting.md @@ -51,6 +51,7 @@ Checking Geo ... GitLab Geo is available ... yes GitLab Geo is enabled ... yes +This machine's Geo node name matches a database record ... yes, found a secondary node named "Shanghai" GitLab Geo secondary database is correctly configured ... yes Database replication enabled? ... yes Database replication working? ... yes @@ -115,34 +116,36 @@ Any **secondary** nodes should point only to read-only instances. #### Can Geo detect the current node correctly? -Geo finds the current machine's name in `/etc/gitlab/gitlab.rb` by: +Geo finds the current machine's Geo node name in `/etc/gitlab/gitlab.rb` by: - Using the `gitlab_rails['geo_node_name']` setting. - If that is not defined, using the `external_url` setting. -To get a machine's name, run: - -```sh -sudo gitlab-rails runner "puts GeoNode.current_node_name" -``` - This name is used to look up the node with the same **Name** in **Admin Area > Geo**. -To check if current machine is correctly finding its node: +To check if the current machine has a node name that matches a node in the +database, run the check task: ```sh -sudo gitlab-rails runner "puts Gitlab::Geo.current_node.inspect" +sudo gitlab-rake gitlab:geo:check ``` -and expect something like: +It displays the current machine's node name and whether the matching database +record is a **primary** or **secondary** node. -```ruby -#<GeoNode id: 2, schema: "https", host: "gitlab.example.com", port: 443, relative_url_root: "", primary: false, ...> +``` +This machine's Geo node name matches a database record ... yes, found a secondary node named "Shanghai" ``` -By running the command above, `primary` should be `true` when executed in -the **primary** node, and `false` on any **secondary** node. +``` +This machine's Geo node name matches a database record ... no + Try fixing it: + You could add or update a Geo node database record, setting the name to "https://example.com/". + Or you could set this machine's Geo node name to match the name of an existing database record: "London", "Shanghai" + For more information see: + doc/administration/geo/replication/troubleshooting.md#can-geo-detect-the-current-node-correctly +``` ## Fixing errors found when running the Geo check rake task @@ -208,9 +211,9 @@ sudo gitlab-rake gitlab:geo:check Checking Geo ... Finished ``` - - Ensure that you have added the secondary node in the admin area of the primary node. + - Ensure that you have added the secondary node in the Admin Area of the primary node. - Ensure that you entered the `external_url` or `gitlab_rails['geo_node_name']` when adding the secondary node in the admin are of the primary node. - - Prior to GitLab 12.4, edit the secondary node in the admin area of the primary node and ensure that there is a trailing `/` in the `Name` field. + - Prior to GitLab 12.4, edit the secondary node in the Admin Area of the primary node and ensure that there is a trailing `/` in the `Name` field. 1. Check returns Exception: PG::UndefinedTable: ERROR: relation "geo_nodes" does not exist @@ -295,7 +298,7 @@ log data to build up in `pg_xlog`. Removing the unused slots can reduce the amou 1. Start a PostgreSQL console session: ```sh - sudo gitlab-psql gitlabhq_production + sudo gitlab-psql ``` Note: **Note:** Using `gitlab-rails dbconsole` will not work, because managing replication slots requires superuser permissions. @@ -318,6 +321,40 @@ Slots where `active` is `f` are not active. SELECT pg_drop_replication_slot('<name_of_extra_slot>'); ``` +### Message: "ERROR: canceling statement due to conflict with recovery" + +This error may rarely occur under normal usage, and the system is resilient +enough to recover. + +However, under certain conditions, some database queries on secondaries may run +excessively long, which increases the frequency of this error. At some point, +some of these queries will never be able to complete due to being canceled +every time. + +These long-running queries are +[planned to be removed in the future](https://gitlab.com/gitlab-org/gitlab/issues/34269), +but as a workaround, we recommend enabling +[hot_standby_feedback](https://www.postgresql.org/docs/10/hot-standby.html#HOT-STANDBY-CONFLICT). +This increases the likelihood of bloat on the **primary** node as it prevents +`VACUUM` from removing recently-dead rows. However, it has been used +successfully in production on GitLab.com. + +To enable `hot_standby_feedback`, add the following to `/etc/gitlab/gitlab.rb` +on the **secondary** node: + +```ruby +postgresql['hot_standby_feedback'] = 'on' +``` + +Then reconfigure GitLab: + +```sh +sudo gitlab-ctl reconfigure +``` + +To help us resolve this problem, consider commenting on +[the issue](https://gitlab.com/gitlab-org/gitlab/issues/4489). + ### Very large repositories never successfully synchronize on the **secondary** node GitLab places a timeout on all repository clones, including project imports @@ -457,16 +494,55 @@ The following steps are for Omnibus installs only. Using Geo with source-based i To check the configuration: +1. SSH into an app node in the **secondary**: + + ```sh + sudo -i + ``` + + Note: An app node is any machine running at least one of the following services: + + - `puma` + - `unicorn` + - `sidekiq` + - `geo-logcursor` + 1. Enter the database console: + If the tracking database is running on the same node: + ```sh gitlab-geo-psql ``` -1. Check whether any tables are present. If everything is working, you - should see something like this: + Or, if the tracking database is running on a different node, you must specify + the user and host when entering the database console: + + ```sh + gitlab-geo-psql -U gitlab_geo -h <IP of tracking database> + ``` + + You will be prompted for the password of the `gitlab_geo` user. You can find + it in plaintext in `/etc/gitlab/gitlab.rb` at: + + ```ruby + geo_secondary['db_password'] = '<geo_tracking_db_password>' + ``` + + This password is normally set on the tracking database during + [Step 3: Configure the tracking database on the secondary node](high_availability.md#step-3-configure-the-tracking-database-on-the-secondary-node), + and it is set on the app nodes during + [Step 4: Configure the frontend application servers on the secondary node](high_availability.md#step-4-configure-the-frontend-application-servers-on-the-secondary-node). + +1. Check whether any tables are present with the following statement: ```sql + SELECT * from information_schema.foreign_tables; + ``` + + If everything is working, you should see something like this: + + ``` gitlabhq_geo_production=# SELECT * from information_schema.foreign_tables; foreign_table_catalog | foreign_table_schema | foreign_table_name | foreign_server_catalog | foreign_server_name -------------------------+----------------------+-------------------------------------------------+-------------------------+--------------------- @@ -482,7 +558,7 @@ To check the configuration: 1. Check that the foreign server mapping is correct via `\des+`. The results should look something like this: - ```sql + ``` gitlabhq_geo_production=# \des+ List of foreign servers -[ RECORD 1 ]--------+------------------------------------------------------------ @@ -518,7 +594,7 @@ To check the configuration: 1. Check that the user mapping is configured properly via `\deu+`: - ```sql + ``` gitlabhq_geo_production=# \deu+ List of user mappings Server | User name | FDW Options diff --git a/doc/administration/geo/replication/version_specific_updates.md b/doc/administration/geo/replication/version_specific_updates.md index 5288cc6e186..7ce38d80c88 100644 --- a/doc/administration/geo/replication/version_specific_updates.md +++ b/doc/administration/geo/replication/version_specific_updates.md @@ -357,7 +357,7 @@ is prepended with the relevant node for better clarity: 1. **(secondary)** Save the snippet below in a file, let's say `/tmp/replica.sh`. Modify the embedded paths if necessary: - ``` + ```bash #!/bin/bash PORT="5432" |
