diff options
Diffstat (limited to 'doc/administration/reference_architectures/troubleshooting.md')
| -rw-r--r-- | doc/administration/reference_architectures/troubleshooting.md | 44 |
1 files changed, 12 insertions, 32 deletions
diff --git a/doc/administration/reference_architectures/troubleshooting.md b/doc/administration/reference_architectures/troubleshooting.md index 406ff57c66d..835231ac584 100644 --- a/doc/administration/reference_architectures/troubleshooting.md +++ b/doc/administration/reference_architectures/troubleshooting.md @@ -23,12 +23,12 @@ with the Fog library that GitLab uses. Symptoms include: ### GitLab Pages requires NFS If you intend to use [GitLab Pages](../../user/project/pages/index.md), this currently requires -[NFS](../nfs.md). There is [work in progress](https://gitlab.com/gitlab-org/gitlab-pages/-/issues/196) -to remove this dependency. In the future, GitLab Pages may use -[object storage](https://gitlab.com/gitlab-org/gitlab/-/issues/208135). +[NFS](../nfs.md). There is [work in progress](https://gitlab.com/groups/gitlab-org/-/epics/3901) +to remove this dependency. In the future, GitLab Pages will use +object storage. The dependency on disk storage also prevents Pages being deployed using the -[GitLab Helm chart](https://gitlab.com/gitlab-org/charts/gitlab/-/issues/37). +[GitLab Helm chart](https://gitlab.com/groups/gitlab-org/-/epics/4283). ### Incremental logging is required for CI to use object storage @@ -206,13 +206,8 @@ To make sure your configuration is correct: ## Troubleshooting Gitaly -### Checking versions when using standalone Gitaly nodes - -When using standalone Gitaly nodes, you must make sure they are the same version -as GitLab to ensure full compatibility. Check **Admin Area > Gitaly Servers** on -your GitLab instance and confirm all Gitaly Servers are `Up to date`. - - +If you have any problems when using standalone Gitaly nodes, first +[check all the versions are up to date](../gitaly/index.md#check-versions-when-using-standalone-gitaly-servers). ### `gitaly-debug` @@ -514,39 +509,24 @@ See the suggested fix [in Geo documentation](../geo/replication/troubleshooting. See the suggested fix [in Geo documentation](../geo/replication/troubleshooting.md#message-log--invalid-ip-mask-md5-name-or-service-not-known). -## Troubleshooting PostgreSQL +## Troubleshooting PostgreSQL with Patroni -In case you are experiencing any issues connecting through PgBouncer, the first place to check is always the logs: +In case you are experiencing any issues connecting through PgBouncer, the first place to check is always the logs for PostgreSQL (which is run through Patroni): ```shell -sudo gitlab-ctl tail postgresql +sudo gitlab-ctl tail patroni ``` -### Consul and PostgreSQL changes not taking effect +### Consul and PostgreSQL with Patroni changes not taking effect Due to the potential impacts, `gitlab-ctl reconfigure` only reloads Consul and PostgreSQL, it will not restart the services. However, not all changes can be activated by reloading. -To restart either service, run `gitlab-ctl restart SERVICE` +To restart either service, run `gitlab-ctl restart consul` or `gitlab-ctl restart patroni` respectively. -For PostgreSQL, it is usually safe to restart the master node by default. Automatic failover defaults to a 1 minute timeout. Provided the database returns before then, nothing else needs to be done. To be safe, you can stop `repmgrd` on the standby nodes first with `gitlab-ctl stop repmgrd`, then start afterwards with `gitlab-ctl start repmgrd`. +For PostgreSQL with Patroni, to prevent the primary node from being failed over automatically, it's safest to stop all secondaries first, then restart the primary and finally restart the secondaries again. On the Consul server nodes, it is important to restart the Consul service in a controlled fashion. Read our [Consul documentation](../consul.md#restart-consul) for instructions on how to restart the service. -### `gitlab-ctl repmgr-check-master` command produces errors - -If this command displays errors about database permissions it is likely that something failed during -install, resulting in the `gitlab-consul` database user getting incorrect permissions. Follow these -steps to fix the problem: - -1. On the master database node, connect to the database prompt - `gitlab-psql -d template1` -1. Delete the `gitlab-consul` user - `DROP USER "gitlab-consul";` -1. Exit the database prompt - `\q` -1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) and the user will be re-added with the proper permissions. -1. Change to the `gitlab-consul` user - `su - gitlab-consul` -1. Try the check command again - `gitlab-ctl repmgr-check-master`. - -Now there should not be errors. If errors still occur then there is another problem. - ### PgBouncer error `ERROR: pgbouncer cannot connect to server` You may get this error when running `gitlab-rake gitlab:db:configure` or you |