diff options
Diffstat (limited to 'doc/administration')
19 files changed, 106 insertions, 108 deletions
diff --git a/doc/administration/geo/disaster_recovery/index.md b/doc/administration/geo/disaster_recovery/index.md index 5eb23422374..ad5284938fa 100644 --- a/doc/administration/geo/disaster_recovery/index.md +++ b/doc/administration/geo/disaster_recovery/index.md @@ -51,7 +51,7 @@ must disable the **primary** node. NOTE: **Note:** (**CentOS only**) In CentOS 6 or older, there is no easy way to prevent GitLab from being - started if the machine reboots isn't available (see [gitlab-org/omnibus-gitlab#3058]). + started if the machine reboots isn't available (see [Omnibus GitLab issue #3058](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/3058)). It may be safest to uninstall the GitLab package completely: ```sh @@ -317,6 +317,5 @@ section to resolve the error. Otherwise, the secret is lost and you'll need to [setup-geo]: ../replication/index.md#setup-instructions [updating-geo]: ../replication/version_specific_updates.md#updating-to-gitlab-105 [sec-tfa]: ../../../security/two_factor_authentication.md#disabling-2fa-for-everyone -[gitlab-org/omnibus-gitlab#3058]: https://gitlab.com/gitlab-org/omnibus-gitlab/issues/3058 [initiate-the-replication-process]: ../replication/database.html#step-3-initiate-the-replication-process [configure-the-primary-server]: ../replication/database.html#step-1-configure-the-primary-server diff --git a/doc/administration/geo/replication/configuration.md b/doc/administration/geo/replication/configuration.md index b8df365e56c..b976db04c00 100644 --- a/doc/administration/geo/replication/configuration.md +++ b/doc/administration/geo/replication/configuration.md @@ -25,7 +25,7 @@ Any change that requires access to the **Admin Area** needs to be done in the GitLab stores a number of secret values in the `/etc/gitlab/gitlab-secrets.json` file which *must* be the same on all nodes. Until there is -a means of automatically replicating these between nodes (see issue [gitlab-org/gitlab#3789]), +a means of automatically replicating these between nodes (see [issue #3789](https://gitlab.com/gitlab-org/gitlab/issues/3789)), they must be manually replicated to the **secondary** node. 1. SSH into the **primary** node, and execute the command below: @@ -75,7 +75,7 @@ they must be manually replicated to the **secondary** node. ### Step 2. Manually replicate the **primary** node's SSH host keys GitLab integrates with the system-installed SSH daemon, designating a user -(typically named git) through which all access requests are handled. +(typically named `git`) through which all access requests are handled. In a [Disaster Recovery] situation, GitLab system administrators will promote a **secondary** node to the **primary** node. DNS records for the @@ -299,7 +299,6 @@ See the [troubleshooting document](troubleshooting.md). [setup-geo-omnibus]: index.md#using-omnibus-gitlab [Hashed Storage]: ../../repository_storage_types.md [Disaster Recovery]: ../disaster_recovery/index.md -[gitlab-org/gitlab#3789]: https://gitlab.com/gitlab-org/gitlab/issues/3789 [gitlab-com/infrastructure#2821]: https://gitlab.com/gitlab-com/infrastructure/issues/2821 [omnibus-ssl]: https://docs.gitlab.com/omnibus/settings/ssl.html [using-geo]: using_a_geo_server.md diff --git a/doc/administration/geo/replication/database.md b/doc/administration/geo/replication/database.md index 33f240ed11f..64b201ef9de 100644 --- a/doc/administration/geo/replication/database.md +++ b/doc/administration/geo/replication/database.md @@ -443,15 +443,15 @@ data before running `pg_basebackup`. The replication process is now complete. -## PGBouncer support (optional) +## PgBouncer support (optional) -[PGBouncer](http://pgbouncer.github.io/) may be used with GitLab Geo to pool -PostgreSQL connections. We recommend using PGBouncer if you use GitLab in a +[PgBouncer](http://pgbouncer.github.io/) may be used with GitLab Geo to pool +PostgreSQL connections. We recommend using PgBouncer if you use GitLab in a high-availability configuration with a cluster of nodes supporting a Geo **primary** node and another cluster of nodes supporting a Geo **secondary** node. For more information, see [High Availability with GitLab Omnibus](../../high_availability/database.md#high-availability-with-gitlab-omnibus-premium-only). -For a Geo **secondary** node to work properly with PGBouncer in front of the database, +For a Geo **secondary** node to work properly with PgBouncer in front of the database, it will need a separate read-only user to make [PostgreSQL FDW queries][FDW] work: diff --git a/doc/administration/geo/replication/faq.md b/doc/administration/geo/replication/faq.md index b3580a706c3..43782b7fc3e 100644 --- a/doc/administration/geo/replication/faq.md +++ b/doc/administration/geo/replication/faq.md @@ -43,9 +43,9 @@ attachments / avatars and the whole database. This means user accounts, issues, merge requests, groups, project data, etc., will be available for query. -## Can I git push to a **secondary** node? +## Can I `git push` to a **secondary** node? -Yes! Pushing directly to a **secondary** node (for both HTTP and SSH, including git-lfs) was [introduced](https://about.gitlab.com/2018/09/22/gitlab-11-3-released/) in [GitLab Premium](https://about.gitlab.com/pricing/#self-managed) 11.3. +Yes! Pushing directly to a **secondary** node (for both HTTP and SSH, including Git LFS) was [introduced](https://about.gitlab.com/2018/09/22/gitlab-11-3-released/) in [GitLab Premium](https://about.gitlab.com/pricing/#self-managed) 11.3. ## How long does it take to have a commit replicated to a **secondary** node? diff --git a/doc/administration/geo/replication/high_availability.md b/doc/administration/geo/replication/high_availability.md index 9d84e10d496..2b6926c1351 100644 --- a/doc/administration/geo/replication/high_availability.md +++ b/doc/administration/geo/replication/high_availability.md @@ -8,7 +8,7 @@ described, it is possible to adapt these instructions to your needs.  -_[diagram source - gitlab employees only][diagram-source]_ +_[diagram source - GitLab employees only][diagram-source]_ The topology above assumes that the **primary** and **secondary** Geo clusters are located in two separate locations, on their own virtual network @@ -274,15 +274,15 @@ After making these changes [Reconfigure GitLab][gitlab-reconfigure] so the chang On the secondary the following GitLab frontend services will be enabled: -- geo-logcursor -- gitlab-pages -- gitlab-workhorse -- logrotate -- nginx -- registry -- remote-syslog -- sidekiq -- unicorn +- `geo-logcursor` +- `gitlab-pages` +- `gitlab-workhorse` +- `logrotate` +- `nginx` +- `registry` +- `remote-syslog` +- `sidekiq` +- `unicorn` Verify these services by running `sudo gitlab-ctl status` on the frontend application servers. diff --git a/doc/administration/geo/replication/index.md b/doc/administration/geo/replication/index.md index f9f56b96e22..58d99863192 100644 --- a/doc/administration/geo/replication/index.md +++ b/doc/administration/geo/replication/index.md @@ -63,7 +63,7 @@ Keep in mind that: - Get user data for logins (API). - Replicate repositories, LFS Objects, and Attachments (HTTPS + JWT). - Since GitLab Premium 10.0, the **primary** node no longer talks to **secondary** nodes to notify for changes (API). -- Pushing directly to a **secondary** node (for both HTTP and SSH, including git-lfs) was [introduced](https://about.gitlab.com/2018/09/22/gitlab-11-3-released/) in [GitLab Premium](https://about.gitlab.com/pricing/#self-managed) 11.3. +- Pushing directly to a **secondary** node (for both HTTP and SSH, including Git LFS) was [introduced](https://about.gitlab.com/2018/09/22/gitlab-11-3-released/) in [GitLab Premium](https://about.gitlab.com/pricing/#self-managed) 11.3. - There are [limitations](#current-limitations) in the current implementation. ### Architecture @@ -240,7 +240,7 @@ This list of limitations only reflects the latest version of GitLab. If you are - Pushing directly to a **secondary** node redirects (for HTTP) or proxies (for SSH) the request to the **primary** node instead of [handling it directly](https://gitlab.com/gitlab-org/gitlab/issues/1381), except when using Git over HTTP with credentials embedded within the URI. For example, `https://user:password@secondary.tld`. - The **primary** node has to be online for OAuth login to happen. Existing sessions and Git are not affected. -- The installation takes multiple manual steps that together can take about an hour depending on circumstances. We are working on improving this experience. See [gitlab-org/omnibus-gitlab#2978](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/2978) for details. +- The installation takes multiple manual steps that together can take about an hour depending on circumstances. We are working on improving this experience. See [Omnibus GitLab issue #2978](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/2978) for details. - Real-time updates of issues/merge requests (for example, via long polling) doesn't work on the **secondary** node. - [Selective synchronization](configuration.md#selective-synchronization) applies only to files and repositories. Other datasets are replicated to the **secondary** node in full, making it inappropriate for use as an access control mechanism. - Object pools for forked project deduplication work only on the **primary** node, and are duplicated on the **secondary** node. diff --git a/doc/administration/geo/replication/security_review.md b/doc/administration/geo/replication/security_review.md index 832d02be9a5..83730d14fa6 100644 --- a/doc/administration/geo/replication/security_review.md +++ b/doc/administration/geo/replication/security_review.md @@ -49,9 +49,9 @@ questions from [owasp.org](https://www.owasp.org). ### How do the end‐users interact with the application? - **Secondary** nodes provide all the interfaces a **primary** node does - (notably a HTTP/HTTPS web application, and HTTP/HTTPS or SSH git repository + (notably a HTTP/HTTPS web application, and HTTP/HTTPS or SSH Git repository access), but is constrained to read-only activities. The principal use case is - envisioned to be cloning git repositories from the **secondary** node in favor of the + envisioned to be cloning Git repositories from the **secondary** node in favor of the **primary** node, but end-users may use the GitLab web interface to view projects, issues, merge requests, snippets, etc. @@ -229,7 +229,7 @@ questions from [owasp.org](https://www.owasp.org). - A static secret shared across all hosts in a GitLab deployment. - In transit, data should be encrypted, although the application does permit communication to proceed unencrypted. The two main transits are the **secondary** node’s - replication process for PostgreSQL, and for git repositories/files. Both should + replication process for PostgreSQL, and for Git repositories/files. Both should be protected using TLS, with the keys for that managed via Omnibus per existing configuration for end-user access to GitLab. diff --git a/doc/administration/geo/replication/troubleshooting.md b/doc/administration/geo/replication/troubleshooting.md index 263fc05dce9..4d64941411a 100644 --- a/doc/administration/geo/replication/troubleshooting.md +++ b/doc/administration/geo/replication/troubleshooting.md @@ -252,7 +252,7 @@ to start again from scratch, there are a few steps that can help you: gitlab-ctl stop geo-logcursor ``` - You can watch sidekiq logs to know when sidekiq jobs processing have finished: + You can watch Sidekiq logs to know when Sidekiq jobs processing have finished: ```sh gitlab-ctl tail sidekiq @@ -280,8 +280,8 @@ to start again from scratch, there are a few steps that can help you: Any uploaded content like file attachments, avatars or LFS objects are stored in a subfolder in one of the two paths below: - - /var/opt/gitlab/gitlab-rails/shared - - /var/opt/gitlab/gitlab-rails/uploads + - `/var/opt/gitlab/gitlab-rails/shared` + - `/var/opt/gitlab/gitlab-rails/uploads` To rename all of them: diff --git a/doc/administration/geo/replication/using_a_geo_server.md b/doc/administration/geo/replication/using_a_geo_server.md index 55b5d486676..fd61e3258e9 100644 --- a/doc/administration/geo/replication/using_a_geo_server.md +++ b/doc/administration/geo/replication/using_a_geo_server.md @@ -4,7 +4,7 @@ After you set up the [database replication and configure the Geo nodes][req], use your closest GitLab node as you would a normal standalone GitLab instance. -Pushing directly to a **secondary** node (for both HTTP, SSH including git-lfs) was [introduced](https://about.gitlab.com/2018/09/22/gitlab-11-3-released/) in [GitLab Premium](https://about.gitlab.com/pricing/#self-managed) 11.3. +Pushing directly to a **secondary** node (for both HTTP, SSH including Git LFS) was [introduced](https://about.gitlab.com/2018/09/22/gitlab-11-3-released/) in [GitLab Premium](https://about.gitlab.com/pricing/#self-managed) 11.3. Example of the output you will see when pushing to a **secondary** node: diff --git a/doc/administration/high_availability/README.md b/doc/administration/high_availability/README.md index e2998e4438a..f59709d063c 100644 --- a/doc/administration/high_availability/README.md +++ b/doc/administration/high_availability/README.md @@ -42,7 +42,7 @@ complexity. - Sidekiq - Asynchronous/Background jobs - PostgreSQL - Database - Consul - Database service discovery and health checks/failover - - PGBouncer - Database pool manager + - PgBouncer - Database pool manager - Redis - Key/Value store (User sessions, cache, queue for Sidekiq) - Sentinel - Redis health check/failover manager - Gitaly - Provides high-level RPC access to Git repositories @@ -138,7 +138,7 @@ the contention. - 3 PostgreSQL nodes - 2 Redis nodes - 3 Consul/Sentinel nodes -- 2 or more GitLab application nodes (Unicorn, Workhorse, Sidekiq, PGBouncer) +- 2 or more GitLab application nodes (Unicorn, Workhorse, Sidekiq, PgBouncer) - 1 NFS/Gitaly server - 1 Monitoring node (Prometheus, Grafana) @@ -165,7 +165,7 @@ contention due to certain workloads. #### Reference Architecture - **Supported Users (approximate):** 10,000 -- **Known Issues:** While validating the reference architecture, slow endpoints were discovered and are being investigated. [gitlab-org/gitlab-foss/issues/64335](https://gitlab.com/gitlab-org/gitlab-foss/issues/64335) +- **Known Issues:** While validating the reference architecture, slow endpoints were discovered and are being investigated. [See issue #64335](https://gitlab.com/gitlab-org/gitlab-foss/issues/64335) The Support and Quality teams built, performance tested, and validated an environment that supports about 10,000 users. The specifications below are a diff --git a/doc/administration/high_availability/consul.md b/doc/administration/high_availability/consul.md index b02a61b9256..25a4ac930f3 100644 --- a/doc/administration/high_availability/consul.md +++ b/doc/administration/high_availability/consul.md @@ -6,7 +6,7 @@ type: reference As part of its High Availability stack, GitLab Premium includes a bundled version of [Consul](https://www.consul.io/) that can be managed through `/etc/gitlab/gitlab.rb`. -A Consul cluster consists of multiple server agents, as well as client agents that run on other nodes which need to talk to the consul cluster. +A Consul cluster consists of multiple server agents, as well as client agents that run on other nodes which need to talk to the Consul cluster. ## Prerequisites @@ -96,7 +96,7 @@ Ideally all nodes will have a `Status` of `alive`. **Note**: This section only applies to server agents. It is safe to restart client agents whenever needed. -If it is necessary to restart the server cluster, it is important to do this in a controlled fashion in order to maintain quorum. If quorum is lost, you will need to follow the consul [outage recovery](#outage-recovery) process to recover the cluster. +If it is necessary to restart the server cluster, it is important to do this in a controlled fashion in order to maintain quorum. If quorum is lost, you will need to follow the Consul [outage recovery](#outage-recovery) process to recover the cluster. To be safe, we recommend you only restart one server agent at a time to ensure the cluster remains intact. @@ -129,7 +129,7 @@ To fix this: 1. Run `gitlab-ctl reconfigure` -If you still see the errors, you may have to [erase the consul database and reinitialize](#recreate-from-scratch) on the affected node. +If you still see the errors, you may have to [erase the Consul database and reinitialize](#recreate-from-scratch) on the affected node. ### Consul agents do not start - Multiple private IPs @@ -162,7 +162,7 @@ If you lost enough server agents in the cluster to break quorum, then the cluste #### Recreate from scratch -By default, GitLab does not store anything in the consul cluster that cannot be recreated. To erase the consul database and reinitialize +By default, GitLab does not store anything in the Consul cluster that cannot be recreated. To erase the Consul database and reinitialize ``` # gitlab-ctl stop consul @@ -174,4 +174,4 @@ After this, the cluster should start back up, and the server agents rejoin. Shor #### Recover a failed cluster -If you have taken advantage of consul to store other data, and want to restore the failed cluster, please follow the [Consul guide](https://www.consul.io/docs/guides/outage.html) to recover a failed cluster. +If you have taken advantage of Consul to store other data, and want to restore the failed cluster, please follow the [Consul guide](https://www.consul.io/docs/guides/outage.html) to recover a failed cluster. diff --git a/doc/administration/high_availability/database.md b/doc/administration/high_availability/database.md index c9f6e768772..61f5c775125 100644 --- a/doc/administration/high_availability/database.md +++ b/doc/administration/high_availability/database.md @@ -153,9 +153,9 @@ Database nodes run two services with PostgreSQL: - Instructing remaining servers to follow the new master node. On failure, the old master node is automatically evicted from the cluster, and should be rejoined manually once recovered. -- Consul. Monitors the status of each node in the database cluster and tracks its health in a service definition on the consul cluster. +- Consul. Monitors the status of each node in the database cluster and tracks its health in a service definition on the Consul cluster. -Alongside pgbouncer, there is a consul agent that watches the status of the PostgreSQL service. If that status changes, consul runs a script which updates the configuration and reloads pgbouncer +Alongside PgBouncer, there is a Consul agent that watches the status of the PostgreSQL service. If that status changes, Consul runs a script which updates the configuration and reloads PgBouncer ##### Connection flow @@ -198,7 +198,7 @@ When using default setup, minimum configuration requires: - `CONSUL_USERNAME`. Defaults to `gitlab-consul` - `CONSUL_DATABASE_PASSWORD`. Password for the database user. -- `CONSUL_PASSWORD_HASH`. This is a hash generated out of consul username/password pair. +- `CONSUL_PASSWORD_HASH`. This is a hash generated out of Consul username/password pair. Can be generated with: ```sh @@ -248,26 +248,26 @@ We will need the following password information for the application's database u sudo gitlab-ctl pg-password-md5 POSTGRESQL_USERNAME ``` -##### Pgbouncer information +##### PgBouncer information When using default setup, minimum configuration requires: - `PGBOUNCER_USERNAME`. Defaults to `pgbouncer` -- `PGBOUNCER_PASSWORD`. This is a password for pgbouncer service. -- `PGBOUNCER_PASSWORD_HASH`. This is a hash generated out of pgbouncer username/password pair. +- `PGBOUNCER_PASSWORD`. This is a password for PgBouncer service. +- `PGBOUNCER_PASSWORD_HASH`. This is a hash generated out of PgBouncer username/password pair. Can be generated with: ```sh sudo gitlab-ctl pg-password-md5 PGBOUNCER_USERNAME ``` -- `PGBOUNCER_NODE`, is the IP address or a FQDN of the node running Pgbouncer. +- `PGBOUNCER_NODE`, is the IP address or a FQDN of the node running PgBouncer. Few notes on the service itself: - The service runs as the same system account as the database - In the package, this is by default `gitlab-psql` -- If you use a non-default user account for Pgbouncer service (by default `pgbouncer`), you will have to specify this username. We will refer to this requirement with `PGBOUNCER_USERNAME`. +- If you use a non-default user account for PgBouncer service (by default `pgbouncer`), you will have to specify this username. We will refer to this requirement with `PGBOUNCER_USERNAME`. - The service will have a regular database user account generated for it - This defaults to `repmgr` - Passwords will be stored in the following locations: @@ -315,7 +315,7 @@ When installing the GitLab package, do not supply `EXTERNAL_URL` value. # Disable automatic database migrations gitlab_rails['auto_migrate'] = false - # Configure the consul agent + # Configure the Consul agent consul['services'] = %w(postgresql) # START user configuration @@ -348,7 +348,7 @@ When installing the GitLab package, do not supply `EXTERNAL_URL` value. 1. On secondary nodes, add all the configuration specified above for primary node to `/etc/gitlab/gitlab.rb`. In addition, append the following configuration - to inform gitlab-ctl that they are standby nodes initially and it need not + to inform `gitlab-ctl` that they are standby nodes initially and it need not attempt to register them as primary node ``` @@ -363,7 +363,7 @@ When installing the GitLab package, do not supply `EXTERNAL_URL` value. > > - If you want your database to listen on a specific interface, change the config: > `postgresql['listen_address'] = '0.0.0.0'`. -> - If your Pgbouncer service runs under a different user account, +> - If your PgBouncer service runs under a different user account, > you also need to specify: `postgresql['pgbouncer_user'] = PGBOUNCER_USERNAME` in > your configuration. @@ -484,9 +484,9 @@ or secondary. The most important thing here is that this command does not produc If there are errors it's most likely due to incorrect `gitlab-consul` database user permissions. Check the [Troubleshooting section](#troubleshooting) before proceeding. -#### Configuring the Pgbouncer node +#### Configuring the PgBouncer node -See our [documentation for Pgbouncer](pgbouncer.md) for information on running Pgbouncer as part of an HA setup. +See our [documentation for PgBouncer](pgbouncer.md) for information on running PgBouncer as part of an HA setup. #### Configuring the Application nodes @@ -515,10 +515,10 @@ Ensure that all migrations ran: gitlab-rake gitlab:db:configure ``` -> **Note**: If you encounter a `rake aborted!` error stating that PGBouncer is failing to connect to -PostgreSQL it may be that your PGBouncer node's IP address is missing from +> **Note**: If you encounter a `rake aborted!` error stating that PgBouncer is failing to connect to +PostgreSQL it may be that your PgBouncer node's IP address is missing from PostgreSQL's `trust_auth_cidr_addresses` in `gitlab.rb` on your database nodes. See -[PGBouncer error `ERROR: pgbouncer cannot connect to server`](#pgbouncer-error-error-pgbouncer-cannot-connect-to-server) +[PgBouncer error `ERROR: pgbouncer cannot connect to server`](#pgbouncer-error-error-pgbouncer-cannot-connect-to-server) in the Troubleshooting section before proceeding. ##### Ensure GitLab is running @@ -533,7 +533,7 @@ Here we'll show you some fully expanded example configurations. ##### Example recommended setup -This example uses 3 consul servers, 3 postgresql servers, and 1 application node. +This example uses 3 Consul servers, 3 PostgreSQL servers, and 1 application node. We start with all servers on the same 10.6.0.0/16 private network range, they can connect to each freely other on those addresses. @@ -589,7 +589,7 @@ postgresql['shared_preload_libraries'] = 'repmgr_funcs' # Disable automatic database migrations gitlab_rails['auto_migrate'] = false -# Configure the consul agent +# Configure the Consul agent consul['services'] = %w(postgresql) postgresql['pgbouncer_user_password'] = '771a8625958a529132abe6f1a4acb19c' @@ -635,7 +635,7 @@ postgresql['enable'] = false pgbouncer['enable'] = true consul['enable'] = true -# Configure Pgbouncer +# Configure PgBouncer pgbouncer['admin_users'] = %w(pgbouncer gitlab-consul) # Configure Consul agent @@ -691,7 +691,7 @@ After deploying the configuration follow these steps: 1. On `10.6.0.31`, our application server - Set gitlab-consul's pgbouncer password to `toomanysecrets` + Set `gitlab-consul` user's PgBouncer password to `toomanysecrets` ```sh gitlab-ctl write-pgpass --host 127.0.0.1 --database pgbouncer --user pgbouncer --hostuser gitlab-consul @@ -705,10 +705,10 @@ After deploying the configuration follow these steps: #### Example minimal setup -This example uses 3 postgresql servers, and 1 application node. +This example uses 3 PostgreSQL servers, and 1 application node. -It differs from the [recommended setup](#example-recommended-setup) by moving the consul servers into the same servers we use for PostgreSQL. -The trade-off is between reducing server counts, against the increased operational complexity of needing to deal with postgres [failover](#failover-procedure) and [restore](#restore-procedure) procedures in addition to [consul outage recovery](consul.md#outage-recovery) on the same set of machines. +It differs from the [recommended setup](#example-recommended-setup) by moving the Consul servers into the same servers we use for PostgreSQL. +The trade-off is between reducing server counts, against the increased operational complexity of needing to deal with postgres [failover](#failover-procedure) and [restore](#restore-procedure) procedures in addition to [Consul outage recovery](consul.md#outage-recovery) on the same set of machines. In this example we start with all servers on the same 10.6.0.0/16 private network range, they can connect to each freely other on those addresses. @@ -744,7 +744,7 @@ postgresql['shared_preload_libraries'] = 'repmgr_funcs' # Disable automatic database migrations gitlab_rails['auto_migrate'] = false -# Configure the consul agent +# Configure the Consul agent consul['services'] = %w(postgresql) postgresql['pgbouncer_user_password'] = '771a8625958a529132abe6f1a4acb19c' @@ -788,7 +788,7 @@ postgresql['enable'] = false pgbouncer['enable'] = true consul['enable'] = true -# Configure Pgbouncer +# Configure PgBouncer pgbouncer['admin_users'] = %w(pgbouncer gitlab-consul) # Configure Consul agent @@ -817,7 +817,7 @@ The manual steps for this configuration are the same as for the [example recomme #### Failover procedure By default, if the master database fails, `repmgrd` should promote one of the -standby nodes to master automatically, and consul will update pgbouncer with +standby nodes to master automatically, and Consul will update PgBouncer with the new master. If you need to failover manually, you have two options: @@ -907,7 +907,7 @@ can require md5 authentication. ##### Trust specific addresses -If you know the IP address, or FQDN of all database and pgbouncer nodes in the +If you know the IP address, or FQDN of all database and PgBouncer nodes in the cluster, you can trust only those nodes. In `/etc/gitlab/gitlab.rb` on all of the database nodes, set @@ -950,7 +950,7 @@ the previous section: 1. Set `postgresql['md5_auth_cidr_addresses']` to the desired value 1. Set `postgresql['sql_replication_user'] = 'gitlab_repmgr'` 1. Reconfigure with `gitlab-ctl reconfigure` - 1. Restart postgresql with `gitlab-ctl restart postgresql` + 1. Restart PostgreSQL with `gitlab-ctl restart postgresql` 1. Create a `.pgpass` file. Enter the `gitlab_repmgr` password twice to when asked: @@ -959,7 +959,7 @@ the previous section: gitlab-ctl write-pgpass --user gitlab_repmgr --hostuser gitlab-psql --database '*' ``` -1. On each pgbouncer node, edit `/etc/gitlab/gitlab.rb`: +1. On each PgBouncer node, edit `/etc/gitlab/gitlab.rb`: 1. Ensure `gitlab_rails['db_password']` is set to the plaintext password for the `gitlab` database user 1. [Reconfigure GitLab] for the changes to take effect @@ -993,7 +993,7 @@ To restart either service, run `gitlab-ctl restart SERVICE` For PostgreSQL, it is usually safe to restart the master node by default. Automatic failover defaults to a 1 minute timeout. Provided the database returns before then, nothing else needs to be done. To be safe, you can stop `repmgrd` on the standby nodes first with `gitlab-ctl stop repmgrd`, then start afterwards with `gitlab-ctl start repmgrd`. -On the consul server nodes, it is important to restart the consul service in a controlled fashion. Read our [consul documentation](consul.md#restarting-the-server-cluster) for instructions on how to restart the service. +On the Consul server nodes, it is important to restart the Consul service in a controlled fashion. Read our [Consul documentation](consul.md#restarting-the-server-cluster) for instructions on how to restart the service. ### `gitlab-ctl repmgr-check-master` command produces errors @@ -1010,16 +1010,16 @@ steps to fix the problem: Now there should not be errors. If errors still occur then there is another problem. -### PGBouncer error `ERROR: pgbouncer cannot connect to server` +### PgBouncer error `ERROR: pgbouncer cannot connect to server` You may get this error when running `gitlab-rake gitlab:db:configure` or you -may see the error in the PGBouncer log file. +may see the error in the PgBouncer log file. ``` PG::ConnectionBad: ERROR: pgbouncer cannot connect to server ``` -The problem may be that your PGBouncer node's IP address is not included in the +The problem may be that your PgBouncer node's IP address is not included in the `trust_auth_cidr_addresses` setting in `/etc/gitlab/gitlab.rb` on the database nodes. You can confirm that this is the issue by checking the PostgreSQL log on the master diff --git a/doc/administration/high_availability/gitlab.md b/doc/administration/high_availability/gitlab.md index 0d1dd06871a..9c6c8dfbc78 100644 --- a/doc/administration/high_availability/gitlab.md +++ b/doc/administration/high_availability/gitlab.md @@ -90,8 +90,8 @@ these additional steps before proceeding with GitLab installation. NOTE: **Note:** When you specify `https` in the `external_url`, as in the example above, GitLab assumes you have SSL certificates in `/etc/gitlab/ssl/`. If - certificates are not present, Nginx will fail to start. See - [Nginx documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https) + certificates are not present, NGINX will fail to start. See + [NGINX documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https) for more information. NOTE: **Note:** It is best to set the `uid` and `gid`s prior to the initial reconfigure @@ -175,7 +175,7 @@ If you enable Monitoring, it must be enabled on **all** GitLab servers. CAUTION: **Warning:** After changing `unicorn['listen']` in `gitlab.rb`, and running `sudo gitlab-ctl reconfigure`, - it can take an extended period of time for unicorn to complete reloading after receiving a `HUP`. + it can take an extended period of time for Unicorn to complete reloading after receiving a `HUP`. For more information, see the [issue](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/4401). ## Troubleshooting diff --git a/doc/administration/high_availability/load_balancer.md b/doc/administration/high_availability/load_balancer.md index f11d27487d1..43a7c442d8c 100644 --- a/doc/administration/high_availability/load_balancer.md +++ b/doc/administration/high_availability/load_balancer.md @@ -27,10 +27,10 @@ options: Configure your load balancer(s) to pass connections on port 443 as 'TCP' rather than 'HTTP(S)' protocol. This will pass the connection to the application nodes -Nginx service untouched. Nginx will have the SSL certificate and listen on port 443. +NGINX service untouched. NGINX will have the SSL certificate and listen on port 443. -See [Nginx HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https) -for details on managing SSL certificates and configuring Nginx. +See [NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https) +for details on managing SSL certificates and configuring NGINX. ### Load Balancer(s) terminate SSL without backend SSL @@ -40,7 +40,7 @@ terminating SSL. Since communication between the load balancer(s) and GitLab will not be secure, there is some additional configuration needed. See -[Nginx Proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#supporting-proxied-ssl) +[NGINX Proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#supporting-proxied-ssl) for details. ### Load Balancer(s) terminate SSL with backend SSL @@ -49,12 +49,12 @@ Configure your load balancer(s) to use the 'HTTP(S)' protocol rather than 'TCP'. The load balancer(s) will be responsible for managing SSL certificates that end users will see. -Traffic will also be secure between the load balancer(s) and Nginx in this +Traffic will also be secure between the load balancer(s) and NGINX in this scenario. There is no need to add configuration for proxied SSL since the connection will be secure all the way. However, configuration will need to be added to GitLab to configure SSL certificates. See -[Nginx HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https) -for details on managing SSL certificates and configuring Nginx. +[NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https) +for details on managing SSL certificates and configuring NGINX. ## Ports diff --git a/doc/administration/high_availability/nfs.md b/doc/administration/high_availability/nfs.md index 987db3c89a5..c4b82ee073a 100644 --- a/doc/administration/high_availability/nfs.md +++ b/doc/administration/high_availability/nfs.md @@ -109,7 +109,7 @@ For more details on another person's experience with EFS, see ## Avoid using CephFS and GlusterFS GitLab strongly recommends against using CephFS and GlusterFS. -These distributed file systems are not well-suited for GitLab's input/output access patterns because git uses many small files and access times and file locking times to propagate will make git activity very slow. +These distributed file systems are not well-suited for GitLab's input/output access patterns because Git uses many small files and access times and file locking times to propagate will make Git activity very slow. ## Avoid using PostgreSQL with NFS @@ -147,7 +147,7 @@ Note there are several options that you should consider using: ## A single NFS mount -It's recommended to nest all gitlab data dirs within a mount, that allows automatic +It's recommended to nest all GitLab data dirs within a mount, that allows automatic restore of backups without manually moving existing data. ``` diff --git a/doc/administration/high_availability/nfs_host_client_setup.md b/doc/administration/high_availability/nfs_host_client_setup.md index 9b0e085fe25..bb8aa61f7b0 100644 --- a/doc/administration/high_availability/nfs_host_client_setup.md +++ b/doc/administration/high_availability/nfs_host_client_setup.md @@ -56,7 +56,7 @@ You may need to update your server's firewall. See the [firewall section](#nfs-i ## Client/ GitLab application node Setup -> Follow the instructions below to connect any GitLab rails application node running +> Follow the instructions below to connect any GitLab Rails application node running inside your HA environment to the NFS server configured above. ### Step 1 - Install NFS Common on Client @@ -108,7 +108,7 @@ When using the default Omnibus configuration you will need to share 5 data locat between all GitLab cluster nodes. No other locations should be shared. Changing the default file locations in `gitlab.rb` on the client allows you to have one main mount point and have all the required locations as subdirectories to use the NFS mount for -git-data. +`git-data`. ```text git_data_dirs({"default" => {"path" => "/nfs/home/var/opt/gitlab-data/git-data"}}) diff --git a/doc/administration/high_availability/pgbouncer.md b/doc/administration/high_availability/pgbouncer.md index b99724d12a2..e7479ad1ecb 100644 --- a/doc/administration/high_availability/pgbouncer.md +++ b/doc/administration/high_availability/pgbouncer.md @@ -2,29 +2,29 @@ type: reference --- -# Working with the bundle Pgbouncer service +# Working with the bundle PgBouncer service -As part of its High Availability stack, GitLab Premium includes a bundled version of [Pgbouncer](https://pgbouncer.github.io/) that can be managed through `/etc/gitlab/gitlab.rb`. +As part of its High Availability stack, GitLab Premium includes a bundled version of [PgBouncer](https://pgbouncer.github.io/) that can be managed through `/etc/gitlab/gitlab.rb`. -In a High Availability setup, Pgbouncer is used to seamlessly migrate database connections between servers in a failover scenario. +In a High Availability setup, PgBouncer is used to seamlessly migrate database connections between servers in a failover scenario. Additionally, it can be used in a non-HA setup to pool connections, speeding up response time while reducing resource usage. -It is recommended to run pgbouncer alongside the `gitlab-rails` service, or on its own dedicated node in a cluster. +It is recommended to run PgBouncer alongside the `gitlab-rails` service, or on its own dedicated node in a cluster. ## Operations -### Running Pgbouncer as part of an HA GitLab installation +### Running PgBouncer as part of an HA GitLab installation 1. Make sure you collect [`CONSUL_SERVER_NODES`](database.md#consul-information), [`CONSUL_PASSWORD_HASH`](database.md#consul-information), and [`PGBOUNCER_PASSWORD_HASH`](database.md#pgbouncer-information) before executing the next step. 1. Edit `/etc/gitlab/gitlab.rb` replacing values noted in the `# START user configuration` section: ```ruby - # Disable all components except Pgbouncer and Consul agent + # Disable all components except PgBouncer and Consul agent roles ['pgbouncer_role'] - # Configure Pgbouncer + # Configure PgBouncer pgbouncer['admin_users'] = %w(pgbouncer gitlab-consul) # Configure Consul agent @@ -59,13 +59,13 @@ It is recommended to run pgbouncer alongside the `gitlab-rails` service, or on i 1. Run `gitlab-ctl reconfigure` 1. Create a `.pgpass` file so Consul is able to - reload pgbouncer. Enter the `PGBOUNCER_PASSWORD` twice when asked: + reload PgBouncer. Enter the `PGBOUNCER_PASSWORD` twice when asked: ```sh gitlab-ctl write-pgpass --host 127.0.0.1 --database pgbouncer --user pgbouncer --hostuser gitlab-consul ``` -#### PGBouncer Checkpoint +#### PgBouncer Checkpoint 1. Ensure the node is talking to the current master: @@ -100,7 +100,7 @@ It is recommended to run pgbouncer alongside the `gitlab-rails` service, or on i (2 rows) ``` -### Running Pgbouncer as part of a non-HA GitLab installation +### Running PgBouncer as part of a non-HA GitLab installation 1. Generate PGBOUNCER_USER_PASSWORD_HASH with the command `gitlab-ctl pg-password-md5 pgbouncer` @@ -119,7 +119,7 @@ It is recommended to run pgbouncer alongside the `gitlab-rails` service, or on i **Note:** If the database was already running, it will need to be restarted after reconfigure by running `gitlab-ctl restart postgresql`. -1. On the node you are running pgbouncer on, make sure the following is set in `/etc/gitlab/gitlab.rb` +1. On the node you are running PgBouncer on, make sure the following is set in `/etc/gitlab/gitlab.rb` ```ruby pgbouncer['enable'] = true @@ -134,7 +134,7 @@ It is recommended to run pgbouncer alongside the `gitlab-rails` service, or on i 1. Run `gitlab-ctl reconfigure` -1. On the node running unicorn, make sure the following is set in `/etc/gitlab/gitlab.rb` +1. On the node running Unicorn, make sure the following is set in `/etc/gitlab/gitlab.rb` ```ruby gitlab_rails['db_host'] = 'PGBOUNCER_HOST' @@ -144,13 +144,13 @@ It is recommended to run pgbouncer alongside the `gitlab-rails` service, or on i 1. Run `gitlab-ctl reconfigure` -1. At this point, your instance should connect to the database through pgbouncer. If you are having issues, see the [Troubleshooting](#troubleshooting) section +1. At this point, your instance should connect to the database through PgBouncer. If you are having issues, see the [Troubleshooting](#troubleshooting) section ## Enable Monitoring > [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/3786) in GitLab 12.0. -If you enable Monitoring, it must be enabled on **all** pgbouncer servers. +If you enable Monitoring, it must be enabled on **all** PgBouncer servers. 1. Create/edit `/etc/gitlab/gitlab.rb` and add the following configuration: @@ -173,11 +173,11 @@ If you enable Monitoring, it must be enabled on **all** pgbouncer servers. 1. Run `sudo gitlab-ctl reconfigure` to compile the configuration. -### Interacting with pgbouncer +### Interacting with PgBouncer #### Administrative console -As part of omnibus-gitlab, we provide a command `gitlab-ctl pgb-console` to automatically connect to the pgbouncer administrative console. Please see the [pgbouncer documentation](https://pgbouncer.github.io/usage.html#admin-console) for detailed instructions on how to interact with the console. +As part of Omnibus GitLab, we provide a command `gitlab-ctl pgb-console` to automatically connect to the PgBouncer administrative console. Please see the [PgBouncer documentation](https://pgbouncer.github.io/usage.html#admin-console) for detailed instructions on how to interact with the console. To start a session, run @@ -235,7 +235,7 @@ ote_pid | tls ## Troubleshooting -In case you are experiencing any issues connecting through pgbouncer, the first place to check is always the logs: +In case you are experiencing any issues connecting through PgBouncer, the first place to check is always the logs: ```shell # gitlab-ctl tail pgbouncer diff --git a/doc/administration/high_availability/redis.md b/doc/administration/high_availability/redis.md index aa616ec91d8..5056413566c 100644 --- a/doc/administration/high_availability/redis.md +++ b/doc/administration/high_availability/redis.md @@ -397,7 +397,7 @@ The prerequisites for a HA Redis setup are the following: 1. [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect. -> Note: You can specify multiple roles like sentinel and redis as: +> Note: You can specify multiple roles like sentinel and Redis as: > `roles ['redis_sentinel_role', 'redis_master_role']`. Read more about high > availability roles at <https://docs.gitlab.com/omnibus/roles/>. @@ -446,7 +446,7 @@ The prerequisites for a HA Redis setup are the following: 1. [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect. 1. Go through the steps again for all the other slave nodes. -> Note: You can specify multiple roles like sentinel and redis as: +> Note: You can specify multiple roles like sentinel and Redis as: > `roles ['redis_sentinel_role', 'redis_slave_role']`. Read more about high > availability roles at <https://docs.gitlab.com/omnibus/roles/>. @@ -628,7 +628,7 @@ single-machine install, to rotate the **Master** to one of the new nodes. Make the required changes in configuration and restart the new nodes again. -To disable redis in the single install, edit `/etc/gitlab/gitlab.rb`: +To disable Redis in the single install, edit `/etc/gitlab/gitlab.rb`: ```ruby redis['enable'] = false @@ -902,7 +902,7 @@ You can check if everything is correct by connecting to each server using /opt/gitlab/embedded/bin/redis-cli -h <redis-host-or-ip> -a '<redis-password>' info replication ``` -When connected to a `master` redis, you will see the number of connected +When connected to a `master` Redis, you will see the number of connected `slaves`, and a list of each with connection details: ``` @@ -948,7 +948,7 @@ to [this issue][gh-531]. You must make sure you are defining the same value in `redis['master_name']` and `redis['master_pasword']` as you defined for your sentinel node. -The way the redis connector `redis-rb` works with sentinel is a bit +The way the Redis connector `redis-rb` works with sentinel is a bit non-intuitive. We try to hide the complexity in omnibus, but it still requires a few extra configs. diff --git a/doc/administration/high_availability/redis_source.md b/doc/administration/high_availability/redis_source.md index 379c64dcf90..9ab9d9a206d 100644 --- a/doc/administration/high_availability/redis_source.md +++ b/doc/administration/high_availability/redis_source.md @@ -341,7 +341,7 @@ to [this upstream issue][gh-531]. You must make sure that `resque.yml` and `sentinel.conf` are configured correctly, otherwise `redis-rb` will not work properly. -The `master-group-name` ('gitlab-redis') defined in (`sentinel.conf`) +The `master-group-name` (`gitlab-redis`) defined in (`sentinel.conf`) **must** be used as the hostname in GitLab (`resque.yml`): ```conf |