diff options
| author | Martin Hanzel <mhanzel@gitlab.com> | 2019-06-05 10:18:12 +0200 |
|---|---|---|
| committer | Martin Hanzel <mhanzel@gitlab.com> | 2019-06-05 10:18:12 +0200 |
| commit | 03c72e998dd8016ccda0a6b9515dd3fc0302978a (patch) | |
| tree | 1f7e1623637de75c2807ef7d40f0feae08c687f3 /doc/administration | |
| parent | 3aeea7fb0c1d6389df6d8643ef40dd54aa84d1a8 (diff) | |
| parent | b560ce1e666733f12c65e8b9f659c89256c1775b (diff) | |
| download | gitlab-ce-mh/notes-spec.tar.gz | |
Merge branch 'master' into mh/notes-specmh/notes-spec
Diffstat (limited to 'doc/administration')
40 files changed, 301 insertions, 862 deletions
diff --git a/doc/administration/auth/google_secure_ldap.md b/doc/administration/auth/google_secure_ldap.md index 65a51fc4aa0..760af0cfd1a 100644 --- a/doc/administration/auth/google_secure_ldap.md +++ b/doc/administration/auth/google_secure_ldap.md @@ -27,7 +27,7 @@ The steps below cover: 'Entire domain (GitLab)' or 'Selected organizational units' for both 'Verify user credentials' and 'Read user information'. Select 'Add LDAP Client' - TIP: **Tip:** If you plan to use GitLab [LDAP Group Sync](https://docs.gitlab.com/ee/administration/auth/ldap-ee.html#group-sync) + TIP: **Tip:** If you plan to use GitLab [LDAP Group Sync](ldap-ee.md#group-sync) , turn on 'Read group information'.  diff --git a/doc/administration/compliance.md b/doc/administration/compliance.md index 9c13ff772b3..246addb6dc9 100644 --- a/doc/administration/compliance.md +++ b/doc/administration/compliance.md @@ -12,7 +12,7 @@ GitLab’s [security features](../security/README.md) may also help you meet rel |**[Email all users of a project, group, or entire server](../user/admin_area/settings/terms.md)**<br>An admin can email groups of users based on project or group membership, or email everyone using the GitLab instance. This is great for scheduled maintenance or upgrades.|Starter+|| |**[Omnibus package supports log forwarding](https://docs.gitlab.com/omnibus/settings/logs.html#udp-log-forwarding)**<br>Forward your logs to a central system.|Starter+|| |**[Lock project membership to group](../user/group/index.md#member-lock-starter)**<br>Group owners can prevent new members from being added to projects within a group.|Starter+|✓| -|**[LDAP group sync](https://docs.gitlab.com/ee/administration/auth/ldap-ee.html#group-sync)**<br>GitLab Enterprise Edition gives admins the ability to automatically sync groups and manage SSH keys, permissions, and authentication, so you can focus on building your product, not configuring your tools.|Starter+|| -|**[LDAP group sync filters](https://docs.gitlab.com/ee/administration/auth/ldap-ee.html#group-sync)**<br>GitLab Enterprise Edition Premium gives more flexibility to synchronize with LDAP based on filters, meaning you can leverage LDAP attributes to map GitLab permissions.|Premium+|| +|**[LDAP group sync](auth/ldap-ee.md#group-sync)**<br>GitLab Enterprise Edition gives admins the ability to automatically sync groups and manage SSH keys, permissions, and authentication, so you can focus on building your product, not configuring your tools.|Starter+|| +|**[LDAP group sync filters](auth/ldap-ee.md#group-sync)**<br>GitLab Enterprise Edition Premium gives more flexibility to synchronize with LDAP based on filters, meaning you can leverage LDAP attributes to map GitLab permissions.|Premium+|| |**[Audit logs](audit_events.md)**<br>To maintain the integrity of your code, GitLab Enterprise Edition Premium gives admins the ability to view any modifications made within the GitLab server in an advanced audit log system, so you can control, analyze and track every change.|Premium+|| |**[Auditor users](auditor_users.md)**<br>Auditor users are users who are given read-only access to all projects, groups, and other resources on the GitLab instance.|Premium+|| diff --git a/doc/administration/custom_hooks.md b/doc/administration/custom_hooks.md index 288cb1bf0bb..113514e1ee8 100644 --- a/doc/administration/custom_hooks.md +++ b/doc/administration/custom_hooks.md @@ -5,11 +5,11 @@ Custom Git hooks must be configured on the filesystem of the GitLab server. Only GitLab server administrators will be able to complete these tasks. Please explore [webhooks] and [CI] as an option if you do not have filesystem access. For a user configurable Git hook interface, see -[Push Rules](https://docs.gitlab.com/ee/push_rules/push_rules.html), +[Push Rules](../push_rules/push_rules.md), available in GitLab Enterprise Edition. NOTE: **Note:** -Custom Git hooks won't be replicated to secondary nodes if you use [GitLab Geo][gitlab-geo] +Custom Git hooks won't be replicated to secondary nodes if you use [GitLab Geo](geo/replication/index.md) Git natively supports hooks that are executed on different actions. Examples of server-side git hooks include pre-receive, post-receive, and update. @@ -123,6 +123,5 @@ exit 1 [CI]: ../ci/README.md [hooks]: https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks#Server-Side-Hooks [webhooks]: ../user/project/integrations/webhooks.md -[gitlab-geo]: https://docs.gitlab.com/ee/administration/geo/replication/index.html [5073]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5073 [93]: https://gitlab.com/gitlab-org/gitlab-shell/merge_requests/93 diff --git a/doc/administration/geo/disaster_recovery/background_verification.md b/doc/administration/geo/disaster_recovery/background_verification.md index c7299b6e196..d4c8c2d3624 100644 --- a/doc/administration/geo/disaster_recovery/background_verification.md +++ b/doc/administration/geo/disaster_recovery/background_verification.md @@ -29,12 +29,7 @@ the node more time before scheduling a planned failover. Run the following commands in a Rails console on the **primary** node: ```sh -# Omnibus GitLab gitlab-rails console - -# Installation from source -cd /home/git/gitlab -sudo -u git -H bin/rails console RAILS_ENV=production ``` To check if automatic background verification is enabled: @@ -102,12 +97,7 @@ disable if you need. Run the following commands in a Rails console on the **primary** node: ```sh -# Omnibus GitLab gitlab-rails console - -# Installation from source -cd /home/git/gitlab -sudo -u git -H bin/rails console RAILS_ENV=production ``` To disable automatic background re-verification: @@ -131,31 +121,15 @@ to be resynced without the backoff period: For repositories: -- Omnibus Installation - - ```sh - sudo gitlab-rake geo:verification:repository:reset - ``` - -- Source Installation - - ```sh - sudo -u git -H bundle exec rake geo:verification:repository:reset RAILS_ENV=production - ``` +```sh +sudo gitlab-rake geo:verification:repository:reset +``` For wikis: -- Omnibus Installation - - ```sh - sudo gitlab-rake geo:verification:wiki:reset - ``` - -- Source Installation - - ```sh - sudo -u git -H bundle exec rake geo:verification:wiki:reset RAILS_ENV=production - ``` +```sh +sudo gitlab-rake geo:verification:wiki:reset +``` ## Reconcile differences with checksum mismatches @@ -169,7 +143,7 @@ If the **primary** and **secondary** nodes have a checksum verification mismatch 1. On the project admin page get the **Gitaly storage name**, and **Gitaly relative path**:  -1. Navigate to the project's repository directory on both **primary** and **secondary** nodes. For an installation from source, the path is usually `/home/git/repositories`. For Omnibus installs, the path is usually `/var/opt/gitlab/git-data/repositories`. Note that if `git_data_dirs` is customized, check the directory layout on your server to be sure. +1. Navigate to the project's repository directory on both **primary** and **secondary** nodes (the path is usually `/var/opt/gitlab/git-data/repositories`). Note that if `git_data_dirs` is customized, check the directory layout on your server to be sure. ```sh cd /var/opt/gitlab/git-data/repositories diff --git a/doc/administration/geo/disaster_recovery/planned_failover.md b/doc/administration/geo/disaster_recovery/planned_failover.md index 88ab12d910a..b8071b5993f 100644 --- a/doc/administration/geo/disaster_recovery/planned_failover.md +++ b/doc/administration/geo/disaster_recovery/planned_failover.md @@ -113,7 +113,7 @@ If any objects are failing to replicate, this should be investigated before scheduling the maintenance window. Following a planned failover, anything that failed to replicate will be **lost**. -You can use the [Geo status API](https://docs.gitlab.com/ee/api/geo_nodes.html#retrieve-project-sync-or-verification-failures-that-occurred-on-the-current-node) to review failed objects and +You can use the [Geo status API](../../../api/geo_nodes.md#retrieve-project-sync-or-verification-failures-that-occurred-on-the-current-node) to review failed objects and the reasons for failure. A common cause of replication failures is the data being missing on the diff --git a/doc/administration/geo/replication/configuration.md b/doc/administration/geo/replication/configuration.md index 57735b21cda..3d4f69d3abe 100644 --- a/doc/administration/geo/replication/configuration.md +++ b/doc/administration/geo/replication/configuration.md @@ -1,9 +1,4 @@ -# Geo configuration (GitLab Omnibus) **[PREMIUM ONLY]** - -NOTE: **Note:** -This is the documentation for the Omnibus GitLab packages. For installations -from source, follow the [**Geo nodes configuration for installations -from source**][configuration-source] guide. +# Geo configuration **[PREMIUM ONLY]** ## Configuring a new **secondary** node @@ -303,7 +298,6 @@ See the [updating the Geo nodes document](updating_the_geo_nodes.md). See the [troubleshooting document](troubleshooting.md). -[configuration-source]: configuration_source.md [setup-geo-omnibus]: index.md#using-omnibus-gitlab [Hashed Storage]: ../../repository_storage_types.md [Disaster Recovery]: ../disaster_recovery/index.md diff --git a/doc/administration/geo/replication/configuration_source.md b/doc/administration/geo/replication/configuration_source.md deleted file mode 100644 index 10dd9405b4b..00000000000 --- a/doc/administration/geo/replication/configuration_source.md +++ /dev/null @@ -1,172 +0,0 @@ -# Geo configuration (source) **[PREMIUM ONLY]** - -NOTE: **Note:** -This documentation applies to GitLab source installations. In GitLab 11.5, this documentation was deprecated and will be removed in a future release. -Please consider [migrating to GitLab Omnibus install](https://docs.gitlab.com/omnibus/update/convert_to_omnibus.html). For installations -using the Omnibus GitLab packages, follow the -[**Omnibus Geo nodes configuration**][configuration] guide. - -## Configuring a new **secondary** node - -NOTE: **Note:** -This is the final step in setting up a **secondary** node. Stages of the setup -process must be completed in the documented order. Before attempting the steps -in this stage, [complete all prior stages](index.md#using-gitlab-installed-from-source-deprecated). - -The basic steps of configuring a **secondary** node are to: - -- Replicate required configurations between the **primary** and **secondary** nodes. -- Configure a tracking database on each **secondary** node. -- Start GitLab on the **secondary** node. - -You are encouraged to first read through all the steps before executing them -in your testing/production environment. - -NOTE: **Note:** -**Do not** set up any custom authentication on **secondary** nodes, this will be handled by the **primary** node. - -NOTE: **Note:** -**Do not** add anything in the **secondary** node's admin area (**Admin Area > Geo**). This is handled solely by the **primary** node. - -### Step 1. Manually replicate secret GitLab values - -GitLab stores a number of secret values in the `/home/git/gitlab/config/secrets.yml` -file which *must* match between the **primary** and **secondary** nodes. Until there is -a means of automatically replicating these between nodes (see [gitlab-org/gitlab-ee#3789]), they must -be manually replicated to **secondary** nodes. - -1. SSH into the **primary** node, and execute the command below: - - ```sh - sudo cat /home/git/gitlab/config/secrets.yml - ``` - - This will display the secrets that need to be replicated, in YAML format. - -1. SSH into the **secondary** node and login as the `git` user: - - ```sh - sudo -i -u git - ``` - -1. Make a backup of any existing secrets: - - ```sh - mv /home/git/gitlab/config/secrets.yml /home/git/gitlab/config/secrets.yml.`date +%F` - ``` - -1. Copy `/home/git/gitlab/config/secrets.yml` from the **primary** node to the **secondary** node, or - copy-and-paste the file contents between nodes: - - ```sh - sudo editor /home/git/gitlab/config/secrets.yml - - # paste the output of the `cat` command you ran on the primary - # save and exit - ``` - -1. Ensure the file permissions are correct: - - ```sh - chown git:git /home/git/gitlab/config/secrets.yml - chmod 0600 /home/git/gitlab/config/secrets.yml - ``` - -1. Restart GitLab - - ```sh - service gitlab restart - ``` - -Once restarted, the **secondary** node will automatically start replicating missing data -from the **primary** node in a process known as backfill. Meanwhile, the **primary** node -will start to notify the **secondary** node of any changes, so that the **secondary** node can -act on those notifications immediately. - -Make sure the **secondary** node is running and accessible. You can login to -the **secondary** node with the same credentials as used for the **primary** node. - -### Step 2. Manually replicate the **primary** node's SSH host keys - -Read [Manually replicate the **primary** node's SSH host keys](configuration.md#step-2-manually-replicate-the-primary-nodes-ssh-host-keys) - -### Step 3. Add the **secondary** GitLab node - -1. Navigate to the **primary** node's **Admin Area > Geo** - (`/admin/geo/nodes`) in your browser. -1. Add the **secondary** node by providing its full URL. **Do NOT** check the - **This is a primary node** checkbox. -1. Optionally, choose which namespaces should be replicated by the - **secondary** node. Leave blank to replicate all. Read more in - [selective synchronization](#selective-synchronization). -1. Click the **Add node** button. -1. SSH into your GitLab **secondary** server and restart the services: - - ```sh - service gitlab restart - ``` - - Check if there are any common issue with your Geo setup by running: - - ```sh - bundle exec rake gitlab:geo:check - ``` - -1. SSH into your GitLab **primary** server and login as root to verify the - **secondary** node is reachable or there are any common issue with your Geo setup: - - ```sh - bundle exec rake gitlab:geo:check - ``` - -Once reconfigured, the **secondary** node will automatically start -replicating missing data from the **primary** node in a process known as backfill. -Meanwhile, the **primary** node will start to notify the **secondary** node of any changes, so -that the **secondary** node can act on those notifications immediately. - -Make sure the **secondary** node is running and accessible. -You can log in to the **secondary** node with the same credentials as used for the **primary** node. - -### Step 4. Enabling Hashed Storage - -Read [Enabling Hashed Storage](configuration.md#step-4-enabling-hashed-storage). - -### Step 5. (Optional) Configuring the secondary to trust the primary - -You can safely skip this step if your **primary** node uses a CA-issued HTTPS certificate. - -If your **primary** node is using a self-signed certificate for *HTTPS* support, you will -need to add that certificate to the **secondary** node's trust store. Retrieve the -certificate from the **primary** node and follow your distribution's instructions for -adding it to the **secondary** node's trust store. In Debian/Ubuntu, you would follow these steps: - -```sh -sudo -i -cp <primary_node_certification_file> /usr/local/share/ca-certificates -update-ca-certificates -``` - -### Step 6. Enable Git access over HTTP/HTTPS - -Geo synchronizes repositories over HTTP/HTTPS, and therefore requires this clone -method to be enabled. Navigate to **Admin Area > Settings** -(`/admin/application_settings`) on the **primary** node, and set -`Enabled Git access protocols` to `Both SSH and HTTP(S)` or `Only HTTP(S)`. - -### Step 7. Verify proper functioning of the secondary node - -Read [Verify proper functioning of the secondary node][configuration-verify-node]. - -## Selective synchronization - -Read [Selective synchronization][configuration-selective-replication]. - -## Troubleshooting - -Read the [troubleshooting document][troubleshooting]. - -[gitlab-org/gitlab-ee#3789]: https://gitlab.com/gitlab-org/gitlab-ee/issues/3789 -[configuration]: configuration.md -[configuration-selective-replication]: configuration.md#selective-synchronization -[configuration-verify-node]: configuration.md#step-7-verify-proper-functioning-of-the-secondary-node -[troubleshooting]: troubleshooting.md diff --git a/doc/administration/geo/replication/database.md b/doc/administration/geo/replication/database.md index e57583a3bf9..c0cdea216cb 100644 --- a/doc/administration/geo/replication/database.md +++ b/doc/administration/geo/replication/database.md @@ -1,9 +1,7 @@ -# Geo database replication (GitLab Omnibus) **[PREMIUM ONLY]** +# Geo database replication **[PREMIUM ONLY]** NOTE: **Note:** -This is the documentation for the Omnibus GitLab packages. For installations -from source, follow the -[Geo database replication (source)](database_source.md) guide. +The following steps are for Omnibus installs only. Using Geo with source-based installs was **deprecated** in GitLab 11.5. NOTE: **Note:** If your GitLab installation uses external (not managed by Omnibus) PostgreSQL @@ -102,10 +100,15 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o else. If you are using an external database not managed by Omnibus GitLab, you need - to create the replicator user and define a password to it manually. - For information on how to create a replication user, refer to the - [appropriate step](database_source.md#step-1-configure-the-primary-server) - in [Geo database replication (source)](database_source.md). + to create the replicator user and define a password to it manually: + + ```sql + --- Create a new user 'replicator' + CREATE USER gitlab_replicator; + + --- Set/change a password and grants replication privilege + ALTER USER gitlab_replicator WITH REPLICATION ENCRYPTED PASSWORD '<replication_password>'; + ``` 1. Configure PostgreSQL to listen on network interfaces: @@ -340,7 +343,7 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o ## ## Secondary address - ## - replace '<secondary_node_ip>' with the public or VPC address of your Geo secondary node + ## - replace '<secondary_node_ip>' with the public or VPC address of your Geo secondary node ## postgresql['listen_address'] = '<secondary_node_ip>' postgresql['md5_auth_cidr_addresses'] = ['<secondary_node_ip>/32'] @@ -383,8 +386,7 @@ the database on the **primary** node, replicates the database, and creates the needed files for streaming replication. The directories used are the defaults that are set up in Omnibus. If you have -changed any defaults or are using a source installation, configure it as you -see fit replacing the directories and paths. +changed any defaults, configure it as you see fit replacing the directories and paths. CAUTION: **Warning:** Make sure to run this on the **secondary** server as it removes all PostgreSQL's @@ -443,7 +445,7 @@ The replication process is now complete. PostgreSQL connections. We recommend using PGBouncer if you use GitLab in a high-availability configuration with a cluster of nodes supporting a Geo **primary** node and another cluster of nodes supporting a Geo **secondary** node. For more -information, see the [Omnibus HA](https://docs.gitlab.com/ee/administration/high_availability/database.html#configure-using-omnibus-for-high-availability) +information, see the [Omnibus HA](../../high_availability/database.md#configure-using-omnibus) documentation. For a Geo **secondary** node to work properly with PGBouncer in front of the database, diff --git a/doc/administration/geo/replication/database_source.md b/doc/administration/geo/replication/database_source.md deleted file mode 100644 index 67cf8b6535f..00000000000 --- a/doc/administration/geo/replication/database_source.md +++ /dev/null @@ -1,439 +0,0 @@ -# Geo database replication (source) **[PREMIUM ONLY]** - -NOTE: **Note:** -This documentation applies to GitLab source installations. In GitLab 11.5, this documentation was deprecated and will be removed in a future release. -Please consider [migrating to GitLab Omnibus install](https://docs.gitlab.com/omnibus/update/convert_to_omnibus.html). For installations -using the Omnibus GitLab packages, follow the -[**database replication for Omnibus GitLab**][database] guide. - -NOTE: **Note:** -The stages of the setup process must be completed in the documented order. -Before attempting the steps in this stage, [complete all prior stages](index.md#using-gitlab-installed-from-source-deprecated). - -This document describes the minimal steps you have to take in order to -replicate your **primary** GitLab database to a **secondary** node's database. You may -have to change some values according to your database setup, how big it is, etc. - -You are encouraged to first read through all the steps before executing them -in your testing/production environment. - -## PostgreSQL replication - -The GitLab **primary** node where the write operations happen will connect to -**primary** database server, and the **secondary** ones which are read-only will -connect to **secondary** database servers (which are read-only too). - -NOTE: **Note:** -In many databases' documentation, you will see "**primary**" being referenced as "master" -and "**secondary**" as either "slave" or "standby" server (read-only). - -We recommend using [PostgreSQL replication slots][replication-slots-article] -to ensure the **primary** node retains all the data necessary for the secondaries to -recover. See below for more details. - -The following guide assumes that: - -- You are using PostgreSQL 9.6 or later which includes the - [`pg_basebackup` tool][pgback] and improved [Foreign Data Wrapper][FDW] support. -- You have a **primary** node already set up (the GitLab server you are - replicating from), running PostgreSQL 9.6 or later, and - you have a new **secondary** server set up with the same versions of the OS, - PostgreSQL, and GitLab on all nodes. -- The IP of the **primary** server for our examples is `198.51.100.1`, whereas the - **secondary** node's IP is `198.51.100.2`. Note that the **primary** and **secondary** servers - **must** be able to communicate over these addresses. These IP addresses can either - be public or private. - -CAUTION: **Warning:** -Geo works with streaming replication. Logical replication is not supported at this time. -There is an [issue where support is being discussed](https://gitlab.com/gitlab-org/gitlab-ee/issues/7420). - -### Step 1. Configure the **primary** server - -1. SSH into your GitLab **primary** server and login as root: - - ```sh - sudo -i - ``` - -1. Add this node as the Geo **primary** by running: - - ```sh - bundle exec rake geo:set_primary_node - ``` - -1. Create a [replication user] named `gitlab_replicator`: - - ```sql - --- Create a new user 'replicator' - CREATE USER gitlab_replicator; - - --- Set/change a password and grants replication privilege - ALTER USER gitlab_replicator WITH REPLICATION ENCRYPTED PASSWORD '<replication_password>'; - ``` - -1. Make sure your the `gitlab` database user has a password defined: - - ```sh - sudo \ - -u postgres psql \ - -d template1 \ - -c "ALTER USER gitlab WITH ENCRYPTED PASSWORD '<database_password>';" - ``` - -1. Edit the content of `database.yml` in `production:` and add the password like the example below: - - ```yaml - # - # PRODUCTION - # - production: - adapter: postgresql - encoding: unicode - database: gitlabhq_production - pool: 10 - username: gitlab - password: <database_password> - host: /var/opt/gitlab/geo-postgresql - ``` - -1. Set up TLS support for the PostgreSQL **primary** server: - - CAUTION: **Warning**: - Only skip this step if you **know** that PostgreSQL traffic - between the **primary** and **secondary** nodes will be secured through some other - means, e.g., a known-safe physical network path or a site-to-site VPN that - you have configured. - - If you are replicating your database across the open Internet, it is - **essential** that the connection is TLS-secured. Correctly configured, this - provides protection against both passive eavesdroppers and active - "man-in-the-middle" attackers. - - To generate a self-signed certificate and key, run this command: - - ```sh - openssl req \ - -nodes \ - -batch \ - -x509 \ - -newkey rsa:4096 \ - -keyout server.key \ - -out server.crt \ - -days 3650 - ``` - - This will create two files - `server.key` and `server.crt` - that you can - use for authentication. - - Copy them to the correct location for your PostgreSQL installation: - - ```sh - # Copying a self-signed certificate and key - install -o postgres -g postgres -m 0400 -T server.crt ~postgres/9.x/main/data/server.crt - install -o postgres -g postgres -m 0400 -T server.key ~postgres/9.x/main/data/server.key - ``` - - Add this configuration to `postgresql.conf`, removing any existing - configuration for `ssl_cert_file` or `ssl_key_file`: - - ``` - ssl = on - ssl_cert_file='server.crt' - ssl_key_file='server.key' - ``` - -1. Edit `postgresql.conf` to configure the **primary** server for streaming replication - (for Debian/Ubuntu that would be `/etc/postgresql/9.x/main/postgresql.conf`): - - ``` - listen_address = '<primary_node_ip>' - wal_level = hot_standby - max_wal_senders = 5 - min_wal_size = 80MB - max_wal_size = 1GB - max_replicaton_slots = 1 # Number of Geo secondary nodes - wal_keep_segments = 10 - hot_standby = on - ``` - - NOTE: **Note**: - Be sure to set `max_replication_slots` to the number of Geo **secondary** - nodes that you may potentially have (at least 1). - - For security reasons, PostgreSQL by default only listens on the local - interface (e.g. 127.0.0.1). However, Geo needs to communicate - between the **primary** and **secondary** nodes over a common network, such as a - corporate LAN or the public Internet. For this reason, we need to - configure PostgreSQL to listen on more interfaces. - - The `listen_address` option opens PostgreSQL up to external connections - with the interface corresponding to the given IP. See [the PostgreSQL - documentation][pg-docs-runtime-conn] for more details. - - You may also want to edit the `wal_keep_segments` and `max_wal_senders` to - match your database replication requirements. Consult the - [PostgreSQL - Replication documentation][pg-docs-runtime-replication] for more information. - -1. Set the access control on the **primary** node to allow TCP connections using the - server's public IP and set the connection from the **secondary** node to require a - password. Edit `pg_hba.conf` (for Debian/Ubuntu that would be - `/etc/postgresql/9.x/main/pg_hba.conf`): - - ```sh - host all all <primary_node_ip>/32 md5 - host replication gitlab_replicator <secondary_node_ip>/32 md5 - ``` - - If you want to add another secondary, add one more row like the replication - one and change the IP address: - - ```sh - host all all <primary_node_ip>/32 md5 - host replication gitlab_replicator <secondary_node_ip>/32 md5 - host replication gitlab_replicator <another_secondary_node_ip>/32 md5 - ``` - -1. Restart PostgreSQL for the changes to take effect. - -1. Choose a database-friendly name to use for your secondary to use as the - replication slot name. For example, if your domain is - `secondary.geo.example.com`, you may use `secondary_example` as the slot - name. - -1. Create the replication slot on the **primary** node: - - ```sh - $ sudo -u postgres psql -c "SELECT * FROM pg_create_physical_replication_slot('secondary_example');" - slot_name | xlog_position - ------------------+--------------- - secondary_example | - (1 row) - ``` - -1. Now that the PostgreSQL server is set up to accept remote connections, run - `netstat -plnt` to make sure that PostgreSQL is listening to the server's - public IP. - -### Step 2. Configure the secondary server - -Follow the first steps in ["configure the secondary server"][database-replication] and note that since you are installing from source, the username and -group listed as `gitlab-psql` in those steps should be replaced by `postgres` -instead. After completing the "Test that the `gitlab-psql` user can connect to -the **primary** node's database" step, continue here: - -1. Edit `postgresql.conf` to configure the secondary for streaming replication - (for Debian/Ubuntu that would be `/etc/postgresql/9.*/main/postgresql.conf`): - - ```sh - wal_level = hot_standby - max_wal_senders = 5 - checkpoint_segments = 10 - wal_keep_segments = 10 - hot_standby = on - ``` - -1. Restart PostgreSQL for the changes to take effect. - -#### Enable tracking database on the secondary server - -Geo secondary nodes use a tracking database to keep track of replication status -and recover automatically from some replication issues. Follow the steps below to create -the tracking database. - -1. On the secondary node, run the following command to create `database_geo.yml` with the - information of your secondary PostgreSQL instance: - - ```sh - sudo cp /home/git/gitlab/config/database_geo.yml.postgresql /home/git/gitlab/config/database_geo.yml - ``` - -1. Edit the content of `database_geo.yml` in `production:` as in the example below: - - ```yaml - # - # PRODUCTION - # - production: - adapter: postgresql - encoding: unicode - database: gitlabhq_geo_production - pool: 10 - username: gitlab_geo - # password: - host: /var/opt/gitlab/geo-postgresql - ``` - -1. Create the database `gitlabhq_geo_production` on the PostgreSQL instance of the **secondary** node. - -1. Set up the Geo tracking database: - - ```sh - bundle exec rake geo:db:migrate - ``` - -1. Configure the [PostgreSQL FDW][FDW] connection and credentials: - - Save the script below in a file, ex. `/tmp/geo_fdw.sh` and modify the connection - params to match your environment. Execute it to set up the FDW connection. - - ```sh - #!/bin/bash - - # Secondary Database connection params: - DB_HOST="/var/opt/gitlab/postgresql" # change to the public IP or VPC private IP if its an external server - DB_NAME="gitlabhq_production" - DB_USER="gitlab" - DB_PORT="5432" - - # Tracking Database connection params: - GEO_DB_HOST="/var/opt/gitlab/geo-postgresql" # change to the public IP or VPC private IP if its an external server - GEO_DB_NAME="gitlabhq_geo_production" - GEO_DB_USER="gitlab_geo" - GEO_DB_PORT="5432" - - query_exec () { - gitlab-psql -h $GEO_DB_HOST -d $GEO_DB_NAME -p $GEO_DB_PORT -c "${1}" - } - - query_exec "CREATE EXTENSION postgres_fdw;" - query_exec "CREATE SERVER gitlab_secondary FOREIGN DATA WRAPPER postgres_fdw OPTIONS (host '${DB_HOST}', dbname '${DB_NAME}', port '${DB_PORT}');" - query_exec "CREATE USER MAPPING FOR ${GEO_DB_USER} SERVER gitlab_secondary OPTIONS (user '${DB_USER}');" - query_exec "CREATE SCHEMA gitlab_secondary;" - query_exec "GRANT USAGE ON FOREIGN SERVER gitlab_secondary TO ${GEO_DB_USER};" - ``` - - And edit the content of `database_geo.yml` and to add `fdw: true` to - the `production:` block. - -### Step 3. Initiate the replication process - -Below we provide a script that connects the database on the **secondary** node to -the database on the **primary** node, replicates the database, and creates the -needed files for streaming replication. - -The directories used are the defaults for Debian/Ubuntu. If you have changed -any defaults, configure it as you see fit replacing the directories and paths. - -CAUTION: **Warning:** -Make sure to run this on the **secondary** server as it removes all PostgreSQL's -data before running `pg_basebackup`. - -1. SSH into your GitLab **secondary** server and login as root: - - ```sh - sudo -i - ``` - -1. Save the snippet below in a file, let's say `/tmp/replica.sh`. Modify the - embedded paths if necessary: - - ``` - #!/bin/bash - - PORT="5432" - USER="gitlab_replicator" - echo --------------------------------------------------------------- - echo WARNING: Make sure this script is run from the secondary server - echo --------------------------------------------------------------- - echo - echo Enter the IP or FQDN of the primary PostgreSQL server - read HOST - echo Enter the password for $USER@$HOST - read -s PASSWORD - echo Enter the required sslmode - read SSLMODE - - echo Stopping PostgreSQL and all GitLab services - sudo service gitlab stop - sudo service postgresql stop - - echo Backing up postgresql.conf - sudo -u postgres mv /var/opt/gitlab/postgresql/data/postgresql.conf /var/opt/gitlab/postgresql/ - - echo Cleaning up old cluster directory - sudo -u postgres rm -rf /var/opt/gitlab/postgresql/data - - echo Starting base backup as the replicator user - echo Enter the password for $USER@$HOST - sudo -u postgres /opt/gitlab/embedded/bin/pg_basebackup -h $HOST -D /var/opt/gitlab/postgresql/data -U gitlab_replicator -v -x -P - - echo Writing recovery.conf file - sudo -u postgres bash -c "cat > /var/opt/gitlab/postgresql/data/recovery.conf <<- _EOF1_ - standby_mode = 'on' - primary_conninfo = 'host=$HOST port=$PORT user=$USER password=$PASSWORD sslmode=$SSLMODE' - _EOF1_ - " - - echo Restoring postgresql.conf - sudo -u postgres mv /var/opt/gitlab/postgresql/postgresql.conf /var/opt/gitlab/postgresql/data/ - - echo Starting PostgreSQL - sudo service postgresql start - ``` - -1. Run it with: - - ```sh - bash /tmp/replica.sh - ``` - - When prompted, enter the IP/FQDN of the **primary** node, and the password you set up - for the `gitlab_replicator` user in the first step. - - You should use `verify-ca` for the `sslmode`. You can use `disable` if you - are happy to skip PostgreSQL TLS authentication altogether (e.g., you know - the network path is secure, or you are using a site-to-site VPN). This is - **not** safe over the public Internet! - - You can read more details about each `sslmode` in the - [PostgreSQL documentation][pg-docs-ssl]; - the instructions above are carefully written to ensure protection against - both passive eavesdroppers and active "man-in-the-middle" attackers. - -The replication process is now over. - -## PGBouncer support (optional) - -1. First, enter the PostgreSQL console as an admin user. - -1. Then create the read-only user: - - ```sql - -- NOTE: Use the password defined earlier - CREATE USER gitlab_geo_fdw WITH password '<your_password_here>'; - GRANT CONNECT ON DATABASE gitlabhq_production to gitlab_geo_fdw; - GRANT USAGE ON SCHEMA public TO gitlab_geo_fdw; - GRANT SELECT ON ALL TABLES IN SCHEMA public TO gitlab_geo_fdw; - GRANT SELECT ON ALL SEQUENCES IN SCHEMA public TO gitlab_geo_fdw; - - -- Tables created by "gitlab" should be made read-only for "gitlab_geo_fdw" - -- automatically. - ALTER DEFAULT PRIVILEGES FOR USER gitlab IN SCHEMA public GRANT SELECT ON TABLES TO gitlab_geo_fdw; - ALTER DEFAULT PRIVILEGES FOR USER gitlab IN SCHEMA public GRANT SELECT ON SEQUENCES TO gitlab_geo_fdw; - ``` - -1. Enter the PostgreSQL console on the **secondary** tracking database and change the user mapping to this new user: - - ``` - ALTER USER MAPPING FOR gitlab_geo SERVER gitlab_secondary OPTIONS (SET user 'gitlab_geo_fdw') - ``` - -## MySQL replication - -MySQL replication is not supported for Geo. - -## Troubleshooting - -Read the [troubleshooting document](troubleshooting.md). - -[replication-slots-article]: https://medium.com/@tk512/replication-slots-in-postgresql-b4b03d277c75 -[pgback]: http://www.postgresql.org/docs/9.6/static/app-pgbasebackup.html -[replication user]:https://wiki.postgresql.org/wiki/Streaming_Replication -[FDW]: https://www.postgresql.org/docs/9.6/static/postgres-fdw.html -[database]: database.md -[add-geo-node]: configuration.md#step-3-add-the-secondary-gitlab-node -[database-replication]: database.md#step-2-configure-the-secondary-server -[pg-docs-ssl]: https://www.postgresql.org/docs/9.6/static/libpq-ssl.html#LIBPQ-SSL-PROTECTION -[pg-docs-runtime-conn]: https://www.postgresql.org/docs/9.6/static/runtime-config-connection.html -[pg-docs-runtime-replication]: https://www.postgresql.org/docs/9.6/static/runtime-config-replication.html diff --git a/doc/administration/geo/replication/external_database.md b/doc/administration/geo/replication/external_database.md index dae5ed911b0..177ca68613e 100644 --- a/doc/administration/geo/replication/external_database.md +++ b/doc/administration/geo/replication/external_database.md @@ -129,7 +129,7 @@ To configure the connection to the external read-replica database and enable Log database to keep track of replication status and automatically recover from potential replication issues. Omnibus automatically configures a tracking database when `roles ['geo_secondary_role']` is set. For high availability, -refer to [Geo High Availability](https://docs.gitlab.com/ee/administration/high_availability). +refer to [Geo High Availability](../../high_availability/README.md). If you want to run this database external to Omnibus, please follow the instructions below. The tracking database requires an [FDW](https://www.postgresql.org/docs/9.6/static/postgres-fdw.html) diff --git a/doc/administration/geo/replication/high_availability.md b/doc/administration/geo/replication/high_availability.md index 715a83a9ff3..921a3ef1c7a 100644 --- a/doc/administration/geo/replication/high_availability.md +++ b/doc/administration/geo/replication/high_availability.md @@ -6,7 +6,7 @@ described, it is possible to adapt these instructions to your needs. ## Architecture overview - + _[diagram source - gitlab employees only][diagram-source]_ @@ -68,7 +68,7 @@ NOTE: **Note:** PostgreSQL and Redis should have already been disabled on the application servers, and connections from the application servers to those services on the backend servers configured, during normal GitLab HA set up. See high availability configuration documentation for -[PostgreSQL](https://docs.gitlab.com/ee/administration/high_availability/database.html#configuring-the-application-nodes) +[PostgreSQL](../../high_availability/database.md#configuring-the-application-nodes) and [Redis](../../high_availability/redis.md#example-configuration-for-the-gitlab-application). The **primary** database will require modification later, as part of diff --git a/doc/administration/geo/replication/index.md b/doc/administration/geo/replication/index.md index 6abea2cf271..54377f7ae4e 100644 --- a/doc/administration/geo/replication/index.md +++ b/doc/administration/geo/replication/index.md @@ -179,37 +179,20 @@ The steps below should be followed in the order they appear. **Make sure the Git If you installed GitLab using the Omnibus packages (highly recommended): 1. [Install GitLab Enterprise Edition](https://about.gitlab.com/installation/) on the server that will serve as the **secondary** node. Do not create an account or log in to the new **secondary** node. -1. [Upload the GitLab License](https://docs.gitlab.com/ee/user/admin_area/license.html) on the **primary** node to unlock Geo. The license must be for [GitLab Premium](https://about.gitlab.com/pricing/) or higher. +1. [Upload the GitLab License](../../../user/admin_area/license.md) on the **primary** node to unlock Geo. The license must be for [GitLab Premium](https://about.gitlab.com/pricing/) or higher. 1. [Set up the database replication](database.md) (`primary (read-write) <-> secondary (read-only)` topology). 1. [Configure fast lookup of authorized SSH keys in the database](../../operations/fast_ssh_key_lookup.md). This step is required and needs to be done on **both** the **primary** and **secondary** nodes. 1. [Configure GitLab](configuration.md) to set the **primary** and **secondary** nodes. 1. Optional: [Configure a secondary LDAP server](../../auth/ldap.md) for the **secondary** node. See [notes on LDAP](#ldap). 1. [Follow the "Using a Geo Server" guide](using_a_geo_server.md). -### Using GitLab installed from source (Deprecated) - -NOTE: **Note:** -In GitLab 11.5, support for using Geo in GitLab source installations was deprecated and will be removed in a future release. Please consider [migrating to GitLab Omnibus install](https://docs.gitlab.com/omnibus/update/convert_to_omnibus.html). - -If you installed GitLab from source: - -1. [Install GitLab Enterprise Edition](../../../install/installation.md) on the server that will serve as the **secondary** node. Do not create an account or log in to the new **secondary** node. -1. [Upload the GitLab License](https://docs.gitlab.com/ee/user/admin_area/license.html) on the **primary** node to unlock Geo. The license must be for [GitLab Premium](https://about.gitlab.com/pricing/) or higher. -1. [Set up the database replication](database_source.md) (`primary (read-write) <-> secondary (read-only)` topology). -1. [Configure fast lookup of authorized SSH keys in the database](../../operations/fast_ssh_key_lookup.md). Do this step for **both** **primary** and **secondary** nodes. -1. [Configure GitLab](configuration_source.md) to set the **primary** and **secondary** nodes. -1. [Follow the "Using a Geo Server" guide](using_a_geo_server.md). - ## Post-installation documentation After installing GitLab on the **secondary** nodes and performing the initial configuration, see the following documentation for post-installation information. ### Configuring Geo -For information on configuring Geo, see: - -- [Geo configuration (GitLab Omnibus)](configuration.md). -- [Geo configuration (source)](configuration_source.md). Configuring Geo in GitLab source installations was **deprecated** in GitLab 11.5. +For information on configuring Geo, see [Geo configuration](configuration.md). ### Updating Geo @@ -278,7 +261,7 @@ Take special note that these examples of GitLab features are both: Examples include: -- [Elasticsearch integration](https://docs.gitlab.com/ee/integration/elasticsearch.html). +- [Elasticsearch integration](../../../integration/elasticsearch.md). - [Container Registry](../../container_registry.md). [Object Storage](object_storage.md) can mitigate this. - [GitLab Pages](../../pages/index.md). - [Mattermost integration](https://docs.gitlab.com/omnibus/gitlab-mattermost/). diff --git a/doc/administration/geo/replication/security_review.md b/doc/administration/geo/replication/security_review.md index 46d3e68ab63..cd54e2dc8c4 100644 --- a/doc/administration/geo/replication/security_review.md +++ b/doc/administration/geo/replication/security_review.md @@ -120,9 +120,7 @@ questions from [owasp.org](https://www.owasp.org). ### What details regarding required OS components and lock‐down needs have been defined? -- The recommended installation method (Omnibus) packages most components itself. - A from-source installation method exists. Both are documented at - <https://docs.gitlab.com/ee/administration/geo/replication/index.html> +- The supported installation method (Omnibus) packages most components itself. - There are significant dependencies on the system-installed OpenSSH daemon (Geo requires users to set up custom authentication methods) and the omnibus or system-provided PostgreSQL daemon (it must be configured to listen on TCP, diff --git a/doc/administration/geo/replication/troubleshooting.md b/doc/administration/geo/replication/troubleshooting.md index 9c95720487d..c5bdd36ba70 100644 --- a/doc/administration/geo/replication/troubleshooting.md +++ b/doc/administration/geo/replication/troubleshooting.md @@ -23,6 +23,8 @@ to help identify if something is wrong:  +For information on how to resolve common errors reported from the UI, see [common errors](#common-errors). + If the UI is not working, or you are unable to log in, you can run the Geo health check manually to get this information as well as a few more details. This rake task can be run on an app node in the **primary** or **secondary** @@ -40,7 +42,8 @@ Checking Geo ... GitLab Geo is available ... yes GitLab Geo is enabled ... yes GitLab Geo secondary database is correctly configured ... yes -Using database streaming replication? ... yes +Database replication enabled? ... yes +Database replication working? ... yes GitLab Geo tracking database is configured to use Foreign Data Wrapper? ... yes GitLab Geo tracking database Foreign Data Wrapper schema is up-to-date? ... yes GitLab Geo HTTP(S) connectivity ... @@ -68,22 +71,22 @@ Example output: ``` http://secondary.example.com/ ----------------------------------------------------- - GitLab Version: 11.8.1-ee + GitLab Version: 11.10.4-ee Geo Role: Secondary Health Status: Healthy - Repositories: 190/190 (100%) - Verified Repositories: 190/190 (100%) - Wikis: 190/190 (100%) - Verified Wikis: 190/190 (100%) - LFS Objects: 35/35 (100%) - Attachments: 528/528 (100%) - CI job artifacts: 477/477 (100%) - Repositories Checked: 0/190 (0%) + Repositories: 289/289 (100%) + Verified Repositories: 289/289 (100%) + Wikis: 289/289 (100%) + Verified Wikis: 289/289 (100%) + LFS Objects: 8/8 (100%) + Attachments: 5/5 (100%) + CI job artifacts: 0/0 (0%) + Repositories Checked: 0/289 (0%) Sync Settings: Full Database replication lag: 0 seconds - Last event ID seen from primary: 2158 (about 2 minute ago) - Last event ID processed by cursor: 2158 (about 2 minute ago) - Last status report was: 4 minutes ago + Last event ID seen from primary: 10215 (about 2 minutes ago) + Last event ID processed by cursor: 10215 (about 2 minutes ago) + Last status report was: 2 minutes ago ``` ## Is Postgres replication working? @@ -307,7 +310,7 @@ same host on different ports. That is, 5432 and 5431 respectively. ### Checking configuration NOTE: **Note:** -The following steps are for Omnibus installs only. Using Geo with source-based installs [is deprecated](index.md#using-gitlab-installed-from-source-deprecated). +The following steps are for Omnibus installs only. Using Geo with source-based installs was **deprecated** in GitLab 11.5. To check the configuration: @@ -455,3 +458,57 @@ reload of the FDW schema. To manually reload the FDW schema: [database-start-replication]: database.md#step-3-initiate-the-replication-process [database-pg-replication]: database.md#postgresql-replication + +## Common errors + +This section documents common errors reported in the admin UI and how to fix them. + +### Geo database configuration file is missing + +GitLab cannot find or doesn't have permission to access the `database_geo.yml` configuration file. + +In an Omnibus GitLab installation, the file should be in `/var/opt/gitlab/gitlab-rails/etc`. +If it doesn't exist or inadvertent changes have been made to it, run `sudo gitlab-ctl reconfigure` to restore it to its correct state. + + +If this path is mounted on a remote volume, please check your volume configuration and that it has correct permissions. + +### Geo node has a database that is writable which is an indication it is not configured for replication with the primary node. + +This error refers to a problem with the database replica on a **secondary** node, +which Geo expects to have access to. It usually means, either: + +- An unsupported replication method was used (for example, logical replication). +- The instructions to setup a [Geo database replication](database.md) were not followed correctly. + +A common source of confusion with **secondary** nodes is that it requires two separate +PostgreSQL instances: + +- A read-only replica of the **primary** node. +- A regular, writable instance that holds replication metadata. That is, the Geo tracking database. + +### Geo node does not appear to be replicating the database from the primary node. + +The most common problems that prevent the database from replicating correctly are: + +- **Secondary** nodes cannot reach the **primary** node. Check credentials, firewall rules, etc. +- SSL certificate problems. Make sure you copied `/etc/gitlab/gitlab-secrets.json` from the **primary** node. +- Database storage disk is full. +- Database replication slot is misconfigured. +- Database is not using a replication slot or another alternative and cannot catch-up because WAL files were purged. + +Make sure you follow the [Geo database replication](database.md) instructions for supported configuration. + +### Geo database version (...) does not match latest migration (...) + +If you are using GitLab Omnibus installation, something might have failed during upgrade. You can: + +- Run `sudo gitlab-ctl reconfigure`. +- Manually trigger the database migration by running: `sudo gitlab-rake geo:db:migrate` as root on the **secondary** node. + +### Geo database is not configured to use Foreign Data Wrapper + +This error means the Geo Tracking Database doesn't have the FDW server and credentials +configured. + +See [How do I fix a "Foreign Data Wrapper (FDW) is not configured" error?](#how-do-i-fix-a-foreign-data-wrapper-fdw-is-not-configured-error). diff --git a/doc/administration/geo/replication/tuning.md b/doc/administration/geo/replication/tuning.md index b9921b2e69f..1943f2230df 100644 --- a/doc/administration/geo/replication/tuning.md +++ b/doc/administration/geo/replication/tuning.md @@ -13,5 +13,5 @@ However, this may not lead to more downloads in parallel unless the number of available Sidekiq threads is also increased. For example, if repository sync capacity is increased from 25 to 50, you may also want to increase the number of Sidekiq threads from 25 to 50. See the -[Sidekiq concurrency documentation](https://docs.gitlab.com/ee/administration/operations/extra_sidekiq_processes.html#number-of-threads) +[Sidekiq concurrency documentation](../../operations/extra_sidekiq_processes.md#number-of-threads) for more details. diff --git a/doc/administration/geo/replication/updating_the_geo_nodes.md b/doc/administration/geo/replication/updating_the_geo_nodes.md index 66728366e24..933a75c47d8 100644 --- a/doc/administration/geo/replication/updating_the_geo_nodes.md +++ b/doc/administration/geo/replication/updating_the_geo_nodes.md @@ -337,6 +337,53 @@ is prepended with the relevant node for better clarity: 1. **[secondary]** Create the `replica.sh` script as described in the [database configuration document][database-source-replication]. + 1. 1. **[secondary]** Save the snippet below in a file, let's say `/tmp/replica.sh`. Modify the + embedded paths if necessary: + + ``` + #!/bin/bash + + PORT="5432" + USER="gitlab_replicator" + echo --------------------------------------------------------------- + echo WARNING: Make sure this script is run from the secondary server + echo --------------------------------------------------------------- + echo + echo Enter the IP or FQDN of the primary PostgreSQL server + read HOST + echo Enter the password for $USER@$HOST + read -s PASSWORD + echo Enter the required sslmode + read SSLMODE + + echo Stopping PostgreSQL and all GitLab services + sudo service gitlab stop + sudo service postgresql stop + + echo Backing up postgresql.conf + sudo -u postgres mv /var/opt/gitlab/postgresql/data/postgresql.conf /var/opt/gitlab/postgresql/ + + echo Cleaning up old cluster directory + sudo -u postgres rm -rf /var/opt/gitlab/postgresql/data + + echo Starting base backup as the replicator user + echo Enter the password for $USER@$HOST + sudo -u postgres /opt/gitlab/embedded/bin/pg_basebackup -h $HOST -D /var/opt/gitlab/postgresql/data -U gitlab_replicator -v -x -P + + echo Writing recovery.conf file + sudo -u postgres bash -c "cat > /var/opt/gitlab/postgresql/data/recovery.conf <<- _EOF1_ + standby_mode = 'on' + primary_conninfo = 'host=$HOST port=$PORT user=$USER password=$PASSWORD sslmode=$SSLMODE' + _EOF1_ + " + + echo Restoring postgresql.conf + sudo -u postgres mv /var/opt/gitlab/postgresql/postgresql.conf /var/opt/gitlab/postgresql/data/ + + echo Starting PostgreSQL + sudo service postgresql start + ``` + 1. **[secondary]** Run the recovery script using the credentials from the previous step: @@ -396,8 +443,6 @@ and it is required since 10.0. [update]: ../../../update/README.md [database]: database.md -[database-replication]: database.md#step-3-initiate-the-replication-process -[database-source-replication]: database_source.md#step-3-initiate-the-replication-process [Hashed Storage]: ../../repository_storage_types.md [hashed-migration]: ../../raketasks/storage.md [ssh-fast-lookup]: ../../operations/fast_ssh_key_lookup.md diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md index f1cedb85455..dcf8d8715ca 100644 --- a/doc/administration/gitaly/index.md +++ b/doc/administration/gitaly/index.md @@ -53,6 +53,10 @@ But since 11.8 the indexer uses Gitaly for data access as well. NFS can still be leveraged for redudancy on block level of the Git data. But only has to be mounted on the Gitaly server. +NOTE: **Note:** While Gitaly can be used as a replacement for NFS, we do not recommend +using EFS as it may impact GitLab's performance. Please review the [relevant documentation](../high_availability/nfs.md#avoid-using-awss-elastic-file-system-efs) +for more details. + ### Network architecture - gitlab-rails shards repositories into "repository storages" @@ -73,18 +77,29 @@ be mounted on the Gitaly server. - Gitaly servers must not be exposed to the public internet Gitaly network traffic is unencrypted by default, but supports -[TLS](#tls-support). Authentication is done through a static token. For -security in depth, its recommended to use a firewall to restrict access -to your Gitaly server. +[TLS](#tls-support). Authentication is done through a static token. + +NOTE: **Note:** Gitaly network traffic is unencrypted so we recommend a firewall to +restrict access to your Gitaly server. Below we describe how to configure a Gitaly server at address `gitaly.internal:8075` with secret token `abc123secret`. We assume your GitLab installation has two repository storages, `default` and `storage1`. +### Installation + +First install Gitaly using either Omnibus or from source. + +Omnibus: [Download/install](https://about.gitlab.com/installation) the Omnibus GitLab +package you want using **steps 1 and 2** from the GitLab downloads page but +**_do not_** provide the `EXTERNAL_URL=` value. + +Source: [Install Gitaly](../../install/installation.md#install-gitaly) + ### Client side token configuration -Start by configuring a token on the client side. +Configure a token on the client side. Omnibus installations: @@ -110,7 +125,7 @@ changes to be picked up. Next, on the Gitaly server, we need to configure storage paths, enable the network listener and configure the token. -Note: if you want to reduce the risk of downtime when you enable +NOTE: **Note:** if you want to reduce the risk of downtime when you enable authentication you can temporarily disable enforcement, see [the documentation on configuring Gitaly authentication](https://gitlab.com/gitlab-org/gitaly/blob/master/doc/configuration/README.md#authentication) @@ -122,12 +137,17 @@ the Gitaly server. The easiest way to accomplish this is to copy `/etc/gitlab/gi from an existing GitLab server to the Gitaly server. Without this shared secret, Git operations in GitLab will result in an API error. -> **NOTE:** In most or all cases the storage paths below end in `/repositories` which is +NOTE: **Note:** In most or all cases the storage paths below end in `/repositories` which is different than `path` in `git_data_dirs` of Omnibus installations. Check the directory layout on your Gitaly server to be sure. Omnibus installations: +<!-- +updates to following example must also be made at +https://gitlab.com/charts/gitlab/blob/master/doc/advanced/external-gitaly/external-omnibus-gitaly.md#configure-omnibus-gitlab +--> + ```ruby # /etc/gitlab/gitlab.rb @@ -147,6 +167,7 @@ gitlab_rails['auto_migrate'] = false # Configure the gitlab-shell API callback URL. Without this, `git push` will # fail. This can be your 'front door' GitLab URL or an internal load # balancer. +# Don't forget to copy `/etc/gitlab/gitlab-secrets.json` from web server to Gitaly server. gitlab_rails['internal_api_url'] = 'https://gitlab.example.com' # Make Gitaly accept connections on all network interfaces. You must use diff --git a/doc/administration/high_availability/README.md b/doc/administration/high_availability/README.md index 24db1c28778..e5701525077 100644 --- a/doc/administration/high_availability/README.md +++ b/doc/administration/high_availability/README.md @@ -64,8 +64,8 @@ larger one. - 1 PostgreSQL node - 1 Redis node -- 2 or more GitLab application nodes (Unicorn, Workhorse, Sidekiq) - 1 NFS/Gitaly storage server +- 2 or more GitLab application nodes (Unicorn, Workhorse, Sidekiq) #### Installation Instructions @@ -88,9 +88,9 @@ in size, indicating that there is contention or not enough resources. - 1 PostgreSQL node - 1 Redis node -- 2 or more GitLab application nodes (Unicorn, Workhorse) -- 2 or more Sidekiq nodes - 2 or more NFS/Gitaly storage servers +- 2 or more Sidekiq nodes +- 2 or more GitLab application nodes (Unicorn, Workhorse) ## High Availability Architecture Examples @@ -135,7 +135,7 @@ the contention. - 2 or more GitLab application nodes (Unicorn, Workhorse, Sidekiq, PGBouncer) - 1 NFS/Gitaly server - + ### Hybrid @@ -153,7 +153,7 @@ contention due to certain workloads. - 1 or more NFS/Gitaly servers - 1 Monitoring node (Prometheus, Grafana) - + #### Reference Architecture @@ -194,7 +194,7 @@ with the added complexity of many more nodes to configure, manage and monitor. - 2 or more Web nodes (All other web requests) - 2 or more NFS/Gitaly servers - + The following pages outline the steps necessary to configure each component separately: diff --git a/doc/administration/high_availability/database.md b/doc/administration/high_availability/database.md index 1648b6b848a..a446ed9637c 100644 --- a/doc/administration/high_availability/database.md +++ b/doc/administration/high_availability/database.md @@ -134,7 +134,7 @@ otherwise the networks will become a single point of failure. #### Architecture - + Database nodes run two services with PostgreSQL: diff --git a/doc/administration/high_availability/gitaly.md b/doc/administration/high_availability/gitaly.md index d44744f2af8..40f85f28cb8 100644 --- a/doc/administration/high_availability/gitaly.md +++ b/doc/administration/high_availability/gitaly.md @@ -12,77 +12,8 @@ environments and [High Availability Architecture](./README.md#high-availability- ## Running Gitaly on its own server -Starting with GitLab 11.4, Gitaly is a replacement for NFS except -when the [Elastic Search indexer](https://gitlab.com/gitlab-org/gitlab-elasticsearch-indexer) -is used. - -NOTE: **Note:** While Gitaly can be used as a replacement for NFS, we do not recommend using EFS as it may impact GitLab's performance. Please review the [relevant documentation](nfs.md#avoid-using-awss-elastic-file-system-efs) for more details. - -NOTE: **Note:** Gitaly network traffic is unencrypted so we recommend a firewall to -restrict access to your Gitaly server. - -The steps below are the minimum necessary to configure a Gitaly server with -Omnibus: - -1. SSH into the Gitaly server. -1. [Download/install](https://about.gitlab.com/installation) the Omnibus GitLab - package you want using **steps 1 and 2** from the GitLab downloads page. - - Do not complete any other steps on the download page. - -1. Edit `/etc/gitlab/gitlab.rb` and add the contents: - - Gitaly must trigger some callbacks to GitLab via GitLab Shell. As a result, - the GitLab Shell secret must be the same between the other GitLab servers and - the Gitaly server. The easiest way to accomplish this is to copy `/etc/gitlab/gitlab-secrets.json` - from an existing GitLab server to the Gitaly server. Without this shared secret, - Git operations in GitLab will result in an API error. - - > **NOTE:** In most or all cases the storage paths below end in `repositories` which is - different than `path` in `git_data_dirs` of Omnibus installations. Check the - directory layout on your Gitaly server to be sure. - - ```ruby - # Enable Gitaly - gitaly['enable'] = true - - ## Disable all other services - sidekiq['enable'] = false - gitlab_workhorse['enable'] = false - unicorn['enable'] = false - postgresql['enable'] = false - nginx['enable'] = false - prometheus['enable'] = false - alertmanager['enable'] = false - pgbouncer_exporter['enable'] = false - redis_exporter['enable'] = false - gitlab_monitor['enable'] = false - - # Prevent database connections during 'gitlab-ctl reconfigure' - gitlab_rails['rake_cache_clear'] = false - gitlab_rails['auto_migrate'] = false - - # Configure the gitlab-shell API callback URL. Without this, `git push` will - # fail. This can be your 'front door' GitLab URL or an internal load - # balancer. - gitlab_rails['internal_api_url'] = 'https://gitlab.example.com' - - # Make Gitaly accept connections on all network interfaces. You must use - # firewalls to restrict access to this address/port. - gitaly['listen_addr'] = "0.0.0.0:8075" - gitaly['auth_token'] = 'abc123secret' - - gitaly['storage'] = [ - { 'name' => 'default', 'path' => '/mnt/gitlab/default/repositories' }, - { 'name' => 'storage1', 'path' => '/mnt/gitlab/storage1/repositories' }, - ] - - # To use tls for gitaly you need to add - gitaly['tls_listen_addr'] = "0.0.0.0:9999" - gitaly['certificate_path'] = "path/to/cert.pem" - gitaly['key_path'] = "path/to/key.pem" - ``` - -Again, reconfigure (Omnibus) or restart (source). +See [Running Gitaly on its own server](../gitaly/index.md#running-gitaly-on-its-own-server) +in Gitaly documentation. Continue configuration of other components by going back to: diff --git a/doc/administration/high_availability/img/fully-distributed.png b/doc/administration/high_availability/img/fully-distributed.png Binary files differnew file mode 100644 index 00000000000..ad23207134e --- /dev/null +++ b/doc/administration/high_availability/img/fully-distributed.png diff --git a/doc/administration/high_availability/img/geo-ha-diagram.png b/doc/administration/high_availability/img/geo-ha-diagram.png Binary files differnew file mode 100644 index 00000000000..da5d612827c --- /dev/null +++ b/doc/administration/high_availability/img/geo-ha-diagram.png diff --git a/doc/administration/high_availability/img/horizontal.png b/doc/administration/high_availability/img/horizontal.png Binary files differnew file mode 100644 index 00000000000..c3bd489d96f --- /dev/null +++ b/doc/administration/high_availability/img/horizontal.png diff --git a/doc/administration/high_availability/img/hybrid.png b/doc/administration/high_availability/img/hybrid.png Binary files differnew file mode 100644 index 00000000000..7d4a56bf0ea --- /dev/null +++ b/doc/administration/high_availability/img/hybrid.png diff --git a/doc/administration/high_availability/pg_ha_architecture.png b/doc/administration/high_availability/img/pg_ha_architecture.png Binary files differindex ef870f652ae..ef870f652ae 100644 --- a/doc/administration/high_availability/pg_ha_architecture.png +++ b/doc/administration/high_availability/img/pg_ha_architecture.png diff --git a/doc/administration/high_availability/nfs.md b/doc/administration/high_availability/nfs.md index caadec3ac4e..f6bbc8935b4 100644 --- a/doc/administration/high_availability/nfs.md +++ b/doc/administration/high_availability/nfs.md @@ -39,13 +39,8 @@ options: ### Improving NFS performance with GitLab -NOTE: **Note:** -This is only available starting in certain versions of GitLab: - * 11.5.11 - * 11.6.11 - * 11.7.12 - * 11.8.8 - * 11.9.0 and up (e.g. 11.10, 11.11, etc.) +NOTE: **Note:** This is only available starting in certain versions of GitLab: 11.5.11, +11.6.11, 11.7.12, 11.8.8, 11.9.0 and up (e.g. 11.10, 11.11, etc.) If you are using NFS to share Git data, we recommend that you enable a number of feature flags that will allow GitLab application processes to @@ -107,6 +102,11 @@ stored on a local volume. For more details on another person's experience with EFS, see [Amazon's Elastic File System: Burst Credits](https://rawkode.com/2017/04/16/amazons-elastic-file-system-burst-credits/) +## Avoid using CephFS and GlusterFS + +GitLab strongly recommends against using CephFS and GlusterFS. +These distributed file systems are not well-suited for GitLab's input/output access patterns because git uses many small files and access times and file locking times to propagate will make git activity very slow. + ## Avoid using PostgreSQL with NFS GitLab strongly recommends against running your PostgreSQL database @@ -122,7 +122,7 @@ Additionally, this configuration is specifically warned against in the >to the NFS server can cause data corruption problems. For supported database architecture, please see our documentation on -[Configuring a Database for GitLab HA](https://docs.gitlab.com/ee/administration/high_availability/database.html). +[Configuring a Database for GitLab HA](database.md). ## NFS Client mount options diff --git a/doc/administration/high_availability/redis.md b/doc/administration/high_availability/redis.md index 46ad3ecd9bb..1f37224a184 100644 --- a/doc/administration/high_availability/redis.md +++ b/doc/administration/high_availability/redis.md @@ -863,7 +863,7 @@ You can check if everything is correct by connecting to each server using `redis-cli` application, and sending the `info replication` command as below. ``` -/opt/gitlab/embedded/bin/redis-cli -a <redis-password> info replication +/opt/gitlab/embedded/bin/redis-cli -h <redis-host-or-ip> -a '<redis-password>' info replication ``` When connected to a `master` redis, you will see the number of connected diff --git a/doc/administration/incoming_email.md b/doc/administration/incoming_email.md index 9ac310af248..8271c579f5b 100644 --- a/doc/administration/incoming_email.md +++ b/doc/administration/incoming_email.md @@ -10,7 +10,7 @@ GitLab has several features based on receiving incoming emails: - [New merge request by email](../user/project/merge_requests/index.md#create-new-merge-requests-by-email): allow GitLab users to create a new merge request by sending an email to a user-specific email address. -- [Service Desk](https://docs.gitlab.com/ee/user/project/service_desk.html): provide e-mail support to +- [Service Desk](../user/project/service_desk.md): provide e-mail support to your customers through GitLab. **[PREMIUM]** ## Requirements diff --git a/doc/administration/index.md b/doc/administration/index.md index 797a7242bd0..95a0e84deb6 100644 --- a/doc/administration/index.md +++ b/doc/administration/index.md @@ -20,7 +20,7 @@ GitLab Community Edition installations only have access to Core features. GitLab.com is administered by GitLab, Inc., therefore, only GitLab team members have access to its admin configurations. If you're a GitLab.com user, please check the -[user documentation](../user/index.html). +[user documentation](../user/index.md). NOTE: **Note:** Non-administrator users don’t have access to GitLab administration tools and settings. @@ -37,9 +37,9 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Omnibus support for log forwarding](https://docs.gitlab.com/omnibus/settings/logs.html#udp-log-shipping-gitlab-enterprise-edition-only) **[STARTER ONLY]** - [High Availability](high_availability/README.md): Configure multiple servers for scaling or high availability. - [High Availability on AWS](../university/high-availability/aws/README.md): Set up GitLab HA on Amazon AWS. -- [Geo](https://docs.gitlab.com/ee/administration/geo/replication/index.html): Replicate your GitLab instance to other geographic locations as a read-only fully operational version. **[PREMIUM ONLY]** -- [Disaster Recovery](https://docs.gitlab.com/ee/administration/geo/disaster_recovery/index.html): Quickly fail-over to a different site with minimal effort in a disaster situation. **[PREMIUM ONLY]** -- [Pivotal Tile](https://docs.gitlab.com/ee/install/pivotal/index.html): Deploy GitLab as a pre-configured appliance using Ops Manager (BOSH) for Pivotal Cloud Foundry. **[PREMIUM ONLY]** +- [Geo](geo/replication/index.md): Replicate your GitLab instance to other geographic locations as a read-only fully operational version. **[PREMIUM ONLY]** +- [Disaster Recovery](geo/disaster_recovery/index.md): Quickly fail-over to a different site with minimal effort in a disaster situation. **[PREMIUM ONLY]** +- [Pivotal Tile](../install/pivotal/index.md): Deploy GitLab as a pre-configured appliance using Ops Manager (BOSH) for Pivotal Cloud Foundry. **[PREMIUM ONLY]** - [Add License](../user/admin_area/license.md): Upload a license at install time to unlock features that are in paid tiers of GitLab. **[STARTER ONLY]** ### Configuring GitLab @@ -61,9 +61,9 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Diff limits](../user/admin_area/diff_limits.md): Configure the diff rendering size limits of branch comparison pages. - [Merge request diffs storage](merge_request_diffs.md): Configure merge requests diffs external storage. - [Broadcast Messages](../user/admin_area/broadcast_messages.md): Send messages to GitLab users through the UI. -- [Elasticsearch](https://docs.gitlab.com/ee/integration/elasticsearch.html): Enable Elasticsearch to empower GitLab's Advanced Global Search. Useful when you deal with a huge amount of data. **[STARTER ONLY]** +- [Elasticsearch](../integration/elasticsearch.md): Enable Elasticsearch to empower GitLab's Advanced Global Search. Useful when you deal with a huge amount of data. **[STARTER ONLY]** - [External Classification Policy Authorization](../user/admin_area/settings/external_authorization.md) **[PREMIUM ONLY]** -- [Upload a license](https://docs.gitlab.com/ee/user/admin_area/license.html): Upload a license to unlock features that are in paid tiers of GitLab. **[STARTER ONLY]** +- [Upload a license](../user/admin_area/license.md): Upload a license to unlock features that are in paid tiers of GitLab. **[STARTER ONLY]** - [Admin Area](../user/admin_area/index.md): for self-managed instance-wide configuration and maintenance. #### Customizing GitLab's appearance @@ -73,7 +73,7 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Branded login page](../customization/branded_login_page.md): Customize the login page with your own logo, title, and description. - [Welcome message](../customization/welcome_message.md): Add a custom welcome message to the sign-in page. - ["New Project" page](../customization/new_project_page.md): Customize the text to be displayed on the page that opens whenever your users create a new project. -- [Additional custom email text](https://docs.gitlab.com/ee/user/admin_area/settings/email.html#custom-additional-text-premium-only): Add additional custom text to emails sent from GitLab. **[PREMIUM ONLY]** +- [Additional custom email text](../user/admin_area/settings/email.md#custom-additional-text-premium-only): Add additional custom text to emails sent from GitLab. **[PREMIUM ONLY]** ### Maintaining GitLab @@ -108,34 +108,29 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Sign-up restrictions](../user/admin_area/settings/sign_up_restrictions.md): block email addresses of specific domains, or whitelist only specific domains. - [Access restrictions](../user/admin_area/settings/visibility_and_access_controls.md#enabled-git-access-protocols): Define which Git access protocols can be used to talk to GitLab (SSH, HTTP, HTTPS). - [Authentication and Authorization](auth/README.md): Configure external authentication with LDAP, SAML, CAS and additional providers. - - [Sync LDAP](https://docs.gitlab.com/ee/administration/auth/ldap-ee.html) **[STARTER ONLY]** - - [Kerberos authentication](https://docs.gitlab.com/ee/integration/kerberos.html) **[STARTER ONLY]** + - [Sync LDAP](auth/ldap-ee.md) **[STARTER ONLY]** + - [Kerberos authentication](../integration/kerberos.md) **[STARTER ONLY]** - See also other [authentication](../topics/authentication/index.md#gitlab-administrators) topics (for example, enforcing 2FA). -- [Email users](https://docs.gitlab.com/ee/tools/email.html): Email GitLab users from within GitLab. **[STARTER ONLY]** +- [Email users](../tools/email.md): Email GitLab users from within GitLab. **[STARTER ONLY]** - [User Cohorts](../user/admin_area/user_cohorts.md): Display the monthly cohorts of new users and their activities over time. - [Audit logs and events](audit_events.md): View the changes made within the GitLab server for: - Groups and projects. **[STARTER]** - Instances. **[PREMIUM ONLY]** - [Auditor users](auditor_users.md): Users with read-only access to all projects, groups, and other resources on the GitLab instance. **[PREMIUM ONLY]** - [Incoming email](incoming_email.md): Configure incoming emails to allow - users to [reply by email], create [issues by email] and - [merge requests by email], and to enable [Service Desk]. + users to [reply by email](reply_by_email.md), create [issues by email](../user/project/issues/create_new_issue.md#new-issue-via-email) and + [merge requests by email](../user/project/merge_requests/index.md#create-new-merge-requests-by-email), and to enable [Service Desk](../user/project/service_desk.md). - [Postfix for incoming email](reply_by_email_postfix_setup.md): Set up a basic Postfix mail server with IMAP authentication on Ubuntu for incoming emails. - [Abuse reports](../user/admin_area/abuse_reports.md): View and resolve abuse reports from your users. -[reply by email]: reply_by_email.md -[issues by email]: ../user/project/issues/create_new_issue.md#new-issue-via-email -[merge requests by email]: ../user/project/merge_requests/index.md#create-new-merge-requests-by-email -[Service Desk]: https://docs.gitlab.com/ee/user/project/service_desk.html - ## Project settings - [Container Registry](container_registry.md): Configure Container Registry with GitLab. - [Issue closing pattern](issue_closing_pattern.md): Customize how to close an issue from commit messages. - [Gitaly](gitaly/index.md): Configuring Gitaly, GitLab's Git repository storage service. -- [Default labels](../user/admin_area/labels.html): Create labels that will be automatically added to every new project. +- [Default labels](../user/admin_area/labels.md): Create labels that will be automatically added to every new project. - [Restrict the use of public or internal projects](../public_access/public_access.md#restricting-the-use-of-public-or-internal-projects): Restrict the use of visibility levels for users when they create a project or a snippet. - [Custom project templates](../user/admin_area/custom_project_templates.md): Configure a set of projects to be used as custom templates when creating a new project. **[PREMIUM ONLY]** - [Packages](packages.md): Enable GitLab to act as a Maven repository or NPM registry. **[PREMIUM ONLY]** @@ -145,7 +140,7 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Repository checks](repository_checks.md): Periodic Git repository checks. - [Repository storage paths](repository_storage_paths.md): Manage the paths used to store repositories. - [Repository storage rake tasks](raketasks/storage.md): A collection of rake tasks to list and migrate existing projects and attachments associated with it from Legacy storage to Hashed storage. -- [Limit repository size](https://docs.gitlab.com/ee/user/admin_area/settings/account_and_limit_settings.html): Set a hard limit for your repositories' size. **[STARTER ONLY]** +- [Limit repository size](../user/admin_area/settings/account_and_limit_settings.md): Set a hard limit for your repositories' size. **[STARTER ONLY]** ## Continuous Integration settings @@ -154,7 +149,7 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Job artifacts](job_artifacts.md): Enable, disable, and configure job artifacts (a set of files and directories which are outputted by a job when it completes successfully). - [Job traces](job_traces.md): Information about the job traces (logs). - [Register Shared and specific Runners](../ci/runners/README.md#registering-a-shared-runner): Learn how to register and configure Shared and specific Runners to your own instance. -- [Shared Runners pipelines quota](https://docs.gitlab.com/ee/user/admin_area/settings/continuous_integration.html#shared-runners-pipeline-minutes-quota-starter-only): Limit the usage of pipeline minutes for Shared Runners. **[STARTER ONLY]** +- [Shared Runners pipelines quota](../user/admin_area/settings/continuous_integration.md#shared-runners-pipeline-minutes-quota-starter-only): Limit the usage of pipeline minutes for Shared Runners. **[STARTER ONLY]** - [Enable/disable Auto DevOps](../topics/autodevops/index.md#enablingdisabling-auto-devops): Enable or disable Auto DevOps for your instance. ## Git configuration options diff --git a/doc/administration/integration/plantuml.md b/doc/administration/integration/plantuml.md index b7b820abb40..82e0c14ffc2 100644 --- a/doc/administration/integration/plantuml.md +++ b/doc/administration/integration/plantuml.md @@ -27,7 +27,7 @@ own PlantUML server is easy in Debian/Ubuntu distributions using Tomcat. First you need to create a `plantuml.war` file from the source code: ``` -sudo apt-get install graphviz openjdk-7-jdk git-core maven +sudo apt-get install graphviz openjdk-8-jdk git-core maven git clone https://github.com/plantuml/plantuml-server.git cd plantuml-server mvn package diff --git a/doc/administration/integration/terminal.md b/doc/administration/integration/terminal.md index 2596e3fe68b..c34858cd0db 100644 --- a/doc/administration/integration/terminal.md +++ b/doc/administration/integration/terminal.md @@ -43,6 +43,11 @@ detail below. ## Enabling and disabling terminal support +NOTE: **Note:** AWS Elastic Load Balancers (ELBs) do not support web sockets. +AWS Application Load Balancers (ALBs) must be used if you want web terminals +to work. See [AWS Elastic Load Balancing Product Comparison](https://aws.amazon.com/elasticloadbalancing/features/#compare) +for more information. + As web terminals use WebSockets, every HTTP/HTTPS reverse proxy in front of Workhorse needs to be configured to pass the `Connection` and `Upgrade` headers through to the next one in the chain. If you installed GitLab using Omnibus, or diff --git a/doc/administration/job_artifacts.md b/doc/administration/job_artifacts.md index e7792106f81..ef370573a98 100644 --- a/doc/administration/job_artifacts.md +++ b/doc/administration/job_artifacts.md @@ -100,6 +100,9 @@ artifacts, you can use an object storage like AWS S3 instead. This configuration relies on valid AWS credentials to be configured already. Use an object storage option like AWS S3 to store job artifacts. +DANGER: **Danger:** +If you're enabling S3 in [GitLab HA](high_availability/README.md), you will need to have an [NFS mount set up for CI traces and artifacts](high_availability/nfs.md#a-single-nfs-mount) or enable [live tracing](job_traces.md#new-live-trace-architecture). If these settings are not set, you will risk job traces disappearing or not being saved. + ### Object Storage Settings For source installations the following settings are nested under `artifacts:` and then `object_store:`. On omnibus installs they are prefixed by `artifacts_object_store_`. diff --git a/doc/administration/logs.md b/doc/administration/logs.md index 3d40cda491a..ac41f9177dd 100644 --- a/doc/administration/logs.md +++ b/doc/administration/logs.md @@ -280,6 +280,28 @@ installations from source. Currently it logs the progress of project imports from the Bitbucket Server importer. Future importers may use this file. +## `auth.log` + +Introduced in GitLab 12.0. This file lives in `/var/log/gitlab/gitlab-rails/auth.log` for +Omnibus GitLab packages or in `/home/git/gitlab/log/auth.log` for +installations from source. + +It logs information whenever [Rack Attack] registers an abusive request. + +## `graphql_json.log` + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/59587) in GitLab 12.0. + +This file lives in `/var/log/gitlab/gitlab-rails/graphql_json.log` for +Omnibus GitLab packages or in `/home/git/gitlab/log/graphql_json.log` for +installations from source. + +GraphQL queries are recorded in that file. For example: + +```json +{"query_string":"query IntrospectionQuery{__schema {queryType { name },mutationType { name }}}...(etc)","variables":{"a":1,"b":2},"complexity":181,"depth":1,"duration":7} +``` + ## Reconfigure Logs Reconfigure log files live in `/var/log/gitlab/reconfigure` for Omnibus GitLab @@ -298,3 +320,4 @@ Omnibus GitLab packages or in `/home/git/gitlab/log/sidekiq_exporter.log` for installations from source. [repocheck]: repository_checks.md +[Rack Attack]: ../security/rack_attack.md diff --git a/doc/administration/monitoring/performance/grafana_configuration.md b/doc/administration/monitoring/performance/grafana_configuration.md index ab43ec2cc4f..187fb2f73a1 100644 --- a/doc/administration/monitoring/performance/grafana_configuration.md +++ b/doc/administration/monitoring/performance/grafana_configuration.md @@ -3,7 +3,7 @@ [Grafana](http://grafana.org/) is a tool that allows you to visualize time series metrics through graphs and dashboards. It supports several backend data stores, including InfluxDB. GitLab writes performance data to InfluxDB -and Grafana will allow you to query InfluxDB to display useful graphs. +and Grafana will allow you to query to display useful graphs. For the easiest installation and configuration, install Grafana on the same server as InfluxDB. For larger installations, you may want to split out these @@ -11,11 +11,13 @@ services. ## Installation -Grafana supplies package repositories (Yum/Apt) for easy installation. +[GitLab Omnibus can help you install Grafana (recommended)](https://docs.gitlab.com/omnibus/settings/grafana.html) +or Grafana supplies package repositories (Yum/Apt) for easy installation. See [Grafana installation documentation](http://docs.grafana.org/installation/) for detailed steps. -> **Note**: Before starting Grafana for the first time, set the admin user +NOTE: **Note:** +Before starting Grafana for the first time, set the admin user and password in `/etc/grafana/grafana.ini`. Otherwise, the default password will be `admin`. diff --git a/doc/administration/monitoring/prometheus/gitlab_metrics.md b/doc/administration/monitoring/prometheus/gitlab_metrics.md index c243dd9edbb..3dcd1593099 100644 --- a/doc/administration/monitoring/prometheus/gitlab_metrics.md +++ b/doc/administration/monitoring/prometheus/gitlab_metrics.md @@ -43,10 +43,11 @@ The following metrics are available: | redis_ping_latency_seconds | Gauge | 9.4 | Round trip time of the redis ping | | user_session_logins_total | Counter | 9.4 | Counter of how many users have logged in | | upload_file_does_not_exist | Counter | 10.7 in EE, 11.5 in CE | Number of times an upload record could not find its file | -| failed_login_captcha_total | Gauge | 11.0 | Counter of failed CAPTCHA attempts during login | -| successful_login_captcha_total | Gauge | 11.0 | Counter of successful CAPTCHA attempts during login | -| unicorn_active_connections | Gauge | 11.0 | The number of active Unicorn connections (workers) | -| unicorn_queued_connections | Gauge | 11.0 | The number of queued Unicorn connections | +| failed_login_captcha_total | Gauge | 11.0 | Counter of failed CAPTCHA attempts during login | +| successful_login_captcha_total | Gauge | 11.0 | Counter of successful CAPTCHA attempts during login | +| unicorn_active_connections | Gauge | 11.0 | The number of active Unicorn connections (workers) | +| unicorn_queued_connections | Gauge | 11.0 | The number of queued Unicorn connections | +| unicorn_workers | Gauge | 12.0 | The number of Unicorn workers | ## Sidekiq Metrics available for Geo **[PREMIUM]** @@ -86,8 +87,8 @@ the `monitoring.sidekiq_exporter` configuration option in `gitlab.yml`. | geo_wikis_checksum_mismatch_count | Gauge | 10.7 | Number of wikis that checksum mismatch on secondary | url | geo_repositories_checked_count | Gauge | 11.1 | Number of repositories that have been checked via `git fsck` | url | geo_repositories_checked_failed_count | Gauge | 11.1 | Number of repositories that have a failure from `git fsck` | url -| geo_repositories_retrying_verification_count | Gauge | 11.2 | Number of repositories verification failures that Geo is actively trying to correct on secondary | url -| geo_wikis_retrying_verification_count | Gauge | 11.2 | Number of wikis verification failures that Geo is actively trying to correct on secondary | url +| geo_repositories_retrying_verification_count | Gauge | 11.2 | Number of repositories verification failures that Geo is actively trying to correct on secondary | url +| geo_wikis_retrying_verification_count | Gauge | 11.2 | Number of wikis verification failures that Geo is actively trying to correct on secondary | url ### Ruby metrics @@ -100,9 +101,31 @@ Some basic Ruby runtime metrics are available: | ruby_file_descriptors | Gauge | 11.1 | File descriptors per process | | ruby_memory_bytes | Gauge | 11.1 | Memory usage by process | | ruby_sampler_duration_seconds_total | Counter | 11.1 | Time spent collecting stats | +| ruby_process_cpu_seconds_total | Gauge | 12.0 | Total amount of CPU time per process | +| ruby_process_max_fds | Gauge | 12.0 | Maximum number of open file descriptors per process | +| ruby_process_resident_memory_bytes | Gauge | 12.0 | Memory usage by process, measured in bytes | +| ruby_process_start_time_seconds | Gauge | 12.0 | The elapsed time between system boot and the process started, measured in seconds | [GC.stat]: https://ruby-doc.org/core-2.3.0/GC.html#method-c-stat +## Puma Metrics **[EXPERIMENTAL]** + +When Puma is used instead of Unicorn, following metrics are available: + +| Metric | Type | Since | Description | +|:-------------------------------------------- |:------- |:----- |:----------- | +| puma_workers | Gauge | 12.0 | Total number of workers | +| puma_running_workers | Gauge | 12.0 | Number of booted workers | +| puma_stale_workers | Gauge | 12.0 | Number of old workers | +| puma_phase | Gauge | 12.0 | Phase number (increased during phased restarts) | +| puma_running | Gauge | 12.0 | Number of running threads | +| puma_queued_connections | Gauge | 12.0 | Number of connections in that worker's "todo" set waiting for a worker thread | +| puma_active_connections | Gauge | 12.0 | Number of threads processing a request | +| puma_pool_capacity | Gauge | 12.0 | Number of requests the worker is capable of taking right now | +| puma_max_threads | Gauge | 12.0 | Maximum number of worker threads | +| puma_idle_threads | Gauge | 12.0 | Number of spawned threads which are not processing a request | +| rack_state_total | Gauge | 12.0 | Number of requests in a given rack state | + ## Metrics shared directory GitLab's Prometheus client requires a directory to store metrics data shared between multi-process services. diff --git a/doc/administration/operations/fast_ssh_key_lookup.md b/doc/administration/operations/fast_ssh_key_lookup.md index 69f110805b7..3631ea0822f 100644 --- a/doc/administration/operations/fast_ssh_key_lookup.md +++ b/doc/administration/operations/fast_ssh_key_lookup.md @@ -34,10 +34,10 @@ feature for CentOS 6, follow [the instructions on how to build and install a cus By default, GitLab manages an `authorized_keys` file, which contains all the public SSH keys for users allowed to access GitLab. However, to maintain a -single source of truth, [Geo](https://docs.gitlab.com/ee/administration/geo/replication/index.html) needs to be configured to perform SSH fingerprint +single source of truth, [Geo](../geo/replication/index.md) needs to be configured to perform SSH fingerprint lookups via database lookup. -As part of [setting up Geo](https://docs.gitlab.com/ee/administration/geo/replication/index.html#setup-instructions), +As part of [setting up Geo](../geo/replication/index.md#setup-instructions), you will be required to follow the steps outlined below for both the primary and secondary nodes, but note that the `Write to "authorized keys" file` checkbox only needs to be unchecked on the primary node since it will be reflected diff --git a/doc/administration/packages.md b/doc/administration/packages.md index 5b9a13e3859..0d5f784b71e 100644 --- a/doc/administration/packages.md +++ b/doc/administration/packages.md @@ -8,12 +8,12 @@ The Packages feature allows GitLab to act as a repository for the following: | Software repository | Description | Available in GitLab version | | ------------------- | ----------- | --------------------------- | -| [Maven Repository](https://docs.gitlab.com/ee/user/project/packages/maven_repository.html) | The GitLab Maven Repository enables every project in GitLab to have its own space to store [Maven](https://maven.apache.org/) packages. | 11.3+ | -| [NPM Registry](https://docs.gitlab.com/ee/user/project/packages/npm_registry.html) | The GitLab NPM Registry enables every project in GitLab to have its own space to store [NPM](https://www.npmjs.com/) packages. | 11.7+ | +| [Maven Repository](../user/project/packages/maven_repository.md) | The GitLab Maven Repository enables every project in GitLab to have its own space to store [Maven](https://maven.apache.org/) packages. | 11.3+ | +| [NPM Registry](../user/project/packages/npm_registry.md) | The GitLab NPM Registry enables every project in GitLab to have its own space to store [NPM](https://www.npmjs.com/) packages. | 11.7+ | Don't you see your package management system supported yet? Please consider contributing -to GitLab. This [development documentation](https://docs.gitlab.com/ee/development/packages.html) will guide you through the process. +to GitLab. This [development documentation](../development/packages.md) will guide you through the process. ## Enabling the Packages feature diff --git a/doc/administration/pages/index.md b/doc/administration/pages/index.md index 1373bd56fe3..3a7ca517d56 100644 --- a/doc/administration/pages/index.md +++ b/doc/administration/pages/index.md @@ -291,6 +291,20 @@ Pages access control is disabled by default. To enable it: 1. [Reconfigure GitLab][reconfigure]. 1. Users can now configure it in their [projects' settings](../../user/project/pages/introduction.md#gitlab-pages-access-control-core-only). +### Running behind a proxy + +Like the rest of GitLab, Pages can be used in those environments where external +internet connectivity is gated by a proxy. In order to use a proxy for GitLab +pages: + +1. Configure in `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_pages['http_proxy'] = 'http://example:8080' + ``` + +1. [Reconfigure Gitlab][reconfigure] for the changes to take effect. + ## Activate verbose logging for daemon Verbose logging was [introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/2533) in @@ -376,6 +390,7 @@ Follow the steps below to configure GitLab Pages in a separate server. gitaly['enable'] = false alertmanager['enable'] = false node_exporter['enable'] = false + gitlab_rails['auto_migrate'] = false ``` 1. Run `sudo gitlab-ctl reconfigure`. 1. On `app1` apply the following changes to `/etc/gitlab/gitlab.rb`: diff --git a/doc/administration/raketasks/maintenance.md b/doc/administration/raketasks/maintenance.md index b295b7d5dc4..0b4c1ae15d6 100644 --- a/doc/administration/raketasks/maintenance.md +++ b/doc/administration/raketasks/maintenance.md @@ -205,25 +205,6 @@ cd /home/git/gitlab sudo -u git -H bundle exec rake gitlab:track_deployment RAILS_ENV=production ``` -## Create or repair repository hooks symlink - -If the GitLab shell hooks directory location changes or another circumstance -leads to the hooks symlink becoming missing or invalid, run this Rake task -to create or repair the symlinks. - -**Omnibus Installation** - -``` -sudo gitlab-rake gitlab:shell:create_hooks -``` - -**Source Installation** - -``` -cd /home/git/gitlab -sudo -u git -H bundle exec rake gitlab:shell:create_hooks RAILS_ENV=production -``` - ## Check TCP connectivity to a remote site Sometimes you need to know if your GitLab installation can connect to a TCP diff --git a/doc/administration/raketasks/project_import_export.md b/doc/administration/raketasks/project_import_export.md index f43bba0a7a7..6ca23aabdec 100644 --- a/doc/administration/raketasks/project_import_export.md +++ b/doc/administration/raketasks/project_import_export.md @@ -32,5 +32,4 @@ bundle exec rake gitlab:import_export:data RAILS_ENV=production ``` [ce-3050]: https://gitlab.com/gitlab-org/gitlab-ce/issues/3050 -[feature-flags]: https://docs.gitlab.com/ee/api/features.html [tmp]: ../../development/shared_files.md |
