diff options
Diffstat (limited to 'doc/administration')
24 files changed, 758 insertions, 719 deletions
diff --git a/doc/administration/audit_events.md b/doc/administration/audit_events.md index ae1f6e00ea5..a80ff330e03 100644 --- a/doc/administration/audit_events.md +++ b/doc/administration/audit_events.md @@ -99,7 +99,7 @@ It is possible to filter particular actions by choosing an audit data type from the filter drop-down. You can further filter by specific group, project or user (for authentication events). - + ### Missing events diff --git a/doc/administration/auditor_users.md b/doc/administration/auditor_users.md index daba5b4ca82..65d36612d85 100644 --- a/doc/administration/auditor_users.md +++ b/doc/administration/auditor_users.md @@ -52,7 +52,7 @@ section. **Admin Area > Users**. You will find the option of the access level under the 'Access' section. -  +  1. Click **Save changes** or **Create user** for the changes to take effect. diff --git a/doc/administration/auth/smartcard.md b/doc/administration/auth/smartcard.md index 2f3289d6102..a0d4e9ef3b5 100644 --- a/doc/administration/auth/smartcard.md +++ b/doc/administration/auth/smartcard.md @@ -184,3 +184,31 @@ attribute. As a prerequisite, you must use an LDAP server that: 1. Save the file and [restart](../restart_gitlab.md#installations-from-source) GitLab for the changes to take effect. + +### Require browser session with smartcard sign-in for Git access + +**For Omnibus installations** + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['smartcard_required_for_git_access'] = true + ``` + +1. Save the file and [reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) + GitLab for the changes to take effect. + +**For installations from source** + +1. Edit `config/gitlab.yml`: + + ```yaml + ## Smartcard authentication settings + smartcard: + # snip... + # Browser session with smartcard sign-in is required for Git access + required_for_git_access: true + ``` + +1. Save the file and [restart](../restart_gitlab.md#installations-from-source) + GitLab for the changes to take effect. diff --git a/doc/administration/container_registry.md b/doc/administration/container_registry.md index 2e4b4efa0ac..04f52783d22 100644 --- a/doc/administration/container_registry.md +++ b/doc/administration/container_registry.md @@ -1,7 +1,5 @@ # GitLab Container Registry administration -> **Notes:** -> > - [Introduced][ce-4040] in GitLab 8.8. > - Container Registry manifest `v1` support was added in GitLab 8.9 to support > Docker versions earlier than 1.10. @@ -125,21 +123,21 @@ otherwise you will run into conflicts. 1. Your `/etc/gitlab/gitlab.rb` should contain the Registry URL as well as the path to the existing TLS certificate and key used by GitLab: - ```ruby - registry_external_url 'https://gitlab.example.com:4567' - ``` + ```ruby + registry_external_url 'https://gitlab.example.com:4567' + ``` - Note how the `registry_external_url` is listening on HTTPS under the - existing GitLab URL, but on a different port. + Note how the `registry_external_url` is listening on HTTPS under the + existing GitLab URL, but on a different port. - If your TLS certificate is not in `/etc/gitlab/ssl/gitlab.example.com.crt` - and key not in `/etc/gitlab/ssl/gitlab.example.com.key` uncomment the lines - below: + If your TLS certificate is not in `/etc/gitlab/ssl/gitlab.example.com.crt` + and key not in `/etc/gitlab/ssl/gitlab.example.com.key` uncomment the lines + below: - ```ruby - registry_nginx['ssl_certificate'] = "/path/to/certificate.pem" - registry_nginx['ssl_certificate_key'] = "/path/to/certificate.key" - ``` + ```ruby + registry_nginx['ssl_certificate'] = "/path/to/certificate.pem" + registry_nginx['ssl_certificate_key'] = "/path/to/certificate.key" + ``` 1. Save the file and [reconfigure GitLab][] for the changes to take effect. @@ -150,12 +148,12 @@ otherwise you will run into conflicts. 1. Open `/home/git/gitlab/config/gitlab.yml`, find the `registry` entry and configure it with the following settings: - ``` - registry: - enabled: true - host: gitlab.example.com - port: 4567 - ``` + ``` + registry: + enabled: true + host: gitlab.example.com + port: 4567 + ``` 1. Save the file and [restart GitLab][] for the changes to take effect. 1. Make the relevant changes in NGINX as well (domain, port, TLS certificates path). @@ -188,17 +186,17 @@ Let's assume that you want the container Registry to be accessible at `/etc/gitlab/ssl/registry.gitlab.example.com.key` and make sure they have correct permissions: - ```bash - chmod 600 /etc/gitlab/ssl/registry.gitlab.example.com.* - ``` + ```bash + chmod 600 /etc/gitlab/ssl/registry.gitlab.example.com.* + ``` 1. Once the TLS certificate is in place, edit `/etc/gitlab/gitlab.rb` with: - ```ruby - registry_external_url 'https://registry.gitlab.example.com' - ``` + ```ruby + registry_external_url 'https://registry.gitlab.example.com' + ``` - Note how the `registry_external_url` is listening on HTTPS. + Note how the `registry_external_url` is listening on HTTPS. 1. Save the file and [reconfigure GitLab][] for the changes to take effect. @@ -219,11 +217,11 @@ look like: 1. Open `/home/git/gitlab/config/gitlab.yml`, find the `registry` entry and configure it with the following settings: - ``` - registry: - enabled: true - host: registry.gitlab.example.com - ``` + ```yaml + registry: + enabled: true + host: registry.gitlab.example.com + ``` 1. Save the file and [restart GitLab][] for the changes to take effect. 1. Make the relevant changes in NGINX as well (domain, port, TLS certificates path). @@ -248,9 +246,9 @@ Registry application itself. 1. Open `/etc/gitlab/gitlab.rb` and set `registry['enable']` to `false`: - ```ruby - registry['enable'] = false - ``` + ```ruby + registry['enable'] = false + ``` 1. Save the file and [reconfigure GitLab][] for the changes to take effect. @@ -261,10 +259,10 @@ Registry application itself. 1. Open `/home/git/gitlab/config/gitlab.yml`, find the `registry` entry and set `enabled` to `false`: - ``` - registry: - enabled: false - ``` + ```yaml + registry: + enabled: false + ``` 1. Save the file and [restart GitLab][] for the changes to take effect. @@ -280,9 +278,9 @@ the Container Registry by themselves, follow the steps below. 1. Edit `/etc/gitlab/gitlab.rb` and add the following line: - ```ruby - gitlab_rails['gitlab_default_projects_features_container_registry'] = false - ``` + ```ruby + gitlab_rails['gitlab_default_projects_features_container_registry'] = false + ``` 1. Save the file and [reconfigure GitLab][] for the changes to take effect. @@ -293,16 +291,16 @@ the Container Registry by themselves, follow the steps below. 1. Open `/home/git/gitlab/config/gitlab.yml`, find the `default_projects_features` entry and configure it so that `container_registry` is set to `false`: - ``` - ## Default project features settings - default_projects_features: - issues: true - merge_requests: true - wiki: true - snippets: false - builds: true - container_registry: false - ``` + ```yaml + ## Default project features settings + default_projects_features: + issues: true + merge_requests: true + wiki: true + snippets: false + builds: true + container_registry: false + ``` 1. Save the file and [restart GitLab][] for the changes to take effect. @@ -332,9 +330,9 @@ The default location where images are stored in Omnibus, is 1. Edit `/etc/gitlab/gitlab.rb`: - ```ruby - gitlab_rails['registry_path'] = "/path/to/registry/storage" - ``` + ```ruby + gitlab_rails['registry_path'] = "/path/to/registry/storage" + ``` 1. Save the file and [reconfigure GitLab][] for the changes to take effect. @@ -348,10 +346,10 @@ The default location where images are stored in source installations, is 1. Open `/home/git/gitlab/config/gitlab.yml`, find the `registry` entry and change the `path` setting: - ``` - registry: - path: shared/registry - ``` + ```yaml + registry: + path: shared/registry + ``` 1. Save the file and [restart GitLab][] for the changes to take effect. @@ -393,17 +391,17 @@ To configure the `s3` storage driver in Omnibus: 1. Edit `/etc/gitlab/gitlab.rb`: - ```ruby - registry['storage'] = { - 's3' => { - 'accesskey' => 's3-access-key', - 'secretkey' => 's3-secret-key-for-access-key', - 'bucket' => 'your-s3-bucket', - 'region' => 'your-s3-region', - 'regionendpoint' => 'your-s3-regionendpoint' - } - } - ``` + ```ruby + registry['storage'] = { + 's3' => { + 'accesskey' => 's3-access-key', + 'secretkey' => 's3-secret-key-for-access-key', + 'bucket' => 'your-s3-bucket', + 'region' => 'your-s3-region', + 'regionendpoint' => 'your-s3-regionendpoint' + } + } + ``` 1. Save the file and [reconfigure GitLab][] for the changes to take effect. @@ -442,9 +440,9 @@ In the examples below we set the Registry's port to `5001`. 1. Open `/etc/gitlab/gitlab.rb` and set `registry['registry_http_addr']`: - ```ruby - registry['registry_http_addr'] = "localhost:5001" - ``` + ```ruby + registry['registry_http_addr'] = "localhost:5001" + ``` 1. Save the file and [reconfigure GitLab][] for the changes to take effect. @@ -455,10 +453,10 @@ In the examples below we set the Registry's port to `5001`. 1. Open the configuration file of your Registry server and edit the [`http:addr`][registry-http-config] value: - ``` - http - addr: localhost:5001 - ``` + ```yaml + http + addr: localhost:5001 + ``` 1. Save the file and restart the Registry server. @@ -476,14 +474,14 @@ You can use GitLab as an auth endpoint and use a non-bundled Container Registry. 1. Open `/etc/gitlab/gitlab.rb` and set necessary configurations: - ```ruby - gitlab_rails['registry_enabled'] = true - gitlab_rails['registry_host'] = "registry.gitlab.example.com" - gitlab_rails['registry_port'] = "5005" - gitlab_rails['registry_api_url'] = "http://localhost:5000" - gitlab_rails['registry_path'] = "/var/opt/gitlab/gitlab-rails/shared/registry" - gitlab_rails['registry_issuer'] = "omnibus-gitlab-issuer" - ``` + ```ruby + gitlab_rails['registry_enabled'] = true + gitlab_rails['registry_host'] = "registry.gitlab.example.com" + gitlab_rails['registry_port'] = "5005" + gitlab_rails['registry_api_url'] = "http://localhost:5000" + gitlab_rails['registry_path'] = "/var/opt/gitlab/gitlab-rails/shared/registry" + gitlab_rails['registry_issuer'] = "omnibus-gitlab-issuer" + ``` 1. A certificate keypair is required for GitLab and the Container Registry to communicate securely. By default omnibus-gitlab will generate one keypair, @@ -492,19 +490,19 @@ You can use GitLab as an auth endpoint and use a non-bundled Container Registry. custom certificate key. To do that, add the following to `/etc/gitlab/gitlab.rb` - ```ruby - gitlab_rails['registry_key_path'] = "/custom/path/to/registry-key.key" - # registry['internal_key'] should contain the contents of the custom key - # file. Line breaks in the key file should be marked using `\n` character - # Example: - registry['internal_key'] = "---BEGIN RSA PRIVATE KEY---\nMIIEpQIBAA\n" - ``` + ```ruby + gitlab_rails['registry_key_path'] = "/custom/path/to/registry-key.key" + # registry['internal_key'] should contain the contents of the custom key + # file. Line breaks in the key file should be marked using `\n` character + # Example: + registry['internal_key'] = "---BEGIN RSA PRIVATE KEY---\nMIIEpQIBAA\n" + ``` - **Note:** The file specified at `registry_key_path` gets populated with the - content specified by `internal_key`, each time reconfigure is executed. If - no file is specified, omnibus-gitlab will default it to - `/var/opt/gitlab/gitlab-rails/etc/gitlab-registry.key` and will populate - it. + **Note:** The file specified at `registry_key_path` gets populated with the + content specified by `internal_key`, each time reconfigure is executed. If + no file is specified, omnibus-gitlab will default it to + `/var/opt/gitlab/gitlab-rails/etc/gitlab-registry.key` and will populate + it. 1. Save the file and [reconfigure GitLab][] for the changes to take effect. @@ -512,18 +510,18 @@ You can use GitLab as an auth endpoint and use a non-bundled Container Registry. 1. Open `/home/git/gitlab/config/gitlab.yml`, and edit the configuration settings under `registry`: - ``` - ## Container Registry + ```yaml + ## Container Registry - registry: - enabled: true - host: "registry.gitlab.example.com" - port: "5005" - api_url: "http://localhost:5000" - path: /var/opt/gitlab/gitlab-rails/shared/registry - key: /var/opt/gitlab/gitlab-rails/certificate.key - issuer: omnibus-gitlab-issuer - ``` + registry: + enabled: true + host: "registry.gitlab.example.com" + port: "5005" + api_url: "http://localhost:5000" + path: /var/opt/gitlab/gitlab-rails/shared/registry + key: /var/opt/gitlab/gitlab-rails/certificate.key + issuer: omnibus-gitlab-issuer + ``` 1. Save the file and [restart GitLab][] for the changes to take effect. @@ -550,20 +548,20 @@ To configure a notification endpoint in Omnibus: 1. Edit `/etc/gitlab/gitlab.rb`: - ```ruby - registry['notifications'] = [ - { - 'name' => 'test_endpoint', - 'url' => 'https://gitlab.example.com/notify', - 'timeout' => '500ms', - 'threshold' => 5, - 'backoff' => '1s', - 'headers' => { - "Authorization" => ["AUTHORIZATION_EXAMPLE_TOKEN"] - } - } - ] - ``` + ```ruby + registry['notifications'] = [ + { + 'name' => 'test_endpoint', + 'url' => 'https://gitlab.example.com/notify', + 'timeout' => '500ms', + 'threshold' => 5, + 'backoff' => '1s', + 'headers' => { + "Authorization" => ["AUTHORIZATION_EXAMPLE_TOKEN"] + } + } + ] + ``` 1. Save the file and [reconfigure GitLab][] for the changes to take effect. @@ -629,16 +627,16 @@ Start with a value between `25000000` (25MB) and `50000000` (50MB). 1. Edit `/etc/gitlab/gitlab.rb`: - ```ruby - registry['storage'] = { - 's3' => { - 'accesskey' => 'AKIAKIAKI', - 'secretkey' => 'secret123', - 'bucket' => 'gitlab-registry-bucket-AKIAKIAKI', - 'chunksize' => 25000000 - } - } - ``` + ```ruby + registry['storage'] = { + 's3' => { + 'accesskey' => 'AKIAKIAKI', + 'secretkey' => 'secret123', + 'bucket' => 'gitlab-registry-bucket-AKIAKIAKI', + 'chunksize' => 25000000 + } + } + ``` 1. Save the file and [reconfigure GitLab][] for the changes to take effect. @@ -648,14 +646,14 @@ Start with a value between `25000000` (25MB) and `50000000` (50MB). 1. Edit `config/gitlab.yml`: - ```yaml - storage: - s3: - accesskey: 'AKIAKIAKI' - secretkey: 'secret123' - bucket: 'gitlab-registry-bucket-AKIAKIAKI' - chunksize: 25000000 - ``` + ```yaml + storage: + s3: + accesskey: 'AKIAKIAKI' + secretkey: 'secret123' + bucket: 'gitlab-registry-bucket-AKIAKIAKI' + chunksize: 25000000 + ``` 1. Save the file and [restart GitLab][] for the changes to take effect. @@ -669,9 +667,9 @@ You can add a configuration option for backwards compatibility. 1. Edit `/etc/gitlab/gitlab.rb`: - ```ruby - registry['compatibility_schema1_enabled'] = true - ``` + ```ruby + registry['compatibility_schema1_enabled'] = true + ``` 1. Save the file and [reconfigure GitLab][] for the changes to take effect. @@ -681,11 +679,11 @@ You can add a configuration option for backwards compatibility. 1. Edit the YML configuration file you created when you [deployed the registry][registry-deploy]. Add the following snippet: - ```yaml - compatibility: - schema1: - enabled: true - ``` + ```yaml + compatibility: + schema1: + enabled: true + ``` 1. Restart the registry for the changes to take affect. @@ -694,9 +692,9 @@ You can add a configuration option for backwards compatibility. A Docker connection error can occur when there are special characters in either the group, project or branch name. Special characters can include: -* Leading underscore -* Trailing hyphen/dash -* Double hyphen/dash +- Leading underscore +- Trailing hyphen/dash +- Double hyphen/dash To get around this, you can [change the group path](../user/group/index.md#changing-a-groups-path), [change the project path](../user/project/settings/index.md#renaming-a-repository) or change the diff --git a/doc/administration/database_load_balancing.md b/doc/administration/database_load_balancing.md index e76d3b14a40..dc4cc401fca 100644 --- a/doc/administration/database_load_balancing.md +++ b/doc/administration/database_load_balancing.md @@ -74,9 +74,9 @@ the following. This will balance the load between `host1.example.com` and 1. Edit `/etc/gitlab/gitlab.rb` and add the following line: - ```ruby - gitlab_rails['db_load_balancing'] = { 'hosts' => ['host1.example.com', 'host2.example.com'] } - ``` + ```ruby + gitlab_rails['db_load_balancing'] = { 'hosts' => ['host1.example.com', 'host2.example.com'] } + ``` 1. Save the file and [reconfigure GitLab][] for the changes to take effect. @@ -86,16 +86,16 @@ the following. This will balance the load between `host1.example.com` and 1. Edit `/home/git/gitlab/config/database.yml` and add or amend the following lines: - ```yaml - production: - username: gitlab - database: gitlab - encoding: unicode - load_balancing: - hosts: - - host1.example.com - - host2.example.com - ``` + ```yaml + production: + username: gitlab + database: gitlab + encoding: unicode + load_balancing: + hosts: + - host1.example.com + - host2.example.com + ``` 1. Save the file and [restart GitLab][] for the changes to take effect. diff --git a/doc/administration/geo/replication/index.md b/doc/administration/geo/replication/index.md index 5cfa6bcae56..f0d329d5296 100644 --- a/doc/administration/geo/replication/index.md +++ b/doc/administration/geo/replication/index.md @@ -178,7 +178,7 @@ The steps below should be followed in the order they appear. **Make sure the Git If you installed GitLab using the Omnibus packages (highly recommended): -1. [Install GitLab Enterprise Edition](https://about.gitlab.com/installation/) on the server that will serve as the **secondary** node. Do not create an account or log in to the new **secondary** node. +1. [Install GitLab Enterprise Edition](https://about.gitlab.com/install/) on the server that will serve as the **secondary** node. Do not create an account or log in to the new **secondary** node. 1. [Upload the GitLab License](../../../user/admin_area/license.md) on the **primary** node to unlock Geo. The license must be for [GitLab Premium](https://about.gitlab.com/pricing/) or higher. 1. [Set up the database replication](database.md) (`primary (read-write) <-> secondary (read-only)` topology). 1. [Configure fast lookup of authorized SSH keys in the database](../../operations/fast_ssh_key_lookup.md). This step is required and needs to be done on **both** the **primary** and **secondary** nodes. diff --git a/doc/administration/geo/replication/security_review.md b/doc/administration/geo/replication/security_review.md index 7c21a348e23..ed3f1faa93e 100644 --- a/doc/administration/geo/replication/security_review.md +++ b/doc/administration/geo/replication/security_review.md @@ -115,7 +115,7 @@ questions from [owasp.org](https://www.owasp.org). ### What operating systems support the application? - Geo imposes no additional restrictions on operating system (see the - [GitLab installation](https://about.gitlab.com/installation/) page for more + [GitLab installation](https://about.gitlab.com/install/) page for more details), however we recommend using the operating systems listed in the [Geo documentation](index.md#requirements-for-running-geo). ### What details regarding required OS components and lock‐down needs have been defined? diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md index b98383f30fc..4407facfca9 100644 --- a/doc/administration/gitaly/index.md +++ b/doc/administration/gitaly/index.md @@ -91,7 +91,7 @@ your GitLab installation has two repository storages, `default` and First install Gitaly using either Omnibus or from source. -Omnibus: [Download/install](https://about.gitlab.com/installation) the Omnibus GitLab +Omnibus: [Download/install](https://about.gitlab.com/install/) the Omnibus GitLab package you want using **steps 1 and 2** from the GitLab downloads page but **_do not_** provide the `EXTERNAL_URL=` value. @@ -220,7 +220,7 @@ network, firewall, or name resolution problem preventing your GitLab server from reaching the Gitaly server then all Gitaly requests will fail. -Additionally, you need to +Additionally, you need to [disable Rugged if previously manually enabled](../high_availability/nfs.md#improving-nfs-performance-with-gitlab). We assume that your Gitaly server can be reached at @@ -436,8 +436,8 @@ particular machine. ## Eliminating NFS altogether -If you are planning to use Gitaly without NFS for your storage needs -and want to eliminate NFS from your environment altogether, there are +If you are planning to use Gitaly without NFS for your storage needs +and want to eliminate NFS from your environment altogether, there are a few things that you need to do: 1. Make sure the [`git` user home directory](https://docs.gitlab.com/omnibus/settings/configuration.html#moving-the-home-directory-for-a-user) is on local disk. diff --git a/doc/administration/high_availability/consul.md b/doc/administration/high_availability/consul.md index 3bdce7bce5e..49199b659bc 100644 --- a/doc/administration/high_availability/consul.md +++ b/doc/administration/high_availability/consul.md @@ -6,6 +6,72 @@ As part of its High Availability stack, GitLab Premium includes a bundled versio A Consul cluster consists of multiple server agents, as well as client agents that run on other nodes which need to talk to the consul cluster. +## Prerequisites + +First, make sure to [download/install](https://about.gitlab.com/install/) +GitLab Omnibus **on each node**. + +Choose an installation method, then make sure you complete steps: + +1. Install and configure the necessary dependencies. +1. Add the GitLab package repository and install the package. + +When installing the GitLab package, do not supply `EXTERNAL_URL` value. + +## Configuring the Consul nodes + +On each Consul node perform the following: + +1. Make sure you collect [`CONSUL_SERVER_NODES`](database.md#consul-information), which are the IP addresses or DNS records of the Consul server nodes, for the next step, before executing the next step. + +1. Edit `/etc/gitlab/gitlab.rb` replacing values noted in the `# START user configuration` section: + + ```ruby + # Disable all components except Consul + roles ['consul_role'] + + # START user configuration + # Replace placeholders: + # + # Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z + # with the addresses gathered for CONSUL_SERVER_NODES + consul['configuration'] = { + server: true, + retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z) + } + + # Disable auto migrations + gitlab_rails['auto_migrate'] = false + # + # END user configuration + ``` + + > `consul_role` was introduced with GitLab 10.3 + +1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes + to take effect. + +### Consul checkpoint + +Before moving on, make sure Consul is configured correctly. Run the following +command to verify all server nodes are communicating: + +```sh +/opt/gitlab/embedded/bin/consul members +``` + +The output should be similar to: + +``` +Node Address Status Type Build Protocol DC +CONSUL_NODE_ONE XXX.XXX.XXX.YYY:8301 alive server 0.9.2 2 gitlab_consul +CONSUL_NODE_TWO XXX.XXX.XXX.YYY:8301 alive server 0.9.2 2 gitlab_consul +CONSUL_NODE_THREE XXX.XXX.XXX.YYY:8301 alive server 0.9.2 2 gitlab_consul +``` + +If any of the nodes isn't `alive` or if any of the three nodes are missing, +check the [Troubleshooting section](#troubleshooting) before proceeding. + ## Operations ### Checking cluster membership diff --git a/doc/administration/high_availability/database.md b/doc/administration/high_availability/database.md index d7acc1e7eb1..c32a73080ff 100644 --- a/doc/administration/high_availability/database.md +++ b/doc/administration/high_availability/database.md @@ -31,7 +31,7 @@ deploy the bundled PostgreSQL. ### Standalone PostgreSQL using GitLab Omnibus **(CORE ONLY)** 1. SSH into the PostgreSQL server. -1. [Download/install](https://about.gitlab.com/installation) the Omnibus GitLab +1. [Download/install](https://about.gitlab.com/install/) the Omnibus GitLab package you want using **steps 1 and 2** from the GitLab downloads page. - Do not complete any other steps on the download page. 1. Generate a password hash for PostgreSQL. This assumes you will use the default @@ -281,68 +281,17 @@ Few notes on the service itself: #### Installing Omnibus GitLab -First, make sure to [download/install](https://about.gitlab.com/installation) +First, make sure to [download/install](https://about.gitlab.com/install/) GitLab Omnibus **on each node**. Make sure you install the necessary dependencies from step 1, add GitLab package repository from step 2. When installing the GitLab package, do not supply `EXTERNAL_URL` value. -#### Configuring the Consul nodes - -On each Consul node perform the following: - -1. Make sure you collect [`CONSUL_SERVER_NODES`](#consul-information) before executing the next step. - -1. Edit `/etc/gitlab/gitlab.rb` replacing values noted in the `# START user configuration` section: - - ```ruby - # Disable all components except Consul - roles ['consul_role'] - - # START user configuration - # Replace placeholders: - # - # Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z - # with the addresses gathered for CONSUL_SERVER_NODES - consul['configuration'] = { - server: true, - retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z) - } - - # Disable auto migrations - gitlab_rails['auto_migrate'] = false - # - # END user configuration - ``` - - > `consul_role` was introduced with GitLab 10.3 - -1. [Reconfigure GitLab] for the changes to take effect. - -##### Consul Checkpoint - -Before moving on, make sure Consul is configured correctly. Run the following -command to verify all server nodes are communicating: - -```sh -/opt/gitlab/embedded/bin/consul members -``` - -The output should be similar to: - -``` -Node Address Status Type Build Protocol DC -CONSUL_NODE_ONE XXX.XXX.XXX.YYY:8301 alive server 0.9.2 2 gitlab_consul -CONSUL_NODE_TWO XXX.XXX.XXX.YYY:8301 alive server 0.9.2 2 gitlab_consul -CONSUL_NODE_THREE XXX.XXX.XXX.YYY:8301 alive server 0.9.2 2 gitlab_consul -``` - -If any of the nodes isn't `alive` or if any of the three nodes are missing, -check the [Troubleshooting section](#troubleshooting) before proceeding. #### Configuring the Database nodes +1. Make sure to [configure the Consul nodes](consul.md). 1. Make sure you collect [`CONSUL_SERVER_NODES`](#consul-information), [`PGBOUNCER_PASSWORD_HASH`](#pgbouncer-information), [`POSTGRESQL_PASSWORD_HASH`](#postgresql-information), the [number of db nodes](#postgresql-information), and the [network address](#network-information) before executing the next step. 1. On the master database node, edit `/etc/gitlab/gitlab.rb` replacing values noted in the `# START user configuration` section: @@ -570,7 +519,7 @@ Check the [Troubleshooting section](#troubleshooting) before proceeding. 1. [Reconfigure GitLab] for the changes to take effect. -1. Create a `.pgpass` file so Consule is able to +1. Create a `.pgpass` file so Consul is able to reload pgbouncer. Enter the `PGBOUNCER_PASSWORD` twice when asked: ```sh diff --git a/doc/administration/high_availability/gitlab.md b/doc/administration/high_availability/gitlab.md index 3045be616a6..9b1b7142e83 100644 --- a/doc/administration/high_availability/gitlab.md +++ b/doc/administration/high_availability/gitlab.md @@ -125,7 +125,7 @@ need some extra configuration. from running on upgrade. Only the primary GitLab application server should handle migrations. -1. **Optional** Configure host keys. Copy all contents(primary and public keys) inside `/etc/ssh/` on +1. **Recommended** Configure host keys. Copy the contents (primary and public keys) of `/etc/ssh/` on the primary application server to `/etc/ssh` on all secondary servers. This prevents false man-in-the-middle-attack alerts when accessing servers in your High Availability cluster behind a load balancer. diff --git a/doc/administration/high_availability/redis.md b/doc/administration/high_availability/redis.md index 984c9688cdf..27310c59755 100644 --- a/doc/administration/high_availability/redis.md +++ b/doc/administration/high_availability/redis.md @@ -41,7 +41,7 @@ The steps below are the minimum necessary to configure a Redis server with Omnibus: 1. SSH into the Redis server. -1. [Download/install](https://about.gitlab.com/installation) the Omnibus GitLab +1. [Download/install](https://about.gitlab.com/install/) the Omnibus GitLab package you want using **steps 1 and 2** from the GitLab downloads page. - Do not complete any other steps on the download page. @@ -357,7 +357,7 @@ The prerequisites for a HA Redis setup are the following: ### Step 1. Configuring the master Redis instance 1. SSH into the **master** Redis server. -1. [Download/install](https://about.gitlab.com/installation) the Omnibus GitLab +1. [Download/install](https://about.gitlab.com/install/) the Omnibus GitLab package you want using **steps 1 and 2** from the GitLab downloads page. - Make sure you select the correct Omnibus package, with the same version and type (Community, Enterprise editions) of your current install. @@ -400,7 +400,7 @@ The prerequisites for a HA Redis setup are the following: ### Step 2. Configuring the slave Redis instances 1. SSH into the **slave** Redis server. -1. [Download/install](https://about.gitlab.com/installation) the Omnibus GitLab +1. [Download/install](https://about.gitlab.com/install/) the Omnibus GitLab package you want using **steps 1 and 2** from the GitLab downloads page. - Make sure you select the correct Omnibus package, with the same version and type (Community, Enterprise editions) of your current install. diff --git a/doc/administration/audit_log.png b/doc/administration/img/audit_log.png Binary files differindex d4f4c2abf38..d4f4c2abf38 100644 --- a/doc/administration/audit_log.png +++ b/doc/administration/img/audit_log.png diff --git a/doc/administration/auditor_access_form.png b/doc/administration/img/auditor_access_form.png Binary files differindex c179a7d3b0a..c179a7d3b0a 100644 --- a/doc/administration/auditor_access_form.png +++ b/doc/administration/img/auditor_access_form.png diff --git a/doc/administration/incoming_email.md b/doc/administration/incoming_email.md index 53a61358670..73a39a6dd35 100644 --- a/doc/administration/incoming_email.md +++ b/doc/administration/incoming_email.md @@ -102,16 +102,16 @@ for a real-world example of this exploit. 1. Reconfigure GitLab for the changes to take effect: - ```sh - sudo gitlab-ctl reconfigure - sudo gitlab-ctl restart - ``` + ```sh + sudo gitlab-ctl reconfigure + sudo gitlab-ctl restart + ``` 1. Verify that everything is configured correctly: - ```sh - sudo gitlab-rake gitlab:incoming_email:check - ``` + ```sh + sudo gitlab-rake gitlab:incoming_email:check + ``` Reply by email should now be working. @@ -119,31 +119,31 @@ Reply by email should now be working. 1. Go to the GitLab installation directory: - ```sh - cd /home/git/gitlab - ``` + ```sh + cd /home/git/gitlab + ``` 1. Find the `incoming_email` section in `config/gitlab.yml`, enable the feature and fill in the details for your specific IMAP server and email account (see [examples](#config-examples) below). 1. Enable `mail_room` in the init script at `/etc/default/gitlab`: - ```sh - sudo mkdir -p /etc/default - echo 'mail_room_enabled=true' | sudo tee -a /etc/default/gitlab - ``` + ```sh + sudo mkdir -p /etc/default + echo 'mail_room_enabled=true' | sudo tee -a /etc/default/gitlab + ``` 1. Restart GitLab: - ```sh - sudo service gitlab restart - ``` + ```sh + sudo service gitlab restart + ``` 1. Verify that everything is configured correctly: - ```sh - sudo -u git -H bundle exec rake gitlab:incoming_email:check RAILS_ENV=production - ``` + ```sh + sudo -u git -H bundle exec rake gitlab:incoming_email:check RAILS_ENV=production + ``` Reply by email should now be working. diff --git a/doc/administration/index.md b/doc/administration/index.md index c5909e700c5..00c8863f200 100644 --- a/doc/administration/index.md +++ b/doc/administration/index.md @@ -11,7 +11,7 @@ GitLab has two product distributions available through [different subscriptions] - The open source [GitLab Community Edition (CE)](https://gitlab.com/gitlab-org/gitlab-ce). - The open core [GitLab Enterprise Edition (EE)](https://gitlab.com/gitlab-org/gitlab-ee). -You can [install either GitLab CE or GitLab EE](https://about.gitlab.com/installation/ce-or-ee/). +You can [install either GitLab CE or GitLab EE](https://about.gitlab.com/install/ce-or-ee/). However, the features you'll have access to depend on the subscription you choose (Core, Starter, Premium, or Ultimate). diff --git a/doc/administration/issue_closing_pattern.md b/doc/administration/issue_closing_pattern.md index f2877c49d05..e1bbabb2878 100644 --- a/doc/administration/issue_closing_pattern.md +++ b/doc/administration/issue_closing_pattern.md @@ -27,9 +27,10 @@ Because Rubular doesn't understand `%{issue_ref}`, you can replace this by 1. Change the value of `gitlab_rails['gitlab_issue_closing_pattern']` to a regular expression of your liking: - ```ruby - gitlab_rails['gitlab_issue_closing_pattern'] = "\b((?:[Cc]los(?:e[sd]|ing)|\b[Ff]ix(?:e[sd]|ing)?) +(?:(?:issues? +)?%{issue_ref}(?:(?:, *| +and +)?))+)" - ``` + ```ruby + gitlab_rails['gitlab_issue_closing_pattern'] = "\b((?:[Cc]los(?:e[sd]|ing)|\b[Ff]ix(?:e[sd]|ing)?) +(?:(?:issues? +)?%{issue_ref}(?:(?:, *| +and +)?))+)" + ``` + 1. [Reconfigure] GitLab for the changes to take effect. **For installations from source** @@ -37,9 +38,9 @@ Because Rubular doesn't understand `%{issue_ref}`, you can replace this by 1. Open `gitlab.yml` with your editor. 1. Change the value of `issue_closing_pattern`: - ```yaml - issue_closing_pattern: "\b((?:[Cc]los(?:e[sd]|ing)|\b[Ff]ix(?:e[sd]|ing)?) +(?:(?:issues? +)?%{issue_ref}(?:(?:, *| +and +)?))+)" - ``` + ```yaml + issue_closing_pattern: "\b((?:[Cc]los(?:e[sd]|ing)|\b[Ff]ix(?:e[sd]|ing)?) +(?:(?:issues? +)?%{issue_ref}(?:(?:, *| +and +)?))+)" + ``` 1. [Restart] GitLab for the changes to take effect. diff --git a/doc/administration/job_artifacts.md b/doc/administration/job_artifacts.md index 05e15fc303b..9df7b2526e2 100644 --- a/doc/administration/job_artifacts.md +++ b/doc/administration/job_artifacts.md @@ -1,7 +1,5 @@ # Jobs artifacts administration -> **Notes:** -> > - Introduced in GitLab 8.2 and GitLab Runner 0.7.0. > - Starting with GitLab 8.4 and GitLab Runner 1.0, the artifacts archive format changed to `ZIP`. > - Starting with GitLab 8.17, builds are renamed to jobs. @@ -21,9 +19,9 @@ To disable artifacts site-wide, follow the steps below. 1. Edit `/etc/gitlab/gitlab.rb` and add the following line: - ```ruby - gitlab_rails['artifacts_enabled'] = false - ``` + ```ruby + gitlab_rails['artifacts_enabled'] = false + ``` 1. Save the file and [reconfigure GitLab][] for the changes to take effect. @@ -33,10 +31,10 @@ To disable artifacts site-wide, follow the steps below. 1. Edit `/home/git/gitlab/config/gitlab.yml` and add or amend the following lines: - ```yaml - artifacts: - enabled: false - ``` + ```yaml + artifacts: + enabled: false + ``` 1. Save the file and [restart GitLab][] for the changes to take effect. @@ -61,9 +59,9 @@ _The artifacts are stored by default in 1. To change the storage path for example to `/mnt/storage/artifacts`, edit `/etc/gitlab/gitlab.rb` and add the following line: - ```ruby - gitlab_rails['artifacts_path'] = "/mnt/storage/artifacts" - ``` + ```ruby + gitlab_rails['artifacts_path'] = "/mnt/storage/artifacts" + ``` 1. Save the file and [reconfigure GitLab][] for the changes to take effect. @@ -77,18 +75,16 @@ _The artifacts are stored by default in 1. To change the storage path for example to `/mnt/storage/artifacts`, edit `/home/git/gitlab/config/gitlab.yml` and add or amend the following lines: - ```yaml - artifacts: - enabled: true - path: /mnt/storage/artifacts - ``` + ```yaml + artifacts: + enabled: true + path: /mnt/storage/artifacts + ``` 1. Save the file and [restart GitLab][] for the changes to take effect. ### Using object storage -> **Notes:** -> > - [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/1762) in > [GitLab Premium](https://about.gitlab.com/pricing/) 9.4. > - Since version 9.5, artifacts are [browsable](../user/project/pipelines/job_artifacts.md#browsing-artifacts), @@ -141,35 +137,35 @@ _The artifacts are stored by default in 1. Edit `/etc/gitlab/gitlab.rb` and add the following lines by replacing with the values you want: - ```ruby - gitlab_rails['artifacts_enabled'] = true - gitlab_rails['artifacts_object_store_enabled'] = true - gitlab_rails['artifacts_object_store_remote_directory'] = "artifacts" - gitlab_rails['artifacts_object_store_connection'] = { - 'provider' => 'AWS', - 'region' => 'eu-central-1', - 'aws_access_key_id' => 'AWS_ACCESS_KEY_ID', - 'aws_secret_access_key' => 'AWS_SECRET_ACCESS_KEY' - } - ``` - - NOTE: For GitLab 9.4+, if you are using AWS IAM profiles, be sure to omit the - AWS access key and secret access key/value pairs. For example: - - ```ruby - gitlab_rails['artifacts_object_store_connection'] = { - 'provider' => 'AWS', - 'region' => 'eu-central-1', - 'use_iam_profile' => true - } - ``` + ```ruby + gitlab_rails['artifacts_enabled'] = true + gitlab_rails['artifacts_object_store_enabled'] = true + gitlab_rails['artifacts_object_store_remote_directory'] = "artifacts" + gitlab_rails['artifacts_object_store_connection'] = { + 'provider' => 'AWS', + 'region' => 'eu-central-1', + 'aws_access_key_id' => 'AWS_ACCESS_KEY_ID', + 'aws_secret_access_key' => 'AWS_SECRET_ACCESS_KEY' + } + ``` + + NOTE: For GitLab 9.4+, if you are using AWS IAM profiles, be sure to omit the + AWS access key and secret access key/value pairs. For example: + + ```ruby + gitlab_rails['artifacts_object_store_connection'] = { + 'provider' => 'AWS', + 'region' => 'eu-central-1', + 'use_iam_profile' => true + } + ``` 1. Save the file and [reconfigure GitLab][] for the changes to take effect. 1. Migrate any existing local artifacts to the object storage: - ```bash - gitlab-rake gitlab:artifacts:migrate - ``` + ```bash + gitlab-rake gitlab:artifacts:migrate + ``` --- @@ -181,25 +177,25 @@ _The artifacts are stored by default in 1. Edit `/home/git/gitlab/config/gitlab.yml` and add or amend the following lines: - ```yaml - artifacts: - enabled: true - object_store: - enabled: true - remote_directory: "artifacts" # The bucket name - connection: - provider: AWS # Only AWS supported at the moment - aws_access_key_id: AWS_ACCESS_KEY_ID - aws_secret_access_key: AWS_SECRET_ACCESS_KEY - region: eu-central-1 - ``` + ```yaml + artifacts: + enabled: true + object_store: + enabled: true + remote_directory: "artifacts" # The bucket name + connection: + provider: AWS # Only AWS supported at the moment + aws_access_key_id: AWS_ACCESS_KEY_ID + aws_secret_access_key: AWS_SECRET_ACCESS_KEY + region: eu-central-1 + ``` 1. Save the file and [restart GitLab][] for the changes to take effect. 1. Migrate any existing local artifacts to the object storage: - ```bash - sudo -u git -H bundle exec rake gitlab:artifacts:migrate RAILS_ENV=production - ``` + ```bash + sudo -u git -H bundle exec rake gitlab:artifacts:migrate RAILS_ENV=production + ``` ## Expiring artifacts @@ -217,9 +213,9 @@ steps below. 1. Edit `/etc/gitlab/gitlab.rb` and comment out or add the following line - ```ruby - gitlab_rails['expire_build_artifacts_worker_cron'] = "50 * * * *" - ``` + ```ruby + gitlab_rails['expire_build_artifacts_worker_cron'] = "50 * * * *" + ``` 1. Save the file and [reconfigure GitLab][] for the changes to take effect. @@ -230,10 +226,10 @@ steps below. 1. Edit `/home/git/gitlab/config/gitlab.yml` and add or amend the following lines: - ```yaml - expire_build_artifacts_worker: - cron: "50 * * * *" - ``` + ```yaml + expire_build_artifacts_worker: + cron: "50 * * * *" + ``` 1. Save the file and [restart GitLab][] for the changes to take effect. @@ -250,15 +246,15 @@ you can flip the feature flag from a Rails console. 1. Enter the Rails console: - ```sh - sudo gitlab-rails console - ``` + ```sh + sudo gitlab-rails console + ``` 1. Flip the switch and disable it: - ```ruby - Feature.enable('ci_disable_validates_dependencies') - ``` + ```ruby + Feature.enable('ci_disable_validates_dependencies') + ``` --- @@ -266,16 +262,16 @@ you can flip the feature flag from a Rails console. 1. Enter the Rails console: - ```sh - cd /home/git/gitlab - RAILS_ENV=production sudo -u git -H bundle exec rails console - ``` + ```sh + cd /home/git/gitlab + RAILS_ENV=production sudo -u git -H bundle exec rails console + ``` 1. Flip the switch and disable it: - ```ruby - Feature.enable('ci_disable_validates_dependencies') - ``` + ```ruby + Feature.enable('ci_disable_validates_dependencies') + ``` ## Set the maximum file size of the artifacts diff --git a/doc/administration/job_traces.md b/doc/administration/job_traces.md index aa9d87562a3..957916893d7 100644 --- a/doc/administration/job_traces.md +++ b/doc/administration/job_traces.md @@ -25,11 +25,11 @@ To change the location where the job logs will be stored, follow the steps below **In Omnibus installations:** -1. Edit `/etc/gitlab/gitlab.rb` and add or amend the following line: +1. Edit `/etc/gitlab/gitlab.rb` and add or amend the following line: - ``` - gitlab_ci['builds_directory'] = '/mnt/to/gitlab-ci/builds' - ``` + ```ruby + gitlab_ci['builds_directory'] = '/mnt/to/gitlab-ci/builds' + ``` 1. Save the file and [reconfigure GitLab][] for the changes to take effect. @@ -39,12 +39,12 @@ To change the location where the job logs will be stored, follow the steps below 1. Edit `/home/git/gitlab/config/gitlab.yml` and add or amend the following lines: - ```yaml - gitlab_ci: - # The location where build traces are stored (default: builds/). - # Relative paths are relative to Rails.root. - builds_path: path/to/builds/ - ``` + ```yaml + gitlab_ci: + # The location where build traces are stored (default: builds/). + # Relative paths are relative to Rails.root. + builds_path: path/to/builds/ + ``` 1. Save the file and [restart GitLab][] for the changes to take effect. @@ -67,24 +67,24 @@ To archive those legacy job traces, please follow the instruction below. 1. Execute the following command - ```bash - gitlab-rake gitlab:traces:archive - ``` + ```bash + gitlab-rake gitlab:traces:archive + ``` - After you executed this task, GitLab instance queues up Sidekiq jobs (asynchronous processes) - for migrating job trace files from local storage to object storage. - It could take time to complete the all migration jobs. You can check the progress by the following command + After you executed this task, GitLab instance queues up Sidekiq jobs (asynchronous processes) + for migrating job trace files from local storage to object storage. + It could take time to complete the all migration jobs. You can check the progress by the following command - ```bash - sudo gitlab-rails console - ``` + ```bash + sudo gitlab-rails console + ``` - ```bash - [1] pry(main)> Sidekiq::Stats.new.queues['pipeline_background:archive_trace'] - => 100 - ``` + ```bash + [1] pry(main)> Sidekiq::Stats.new.queues['pipeline_background:archive_trace'] + => 100 + ``` - If the count becomes zero, the archiving processes are done + If the count becomes zero, the archiving processes are done ## How to migrate archived job traces to object storage diff --git a/doc/administration/merge_request_diffs.md b/doc/administration/merge_request_diffs.md index c328d4c3aaa..99cd9051778 100644 --- a/doc/administration/merge_request_diffs.md +++ b/doc/administration/merge_request_diffs.md @@ -21,18 +21,18 @@ To enable external storage of merge request diffs, follow the instructions below 1. Edit `/etc/gitlab/gitlab.rb` and add the following line: - ```ruby - gitlab_rails['external_diffs_enabled'] = true - ``` + ```ruby + gitlab_rails['external_diffs_enabled'] = true + ``` 1. _The external diffs will be stored in in `/var/opt/gitlab/gitlab-rails/shared/external-diffs`._ To change the path, for example, to `/mnt/storage/external-diffs`, edit `/etc/gitlab/gitlab.rb` and add the following line: - ```ruby - gitlab_rails['external_diffs_storage_path'] = "/mnt/storage/external-diffs" - ``` + ```ruby + gitlab_rails['external_diffs_storage_path'] = "/mnt/storage/external-diffs" + ``` 1. Save the file and [reconfigure GitLab](restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. @@ -41,21 +41,21 @@ To enable external storage of merge request diffs, follow the instructions below 1. Edit `/home/git/gitlab/config/gitlab.yml` and add or amend the following lines: - ```yaml - external_diffs: - enabled: true - ``` + ```yaml + external_diffs: + enabled: true + ``` 1. _The external diffs will be stored in `/home/git/gitlab/shared/external-diffs`._ To change the path, for example, to `/mnt/storage/external-diffs`, edit `/home/git/gitlab/config/gitlab.yml` and add or amend the following lines: - ```yaml - external_diffs: - enabled: true - storage_path: /mnt/storage/external-diffs - ``` + ```yaml + external_diffs: + enabled: true + storage_path: /mnt/storage/external-diffs + ``` 1. Save the file and [restart GitLab](restart_gitlab.md#installations-from-source) for the changes to take effect. @@ -101,28 +101,28 @@ The connection settings match those provided by [Fog](https://github.com/fog), a 1. Edit `/etc/gitlab/gitlab.rb` and add the following lines by replacing with the values you want: - ```ruby - gitlab_rails['external_diffs_enabled'] = true - gitlab_rails['external_diffs_object_store_enabled'] = true - gitlab_rails['external_diffs_object_store_remote_directory'] = "external-diffs" - gitlab_rails['external_diffs_object_store_connection'] = { - 'provider' => 'AWS', - 'region' => 'eu-central-1', - 'aws_access_key_id' => 'AWS_ACCESS_KEY_ID', - 'aws_secret_access_key' => 'AWS_SECRET_ACCESS_KEY' - } - ``` - - Note that, if you are using AWS IAM profiles, be sure to omit the - AWS access key and secret access key/value pairs. For example: - - ```ruby - gitlab_rails['external_diffs_object_store_connection'] = { - 'provider' => 'AWS', - 'region' => 'eu-central-1', - 'use_iam_profile' => true - } - ``` + ```ruby + gitlab_rails['external_diffs_enabled'] = true + gitlab_rails['external_diffs_object_store_enabled'] = true + gitlab_rails['external_diffs_object_store_remote_directory'] = "external-diffs" + gitlab_rails['external_diffs_object_store_connection'] = { + 'provider' => 'AWS', + 'region' => 'eu-central-1', + 'aws_access_key_id' => 'AWS_ACCESS_KEY_ID', + 'aws_secret_access_key' => 'AWS_SECRET_ACCESS_KEY' + } + ``` + + Note that, if you are using AWS IAM profiles, be sure to omit the + AWS access key and secret access key/value pairs. For example: + + ```ruby + gitlab_rails['external_diffs_object_store_connection'] = { + 'provider' => 'AWS', + 'region' => 'eu-central-1', + 'use_iam_profile' => true + } + ``` 1. Save the file and [reconfigure GitLab](restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. @@ -131,18 +131,18 @@ The connection settings match those provided by [Fog](https://github.com/fog), a 1. Edit `/home/git/gitlab/config/gitlab.yml` and add or amend the following lines: - ```yaml - external_diffs: - enabled: true - object_store: - enabled: true - remote_directory: "external-diffs" # The bucket name - connection: - provider: AWS # Only AWS supported at the moment - aws_access_key_id: AWS_ACCESS_KEY_ID - aws_secret_access_key: AWS_SECRET_ACCESS_KEY - region: eu-central-1 - ``` + ```yaml + external_diffs: + enabled: true + object_store: + enabled: true + remote_directory: "external-diffs" # The bucket name + connection: + provider: AWS # Only AWS supported at the moment + aws_access_key_id: AWS_ACCESS_KEY_ID + aws_secret_access_key: AWS_SECRET_ACCESS_KEY + region: eu-central-1 + ``` 1. Save the file and [restart GitLab](restart_gitlab.md#installations-from-source) for the changes to take effect. @@ -157,11 +157,11 @@ To enable this feature, perform the following steps: **In Omnibus installations:** -1. Edit `/etc/gitlab/gitlab.rb` and add the following line: +1. Edit `/etc/gitlab/gitlab.rb` and add the following line: - ```ruby - gitlab_rails['external_diffs_when'] = 'outdated' - ``` + ```ruby + gitlab_rails['external_diffs_when'] = 'outdated' + ``` 1. Save the file and [reconfigure GitLab](restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. @@ -170,11 +170,11 @@ To enable this feature, perform the following steps: 1. Edit `/home/git/gitlab/config/gitlab.yml` and add or amend the following lines: - ```yaml - external_diffs: - enabled: true - when: outdated - ``` + ```yaml + external_diffs: + enabled: true + when: outdated + ``` 1. Save the file and [restart GitLab](restart_gitlab.md#installations-from-source) for the changes to take effect. diff --git a/doc/administration/packages.md b/doc/administration/packages.md index cc7d76d61f6..c0f8777a8c0 100644 --- a/doc/administration/packages.md +++ b/doc/administration/packages.md @@ -11,7 +11,7 @@ The Packages feature allows GitLab to act as a repository for the following: | [Maven Repository](../user/project/packages/maven_repository.md) | The GitLab Maven Repository enables every project in GitLab to have its own space to store [Maven](https://maven.apache.org/) packages. | 11.3+ | | [NPM Registry](../user/project/packages/npm_registry.md) | The GitLab NPM Registry enables every project in GitLab to have its own space to store [NPM](https://www.npmjs.com/) packages. | 11.7+ | -Don't you see your package management system supported yet? +Don't you see your package management system supported yet? Please consider contributing to GitLab. This [development documentation](../development/packages.md) will guide you through the process. @@ -28,9 +28,9 @@ To enable the Packages feature: 1. Edit `/etc/gitlab/gitlab.rb` and add the following line: - ```ruby - gitlab_rails['packages_enabled'] = true - ``` + ```ruby + gitlab_rails['packages_enabled'] = true + ``` 1. Save the file and [reconfigure GitLab][] for the changes to take effect. @@ -39,10 +39,11 @@ To enable the Packages feature: 1. After the installation is complete, you will have to configure the `packages` section in `config/gitlab.yml`. Set to `true` to enable it: - ```yaml - packages: - enabled: true - ``` + ```yaml + packages: + enabled: true + ``` + 1. [Restart GitLab] for the changes to take effect. ## Changing the storage path @@ -61,9 +62,9 @@ To change the local storage path: 1. Edit `/etc/gitlab/gitlab.rb` and add the following line: - ```ruby - gitlab_rails['packages_storage_path'] = "/mnt/packages" - ``` + ```ruby + gitlab_rails['packages_storage_path'] = "/mnt/packages" + ``` 1. Save the file and [reconfigure GitLab][] for the changes to take effect. @@ -71,11 +72,12 @@ To change the local storage path: 1. Edit the `packages` section in `config/gitlab.yml`: - ```yaml - packages: - enabled: true - storage_path: shared/packages - ``` + ```yaml + packages: + enabled: true + storage_path: shared/packages + ``` + 1. [Restart GitLab] for the changes to take effect. ### Using object storage @@ -88,31 +90,31 @@ upload packages: 1. Edit `/etc/gitlab/gitlab.rb` and add the following lines (uncomment where necessary): - ```ruby - gitlab_rails['packages_enabled'] = true - gitlab_rails['packages_storage_path'] = "/var/opt/gitlab/gitlab-rails/shared/packages" - gitlab_rails['packages_object_store_enabled'] = true - gitlab_rails['packages_object_store_remote_directory'] = "packages" # The bucket name. - gitlab_rails['packages_object_store_direct_upload'] = false # Use Object Storage directly for uploads instead of background uploads if enabled (Default: false). - gitlab_rails['packages_object_store_background_upload'] = true # Temporary option to limit automatic upload (Default: true). - gitlab_rails['packages_object_store_proxy_download'] = false # Passthrough all downloads via GitLab instead of using Redirects to Object Storage. - gitlab_rails['packages_object_store_connection'] = { - ## - ## If the provider is AWS S3, uncomment the following - ## - #'provider' => 'AWS', - #'region' => 'eu-west-1', - #'aws_access_key_id' => 'AWS_ACCESS_KEY_ID', - #'aws_secret_access_key' => 'AWS_SECRET_ACCESS_KEY', - ## - ## If the provider is other than AWS (an S3-compatible one), uncomment the following - ## - #'host' => 's3.amazonaws.com', - #'aws_signature_version' => 4 # For creation of signed URLs. Set to 2 if provider does not support v4. - #'endpoint' => 'https://s3.amazonaws.com' # Useful for S3-compliant services such as DigitalOcean Spaces. - #'path_style' => false # If true, use 'host/bucket_name/object' instead of 'bucket_name.host/object'. - } - ``` + ```ruby + gitlab_rails['packages_enabled'] = true + gitlab_rails['packages_storage_path'] = "/var/opt/gitlab/gitlab-rails/shared/packages" + gitlab_rails['packages_object_store_enabled'] = true + gitlab_rails['packages_object_store_remote_directory'] = "packages" # The bucket name. + gitlab_rails['packages_object_store_direct_upload'] = false # Use Object Storage directly for uploads instead of background uploads if enabled (Default: false). + gitlab_rails['packages_object_store_background_upload'] = true # Temporary option to limit automatic upload (Default: true). + gitlab_rails['packages_object_store_proxy_download'] = false # Passthrough all downloads via GitLab instead of using Redirects to Object Storage. + gitlab_rails['packages_object_store_connection'] = { + ## + ## If the provider is AWS S3, uncomment the following + ## + #'provider' => 'AWS', + #'region' => 'eu-west-1', + #'aws_access_key_id' => 'AWS_ACCESS_KEY_ID', + #'aws_secret_access_key' => 'AWS_SECRET_ACCESS_KEY', + ## + ## If the provider is other than AWS (an S3-compatible one), uncomment the following + ## + #'host' => 's3.amazonaws.com', + #'aws_signature_version' => 4 # For creation of signed URLs. Set to 2 if provider does not support v4. + #'endpoint' => 'https://s3.amazonaws.com' # Useful for S3-compliant services such as DigitalOcean Spaces. + #'path_style' => false # If true, use 'host/bucket_name/object' instead of 'bucket_name.host/object'. + } + ``` 1. Save the file and [reconfigure GitLab][] for the changes to take effect. @@ -120,35 +122,35 @@ upload packages: 1. Edit the `packages` section in `config/gitlab.yml` (uncomment where necessary): - ```yaml - packages: - enabled: true - ## - ## The location where build packages are stored (default: shared/packages). - ## - #storage_path: shared/packages - object_store: - enabled: false - remote_directory: packages # The bucket name. - #direct_upload: false # Use Object Storage directly for uploads instead of background uploads if enabled (Default: false). - #background_upload: true # Temporary option to limit automatic upload (Default: true). - #proxy_download: false # Passthrough all downloads via GitLab instead of using Redirects to Object Storage. - connection: - ## - ## If the provider is AWS S3, uncomment the following - ## - #provider: AWS - #region: us-east-1 - #aws_access_key_id: AWS_ACCESS_KEY_ID - #aws_secret_access_key: AWS_SECRET_ACCESS_KEY - ## - ## If the provider is other than AWS (an S3-compatible one), uncomment the following - ## - #host: 's3.amazonaws.com' # default: s3.amazonaws.com. - #aws_signature_version: 4 # For creation of signed URLs. Set to 2 if provider does not support v4. - #endpoint: 'https://s3.amazonaws.com' # Useful for S3-compliant services such as DigitalOcean Spaces. - #path_style: false # If true, use 'host/bucket_name/object' instead of 'bucket_name.host/object'. - ``` + ```yaml + packages: + enabled: true + ## + ## The location where build packages are stored (default: shared/packages). + ## + #storage_path: shared/packages + object_store: + enabled: false + remote_directory: packages # The bucket name. + #direct_upload: false # Use Object Storage directly for uploads instead of background uploads if enabled (Default: false). + #background_upload: true # Temporary option to limit automatic upload (Default: true). + #proxy_download: false # Passthrough all downloads via GitLab instead of using Redirects to Object Storage. + connection: + ## + ## If the provider is AWS S3, uncomment the following + ## + #provider: AWS + #region: us-east-1 + #aws_access_key_id: AWS_ACCESS_KEY_ID + #aws_secret_access_key: AWS_SECRET_ACCESS_KEY + ## + ## If the provider is other than AWS (an S3-compatible one), uncomment the following + ## + #host: 's3.amazonaws.com' # default: s3.amazonaws.com. + #aws_signature_version: 4 # For creation of signed URLs. Set to 2 if provider does not support v4. + #endpoint: 'https://s3.amazonaws.com' # Useful for S3-compliant services such as DigitalOcean Spaces. + #path_style: false # If true, use 'host/bucket_name/object' instead of 'bucket_name.host/object'. + ``` 1. [Restart GitLab] for the changes to take effect. diff --git a/doc/administration/reply_by_email_postfix_setup.md b/doc/administration/reply_by_email_postfix_setup.md index d57fc67c83e..406f7e8a034 100644 --- a/doc/administration/reply_by_email_postfix_setup.md +++ b/doc/administration/reply_by_email_postfix_setup.md @@ -14,109 +14,109 @@ The instructions make the assumption that you will be using the email address `i 1. Install the `postfix` package if it is not installed already: - ```sh - sudo apt-get install postfix - ``` + ```sh + sudo apt-get install postfix + ``` - When asked about the environment, select 'Internet Site'. When asked to confirm the hostname, make sure it matches `gitlab.example.com`. + When asked about the environment, select 'Internet Site'. When asked to confirm the hostname, make sure it matches gitlab.example.com`. 1. Install the `mailutils` package. - ```sh - sudo apt-get install mailutils - ``` + ```sh + sudo apt-get install mailutils + ``` ## Create user 1. Create a user for incoming email. - ```sh - sudo useradd -m -s /bin/bash incoming - ``` + ```sh + sudo useradd -m -s /bin/bash incoming + ``` 1. Set a password for this user. - ```sh - sudo passwd incoming - ``` + ```sh + sudo passwd incoming + ``` - Be sure not to forget this, you'll need it later. + Be sure not to forget this, you'll need it later. ## Test the out-of-the-box setup 1. Connect to the local SMTP server: - ```sh - telnet localhost 25 - ``` + ```sh + telnet localhost 25 + ``` - You should see a prompt like this: + You should see a prompt like this: - ```sh - Trying 127.0.0.1... - Connected to localhost. - Escape character is '^]'. - 220 gitlab.example.com ESMTP Postfix (Ubuntu) - ``` + ```sh + Trying 127.0.0.1... + Connected to localhost. + Escape character is '^]'. + 220 gitlab.example.com ESMTP Postfix (Ubuntu) + ``` - If you get a `Connection refused` error instead, verify that `postfix` is running: + If you get a `Connection refused` error instead, verify that `postfix` is running: - ```sh - sudo postfix status - ``` + ```sh + sudo postfix status + ``` - If it is not, start it: + If it is not, start it: - ```sh - sudo postfix start - ``` + ```sh + sudo postfix start + ``` 1. Send the new `incoming` user a dummy email to test SMTP, by entering the following into the SMTP prompt: - ``` - ehlo localhost - mail from: root@localhost - rcpt to: incoming@localhost - data - Subject: Re: Some issue + ``` + ehlo localhost + mail from: root@localhost + rcpt to: incoming@localhost + data + Subject: Re: Some issue - Sounds good! - . - quit - ``` + Sounds good! + . + quit + ``` - _**Note:** The `.` is a literal period on its own line._ + _**Note:** The `.` is a literal period on its own line._ - _**Note:** If you receive an error after entering `rcpt to: incoming@localhost` - then your Postfix `my_network` configuration is not correct. The error will - say 'Temporary lookup failure'. See - [Configure Postfix to receive email from the Internet](#configure-postfix-to-receive-email-from-the-internet)._ + _**Note:** If you receive an error after entering `rcpt to: incoming@localhost` + then your Postfix `my_network` configuration is not correct. The error will + say 'Temporary lookup failure'. See + [Configure Postfix to receive email from the Internet](#configure-postfix-to-receive-email-from-the-internet)._ 1. Check if the `incoming` user received the email: - ```sh - su - incoming - mail - ``` + ```sh + su - incoming + mail + ``` - You should see output like this: + You should see output like this: - ``` - "/var/mail/incoming": 1 message 1 unread - >U 1 root@localhost 59/2842 Re: Some issue - ``` + ``` + "/var/mail/incoming": 1 message 1 unread + >U 1 root@localhost 59/2842 Re: Some issue + ``` - Quit the mail app: + Quit the mail app: - ```sh - q - ``` + ```sh + q + ``` 1. Log out of the `incoming` account and go back to being `root`: - ```sh - logout - ``` + ```sh + logout + ``` ## Configure Postfix to use Maildir-style mailboxes @@ -124,208 +124,212 @@ Courier, which we will install later to add IMAP authentication, requires mailbo 1. Configure Postfix to use Maildir-style mailboxes: - ```sh - sudo postconf -e "home_mailbox = Maildir/" - ``` + ```sh + sudo postconf -e "home_mailbox = Maildir/" + ``` 1. Restart Postfix: - ```sh - sudo /etc/init.d/postfix restart - ``` + ```sh + sudo /etc/init.d/postfix restart + ``` 1. Test the new setup: - 1. Follow steps 1 and 2 of _[Test the out-of-the-box setup](#test-the-out-of-the-box-setup)_. - 1. Check if the `incoming` user received the email: + 1. Follow steps 1 and 2 of _[Test the out-of-the-box setup](#test-the-out-of-the-box-setup)_. + 1. Check if the `incoming` user received the email: - ```sh - su - incoming - MAIL=/home/incoming/Maildir - mail - ``` + ```sh + su - incoming + MAIL=/home/incoming/Maildir + mail + ``` - You should see output like this: + You should see output like this: - ``` - "/home/incoming/Maildir": 1 message 1 unread - >U 1 root@localhost 59/2842 Re: Some issue - ``` + ``` + "/home/incoming/Maildir": 1 message 1 unread + >U 1 root@localhost 59/2842 Re: Some issue + ``` - Quit the mail app: + Quit the mail app: - ```sh - q - ``` + ```sh + q + ``` - _**Note:** If `mail` returns an error `Maildir: Is a directory` then your - version of `mail` doesn't support Maildir style mailboxes. Install - `heirloom-mailx` by running `sudo apt-get install heirloom-mailx`. Then, - try the above steps again, substituting `heirloom-mailx` for the `mail` - command._ + _**Note:** If `mail` returns an error `Maildir: Is a directory` then your + version of `mail` doesn't support Maildir style mailboxes. Install + `heirloom-mailx` by running `sudo apt-get install heirloom-mailx`. Then, + try the above steps again, substituting `heirloom-mailx` for the `mail` + command._ 1. Log out of the `incoming` account and go back to being `root`: - ```sh - logout - ``` + ```sh + logout + ``` ## Install the Courier IMAP server 1. Install the `courier-imap` package: - ```sh - sudo apt-get install courier-imap - ``` + ```sh + sudo apt-get install courier-imap + ``` - And start `imapd`: - ```sh - imapd start - ``` + And start `imapd`: + + ```sh + imapd start + ``` 1. The courier-authdaemon isn't started after installation. Without it, imap authentication will fail: - ```sh - sudo service courier-authdaemon start - ``` - You can also configure courier-authdaemon to start on boot: - ```sh - sudo systemctl enable courier-authdaemon - ``` + + ```sh + sudo service courier-authdaemon start + ``` + + You can also configure courier-authdaemon to start on boot: + + ```sh + sudo systemctl enable courier-authdaemon + ``` ## Configure Postfix to receive email from the internet 1. Let Postfix know about the domains that it should consider local: - ```sh - sudo postconf -e "mydestination = gitlab.example.com, localhost.localdomain, localhost" - ``` + ```sh + sudo postconf -e "mydestination = gitlab.example.com, localhost.localdomain, localhost" + ``` 1. Let Postfix know about the IPs that it should consider part of the LAN: - We'll assume `192.168.1.0/24` is your local LAN. You can safely skip this step if you don't have other machines in the same local network. + We'll assume `192.168.1.0/24` is your local LAN. You can safely skip this step if you don't have other machines in the same local network. - ```sh - sudo postconf -e "mynetworks = 127.0.0.0/8, 192.168.1.0/24" - ``` + ```sh + sudo postconf -e "mynetworks = 127.0.0.0/8, 192.168.1.0/24" + ``` 1. Configure Postfix to receive mail on all interfaces, which includes the internet: - ```sh - sudo postconf -e "inet_interfaces = all" - ``` + ```sh + sudo postconf -e "inet_interfaces = all" + ``` 1. Configure Postfix to use the `+` delimiter for sub-addressing: - ```sh - sudo postconf -e "recipient_delimiter = +" - ``` + ```sh + sudo postconf -e "recipient_delimiter = +" + ``` 1. Restart Postfix: - ```sh - sudo service postfix restart - ``` + ```sh + sudo service postfix restart + ``` ## Test the final setup 1. Test SMTP under the new setup: - 1. Connect to the SMTP server: + 1. Connect to the SMTP server: - ```sh - telnet gitlab.example.com 25 - ``` + ```sh + telnet gitlab.example.com 25 + ``` - You should see a prompt like this: + You should see a prompt like this: - ```sh - Trying 123.123.123.123... - Connected to gitlab.example.com. - Escape character is '^]'. - 220 gitlab.example.com ESMTP Postfix (Ubuntu) - ``` + ```sh + Trying 123.123.123.123... + Connected to gitlab.example.com. + Escape character is '^]'. + 220 gitlab.example.com ESMTP Postfix (Ubuntu) + ``` - If you get a `Connection refused` error instead, make sure your firewall is set up to allow inbound traffic on port 25. + If you get a `Connection refused` error instead, make sure your firewall is set up to allow inbound traffic on port 25. - 1. Send the `incoming` user a dummy email to test SMTP, by entering the following into the SMTP prompt: + 1. Send the `incoming` user a dummy email to test SMTP, by entering the following into the SMTP prompt: - ``` - ehlo gitlab.example.com - mail from: root@gitlab.example.com - rcpt to: incoming@gitlab.example.com - data - Subject: Re: Some issue + ``` + ehlo gitlab.example.com + mail from: root@gitlab.example.com + rcpt to: incoming@gitlab.example.com + data + Subject: Re: Some issue - Sounds good! - . - quit - ``` + Sounds good! + . + quit + ``` - (Note: The `.` is a literal period on its own line) + (Note: The `.` is a literal period on its own line) - 1. Check if the `incoming` user received the email: + 1. Check if the `incoming` user received the email: - ```sh - su - incoming - MAIL=/home/incoming/Maildir - mail - ``` + ```sh + su - incoming + MAIL=/home/incoming/Maildir + mail + ``` - You should see output like this: + You should see output like this: - ``` - "/home/incoming/Maildir": 1 message 1 unread - >U 1 root@gitlab.example.com 59/2842 Re: Some issue - ``` + ``` + "/home/incoming/Maildir": 1 message 1 unread + >U 1 root@gitlab.example.com 59/2842 Re: Some issue + ``` - Quit the mail app: + Quit the mail app: - ```sh - q - ``` + ```sh + q + ``` - 1. Log out of the `incoming` account and go back to being `root`: + 1. Log out of the `incoming` account and go back to being `root`: - ```sh - logout - ``` + ```sh + logout + ``` 1. Test IMAP under the new setup: - 1. Connect to the IMAP server: + 1. Connect to the IMAP server: - ```sh - telnet gitlab.example.com 143 - ``` + ```sh + telnet gitlab.example.com 143 + ``` - You should see a prompt like this: + You should see a prompt like this: - ```sh - Trying 123.123.123.123... - Connected to mail.example.gitlab.com. - Escape character is '^]'. - - OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION] Courier-IMAP ready. Copyright 1998-2011 Double Precision, Inc. See COPYING for distribution information. - ``` + ```sh + Trying 123.123.123.123... + Connected to mail.example.gitlab.com. + Escape character is '^]'. + - OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION] Courier-IMAP ready. Copyright 1998-2011 Double Precision, Inc. See COPYING for distribution information. + ``` - 1. Sign in as the `incoming` user to test IMAP, by entering the following into the IMAP prompt: + 1. Sign in as the `incoming` user to test IMAP, by entering the following into the IMAP prompt: - ``` - a login incoming PASSWORD - ``` + ``` + a login incoming PASSWORD + ``` - Replace PASSWORD with the password you set on the `incoming` user earlier. + Replace PASSWORD with the password you set on the `incoming` user earlier. - You should see output like this: + You should see output like this: - ``` - a OK LOGIN Ok. - ``` + ``` + a OK LOGIN Ok. + ``` - 1. Disconnect from the IMAP server: + 1. Disconnect from the IMAP server: - ```sh - a logout - ``` + ```sh + a logout + ``` ## Done! diff --git a/doc/administration/repository_storage_paths.md b/doc/administration/repository_storage_paths.md index 4aafc06cfdc..3de860f9240 100644 --- a/doc/administration/repository_storage_paths.md +++ b/doc/administration/repository_storage_paths.md @@ -68,18 +68,18 @@ NOTE: **Note:** This example uses NFS and CephFS. We do not recommend using EFS 1. Edit `gitlab.yml` and add the storage paths: - ```yaml - repositories: - # Paths where repositories can be stored. Give the canonicalized absolute pathname. - # NOTE: REPOS PATHS MUST NOT CONTAIN ANY SYMLINK!!! - storages: # You must have at least a 'default' storage path. - default: - path: /home/git/repositories - nfs: - path: /mnt/nfs/repositories - cephfs: - path: /mnt/cephfs/repositories - ``` + ```yaml + repositories: + # Paths where repositories can be stored. Give the canonicalized absolute pathname. + # NOTE: REPOS PATHS MUST NOT CONTAIN ANY SYMLINK!!! + storages: # You must have at least a 'default' storage path. + default: + path: /home/git/repositories + nfs: + path: /mnt/nfs/repositories + cephfs: + path: /mnt/cephfs/repositories + ``` 1. [Restart GitLab][restart-gitlab] for the changes to take effect. @@ -97,16 +97,16 @@ working, you can remove the `repos_path` line. 1. Edit `/etc/gitlab/gitlab.rb` by appending the rest of the paths to the default one: - ```ruby - git_data_dirs({ - "default" => { "path" => "/var/opt/gitlab/git-data" }, - "nfs" => { "path" => "/mnt/nfs/git-data" }, - "cephfs" => { "path" => "/mnt/cephfs/git-data" } - }) - ``` + ```ruby + git_data_dirs({ + "default" => { "path" => "/var/opt/gitlab/git-data" }, + "nfs" => { "path" => "/mnt/nfs/git-data" }, + "cephfs" => { "path" => "/mnt/cephfs/git-data" } + }) + ``` - Note that Omnibus stores the repositories in a `repositories` subdirectory - of the `git-data` directory. + Note that Omnibus stores the repositories in a `repositories` subdirectory + of the `git-data` directory. ## Choose where new project repositories will be stored diff --git a/doc/administration/repository_storage_types.md b/doc/administration/repository_storage_types.md index 834b41b3a2c..9dea6074a3f 100644 --- a/doc/administration/repository_storage_types.md +++ b/doc/administration/repository_storage_types.md @@ -6,19 +6,19 @@ Two different storage layouts can be used to store the repositories on disk and their characteristics. GitLab can be configured to use one or multiple repository shard locations -that can be: +that can be: - Mounted to the local disk - Exposed as an NFS shared volume - Acessed via [gitaly] on its own machine. In GitLab, this is configured in `/etc/gitlab/gitlab.rb` by the `git_data_dirs({})` -configuration hash. The storage layouts discussed here will apply to any shard +configuration hash. The storage layouts discussed here will apply to any shard defined in it. The `default` repository shard that is available in any installations that haven't customized it, points to the local folder: `/var/opt/gitlab/git-data`. -Anything discussed below is expected to be part of that folder. +Anything discussed below is expected to be part of that folder. ## Legacy Storage @@ -80,25 +80,20 @@ by another folder with the next 2 characters. They are both stored in a special ### Hashed object pools -CAUTION: **Beta:** -Hashed objects pools are considered beta, and are not ready for production use. -Follow [gitaly#1548](https://gitlab.com/gitlab-org/gitaly/issues/1548) for -updates. +> [Introduced](https://gitlab.com/gitlab-org/gitaly/issues/1606) in GitLab 12.1. -For deduplication of public forks and their parent repository, objects are pooled -in an object pool. These object pools are a third repository where shared objects -are stored. +Forks of public projects are deduplicated by creating a third repository, the object pool, containing the objects from the source project. Using `objects/info/alternates`, the source project and forks use the object pool for shared objects. Objects are moved from the source project to the object pool when housekeeping is run on the source project. ```ruby # object pool paths "@pools/#{hash[0..1]}/#{hash[2..3]}/#{hash}.git" ``` -The object pool feature is behind the `object_pools` feature flag, and can be -enabled for individual projects by executing -`Feature.enable(:object_pools, Project.find(<id>))`. Note that the project has to -be on hashed storage, should not be a fork itself, and hashed storage should be -enabled for all new projects. +Object pools can be disabled using the `object_pools` feature flag, and can be +disabled for individual projects by executing +`Feature.disable(:object_pools, Project.find(<id>))`. Disabling object pools +will not change existing deduplicated forks, but will prevent new forks from +being deduplicated. DANGER: **Danger:** Do not run `git prune` or `git gc` in pool repositories! This can @@ -108,7 +103,7 @@ question. ### How to migrate to Hashed Storage To start a migration, enable Hashed Storage for new projects: - + 1. Go to **Admin > Settings > Repository** and expand the **Repository Storage** section. 2. Select the **Use hashed storage paths for newly created and renamed projects** checkbox. @@ -129,7 +124,7 @@ an Omnibus Gitlab installation: sudo gitlab-rake gitlab:storage:migrate_to_hashed ID_FROM=50 ID_TO=100 ``` -Check the [documentation][rake/migrate-to-hashed] for additional information and instructions for +Check the [documentation][rake/migrate-to-hashed] for additional information and instructions for source-based installation. #### Rollback @@ -140,12 +135,12 @@ projects: 1. Go to **Admin > Settings > Repository** and expand the **Repository Storage** section. 2. Uncheck the **Use hashed storage paths for newly created and renamed projects** checkbox. -To schedule a complete rollback, see the +To schedule a complete rollback, see the [rake task documentation for storage rollback](raketasks/storage.md#rollback-from-hashed-storage-to-legacy-storage) for instructions. The rollback task also supports specifying a range of Project IDs. Here is an example of limiting the rollout to Project IDs 50 to 100, in an Omnibus Gitlab installation: - + ```bash sudo gitlab-rake gitlab:storage:rollback_to_legacy ID_FROM=50 ID_TO=100 ``` |
