diff options
Diffstat (limited to 'doc/administration/high_availability')
| -rw-r--r-- | doc/administration/high_availability/README.md | 18 | ||||
| -rw-r--r-- | doc/administration/high_availability/database.md | 4 | ||||
| -rw-r--r-- | doc/administration/high_availability/load_balancer.md | 5 | ||||
| -rw-r--r-- | doc/administration/high_availability/nfs.md | 36 | ||||
| -rw-r--r-- | doc/administration/high_availability/redis.md | 19 |
5 files changed, 60 insertions, 22 deletions
diff --git a/doc/administration/high_availability/README.md b/doc/administration/high_availability/README.md index d5a5aef7ec0..4d3be0ab8f6 100644 --- a/doc/administration/high_availability/README.md +++ b/doc/administration/high_availability/README.md @@ -5,6 +5,20 @@ The solution you choose will be based on the level of scalability and availability you require. The easiest solutions are scalable, but not necessarily highly available. +GitLab provides a service that is usually essential to most organizations: it +enables people to collaborate on code in a timely fashion. Any downtime should +therefore be short and planned. Luckily, GitLab provides a solid setup even on +a single server without special measures. Due to the distributed nature +of Git, developers can still commit code locally even when GitLab is not +available. However, some GitLab features such as the issue tracker and +Continuous Integration are not available when GitLab is down. + +**Keep in mind that all Highly Available solutions come with a trade-off between +cost/complexity and uptime**. The more uptime you want, the more complex the +solution. And the more complex the solution, the more work is involved in +setting up and maintaining it. High availability is not free and every HA +solution should balance the costs against the benefits. + ## Architecture There are two kinds of setups: @@ -37,6 +51,10 @@ Block Device) to keep all data in sync. DRBD requires a low latency link to remain in sync. It is not advisable to attempt to run DRBD between data centers or in different cloud availability zones. +> **Note:** GitLab recommends against choosing this HA method because of the + complexity of managing DRBD and crafting automatic failover. This is + *compatible* with GitLab, but not officially *supported*. + Components/Servers Required: 2 servers/virtual machines (one active/one passive)  diff --git a/doc/administration/high_availability/database.md b/doc/administration/high_availability/database.md index c22b1af8bfb..da9687aa849 100644 --- a/doc/administration/high_availability/database.md +++ b/doc/administration/high_availability/database.md @@ -27,7 +27,7 @@ If you use a cloud-managed service, or provide your own PostgreSQL: steps on the download page. 1. Create/edit `/etc/gitlab/gitlab.rb` and use the following configuration. Be sure to change the `external_url` to match your eventual GitLab front-end - URL. + URL. If there is a directive listed below that you do not see in the configuration, be sure to add it. ```ruby external_url 'https://gitlab.example.com' @@ -39,6 +39,8 @@ If you use a cloud-managed service, or provide your own PostgreSQL: unicorn['enable'] = false sidekiq['enable'] = false redis['enable'] = false + prometheus['enable'] = false + gitaly['enable'] = false gitlab_workhorse['enable'] = false mailroom['enable'] = false diff --git a/doc/administration/high_availability/load_balancer.md b/doc/administration/high_availability/load_balancer.md index 3245988fc14..359de0efadb 100644 --- a/doc/administration/high_availability/load_balancer.md +++ b/doc/administration/high_availability/load_balancer.md @@ -13,12 +13,13 @@ you need to use with GitLab. | LB Port | Backend Port | Protocol | | ------- | ------------ | --------------- | | 80 | 80 | HTTP [^1] | -| 443 | 443 | HTTPS [^1] [^2] | +| 443 | 443 | TCP or HTTPS [^1] [^2] | | 22 | 22 | TCP | ## GitLab Pages Ports -If you're using GitLab Pages you will need some additional port configurations. +If you're using GitLab Pages with custom domain support you will need some +additional port configurations. GitLab Pages requires a separate virtual IP address. Configure DNS to point the `pages_external_url` from `/etc/gitlab/gitlab.rb` at the new virtual IP address. See the [GitLab Pages documentation][gitlab-pages] for more information. diff --git a/doc/administration/high_availability/nfs.md b/doc/administration/high_availability/nfs.md index bf1aa6b9ac5..d8e76d6ab94 100644 --- a/doc/administration/high_availability/nfs.md +++ b/doc/administration/high_availability/nfs.md @@ -7,21 +7,39 @@ supported natively in NFS version 4. NFSv3 also supports locking as long as Linux Kernel 2.6.5+ is used. We recommend using version 4 and do not specifically test NFSv3. -**no_root_squash**: NFS normally changes the `root` user to `nobody`. This is -a good security measure when NFS shares will be accessed by many different -users. However, in this case only GitLab will use the NFS share so it -is safe. GitLab requires the `no_root_squash` setting because we need to -manage file permissions automatically. Without the setting you will receive -errors when the Omnibus package tries to alter permissions. Note that GitLab -and other bundled components do **not** run as `root` but as non-privileged -users. The requirement for `no_root_squash` is to allow the Omnibus package to -set ownership and permissions on files, as needed. +## AWS Elastic File System + +GitLab does not recommend using AWS Elastic File System (EFS). + +Customers and users have reported that AWS EFS does not perform well for GitLab's +use-case. There are several issues that can cause problems. For these reasons +GitLab does not recommend using EFS with GitLab. + +- EFS bases allowed IOPS on volume size. The larger the volume, the more IOPS + are allocated. For smaller volumes, users may experience decent performance + for a period of time due to 'Burst Credits'. Over a period of weeks to months + credits may run out and performance will bottom out. +- For larger volumes, allocated IOPS may not be the problem. Workloads where + many small files are written in a serialized manner are not well-suited for EFS. + EBS with an NFS server on top will perform much better. + +For more details on another person's experience with EFS, see +[Amazon's Elastic File System: Burst Credits](https://www.rawkode.io/2017/04/amazons-elastic-file-system-burst-credits/) ### Recommended options When you define your NFS exports, we recommend you also add the following options: +- `no_root_squash` - NFS normally changes the `root` user to `nobody`. This is + a good security measure when NFS shares will be accessed by many different + users. However, in this case only GitLab will use the NFS share so it + is safe. GitLab recommends the `no_root_squash` setting because we need to + manage file permissions automatically. Without the setting you may receive + errors when the Omnibus package tries to alter permissions. Note that GitLab + and other bundled components do **not** run as `root` but as non-privileged + users. The recommendation for `no_root_squash` is to allow the Omnibus package + to set ownership and permissions on files, as needed. - `sync` - Force synchronous behavior. Default is asynchronous and under certain circumstances it could lead to data loss if a failure occurs before data has synced. diff --git a/doc/administration/high_availability/redis.md b/doc/administration/high_availability/redis.md index b4e7bf21e35..0e92f7c5a34 100644 --- a/doc/administration/high_availability/redis.md +++ b/doc/administration/high_availability/redis.md @@ -42,10 +42,10 @@ instances run in different machines. If you fail to provision the machines in that specific way, any issue with the shared environment can bring your entire setup down. -It is OK to run a Sentinel along with a master or slave Redis instance. -No more than one Sentinel in the same machine though. +It is OK to run a Sentinel alongside of a master or slave Redis instance. +There should be no more than one Sentinel on the same machine though. -You also need to take in consideration the underlying network topology, +You also need to take into consideration the underlying network topology, making sure you have redundant connectivity between Redis / Sentinel and GitLab instances, otherwise the networks will become a single point of failure. @@ -113,7 +113,7 @@ the Omnibus GitLab package in `5` **independent** machines, both with ### Redis setup overview You must have at least `3` Redis servers: `1` Master, `2` Slaves, and they -need to be each in a independent machine (see explanation above). +need to each be on independent machines (see explanation above). You can have additional Redis nodes, that will help survive a situation where more nodes goes down. Whenever there is only `2` nodes online, a failover @@ -232,7 +232,7 @@ Pick the one that suits your needs. This is the section where we install and setup the new Redis instances. >**Notes:** -- We assume that you install GitLab and all HA components from scratch. If you +- We assume that you have installed GitLab and all HA components from scratch. If you already have it installed and running, read how to [switch from a single-machine installation to Redis HA](#switching-from-an-existing-single-machine-installation-to-redis-ha). - Redis nodes (both master and slaves) will need the same password defined in @@ -245,10 +245,9 @@ The prerequisites for a HA Redis setup are the following: 1. Provision the minimum required number of instances as specified in the [recommended setup](#recommended-setup) section. -1. **Do NOT** install Redis or Redis Sentinel in the same machines your - GitLab application is running on. You can however opt in to install Redis - and Sentinel in the same machine (each in independent ones is recommended - though). +1. We **Do not** recommend installing Redis or Redis Sentinel in the same machines your + GitLab application is running on as this weakens your HA configuration. You can however opt in to install Redis + and Sentinel in the same machine. 1. All Redis nodes must be able to talk to each other and accept incoming connections over Redis (`6379`) and Sentinel (`26379`) ports (unless you change the default ones). @@ -492,7 +491,7 @@ which ideally should not have Redis or Sentinels on it for a HA setup. redis['master_name'] = 'gitlab-redis' ## The same password for Redis authentication you set up for the master node. - redis['password'] = 'redis-password-goes-here' + redis['master_password'] = 'redis-password-goes-here' ## A list of sentinels with `host` and `port` gitlab_rails['redis_sentinels'] = [ |