diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2020-05-20 14:34:42 +0000 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2020-05-20 14:34:42 +0000 |
commit | 9f46488805e86b1bc341ea1620b866016c2ce5ed (patch) | |
tree | f9748c7e287041e37d6da49e0a29c9511dc34768 /doc/install | |
parent | dfc92d081ea0332d69c8aca2f0e745cb48ae5e6d (diff) | |
download | gitlab-ce-9f46488805e86b1bc341ea1620b866016c2ce5ed.tar.gz |
Add latest changes from gitlab-org/gitlab@13-0-stable-ee
Diffstat (limited to 'doc/install')
-rw-r--r-- | doc/install/README.md | 12 | ||||
-rw-r--r-- | doc/install/aws/index.md | 234 | ||||
-rw-r--r-- | doc/install/azure/index.md | 47 | ||||
-rw-r--r-- | doc/install/google_cloud_platform/index.md | 19 | ||||
-rw-r--r-- | doc/install/installation.md | 39 | ||||
-rw-r--r-- | doc/install/openshift_and_gitlab/index.md | 27 | ||||
-rw-r--r-- | doc/install/relative_url.md | 9 | ||||
-rw-r--r-- | doc/install/requirements.md | 81 |
8 files changed, 264 insertions, 204 deletions
diff --git a/doc/install/README.md b/doc/install/README.md index 83910b0a281..f1ef368685e 100644 --- a/doc/install/README.md +++ b/doc/install/README.md @@ -14,15 +14,15 @@ and cost of hosting. There are many ways you can install GitLab depending on your platform: 1. **Omnibus GitLab**: The official deb/rpm packages that contain a bundle of GitLab - and the various components it depends on like PostgreSQL, Redis, Sidekiq, etc. + and the various components it depends on, like PostgreSQL, Redis, Sidekiq, etc. 1. **GitLab Helm chart**: The cloud native Helm chart for installing GitLab and all its components on Kubernetes. 1. **Docker**: The Omnibus GitLab packages dockerized. 1. **Source**: Install GitLab and all its components from scratch. TIP: **If in doubt, choose Omnibus:** -The Omnibus GitLab packages are mature, scalable, support -[high availability](../administration/availability/index.md) and are used +The Omnibus GitLab packages are mature, +[scalable](../administration/reference_architectures/index.md) and are used today on GitLab.com. The Helm charts are recommended for those who are familiar with Kubernetes. @@ -36,7 +36,7 @@ The Omnibus GitLab package uses our official deb/rpm repositories. This is recommended for most users. If you need additional flexibility and resilience, we recommend deploying -GitLab as described in our [High Availability documentation](../administration/availability/index.md). +GitLab as described in our [reference architecture documentation](../administration/reference_architectures/index.md). [**> Install GitLab using the Omnibus GitLab package.**](https://about.gitlab.com/install/) @@ -66,8 +66,8 @@ GitLab maintains a set of official Docker images based on the Omnibus GitLab pac ## Installing GitLab from source -If the GitLab Omnibus package is not available in your distribution, you can -install GitLab from source: Useful for unsupported systems like *BSD. For an +If the Omnibus GitLab package is not available in your distribution, you can +install GitLab from source: Useful for unsupported systems like \*BSD. For an overview of the directory structure, read the [structure documentation](structure.md). [**> Install GitLab from source.**](installation.md) diff --git a/doc/install/aws/index.md b/doc/install/aws/index.md index 48de5e274b0..41f8d7babac 100644 --- a/doc/install/aws/index.md +++ b/doc/install/aws/index.md @@ -2,9 +2,9 @@ type: howto --- -# Installing GitLab HA on Amazon Web Services (AWS) +# Installing GitLab on Amazon Web Services (AWS) -This page offers a walkthrough of a common HA (Highly Available) configuration +This page offers a walkthrough of a common configuration for GitLab on AWS. You should customize it to accommodate your needs. NOTE: **Note** @@ -13,11 +13,10 @@ For organizations with 300 users or less, the recommended AWS installation metho ## Introduction GitLab on AWS can leverage many of the services that are already -configurable with GitLab High Availability (HA). These services offer a great deal of -flexibility and can be adapted to the needs of most companies, while enabling the -automation of both vertical and horizontal scaling. +configurable. These services offer a great deal of +flexibility and can be adapted to the needs of most companies. -In this guide, we'll go through a basic HA setup where we'll start by +In this guide, we'll go through a multi-node setup where we'll start by configuring our Virtual Private Cloud and subnets to later integrate services such as RDS for our database server and ElastiCache as a Redis cluster to finally manage them within an auto scaling group with custom @@ -54,26 +53,60 @@ Here's a list of the AWS services we will use, with links to pricing information [Amazon S3 pricing](https://aws.amazon.com/s3/pricing/). - **ELB**: A Classic Load Balancer will be used to route requests to the GitLab instances. See the [Amazon ELB pricing](https://aws.amazon.com/elasticloadbalancing/pricing/). -- **RDS**: An Amazon Relational Database Service using PostgreSQL will be used - to provide a High Availability database configuration. See the +- **RDS**: An Amazon Relational Database Service using PostgreSQL will be used. See the [Amazon RDS pricing](https://aws.amazon.com/rds/postgresql/pricing/). - **ElastiCache**: An in-memory cache environment will be used to provide a - High Availability Redis configuration. See the + Redis configuration. See the [Amazon ElastiCache pricing](https://aws.amazon.com/elasticache/pricing/). NOTE: **Note:** Please note that while we will be using EBS for storage, we do not recommend using EFS as it may negatively impact GitLab's performance. You can review the [relevant documentation](../../administration/high_availability/nfs.md#avoid-using-awss-elastic-file-system-efs) for more details. -## Creating an IAM EC2 instance role and profile +## Create an IAM EC2 instance role and profile + +As we'll be using [Amazon S3 object storage](#amazon-s3-object-storage), our EC2 instances need to have read, write, and list permissions for our S3 buckets. To avoid embedding AWS keys in our GitLab config, we'll make use of an [IAM Role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html) to allow our GitLab instance with this access. We'll need to create an IAM policy to attach to our IAM role: + +### Create an IAM Policy + +1. Navigate to the IAM dashboard and click on **Policies** in the left menu. +1. Click **Create policy**, select the `JSON` tab, and add a policy. We want to [follow security best practices and grant _least privilege_](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#grant-least-privilege), giving our role only the permissions needed to perform the required actions. + 1. Assuming you prefix the S3 bucket names with `gl-` as shown in the diagram, add the following policy: + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "s3:AbortMultipartUpload", + "s3:CompleteMultipartUpload", + "s3:ListBucket", + "s3:PutObject", + "s3:GetObject", + "s3:DeleteObject", + "s3:PutObjectAcl" + ], + "Resource": [ + "arn:aws:s3:::gl-*/*" + ] + } + ] +} +``` + +1. Click **Review policy**, give your policy a name (we'll use `gl-s3-policy`), and click **Create policy**. -To minimize the permissions of the user, we'll create a new [IAM](https://docs.aws.amazon.com/IAM/latest/UserGuide/introduction.html) -role with limited access: +### Create an IAM Role -1. Navigate to the IAM dashboard <https://console.aws.amazon.com/iam/home> and +1. Still on the IAM dashboard, click on **Roles** in the left menu, and click **Create role**. 1. Create a new role by selecting **AWS service > EC2**, then click **Next: Permissions**. -1. Choose **AmazonEC2FullAccess** and **AmazonS3FullAccess**, then click **Next: Review**. -1. Give the role the name `GitLabAdmin` and click **Create role**. +1. In the policy filter, search for the `gl-s3-policy` we created above, select it, and click **Tags**. +1. Add tags if needed and click **Review**. +1. Give the role a name (we'll use `GitLabS3Access`) and click **Create Role**. + +We'll use this role when we [create a launch configuration](#create-a-launch-configuration) later on. ## Configuring the network @@ -94,6 +127,8 @@ We'll now create a VPC, a virtual networking environment that you'll control: ![Create VPC](img/create_vpc.png) +1. Select the VPC, click **Actions**, click **Edit DNS resolution**, and enable DNS resolution. Hit **Save** when done. + ### Subnets Now, let's create some subnets in different Availability Zones. Make sure @@ -106,7 +141,7 @@ RDS instances as well: 1. Select **Subnets** from the left menu. 1. Click **Create subnet**. Give it a descriptive name tag based on the IP, - for example `gitlab-public-10.0.0.0`, select the VPC we created previously, + for example `gitlab-public-10.0.0.0`, select the VPC we created previously, select an availability zone (we'll use `us-west-2a`), and at the IPv4 CIDR block let's give it a 24 subnet `10.0.0.0/24`: ![Create subnet](img/create_subnet.png) @@ -120,18 +155,8 @@ RDS instances as well: | `gitlab-public-10.0.2.0` | public | `us-west-2b` | `10.0.2.0/24` | | `gitlab-private-10.0.3.0` | private | `us-west-2b` | `10.0.3.0/24` | -### Create NAT Gateways - -Instances deployed in our private subnets need to connect to the internet for updates, but should not be reachable from the public internet. To achieve this, we'll make use of [NAT Gateways](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-nat-gateway.html) deployed in each of our public subnets: - -1. Navigate to the VPC dashboard and click on **NAT Gateways** in the left menu bar. -1. Click **Create NAT Gateway** and complete the following: - 1. **Subnet**: Select `gitlab-public-10.0.0.0` from the dropdown. - 1. **Elastic IP Allocation ID**: Enter an existing Elastic IP or click **Allocate Elastic IP address** to allocate a new IP to your NAT gateway. - 1. Add tags if needed. - 1. Click **Create NAT Gateway**. - -Create a second NAT gateway but this time place it in the second public subnet, `gitlab-public-10.0.2.0`. +1. Once all the subnets are created, enable **Auto-assign IPv4** for the two public subnets: + 1. Select each public subnet in turn, click **Actions**, and click **Modify auto-assign IP settings**. Enable the option and save. ### Internet Gateway @@ -148,6 +173,19 @@ create a new one: 1. Choose `gitlab-vpc` from the list and hit **Attach**. +### Create NAT Gateways + +Instances deployed in our private subnets need to connect to the internet for updates, but should not be reachable from the public internet. To achieve this, we'll make use of [NAT Gateways](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-nat-gateway.html) deployed in each of our public subnets: + +1. Navigate to the VPC dashboard and click on **NAT Gateways** in the left menu bar. +1. Click **Create NAT Gateway** and complete the following: + 1. **Subnet**: Select `gitlab-public-10.0.0.0` from the dropdown. + 1. **Elastic IP Allocation ID**: Enter an existing Elastic IP or click **Allocate Elastic IP address** to allocate a new IP to your NAT gateway. + 1. Add tags if needed. + 1. Click **Create NAT Gateway**. + +Create a second NAT gateway but this time place it in the second public subnet, `gitlab-public-10.0.2.0`. + ### Route Tables #### Public Route Table @@ -179,13 +217,13 @@ Next, we must associate the **public** subnets to the route table: We also need to create two private route tables so that instances in each private subnet can reach the internet via the NAT gateway in the corresponding public subnet in the same availability zone. -1. Follow the same steps as above to create two private route tables. Name them `gitlab-public-a` and `gitlab-public-b` respectively. +1. Follow the same steps as above to create two private route tables. Name them `gitlab-private-a` and `gitlab-private-b` respectively. 1. Next, add a new route to each of the private route tables where the destination is `0.0.0.0/0` and the target is one of the NAT gateways we created earlier. - 1. Add the NAT gateway we created in `gitlab-public-10.0.0.0` as the target for the new route in the `gitlab-public-a` route table. - 1. Similarly, add the NAT gateway in `gitlab-public-10.0.2.0` as the target for the new route in the `gitlab-public-b`. + 1. Add the NAT gateway we created in `gitlab-public-10.0.0.0` as the target for the new route in the `gitlab-private-a` route table. + 1. Similarly, add the NAT gateway in `gitlab-public-10.0.2.0` as the target for the new route in the `gitlab-private-b`. 1. Lastly, associate each private subnet with a private route table. - 1. Associate `gitlab-private-10.0.1.0` with `gitlab-public-a`. - 1. Associate `gitlab-private-10.0.3.0` with `gitlab-public-b`. + 1. Associate `gitlab-private-10.0.1.0` with `gitlab-private-a`. + 1. Associate `gitlab-private-10.0.3.0` with `gitlab-private-b`. ## Load Balancer @@ -198,7 +236,7 @@ On the EC2 dashboard, look for Load Balancer in the left navigation bar: 1. In the **Select Subnets** section, select both public subnets from the list. 1. Click **Assign Security Groups** and select **Create a new security group**, give it a name (we'll use `gitlab-loadbalancer-sec-group`) and description, and allow both HTTP and HTTPS traffic - from anywhere (`0.0.0.0/0, ::/0`). + from anywhere (`0.0.0.0/0, ::/0`). Also allow SSH traffic from a single IP address or an IP address range in CIDR notation. 1. Click **Configure Security Settings** and select an SSL/TLS certificate from ACM or upload a certificate to IAM. 1. Click **Configure Health Check** and set up a health check for your EC2 instances. 1. For **Ping Protocol**, select HTTP. @@ -232,19 +270,9 @@ On the Route 53 dashboard, click **Hosted zones** in the left navigation bar: ## PostgreSQL with RDS For our database server we will use Amazon RDS which offers Multi AZ -for redundancy. Let's start by creating a subnet group and then we'll +for redundancy. First we'll create a security group and subnet group, then we'll create the actual RDS instance. -### RDS Subnet Group - -1. Navigate to the RDS dashboard and select **Subnet Groups** from the left menu. -1. Click on **Create DB Subnet Group**. -1. Under **Subnet group details**, enter a name (we'll use `gitlab-rds-group`), a description, and choose the `gitlab-vpc` from the VPC dropdown. -1. Under **Add subnets**, click **Add all the subnets related to this VPC** and remove the public ones, we only want the **private subnets**. In the end, you should see `10.0.1.0/24` and `10.0.3.0/24` (as we defined them in the [subnets section](#subnets)). -1. Click **Create** when ready. - - ![RDS Subnet Group](img/rds_subnet_group.png) - ### RDS Security Group We need a security group for our database that will allow inbound traffic from the instances we'll deploy in our `gitlab-loadbalancer-sec-group` later on: @@ -255,21 +283,33 @@ We need a security group for our database that will allow inbound traffic from t 1. In the **Inbound rules** section, click **Add rule** and add a **PostgreSQL** rule, and set the "Custom" source as the `gitlab-loadbalancer-sec-group` we created earlier. The default PostgreSQL port is `5432`, which we'll also use when creating our database below. 1. When done, click **Create security group**. +### RDS Subnet Group + +1. Navigate to the RDS dashboard and select **Subnet Groups** from the left menu. +1. Click on **Create DB Subnet Group**. +1. Under **Subnet group details**, enter a name (we'll use `gitlab-rds-group`), a description, and choose the `gitlab-vpc` from the VPC dropdown. +1. Under **Add subnets**, click **Add all the subnets related to this VPC** and remove the public ones, we only want the **private subnets**. In the end, you should see `10.0.1.0/24` and `10.0.3.0/24` (as we defined them in the [subnets section](#subnets)). +1. Click **Create** when ready. + + ![RDS Subnet Group](img/rds_subnet_group.png) + ### Create the database +DANGER: **Danger:** Avoid using burstable instances (t class instances) for the database as this could lead to performance issues due to CPU credits running out during sustained periods of high load. + Now, it's time to create the database: -1. Select **Databases** from the left menu and click **Create database**. +1. Navigate to the RDS dashboard, select **Databases** from the left menu, and click **Create database**. 1. Select **Standard Create** for the database creation method. 1. Select **PostgreSQL** as the database engine and select **PostgreSQL 10.9-R1** from the version dropdown menu (check the [database requirements](../../install/requirements.md#postgresql-requirements) to see if there are any updates on this for your chosen version of GitLab). 1. Since this is a production server, let's choose **Production** from the **Templates** section. 1. Under **Settings**, set a DB instance identifier, a master username, and a master password. We'll use `gitlab-db-ha`, `gitlab`, and a very secure password respectively. Make a note of these as we'll need them later. 1. For the DB instance size, select **Standard classes** and select an instance size that meets your requirements from the dropdown menu. We'll use a `db.m4.large` instance. 1. Under **Storage**, configure the following: - 1. Select **Provisioned IOPS (SSD)** from the storage type dropdown menu. Provisioned IOPS (SSD) storage is best suited for HA (though you can choose General Purpose (SSD) to reduce the costs). Read more about it at [Storage for Amazon RDS](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Storage.html). + 1. Select **Provisioned IOPS (SSD)** from the storage type dropdown menu. Provisioned IOPS (SSD) storage is best suited for this use (though you can choose General Purpose (SSD) to reduce the costs). Read more about it at [Storage for Amazon RDS](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Storage.html). 1. Allocate storage and set provisioned IOPS. We'll use the minimum values, `100` and `1000`, respectively. 1. Enable storage autoscaling (optional) and set a maximum storage threshold. -1. Under **Availability & durability**, select **Create a standby instance** to have a standby RDS instance provisioned in a different Availability Zone. Read more at [High Availability (Multi-AZ)](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.MultiAZ.html). +1. Under **Availability & durability**, select **Create a standby instance** to have a standby RDS instance provisioned in a different [Availability Zone](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.MultiAZ.html). 1. Under **Connectivity**, configure the following: 1. Select the VPC we created earlier (`gitlab-vpc`) from the **Virtual Private Cloud (VPC)** dropdown menu. 1. Expand the **Additional connectivity configuration** section and select the subnet group (`gitlab-rds-group`) we created earlier. @@ -291,15 +331,6 @@ Now that the database is created, let's move on to setting up Redis with ElastiC ElastiCache is an in-memory hosted caching solution. Redis maintains its own persistence and is used for certain types of the GitLab application. -### Redis Subnet Group - -1. Navigate to the ElastiCache dashboard from your AWS console. -1. Go to **Subnet Groups** in the left menu, and create a new subnet group. - Make sure to select our VPC and its [private subnets](#subnets). Click - **Create** when ready. - - ![ElastiCache subnet](img/ec_subnet.png) - ### Create a Redis Security Group 1. Navigate to the EC2 dashboard. @@ -309,6 +340,15 @@ persistence and is used for certain types of the GitLab application. 1. In the **Inbound rules** section, click **Add rule** and add a **Custom TCP** rule, set port `6379`, and set the "Custom" source as the `gitlab-loadbalancer-sec-group` we created earlier. 1. When done, click **Create security group**. +### Redis Subnet Group + +1. Navigate to the ElastiCache dashboard from your AWS console. +1. Go to **Subnet Groups** in the left menu, and create a new subnet group (we'll name ours `gitlab-redis-group`). + Make sure to select our VPC and its [private subnets](#subnets). Click + **Create** when ready. + + ![ElastiCache subnet](img/ec_subnet.png) + ### Create the Redis Cluster 1. Navigate back to the ElastiCache dashboard. @@ -392,7 +432,7 @@ From the EC2 dashboard: 1. Select an instance type based on your workload. Consult the [hardware requirements](../../install/requirements.md#hardware-requirements) to choose one that fits your needs (at least `c5.xlarge`, which is sufficient to accommodate 100 users). 1. Click **Configure Instance Details**: 1. In the **Network** dropdown, select `gitlab-vpc`, the VPC we created earlier. - 1. In the **Subnet** dropdown, `select gitlab-private-10.0.1.0` from the list of subnets we created earlier. + 1. In the **Subnet** dropdown, select `gitlab-private-10.0.1.0` from the list of subnets we created earlier. 1. Double check that **Auto-assign Public IP** is set to `Use subnet setting (Disable)`. 1. Click **Add Storage**. 1. The root volume is 8GiB by default and should be enough given that we won’t store any data there. @@ -405,6 +445,22 @@ From the EC2 dashboard: Connect to your GitLab instance via **Bastion Host A** using [SSH Agent Forwarding](#use-ssh-agent-forwarding). Once connected, add the following custom configuration: +#### Disable Let's Encrypt + +Since we're adding our SSL certificate at the load balancer, we do not need GitLab's built-in support for Let's Encrypt. Let's Encrypt [is enabled by default](https://docs.gitlab.com/omnibus/settings/ssl.html#lets-encrypt-integration) when using an `https` domain since GitLab 10.7, so we need to explicitly disable it: + +1. Open `/etc/gitlab/gitlab.rb` and disable it: + + ```ruby + letsencrypt['enable'] = false + ``` + +1. Save the file and reconfigure for the changes to take effect: + + ```shell + sudo gitlab-ctl reconfigure + ``` + #### Install the `pg_trgm` extension for PostgreSQL From your GitLab instance, connect to the RDS instance to verify access and to install the required `pg_trgm` extension. @@ -477,7 +533,7 @@ gitlab=# \q #### Set up Gitaly -CAUTION: **Caution:** In this architecture, having a single Gitaly server creates a single point of failure. This limitation will be removed once [Gitaly HA](https://gitlab.com/groups/gitlab-org/-/epics/842) is released. +CAUTION: **Caution:** In this architecture, having a single Gitaly server creates a single point of failure. This limitation will be removed once [Gitaly Cluster](https://gitlab.com/groups/gitlab-org/-/epics/1489) is released. Gitaly is a service that provides high-level RPC access to Git repositories. It should be enabled and configured on a separate EC2 instance in one of the @@ -499,6 +555,7 @@ Let's create an EC2 instance where we'll install Gitaly: 1. Click on **Configure Security Group** and let's **Create a new security group**. 1. Give your security group a name and description. We'll use `gitlab-gitaly-sec-group` for both. 1. Create a **Custom TCP** rule and add port `8075` to the **Port Range**. For the **Source**, select the `gitlab-loadbalancer-sec-group`. + 1. Also add an inbound rule for SSH from the `bastion-sec-group` so that we can connect using [SSH Agent Forwarding](#use-ssh-agent-forwarding) from the Bastion hosts. 1. Click **Review and launch** followed by **Launch** if you're happy with your settings. 1. Finally, acknowledge that you have access to the selected private key file or create a new one. Click **Launch Instances**. @@ -512,48 +569,51 @@ As we are terminating SSL at our [load balancer](#load-balancer), follow the ste Remember to run `sudo gitlab-ctl reconfigure` after saving the changes to the `gitlab.rb` file. -#### Disable Let's Encrypt +#### Fast lookup of authorized SSH keys -Since we're adding our SSL certificate at the load balancer, we do not need GitLab's built-in support for Let's Encrypt. Let's Encrypt [is enabled by default](https://docs.gitlab.com/omnibus/settings/ssl.html#lets-encrypt-integration) when using an `https` domain since GitLab 10.7, so we need to explicitly disable it: +The public SSH keys for users allowed to access GitLab are stored in `/var/opt/gitlab/.ssh/authorized_keys`. Typically we'd use shared storage so that all the instances are able to access this file when a user performs a Git action over SSH. Since we do not have shared storage in our setup, we'll update our configuration to authorize SSH users via indexed lookup in the GitLab database. -1. Open `/etc/gitlab/gitlab.rb` and disable it: +Follow the instructions at [Setting up fast lookup via GitLab Shell](../../administration/operations/fast_ssh_key_lookup.md#setting-up-fast-lookup-via-gitlab-shell) to switch from using the `authorized_keys` file to the database. - ```ruby - letsencrypt['enable'] = false - ``` +If you do not configure fast lookup, Git actions over SSH will result in the following error: -1. Save the file and reconfigure for the changes to take effect: +```shell +Permission denied (publickey). +fatal: Could not read from remote repository. - ```shell - sudo gitlab-ctl reconfigure - ``` +Please make sure you have the correct access rights +and the repository exists. +``` #### Configure host keys -Ordinarily we would manually copy the contents (primary and public keys) of `/etc/ssh/` on the primary application server to `/etc/ssh` on all secondary servers. This prevents false man-in-the-middle-attack alerts when accessing servers in your High Availability cluster behind a load balancer. +Ordinarily we would manually copy the contents (primary and public keys) of `/etc/ssh/` on the primary application server to `/etc/ssh` on all secondary servers. This prevents false man-in-the-middle-attack alerts when accessing servers in your cluster behind a load balancer. We'll automate this by creating static host keys as part of our custom AMI. As these host keys are also rotated every time an EC2 instance boots up, "hard coding" them into our custom AMI serves as a handy workaround. On your GitLab instance run the following: ```shell -mkdir /etc/ssh_static -cp -R /etc/ssh/* /etc/ssh_static +sudo mkdir /etc/ssh_static +sudo cp -R /etc/ssh/* /etc/ssh_static ``` In `/etc/ssh/sshd_config` update the following: ```bash - # HostKeys for protocol version 2 - HostKey /etc/ssh_static/ssh_host_rsa_key - HostKey /etc/ssh_static/ssh_host_dsa_key - HostKey /etc/ssh_static/ssh_host_ecdsa_key - HosstKey /etc/ssh_static/ssh_host_ed25519_key +# HostKeys for protocol version 2 +HostKey /etc/ssh_static/ssh_host_rsa_key +HostKey /etc/ssh_static/ssh_host_dsa_key +HostKey /etc/ssh_static/ssh_host_ecdsa_key +HostKey /etc/ssh_static/ssh_host_ed25519_key ``` #### Amazon S3 object storage -Since we're not using NFS for shared storage, we will use [Amazon S3](https://aws.amazon.com/s3/) buckets to store backups, artifacts, LFS objects, uploads, merge request diffs, container registry images, and more. Our [documentation includes configuration instructions](../../administration/object_storage.md) for each of these, and other information about using object storage with GitLab. +Since we're not using NFS for shared storage, we will use [Amazon S3](https://aws.amazon.com/s3/) buckets to store backups, artifacts, LFS objects, uploads, merge request diffs, container registry images, and more. Our documentation includes [instructions on how to configure object storage](../../administration/object_storage.md) for each of these data types, and other information about using object storage with GitLab. + +NOTE: **Note:** +Since we are using the [AWS IAM profile](#create-an-iam-role) we created earlier, be sure to omit the AWS access key and secret access key/value pairs when configuring object storage. Instead, use `'use_iam_profile' => true` in your configuration as shown in the object storage documentation linked above. Remember to run `sudo gitlab-ctl reconfigure` after saving the changes to the `gitlab.rb` file. @@ -589,7 +649,7 @@ From the EC2 dashboard: 1. Select an instance type best suited for your needs (at least a `c5.xlarge`) and click **Configure details**. 1. Enter a name for your launch configuration (we'll use `gitlab-ha-launch-config`). 1. **Do not** check **Request Spot Instance**. -1. From the **IAM Role** dropdown, pick the `GitLabAdmin` instance role we [created earlier](#creating-an-iam-ec2-instance-role-and-profile). +1. From the **IAM Role** dropdown, pick the `GitLabAdmin` instance role we [created earlier](#create-an-iam-ec2-instance-role-and-profile). 1. Leave the rest as defaults and click **Add Storage**. 1. The root volume is 8GiB by default and should be enough given that we won’t store any data there. Click **Configure Security Group**. 1. Check **Select and existing security group** and select the `gitlab-loadbalancer-sec-group` we created earlier. @@ -604,7 +664,7 @@ From the EC2 dashboard: 1. Select the `gitlab-vpc` from the **Network** dropdown. 1. Add both the private [subnets we created earlier](#subnets). 1. Expand the **Advanced Details** section and check the **Receive traffic from one or more load balancers** option. -1. From the **Classic Load Balancers** dropdown, Select the load balancer we created earlier. +1. From the **Classic Load Balancers** dropdown, select the load balancer we created earlier. 1. For **Health Check Type**, select **ELB**. 1. We'll leave our **Health Check Grace Period** as the default `300` seconds. Click **Configure scaling policies**. 1. Check **Use scaling policies to adjust the capacity of this group**. @@ -635,7 +695,7 @@ GitLab provides its own integrated monitoring solution based on Prometheus. For more information on how to set it up, visit the [GitLab Prometheus documentation](../../administration/monitoring/prometheus/index.md) -GitLab also has various [health check endpoints](../..//user/admin_area/monitoring/health_check.md) +GitLab also has various [health check endpoints](../../user/admin_area/monitoring/health_check.md) that you can ping and get reports. ## GitLab Runners @@ -648,8 +708,8 @@ Read more on configuring an ## Backup and restore -GitLab provides [a tool to backup](../../raketasks/backup_restore.md#creating-a-backup-of-the-gitlab-system) -and restore its Git data, database, attachments, LFS objects, etc. +GitLab provides [a tool to back up](../../raketasks/backup_restore.md#back-up-gitlab) +and restore its Git data, database, attachments, LFS objects, and so on. Some important things to know: @@ -675,7 +735,7 @@ For GitLab 12.1 and earlier, use `gitlab-rake gitlab:backup:create`. ### Restoring GitLab from a backup -To restore GitLab, first review the [restore documentation](../../raketasks/backup_restore.md#restore), +To restore GitLab, first review the [restore documentation](../../raketasks/backup_restore.md#restore-gitlab), and primarily the restore prerequisites. Then, follow the steps under the [Omnibus installations section](../../raketasks/backup_restore.md#restore-for-omnibus-gitlab-installations). @@ -708,7 +768,7 @@ After a few minutes, the new version should be up and running. In this guide, we went mostly through scaling and some redundancy options, your mileage may vary. -Keep in mind that all Highly Available solutions come with a trade-off between +Keep in mind that all solutions come with a trade-off between cost/complexity and uptime. The more uptime you want, the more complex the solution. And the more complex the solution, the more work is involved in setting up and maintaining it. @@ -717,8 +777,8 @@ Have a read through these other resources and feel free to [open an issue](https://gitlab.com/gitlab-org/gitlab/issues/new) to request additional material: -- [Scaling GitLab](../../administration/scaling/index.md): - GitLab supports several different types of clustering and high-availability. +- [Scaling GitLab](../../administration/reference_architectures/index.md): + GitLab supports several different types of clustering. - [Geo replication](../../administration/geo/replication/index.md): Geo is the solution for widely distributed development teams. - [Omnibus GitLab](https://docs.gitlab.com/omnibus/) - Everything you need to know diff --git a/doc/install/azure/index.md b/doc/install/azure/index.md index b782f1c82cd..fbc81da20d4 100644 --- a/doc/install/azure/index.md +++ b/doc/install/azure/index.md @@ -21,7 +21,7 @@ First, you'll need an account on Azure. There are three ways to do this: services, exploring Microsoft's cloud for free. Even after the first 30 days, you never have to pay anything unless you decide to transition to paid services with a Pay-As-You-Go Azure subscription. This is a great way to try out Azure and cloud computing, and you can - [read more in their comprehensive FAQ][Azure-Free-Account-FAQ]. + [read more in their comprehensive FAQ](https://azure.microsoft.com/en-us/free/free-account-faq/). - If you have an MSDN subscription, you can activate your Azure subscriber benefits. Your MSDN subscription gives you recurring Azure credits every month, so why not put those credits to use and try out GitLab right now? @@ -73,7 +73,7 @@ The first items we need to configure are the basic settings of the underlying vi _(read the [SSH documentation](../../ssh/README.md) to learn more about how to set up SSH public keys)_ 1. If you chose **Password** - enter the password you wish to use _(this is the password that you - will use later in this tutorial to [SSH] into the VM, so make sure it's a strong password/passphrase)_ + will use later in this tutorial to [SSH](https://en.wikipedia.org/wiki/Secure_Shell) into the VM, so make sure it's a strong password/passphrase)_ 1. Choose the appropriate `Subscription` tier for your Azure account 1. Choose an existing `Resource Group` or create a new one - e.g. **"GitLab-CE-Azure"** @@ -177,7 +177,7 @@ Click **"Save"** for the changes to take effect. domain registrar which points to the public IP address of your Azure VM. If you do this, you'll need to make sure your VM is configured to use a _static_ public IP address (i.e. not a _dynamic_ one) or you will have to reconfigure the DNS `A` record each time Azure reassigns your VM a new public IP -address. Read [IP address types and allocation methods in Azure][Azure-IP-Address-Types] to learn more. +address. Read [IP address types and allocation methods in Azure](https://docs.microsoft.com/en-us/azure/virtual-network/virtual-network-ip-addresses-overview-arm) to learn more. ## Let's open some ports @@ -216,7 +216,7 @@ ports to enable public internet access to two services in particular: public access to the instance of GitLab running on our VM. 1. **SSH** (port 22) - opening port 22 will enable our VM to respond to SSH connection requests, allowing public access (with authentication) to remote terminal sessions - _(you'll see why we need [SSH] access to our VM [later on in this tutorial](#maintaining-your-gitlab-instance))_ + _(you'll see why we need [SSH](https://en.wikipedia.org/wiki/Secure_Shell) access to our VM [later on in this tutorial](#maintaining-your-gitlab-instance))_ ### Open HTTP on Port 80 @@ -233,7 +233,7 @@ connections: ### Open SSH on Port 22 Repeat the above process, adding a second Inbound security rule to open port 22, enabling our VM to -accept [SSH] connections: +accept [SSH](https://en.wikipedia.org/wiki/Secure_Shell) connections: ![Azure - Add inbound security rules - SSH](img/azure-add-inbound-sec-rule-ssh.png) @@ -327,7 +327,7 @@ process will still be the same. To perform an update, we need to connect directly to our Azure VM instance and run some commands from the terminal. Our Azure VM is actually a server running Linux (Ubuntu), so we'll need to -connect to it using SSH ([Secure Shell][SSH]). +connect to it using SSH ([Secure Shell](https://en.wikipedia.org/wiki/Secure_Shell)). If you're running Windows, you'll need to connect using [PuTTY](https://www.putty.org) or an equivalent Windows SSH client. If you're running Linux or macOS, then you already have an SSH client installed. @@ -341,7 +341,7 @@ If you're running Linux or macOS, then you already have an SSH client installed. #### SSH from the command-line -If you're running [SSH] from the command-line (terminal), then type in the following command to +If you're running [SSH](https://en.wikipedia.org/wiki/Secure_Shell) from the command-line (terminal), then type in the following command to connect to your VM, substituting `username` and `your-azure-domain-name.com` for the correct values. Again, remember that your Azure VM domain name will be the one you @@ -356,8 +356,8 @@ Provide your password at the prompt to authenticate. #### SSH from Windows (PuTTY) -If you're using [PuTTY](https://www.putty.org) in Windows as your [SSH] client, then you might want to take a quick -read on [using PuTTY in Windows][Using-SSH-In-Putty]. +If you're using [PuTTY](https://www.putty.org) in Windows as your [SSH](https://en.wikipedia.org/wiki/Secure_Shell) client, then you might want to take a quick +read on [using PuTTY in Windows](https://mediatemple.net/community/products/dv/204404604/using-ssh-in-putty-). ### Updating GitLab @@ -412,31 +412,16 @@ Check out our other [Technical Articles](../../articles/index.md) or browse the ### Useful links -- [GitLab Community Edition][CE] -- [GitLab Enterprise Edition][EE] -- [Microsoft Azure][Azure] - - [Azure - Free Account FAQ][Azure-Free-Account-FAQ] +- [GitLab Community Edition](https://about.gitlab.com/features/) +- [GitLab Enterprise Edition](https://about.gitlab.com/features/#ee-starter) +- [Microsoft Azure](https://azure.microsoft.com/en-us/) + - [Azure - Free Account FAQ](https://azure.microsoft.com/en-us/free/free-account-faq/) - [Azure - Marketplace](https://azuremarketplace.microsoft.com/en-us/marketplace/) - - [Azure Portal][Azure-Portal] - - [Azure - Pricing Calculator][Azure-Pricing-Calculator] + - [Azure Portal](https://portal.azure.com) + - [Azure - Pricing Calculator](https://azure.microsoft.com/en-us/pricing/calculator/) - [Azure - Troubleshoot SSH Connections to an Azure Linux VM](https://docs.microsoft.com/en-us/azure/virtual-machines/troubleshooting/troubleshoot-ssh-connection) - [Azure - Properly Shutdown an Azure VM](https://build5nines.com/properly-shutdown-azure-vm-to-save-money/) -- [SSH], [PuTTY](https://www.putty.org) and [Using SSH in PuTTY][Using-SSH-In-Putty] - -[Original-Blog-Post]: https://about.gitlab.com/blog/2016/07/13/how-to-setup-a-gitlab-instance-on-microsoft-azure/ "How to Set up a GitLab Instance on Microsoft Azure" -[CE]: https://about.gitlab.com/features/ -[EE]: https://about.gitlab.com/features/#ee-starter - -[Azure-Troubleshoot-Linux-VM]: https://docs.microsoft.com/en-us/azure/virtual-machines/linux/troubleshoot-app-connection "Troubleshoot application connectivity issues on a Linux virtual machine in Azure" -[Azure-IP-Address-Types]: https://docs.microsoft.com/en-us/azure/virtual-network/virtual-network-ip-addresses-overview-arm "IP address types and allocation methods in Azure" -[Azure-How-To-Open-Ports]: https://docs.microsoft.com/en-us/azure/virtual-machines/windows/nsg-quickstart-portal "How to open ports to a virtual machine with the Azure portal" -[Azure]: https://azure.microsoft.com/en-us/ -[Azure-Free-Account-FAQ]: https://azure.microsoft.com/en-us/free/free-account-faq/ -[Azure-Portal]: https://portal.azure.com -[Azure-Pricing-Calculator]: https://azure.microsoft.com/en-us/pricing/calculator/ - -[SSH]: https://en.wikipedia.org/wiki/Secure_Shell -[Using-SSH-In-Putty]: https://mediatemple.net/community/products/dv/204404604/using-ssh-in-putty- +- [SSH](https://en.wikipedia.org/wiki/Secure_Shell), [PuTTY](https://www.putty.org) and [Using SSH in PuTTY](https://mediatemple.net/community/products/dv/204404604/using-ssh-in-putty-) <!-- ## Troubleshooting diff --git a/doc/install/google_cloud_platform/index.md b/doc/install/google_cloud_platform/index.md index 9fcf6e6420c..433eeda00b1 100644 --- a/doc/install/google_cloud_platform/index.md +++ b/doc/install/google_cloud_platform/index.md @@ -5,7 +5,7 @@ type: howto # Installing GitLab on Google Cloud Platform -This guide will help you install GitLab on a [Google Cloud Platform (GCP)][gcp] instance. +This guide will help you install GitLab on a [Google Cloud Platform (GCP)](https://cloud.google.com/) instance. NOTE: **Alternative installation method:** Google provides a whitepaper for [deploying production-ready GitLab on @@ -19,7 +19,7 @@ There are only two prerequisites in order to install GitLab on GCP: 1. You need to have a Google account. 1. You need to sign up for the GCP program. If this is your first time, Google - gives you [$300 credit for free][freetrial] to consume over a 60-day period. + gives you [$300 credit for free](https://console.cloud.google.com/freetrial) to consume over a 60-day period. Once you have performed those two steps, you can [create a VM](#creating-the-vm). @@ -37,7 +37,7 @@ To deploy GitLab on GCP you first need to create a virtual machine: ![Launch on Compute Engine](img/vm_details.png) -1. Click **Change** under Boot disk to select the size, type, and desired operating system. GitLab supports a [variety of linux operating systems][req], including Ubuntu and Debian. Click **Select** when finished. +1. Click **Change** under Boot disk to select the size, type, and desired operating system. GitLab supports a [variety of linux operating systems](../requirements.md), including Ubuntu and Debian. Click **Select** when finished. ![Deploy in progress](img/boot_disk.png) @@ -85,7 +85,7 @@ here's how you configure GitLab to be aware of the change: ![SSH button](img/vm_created.png) - In the future you might want to set up [connecting with an SSH key][ssh] + In the future you might want to set up [connecting with an SSH key](https://cloud.google.com/compute/docs/instances/connecting-to-instance) instead. 1. Edit the config file of Omnibus GitLab using your favorite text editor: @@ -114,13 +114,13 @@ here's how you configure GitLab to be aware of the change: ### Configuring HTTPS with the domain name Although not needed, it's strongly recommended to secure GitLab with a TLS -certificate. Follow the steps in the [Omnibus documentation][omni-ssl]. +certificate. Follow the steps in the [Omnibus documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https). ### Configuring the email SMTP settings You need to configure the email SMTP settings correctly otherwise GitLab will not be able to send notification emails, like comments, and password changes. -Check the [Omnibus documentation][omni-smtp] how to do so. +Check the [Omnibus documentation](https://docs.gitlab.com/omnibus/settings/smtp.html#smtp-settings) how to do so. ## Further reading @@ -132,13 +132,6 @@ Kerberos, etc. Here are some documents you might be interested in reading: - [GitLab Pages configuration](../../administration/pages/index.md) - [GitLab Container Registry configuration](../../administration/packages/container_registry.md) -[freetrial]: https://console.cloud.google.com/freetrial "GCP free trial" -[gcp]: https://cloud.google.com/ "Google Cloud Platform" -[req]: ../requirements.md "GitLab hardware and software requirements" -[ssh]: https://cloud.google.com/compute/docs/instances/connecting-to-instance "Connecting to Linux Instances" -[omni-smtp]: https://docs.gitlab.com/omnibus/settings/smtp.html#smtp-settings "Omnibus GitLab SMTP settings" -[omni-ssl]: https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https "Omnibus GitLab enable HTTPS" - <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/install/installation.md b/doc/install/installation.md index 740cd12b7fd..dd619e9e7b3 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -142,21 +142,31 @@ Starting with GitLab 12.0, Git is required to be compiled with `libpcre2`. Find out if that's the case: ```shell -ldd /usr/local/bin/git | grep pcre2 +ldd $(which git) | grep pcre2 ``` -The output should be similar to: +The output should contain `libpcre2-8.so.0`. -```plaintext -libpcre2-8.so.0 => /usr/lib/libpcre2-8.so.0 (0x00007f08461c3000) +Is the system packaged Git too old, or not compiled with pcre2? +Remove it: + +```shell +sudo apt-get remove git-core ``` -Is the system packaged Git too old, or not compiled with pcre2? Remove it and compile from source: +On Ubuntu, install Git from [its official PPA](https://git-scm.com/download/linux): ```shell -# Remove packaged Git -sudo apt-get remove git-core +# run as root! +add-apt-repository ppa:git-core/ppa +apt update +apt install git +# repeat libpcre2 check as above +``` + +On Debian, use the following compilation instructions: +```shell # Install dependencies sudo apt-get install -y libcurl4-openssl-dev libexpat1-dev gettext libz-dev libssl-dev build-essential @@ -180,7 +190,7 @@ make prefix=/usr/local all # Install into /usr/local/bin sudo make prefix=/usr/local install -# When editing config/gitlab.yml (Step 5), change the git -> bin_path to /usr/local/bin/git +# When editing config/gitlab.yml later, change the git -> bin_path to /usr/local/bin/git ``` For the [Custom Favicon](../user/admin_area/appearance.md#favicon) to work, GraphicsMagick @@ -205,7 +215,7 @@ The Ruby interpreter is required to run GitLab. **Note:** The current supported Ruby (MRI) version is 2.6.x. GitLab 12.2 dropped support for Ruby 2.5.x. -The use of Ruby version managers such as [RVM], [rbenv](https://github.com/rbenv/rbenv) or [chruby] with GitLab +The use of Ruby version managers such as [RVM](https://rvm.io/), [rbenv](https://github.com/rbenv/rbenv) or [chruby](https://github.com/postmodern/chruby) with GitLab in production, frequently leads to hard to diagnose problems. For example, GitLab Shell is called from OpenSSH, and having a version manager can prevent pushing and pulling over SSH. Version managers are not supported and we strongly @@ -360,10 +370,10 @@ use of extensions and concurrent index removal, you need at least PostgreSQL 9.2 ## 7. Redis -GitLab requires at least Redis 2.8. +GitLab requires at least Redis 5.0. -If you are using Debian 8 or Ubuntu 14.04 and up, you can simply install -Redis 2.8 with: +If you are using Debian 10 or Ubuntu 20.04 and up, you can install +Redis 5.0 with: ```shell sudo apt-get install redis-server @@ -966,7 +976,7 @@ If you want to switch back to Unicorn, follow these steps: sudo -u git -H cp config/unicorn.rb.example config/unicorn.rb ``` -1. Edit the system `init.d` script to set the `USE_UNICORN=1` flag. If you have `/etc/default/gitlab`, then you should edit it instead. +1. Edit the system `init.d` script and set `USE_WEB_SERVER="unicorn"`. If you have `/etc/default/gitlab`, then you should edit it instead. 1. Restart GitLab. ### Using Sidekiq instead of Sidekiq Cluster @@ -1035,6 +1045,3 @@ On RedHat/CentOS: ```shell sudo yum groupinstall 'Development Tools' ``` - -[RVM]: https://rvm.io/ "RVM Homepage" -[chruby]: https://github.com/postmodern/chruby "chruby on GitHub" diff --git a/doc/install/openshift_and_gitlab/index.md b/doc/install/openshift_and_gitlab/index.md index a71d839e742..fd81b7f6caf 100644 --- a/doc/install/openshift_and_gitlab/index.md +++ b/doc/install/openshift_and_gitlab/index.md @@ -14,7 +14,7 @@ for details. ## Introduction [OpenShift Origin](https://www.okd.io/) (**Note:** renamed to OKD in Aug 2018) is an open source container application -platform created by [RedHat], based on [Kubernetes](https://kubernetes.io/) and [Docker]. That means +platform created by [RedHat](https://www.redhat.com/en), based on [Kubernetes](https://kubernetes.io/) and [Docker](https://www.docker.com). That means you can host your own PaaS for free and almost with no hassle. In this tutorial, we will see how to deploy GitLab in OpenShift using GitLab's @@ -34,16 +34,16 @@ offered by the OpenShift developers and managed by Vagrant. If you haven't done already, go ahead and install the following components as they are essential to test OpenShift easily: -- [VirtualBox] -- [Vagrant] +- [VirtualBox](https://www.virtualbox.org/wiki/Downloads) +- [Vagrant](https://www.vagrantup.com/downloads.html) - [OpenShift Client](https://docs.okd.io/3.11/cli_reference/get_started_cli.html) (`oc` for short) It is also important to mention that for the purposes of this tutorial, the latest Origin release is used: -- **oc** `v1.3.0` (must be [installed][oc-gh] locally on your computer) -- **OpenShift** `v1.3.0` (is pre-installed in the [VM image][vm-new]) -- **Kubernetes** `v1.3.0` (is pre-installed in the [VM image][vm-new]) +- **oc** `v1.3.0` (must be [installed](https://github.com/openshift/origin/releases/tag/v1.3.0) locally on your computer) +- **OpenShift** `v1.3.0` (is pre-installed in the [VM image](https://app.vagrantup.com/openshift/boxes/origin-all-in-one)) +- **Kubernetes** `v1.3.0` (is pre-installed in the [VM image](https://app.vagrantup.com/openshift/boxes/origin-all-in-one)) >**Note:** If you intend to deploy GitLab on a production OpenShift cluster, there are some @@ -302,7 +302,7 @@ template: - `gitlab-ce-postgresql` While PostgreSQL and Redis are bundled in Omnibus GitLab, the template is using -separate images as you can see from [this line][line] in the template. +separate images as you can see from [this line](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/658c065c8d022ce858dd63eaeeadb0b2ddc8deea/docker/openshift-template.json#L239) in the template. The predefined values have been calculated for the purposes of testing out GitLab in the all-in-one VM. You don't need to change anything here, hit @@ -371,7 +371,7 @@ running scaled to 2. Upping the GitLab pods is actually like adding new application servers to your cluster. You can see how that would work if you didn't use GitLab with -OpenShift by following the [HA documentation][ha] for the application servers. +OpenShift by following the [HA documentation](../../administration/high_availability/gitlab.md) for the application servers. Bare in mind that you may need more resources (CPU, RAM, disk space) when you scale up. If a pod is in pending state for too long, you can navigate to @@ -505,14 +505,3 @@ And remember that in this tutorial we just scratched the surface of what Origin is capable of. As always, you can refer to the detailed [documentation](https://docs.okd.io) to learn more about deploying your own OpenShift PaaS and managing your applications with the ease of containers. - -[RedHat]: https://www.redhat.com/en "RedHat website" -[vm-new]: https://app.vagrantup.com/openshift/boxes/origin-all-in-one "Official OpenShift Vagrant box on Vagrant Cloud" -[template]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/docker/openshift-template.json "OpenShift template for GitLab" -[Docker]: https://www.docker.com "Docker website" -[VirtualBox]: https://www.virtualbox.org/wiki/Downloads "VirtualBox downloads" -[Vagrant]: https://www.vagrantup.com/downloads.html "Vagrant downloads" -[old-post]: https://blog.openshift.com/deploy-gitlab-openshift/ "Old post - Deploy GitLab on OpenShift" -[line]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/658c065c8d022ce858dd63eaeeadb0b2ddc8deea/docker/openshift-template.json#L239 "GitLab - OpenShift template" -[oc-gh]: https://github.com/openshift/origin/releases/tag/v1.3.0 "OpenShift Origin 1.3.0 release on GitHub" -[ha]: ../../administration/high_availability/gitlab.md "Documentation - GitLab High Availability" diff --git a/doc/install/relative_url.md b/doc/install/relative_url.md index 595304e27e2..f8ff8e75c4d 100644 --- a/doc/install/relative_url.md +++ b/doc/install/relative_url.md @@ -10,7 +10,7 @@ be installed under a relative URL, for example `https://example.com/gitlab`. This document describes how to run GitLab under a relative URL for installations from source. If you are using an Omnibus package, -[the steps are different][omnibus-rel]. Use this guide along with the +[the steps are different](https://docs.gitlab.com/omnibus/settings/configuration.html#configuring-a-relative-url-for-gitlab). Use this guide along with the [installation guide](installation.md) if you are installing GitLab for the first time. @@ -30,7 +30,7 @@ serve GitLab under a relative URL is: - `/home/git/gitlab-shell/config.yml` - `/etc/default/gitlab` -After all the changes you need to recompile the assets and [restart GitLab]. +After all the changes you need to recompile the assets and [restart GitLab](../administration/restart_gitlab.md#installations-from-source). ## Relative URL requirements @@ -112,7 +112,7 @@ Make sure to follow all steps below: If you are using a custom init script, make sure to edit the above GitLab Workhorse setting as needed. -1. [Restart GitLab][] for the changes to take effect. +1. [Restart GitLab](../administration/restart_gitlab.md#installations-from-source) for the changes to take effect. ## Disable relative URL in GitLab @@ -123,9 +123,6 @@ To disable the relative URL: 1. Follow the same as above starting from 2. and set up the GitLab URL to one that doesn't contain a relative path. -[omnibus-rel]: https://docs.gitlab.com/omnibus/settings/configuration.html#configuring-a-relative-url-for-gitlab "How to set up relative URL in Omnibus GitLab" -[restart gitlab]: ../administration/restart_gitlab.md#installations-from-source "How to restart GitLab" - <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/install/requirements.md b/doc/install/requirements.md index f78525659f2..09ad5f9afd7 100644 --- a/doc/install/requirements.md +++ b/doc/install/requirements.md @@ -11,15 +11,15 @@ as the hardware requirements that are needed to install and use GitLab. ### Supported Linux distributions -- Ubuntu -- Debian -- CentOS -- openSUSE +- Ubuntu (16.04/18.04) +- Debian (8/9/10) +- CentOS (6/7/8) +- openSUSE (Leap 15.1/Enterprise Server 12.2) - Red Hat Enterprise Linux (please use the CentOS packages and instructions) - Scientific Linux (please use the CentOS packages and instructions) - Oracle Linux (please use the CentOS packages and instructions) -For the installations options, see [the main installation page](README.md). +For the installation options, see [the main installation page](README.md). ### Unsupported Linux distributions and Unix-like operating systems @@ -68,10 +68,14 @@ GitLab uses [webpack](https://webpack.js.org/) to compile frontend assets, which version of Node.js 10.13.0. You can check which version you are running with `node -v`. If you are running -a version older than `v10.13.0`, you need to update to a newer version. You +a version older than `v10.13.0`, you need to update it to a newer version. You can find instructions to install from community maintained packages or compile from source at the [Node.js website](https://nodejs.org/en/download/). +## Redis versions + +GitLab requires Redis 5.0+. Beginning in GitLab 13.0, lower versions are not supported. + ## Hardware requirements ### Storage @@ -95,7 +99,7 @@ This is the recommended minimum hardware for a handful of example GitLab user ba - 4 cores supports up to 500 users - 8 cores supports up to 1,000 users - 32 cores supports up to 5,000 users -- More users? Run it high-availability on [multiple application servers](https://about.gitlab.com/solutions/high-availability/) +- More users? Consult the [reference architectures page](../administration/reference_architectures/index.md) ### Memory @@ -112,7 +116,7 @@ errors during usage. - 16GB RAM supports up to 500 users - 32GB RAM supports up to 1,000 users - 128GB RAM supports up to 5,000 users -- More users? Run it high-availability on [multiple application servers](https://about.gitlab.com/solutions/high-availability/) +- More users? Consult the [reference architectures page](../administration/reference_architectures/index.md) We recommend having at least [2GB of swap on your server](https://askubuntu.com/a/505344/310789), even if you currently have enough available RAM. Having swap will help reduce the chance of errors occurring @@ -122,7 +126,7 @@ available when needed. Our [Memory Team](https://about.gitlab.com/handbook/engineering/development/enablement/memory/) is actively working to reduce the memory requirement. -NOTE: **Note:** The 25 workers of Sidekiq will show up as separate processes in your process overview (such as `top` or `htop`) but they share the same RAM allocation since Sidekiq is a multithreaded application. Please see the section below about Unicorn workers for information about how many you need of those. +NOTE: **Note:** The 25 workers of Sidekiq will show up as separate processes in your process overview (such as `top` or `htop`) but they share the same RAM allocation since Sidekiq is a multithreaded application. Please see the section below about Unicorn workers for information about how many you need for those. ## Database @@ -139,9 +143,12 @@ MySQL/MariaDB are advised to [migrate to PostgreSQL](../update/mysql_to_postgres ### PostgreSQL Requirements -As of GitLab 10.0, PostgreSQL 9.6 or newer is required, and earlier versions are -not supported. We highly recommend users to use PostgreSQL 9.6 as this -is the PostgreSQL version used for development and testing. +We highly recommend users to use the minimum PostgreSQL versions specified below as these are the versions used for development and testing. + +GitLab version | Minimum PostgreSQL version +-|- +10.0 | 9.6 +12.10 | 11 Users using PostgreSQL must ensure the `pg_trgm` extension is loaded into every GitLab database. This extension can be enabled (using a PostgreSQL super user) @@ -167,7 +174,7 @@ If you are using [GitLab Geo](../development/geo.md): - The [tracking database](../development/geo.md#using-the-tracking-database) requires the - [postgres_fdw](https://www.postgresql.org/docs/9.6/postgres-fdw.html) + [postgres_fdw](https://www.postgresql.org/docs/11/postgres-fdw.html) extension. ```sql @@ -180,28 +187,50 @@ For most instances we recommend using: (CPU cores * 1.5) + 1 = Unicorn workers. For example a node with 4 cores would have 7 Unicorn workers. For all machines that have 2GB and up we recommend a minimum of three Unicorn workers. -If you have a 1GB machine we recommend to configure only two Unicorn workers to prevent excessive swapping. +If you have a 1GB machine we recommend to configure only two Unicorn workers to prevent excessive +swapping. + +As long as you have enough available CPU and memory capacity, it's okay to increase the number of +Unicorn workers and this will usually help to reduce the response time of the applications and +increase the ability to handle parallel requests. + +To change the Unicorn workers when you have the Omnibus package (which defaults to the +recommendation above) please see [the Unicorn settings in the Omnibus GitLab documentation](https://docs.gitlab.com/omnibus/settings/unicorn.html). + +## Puma settings + +The recommended settings for Puma are determined by the infrastructure on which it's running. +Omnibus GitLab defaults to the recommended Puma settings. Regardless of installation method, you can +tune the Puma settings. -As long as you have enough available CPU and memory capacity, it's okay to increase the number of Unicorn workers and this will usually help to reduce the response time of the applications and increase the ability to handle parallel requests. +If you're using Omnibus GitLab, see [Puma settings](https://docs.gitlab.com/omnibus/settings/puma.html) +for instructions on changing the Puma settings. -To change the Unicorn workers when you have the Omnibus package (which defaults to the recommendation above) please see [the Unicorn settings in the Omnibus GitLab documentation](https://docs.gitlab.com/omnibus/settings/unicorn.html). +### Puma workers -## Puma Workers +The recommended number of workers is calculated as the highest of the following: -For most instances we recommend using: max(CPU cores * 0.9, 2) = Puma workers. -For example a node with 4 cores would have 3 Puma workers. +- `2` +- Number of CPU cores - 1 -For all machines that have 4GB and up we recommend a minimum of three Puma workers. -If you have a 2GB machine we recommend to configure only one Puma worker to prevent excessive swapping. +For example a node with 4 cores should be configured with 3 Puma workers. -By default each Puma worker runs with 4 threads. We do not recommend setting more, -due to how [Ruby MRI multi-threading](https://en.wikipedia.org/wiki/Global_interpreter_lock) works. +You can increase the number of Puma workers, providing enough CPU and memory capacity is available. +A higher number of Puma workers will usually help to reduce the response time of the application +and increase the ability to handle parallel requests. You must perform testing to verify the +optimal settings for your infrastructure. -For cases when you have to use [Legacy Rugged code](../development/gitaly.md#legacy-rugged-code) it is recommended to set number of threads to 1. +### Puma threads -As long as you have enough available CPU and memory capacity, it's okay to increase the number of Puma workers and this will usually help to reduce the response time of the applications and increase the ability to handle parallel requests. +The recommended number of threads is dependent on several factors, including total memory, and use +of [legacy Rugged code](../development/gitaly.md#legacy-rugged-code). -To change the Puma workers when you have the Omnibus package (which defaults to the recommendation above) please see [the Puma settings in the Omnibus GitLab documentation](https://docs.gitlab.com/omnibus/settings/puma.html). +- If the operating system has a maximum 2 GB of memory, the recommended number of threads is `1`. + A higher value will result in excess swapping, and decrease performance. +- If legacy Rugged code is in use, the recommended number of threads is `1`. +- In all other cases, the recommended number of threads is `4`. We do not recommend setting this +higher, due to how [Ruby MRI multi-threading](https://en.wikipedia.org/wiki/Global_interpreter_lock) +works. ## Redis and Sidekiq |