diff options
author | Lin Jen-Shin <godfat@godfat.org> | 2016-11-17 23:28:58 +0800 |
---|---|---|
committer | Lin Jen-Shin <godfat@godfat.org> | 2016-11-17 23:28:58 +0800 |
commit | a8a879eebc805f27de9eb27fa05bce291ee931ff (patch) | |
tree | bc7054b990df703cbfa1fc4aac77e87be2359fe8 /doc | |
parent | 48090a9188e13e3ddaffb5957a7b5a264024f060 (diff) | |
parent | aea8baed3093c513560e9ac5ac0c5c99508d3001 (diff) | |
download | gitlab-ce-a8a879eebc805f27de9eb27fa05bce291ee931ff.tar.gz |
Merge remote-tracking branch 'upstream/master' into feature/1376-allow-write-access-deploy-keys
* upstream/master: (236 commits)
Mention Git strategy none
Remove ToC since it's now supported in the docs portal itself
Add 8.14 to versions with further additions to review apps
Add Limitations sections to environments and review apps docs
Add link to environments docs
Fix URL to review apps docs
Add a prerequisites section, add some links
Link to NGINX example project for the time being
Get rid most of the irrelevant sections
Add note about current limitation in $CI_BUILD_REF_NAME
Add an intro and an Overview section for Review Apps
WIP review apps
Add Review apps link to CI README
Add stop environment permissions and remove delete
Add note about auto-stopping of environments
Update CHANGELOG.md for 8.13.6
Finish "Stopping envs" and "Grouping similar envs" sections
Highlight first user autocomplete option
Remove ToC and fix headings in Markdown docs
Revert "Merge branch '22680-unlabel-limit-autocomplete-to-selected-items' into 'master'"
...
Diffstat (limited to 'doc')
103 files changed, 2731 insertions, 392 deletions
diff --git a/doc/administration/auth/ldap.md b/doc/administration/auth/ldap.md index fd23047f027..d3f216fb3bf 100644 --- a/doc/administration/auth/ldap.md +++ b/doc/administration/auth/ldap.md @@ -257,6 +257,24 @@ the LDAP server's SSL certificate is performed. ## Troubleshooting +### Debug LDAP user filter with ldapsearch + +This example uses ldapsearch and assumes you are using ActiveDirectory. The +following query returns the login names of the users that will be allowed to +log in to GitLab if you configure your own user_filter. + +``` +ldapsearch -H ldaps://$host:$port -D "$bind_dn" -y bind_dn_password.txt -b "$base" "$user_filter" sAMAccountName +``` + +- Variables beginning with a `$` refer to a variable from the LDAP section of + your configuration file. +- Replace ldaps:// with ldap:// if you are using the plain authentication method. + Port `389` is the default `ldap://` port and `636` is the default `ldaps://` + port. +- We are assuming the password for the bind_dn user is in bind_dn_password.txt. + + ### Invalid credentials when logging in - Make sure the user you are binding with has enough permissions to read the user's diff --git a/doc/administration/high_availability/README.md b/doc/administration/high_availability/README.md index d74a786ac24..d5a5aef7ec0 100644 --- a/doc/administration/high_availability/README.md +++ b/doc/administration/high_availability/README.md @@ -7,19 +7,10 @@ highly available. ## Architecture -### Active/Passive - -For pure high-availability/failover with no scaling you can use an -active/passive configuration. This utilizes DRBD (Distributed Replicated -Block Device) to keep all data in sync. DRBD requires a low latency link to -remain in sync. It is not advisable to attempt to run DRBD between data centers -or in different cloud availability zones. +There are two kinds of setups: -Components/Servers Required: - -- 2 servers/virtual machines (one active/one passive) - -![Active/Passive HA Diagram](../img/high_availability/active-passive-diagram.png) +- active/active +- active/passive ### Active/Active @@ -28,12 +19,24 @@ user requests simultaneously. The database, Redis, and GitLab application are all deployed on separate servers. The configuration is **only** highly-available if the database, Redis and storage are also configured as such. -![Active/Active HA Diagram](../img/high_availability/active-active-diagram.png) - -**Steps to configure active/active:** +Follow the steps below to configure an active/active setup: 1. [Configure the database](database.md) 1. [Configure Redis](redis.md) 1. [Configure NFS](nfs.md) 1. [Configure the GitLab application servers](gitlab.md) 1. [Configure the load balancers](load_balancer.md) + +![Active/Active HA Diagram](../img/high_availability/active-active-diagram.png) + +### Active/Passive + +For pure high-availability/failover with no scaling you can use an +active/passive configuration. This utilizes DRBD (Distributed Replicated +Block Device) to keep all data in sync. DRBD requires a low latency link to +remain in sync. It is not advisable to attempt to run DRBD between data centers +or in different cloud availability zones. + +Components/Servers Required: 2 servers/virtual machines (one active/one passive) + +![Active/Passive HA Diagram](../img/high_availability/active-passive-diagram.png) diff --git a/doc/administration/high_availability/nfs.md b/doc/administration/high_availability/nfs.md index 537f4f3501d..5602d70f1ef 100644 --- a/doc/administration/high_availability/nfs.md +++ b/doc/administration/high_availability/nfs.md @@ -76,7 +76,7 @@ configuration to move each data location to a subdirectory: user['home'] = '/gitlab-data/home' git_data_dir '/gitlab-data/git-data' gitlab_rails['shared_path'] = '/gitlab-data/shared' -gitlab_rails['uploads_directory'] = "/gitlab-data/uploads" +gitlab_rails['uploads_directory'] = '/gitlab-data/uploads' gitlab_ci['builds_directory'] = '/gitlab-data/builds' ``` diff --git a/doc/administration/high_availability/redis.md b/doc/administration/high_availability/redis.md index bc424330656..f532a106bc6 100644 --- a/doc/administration/high_availability/redis.md +++ b/doc/administration/high_availability/redis.md @@ -1,265 +1,780 @@ # Configuring Redis for GitLab HA -You can choose to install and manage Redis yourself, or you can use the one -that comes bundled with GitLab Omnibus packages. - -> **Note:** Redis does not require authentication by default. See +> +Experimental Redis Sentinel support was [Introduced][ce-1877] in GitLab 8.11. +Starting with 8.14, Redis Sentinel is no longer experimental. +If you've used it with versions `< 8.14` before, please check the updated +documentation here. + +High Availability with [Redis] is possible using a **Master** x **Slave** +topology with a [Redis Sentinel][sentinel] service to watch and automatically +start the failover procedure. + +You can choose to install and manage Redis and Sentinel yourself, use +a hosted cloud solution or you can use the one that comes bundled with +Omnibus GitLab packages. + +> **Notes:** +- Redis requires authentication for High Availability. See [Redis Security](http://redis.io/topics/security) documentation for more information. We recommend using a combination of a Redis password and tight firewall rules to secure your Redis service. +- You are highly encouraged to read the [Redis Sentinel][sentinel] documentation + before configuring Redis HA with GitLab to fully understand the topology and + architecture. +- This is the documentation for the Omnibus GitLab packages. For installations + from source, follow the [Redis HA source installation](redis_source.md) guide. +- Redis Sentinel daemon is bundled with Omnibus GitLab Enterprise Edition only. + For configuring Sentinel with the Omnibus GitLab Community Edition and + installations from source, read the + [Available configuration setups](#available-configuration-setups) section + below. + +## Overview + +Before diving into the details of setting up Redis and Redis Sentinel for HA, +make sure you read this Overview section to better understand how the components +are tied together. + +You need at least `3` independent machines: physical, or VMs running into +distinct physical machines. It is essential that all master and slaves Redis +instances run in different machines. If you fail to provision the machines in +that specific way, any issue with the shared environment can bring your entire +setup down. + +It is OK to run a Sentinel along with a master or slave Redis instance. +No more than one Sentinel in the same machine though. + +You also need to take in consideration the underlying network topology, +making sure you have redundant connectivity between Redis / Sentinel and +GitLab instances, otherwise the networks will become a single point of +failure. + +Make sure that you read this document once as a whole before configuring the +components below. + +### High Availability with Sentinel + +>**Notes:** +- Starting with GitLab `8.11`, you can configure a list of Redis Sentinel + servers that will monitor a group of Redis servers to provide failover support. +- Starting with GitLab `8.14`, the Omnibus GitLab Enterprise Edition package + comes with Redis Sentinel daemon built-in. + +High Availability with Redis requires a few things: + +- Multiple Redis instances +- Run Redis in a **Master** x **Slave** topology +- Multiple Sentinel instances +- Application support and visibility to all Sentinel and Redis instances + +Redis Sentinel can handle the most important tasks in an HA environment and that's +to help keep servers online with minimal to no downtime. Redis Sentinel: + +- Monitors **Master** and **Slaves** instances to see if they are available +- Promotes a **Slave** to **Master** when the **Master** fails +- Demotes a **Master** to **Slave** when the failed **Master** comes back online + (to prevent data-partitioning) +- Can be queried by the application to always connect to the current **Master** + server + +When a **Master** fails to respond, it's the application's responsibility +(in our case GitLab) to handle timeout and reconnect (querying a **Sentinel** +for a new **Master**). -## Configure your own Redis server +To get a better understanding on how to correctly setup Sentinel, please read +the [Redis Sentinel documentation](http://redis.io/topics/sentinel) first, as +failing to configure it correctly can lead to data loss or can bring your +whole cluster down, invalidating the failover effort. -If you're hosting GitLab on a cloud provider, you can optionally use a -managed service for Redis. For example, AWS offers a managed ElastiCache service -that runs Redis. +### Recommended setup -## Configure Redis using Omnibus +For a minimal setup, you will install the Omnibus GitLab package in `3` +**independent** machines, both with **Redis** and **Sentinel**: -If you don't want to bother setting up your own Redis server, you can use the -one bundled with Omnibus. In this case, you should disable all services except -Redis. +- Redis Master + Sentinel +- Redis Slave + Sentinel +- Redis Slave + Sentinel -1. Download/install GitLab Omnibus using **steps 1 and 2** from - [GitLab downloads](https://about.gitlab.com/downloads). Do not complete other - steps on the download page. -1. Create/edit `/etc/gitlab/gitlab.rb` and use the following configuration. - Be sure to change the `external_url` to match your eventual GitLab front-end - URL: +If you are not sure or don't understand why and where the amount of nodes come +from, read [Redis setup overview](#redis-setup-overview) and +[Sentinel setup overview](#sentinel-setup-overview). - ```ruby - external_url 'https://gitlab.example.com' - - # Disable all services except Redis - redis['enable'] = true - bootstrap['enable'] = false - nginx['enable'] = false - unicorn['enable'] = false - sidekiq['enable'] = false - postgresql['enable'] = false - gitlab_workhorse['enable'] = false - mailroom['enable'] = false - - # Redis configuration - redis['port'] = 6379 - redis['bind'] = '0.0.0.0' +For a recommended setup that can resist more failures, you will install +the Omnibus GitLab package in `5` **independent** machines, both with +**Redis** and **Sentinel**: - # If you wish to use Redis authentication (recommended) - redis['password'] = 'Redis Password' - ``` +- Redis Master + Sentinel +- Redis Slave + Sentinel +- Redis Slave + Sentinel +- Redis Slave + Sentinel +- Redis Slave + Sentinel -1. Run `sudo gitlab-ctl reconfigure` to install and configure PostgreSQL. +### Redis setup overview - > **Note**: This `reconfigure` step will result in some errors. - That's OK - don't be alarmed. +You must have at least `3` Redis servers: `1` Master, `2` Slaves, and they +need to be each in a independent machine (see explanation above). -1. Run `touch /etc/gitlab/skip-auto-migrations` to prevent database migrations - from running on upgrade. Only the primary GitLab application server should - handle migrations. +You can have additional Redis nodes, that will help survive a situation +where more nodes goes down. Whenever there is only `2` nodes online, a failover +will not be initiated. -## Experimental Redis Sentinel support +As an example, if you have `6` Redis nodes, a maximum of `3` can be +simultaneously down. -> [Introduced][ce-1877] in GitLab 8.11. +Please note that there are different requirements for Sentinel nodes. +If you host them in the same Redis machines, you may need to take +that restrictions into consideration when calculating the amount of +nodes to be provisioned. See [Sentinel setup overview](#sentinel-setup-overview) +documentation for more information. -Since GitLab 8.11, you can configure a list of Redis Sentinel servers that -will monitor a group of Redis servers to provide you with a standard failover -support. +All Redis nodes should be configured the same way and with similar server specs, as +in a failover situation, any **Slave** can be promoted as the new **Master** by +the Sentinel servers. -There is currently one exception to the Sentinel support: `mail_room`, the -component that processes incoming emails. It doesn't support Sentinel yet, but -we hope to integrate a future release that does support it. +The replication requires authentication, so you need to define a password to +protect all Redis nodes and the Sentinels. They will all share the same +password, and all instances must be able to talk to +each other over the network. -To get a better understanding on how to correctly setup Sentinel, please read -the [Redis Sentinel documentation](http://redis.io/topics/sentinel) first, as -failing to configure it correctly can lead to data loss. +### Sentinel setup overview -The configuration consists of three parts: +Sentinels watch both other Sentinels and Redis nodes. Whenever a Sentinel +detects that a Redis node is not responding, it will announce that to the +other Sentinels. They have to reach the **quorum**, that is the minimum amount +of Sentinels that agrees a node is down, in order to be able to start a failover. -- Redis setup -- Sentinel setup -- GitLab setup +Whenever the **quorum** is met, the **majority** of all known Sentinel nodes +need to be available and reachable, so that they can elect the Sentinel **leader** +who will take all the decisions to restore the service availability by: -Read carefully how to configure those components below. +- Promoting a new **Master** +- Reconfiguring the other **Slaves** and make them point to the new **Master** +- Announce the new **Master** to every other Sentinel peer +- Reconfigure the old **Master** and demote to **Slave** when it comes back online -### Redis setup +You must have at least `3` Redis Sentinel servers, and they need to +be each in a independent machine (that are believed to fail independently), +ideally in different geographical areas. -You must have at least 2 Redis servers: 1 Master, 1 or more Slaves. -They should be configured the same way and with similar server specs, as -in a failover situation, any Slave can be elected as the new Master by -the Sentinel servers. +You can configure them in the same machines where you've configured the other +Redis servers, but understand that if a whole node goes down, you loose both +a Sentinel and a Redis instance. -In a minimal setup, the only required change for the slaves in `redis.conf` -is the addition of a `slaveof` line pointing to the initial master. -You can increase the security by defining a `requirepass` configuration in -the master, and `masterauth` in slaves. +The number of sentinels should ideally always be an **odd** number, for the +consensus algorithm to be effective in the case of a failure. ---- +In a `3` nodes topology, you can only afford `1` Sentinel node going down. +Whenever the **majority** of the Sentinels goes down, the network partition +protection prevents destructive actions and a failover **will not be started**. -**Configuring your own Redis server** +Here are some examples: -1. Add to the slaves' `redis.conf`: +- With `5` or `6` sentinels, a maximum of `2` can go down for a failover begin. +- With `7` sentinels, a maximum of `3` nodes can go down. - ```conf - # IP and port of the master Redis server - slaveof 10.10.10.10 6379 - ``` +The **Leader** election can sometimes fail the voting round when **consensus** +is not achieved (see the odd number of nodes requirement above). In that case, +a new attempt will be made after the amount of time defined in +`sentinel['failover_timeout']` (in milliseconds). -1. Optionally, set up password authentication for increased security. - Add the following to master's `redis.conf`: +>**Note:** +We will see where `sentinel['failover_timeout']` is defined later. + +The `failover_timeout` variable has a lot of different use cases. According to +the official documentation: + +- The time needed to re-start a failover after a previous failover was + already tried against the same master by a given Sentinel, is two + times the failover timeout. + +- The time needed for a slave replicating to a wrong master according + to a Sentinel current configuration, to be forced to replicate + with the right master, is exactly the failover timeout (counting since + the moment a Sentinel detected the misconfiguration). + +- The time needed to cancel a failover that is already in progress but + did not produced any configuration change (SLAVEOF NO ONE yet not + acknowledged by the promoted slave). + +- The maximum time a failover in progress waits for all the slaves to be + reconfigured as slaves of the new master. However even after this time + the slaves will be reconfigured by the Sentinels anyway, but not with + the exact parallel-syncs progression as specified. + +### Available configuration setups + +Based on your infrastructure setup and how you have installed GitLab, there are +multiple ways to configure Redis HA. Omnibus GitLab packages have Redis and/or +Redis Sentinel bundled with them so you only need to focus on configuration. +Pick the one that suits your needs. + +- [Installations from source][source]: You need to install Redis and Sentinel + yourself. Use the [Redis HA installation from source](redis_source.md) + documentation. +- [Omnibus GitLab **Community Edition** (CE) package][ce]: Redis is bundled, so you + can use the package with only the Redis service enabled as described in steps + 1 and 2 of this document (works for both master and slave setups). To install + and configure Sentinel, jump directly to the Sentinel section in the + [Redis HA installation from source](redis_source.md#step-3-configuring-the-redis-sentinel-instances) documentation. +- [Omnibus GitLab **Enterprise Edition** (EE) package][ee]: Both Redis and Sentinel + are bundled in the package, so you can use the EE package to setup the whole + Redis HA infrastructure (master, slave and Sentinel) which is described in + this document. +- If you have installed GitLab using the Omnibus GitLab packages (CE or EE), + but you want to use your own external Redis server, follow steps 1-3 in the + [Redis HA installation from source](redis_source.md) documentation, then go + straight to step 4 in this guide to + [set up the GitLab application](#step-4-configuring-the-gitlab-application). + +## Configuring Redis HA + +This is the section where we install and setup the new Redis instances. + +>**Notes:** +- We assume that you install GitLab and all HA components from scratch. If you + already have it installed and running, read how to + [switch from a single-machine installation to Redis HA](#switching-from-an-existing-single-machine-installation-to-redis-ha). +- Redis nodes (both master and slaves) will need the same password defined in + `redis['password']`. At any time during a failover the Sentinels can + reconfigure a node and change its status from master to slave and vice versa. + +### Prerequisites + +The prerequisites for a HA Redis setup are the following: + +1. Provision the minimum required number of instances as specified in the + [recommended setup](#recommended-setup) section. +1. **Do NOT** install Redis or Redis Sentinel in the same machines your + GitLab application is running on. You can however opt in to install Redis + and Sentinel in the same machine (each in independent ones is recommended + though). +1. All Redis nodes must be able to talk to each other and accept incoming + connections over Redis (`6379`) and Sentinel (`26379`) ports (unless you + change the default ones). +1. The server that hosts the GitLab application must be able to access the + Redis nodes. +1. Protect the nodes from access from external networks ([Internet][it]), using + firewall. + +### Step 1. Configuring the master Redis instance + +1. SSH into the **master** Redis server. +1. [Download/install](https://about.gitlab.com/installation) the Omnibus GitLab + package you want using **steps 1 and 2** from the GitLab downloads page. + - Make sure you select the correct Omnibus package, with the same version + and type (Community, Enterprise editions) of your current install. + - Do not complete any other steps on the download page. + +1. Edit `/etc/gitlab/gitlab.rb` and add the contents: - ```conf - # Optional password authentication for increased security - requirepass "<password>" + ```ruby + # Enable the master role and disable all other services in the machine + # (you can still enable Sentinel). + redis_master_role['enable'] = true + + # IP address pointing to a local IP that the other machines can reach to. + # You can also set bind to '0.0.0.0' which listen in all interfaces. + # If you really need to bind to an external accessible IP, make + # sure you add extra firewall rules to prevent unauthorized access. + redis['bind'] = '10.0.0.1' + + # Define a port so Redis can listen for TCP requests which will allow other + # machines to connect to it. + redis['port'] = 6379 + + # Set up password authentication for Redis (use the same password in all nodes). + redis['password'] = 'redis-password-goes-here' ``` -1. Then add this line to all the slave servers' `redis.conf`: +1. To prevent database migrations from running on upgrade, run: - ```conf - masterauth "<password>" + ``` + sudo touch /etc/gitlab/skip-auto-migrations ``` -1. Restart the Redis services for the changes to take effect. + Only the primary GitLab application server should handle migrations. ---- +1. [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect. + +### Step 2. Configuring the slave Redis instances -**Using Redis via Omnibus** +1. SSH into the **slave** Redis server. +1. [Download/install](https://about.gitlab.com/installation) the Omnibus GitLab + package you want using **steps 1 and 2** from the GitLab downloads page. + - Make sure you select the correct Omnibus package, with the same version + and type (Community, Enterprise editions) of your current install. + - Do not complete any other steps on the download page. -1. Edit `/etc/gitlab/gitlab.rb` of a master Redis machine (usualy a single machine): +1. Edit `/etc/gitlab/gitlab.rb` and add the contents: ```ruby - ## Redis TCP support (will disable UNIX socket transport) - redis['bind'] = '0.0.0.0' # or specify an IP to bind to a single one + # Enable the slave role and disable all other services in the machine + # (you can still enable Sentinel). This will also set automatically + # `redis['master'] = false`. + redis_slave_role['enable'] = true + + # IP address pointing to a local IP that the other machines can reach to. + # You can also set bind to '0.0.0.0' which listen in all interfaces. + # If you really need to bind to an external accessible IP, make + # sure you add extra firewall rules to prevent unauthorized access. + redis['bind'] = '10.0.0.2' + + # Define a port so Redis can listen for TCP requests which will allow other + # machines to connect to it. redis['port'] = 6379 - ## Master redis instance - redis['password'] = '<huge password string here>' - ``` + # The same password for Redeis authentication you set up for the master node. + redis['password'] = 'redis-password-goes-here' -1. Edit `/etc/gitlab/gitlab.rb` of a slave Redis machine (should be one or more machines): + # The IP of the master Redis node. + redis['master_ip'] = '10.0.0.1' - ```ruby - ## Redis TCP support (will disable UNIX socket transport) - redis['bind'] = '0.0.0.0' # or specify an IP to bind to a single one - redis['port'] = 6379 + # Port of master Redis server, uncomment to change to non default. Defaults + # to `6379`. + #redis['master_port'] = 6379 + ``` + +1. To prevent database migrations from running on upgrade, run: - ## Slave redis instance - redis['master_ip'] = '10.10.10.10' # IP of master Redis server - redis['master_port'] = 6379 # Port of master Redis server - redis['master_password'] = "<huge password string here>" ``` + sudo touch /etc/gitlab/skip-auto-migrations + ``` + + Only the primary GitLab application server should handle migrations. -1. Reconfigure the GitLab for the changes to take effect: `sudo gitlab-ctl reconfigure` +1. [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect. +1. Go through the steps again for all the other slave nodes. --- +These values don't have to be changed again in `/etc/gitlab/gitlab.rb` after +a failover, as the nodes will be managed by the Sentinels, and even after a +`gitlab-ctl reconfigure`, they will get their configuration restored by +the same Sentinels. + +### Step 3. Configuring the Redis Sentinel instances + +>**Note:** +Redis Sentinel is bundled with Omnibus GitLab Enterprise Edition only. The +following section assumes you are using Omnibus GitLab Enterprise Edition. +For the Omnibus Community Edition and installations from source, follow the +[Redis HA source install](redis_source.md) guide. + Now that the Redis servers are all set up, let's configure the Sentinel servers. -### Sentinel setup +If you are not sure if your Redis servers are working and replicating +correctly, please read the [Troubleshooting Replication](#troubleshooting-replication) +and fix it before proceeding with Sentinel setup. -We don't provide yet an automated way to setup and run the Sentinel daemon -from Omnibus installation method. You must follow the instructions below and -run it by yourself. +You must have at least `3` Redis Sentinel servers, and they need to +be each in an independent machine. You can configure them in the same +machines where you've configured the other Redis servers. -The support for Sentinel in Ruby has some [caveats](https://github.com/redis/redis-rb/issues/531). -While you can give any name for the `master-group-name` part of the -configuration, as in this example: +With GitLab Enterprise Edition, you can use the Omnibus package to setup +multiple machines with the Sentinel daemon. -```conf -sentinel monitor <master-group-name> <ip> <port> <quorum> -``` +--- -,for it to work in Ruby, you have to use the "hostname" of the master Redis -server, otherwise you will get an error message like: -`Redis::CannotConnectError: No sentinels available.`. Read -[Sentinel troubleshooting](#sentinel-troubleshooting) for more information. +1. SSH into the server that will host Redis Sentinel. +1. **You can omit this step if the Sentinels will be hosted in the same node as + the other Redis instances.** -Here is an example configuration file (`sentinel.conf`) for a Sentinel node: + [Download/install](https://about.gitlab.com/downloads-ee) the + Omnibus GitLab Enterprise Edition package using **steps 1 and 2** from the + GitLab downloads page. + - Make sure you select the correct Omnibus package, with the same version + the GitLab application is running. + - Do not complete any other steps on the download page. -```conf -port 26379 -sentinel monitor master-redis.example.com 10.10.10.10 6379 1 -sentinel down-after-milliseconds master-redis.example.com 10000 -sentinel config-epoch master-redis.example.com 0 -sentinel leader-epoch master-redis.example.com 0 -``` +1. Edit `/etc/gitlab/gitlab.rb` and add the contents (if you are installing the + Sentinels in the same node as the other Redis instances, some values might + be duplicate below): ---- + ```ruby + redis_sentinel_role['enable'] = true -The final part is to inform the main GitLab application server of the Redis -master and the new sentinels servers. + # Must be the same in every sentinel node + redis['master_name'] = 'gitlab-redis' -### GitLab setup + # The same password for Redis authentication you set up for the master node. + redis['password'] = 'redis-password-goes-here' -You can enable or disable sentinel support at any time in new or existing -installations. From the GitLab application perspective, all it requires is -the correct credentials for the master Redis and for a few Sentinel nodes. + # The IP of the master Redis node. + redis['master_ip'] = '10.0.0.1' -It doesn't require a list of all Sentinel nodes, as in case of a failure, -the application will need to query only one of them. + # Define a port so Redis can listen for TCP requests which will allow other + # machines to connect to it. + redis['port'] = 6379 ->**Note:** -The following steps should be performed in the [GitLab application server](gitlab.md). + # Port of master Redis server, uncomment to change to non default. Defaults + # to `6379`. + #redis['master_port'] = 6379 + + ## Configure Sentinel + sentinel['bind'] = '10.0.0.1' + + # Port that Sentinel listens on, uncomment to change to non default. Defaults + # to `26379`. + # sentinel['port'] = 26379 + + ## Quorum must reflect the amount of voting sentinels it take to start a failover. + ## Value must NOT be greater then the amount of sentinels. + ## + ## The quorum can be used to tune Sentinel in two ways: + ## 1. If a the quorum is set to a value smaller than the majority of Sentinels + ## we deploy, we are basically making Sentinel more sensible to master failures, + ## triggering a failover as soon as even just a minority of Sentinels is no longer + ## able to talk with the master. + ## 1. If a quorum is set to a value greater than the majority of Sentinels, we are + ## making Sentinel able to failover only when there are a very large number (larger + ## than majority) of well connected Sentinels which agree about the master being down.s + sentinel['quorum'] = 2 + + ## Consider unresponsive server down after x amount of ms. + # sentinel['down_after_milliseconds'] = 10000 + + ## Specifies the failover timeout in milliseconds. It is used in many ways: + ## + ## - The time needed to re-start a failover after a previous failover was + ## already tried against the same master by a given Sentinel, is two + ## times the failover timeout. + ## + ## - The time needed for a slave replicating to a wrong master according + ## to a Sentinel current configuration, to be forced to replicate + ## with the right master, is exactly the failover timeout (counting since + ## the moment a Sentinel detected the misconfiguration). + ## + ## - The time needed to cancel a failover that is already in progress but + ## did not produced any configuration change (SLAVEOF NO ONE yet not + ## acknowledged by the promoted slave). + ## + ## - The maximum time a failover in progress waits for all the slaves to be + ## reconfigured as slaves of the new master. However even after this time + ## the slaves will be reconfigured by the Sentinels anyway, but not with + ## the exact parallel-syncs progression as specified. + # sentinel['failover_timeout'] = 60000 + ``` + +1. To prevent database migrations from running on upgrade, run: + + ``` + sudo touch /etc/gitlab/skip-auto-migrations + ``` -**For source based installations** + Only the primary GitLab application server should handle migrations. -1. Edit `/home/git/gitlab/config/resque.yml` following the example in - `/home/git/gitlab/config/resque.yml.example`, and uncomment the sentinels - line, changing to the correct server credentials. -1. Restart GitLab for the changes to take effect. +1. [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect. +1. Go through the steps again for all the other Sentinel nodes. -**For Omnibus installations** +### Step 4. Configuring the GitLab application +The final part is to inform the main GitLab application server of the Redis +Sentinels servers and authentication credentials. + +You can enable or disable Sentinel support at any time in new or existing +installations. From the GitLab application perspective, all it requires is +the correct credentials for the Sentinel nodes. + +While it doesn't require a list of all Sentinel nodes, in case of a failure, +it needs to access at least one of the listed. + +>**Note:** +The following steps should be performed in the [GitLab application server](gitlab.md) +which ideally should not have Redis or Sentinels on it for a HA setup. + +1. SSH into the server where the GitLab application is installed. 1. Edit `/etc/gitlab/gitlab.rb` and add/change the following lines: - ```ruby - gitlab-rails['redis_host'] = "master-redis.example.com" - gitlab-rails['redis_port'] = 6379 - gitlab-rails['redis_password'] = '<huge password string here>' - gitlab-rails['redis_sentinels'] = [ - {'host' => '10.10.10.1', 'port' => 26379}, - {'host' => '10.10.10.2', 'port' => 26379}, - {'host' => '10.10.10.3', 'port' => 26379} + ``` + ## Must be the same in every sentinel node + redis['master_name'] = 'gitlab-redis' + + ## The same password for Redis authentication you set up for the master node. + redis['password'] = 'redis-password-goes-here' + + ## A list of sentinels with `host` and `port` + gitlab_rails['redis_sentinels'] = [ + {'host' => '10.0.0.1', 'port' => 26379}, + {'host' => '10.0.0.2', 'port' => 26379}, + {'host' => '10.0.0.3', 'port' => 26379} ] ``` -1. [Reconfigure] the GitLab for the changes to take effect. +1. [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect. -### Sentinel troubleshooting +## Switching from an existing single-machine installation to Redis HA -If you get an error like: `Redis::CannotConnectError: No sentinels available.`, -there may be something wrong with your configuration files or it can be related -to [this issue][gh-531] ([pull request][gh-534] that should make things better). +If you already have a single-machine GitLab install running, you will need to +replicate from this machine first, before de-activating the Redis instance +inside it. + +Your single-machine install will be the initial **Master**, and the `3` others +should be configured as **Slave** pointing to this machine. -It's a bit rigid the way you have to config `resque.yml` and `sentinel.conf`, -otherwise `redis-rb` will not work properly. +After replication catches up, you will need to stop services in the +single-machine install, to rotate the **Master** to one of the new nodes. -The hostname ('my-primary-redis') of the primary Redis server (`sentinel.conf`) -**must** match the one configured in GitLab (`resque.yml` for source installations -or `gitlab-rails['redis_*']` in Omnibus) and it must be valid ex: +Make the required changes in configuration and restart the new nodes again. -```conf -# sentinel.conf: -sentinel monitor my-primary-redis 10.10.10.10 6379 1 -sentinel down-after-milliseconds my-primary-redis 10000 -sentinel config-epoch my-primary-redis 0 -sentinel leader-epoch my-primary-redis 0 +To disable redis in the single install, edit `/etc/gitlab/gitlab.rb`: + +```ruby +redis['enable'] = false +``` + +If you fail to replicate first, you may loose data (unprocessed background jobs). + +## Example of a minimal configuration with 1 master, 2 slaves and 3 Sentinels + +>**Note:** +Redis Sentinel is bundled with Omnibus GitLab Enterprise Edition only. For +different setups, read the +[available configuration setups](#available-configuration-setups) section. + +In this example we consider that all servers have an internal network +interface with IPs in the `10.0.0.x` range, and that they can connect +to each other using these IPs. + +In a real world usage, you would also setup firewall rules to prevent +unauthorized access from other machines and block traffic from the +outside (Internet). + +We will use the same `3` nodes with **Redis** + **Sentinel** topology +discussed in [Redis setup overview](#redis-setup-overview) and +[Sentinel setup overview](#sentinel-setup-overview) documentation. + +Here is a list and description of each **machine** and the assigned **IP**: + +* `10.0.0.1`: Redis Master + Sentinel 1 +* `10.0.0.2`: Redis Slave 1 + Sentinel 2 +* `10.0.0.3`: Redis Slave 2 + Sentinel 3 +* `10.0.0.4`: GitLab application + +Please note that after the initial configuration, if a failover is initiated +by the Sentinel nodes, the Redis nodes will be reconfigured and the **Master** +will change permanently (including in `redis.conf`) from one node to the other, +until a new failover is initiated again. + +The same thing will happen with `sentinel.conf` that will be overridden after the +initial execution, after any new sentinel node starts watching the **Master**, +or a failover promotes a different **Master** node. + +### Example configuration for Redis master and Sentinel 1 + +In `/etc/gitlab/gitlab.rb`: + +```ruby +redis_master_role['enable'] = true +redis_sentinel_role['enable'] = true +redis['bind'] = '10.0.0.1' +redis['port'] = 6379 +redis['password'] = 'redis-password-goes-here' +redis['master_name'] = 'gitlab-redis' # must be the same in every sentinel node +redis['master_password'] = 'redis-password-goes-here' # the same value defined in redis['password'] in the master instance +redis['master_ip'] = '10.0.0.1' # ip of the initial master redis instance +#redis['master_port'] = 6379 # port of the initial master redis instance, uncomment to change to non default +sentinel['bind'] = '10.0.0.1' +# sentinel['port'] = 26379 # uncomment to change default port +sentinel['quorum'] = 2 +# sentinel['down_after_milliseconds'] = 10000 +# sentinel['failover_timeout'] = 60000 +``` + +[Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect. + +### Example configuration for Redis slave 1 and Sentinel 2 + +In `/etc/gitlab/gitlab.rb`: + +```ruby +redis_slave_role['enable'] = true +redis_sentinel_role['enable'] = true +redis['bind'] = '10.0.0.2' +redis['port'] = 6379 +redis['password'] = 'redis-password-goes-here' +redis['master_password'] = 'redis-password-goes-here' +redis['master_ip'] = '10.0.0.1' # IP of master Redis server +#redis['master_port'] = 6379 # Port of master Redis server, uncomment to change to non default +redis['master_name'] = 'gitlab-redis' # must be the same in every sentinel node +sentinel['bind'] = '10.0.0.2' +# sentinel['port'] = 26379 # uncomment to change default port +sentinel['quorum'] = 2 +# sentinel['down_after_milliseconds'] = 10000 +# sentinel['failover_timeout'] = 60000 +``` + +[Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect. + +### Example configuration for Redis slave 2 and Sentinel 3 + +In `/etc/gitlab/gitlab.rb`: + +```ruby +redis_slave_role['enable'] = true +redis_sentinel_role['enable'] = true +redis['bind'] = '10.0.0.3' +redis['port'] = 6379 +redis['password'] = 'redis-password-goes-here' +redis['master_password'] = 'redis-password-goes-here' +redis['master_ip'] = '10.0.0.1' # IP of master Redis server +#redis['master_port'] = 6379 # Port of master Redis server, uncomment to change to non default +redis['master_name'] = 'gitlab-redis' # must be the same in every sentinel node +sentinel['bind'] = '10.0.0.3' +# sentinel['port'] = 26379 # uncomment to change default port +sentinel['quorum'] = 2 +# sentinel['down_after_milliseconds'] = 10000 +# sentinel['failover_timeout'] = 60000 +``` + +[Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect. + +### Example configuration for the GitLab application + +In `/etc/gitlab/gitlab.rb`: + +```ruby +redis['master_name'] = 'gitlab-redis' +redis['password'] = 'redis-password-goes-here' +gitlab_rails['redis_sentinels'] = [ + {'host' => '10.0.0.1', 'port' => 26379}, + {'host' => '10.0.0.2', 'port' => 26379}, + {'host' => '10.0.0.3', 'port' => 26379} +] +``` + +[Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect. + +## Advanced configuration + +Omnibus GitLab configures some things behind the curtains to make the sysadmins' +lives easier. If you want to know what happens underneath keep reading. + +### Control running services + +In the previous example, we've used `redis_sentinel_role` and +`redis_master_role` which simplifies the amount of configuration changes. + +If you want more control, here is what each one sets for you automatically +when enabled: + +```ruby +## Redis Sentinel Role +redis_sentinel_role['enable'] = true + +# When Sentinel Role is enabled, the following services are also enabled +sentinel['enable'] = true + +# The following services are disabled +redis['enable'] = false +bootstrap['enable'] = false +nginx['enable'] = false +postgresql['enable'] = false +gitlab_rails['enable'] = false +mailroom['enable'] = false + +------- + +## Redis master/slave Role +redis_master_role['enable'] = true # enable only one of them +redis_slave_role['enable'] = true # enable only one of them + +# When Redis Master or Slave role are enabled, the following services are +# enabled/disabled. Note that if Redis and Sentinel roles are combined, both +# services will be enabled. + +# The following services are disabled +sentinel['enable'] = false +bootstrap['enable'] = false +nginx['enable'] = false +postgresql['enable'] = false +gitlab_rails['enable'] = false +mailroom['enable'] = false + +# For Redis Slave role, also change this setting from default 'true' to 'false': +redis['master'] = false ``` -```yaml -# resque.yaml -production: - url: redis://my-primary-redis:6378 - sentinels: - - - host: slave1 - port: 26380 # point to sentinel, not to redis port - - - host: slave2 - port: 26381 # point to sentinel, not to redis port +You can find the relevant attributes defined in [gitlab_rails.rb][omnifile]. + +## Troubleshooting + +There are a lot of moving parts that needs to be taken care carefully +in order for the HA setup to work as expected. + +Before proceeding with the troubleshooting below, check your firewall rules: + +- Redis machines + - Accept TCP connection in `6379` + - Connect to the other Redis machines via TCP in `6379` +- Sentinel machines + - Accept TCP connection in `26379` + - Connect to other Sentinel machines via TCP in `26379` + - Connect to the Redis machines via TCP in `6379` + +### Troubleshooting Redis replication + +You can check if everything is correct by connecting to each server using +`redis-cli` application, and sending the `INFO` command. + +If authentication was correctly defined, it should fail with: +`NOAUTH Authentication required` error. Try to authenticate with the +previous defined password with `AUTH redis-password-goes-here` and +try the `INFO` command again. + +Look for the `# Replication` section where you should see some important +information like the `role` of the server. + +When connected to a `master` redis, you will see the number of connected +`slaves`, and a list of each with connection details: + +``` +# Replication +role:master +connected_slaves:1 +slave0:ip=10.133.5.21,port=6379,state=online,offset=208037514,lag=1 +master_repl_offset:208037658 +repl_backlog_active:1 +repl_backlog_size:1048576 +repl_backlog_first_byte_offset:206989083 +repl_backlog_histlen:1048576 ``` -When in doubt, please read [Redis Sentinel documentation](http://redis.io/topics/sentinel) +When it's a `slave`, you will see details of the master connection and if +its `up` or `down`: + +``` +# Replication +role:slave +master_host:10.133.1.58 +master_port:6379 +master_link_status:up +master_last_io_seconds_ago:1 +master_sync_in_progress:0 +slave_repl_offset:208096498 +slave_priority:100 +slave_read_only:1 +connected_slaves:0 +master_repl_offset:0 +repl_backlog_active:0 +repl_backlog_size:1048576 +repl_backlog_first_byte_offset:0 +repl_backlog_histlen:0 +``` + +### Troubleshooting Sentinel + +If you get an error like: `Redis::CannotConnectError: No sentinels available.`, +there may be something wrong with your configuration files or it can be related +to [this issue][gh-531]. + +You must make sure you are defining the same value in `redis['master_name']` +and `redis['master_pasword']` as you defined for your sentinel node. + +The way the redis connector `redis-rb` works with sentinel is a bit +non-intuitive. We try to hide the complexity in omnibus, but it still requires +a few extra configs. --- @@ -273,7 +788,7 @@ To make sure your configuration is correct: sudo gitlab-rails console # For source installations - sudo -u git rails console RAILS_ENV=production + sudo -u git rails console production ``` 1. Run in the console: @@ -288,8 +803,8 @@ To make sure your configuration is correct: 1. To simulate a failover on master Redis, SSH into the Redis server and run: ```bash - # port must match your master redis port - redis-cli -h localhost -p 6379 DEBUG sleep 60 + # port must match your master redis port, and the sleep time must be a few seconds bigger than defined one + redis-cli -h localhost -p 6379 DEBUG sleep 20 ``` 1. Then back in the Rails console from the first step, run: @@ -301,10 +816,26 @@ To make sure your configuration is correct: You should see a different port after a few seconds delay (the failover/reconnect time). ---- -Read more on high-availability configuration: +## Changelog + +Changes to Redis HA over time. + +**8.14** + +- Redis Sentinel support is production-ready and bundled in the Omnibus GitLab + Enterprise Edition package +- Documentation restructure for better readability + +**8.11** + +- Experimental Redis Sentinel support was added + +## Further reading + +Read more on High Availability: +1. [High Availability Overview](README.md) 1. [Configure the database](database.md) 1. [Configure NFS](nfs.md) 1. [Configure the GitLab application servers](gitlab.md) @@ -315,3 +846,10 @@ Read more on high-availability configuration: [reconfigure]: ../restart_gitlab.md#omnibus-gitlab-reconfigure [gh-531]: https://github.com/redis/redis-rb/issues/531 [gh-534]: https://github.com/redis/redis-rb/issues/534 +[redis]: http://redis.io/ +[sentinel]: http://redis.io/topics/sentinel +[omnifile]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-cookbooks/gitlab/libraries/gitlab_rails.rb +[source]: ../../install/installation.md +[ce]: https://about.gitlab.com/downloads +[ee]: https://about.gitlab.com/downloads-ee +[it]: https://gitlab.com/gitlab-org/gitlab-ce/uploads/c4cc8cd353604bd80315f9384035ff9e/The_Internet_IT_Crowd.png diff --git a/doc/administration/high_availability/redis_source.md b/doc/administration/high_availability/redis_source.md new file mode 100644 index 00000000000..3629772b8af --- /dev/null +++ b/doc/administration/high_availability/redis_source.md @@ -0,0 +1,366 @@ +# Configuring non-Omnibus Redis for GitLab HA + +This is the documentation for configuring a Highly Available Redis setup when +you have installed Redis all by yourself and not using the bundled one that +comes with the Omnibus packages. + +We cannot stress enough the importance of reading the +[Overview section](redis.md#overview) of the Omnibus Redis HA as it provides +some invaluable information to the configuration of Redis. Please proceed to +read it before going forward with this guide. + +We also highly recommend that you use the Omnibus GitLab packages, as we +optimize them specifically for GitLab, and we will take care of upgrading Redis +to the latest supported version. + +If you're not sure whether this guide is for you, please refer to +[Available configuration setups](redis.md#available-configuration-setups) in +the Omnibus Redis HA documentation. + +## Configuring your own Redis server + +This is the section where we install and setup the new Redis instances. + +### Prerequisites + +- All Redis servers in this guide must be configured to use a TCP connection + instead of a socket. To configure Redis to use TCP connections you need to + define both `bind` and `port` in the Redis config file. You can bind to all + interfaces (`0.0.0.0`) or specify the IP of the desired interface + (e.g., one from an internal network). +- Since Redis 3.2, you must define a password to receive external connections + (`requirepass`). +- If you are using Redis with Sentinel, you will also need to define the same + password for the slave password definition (`masterauth`) in the same instance. + +In addition, read the prerequisites as described in the +[Omnibus Redis HA document](redis.md#prerequisites) since they provide some +valuable information for the general setup. + +### Step 1. Configuring the master Redis instance + +Assuming that the Redis master instance IP is `10.0.0.1`: + +1. [Install Redis](../../install/installation.md#6-redis) +1. Edit `/etc/redis/redis.conf`: + + ```conf + ## Define a `bind` address pointing to a local IP that your other machines + ## can reach you. If you really need to bind to an external accessible IP, make + ## sure you add extra firewall rules to prevent unauthorized access: + bind 10.0.0.1 + + ## Define a `port` to force redis to listen on TCP so other machines can + ## connect to it (default port is `6379`). + port 6379 + + ## Set up password authentication (use the same password in all nodes). + ## The password should be defined equal for both `requirepass` and `masterauth` + ## when setting up Redis to use with Sentinel. + requirepass redis-password-goes-here + masterauth redis-password-goes-here + ``` + +1. Restart the Redis service for the changes to take effect. + +### Step 2. Configuring the slave Redis instances + +Assuming that the Redis slave instance IP is `10.0.0.2`: + +1. [Install Redis](../../install/installation.md#6-redis) +1. Edit `/etc/redis/redis.conf`: + + ```conf + ## Define a `bind` address pointing to a local IP that your other machines + ## can reach you. If you really need to bind to an external accessible IP, make + ## sure you add extra firewall rules to prevent unauthorized access: + bind 10.0.0.2 + + ## Define a `port` to force redis to listen on TCP so other machines can + ## connect to it (default port is `6379`). + port 6379 + + ## Set up password authentication (use the same password in all nodes). + ## The password should be defined equal for both `requirepass` and `masterauth` + ## when setting up Redis to use with Sentinel. + requirepass redis-password-goes-here + masterauth redis-password-goes-here + + ## Define `slaveof` pointing to the Redis master instance with IP and port. + slaveof 10.0.0.1 6379 + ``` + +1. Restart the Redis service for the changes to take effect. +1. Go through the steps again for all the other slave nodes. + +### Step 3. Configuring the Redis Sentinel instances + +Sentinel is a special type of Redis server. It inherits most of the basic +configuration options you can define in `redis.conf`, with specific ones +starting with `sentinel` prefix. + +Assuming that the Redis Sentinel is installed on the same instance as Redis +master with IP `10.0.0.1` (some settings might overlap with the master): + +1. [Install Redis Sentinel](http://redis.io/topics/sentinel) +1. Edit `/etc/redis/sentinel.conf`: + + ```conf + ## Define a `bind` address pointing to a local IP that your other machines + ## can reach you. If you really need to bind to an external accessible IP, make + ## sure you add extra firewall rules to prevent unauthorized access: + bind 10.0.0.1 + + ## Define a `port` to force Sentinel to listen on TCP so other machines can + ## connect to it (default port is `6379`). + port 26379 + + ## Set up password authentication (use the same password in all nodes). + ## The password should be defined equal for both `requirepass` and `masterauth` + ## when setting up Redis to use with Sentinel. + requirepass redis-password-goes-here + masterauth redis-password-goes-here + + ## Define with `sentinel auth-pass` the same shared password you have + ## defined for both Redis master and slaves instances. + sentinel auth-pass gitlab-redis redis-password-goes-here + + ## Define with `sentinel monitor` the IP and port of the Redis + ## master node, and the quorum required to start a failover. + sentinel monitor gitlab-redis 10.0.0.1 6379 2 + + ## Define with `sentinel down-after-milliseconds` the time in `ms` + ## that an unresponsive server will be considered down. + sentinel down-after-milliseconds gitlab-redis 10000 + + ## Define a value for `sentinel failover_timeout` in `ms`. This has multiple + ## meanings: + ## + ## * The time needed to re-start a failover after a previous failover was + ## already tried against the same master by a given Sentinel, is two + ## times the failover timeout. + ## + ## * The time needed for a slave replicating to a wrong master according + ## to a Sentinel current configuration, to be forced to replicate + ## with the right master, is exactly the failover timeout (counting since + ## the moment a Sentinel detected the misconfiguration). + ## + ## * The time needed to cancel a failover that is already in progress but + ## did not produced any configuration change (SLAVEOF NO ONE yet not + ## acknowledged by the promoted slave). + ## + ## * The maximum time a failover in progress waits for all the slaves to be + ## reconfigured as slaves of the new master. However even after this time + ## the slaves will be reconfigured by the Sentinels anyway, but not with + ## the exact parallel-syncs progression as specified. + sentinel failover_timeout 30000 + ``` +1. Restart the Redis service for the changes to take effect. +1. Go through the steps again for all the other Sentinel nodes. + +### Step 4. Configuring the GitLab application + +You can enable or disable Sentinel support at any time in new or existing +installations. From the GitLab application perspective, all it requires is +the correct credentials for the Sentinel nodes. + +While it doesn't require a list of all Sentinel nodes, in case of a failure, +it needs to access at least one of listed ones. + +The following steps should be performed in the [GitLab application server](gitlab.md) +which ideally should not have Redis or Sentinels in the same machine for a HA +setup: + +1. Edit `/home/git/gitlab/config/resque.yml` following the example in + [resque.yml.example][resque], and uncomment the Sentinel lines, pointing to + the correct server credentials: + + ```yaml + # resque.yaml + production: + url: redis://:redi-password-goes-here@gitlab-redis/ + sentinels: + - + host: 10.0.0.1 + port: 26379 # point to sentinel, not to redis port + - + host: 10.0.0.2 + port: 26379 # point to sentinel, not to redis port + - + host: 10.0.0.3 + port: 26379 # point to sentinel, not to redis port + ``` + +1. [Restart GitLab][restart] for the changes to take effect. + +## Example of minimal configuration with 1 master, 2 slaves and 3 Sentinels + +In this example we consider that all servers have an internal network +interface with IPs in the `10.0.0.x` range, and that they can connect +to each other using these IPs. + +In a real world usage, you would also setup firewall rules to prevent +unauthorized access from other machines, and block traffic from the +outside ([Internet][it]). + +For this example, **Sentinel 1** will be configured in the same machine as the +**Redis Master**, **Sentinel 2** and **Sentinel 3** in the same machines as the +**Slave 1** and **Slave 2** respectively. + +Here is a list and description of each **machine** and the assigned **IP**: + +* `10.0.0.1`: Redis Master + Sentinel 1 +* `10.0.0.2`: Redis Slave 1 + Sentinel 2 +* `10.0.0.3`: Redis Slave 2 + Sentinel 3 +* `10.0.0.4`: GitLab application + +Please note that after the initial configuration, if a failover is initiated +by the Sentinel nodes, the Redis nodes will be reconfigured and the **Master** +will change permanently (including in `redis.conf`) from one node to the other, +until a new failover is initiated again. + +The same thing will happen with `sentinel.conf` that will be overridden after the +initial execution, after any new sentinel node starts watching the **Master**, +or a failover promotes a different **Master** node. + +### Example configuration for Redis master and Sentinel 1 + +1. In `/etc/redis/redis.conf`: + + ```conf + bind 10.0.0.1 + port 6379 + requirepass redis-password-goes-here + masterauth redis-password-goes-here + ``` + +1. In `/etc/redis/sentinel.conf`: + + ```conf + bind 10.0.0.1 + port 26379 + sentinel auth-pass gitlab-redis redis-password-goes-here + sentinel monitor gitlab-redis 10.0.0.1 6379 2 + sentinel down-after-milliseconds gitlab-redis 10000 + sentinel failover_timeout 30000 + ``` + +1. Restart the Redis service for the changes to take effect. + +### Example configuration for Redis slave 1 and Sentinel 2 + +1. In `/etc/redis/redis.conf`: + + ```conf + bind 10.0.0.2 + port 6379 + requirepass redis-password-goes-here + masterauth redis-password-goes-here + slaveof 10.0.0.1 6379 + ``` + +1. In `/etc/redis/sentinel.conf`: + + ```conf + bind 10.0.0.2 + port 26379 + sentinel auth-pass gitlab-redis redis-password-goes-here + sentinel monitor gitlab-redis 10.0.0.1 6379 2 + sentinel down-after-milliseconds gitlab-redis 10000 + sentinel failover_timeout 30000 + ``` + +1. Restart the Redis service for the changes to take effect. + +### Example configuration for Redis slave 2 and Sentinel 3 + +1. In `/etc/redis/redis.conf`: + + ```conf + bind 10.0.0.3 + port 6379 + requirepass redis-password-goes-here + masterauth redis-password-goes-here + slaveof 10.0.0.1 6379 + ``` + +1. In `/etc/redis/sentinel.conf`: + + ```conf + bind 10.0.0.3 + port 26379 + sentinel auth-pass gitlab-redis redis-password-goes-here + sentinel monitor gitlab-redis 10.0.0.1 6379 2 + sentinel down-after-milliseconds gitlab-redis 10000 + sentinel failover_timeout 30000 + ``` + +1. Restart the Redis service for the changes to take effect. + +### Example configuration of the GitLab application + +1. Edit `/home/git/gitlab/config/resque.yml`: + + ```yaml + production: + url: redis://:redi-password-goes-here@gitlab-redis/ + sentinels: + - + host: 10.0.0.1 + port: 26379 # point to sentinel, not to redis port + - + host: 10.0.0.2 + port: 26379 # point to sentinel, not to redis port + - + host: 10.0.0.3 + port: 26379 # point to sentinel, not to redis port + ``` + +1. [Restart GitLab][restart] for the changes to take effect. + +## Troubleshooting + +We have a more detailed [Troubleshooting](redis.md#troubleshooting) explained +in the documentation for Omnibus GitLab installations. Here we will list only +the things that are specific to a source installation. + +If you get an error in GitLab like `Redis::CannotConnectError: No sentinels available.`, +there may be something wrong with your configuration files or it can be related +to [this upstream issue][gh-531]. + +You must make sure that `resque.yml` and `sentinel.conf` are configured correctly, +otherwise `redis-rb` will not work properly. + +The `master-group-name` ('gitlab-redis') defined in (`sentinel.conf`) +**must** be used as the hostname in GitLab (`resque.yml`): + +```conf +# sentinel.conf: +sentinel monitor gitlab-redis 10.0.0.1 6379 2 +sentinel down-after-milliseconds gitlab-redis 10000 +sentinel config-epoch gitlab-redis 0 +sentinel leader-epoch gitlab-redis 0 +``` + +```yaml +# resque.yaml +production: + url: redis://:myredispassword@gitlab-redis/ + sentinels: + - + host: 10.0.0.1 + port: 26379 # point to sentinel, not to redis port + - + host: 10.0.0.2 + port: 26379 # point to sentinel, not to redis port + - + host: 10.0.0.3 + port: 26379 # point to sentinel, not to redis port +``` + +When in doubt, please read [Redis Sentinel documentation](http://redis.io/topics/sentinel). + +[gh-531]: https://github.com/redis/redis-rb/issues/531 +[downloads]: https://about.gitlab.com/downloads +[restart]: ../restart_gitlab.md#installations-from-source +[it]: https://gitlab.com/gitlab-org/gitlab-ce/uploads/c4cc8cd353604bd80315f9384035ff9e/The_Internet_IT_Crowd.png diff --git a/doc/administration/operations.md b/doc/administration/operations.md index 4b582d16b64..0daceb98d99 100644 --- a/doc/administration/operations.md +++ b/doc/administration/operations.md @@ -1,6 +1,7 @@ # GitLab operations - [Sidekiq MemoryKiller](operations/sidekiq_memory_killer.md) +- [Sidekiq Job throttling](operations/sidekiq_job_throttling.md) - [Cleaning up Redis sessions](operations/cleaning_up_redis_sessions.md) - [Understanding Unicorn and unicorn-worker-killer](operations/unicorn.md) - [Moving repositories to a new location](operations/moving_repositories.md) diff --git a/doc/administration/operations/img/sidekiq_job_throttling.png b/doc/administration/operations/img/sidekiq_job_throttling.png Binary files differnew file mode 100644 index 00000000000..7f29a4d3c46 --- /dev/null +++ b/doc/administration/operations/img/sidekiq_job_throttling.png diff --git a/doc/administration/operations/sidekiq_job_throttling.md b/doc/administration/operations/sidekiq_job_throttling.md new file mode 100644 index 00000000000..ddeaa22e288 --- /dev/null +++ b/doc/administration/operations/sidekiq_job_throttling.md @@ -0,0 +1,33 @@ +# Sidekiq Job throttling + +> Note: Introduced with GitLab 8.14 + +When your GitLab installation needs to handle tens of thousands of background +jobs, it can be convenient to throttle queues that do not need to be executed +immediately, e.g. long running jobs like Pipelines, thus allowing jobs that do +need to be executed immediately to have access to more resources. + +In order to accomplish this, you can limit the amount of workers that certain +slow running queues can have available. This is what we call Sidekiq Job +Throttling. Depending on your infrastructure, you might have different slow +running queues, which is why you can choose which queues you want to throttle +and by how much you want to throttle them. + +These settings are available in the Application Settings of your GitLab +installation. + +![Sidekiq Job Throttling](img/sidekiq_job_throttling.png) + +The throttle factor determines the maximum number of workers a queue can run on. +This value gets multiplied by `:concurrency` value set in the Sidekiq settings +and rounded up to the closest full integer. + +So, for example, you set the `:concurrency` to 25 and the `Throttling factor` to +0.1, the maximum workers assigned to the selected queues would be 3. + +```ruby +queue_limit = (factor * Sidekiq.options[:concurrency]).ceil +``` + +After enabling the job throttling, you will need to restart your GitLab +instance, in order for the changes to take effect.
\ No newline at end of file diff --git a/doc/administration/reply_by_email.md b/doc/administration/reply_by_email.md index 5a9a1582877..14cd7a03826 100644 --- a/doc/administration/reply_by_email.md +++ b/doc/administration/reply_by_email.md @@ -105,6 +105,8 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow # The mailbox where incoming mail will end up. Usually "inbox". gitlab_rails['incoming_email_mailbox_name'] = "inbox" + # The IDLE command timeout. + gitlab_rails['incoming_email_idle_timeout'] = 60 ``` ```ruby @@ -133,6 +135,8 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow # The mailbox where incoming mail will end up. Usually "inbox". gitlab_rails['incoming_email_mailbox_name'] = "inbox" + # The IDLE command timeout. + gitlab_rails['incoming_email_idle_timeout'] = 60 ``` 1. Reconfigure GitLab and restart mailroom for the changes to take effect: @@ -192,6 +196,8 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow # The mailbox where incoming mail will end up. Usually "inbox". mailbox: "inbox" + # The IDLE command timeout. + idle_timeout: 60 ``` ```yaml @@ -221,6 +227,8 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow # The mailbox where incoming mail will end up. Usually "inbox". mailbox: "inbox" + # The IDLE command timeout. + idle_timeout: 60 ``` 1. Enable `mail_room` in the init script at `/etc/default/gitlab`: @@ -277,6 +285,8 @@ To set up a basic Postfix mail server with IMAP access on Ubuntu, follow # The mailbox where incoming mail will end up. Usually "inbox". mailbox: "inbox" + # The IDLE command timeout. + idle_timeout: 60 ``` As mentioned, the part after `+` is ignored, and this will end up in the mailbox for `gitlab-incoming@gmail.com`. diff --git a/doc/api/branches.md b/doc/api/branches.md index 0b5f7778fc7..f68eeb9f86b 100644 --- a/doc/api/branches.md +++ b/doc/api/branches.md @@ -240,3 +240,21 @@ Example response: "branch_name": "newbranch" } ``` + +## Delete merged branches + +Will delete all branches that are merged into the project's default branch. + +``` +DELETE /projects/:id/repository/merged_branches +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of a project | + +It returns `200` to indicate deletion of all merged branches was started. + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" "https://gitlab.example.com/api/v3/projects/5/repository/merged_branches" +``` diff --git a/doc/api/builds.md b/doc/api/builds.md index 0476cac0eda..bca2f9e44ef 100644 --- a/doc/api/builds.md +++ b/doc/api/builds.md @@ -45,7 +45,7 @@ Example of response "ref": "master", "sha": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd", "status": "pending" - } + }, "ref": "master", "runner": null, "stage": "test", @@ -89,7 +89,7 @@ Example of response "ref": "master", "sha": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd", "status": "pending" - } + }, "ref": "master", "runner": null, "stage": "test", @@ -163,7 +163,7 @@ Example of response "ref": "master", "sha": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd", "status": "pending" - } + }, "ref": "master", "runner": null, "stage": "test", @@ -193,7 +193,7 @@ Example of response "ref": "master", "sha": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd", "status": "pending" - } + }, "ref": "master", "runner": null, "stage": "test", @@ -260,7 +260,7 @@ Example of response "ref": "master", "sha": "0ff3ae198f8601a285adcf5c0fff204ee6fba5fd", "status": "pending" - } + }, "ref": "master", "runner": null, "stage": "test", diff --git a/doc/api/repositories.md b/doc/api/repositories.md index b6cca5d4e2a..bcf8b955044 100644 --- a/doc/api/repositories.md +++ b/doc/api/repositories.md @@ -13,44 +13,58 @@ Parameters: - `id` (required) - The ID of a project - `path` (optional) - The path inside repository. Used to get contend of subdirectories - `ref_name` (optional) - The name of a repository branch or tag or if not given the default branch +- `recursive` (optional) - Boolean value used to get a recursive tree (false by default) ```json [ { - "name": "assets", + "id": "a1e8f8d745cc87e3a9248358d9352bb7f9a0aeba", + "name": "html", "type": "tree", - "mode": "040000", - "id": "6229c43a7e16fcc7e95f923f8ddadb8281d9c6c6" + "path": "files/html", + "mode": "040000" }, { - "name": "contexts", + "id": "4535904260b1082e14f867f7a24fd8c21495bde3", + "name": "images", "type": "tree", - "mode": "040000", - "id": "faf1cdf33feadc7973118ca42d35f1e62977e91f" + "path": "files/images", + "mode": "040000" }, { - "name": "controllers", + "id": "31405c5ddef582c5a9b7a85230413ff90e2fe720", + "name": "js", "type": "tree", - "mode": "040000", - "id": "95633e8d258bf3dfba3a5268fb8440d263218d74" + "path": "files/js", + "mode": "040000" }, { - "name": "Rakefile", - "type": "blob", - "mode": "100644", - "id": "35b2f05cbb4566b71b34554cf184a9d0bd9d46d6" + "id": "cc71111cfad871212dc99572599a568bfe1e7e00", + "name": "lfs", + "type": "tree", + "path": "files/lfs", + "mode": "040000" }, { - "name": "VERSION", - "type": "blob", - "mode": "100644", - "id": "803e4a4f3727286c3093c63870c2b6524d30ec4f" + "id": "fd581c619bf59cfdfa9c8282377bb09c2f897520", + "name": "markdown", + "type": "tree", + "path": "files/markdown", + "mode": "040000" + }, + { + "id": "23ea4d11a4bdd960ee5320c5cb65b5b3fdbc60db", + "name": "ruby", + "type": "tree", + "path": "files/ruby", + "mode": "040000" }, { - "name": "config.ru", + "id": "7d70e02340bac451f281cecf0a980907974bd8be", + "name": "whitespace", "type": "blob", - "mode": "100644", - "id": "dfd2d862237323aa599be31b473d70a8a817943b" + "path": "files/whitespace", + "mode": "100644" } ] ``` diff --git a/doc/ci/README.md b/doc/ci/README.md index 6b90940c047..545cc72682d 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -1,6 +1,6 @@ -## GitLab CI Documentation +# GitLab CI Documentation -### CI User documentation +## CI User documentation - [Get started with GitLab CI](quick_start/README.md) - [CI examples for various languages](examples/README.md) @@ -20,4 +20,8 @@ - [API](../api/ci/README.md) - [CI services (linked docker containers)](services/README.md) - [CI/CD pipelines settings](../user/project/pipelines/settings.md) -- [**New CI build permissions model**](../user/project/new_ci_build_permissions_model.md) Read about what changed in GitLab 8.12 and how that affects your builds. There's a new way to access your Git submodules and LFS objects in builds. +- [Review Apps](review_apps/index.md) + +## Breaking changes + +- [New CI build permissions model](../user/project/new_ci_build_permissions_model.md) Read about what changed in GitLab 8.12 and how that affects your builds. There's a new way to access your Git submodules and LFS objects in builds. diff --git a/doc/ci/environments.md b/doc/ci/environments.md index e070302fb82..cfb41307c43 100644 --- a/doc/ci/environments.md +++ b/doc/ci/environments.md @@ -3,69 +3,517 @@ >**Note:** Introduced in GitLab 8.9. -## Environments +During the development of software, there can be many stages until it's ready +for public consumption. You sure want to first test your code and then deploy it +in a testing or staging environment before you release it to the public. That +way you can prevent bugs not only in your software, but in the deployment +process as well. -Environments are places where code gets deployed, such as staging or production. -CI/CD [Pipelines] usually have one or more [jobs] that deploy to an environment. -Defining environments in a project's `.gitlab-ci.yml` lets developers track -[deployments] to these environments. +GitLab CI is capable of not only testing or building your projects, but also +deploying them in your infrastructure, with the added benefit of giving you a +way to track your deployments. In other words, you can always know what is +currently being deployed or has been deployed on your servers. -## Deployments +## Overview -Deployments are created when [jobs] deploy versions of code to [environments]. +With environments, you can control the Continuous Deployment of your software +all within GitLab. All you need to do is define them in your project's +[`.gitlab-ci.yml`][yaml] as we will explore below. GitLab provides a full +history of your deployments per every environment. -### Checkout deployments locally +Environments are like tags for your CI jobs, describing where code gets deployed. +Deployments are created when [jobs] deploy versions of code to environments, +so every environment can have one or more deployments. GitLab keeps track of +your deployments, so you always know what is currently being deployed on your +servers. -Since 8.13, a reference in the git repository is saved for each deployment. So -knowing what the state is of your current environments is only a `git fetch` -away. +To better understand how environments and deployments work, let's consider an +example. We assume that you have already created a project in GitLab and set up +a Runner. The example will cover the following: -In your git config, append the `[remote "<your-remote>"]` block with an extra -fetch line: +- We are developing an application +- We want to run tests and build our app on all branches +- Our default branch is `master` +- We deploy the app only when a pipeline on `master` branch is run -``` -fetch = +refs/environments/*:refs/remotes/origin/environments/* -``` +Let's see how it all ties together. ## Defining environments -You can create and delete environments manually in the web interface, but we -recommend that you define your environments in `.gitlab-ci.yml` first, which -will automatically create environments for you after the first deploy. +Let's consider the following `.gitlab-ci.yml` example: -The `environment` is just a hint for GitLab that this job actually deploys to -this environment. Each time the job succeeds, a deployment is recorded, -remembering the git SHA and environment. +```yaml +stages: + - test + - build + - deploy -Add something like this to your `.gitlab-ci.yml`: +test: + stage: test + script: echo "Running tests" + +build: + stage: build + script: echo "Building the app" + +deploy_staging: + stage: deploy + script: + - echo "Deploy to staging server" + environment: + name: staging + url: https://staging.example.com + only: + - master ``` -production: + +We have defined 3 [stages](yaml/README.md#stages): + +- test +- build +- deploy + +The jobs assigned to these stages will run in this order. If a job fails, then +the builds that are assigned to the next stage won't run, rendering the pipeline +as failed. In our case, the `test` job will run first, then the `build` and +lastly the `deploy_staging`. With this, we ensure that first the tests pass, +then our app is able to be built successfully, and lastly we deploy to the +staging server. + +The `environment` keyword is just a hint for GitLab that this job actually +deploys to this environment's `name`. It can also have a `url` which, as we +will later see, is exposed in various places within GitLab. Each time a job that +has an environment specified and succeeds, a deployment is recorded, remembering +the Git SHA and environment name. + +To sum up, with the above `.gitlab-ci.yml` we have achieved that: + +- All branches will run the `test` and `build` jobs. +- The `deploy_staging` job will run [only](yaml/README.md#only) on the `master` + branch which means all merge requests that are created from branches don't + get to deploy to the staging server +- When a merge request is merged, all jobs will run and the `deploy_staging` + in particular will deploy our code to a staging server while the deployment + will be recorded in an environment named `staging`. + +Let's now see how that information is exposed within GitLab. + +## Viewing the current status of an environment + +The environment list under your project's **Pipelines ➔ Environments**, is +where you can find information of the last deployment status of an environment. + +Here's how the Environments page looks so far. + +![Staging environment view](img/environments_available_staging.png) + +There's a bunch of information there, specifically you can see: + +- The environment's name with a link to its deployments +- The last deployment ID number and who performed it +- The build ID of the last deployment with its respective job name +- The commit information of the last deployment such as who committed, to what + branch and the Git SHA of the commit +- The exact time the last deployment was performed +- A button that takes you to the URL that you have defined under the + `environment` keyword in `.gitlab-ci.yml` +- A button that re-deploys the latest deployment, meaning it runs the job + defined by the environment name for that specific commit + +>**Notes:** +- While you can create environments manually in the web interface, we recommend + that you define your environments in `.gitlab-ci.yml` first. They will + be automatically created for you after the first deploy. +- The environments page can only be viewed by Reporters and above. For more + information on the permissions, see the [permissions documentation][permissions]. +- Only deploys that happen after your `.gitlab-ci.yml` is properly configured + will show up in the "Environment" and "Last deployment" lists. + +The information shown in the Environments page is limited to the latest +deployments, but as you may have guessed an environment can have multiple +deployments. + +## Viewing the deployment history of an environment + +GitLab keeps track of your deployments, so you always know what is currently +being deployed on your servers. That way you can have the full history of your +deployments per every environment right in your browser. Clicking on an +environment will show the history of its deployments. Assuming you have deployed +multiple times already, here's how a specific environment's page looks like. + +![Deployments](img/deployments_view.png) + +We can see the same information as when in the Environments page, but this time +all deployments are shown. As you may have noticed, apart from the **Re-deploy** +button there are now **Rollback** buttons for each deployment. Let's see how +that works. + +## Rolling back changes + +You can't control everything, so sometimes things go wrong. When that unfortunate +time comes GitLab has you covered. Simply by clicking the **Rollback** button +that can be found in the deployments page +(**Pipelines ➔ Environments ➔ `environment name`**) you can relaunch the +job with the commit associated with it. + +>**Note:** +Bare in mind that your mileage will vary and it's entirely up to how you define +the deployment process in the job's `script` whether the rollback succeeds or not. +GitLab CI is just following orders. + +Thankfully that was the staging server that we had to rollback, and since we +learn from our mistakes, we decided to not make the same again when we deploy +to the production server. Enter manual actions for deployments. + +## Manually deploying to environments + +Turning a job from running automatically to a manual action is as simple as +adding `when: manual` to it. To expand on our previous example, let's add +another job that this time deploys our app to a production server and is +tracked by a `production` environment. The `.gitlab-ci.yml` looks like this +so far: + +```yaml +stages: + - test + - build + - deploy + +test: + stage: test + script: echo "Running tests" + +build: + stage: build + script: echo "Building the app" + +deploy_staging: + stage: deploy + script: + - echo "Deploy to staging server" + environment: + name: staging + url: https://staging.example.com + only: + - master + +deploy_prod: stage: deploy - script: dpl... - environment: production + script: + - echo "Deploy to production server" + environment: + name: production + url: https://example.com + when: manual + only: + - master ``` -See full [documentation](yaml/README.md#environment). +The `when: manual` action exposes a play button in GitLab's UI and the +`deploy_prod` job will only be triggered if and when we click that play button. +You can find it in the pipeline, build, environment, and deployment views. -## Seeing environment status +| Pipelines | Single pipeline | Environments | Deployments | Builds | +| --------- | ----------------| ------------ | ----------- | -------| +| ![Pipelines manual action](img/environments_manual_action_pipelines.png) | ![Pipelines manual action](img/environments_manual_action_single_pipeline.png) | ![Environments manual action](img/environments_manual_action_environments.png) | ![Deployments manual action](img/environments_manual_action_deployments.png) | ![Builds manual action](img/environments_manual_action_builds.png) | -You can find the environment list under **Pipelines > Environments** for your -project. You'll see the git SHA and date of the last deployment to each -environment defined. +Clicking on the play button in either of these places will trigger the +`deploy_prod` job, and the deployment will be recorded under a new +environment named `production`. >**Note:** -Only deploys that happen after your `.gitlab-ci.yml` is properly configured will -show up in the environments and deployments lists. +Remember that if your environment's name is `production` (all lowercase), then +it will get recorded in [Cycle Analytics](../user/project/cycle_analytics.md). +Double the benefit! + +While this is fine for deploying to some stable environments like staging or +production, what happens for branches? So far we haven't defined anything +regarding deployments for branches other than `master`. Dynamic environments +will help us achieve that. + +## Dynamic environments + +As the name suggests, it is possible to create environments on the fly by just +declaring their names dynamically in `.gitlab-ci.yml`. Dynamic environments is +the base of [Review apps](review_apps.md). -## Seeing deployment history +GitLab Runner exposes various [environment variables][variables] when a job runs, +and as such, you can use them as environment names. Let's add another job in +our example which will deploy to all branches except `master`: -Clicking on an environment will show the history of deployments. +```yaml +deploy_review: + stage: deploy + script: + - echo "Deploy a review app" + environment: + name: review/$CI_BUILD_REF_NAME + url: https://$CI_BUILD_REF_NAME.example.com + only: + - branches + except: + - master +``` + +Let's break it down in pieces. The job's name is `deploy_review` and it runs +on the `deploy` stage. The `script` at this point is fictional, you'd have to +use your own based on your deployment. Then, we set the `environment` with the +`environment:name` being `review/$CI_BUILD_REF_NAME`. Now that's an interesting +one. Since the [environment name][env-name] can contain also slashes (`/`), we +can use this pattern to distinguish between dynamic environments and the regular +ones. + +So, the first part is `review`, followed by a `/` and then `$CI_BUILD_REF_NAME` +which takes the value of the branch name. We also use the same +`$CI_BUILD_REF_NAME` value in the `environment:url` so that the environment +can get a specific and distinct URL for each branch. Again, the way you set up +the webserver to serve these requests is based on your setup. + +Last but not least, we tell the job to run [`only`][only] on branches +[`except`][only] master. >**Note:** -Only deploys that happen after your `.gitlab-ci.yml` is properly configured will -show up in the environments and deployments lists. +You are not bound to use the same prefix or only slashes in the dynamic +environments' names (`/`), but as we will see later, this will enable the +[grouping similar environments](#grouping-similar-environments) feature. + +The whole `.gitlab-ci.yml` looks like this so far: + +```yaml +stages: + - test + - build + - deploy + +test: + stage: test + script: echo "Running tests" + +build: + stage: build + script: echo "Building the app" + +deploy_review: + stage: deploy + script: + - echo "Deploy a review app" + environment: + name: review/$CI_BUILD_REF_NAME + url: https://$CI_BUILD_REF_NAME.example.com + only: + - branches + except: + - master + +deploy_staging: + stage: deploy + script: + - echo "Deploy to staging server" + environment: + name: staging + url: https://staging.example.com + only: + - master + +deploy_prod: + stage: deploy + script: + - echo "Deploy to production server" + environment: + name: production + url: https://example.com + when: manual + only: + - master +``` + +A more realistic example would include copying files to a location where a +webserver (NGINX) could then read and serve. The example below will copy the +`public` directory to `/srv/nginx/$CI_BUILD_REF_NAME/public`: + +```yaml +review_app: + stage: deploy + script: + - rsync -av --delete public /srv/nginx/$CI_BUILD_REF_NAME + environment: + name: review/$CI_BUILD_REF_NAME + url: https://$CI_BUILD_REF_NAME.example.com +``` + +It is assumed that the user has already setup NGINX and GitLab Runner in the +server this job will run on. + +--- + +The development workflow would now be: + +- Developer creates a branch locally +- Developer makes changes, commits and pushes the branch to GitLab +- Developer creates a merge request + +Behind the scenes: + +- GitLab Runner picks up the changes and starts running the jobs +- The jobs run sequentially as defined in `stages` + - First, the tests pass + - Then, the build begins and successfully also passes + - Lastly, the app is deployed to an environment with a name specific to the + branch + +So now, every branch gets its own environment and is deployed to its own place +with the added benefit of having a [history of deployments](#viewing-the-deployment-history-of-an-environment) +and also being able to [rollback changes](#rolling-back-changes) if needed. +Let's briefly see where URL that's defined in the environments is exposed. + +## Making use of the environment URL + +The environment URL is exposed in a few places within GitLab. + +| In a merge request widget as a link | In the Environments view as a button | In the Deployments view as a button | +| -------------------- | ------------ | ----------- | +| ![Environment URL in merge request](img/environments_mr_review_app.png) | ![Environment URL in environments](img/environments_link_url.png) | ![Environment URL in deployments](img/environments_link_url_deployments.png) | + +If a merge request is eventually merged to the default branch (in our case +`master`) and that branch also deploys to an environment (in our case `staging` +and/or `production`) you can see this information in the merge request itself. + +![Environment URLs in merge request](img/environments_link_url_mr.png) + +--- + +We now have a full development cycle, where our app is tested, built, deployed +as a Review app, deployed to a staging server once the merge request is merged, +and finally manually deployed to the production server. What we just described +is a single workflow, but imagine tens of developers working on a project +at the same time. They each push to their branches, and dynamic environments are +created all the time. In that case, we probably need to do some clean up. Read +next how environments can be stopped. + +## Stopping an environment + +By stopping an environment, you are effectively terminating its recording of the +deployments that happen in it. + +A branch is associated with an environment when the CI pipeline that is created +for this branch, was recently deployed to this environment. You can think of +the CI pipeline as the glue between the branch and the environment: +`branch ➔ CI pipeline ➔ environment`. + +There is a special case where environments can be manually stopped. That can +happen if you provide another job for that matter. The syntax is a little +tricky since a job calls another job to do the job. + +Consider the following example where the `deploy_review` calls the `stop_review` +to clean up and stop the environment: + +```yaml +deploy_review: + stage: deploy + script: + - echo "Deploy a review app" + environment: + name: review/$CI_BUILD_REF_NAME + url: https://$CI_BUILD_REF_NAME.example.com + on_stop: stop_review + only: + - branches + except: + - master + +stop_review: + variables: + GIT_STRATEGY: none + script: + - echo "Remove review app" + when: manual + environment: + name: review/$CI_BUILD_REF_NAME + action: stop +``` + +Setting the [`GIT_STRATEGY`][git-strategy] to `none` is necessary on the +`stop_review` job so that the [GitLab Runner] won't try to checkout the code +after the branch is deleted. + +>**Note:** +Starting with GitLab 8.14, dynamic environments will be stopped automatically +when their associated branch is deleted. + +When you have an environment that has a stop action defined (typically when +the environment describes a review app), GitLab will automatically trigger a +stop action when the associated branch is deleted. + +You can read more in the [`.gitlab-ci.yml` reference][onstop]. + +## Grouping similar environments + +> [Introduced][ce-7015] in GitLab 8.14. + +As we've seen in the [dynamic environments](#dynamic-environments), you can +prepend their name with a word, then followed by a `/` and finally the branch +name which is automatically defined by the `CI_BUILD_REF_NAME` variable. + +In short, environments that are named like `type/foo` are presented under a +group named `type`. + +In our minimal example, we name the environments `review/$CI_BUILD_REF_NAME` +where `$CI_BUILD_REF_NAME` is the branch name: + +```yaml +deploy_review: + stage: deploy + script: + - echo "Deploy a review app" + environment: + name: review/$CI_BUILD_REF_NAME +``` + +In that case, if you visit the Environments page, and provided the branches +exist, you should see something like: + +![Environment groups](img/environments_dynamic_groups.png) + +## Checkout deployments locally + +Since 8.13, a reference in the git repository is saved for each deployment. So +knowing what the state is of your current environments is only a `git fetch` +away. + +In your git config, append the `[remote "<your-remote>"]` block with an extra +fetch line: + +``` +fetch = +refs/environments/*:refs/remotes/origin/environments/* +``` + +## Limitations + +- You are limited to use only the [CI predefined variables][variables] in the + `environment: name`. Any variables defined inside `script` will not work. +- If the branch name contains special characters and you use the + `$CI_BUILD_REF_NAME` variable to dynamically create environments, there might + be complications during deployment. Follow the [issue 22849][ce-22849] for + more information. + +## Further reading + +Below are some links you may find interesting: + +- [The `.gitlab-ci.yml` definition of environments](yaml/README.md#environment) +- [A blog post on Deployments & Environments](https://about.gitlab.com/2016/08/26/ci-deployment-and-environments/) +- [Review Apps - Use dynamic environments to deploy your code for every branch](review_apps/index.md) [Pipelines]: pipelines.md [jobs]: yaml/README.md#jobs +[yaml]: yaml/README.md [environments]: #environments [deployments]: #deployments +[permissions]: ../user/permissions.md +[variables]: variables/README.md +[env-name]: yaml/README.md#environment-name +[only]: yaml/README.md#only-and-except +[onstop]: yaml/README.md#environment-on_stop +[ce-7015]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7015 +[gitlab runner]: https://docs.gitlab.com/runner/ +[git-strategy]: yaml/README.md#git-strategy diff --git a/doc/ci/img/deployments_view.png b/doc/ci/img/deployments_view.png Binary files differnew file mode 100644 index 00000000000..ca6097cbea4 --- /dev/null +++ b/doc/ci/img/deployments_view.png diff --git a/doc/ci/img/environments_available_staging.png b/doc/ci/img/environments_available_staging.png Binary files differnew file mode 100644 index 00000000000..784c4fd944c --- /dev/null +++ b/doc/ci/img/environments_available_staging.png diff --git a/doc/ci/img/environments_dynamic_groups.png b/doc/ci/img/environments_dynamic_groups.png Binary files differnew file mode 100644 index 00000000000..e89b66c502c --- /dev/null +++ b/doc/ci/img/environments_dynamic_groups.png diff --git a/doc/ci/img/environments_link_url.png b/doc/ci/img/environments_link_url.png Binary files differnew file mode 100644 index 00000000000..224c21adfb5 --- /dev/null +++ b/doc/ci/img/environments_link_url.png diff --git a/doc/ci/img/environments_link_url_deployments.png b/doc/ci/img/environments_link_url_deployments.png Binary files differnew file mode 100644 index 00000000000..9419668a9bd --- /dev/null +++ b/doc/ci/img/environments_link_url_deployments.png diff --git a/doc/ci/img/environments_link_url_mr.png b/doc/ci/img/environments_link_url_mr.png Binary files differnew file mode 100644 index 00000000000..3276dfb6096 --- /dev/null +++ b/doc/ci/img/environments_link_url_mr.png diff --git a/doc/ci/img/environments_manual_action_builds.png b/doc/ci/img/environments_manual_action_builds.png Binary files differnew file mode 100644 index 00000000000..d4bb7ccdbae --- /dev/null +++ b/doc/ci/img/environments_manual_action_builds.png diff --git a/doc/ci/img/environments_manual_action_deployments.png b/doc/ci/img/environments_manual_action_deployments.png Binary files differnew file mode 100644 index 00000000000..c2477381c80 --- /dev/null +++ b/doc/ci/img/environments_manual_action_deployments.png diff --git a/doc/ci/img/environments_manual_action_environments.png b/doc/ci/img/environments_manual_action_environments.png Binary files differnew file mode 100644 index 00000000000..56601c0db2d --- /dev/null +++ b/doc/ci/img/environments_manual_action_environments.png diff --git a/doc/ci/img/environments_manual_action_pipelines.png b/doc/ci/img/environments_manual_action_pipelines.png Binary files differnew file mode 100644 index 00000000000..eb6e87cd956 --- /dev/null +++ b/doc/ci/img/environments_manual_action_pipelines.png diff --git a/doc/ci/img/environments_manual_action_single_pipeline.png b/doc/ci/img/environments_manual_action_single_pipeline.png Binary files differnew file mode 100644 index 00000000000..9713ad212e2 --- /dev/null +++ b/doc/ci/img/environments_manual_action_single_pipeline.png diff --git a/doc/ci/img/environments_mr_review_app.png b/doc/ci/img/environments_mr_review_app.png Binary files differnew file mode 100644 index 00000000000..a2ae25d62fa --- /dev/null +++ b/doc/ci/img/environments_mr_review_app.png diff --git a/doc/ci/img/environments_view.png b/doc/ci/img/environments_view.png Binary files differnew file mode 100644 index 00000000000..131a9718cc4 --- /dev/null +++ b/doc/ci/img/environments_view.png diff --git a/doc/ci/review_apps/img/review_apps_preview_in_mr.png b/doc/ci/review_apps/img/review_apps_preview_in_mr.png Binary files differnew file mode 100644 index 00000000000..15bcb90518c --- /dev/null +++ b/doc/ci/review_apps/img/review_apps_preview_in_mr.png diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md new file mode 100644 index 00000000000..b41ae130bc2 --- /dev/null +++ b/doc/ci/review_apps/index.md @@ -0,0 +1,124 @@ +# Getting started with Review Apps + +> +- [Introduced][ce-21971] in GitLab 8.12. Further additions were made in GitLab + 8.13 and 8.14. +- Inspired by [Heroku's Review Apps][heroku-apps] which itself was inspired by + [Fourchette]. + +The base of Review Apps is the [dynamic environments] which allow you to create +a new environment (dynamically) for each one of your branches. + +A Review App can then be visible as a link when you visit the [merge request] +relevant to the branch. That way, you are able to see live all changes introduced +by the merge request changes. Reviewing anything, from performance to interface +changes, becomes much easier with a live environment and as such, Review Apps +can make a huge impact on your development flow. + +They mostly make sense to be used with web applications, but you can use them +any way you'd like. + +## Overview + +Simply put, a Review App is a mapping of a branch with an environment as there +is a 1:1 relation between them. + +Here's an example of what it looks like when viewing a merge request with a +dynamically set environment. + +![Review App in merge request](img/review_apps_preview_in_mr.png) + +In the image above you can see that the `add-new-line` branch was successfully +built and deployed under a dynamic environment and can be previewed with an +also dynamically URL. + +The details of the Review Apps implementation depend widely on your real +technology stack and on your deployment process. The simplest case it to +deploy a simple static HTML website, but it will not be that straightforward +when your app is using a database for example. To make a branch be deployed +on a temporary instance and booting up this instance with all required software +and services automatically on the fly is not a trivial task. However, it is +doable, especially if you use Docker, or at least a configuration management +tool like Chef, Puppet, Ansible or Salt. + +## Prerequisites + +To get a better understanding of Review Apps, you must first learn how +environments and deployments work. The following docs will help you grasp that +knowledge: + +1. First, learn about [environments][] and their role in the development workflow. +1. Then make a small stop to learn about [CI variables][variables] and how they + can be used in your CI jobs. +1. Next, explore the [`environment` syntax][yaml-env] as defined in `.gitlab-ci.yml`. + This will be your primary reference when you are finally comfortable with + how environments work. +1. Additionally, find out about [manual actions][] and how you can use them to + deploy to critical environments like production with the push of a button. +1. And as a last step, follow the [example tutorials](#examples) which will + guide you step by step to set up the infrastructure and make use of + Review Apps. + +## Configuration + +The configuration of Review apps depends on your technology stack and your +infrastructure. Read the [dynamic environments] documentation to understand +how to define and create them. + +## Creating and destroying Review Apps + +The creation and destruction of a Review App is defined in `.gitlab-ci.yml` +at a job level under the `environment` keyword. + +Check the [environments] documentation how to do so. + +## A simple workflow + +The process of adding Review Apps in your workflow would look like: + +1. Set up the infrastructure to host and deploy the Review Apps. +1. [Install][install-runner] and [configure][conf-runner] a Runner that does + the deployment. +1. Set up a job in `.gitlab-ci.yml` that uses the predefined + [predefined CI environment variable][variables] `${CI_BUILD_REF_NAME}` to + create dynamic environments and restrict it to run only on branches. +1. Optionally set a job that [manually stops][manual-env] the Review Apps. + +From there on, you would follow the branched Git flow: + +1. Push a branch and let the Runner deploy the Review App based on the `script` + definition of the dynamic environment job. +1. Wait for the Runner to build and/or deploy your web app. +1. Click on the link that's present in the MR related to the branch and see the + changes live. + +## Limitations + +Check the [environments limitations](../environments.md#limitations). + +## Examples + +A list of examples used with Review Apps can be found below: + +- [Use with NGINX][app-nginx] - Use NGINX and the shell executor of GitLab Runner + to deploy a simple HTML website. + +And below is a soon to be added examples list: + +- Use with Amazon S3 +- Use on Heroku with dpl +- Use with OpenShift/kubernetes + +[app-nginx]: https://gitlab.com/gitlab-examples/review-apps-nginx +[ce-21971]: https://gitlab.com/gitlab-org/gitlab-ce/issues/21971 +[dynamic environments]: ../environments.md#dynamic-environments +[environments]: ../environments.md +[fourchette]: https://github.com/rainforestapp/fourchette +[heroku-apps]: https://devcenter.heroku.com/articles/github-integration-review-apps +[manual actions]: ../environments.md#manual-actions +[merge request]: ../../user/project/merge_requests.md +[variables]: ../variables/README.md +[yaml-env]: ../yaml/README.md#environment +[install-runner]: https://docs.gitlab.com/runner/install/ +[conf-runner]: https://docs.gitlab.com/runner/commands/ +[manual-env]: ../environments.md#stopping-an-environment diff --git a/doc/ci/triggers/README.md b/doc/ci/triggers/README.md index 84048f1d25f..cf7c55f75f2 100644 --- a/doc/ci/triggers/README.md +++ b/doc/ci/triggers/README.md @@ -58,6 +58,22 @@ below. See the [Examples](#examples) section for more details on how to actually trigger a rebuild. +## Trigger a build from webhook + +> Introduced in GitLab 8.14. + +To trigger a build from webhook of another project you need to add the following +webhook url for Push and Tag push events: + +``` +https://gitlab.example.com/api/v3/projects/:id/ref/:ref/trigger/builds?token=TOKEN +``` + +> **Note**: +- `ref` should be passed as part of url in order to take precedence over `ref` + from webhook body that designates the branchref that fired the trigger in the source repository. +- `ref` should be url encoded if contains slashes. + ## Pass build variables to a trigger You can pass any number of arbitrary variables in the trigger API call and they @@ -169,6 +185,14 @@ curl --request POST \ https://gitlab.example.com/api/v3/projects/9/trigger/builds ``` +### Using webhook to trigger builds + +You can add the following webhook to another project in order to trigger a build: + +``` +https://gitlab.example.com/api/v3/projects/9/ref/master/trigger/builds?token=TOKEN&variables[UPLOAD_TO_S3]=true +``` + ### Using cron to trigger nightly builds Whether you craft a script or just run cURL directly, you can trigger builds diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 5c0e1c44e3f..6fee750c709 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -541,6 +541,8 @@ same manual action multiple times. An example usage of manual actions is deployment to production. +Read more at the [environments documentation][env-manual]. + ### environment > Introduced in GitLab 8.9. @@ -552,28 +554,14 @@ An example usage of manual actions is deployment to production. If `environment` is specified and no environment under that name exists, a new one will be created automatically. -The `environment` name can contain: - -- letters -- digits -- spaces -- `-` -- `_` -- `/` -- `$` -- `{` -- `}` - -Common names are `qa`, `staging`, and `production`, but you can use whatever -name works with your workflow. - In its simplest form, the `environment` keyword can be defined like: ``` deploy to production: stage: deploy script: git push production HEAD:master - environment: production + environment: + name: production ``` In the above example, the `deploy to production` job will be marked as doing a @@ -588,6 +576,21 @@ Before GitLab 8.11, the name of an environment could be defined as a string like `environment: production`. The recommended way now is to define it under the `name` keyword. +The `environment` name can contain: + +- letters +- digits +- spaces +- `-` +- `_` +- `/` +- `$` +- `{` +- `}` + +Common names are `qa`, `staging`, and `production`, but you can use whatever +name works with your workflow. + Instead of defining the name of the environment right after the `environment` keyword, it is also possible to define it as a separate value. For that, use the `name` keyword under `environment`: @@ -626,7 +629,12 @@ deploy to production: #### environment:on_stop -> [Introduced][ce-6669] in GitLab 8.13. +> +**Notes:** +- [Introduced][ce-6669] in GitLab 8.13. +- Starting with GitLab 8.14, when you have an environment that has a stop action + defined, GitLab will automatically trigger a stop action when the associated + branch is deleted. Closing (stoping) environments can be achieved with the `on_stop` keyword defined under `environment`. It declares a different job that runs in order to close @@ -681,6 +689,13 @@ The `stop_review_app` job is **required** to have the following keywords defined These parameters can use any of the defined [CI variables](#variables) (including predefined, secure variables and `.gitlab-ci.yml` variables). +>**Note:** +Be aware than if the branch name contains special characters and you use the +`$CI_BUILD_REF_NAME` variable to dynamically create environments, there might +be complications during deployment. Follow the +[issue 22849](https://gitlab.com/gitlab-org/gitlab-ce/issues/22849) for more +information. + For example: ``` @@ -1210,6 +1225,7 @@ capitalization, the commit will be created but the builds will be skipped. Visit the [examples README][examples] to see a list of examples using GitLab CI with various languages. +[env-manual]: ../environments.md#manually-deploying-to-environments [examples]: ../examples/README.md [ce-6323]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6323 [environment]: ../environments.md diff --git a/doc/development/README.md b/doc/development/README.md index bf1f054b7d5..f88456a7a7a 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -14,7 +14,7 @@ contributing to documentation. - [SQL Migration Style Guide](migration_style_guide.md) for creating safe SQL migrations - [Testing standards and style guidelines](testing.md) -- [UI guide](ui_guide.md) for building GitLab with existing CSS styles and elements +- [UX guide](ux_guide/index.md) for building GitLab with existing CSS styles and elements - [Frontend guidelines](frontend.md) - [SQL guidelines](sql.md) for working with SQL queries - [Sidekiq guidelines](sidekiq_style_guide.md) for working with Sidekiq workers diff --git a/doc/development/gotchas.md b/doc/development/gotchas.md index b25ce79e89f..7bfc9cb361f 100644 --- a/doc/development/gotchas.md +++ b/doc/development/gotchas.md @@ -32,6 +32,95 @@ spec/models/user_spec.rb|6 error| Failure/Error: u = described_class.new NoMeth Except for the top-level `describe` block, always provide a String argument to `describe`. +## Don't assert against the absolute value of a sequence-generated attribute + +Consider the following factory: + +```ruby +FactoryGirl.define do + factory :label do + sequence(:title) { |n| "label#{n}" } + end +end +``` + +Consider the following API spec: + +```ruby +require 'rails_helper' + +describe API::Labels do + it 'creates a first label' do + create(:label) + + get api("/projects/#{project.id}/labels", user) + + expect(response).to have_http_status(200) + expect(json_response.first['name']).to eq('label1') + end + + it 'creates a second label' do + create(:label) + + get api("/projects/#{project.id}/labels", user) + + expect(response).to have_http_status(200) + expect(json_response.first['name']).to eq('label1') + end +end +``` + +When run, this spec doesn't do what we might expect: + +```sh +1) API::API reproduce sequence issue creates a second label + Failure/Error: expect(json_response.first['name']).to eq('label1') + + expected: "label1" + got: "label2" + + (compared using ==) +``` + +That's because FactoryGirl sequences are not reseted for each example. + +Please remember that sequence-generated values exist only to avoid having to +explicitly set attributes that have a uniqueness constraint when using a factory. + +### Solution + +If you assert against a sequence-generated attribute's value, you should set it +explicitly. Also, the value you set shouldn't match the sequence pattern. + +For instance, using our `:label` factory, writing `create(:label, title: 'foo')` +is ok, but `create(:label, title: 'label1')` is not. + +Following is the fixed API spec: + +```ruby +require 'rails_helper' + +describe API::Labels do + it 'creates a first label' do + create(:label, title: 'foo') + + get api("/projects/#{project.id}/labels", user) + + expect(response).to have_http_status(200) + expect(json_response.first['name']).to eq('foo') + end + + it 'creates a second label' do + create(:label, title: 'bar') + + get api("/projects/#{project.id}/labels", user) + + expect(response).to have_http_status(200) + expect(json_response.first['name']).to eq('bar') + end +end +``` + ## Don't `rescue Exception` See ["Why is it bad style to `rescue Exception => e` in Ruby?"][Exception]. diff --git a/doc/development/testing.md b/doc/development/testing.md index b0b26ccf57a..4dc535fb359 100644 --- a/doc/development/testing.md +++ b/doc/development/testing.md @@ -64,6 +64,7 @@ the command line via `bundle exec teaspoon`, or via a web browser at methods. - Use `context` to test branching logic. - Don't `describe` symbols (see [Gotchas](gotchas.md#dont-describe-symbols)). +- Don't assert against the absolute value of a sequence-generated attribute (see [Gotchas](gotchas.md#dont-assert-against-the-absolute-value-of-a-sequence-generated-attribute)). - Don't supply the `:each` argument to hooks since it's the default. - Prefer `not_to` to `to_not` (_this is enforced by Rubocop_). - Try to match the ordering of tests to the ordering within the class. diff --git a/doc/development/ux_guide/basics.md b/doc/development/ux_guide/basics.md new file mode 100644 index 00000000000..62ac56a6bce --- /dev/null +++ b/doc/development/ux_guide/basics.md @@ -0,0 +1,94 @@ +# Basics + +## Contents +* [Responsive](#responsive) +* [Typography](#typography) +* [Icons](#icons) +* [Color](#color) +* [Motion](#motion) +* [Voice and tone](#voice-and-tone) + +--- + +## Responsive +GitLab is a responsive experience that works well across all screen sizes, from mobile devices to large monitors. In order to provide a great user experience, the core functionality (browsing files, creating issues, writing comments, etc.) must be available at all resolutions. However, due to size limitations, some secondary functionality may be hidden on smaller screens. Please keep this functionality limited to rare actions that aren't expected to be needed on small devices. + +--- + +## Typography +### Primary typeface +GitLab's main typeface used throughout the UI is **Source Sans Pro**. We support both the bold and regular weight. + +![Source Sans Pro sample](img/sourcesanspro-sample.png) + + +### Monospace typeface +This is the typeface used for code blocks. GitLab uses the OS default font. +- **Menlo** (Mac) +- **Consolas** (Windows) +- **Liberation Mono** (Linux) + +![Monospace font sample](img/monospacefont-sample.png) + +--- + +## Icons +GitLab uses Font Awesome icons throughout our interface. + +![Trash icon](img/icon-trash.png) +The trash icon is used for destructive actions that deletes information. + +![Edit icon](img/icon-edit.png) +The pencil icon is used for editing content such as comments. + +![Notification icon](img/icon-notification.png) +The bell icon is for notifications, such as Todos. + +![Subscribe icon](img/icon-subscribe.png) +The eye icon is for subscribing to updates. For example, you can subscribe to a label and get updated on issues with that label. + +![RSS icon](img/icon-rss.png) +The standard RSS icon is used for linking to RSS/atom feeds. + +![Close icon](img/icon-close.png) +An 'x' is used for closing UI elements such as dropdowns. + +![Add icon](img/icon-add.png) +A plus is used when creating new objects, such as issues, projects, etc. + +> TODO: update this section, add more general guidance to icon usage and personality, etc. + +--- + +## Color + +![Blue](img/color-blue.png) +Blue is used to highlight primary active elements (such as current tab), as well as other organization and managing commands. + +![Green](img/color-green.png) +Green is for actions that create new objects. + +![Orange](img/color-orange.png) +Orange is used for warnings + +![Red](img/color-red.png) +Red is reserved for delete and other destructive commands + +![Grey](img/color-grey.png) +Grey, and white (depending on context) is used for netral, secondary elements + +> TODO: Establish a perspective for color in terms of our personality and rationalize with Marketing usage. + +--- + +## Motion + +Motion is a tool to help convey important relationships, changes or transitions between elements. It should be used sparingly and intentionally, highlighting the right elements at the right moment. + +> TODO: Determine a more concrete perspective on motion, create consistent easing/timing curves to follow. + +--- + +## Voice and tone + +The copy for GitLab is clear and direct. We strike a clear balance between professional and friendly. We can empathesize with users (such as celebrating completing all Todos), and remain respectful of the importance of the work. We are that trusted, friendly coworker that is helpful and understanding. diff --git a/doc/development/ux_guide/components.md b/doc/development/ux_guide/components.md new file mode 100644 index 00000000000..764c3355714 --- /dev/null +++ b/doc/development/ux_guide/components.md @@ -0,0 +1,254 @@ +# Components + +## Contents +* [Tooltips](#tooltips) +* [Anchor links](#anchor-links) +* [Buttons](#buttons) +* [Dropdowns](#dropdowns) +* [Counts](#counts) +* [Lists](#lists) +* [Tables](#tables) +* [Blocks](#blocks) +* [Panels](#panels) +* [Alerts](#alerts) +* [Forms](#forms) +* [File holders](#file-holders) +* [Data formats](#data-formats) + +--- + +## Tooltips + +### Usage +A tooltip should only be added if additional information is required. + +![Tooltip usage](img/tooltip-usage.png) + +### Placement +By default, tooltips should be placed below the element that they refer to. However, if there is not enough space in the viewpoint, the tooltip should be moved to the side as needed. + +![Tooltip placement location](img/tooltip-placement.png) + +--- + +## Anchor links + +Anchor links are used for navigational actions and lone, secondary commands (such as 'Reset filters' on the Issues List) when deemed appropriate by the UX team. + +### States + +#### Rest + +Primary links are blue in their rest state. Secondary links (such as the time stamp on comments) are a neutral gray color in rest. Details on the main GitLab navigation links can be found on the [features](features.md#navigation) page. + +#### Hover + +An underline should always be added on hover. A gray link becomes blue on hover. + +#### Focus + +The focus state should match the hover state. + +![Anchor link states ](img/components-anchorlinks.png) + +--- + +## Buttons + +Buttons communicate the command that will occur when the user clicks on them. + +### Types + +#### Primary +Primary buttons communicate the main call to action. There should only be one call to action in any given experience. Visually, primary buttons are conveyed with a full background fill + +![Primary button example](img/button-primary.png) + +#### Secondary +Secondary buttons are for alternative commands. They should be conveyed by a button with an stroke, and no background fill. + +![Secondary button example](img/button-secondary.png) + +### Icon and text treatment +Text should be in sentence case, where only the first word is capitalized. "Create issue" is correct, not "Create Issue". Buttons should only contain an icon or a text, not both. + +>>> +TODO: Rationalize this. Ensure that we still believe this. +>>> + +### Colors +Follow the color guidance on the [basics](basics.md#color) page. The default color treatment is the white/grey button. + +--- + +## Dropdowns + +Dropdowns are used to allow users to choose one (or many) options from a list of options. If this list of options is more 20, there should generally be a way to search through and filter the options (see the complex filter dropdowns below.) + +>>> +TODO: Will update this section when the new filters UI is implemented. +>>> + +![Dropdown states](img/components-dropdown.png) + + + +--- + +## Counts + +A count element is used in navigation contexts where it is helpful to indicate the count, or number of items, in a list. Always use the [`number_with_delimiter`][number_with_delimiter] helper to display counts in the UI. + +![Counts example](img/components-counts.png) + +[number_with_delimiter]: http://api.rubyonrails.org/classes/ActionView/Helpers/NumberHelper.html#method-i-number_with_delimiter + +--- + +## Lists + +Lists are used where ever there is a single column of information to display. Ths [issues list](https://gitlab.com/gitlab-org/gitlab-ce/issues) is an example of a important list in the GitLab UI. + +### Types + +Simple list using .content-list + +![Simple list](img/components-simplelist.png) + +List with avatar, title and description using .content-list + +![List with avatar](img/components-listwithavatar.png) + +List with hover effect .well-list + +![List with hover effect](img/components-listwithhover.png) + +List inside panel + +![List inside panel](img/components-listinsidepanel.png) + +--- + +## Tables + +When the information is too complex for a list, with multiple columns of information, a table can be used. For example, the [pipelines page](https://gitlab.com/gitlab-org/gitlab-ce/pipelines) uses a table. + +![Table](img/components-table.png) + +--- + +## Blocks + +Blocks are a way to group related information. + +### Types + +#### Content blocks + +Content blocks (`.content-block`) are the basic grouping of content. They are commonly used in [lists](#lists), and are separated by a botton border. + +![Content block](img/components-contentblock.png) + +#### Row content blocks + +A background color can be added to this blocks. For example, items in the [issue list](https://gitlab.com/gitlab-org/gitlab-ce/issues) have a green background if they were created recently. Below is an example of a gray content block with side padding using `.row-content-block`. + +![Row content block](img/components-rowcontentblock.png) + +#### Cover blocks +Cover blocks are generally used to create a heading element for a page, such as a new project, or a user profile page. Below is a cover block (`.cover-block`) for the profile page with an avatar, name and description. + +![Cover block](img/components-coverblock.png) + +--- + +## Panels + +>>> +TODO: Catalog how we are currently using panels and rationalize how they relate to alerts +>>> + +![Panels](img/components-panels.png) + +--- + +## Alerts + +>>> +TODO: Catalog how we are currently using alerts +>>> + +![Alerts](img/components-alerts.png) + +--- + +## Forms + +There are two options shown below regarding the positioning of labels in forms. Both are options to consider based on context and available size. However, it is important to have a consistent treatment of labels in the same form. + +### Types + +#### Labels stack vertically + +Form (`form`) with label rendered above input. + +![Vertical form](img/components-verticalform.png) + +#### Labels side-by-side + +Horizontal form (`form.horizontal-form`) with label rendered inline with input. + +![Horizontal form](img/components-horizontalform.png) + +--- + +## File holders +A file holder (`.file-holder`) is used to show the contents of a file inline on a page of GitLab. + +![File Holder component](img/components-fileholder.png) + +--- + +## Data formats + +### Dates + +#### Exact + +Format for exacts dates should be ‘Mon DD, YYYY’, such as the examples below. + +![Exact date](img/components-dateexact.png) + +#### Relative + +This format relates how long since an action has occurred. The exact date can be shown as a tooltip on hover. + +![Relative date](img/components-daterelative.png) + +### References + +Referencing GitLab items depends on a symbol for each type of item. Typing that symbol will invoke a dropdown that allows you to search for and autocomplete the item you were looking for. References are shown as [links](#links) in context, and hovering on them shows the full title or name of the item. + +![Hovering on a reference](img/components-referencehover.png) + +#### `%` Milestones + +![Milestone reference](img/components-referencemilestone.png) + +#### `#` Issues + +![Issue reference](img/components-referenceissues.png) + +#### `!` Merge Requests + +![Merge request reference](img/components-referencemrs.png) + +#### `~` Labels + +![Labels reference](img/components-referencelabels.png) + +#### `@` People + +![People reference](img/components-referencepeople.png) + +> TODO: Open issue: Some commit references use monospace fonts, but others don't. Need to standardize this. diff --git a/doc/development/ux_guide/copy.md b/doc/development/ux_guide/copy.md new file mode 100644 index 00000000000..03392a003ee --- /dev/null +++ b/doc/development/ux_guide/copy.md @@ -0,0 +1,78 @@ +# Copy + +The copy and messaging is a core part of the experience of GitLab and the conversation with our users. Follow the below conventions throughout GitLab. + +>**Note:** +We are currently inconsistent with this guidance. Images below are created to illustrate the point. As this guidance is refined, we will ensure that our experiences align. + +## Contents +* [Brevity](#brevity) +* [Forms](#forms) +* [Terminology](#terminology) + +--- + +## Brevity +Users will skim content, rather than read text carefully. +When familiar with a web app, users rely on muscle memory, and may read even less when moving quickly. +A good experience should quickly orient a user, regardless of their experience, to the purpose of the current screen. This should happen without the user having to consciously read long strings of text. +In general, text is burdensome and adds cognitive load. This is especially pronounced in a powerful productivity tool such as GitLab. +We should _not_ rely on words as a crutch to explain the purpose of a screen. +The current navigation and composition of the elements on the screen should get the user 95% there, with the remaining 5% being specific elements such as text. +This means that, as a rule, copy should be very short. A long message or label is a red flag hinting at design that needs improvement. + +>**Example:** +Use `Add` instead of `Add issue` as a button label. +Preferrably use context and placement of controls to make it obvious what clicking on them will do. + +--- + +## Forms + +### Adding items + +When viewing a list of issues, there is a button that is labeled `Add`. Given the context in the example, it is clearly referring to issues. If the context were not clear enough, the label could be `Add issue`. Clicking the button will bring you to the `Add issue` form. Other add flows should be similar. + +![Add issue button](img/copy-form-addissuebutton.png) + +The form should be titled `Add issue`. The submit button should be labeled `Save` or `Submit`. Do not use `Add`, `Create`, `New`, or `Save Changes`. The cancel button should be labeled `Cancel`. Do not use `Back`. + +![Add issue form](img/copy-form-addissueform.png) + +### Editing items + +When in context of an issue, the affordance to edit it is labeled `Edit`. If the context is not clear enough, `Edit issue` could be considered. Other edit flows should be similar. + +![Edit issue button](img/copy-form-editissuebutton.png) + +The form should be titled `Edit Issue`. The submit button should be labeled `Save`. Do not use `Edit`, `Update`, `New`, or `Save Changes`. The cancel button should be labeled `Cancel`. Do not use `Back`. + +![Edit issue form](img/copy-form-editissueform.png) + +--- + +## Terminology + +### Issues + +#### Adjectives (states) + +| Term | Use | +| ---- | --- | +| Open | Issue is active | +| Closed | Issue is no longer active | + +>**Example:** +Use `5 open issues` and do not use `5 pending issues`. +Only use the adjectives in the table above. + +#### Verbs (actions) + +| Term | Use | +| ---- | --- | +| Add | For adding an issue. Do not use `create` or `new` | +| View | View an issue | +| Edit | Edit an issue. Do not use `update` | +| Close | Closing an issue | +| Re-open | Re-open an issue. There should never be a need to use `open` as a verb | +| Delete | Deleting an issue. Do not use `remove` |
\ No newline at end of file diff --git a/doc/development/ux_guide/features.md b/doc/development/ux_guide/features.md new file mode 100644 index 00000000000..9472995c68c --- /dev/null +++ b/doc/development/ux_guide/features.md @@ -0,0 +1,57 @@ +# Features + +## Contents +* [Navigation](#navigation) +* [Filtering](#filtering) +* [Search results](#search-results) +* [Conversations](#conversations) +* [Empty states](#empty-states) + +--- + +## Navigation + +### Global navigation + +The global navigation is accessible via the menu button on the top left of the screen, and can be pinned to keep it open. It contains a consistent list of pages that allow you to view content that is across GitLab. For example, you can view your todos, issues and merge requests across projects and groups. + +![Global nav](img/features-globalnav.png) + + +### Contextual navigation + +The navigation in the header is contextual to each page. These options change depending on if you are looking at a project, group, or settings page. There should be no more than 10 items on a level in the contextual navigation, allowing it to comfortably fit on a typical laptop screen. There can be up to too levels of navigation. Each sub nav group should be a self-contained group of functionality. For example, everything related to the issue tracker should be under the 'Issue' tab, while everything relating to the wiki will be grouped under the 'Wiki' tab. The names used for each section should be short and easy to remember, ideally 1-2 words in length. + +![Contextual nav](img/features-contextualnav.png) + +### Information architecture + +The [GitLab Product Map](https://gitlab.com/gitlab-org/gitlab-design/raw/master/production/resources/gitlab-map.png) shows a visual representation of the information architecture for GitLab. + +--- + +## Filtering + +Today, lists are filtered by a series of dropdowns. Some of these dropdowns allow multiselect (labels), while others allow you to filter to one option (milestones). However, we are currently implementing a [new model](https://gitlab.com/gitlab-org/gitlab-ce/issues/21747) for this, and will update the guide when it is ready. + +![Filters](img/features-filters.png) + +--- + +## Search results + +### Global search + +[Global search](https://gitlab.com/search?group_id=&project_id=13083&repository_ref=&scope=issues&search=mobile) allows you to search across items in a project, or even across multiple projects. You can switch tabs to filter on type of object, or filter by group. + +### List search + +There are several core lists in the GitLab experience, such as the Issue list and the Merge Request list. You are also able to [filter and search these lists](https://gitlab.com/gitlab-org/gitlab-ce/issues?utf8=%E2%9C%93&search=mobile). This UI will be updated with the [new filtering model](https://gitlab.com/gitlab-org/gitlab-ce/issues/21747). + +--- + +## Empty states + +Empty states need to be considered in the design of features. They are vital to helping onboard new users, making the experience feel more approachable and understandable. Empty states should feel inviting and provide just enough information to get people started. There should be a single call to action and a clear explanation of what to use the feature for. + +![Empty states](img/features-emptystates.png) diff --git a/doc/development/ux_guide/img/button-primary.png b/doc/development/ux_guide/img/button-primary.png Binary files differnew file mode 100644 index 00000000000..f4c673f5b88 --- /dev/null +++ b/doc/development/ux_guide/img/button-primary.png diff --git a/doc/development/ux_guide/img/button-secondary.png b/doc/development/ux_guide/img/button-secondary.png Binary files differnew file mode 100644 index 00000000000..57fa65b247c --- /dev/null +++ b/doc/development/ux_guide/img/button-secondary.png diff --git a/doc/development/ux_guide/img/color-blue.png b/doc/development/ux_guide/img/color-blue.png Binary files differnew file mode 100644 index 00000000000..6449613eb16 --- /dev/null +++ b/doc/development/ux_guide/img/color-blue.png diff --git a/doc/development/ux_guide/img/color-green.png b/doc/development/ux_guide/img/color-green.png Binary files differnew file mode 100644 index 00000000000..15475b36f02 --- /dev/null +++ b/doc/development/ux_guide/img/color-green.png diff --git a/doc/development/ux_guide/img/color-grey.png b/doc/development/ux_guide/img/color-grey.png Binary files differnew file mode 100644 index 00000000000..58c474d5ce9 --- /dev/null +++ b/doc/development/ux_guide/img/color-grey.png diff --git a/doc/development/ux_guide/img/color-orange.png b/doc/development/ux_guide/img/color-orange.png Binary files differnew file mode 100644 index 00000000000..f4fc09b2d9b --- /dev/null +++ b/doc/development/ux_guide/img/color-orange.png diff --git a/doc/development/ux_guide/img/color-red.png b/doc/development/ux_guide/img/color-red.png Binary files differnew file mode 100644 index 00000000000..6fbbf0a885d --- /dev/null +++ b/doc/development/ux_guide/img/color-red.png diff --git a/doc/development/ux_guide/img/components-alerts.png b/doc/development/ux_guide/img/components-alerts.png Binary files differnew file mode 100644 index 00000000000..0b2ecc16a5f --- /dev/null +++ b/doc/development/ux_guide/img/components-alerts.png diff --git a/doc/development/ux_guide/img/components-anchorlinks.png b/doc/development/ux_guide/img/components-anchorlinks.png Binary files differnew file mode 100644 index 00000000000..950f348277d --- /dev/null +++ b/doc/development/ux_guide/img/components-anchorlinks.png diff --git a/doc/development/ux_guide/img/components-contentblock.png b/doc/development/ux_guide/img/components-contentblock.png Binary files differnew file mode 100644 index 00000000000..31fc1eec9df --- /dev/null +++ b/doc/development/ux_guide/img/components-contentblock.png diff --git a/doc/development/ux_guide/img/components-counts.png b/doc/development/ux_guide/img/components-counts.png Binary files differnew file mode 100644 index 00000000000..19280e988a0 --- /dev/null +++ b/doc/development/ux_guide/img/components-counts.png diff --git a/doc/development/ux_guide/img/components-coverblock.png b/doc/development/ux_guide/img/components-coverblock.png Binary files differnew file mode 100644 index 00000000000..c8f1f87a108 --- /dev/null +++ b/doc/development/ux_guide/img/components-coverblock.png diff --git a/doc/development/ux_guide/img/components-dateexact.png b/doc/development/ux_guide/img/components-dateexact.png Binary files differnew file mode 100644 index 00000000000..8c0c5c1be40 --- /dev/null +++ b/doc/development/ux_guide/img/components-dateexact.png diff --git a/doc/development/ux_guide/img/components-daterelative.png b/doc/development/ux_guide/img/components-daterelative.png Binary files differnew file mode 100644 index 00000000000..1dc6d89e4ef --- /dev/null +++ b/doc/development/ux_guide/img/components-daterelative.png diff --git a/doc/development/ux_guide/img/components-dropdown.png b/doc/development/ux_guide/img/components-dropdown.png Binary files differnew file mode 100644 index 00000000000..5770a393b37 --- /dev/null +++ b/doc/development/ux_guide/img/components-dropdown.png diff --git a/doc/development/ux_guide/img/components-fileholder.png b/doc/development/ux_guide/img/components-fileholder.png Binary files differnew file mode 100644 index 00000000000..4b8962905d6 --- /dev/null +++ b/doc/development/ux_guide/img/components-fileholder.png diff --git a/doc/development/ux_guide/img/components-horizontalform.png b/doc/development/ux_guide/img/components-horizontalform.png Binary files differnew file mode 100644 index 00000000000..92e28cf9afc --- /dev/null +++ b/doc/development/ux_guide/img/components-horizontalform.png diff --git a/doc/development/ux_guide/img/components-listinsidepanel.png b/doc/development/ux_guide/img/components-listinsidepanel.png Binary files differnew file mode 100644 index 00000000000..30ceb3eaa08 --- /dev/null +++ b/doc/development/ux_guide/img/components-listinsidepanel.png diff --git a/doc/development/ux_guide/img/components-listwithavatar.png b/doc/development/ux_guide/img/components-listwithavatar.png Binary files differnew file mode 100644 index 00000000000..d3cb0ebc02b --- /dev/null +++ b/doc/development/ux_guide/img/components-listwithavatar.png diff --git a/doc/development/ux_guide/img/components-listwithhover.png b/doc/development/ux_guide/img/components-listwithhover.png Binary files differnew file mode 100644 index 00000000000..1484ecba6a0 --- /dev/null +++ b/doc/development/ux_guide/img/components-listwithhover.png diff --git a/doc/development/ux_guide/img/components-panels.png b/doc/development/ux_guide/img/components-panels.png Binary files differnew file mode 100644 index 00000000000..6e71d0ad9c9 --- /dev/null +++ b/doc/development/ux_guide/img/components-panels.png diff --git a/doc/development/ux_guide/img/components-referencehover.png b/doc/development/ux_guide/img/components-referencehover.png Binary files differnew file mode 100644 index 00000000000..e9fb27e2aa9 --- /dev/null +++ b/doc/development/ux_guide/img/components-referencehover.png diff --git a/doc/development/ux_guide/img/components-referenceissues.png b/doc/development/ux_guide/img/components-referenceissues.png Binary files differnew file mode 100644 index 00000000000..caf9477db38 --- /dev/null +++ b/doc/development/ux_guide/img/components-referenceissues.png diff --git a/doc/development/ux_guide/img/components-referencelabels.png b/doc/development/ux_guide/img/components-referencelabels.png Binary files differnew file mode 100644 index 00000000000..a122b45d1f1 --- /dev/null +++ b/doc/development/ux_guide/img/components-referencelabels.png diff --git a/doc/development/ux_guide/img/components-referencemilestone.png b/doc/development/ux_guide/img/components-referencemilestone.png Binary files differnew file mode 100644 index 00000000000..5aa9ecd1a78 --- /dev/null +++ b/doc/development/ux_guide/img/components-referencemilestone.png diff --git a/doc/development/ux_guide/img/components-referencemrs.png b/doc/development/ux_guide/img/components-referencemrs.png Binary files differnew file mode 100644 index 00000000000..6280243859a --- /dev/null +++ b/doc/development/ux_guide/img/components-referencemrs.png diff --git a/doc/development/ux_guide/img/components-referencepeople.png b/doc/development/ux_guide/img/components-referencepeople.png Binary files differnew file mode 100644 index 00000000000..99772a539cf --- /dev/null +++ b/doc/development/ux_guide/img/components-referencepeople.png diff --git a/doc/development/ux_guide/img/components-rowcontentblock.png b/doc/development/ux_guide/img/components-rowcontentblock.png Binary files differnew file mode 100644 index 00000000000..1c2d7096955 --- /dev/null +++ b/doc/development/ux_guide/img/components-rowcontentblock.png diff --git a/doc/development/ux_guide/img/components-simplelist.png b/doc/development/ux_guide/img/components-simplelist.png Binary files differnew file mode 100644 index 00000000000..892f507cfc2 --- /dev/null +++ b/doc/development/ux_guide/img/components-simplelist.png diff --git a/doc/development/ux_guide/img/components-table.png b/doc/development/ux_guide/img/components-table.png Binary files differnew file mode 100644 index 00000000000..7e964c885cf --- /dev/null +++ b/doc/development/ux_guide/img/components-table.png diff --git a/doc/development/ux_guide/img/components-verticalform.png b/doc/development/ux_guide/img/components-verticalform.png Binary files differnew file mode 100644 index 00000000000..38863ad3c1c --- /dev/null +++ b/doc/development/ux_guide/img/components-verticalform.png diff --git a/doc/development/ux_guide/img/copy-form-addissuebutton.png b/doc/development/ux_guide/img/copy-form-addissuebutton.png Binary files differnew file mode 100644 index 00000000000..18839d447e8 --- /dev/null +++ b/doc/development/ux_guide/img/copy-form-addissuebutton.png diff --git a/doc/development/ux_guide/img/copy-form-addissueform.png b/doc/development/ux_guide/img/copy-form-addissueform.png Binary files differnew file mode 100644 index 00000000000..e6838c06eca --- /dev/null +++ b/doc/development/ux_guide/img/copy-form-addissueform.png diff --git a/doc/development/ux_guide/img/copy-form-editissuebutton.png b/doc/development/ux_guide/img/copy-form-editissuebutton.png Binary files differnew file mode 100644 index 00000000000..2435820e14f --- /dev/null +++ b/doc/development/ux_guide/img/copy-form-editissuebutton.png diff --git a/doc/development/ux_guide/img/copy-form-editissueform.png b/doc/development/ux_guide/img/copy-form-editissueform.png Binary files differnew file mode 100644 index 00000000000..5ddeda33e68 --- /dev/null +++ b/doc/development/ux_guide/img/copy-form-editissueform.png diff --git a/doc/development/ux_guide/img/features-contextualnav.png b/doc/development/ux_guide/img/features-contextualnav.png Binary files differnew file mode 100644 index 00000000000..df157f54c84 --- /dev/null +++ b/doc/development/ux_guide/img/features-contextualnav.png diff --git a/doc/development/ux_guide/img/features-emptystates.png b/doc/development/ux_guide/img/features-emptystates.png Binary files differnew file mode 100644 index 00000000000..3befc14588e --- /dev/null +++ b/doc/development/ux_guide/img/features-emptystates.png diff --git a/doc/development/ux_guide/img/features-filters.png b/doc/development/ux_guide/img/features-filters.png Binary files differnew file mode 100644 index 00000000000..281e55d590c --- /dev/null +++ b/doc/development/ux_guide/img/features-filters.png diff --git a/doc/development/ux_guide/img/features-globalnav.png b/doc/development/ux_guide/img/features-globalnav.png Binary files differnew file mode 100644 index 00000000000..3c0db2247ca --- /dev/null +++ b/doc/development/ux_guide/img/features-globalnav.png diff --git a/doc/development/ux_guide/img/icon-add.png b/doc/development/ux_guide/img/icon-add.png Binary files differnew file mode 100644 index 00000000000..0d4c1a7692a --- /dev/null +++ b/doc/development/ux_guide/img/icon-add.png diff --git a/doc/development/ux_guide/img/icon-close.png b/doc/development/ux_guide/img/icon-close.png Binary files differnew file mode 100644 index 00000000000..88d2b3b0c6d --- /dev/null +++ b/doc/development/ux_guide/img/icon-close.png diff --git a/doc/development/ux_guide/img/icon-edit.png b/doc/development/ux_guide/img/icon-edit.png Binary files differnew file mode 100644 index 00000000000..f73be7a10fb --- /dev/null +++ b/doc/development/ux_guide/img/icon-edit.png diff --git a/doc/development/ux_guide/img/icon-notification.png b/doc/development/ux_guide/img/icon-notification.png Binary files differnew file mode 100644 index 00000000000..4758632edd7 --- /dev/null +++ b/doc/development/ux_guide/img/icon-notification.png diff --git a/doc/development/ux_guide/img/icon-rss.png b/doc/development/ux_guide/img/icon-rss.png Binary files differnew file mode 100644 index 00000000000..c7ac9fb1349 --- /dev/null +++ b/doc/development/ux_guide/img/icon-rss.png diff --git a/doc/development/ux_guide/img/icon-subscribe.png b/doc/development/ux_guide/img/icon-subscribe.png Binary files differnew file mode 100644 index 00000000000..5cb277bfd5d --- /dev/null +++ b/doc/development/ux_guide/img/icon-subscribe.png diff --git a/doc/development/ux_guide/img/icon-trash.png b/doc/development/ux_guide/img/icon-trash.png Binary files differnew file mode 100644 index 00000000000..357289a6fff --- /dev/null +++ b/doc/development/ux_guide/img/icon-trash.png diff --git a/doc/development/ux_guide/img/monospacefont-sample.png b/doc/development/ux_guide/img/monospacefont-sample.png Binary files differnew file mode 100644 index 00000000000..1cd290b713c --- /dev/null +++ b/doc/development/ux_guide/img/monospacefont-sample.png diff --git a/doc/development/ux_guide/img/sourcesanspro-sample.png b/doc/development/ux_guide/img/sourcesanspro-sample.png Binary files differnew file mode 100644 index 00000000000..f7ecf0c7c66 --- /dev/null +++ b/doc/development/ux_guide/img/sourcesanspro-sample.png diff --git a/doc/development/ux_guide/img/surfaces-contentitemtitle.png b/doc/development/ux_guide/img/surfaces-contentitemtitle.png Binary files differnew file mode 100644 index 00000000000..2eb926c1c43 --- /dev/null +++ b/doc/development/ux_guide/img/surfaces-contentitemtitle.png diff --git a/doc/development/ux_guide/img/surfaces-header.png b/doc/development/ux_guide/img/surfaces-header.png Binary files differnew file mode 100644 index 00000000000..ab44d4de696 --- /dev/null +++ b/doc/development/ux_guide/img/surfaces-header.png diff --git a/doc/development/ux_guide/img/surfaces-systeminformationblock.png b/doc/development/ux_guide/img/surfaces-systeminformationblock.png Binary files differnew file mode 100644 index 00000000000..5d91e993e24 --- /dev/null +++ b/doc/development/ux_guide/img/surfaces-systeminformationblock.png diff --git a/doc/development/ux_guide/img/surfaces-ux.png b/doc/development/ux_guide/img/surfaces-ux.png Binary files differnew file mode 100644 index 00000000000..e692c51e8c0 --- /dev/null +++ b/doc/development/ux_guide/img/surfaces-ux.png diff --git a/doc/development/ux_guide/img/tooltip-placement.png b/doc/development/ux_guide/img/tooltip-placement.png Binary files differnew file mode 100644 index 00000000000..29a61c8400a --- /dev/null +++ b/doc/development/ux_guide/img/tooltip-placement.png diff --git a/doc/development/ux_guide/img/tooltip-usage.png b/doc/development/ux_guide/img/tooltip-usage.png Binary files differnew file mode 100644 index 00000000000..e8e4c6ded91 --- /dev/null +++ b/doc/development/ux_guide/img/tooltip-usage.png diff --git a/doc/development/ux_guide/index.md b/doc/development/ux_guide/index.md new file mode 100644 index 00000000000..8aed11ebac3 --- /dev/null +++ b/doc/development/ux_guide/index.md @@ -0,0 +1,58 @@ +# GitLab UX Guide + +The goal of this guide is to provide standards, principles and in-depth information to design beautiful and effective GitLab features. This will be a living document, and we welcome contributions, feedback and suggestions. + +## Design + +--- + +### [Principles](principles.md) +These guiding principles set a solid foundation for our design system, and should remain relatively stable over multiple releases. They should be referenced as new design patterns are created. + +--- + +### [Basics](basics.md) +The basic ingredients of our experience establish our personality and feel. This section includes details about typography, color, and motion. + +--- + +### [Components](components.md) +Components are the controls that make up the GitLab experience, including guidance around buttons, links, dropdowns, etc. + +--- + +### [Surfaces](surfaces.md) +The GitLab experience is broken apart into several surfaces. Each of these surfaces is designated for a specific scope or type of content. Examples include the header, global menu, side pane, etc. + +--- + +### [Copy](copy.md) +Conventions on text and messaging within labels, buttons, and other components. + +--- + +### [Features](features.md) +The previous building blocks are combined into complete features in the GitLab UX. Examples include our navigation, filters, search results, and empty states. + +--- + +## Research + +--- + +### [Users](users.md) +How we think about the variety of users of GitLab, from small to large teams, comparing opensource usage to enterprise, etc. + +--- + +## Other + +--- + +### [Tips for designers](tips.md) +Tips for exporting assets, and other guidance. + +--- + +### [Resources](resources.md) +Resources for GitLab UX diff --git a/doc/development/ux_guide/principles.md b/doc/development/ux_guide/principles.md new file mode 100644 index 00000000000..1a297cba2cc --- /dev/null +++ b/doc/development/ux_guide/principles.md @@ -0,0 +1,17 @@ +# Principles + +These are the guiding principles that we should strive for to establish a solid foundation for the GitLab experience. + +## Professional and productive +GitLab is a tool to support what people do, day in, day out. We need to respect the importance of their work, and avoid gimicky details. + +## Minimal and efficient +While work can get complicated, GitLab is about bringing a sharp focus, helping our customers know what matters now. + +## Immediately recognizable +When you look at any screen, you should know immediately that it is GitLab. Our personality is strong and consistent across product and marketing experiences. + +## Human and quirky +We need to build empathy with our users, understanding their state of mind, and connect with them at a human level. Quirkiness is part of our DNA, and we should embrace it in the right moments and contexts. + +> TODO: Ensure these principles align well with the goals of the Marketing team diff --git a/doc/development/ux_guide/resources.md b/doc/development/ux_guide/resources.md new file mode 100644 index 00000000000..2f760c94414 --- /dev/null +++ b/doc/development/ux_guide/resources.md @@ -0,0 +1,13 @@ +# Resources + +## GitLab UI development kit + +We created a page inside GitLab where you can check commonly used html and css elements. + +When you run GitLab instance locally - just visit http://localhost:3000/help/ui page to see UI examples +you can use during GitLab development. + +## Design repository + +All design files are stored in the [gitlab-design](https://gitlab.com/gitlab-org/gitlab-design) +repository and maintained by GitLab UX designers.
\ No newline at end of file diff --git a/doc/development/ux_guide/surfaces.md b/doc/development/ux_guide/surfaces.md new file mode 100644 index 00000000000..881d6aa4cd6 --- /dev/null +++ b/doc/development/ux_guide/surfaces.md @@ -0,0 +1,47 @@ +# Surfaces + +## Contents +* [Header](#header) +* [Global menu](#global-menu) +* [Side pane](#side-pane) +* [Content area](#content-area) + +--- + +![Surfaces UX](img/surfaces-ux.png) + +## Global menu + +This menu is to navigate to pages that contain content global to GitLab. + +--- + +## Header + +The header contains 3 main elements: Project switching and searching, user account avatar and settings, and a contextual menu that changes based on the current page. + +![Surfaces Header](img/surfaces-header.png) + +--- + +## Side pane + +The side pane holds supporting information and meta data for the information in the content area. + +--- + +## Content area + +The main content of the page. The content area can include other surfaces. + +### Item title bar + +The item title bar contains the top level information to identify the item, such as the name, id and status. + +![Item title](img/surfaces-contentitemtitle.png) + +### Item system information + +The system information block contains relevant system controlled information. + +![Item system information](img/surfaces-systeminformationblock.png) diff --git a/doc/development/ux_guide/tips.md b/doc/development/ux_guide/tips.md new file mode 100644 index 00000000000..8348de4f8a2 --- /dev/null +++ b/doc/development/ux_guide/tips.md @@ -0,0 +1,44 @@ +# Tips + +## Contents +* [SVGs](#svgs) + +--- + +## SVGs + +When exporting SVGs, be sure to follow the following guidelines: + +1. Convert all strokes to outlines. +2. Use pathfinder tools to combine overlapping paths and create compound paths. +3. SVGs that are limited to one color should be exported without a fill color so the color can be set using CSS. +4. Ensure that exported SVGs have been run through an [SVG cleaner](https://github.com/RazrFalcon/SVGCleaner) to remove unused elements and attributes. + +You can open your svg in a text editor to ensure that it is clean. +Incorrect files will look like this: + +```xml +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<svg width="16px" height="17px" viewBox="0 0 16 17" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"> + <!-- Generator: Sketch 3.7.2 (28276) - http://www.bohemiancoding.com/sketch --> + <title>Group</title> + <desc>Created with Sketch.</desc> + <defs></defs> + <g id="Page-1" stroke="none" stroke-width="1" fill="none" fill-rule="evenodd"> + <g id="Group" fill="#7E7C7C"> + <path d="M15.1111,1 L0.8891,1 C0.3981,1 0.0001,1.446 0.0001,1.996 L0.0001,15.945 C0.0001,16.495 0.3981,16.941 0.8891,16.941 L15.1111,16.941 C15.6021,16.941 16.0001,16.495 16.0001,15.945 L16.0001,1.996 C16.0001,1.446 15.6021,1 15.1111,1 L15.1111,1 L15.1111,1 Z M14.0001,6.0002 L14.0001,14.949 L2.0001,14.949 L2.0001,6.0002 L14.0001,6.0002 Z M14.0001,4.0002 L14.0001,2.993 L2.0001,2.993 L2.0001,4.0002 L14.0001,4.0002 Z" id="Combined-Shape"></path> + <polygon id="Fill-11" points="3 2.0002 5 2.0002 5 0.0002 3 0.0002"></polygon> + <polygon id="Fill-16" points="11 2.0002 13 2.0002 13 0.0002 11 0.0002"></polygon> + <path d="M5.37709616,11.5511984 L6.92309616,12.7821984 C7.35112915,13.123019 7.97359761,13.0565604 8.32002627,12.6330535 L10.7740263,9.63305349 C11.1237073,9.20557058 11.0606364,8.57555475 10.6331535,8.22587373 C10.2056706,7.87619272 9.57565475,7.93926361 9.22597373,8.36674651 L6.77197373,11.3667465 L8.16890384,11.2176016 L6.62290384,9.98660159 C6.19085236,9.6425813 5.56172188,9.71394467 5.21770159,10.1459962 C4.8736813,10.5780476 4.94504467,11.2071781 5.37709616,11.5511984 L5.37709616,11.5511984 Z" id="Stroke-21"></path> + </g> + </g> +</svg> +``` + +Correct file will look like this: + +```xml +<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 17" enable-background="new 0 0 16 17"><path d="m15.1 1h-2.1v-1h-2v1h-6v-1h-2v1h-2.1c-.5 0-.9.5-.9 1v14c0 .6.4 1 .9 1h14.2c.5 0 .9-.4.9-1v-14c0-.5-.4-1-.9-1m-1.1 14h-12v-9h12v9m0-11h-12v-1h12v1"/><path d="m5.4 11.6l1.5 1.2c.4.3 1.1.3 1.4-.1l2.5-3c.3-.4.3-1.1-.1-1.4-.5-.4-1.1-.3-1.5.1l-1.8 2.2-.8-.6c-.4-.3-1.1-.3-1.4.2-.3.4-.3 1 .2 1.4"/></svg> +``` + +> TODO: Checkout [https://github.com/svg/svgo](https://github.com/svg/svgo) diff --git a/doc/development/ux_guide/users.md b/doc/development/ux_guide/users.md new file mode 100644 index 00000000000..717a902c424 --- /dev/null +++ b/doc/development/ux_guide/users.md @@ -0,0 +1,16 @@ +# Users + +> TODO: Create personas. Understand the similarities and differences across the below spectrums. + +## Users by organization + +- Enterprise +- Medium company +- Small company +- Open source communities + +## Users by role + +- Admin +- Manager +- Developer diff --git a/doc/gitlab-basics/create-project.md b/doc/gitlab-basics/create-project.md index 3f45a631b3a..1c549844ee1 100644 --- a/doc/gitlab-basics/create-project.md +++ b/doc/gitlab-basics/create-project.md @@ -14,8 +14,8 @@ There are two ways to create a new project in GitLab. 1. Fill out the information: - 1. "Project name" is the name of your project (you can't use spaces, but you - can use hyphens or underscores). + 1. "Project name" is the name of your project (you can't use special characters, + but you can use spaces, hyphens, underscores or even emojis). 1. The "Project description" is optional and will be shown in your project's dashboard so others can briefly understand what your project is about. 1. Select a [visibility level](../public_access/public_access.md). diff --git a/doc/integration/README.md b/doc/integration/README.md index c2fd299db07..ae4387e2577 100644 --- a/doc/integration/README.md +++ b/doc/integration/README.md @@ -44,11 +44,15 @@ This [resource](http://kb.kerio.com/product/kerio-connect/server-configuration/s has all the information you need to add a certificate to the main trusted chain. This [answer](http://superuser.com/questions/437330/how-do-you-add-a-certificate-authority-ca-to-ubuntu) -at SuperUser also has relevant information. +at Super User also has relevant information. **Omnibus Trusted Chain** -It is enough to concatenate the certificate to the main trusted certificate: +[Install the self signed certificate or custom certificate authorities](http://docs.gitlab.com/omnibus/common_installation_problems/README.html#using-self-signed-certificate-or-custom-certificate-authorities) +in to GitLab Omnibus. + +It is enough to concatenate the certificate to the main trusted certificate +however it may be overwritten during upgrades: ```bash cat jira.pem >> /opt/gitlab/embedded/ssl/certs/cacert.pem diff --git a/doc/integration/shibboleth.md b/doc/integration/shibboleth.md index 5210ce0de9a..eb9bbb67e7d 100644 --- a/doc/integration/shibboleth.md +++ b/doc/integration/shibboleth.md @@ -10,7 +10,7 @@ To enable the Shibboleth OmniAuth provider you must: 1. Configure Apache shibboleth module. Installation and configuration of module it self is out of scope of this document. Check https://wiki.shibboleth.net/ for more info. -1. You can find Apache config in gitlab-recipes (https://github.com/gitlabhq/gitlab-recipes/blob/master/web-server/apache/gitlab-ssl.conf) +1. You can find Apache config in gitlab-recipes (https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/web-server/apache) Following changes are needed to enable shibboleth: diff --git a/doc/user/markdown.md b/doc/user/markdown.md index 7a7a0b864bd..162d1bd7ed4 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -1,43 +1,5 @@ # Markdown -## Table of Contents - -**[GitLab Flavored Markdown](#gitlab-flavored-markdown-gfm)** - -* [Newlines](#newlines) -* [Multiple underscores in words](#multiple-underscores-in-words) -* [URL auto-linking](#url-auto-linking) -* [Multiline Blockquote](#multiline-blockquote) -* [Code and Syntax Highlighting](#code-and-syntax-highlighting) -* [Inline Diff](#inline-diff) -* [Emoji](#emoji) -* [Special GitLab references](#special-gitlab-references) -* [Task Lists](#task-lists) -* [Videos](#videos) - -**[Standard Markdown](#standard-markdown)** - -* [Headers](#headers) -* [Emphasis](#emphasis) -* [Lists](#lists) -* [Links](#links) -* [Images](#images) -* [Blockquotes](#blockquotes) -* [Inline HTML](#inline-html) -* [Horizontal Rule](#horizontal-rule) -* [Line Breaks](#line-breaks) -* [Tables](#tables) -* [Footnotes](#footnotes) - -**[Wiki-Specific Markdown](#wiki-specific-markdown)** - -* [Wiki - Direct page link](#wiki-direct-page-link) -* [Wiki - Direct file link](#wiki-direct-file-link) -* [Wiki - Hierarchical link](#wiki-hierarchical-link) -* [Wiki - Root link](#wiki-root-link) - -**[References](#references)** - ## GitLab Flavored Markdown (GFM) > **Note:** @@ -64,7 +26,7 @@ You can use GFM in the following areas: You can also use other rich text files in GitLab. You might have to install a dependency to do so. Please see the [github-markup gem readme](https://github.com/gitlabhq/markup#markups) for more information. -## Newlines +### Newlines > If this is not rendered correctly, see https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#newlines @@ -84,7 +46,7 @@ Violets are blue Sugar is sweet -## Multiple underscores in words +### Multiple underscores in words > If this is not rendered correctly, see https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#multiple-underscores-in-words @@ -99,7 +61,7 @@ perform_complicated_task do_this_and_do_that_and_another_thing -## URL auto-linking +### URL auto-linking > If this is not rendered correctly, see https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#url-auto-linking @@ -120,7 +82,7 @@ GFM will autolink almost any URL you copy and paste into your text: * irc://irc.freenode.net/gitlab * http://localhost:3000 -## Multiline Blockquote +### Multiline Blockquote > If this is not rendered correctly, see https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#multiline-blockquote @@ -154,7 +116,7 @@ multiple lines, you can quote that without having to manually prepend `>` to every line! >>> -## Code and Syntax Highlighting +### Code and Syntax Highlighting > If this is not rendered correctly, see https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#code-and-syntax-highlighting @@ -224,7 +186,7 @@ s = "There is no highlighting for this." But let's throw in a <b>tag</b>. ``` -## Inline Diff +### Inline Diff > If this is not rendered correctly, see https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#inline-diff @@ -240,7 +202,7 @@ However the wrapping tags cannot be mixed as such: - {- deletions -] - [- deletions -} -## Emoji +### Emoji > If this is not rendered correctly, see https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#emoji @@ -265,7 +227,7 @@ If you are new to this, don't be :fearful:. You can easily join the emoji :famil Consult the [Emoji Cheat Sheet](http://emoji.codes) for a list of all supported emoji codes. :thumbsup: -## Special GitLab References +### Special GitLab References GFM recognizes special references. @@ -305,7 +267,7 @@ GFM also recognizes certain cross-project references: | `namespace/project@9ba12248...b19a04f5` | commit range comparison | | `namespace/project~"Some label"` | issues with given label | -## Task Lists +### Task Lists > If this is not rendered correctly, see https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#task-lists @@ -328,7 +290,7 @@ You can add task lists to issues, merge requests and comments. To create a task Task lists can only be created in descriptions, not in titles. Task item state can be managed by editing the description's Markdown or by toggling the rendered check boxes. -## Videos +### Videos > If this is not rendered correctly, see https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#videos @@ -345,9 +307,9 @@ Here's a sample video: ![Sample Video](img/markdown_video.mp4) -# Standard Markdown +## Standard Markdown -## Headers +### Headers ```no-highlight # H1 @@ -366,21 +328,6 @@ Alt-H2 ------ ``` -# H1 -## H2 -### H3 -#### H4 -##### H5 -###### H6 - -Alternatively, for H1 and H2, an underline-ish style: - -Alt-H1 -====== - -Alt-H2 ------- - ### Header IDs and links All Markdown-rendered headers automatically get IDs, except in comments. @@ -416,7 +363,7 @@ Would generate the following link IDs: Note that the Emoji processing happens before the header IDs are generated, so the Emoji is converted to an image which then gets removed from the ID. -## Emphasis +### Emphasis ```no-highlight Emphasis, aka italics, with *asterisks* or _underscores_. @@ -436,7 +383,7 @@ Combined emphasis with **asterisks and _underscores_**. Strikethrough uses two tildes. ~~Scratch this.~~ -## Lists +### Lists ```no-highlight 1. First ordered list item @@ -492,7 +439,7 @@ the second list item will be incorrectly labeled as `1`. Second paragraph of first item. 2. Another item -## Links +### Links There are two ways to create links, inline-style and reference-style. @@ -501,9 +448,9 @@ There are two ways to create links, inline-style and reference-style. [I'm a reference-style link][Arbitrary case-insensitive reference text] [I'm a relative reference to a repository file](LICENSE) - + [I am an absolute reference within the repository](/doc/user/markdown.md) - + [I link to the Milestones page](/../milestones) [You can use numbers for reference-style link definitions][1] @@ -523,9 +470,9 @@ There are two ways to create links, inline-style and reference-style. [I'm a relative reference to a repository file](LICENSE)[^1] [I am an absolute reference within the repository](/doc/user/markdown.md) - + [I link to the Milestones page](/../milestones) - + [You can use numbers for reference-style link definitions][1] Or leave it empty and use the [link text itself][] @@ -544,7 +491,8 @@ Relative links do not allow referencing project files in a wiki page or wiki pag will point the link to `wikis/style` when the link is inside of a wiki markdown file. -## Images + +### Images Here's our logo (hover to see the title text): @@ -568,7 +516,7 @@ Reference-style: [logo]: img/markdown_logo.png -## Blockquotes +### Blockquotes ```no-highlight > Blockquotes are very handy in email to emulate reply text. @@ -586,11 +534,11 @@ Quote break. > This is a very long line that will still be quoted properly when it wraps. Oh boy let's keep writing to make sure this is long enough to actually wrap for everyone. Oh, you can *put* **Markdown** into a blockquote. -## Inline HTML +### Inline HTML You can also use raw HTML in your Markdown, and it'll mostly work pretty well. -See the documentation for HTML::Pipeline's [SanitizationFilter](http://www.rubydoc.info/gems/html-pipeline/HTML/Pipeline/SanitizationFilter#WHITELIST-constant) class for the list of allowed HTML tags and attributes. In addition to the default `SanitizationFilter` whitelist, GitLab allows `span` elements. +See the documentation for HTML::Pipeline's [SanitizationFilter](http://www.rubydoc.info/gems/html-pipeline/1.11.0/HTML/Pipeline/SanitizationFilter#WHITELIST-constant) class for the list of allowed HTML tags and attributes. In addition to the default `SanitizationFilter` whitelist, GitLab allows `span` elements. ```no-highlight <dl> @@ -610,7 +558,7 @@ See the documentation for HTML::Pipeline's [SanitizationFilter](http://www.rubyd <dd>Does *not* work **very** well. Use HTML <em>tags</em>.</dd> </dl> -## Horizontal Rule +### Horizontal Rule ``` Three or more... @@ -642,7 +590,7 @@ ___ Underscores -## Line Breaks +### Line Breaks My basic recommendation for learning how line breaks work is to experiment and discover -- hit <Enter> once (i.e., insert one newline), then hit it twice (i.e., insert two newlines), see what happens. You'll soon learn to get what you want. "Markdown Toggle" is your friend. @@ -672,7 +620,7 @@ This line is also a separate paragraph, and... This line is on its own line, because the previous line ends with two spaces. -## Tables +### Tables Tables aren't part of the core Markdown spec, but they are part of GFM and Markdown Here supports them. @@ -708,16 +656,15 @@ By including colons in the header row, you can align the text within that column | Cell 1 | Cell 2 | Cell 3 | Cell 4 | Cell 5 | Cell 6 | | Cell 7 | Cell 8 | Cell 9 | Cell 10 | Cell 11 | Cell 12 | -## Footnotes - -You can add footnotes to your text as follows.[^1] -[^1]: This is my awesome footnote. +### Footnotes ``` -You can add footnotes to your text as follows.[^1] -[^1]: This is my awesome footnote. +You can add footnotes to your text as follows.[^2] +[^2]: This is my awesome footnote. ``` +You can add footnotes to your text as follows.[^2] + ## Wiki-specific Markdown The following examples show how links inside wikis behave. @@ -752,30 +699,30 @@ A link can be constructed relative to the current wiki page using `./<page>`, - If this snippet was placed on a page at `<your_wiki>/documentation/main`, it would link to `<your_wiki>/documentation/related`: - ```markdown - [Link to Related Page](./related) - ``` + ```markdown + [Link to Related Page](./related) + ``` - If this snippet was placed on a page at `<your_wiki>/documentation/related/content`, it would link to `<your_wiki>/documentation/main`: - ```markdown - [Link to Related Page](../main) - ``` + ```markdown + [Link to Related Page](../main) + ``` - If this snippet was placed on a page at `<your_wiki>/documentation/main`, it would link to `<your_wiki>/documentation/related.md`: - ```markdown - [Link to Related Page](./related.md) - ``` + ```markdown + [Link to Related Page](./related.md) + ``` - If this snippet was placed on a page at `<your_wiki>/documentation/related/content`, it would link to `<your_wiki>/documentation/main.md`: - ```markdown - [Link to Related Page](../main.md) - ``` + ```markdown + [Link to Related Page](../main.md) + ``` ### Wiki - Root link @@ -783,22 +730,25 @@ A link starting with a `/` is relative to the wiki root. - This snippet links to `<wiki_root>/documentation`: - ```markdown - [Link to Related Page](/documentation) - ``` + ```markdown + [Link to Related Page](/documentation) + ``` - This snippet links to `<wiki_root>/miscellaneous.md`: - ```markdown - [Link to Related Page](/miscellaneous.md) - ``` + ```markdown + [Link to Related Page](/miscellaneous.md) + ``` + ## References - This document leveraged heavily from the [Markdown-Cheatsheet](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet). - The [Markdown Syntax Guide](https://daringfireball.net/projects/markdown/syntax) at Daring Fireball is an excellent resource for a detailed explanation of standard markdown. - [Dillinger.io](http://dillinger.io) is a handy tool for testing standard markdown. +[^1]: This link will be broken if you see this document from the Help page or docs.gitlab.com +[^2]: This is my awesome footnote. + [markdown.md]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md [rouge]: http://rouge.jneen.net/ "Rouge website" [redcarpet]: https://github.com/vmg/redcarpet "Redcarpet website" -[^1]: This link will be broken if you see this document from the Help page or docs.gitlab.com diff --git a/doc/user/permissions.md b/doc/user/permissions.md index d6216a8dd50..cea78864df2 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -32,6 +32,8 @@ The following table depicts the various user permission levels in a project. | See a commit status | | ✓ | ✓ | ✓ | ✓ | | See a container registry | | ✓ | ✓ | ✓ | ✓ | | See environments | | ✓ | ✓ | ✓ | ✓ | +| Create new environments | | | ✓ | ✓ | ✓ | +| Stop environments | | | ✓ | ✓ | ✓ | | See a list of merge requests | | ✓ | ✓ | ✓ | ✓ | | Manage/Accept merge requests | | | ✓ | ✓ | ✓ | | Create new merge request | | | ✓ | ✓ | ✓ | @@ -45,7 +47,6 @@ The following table depicts the various user permission levels in a project. | Create or update commit status | | | ✓ | ✓ | ✓ | | Update a container registry | | | ✓ | ✓ | ✓ | | Remove a container registry image | | | ✓ | ✓ | ✓ | -| Create new environments | | | ✓ | ✓ | ✓ | | Create new milestones | | | | ✓ | ✓ | | Add new team members | | | | ✓ | ✓ | | Push to protected branches | | | | ✓ | ✓ | @@ -58,7 +59,6 @@ The following table depicts the various user permission levels in a project. | Manage runners | | | | ✓ | ✓ | | Manage build triggers | | | | ✓ | ✓ | | Manage variables | | | | ✓ | ✓ | -| Delete environments | | | | ✓ | ✓ | | Switch visibility level | | | | | ✓ | | Transfer project to another namespace | | | | | ✓ | | Remove project | | | | | ✓ | |