diff options
| author | DJ Mountney <dj@gitlab.com> | 2019-06-28 16:27:39 +0000 |
|---|---|---|
| committer | DJ Mountney <dj@gitlab.com> | 2019-06-28 16:27:39 +0000 |
| commit | c1b1a1fb5e648d44286e6312daa9fa7980202914 (patch) | |
| tree | 152b36c6bdd8384cc2c842d787f9058bee6bb67c /doc | |
| parent | de93bf1fbbd091075ef7ebafb2ab9dabc2e6563c (diff) | |
| parent | 1cd8fb49f9b9150faf50767edbdfb564fde8576b (diff) | |
| download | gitlab-ce-c1b1a1fb5e648d44286e6312daa9fa7980202914.tar.gz | |
Merge branch 'master' into 'check-min-schema-migrate'
# Conflicts:
# lib/gitlab/database.rb
Diffstat (limited to 'doc')
70 files changed, 2397 insertions, 1337 deletions
diff --git a/doc/README.md b/doc/README.md index 3863e17c268..489c8117b9d 100644 --- a/doc/README.md +++ b/doc/README.md @@ -54,7 +54,7 @@ GitLab provides solutions for [all the stages of the DevOps lifecycle](https://a  GitLab is like a top-of-the-line kitchen for making software. As the executive -chef, you decide what software you want serve. Using your recipe, GitLab handles +chef, you decide what software you want to serve. Using your recipe, GitLab handles all the prep work, cooking, and delivery, so you can turn around orders faster than ever. @@ -209,7 +209,7 @@ The following documentation relates to the DevOps **Create** stage: | [GitLab API](api/README.md) | Integrate GitLab via a simple and powerful API. | | [GitLab Integration](integration/README.md) | Integrate with multiple third-party services with GitLab to allow external issue trackers and external authentication. | | [GitLab Webhooks](user/project/integrations/webhooks.md) | Let GitLab notify you when new code has been pushed to your project. | -| [JIRA Development Panel](integration/jira_development_panel.md) **[PREMIUM]** | See GitLab information in the JIRA Development Panel. | +| [Jira Development Panel](integration/jira_development_panel.md) **[PREMIUM]** | See GitLab information in the Jira Development Panel. | | [Project Services](user/project/integrations/project_services.md) | Integrate a project with external services, such as CI and chat. | | [Trello Power-Up](integration/trello_power_up.md) | Integrate with GitLab's Trello Power-Up. | diff --git a/doc/administration/high_availability/gitaly.md b/doc/administration/high_availability/gitaly.md index 90e5f71d835..b7eaa4ce105 100644 --- a/doc/administration/high_availability/gitaly.md +++ b/doc/administration/high_availability/gitaly.md @@ -2,13 +2,13 @@ Gitaly does not yet support full high availability. However, Gitaly is quite stable and is in use on GitLab.com. Scaled and highly available GitLab environments -should consider using Gitaly on a separate node. +should consider using Gitaly on a separate node. -See the [Gitaly HA Epic](https://gitlab.com/groups/gitlab-org/-/epics/289) to -track plans and progress toward high availability support. +See the [Gitaly HA Epic](https://gitlab.com/groups/gitlab-org/-/epics/289) to +track plans and progress toward high availability support. This document is relevant for [Scaled Architecture](README.md#scalable-architecture-examples) -environments and [High Availability Architecture](README.md#high-availability-architecture-examples). +environments and [High Availability Architecture](README.md#high-availability-architecture-examples). ## Running Gitaly on its own server @@ -24,23 +24,25 @@ Continue configuration of other components by going back to: > [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/3786) in GitLab 12.0. - 1. Create/edit `/etc/gitlab/gitlab.rb` and add the following configuration: +1. Make sure to collect [`CONSUL_SERVER_NODES`](database.md#consul-information), which are the IP addresses or DNS records of the Consul server nodes, for the next step. Note they are presented as `Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z` - ```ruby - # Enable service discovery for Prometheus - consul['enable'] = true - consul['monitoring_service_discovery'] = true +1. Create/edit `/etc/gitlab/gitlab.rb` and add the following configuration: - # Replace placeholders - # Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z - # with the addresses of the Consul server nodes - consul['configuration'] = { - retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z), - } + ```ruby + # Enable service discovery for Prometheus + consul['enable'] = true + consul['monitoring_service_discovery'] = true - # Set the network addresses that the exporters will listen on - node_exporter['listen_address'] = '0.0.0.0:9100' - gitaly['prometheus_listen_addr'] = "0.0.0.0:9236" - ``` + # Replace placeholders + # Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z + # with the addresses of the Consul server nodes + consul['configuration'] = { + retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z), + } - 1. Run `sudo gitlab-ctl reconfigure` to compile the configuration. + # Set the network addresses that the exporters will listen on + node_exporter['listen_address'] = '0.0.0.0:9100' + gitaly['prometheus_listen_addr'] = "0.0.0.0:9236" + ``` + +1. Run `sudo gitlab-ctl reconfigure` to compile the configuration. diff --git a/doc/administration/high_availability/gitlab.md b/doc/administration/high_availability/gitlab.md index 0e655e49922..3045be616a6 100644 --- a/doc/administration/high_availability/gitlab.md +++ b/doc/administration/high_availability/gitlab.md @@ -138,6 +138,8 @@ need some extra configuration. If you enable Monitoring, it must be enabled on **all** GitLab servers. +1. Make sure to collect [`CONSUL_SERVER_NODES`](database.md#consul-information), which are the IP addresses or DNS records of the Consul server nodes, for the next step. Note they are presented as `Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z` + 1. Create/edit `/etc/gitlab/gitlab.rb` and add the following configuration: ```ruby @@ -158,12 +160,11 @@ If you enable Monitoring, it must be enabled on **all** GitLab servers. sidekiq['listen_address'] = "0.0.0.0" unicorn['listen'] = '0.0.0.0' - # Add the monitoring node's IP address to the monitoring whitelist and allow it to scrape the NGINX metrics - # Replace placeholder - # monitoring.gitlab.example.com - # with the addresses gathered for the monitoring node - gitlab_rails['monitoring_whitelist'] = ['monitoring.gitlab.example.com'] - nginx['status']['options']['allow'] = ['monitoring.gitlab.example.com'] + # Add the monitoring node's IP address to the monitoring whitelist and allow it to + # scrape the NGINX metrics. Replace placeholder `monitoring.gitlab.example.com` with + # the address and/or subnets gathered from the monitoring node(s). + gitlab_rails['monitoring_whitelist'] = ['monitoring.gitlab.example.com', '127.0.0.0/8'] + nginx['status']['options']['allow'] = ['monitoring.gitlab.example.com', '127.0.0.0/8'] ``` 1. Run `sudo gitlab-ctl reconfigure` to compile the configuration. diff --git a/doc/administration/high_availability/monitoring_node.md b/doc/administration/high_availability/monitoring_node.md index d16bf7dc0f0..ef415dde10a 100644 --- a/doc/administration/high_availability/monitoring_node.md +++ b/doc/administration/high_availability/monitoring_node.md @@ -16,6 +16,8 @@ Omnibus: package you want using **steps 1 and 2** from the GitLab downloads page. - Do not complete any other steps on the download page. +1. Make sure to collect [`CONSUL_SERVER_NODES`](database.md#consul-information), which are the IP addresses or DNS records of the Consul server nodes, for the next step. Note they are presented as `Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z` + 1. Edit `/etc/gitlab/gitlab.rb` and add the contents: ```ruby diff --git a/doc/administration/high_availability/pgbouncer.md b/doc/administration/high_availability/pgbouncer.md index 762179cf756..053dae25823 100644 --- a/doc/administration/high_availability/pgbouncer.md +++ b/doc/administration/high_availability/pgbouncer.md @@ -62,6 +62,33 @@ See our [HA documentation for PostgreSQL](database.md) for information on runnin 1. At this point, your instance should connect to the database through pgbouncer. If you are having issues, see the [Troubleshooting](#troubleshooting) section +## Enable Monitoring + +> [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/3786) in GitLab 12.0. + + If you enable Monitoring, it must be enabled on **all** pgbouncer servers. + + 1. Create/edit `/etc/gitlab/gitlab.rb` and add the following configuration: + + ```ruby + # Enable service discovery for Prometheus + consul['enable'] = true + consul['monitoring_service_discovery'] = true + + # Replace placeholders + # Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z + # with the addresses of the Consul server nodes + consul['configuration'] = { + retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z), + } + + # Set the network addresses that the exporters will listen on + node_exporter['listen_address'] = '0.0.0.0:9100' + pgbouncer_exporter['listen_address'] = '0.0.0.0:9188' + ``` + + 1. Run `sudo gitlab-ctl reconfigure` to compile the configuration. + ### Interacting with pgbouncer #### Administrative console diff --git a/doc/administration/high_availability/redis.md b/doc/administration/high_availability/redis.md index f61a8834af3..8621224272c 100644 --- a/doc/administration/high_availability/redis.md +++ b/doc/administration/high_availability/redis.md @@ -22,10 +22,10 @@ environments including [Basic Scaling](README.md#basic-scaling) and ### Provide your own Redis instance **[CORE ONLY]** -If you want to use your own deployed Redis instance(s), -see [Provide your own Redis instance](#provide-your-own-redis-instance-core-only) -for more details. However, you can use the GitLab Omnibus package to easily -deploy the bundled Redis. +If you want to use your own deployed Redis instance(s), +see [Provide your own Redis instance](#provide-your-own-redis-instance-core-only) +for more details. However, you can use the GitLab Omnibus package to easily +deploy the bundled Redis. ### Standalone Redis using GitLab Omnibus **[CORE ONLY]** @@ -62,11 +62,11 @@ Omnibus: pgbouncer_exporter['enable'] = false gitlab_monitor['enable'] = false gitaly['enable'] = false - + redis['bind'] = '0.0.0.0' redis['port'] = '6379' redis['password'] = 'SECRET_PASSWORD_HERE' - + gitlab_rails['auto_migrate'] = false ``` @@ -74,7 +74,7 @@ Omnibus: 1. Note the Redis node's IP address or hostname, port, and Redis password. These will be necessary when configuring the GitLab application servers later. -1. [Enable Monitoring](#enable-monitoring) +1. [Enable Monitoring](#enable-monitoring) Advanced configuration options are supported and can be added if needed. @@ -91,10 +91,10 @@ environments including [Horizontal](README.md#horizontal), ### Provide your own Redis instance **[CORE ONLY]** -If you want to use your own deployed Redis instance(s), -see [Provide your own Redis instance](#provide-your-own-redis-instance-core-only) -for more details. However, you can use the GitLab Omnibus package to easily -deploy the bundled Redis. +If you want to use your own deployed Redis instance(s), +see [Provide your own Redis instance](#provide-your-own-redis-instance-core-only) +for more details. However, you can use the GitLab Omnibus package to easily +deploy the bundled Redis. ### High Availability with GitLab Omnibus **[PREMIUM ONLY]** @@ -368,7 +368,7 @@ The prerequisites for a HA Redis setup are the following: ```ruby # Specify server role as 'redis_master_role' roles ['redis_master_role'] - + # IP address pointing to a local IP that the other machines can reach to. # You can also set bind to '0.0.0.0' which listen in all interfaces. # If you really need to bind to an external accessible IP, make @@ -382,7 +382,7 @@ The prerequisites for a HA Redis setup are the following: # Set up password authentication for Redis (use the same password in all nodes). redis['password'] = 'redis-password-goes-here' ``` - + 1. Only the primary GitLab application server should handle migrations. To prevent database migrations from running on upgrade, add the following @@ -394,8 +394,8 @@ The prerequisites for a HA Redis setup are the following: 1. [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect. -> Note: You can specify multiple roles like sentinel and redis as: -> roles ['redis_sentinel_role', 'redis_master_role']. Read more about high +> Note: You can specify multiple roles like sentinel and redis as: +> roles ['redis_sentinel_role', 'redis_master_role']. Read more about high > availability roles at https://docs.gitlab.com/omnibus/roles/ ### Step 2. Configuring the slave Redis instances @@ -412,7 +412,7 @@ The prerequisites for a HA Redis setup are the following: ```ruby # Specify server role as 'redis_slave_role' roles ['redis_slave_role'] - + # IP address pointing to a local IP that the other machines can reach to. # You can also set bind to '0.0.0.0' which listen in all interfaces. # If you really need to bind to an external accessible IP, make @@ -443,8 +443,8 @@ The prerequisites for a HA Redis setup are the following: 1. [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect. 1. Go through the steps again for all the other slave nodes. -> Note: You can specify multiple roles like sentinel and redis as: -> roles ['redis_sentinel_role', 'redis_slave_role']. Read more about high +> Note: You can specify multiple roles like sentinel and redis as: +> roles ['redis_sentinel_role', 'redis_slave_role']. Read more about high > availability roles at https://docs.gitlab.com/omnibus/roles/ --- @@ -754,28 +754,30 @@ gitlab_rails['redis_sentinels'] = [ > [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/3786) in GitLab 12.0. - If you enable Monitoring, it must be enabled on **all** Redis servers. +If you enable Monitoring, it must be enabled on **all** Redis servers. + +1. Make sure to collect [`CONSUL_SERVER_NODES`](database.md#consul-information), which are the IP addresses or DNS records of the Consul server nodes, for the next step. Note they are presented as `Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z` - 1. Create/edit `/etc/gitlab/gitlab.rb` and add the following configuration: +1. Create/edit `/etc/gitlab/gitlab.rb` and add the following configuration: - ```ruby - # Enable service discovery for Prometheus - consul['enable'] = true - consul['monitoring_service_discovery'] = true + ```ruby + # Enable service discovery for Prometheus + consul['enable'] = true + consul['monitoring_service_discovery'] = true - # Replace placeholders - # Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z - # with the addresses of the Consul server nodes - consul['configuration'] = { - retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z), - } + # Replace placeholders + # Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z + # with the addresses of the Consul server nodes + consul['configuration'] = { + retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z), + } - # Set the network addresses that the exporters will listen on - node_exporter['listen_address'] = '0.0.0.0:9100' - redis_exporter['listen_address'] = '0.0.0.0:9121' - ``` + # Set the network addresses that the exporters will listen on + node_exporter['listen_address'] = '0.0.0.0:9100' + redis_exporter['listen_address'] = '0.0.0.0:9121' + ``` - 1. Run `sudo gitlab-ctl reconfigure` to compile the configuration. +1. Run `sudo gitlab-ctl reconfigure` to compile the configuration. ## Advanced configuration diff --git a/doc/administration/logs.md b/doc/administration/logs.md index 022c23d02ce..9921ffd8ea0 100644 --- a/doc/administration/logs.md +++ b/doc/administration/logs.md @@ -125,7 +125,7 @@ This file lives in `/var/log/gitlab/gitlab-rails/integrations_json.log` for Omnibus GitLab packages or in `/home/git/gitlab/log/integrations_json.log` for installations from source. -It contains information about [integrations](../user/project/integrations/project_services.md) activities such as JIRA, Asana and Irker services. It uses JSON format like the example below: +It contains information about [integrations](../user/project/integrations/project_services.md) activities such as Jira, Asana and Irker services. It uses JSON format like the example below: ``` json {"severity":"ERROR","time":"2018-09-06T14:56:20.439Z","service_class":"JiraService","project_id":8,"project_path":"h5bp/html5-boilerplate","message":"Error sending message","client_url":"http://jira.gitlap.com:8080","error":"execution expired"} diff --git a/doc/administration/operations/extra_sidekiq_processes.md b/doc/administration/operations/extra_sidekiq_processes.md index 286b99aceb5..7297507f599 100644 --- a/doc/administration/operations/extra_sidekiq_processes.md +++ b/doc/administration/operations/extra_sidekiq_processes.md @@ -1,70 +1,132 @@ # Extra Sidekiq processes **[STARTER ONLY]** -GitLab Enterprise Edition allows one to start an extra set of Sidekiq processes +NOTE: **Note:** +The information in this page applies only to Omnibus GitLab. + +GitLab Starter allows one to start an extra set of Sidekiq processes besides the default one. These processes can be used to consume a dedicated set of queues. This can be used to ensure certain queues always have dedicated workers, no matter the number of jobs that need to be processed. -## Starting extra processes via Omnibus GitLab +## Available Sidekiq queues -To enable `sidekiq-cluster`, you must apply the `sidekiq_cluster['enable'] = true` -setting `/etc/gitlab/gitlab.rb`: +For a list of the existing Sidekiq queues, check the following files: -```ruby -sidekiq_cluster['enable'] = true -``` +- [Queues for both GitLab Community and Enterprise Editions](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/app/workers/all_queues.yml) +- [Queues for GitLab Enterprise Editions only](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/app/workers/all_queues.yml) -You will then specify how many additional processes to create via `sidekiq-cluster` -as well as which queues for them to handle. This is done via the -`sidekiq_cluster['queue_groups']` setting. This is an array whose items contain -which queues to process. Each item in the array will equate to one additional -sidekiq process. +Each entry in the above files represents a queue on which extra Sidekiq processes +can be started. -As an example, to make additional sidekiq processes that process the -`elastic_indexer` and `mailers` queues, you would apply the following: +## Starting extra processes -```ruby -sidekiq_cluster['queue_groups'] = [ - "elastic_indexer", - "mailers" -] -``` +To start extra Sidekiq processes, you must enable `sidekiq-cluster`: -To have an additional sidekiq process handle multiple queues, you simply put a -comma after the first queue name and then put the next queue name: +1. Edit `/etc/gitlab/gitlab.rb` and add: -```ruby -sidekiq_cluster['queue_groups'] = [ - "elastic_indexer,elastic_commit_indexer", - "mailers" -] -``` + ```ruby + sidekiq_cluster['enable'] = true + ``` -Keep in mind, all changes must be followed by reconfiguring your GitLab -application via `sudo gitlab-ctl reconfigure`. +1. You will then need to specify how many additional processes to create via `sidekiq-cluster` + and which queue they should handle via the `sidekiq_cluster['queue_groups']` + array setting. Each item in the array equates to one additional Sidekiq + process, and values in each item determine the queues it works on. -### Monitoring + For example, the following setting adds additional Sidekiq processes to two + queues, one to `elastic_indexer` and one to `mailers`: -Once the Sidekiq processes are added, you can visit the "Background Jobs" + ```ruby + sidekiq_cluster['queue_groups'] = [ + "elastic_indexer", + "mailers" + ] + ``` + + To have an additional Sidekiq process handle multiple queues, add multiple + queue names to its item delimited by commas. For example: + + ```ruby + sidekiq_cluster['queue_groups'] = [ + "elastic_indexer, elastic_commit_indexer", + "mailers" + ] + ``` + +1. Save the file and reconfigure GitLab for the changes to take effect: + + ```sh + sudo gitlab-ctl reconfigure + ``` + +Once the extra Sidekiq processes are added, you can visit the "Background Jobs" section under the admin area in GitLab (`/admin/background_jobs`). - + -### All queues with exceptions +## Negating settings -To have the additional sidekiq processes work on every queue EXCEPT the ones +To have the additional Sidekiq processes work on every queue **except** the ones you list: +1. After you follow the steps for [starting extra processes](#starting-extra-processes), + edit `/etc/gitlab/gitlab.rb` and add: + + ```ruby + sidekiq_cluster['negate'] = true + ``` + +1. Save the file and reconfigure GitLab for the changes to take effect: + + ```sh + sudo gitlab-ctl reconfigure + ``` + +## Ignore all GitHub import queues + +When [importing from GitHub](../../user/project/import/github.md), Sidekiq might +use all of its resources to perform those operations. To set up a separate +`sidekiq-cluster` process to ignore all GitHub import-related queues: + 1. Edit `/etc/gitlab/gitlab.rb` and add: ```ruby + sidekiq_cluster['enable'] = true sidekiq_cluster['negate'] = true + sidekiq_cluster['queue_groups'] = [ + "github_import_advance_stage", + "github_importer:github_import_import_diff_note", + "github_importer:github_import_import_issue", + "github_importer:github_import_import_note", + "github_importer:github_import_import_lfs_object", + "github_importer:github_import_import_pull_request", + "github_importer:github_import_refresh_import_jid", + "github_importer:github_import_stage_finish_import", + "github_importer:github_import_stage_import_base_data", + "github_importer:github_import_stage_import_issues_and_diff_notes", + "github_importer:github_import_stage_import_notes", + "github_importer:github_import_stage_import_lfs_objects", + "github_importer:github_import_stage_import_pull_requests", + "github_importer:github_import_stage_import_repository" + ] ``` -1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. +1. Save the file and reconfigure GitLab for the changes to take effect: + ```sh + sudo gitlab-ctl reconfigure + ``` -### Limiting concurrency +## Number of threads + +Each process defined under `sidekiq_cluster` starts with a +number of threads that equals the number of queues, plus one spare thread. +For example, a process that handles the `process_commit` and `post_receive` +queues will use three threads in total. + +## Limiting concurrency + +To limit the concurrency of the Sidekiq processes: 1. Edit `/etc/gitlab/gitlab.rb` and add: @@ -72,11 +134,22 @@ you list: sidekiq_cluster['concurrency'] = 25 ``` -1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. +1. Save the file and reconfigure GitLab for the changes to take effect: -Keep in mind, this normally would not exceed the number of CPU cores available. + ```sh + sudo gitlab-ctl reconfigure + ``` -### Modifying the check interval +For each queue group, the concurrency factor will be set to `min(number of queues, N)`. +Setting the value to 0 will disable the limit. Keep in mind this normally would +not exceed the number of CPU cores available. + +Each thread requires a Redis connection, so adding threads may +increase Redis latency and potentially cause client timeouts. See the [Sidekiq +documentation about Redis](https://github.com/mperham/sidekiq/wiki/Using-Redis) +for more details. + +## Modifying the check interval To modify the check interval for the additional Sidekiq processes: @@ -90,9 +163,14 @@ To modify the check interval for the additional Sidekiq processes: This tells the additional processes how often to check for enqueued jobs. -## Starting extra processes via command line +## Troubleshooting using the CLI -Starting extra Sidekiq processes can be done using the command +CAUTION: **Warning:** +It's recommended to use `/etc/gitlab/gitlab.rb` to configure the Sidekiq processes. +If you experience a problem, you should contact GitLab support. Use the command +line at your own risk. + +For debugging purposes, you can start extra Sidekiq processes by using the command `/opt/gitlab/embedded/service/gitlab-rails/ee/bin/sidekiq-cluster`. This command takes arguments using the following syntax: @@ -111,29 +189,29 @@ see the relevant section in the [Sidekiq style guide](../../development/sidekiq_style_guide.md#queue-namespaces). For example, say you want to start 2 extra processes: one to process the -"process_commit" queue, and one to process the "post_receive" queue. This can be +`process_commit` queue, and one to process the `post_receive` queue. This can be done as follows: ```bash /opt/gitlab/embedded/service/gitlab-rails/ee/bin/sidekiq-cluster process_commit post_receive ``` -If you instead want to start one process processing both queues you'd use the +If you instead want to start one process processing both queues, you'd use the following syntax: ```bash /opt/gitlab/embedded/service/gitlab-rails/ee/bin/sidekiq-cluster process_commit,post_receive ``` -If you want to have one Sidekiq process process the "process_commit" and -"post_receive" queues, and one process to process the "gitlab_shell" queue, +If you want to have one Sidekiq process dealing with the `process_commit` and +`post_receive` queues, and one process to process the `gitlab_shell` queue, you'd use the following: ```bash /opt/gitlab/embedded/service/gitlab-rails/ee/bin/sidekiq-cluster process_commit,post_receive gitlab_shell ``` -### Monitoring +### Monitoring the `sidekiq-cluster` command The `sidekiq-cluster` command will not terminate once it has started the desired amount of Sidekiq processes. Instead, the process will continue running and @@ -172,24 +250,24 @@ command and not the PID(s) of the started Sidekiq processes. The Rails environment can be set by passing the `--environment` flag to the `sidekiq-cluster` command, or by setting `RAILS_ENV` to a non-empty value. The -default value is "development". +default value can be found in `/opt/gitlab/etc/gitlab-rails/env/RAILS_ENV`. -### All queues with exceptions +### Using negation You're able to run all queues in `sidekiq_queues.yml` file on a single or multiple processes with exceptions using the `--negate` flag. For example, say you want to run a single process for all queues, -except "process_commit" and "post_receive". You can do so by executing: +except `process_commit` and `post_receive`: ```bash -sidekiq-cluster process_commit,post_receive --negate +/opt/gitlab/embedded/service/gitlab-rails/ee/bin/sidekiq-cluster process_commit,post_receive --negate ``` -For multiple processes of all queues (except "process_commit" and "post_receive"): +For multiple processes of all queues (except `process_commit` and `post_receive`): ```bash -sidekiq-cluster process_commit,post_receive process_commit,post_receive --negate +/opt/gitlab/embedded/service/gitlab-rails/ee/bin/sidekiq-cluster process_commit,post_receive process_commit,post_receive --negate ``` ### Limiting concurrency @@ -201,18 +279,3 @@ the `-m N` option. For example, this would cap the maximum number of threads to ```bash /opt/gitlab/embedded/service/gitlab-rails/ee/bin/sidekiq-cluster process_commit,post_receive -m 1 ``` - -For each queue group, the concurrency factor will be set to min(number of -queues, N). Setting the value to 0 will disable the limit. - -Note that each thread requires a Redis connection, so adding threads may -increase Redis latency and potentially cause client timeouts. See the [Sidekiq -documentation about Redis](https://github.com/mperham/sidekiq/wiki/Using-Redis) -for more details. - -## Number of threads - -Each process started using `sidekiq-cluster` (whether it be via command line or -via the gitlab.rb file) starts with a number of threads that equals the number -of queues, plus one spare thread. For example, a process that handles the -"process_commit" and "post_receive" queues will use 3 threads in total. diff --git a/doc/api/merge_request_approvals.md b/doc/api/merge_request_approvals.md index ddac81328b9..49aaac06b46 100644 --- a/doc/api/merge_request_approvals.md +++ b/doc/api/merge_request_approvals.md @@ -222,7 +222,7 @@ GET /projects/:id/merge_requests/:merge_request_iid/approvals "id": 1, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80\u0026d=identicon", - "web_url": "http://localhost:3000/u/root" + "web_url": "http://localhost:3000/root" } } ], @@ -314,7 +314,7 @@ PUT /projects/:id/merge_requests/:merge_request_iid/approvers "id": 1, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80\u0026d=identicon", - "web_url": "http://localhost:3000/u/root" + "web_url": "http://localhost:3000/root" } } ], @@ -387,7 +387,7 @@ does not match, the response code will be `409`. "id": 1, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80\u0026d=identicon", - "web_url": "http://localhost:3000/u/root" + "web_url": "http://localhost:3000/root" } }, { @@ -397,7 +397,7 @@ does not match, the response code will be `409`. "id": 2, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/cf7ad14b34162a76d593e3affca2adca?s=80\u0026d=identicon", - "web_url": "http://localhost:3000/u/ryley" + "web_url": "http://localhost:3000/ryley" } } ], diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index 7b58aa3100e..85a07589956 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -1473,7 +1473,7 @@ Example response when the GitLab issue tracker is used: ] ``` -Example response when an external issue tracker (e.g. JIRA) is used: +Example response when an external issue tracker (e.g. Jira) is used: ```json [ diff --git a/doc/api/project_aliases.md b/doc/api/project_aliases.md new file mode 100644 index 00000000000..65845579409 --- /dev/null +++ b/doc/api/project_aliases.md @@ -0,0 +1,105 @@ +# Project Aliases API **[PREMIUM ONLY]** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/3264) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.1. + +All methods require administrator authorization. + +## List all project aliases + +Get a list of all project aliases: + +``` +GET /project_aliases +``` + +``` +curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/project_aliases" +``` + +Example response: + +```json +[ + { + "id": 1, + "project_id": 1, + "name": "gitlab-ce" + }, + { + "id": 2, + "project_id": 2, + "name": "gitlab-ee" + } +] +``` + +## Get project alias' details + +Get details of a project alias: + +``` +GET /project_aliases/:name +``` + +| Attribute | Type | Required | Description | +|-----------|--------|----------|-----------------------| +| `name` | string | yes | The name of the alias | + +``` +curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/project_aliases/gitlab-ee" +``` + +Example response: + +```json +{ + "id": 1, + "project_id": 1, + "name": "gitlab-ee" +} +``` + +## Create a project alias + +Add a new alias for a project. Responds with a 201 when successful, +400 when there are validation errors (e.g. alias already exists): + +``` +POST /project_aliases +``` + +| Attribute | Type | Required | Description | +|--------------|--------|----------|-----------------------------------------------| +| `project_id` | string | yes | The ID or URL-encoded path of the project. | +| `name` | string | yes | The name of the alias. Must be unique. | + +``` +curl --request POST "https://gitlab.example.com/api/v4/project_aliases" --form "project_id=gitlab-org%2Fgitlab-ee" --form "name=gitlab-ee" +``` + +Example response: + +```json +{ + "id": 1, + "project_id": 1, + "name": "gitlab-ee" +} +``` + +## Delete a project alias + +Removes a project aliases. Responds with a 204 when project alias +exists, 404 when it doesn't: + +``` +DELETE /project_aliases/:name +``` + +| Attribute | Type | Required | Description | +|-----------|--------|----------|-----------------------| +| `name` | string | yes | The name of the alias | + +``` +curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/project_aliases/gitlab-ee" +``` diff --git a/doc/api/services.md b/doc/api/services.md index 042fee4a21a..2368f36e444 100644 --- a/doc/api/services.md +++ b/doc/api/services.md @@ -551,21 +551,21 @@ Get Irker (IRC gateway) service settings for a project. GET /projects/:id/services/irker ``` -## JIRA +## Jira -JIRA issue tracker. +Jira issue tracker. -### Get JIRA service settings +### Get Jira service settings -Get JIRA service settings for a project. +Get Jira service settings for a project. ``` GET /projects/:id/services/jira ``` -### Create/Edit JIRA service +### Create/Edit Jira service -Set JIRA service for a project. +Set Jira service for a project. > Starting with GitLab 8.14, `api_url`, `issues_url`, `new_issue_url` and > `project_url` are replaced by `url`. If you are using an @@ -579,18 +579,18 @@ Parameters: | Parameter | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `url` | string | yes | The URL to the JIRA project which is being linked to this GitLab project. For example, `https://jira.example.com`. | -| `api_url` | string | no | The base URL to the JIRA instance API. Web URL value will be used if not set. For example, `https://jira-api.example.com`. | -| `username` | string | yes | The username of the user created to be used with GitLab/JIRA. | -| `password` | string | yes | The password of the user created to be used with GitLab/JIRA. | +| `url` | string | yes | The URL to the Jira project which is being linked to this GitLab project. For example, `https://jira.example.com`. | +| `api_url` | string | no | The base URL to the Jira instance API. Web URL value will be used if not set. For example, `https://jira-api.example.com`. | +| `username` | string | yes | The username of the user created to be used with GitLab/Jira. | +| `password` | string | yes | The password of the user created to be used with GitLab/Jira. | | `active` | boolean | no | Activates or deactivates the service. Defaults to false (deactivated). | -| `jira_issue_transition_id` | string | no | The ID of a transition that moves issues to a closed state. You can find this number under the JIRA workflow administration (**Administration > Issues > Workflows**) by selecting **View** under **Operations** of the desired workflow of your project. The ID of each state can be found inside the parenthesis of each transition name under the **Transitions (id)** column ([see screenshot][trans]). By default, this ID is set to `2`. | +| `jira_issue_transition_id` | string | no | The ID of a transition that moves issues to a closed state. You can find this number under the Jira workflow administration (**Administration > Issues > Workflows**) by selecting **View** under **Operations** of the desired workflow of your project. The ID of each state can be found inside the parenthesis of each transition name under the **Transitions (id)** column ([see screenshot][trans]). By default, this ID is set to `2`. | | `commit_events` | boolean | false | Enable notifications for commit events | | `merge_requests_events` | boolean | false | Enable notifications for merge request events | -### Delete JIRA service +### Delete Jira service -Remove all previously JIRA settings from a project. +Remove all previously Jira settings from a project. ``` DELETE /projects/:id/services/jira diff --git a/doc/api/settings.md b/doc/api/settings.md index eb3f39e6670..b01cec64837 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -142,8 +142,6 @@ are listed in the descriptions of the relevant settings. | `authorized_keys_enabled` | boolean | no | By default, we write to the `authorized_keys` file to support Git over SSH without additional configuration. GitLab can be optimized to authenticate SSH keys via the database file. Only disable this if you have configured your OpenSSH server to use the AuthorizedKeysCommand. | | `auto_devops_domain` | string | no | Specify a domain to use by default for every project's Auto Review Apps and Auto Deploy stages. | | `auto_devops_enabled` | boolean | no | Enable Auto DevOps for projects by default. It will automatically build, test, and deploy applications based on a predefined CI/CD configuration. | -| `clientside_sentry_dsn` | string | required by: `clientside_sentry_enabled` | Clientside Sentry Data Source Name. | -| `clientside_sentry_enabled` | boolean | no | (**If enabled, requires:** `clientside_sentry_dsn`) Enable Sentry error reporting for the client side. | | `container_registry_token_expire_delay` | integer | no | Container Registry token duration in minutes. | | `default_artifacts_expire_in` | string | no | Set the default expiration time for each job's artifacts. | | `default_branch_protection` | integer | no | Determine if developers can push to master. Can take: `0` _(not protected, both developers and maintainers can push new commits, force push, or delete the branch)_, `1` _(partially protected, developers and maintainers can push new commits, but cannot force push or delete the branch)_ or `2` _(fully protected, developers cannot push new commits, but maintainers can; no-one can force push or delete the branch)_ as a parameter. Default is `2`. | @@ -212,8 +210,6 @@ are listed in the descriptions of the relevant settings. | `restricted_visibility_levels` | array of strings | no | Selected levels cannot be used by non-admin users for groups, projects or snippets. Can take `private`, `internal` and `public` as a parameter. Default is `null` which means there is no restriction. | | `rsa_key_restriction` | integer | no | The minimum allowed bit length of an uploaded RSA key. Default is `0` (no restriction). `-1` disables RSA keys. | | `send_user_confirmation_email` | boolean | no | Send confirmation email on sign-up. | -| `sentry_dsn` | string | required by: `sentry_enabled` | Sentry Data Source Name. | -| `sentry_enabled` | boolean | no | (**If enabled, requires:** `sentry_dsn`) Sentry is an error reporting and logging tool which is currently not shipped with GitLab, available at <https://sentry.io>. | | `session_expire_delay` | integer | no | Session duration in minutes. GitLab restart is required to apply changes | | `shared_runners_enabled` | boolean | no | (**If enabled, requires:** `shared_runners_text`) Enable shared runners for new projects. | | `shared_runners_text` | string | required by: `shared_runners_enabled` | Shared runners text. | diff --git a/doc/api/users.md b/doc/api/users.md index 4bc0335ae33..4667a985eb9 100644 --- a/doc/api/users.md +++ b/doc/api/users.md @@ -272,7 +272,14 @@ GET /users/:id?with_custom_attributes=true ## User creation -Creates a new user. Note only administrators can create new users. Either `password` or `reset_password` should be specified (`reset_password` takes priority). If `reset_password` is `false`, then `password` is required. +Creates a new user. Note only administrators can create new +users. Either `password`, `reset_password`, or `force_random_password` +must be specified. If `reset_password` and `force_random_password` are +both `false`, then `password` is required. + +Note that `force_random_password` and `reset_password` take priority +over `password`. In addition, `reset_password` and +`force_random_password` can be used together. ``` POST /users @@ -280,29 +287,30 @@ POST /users Parameters: -- `email` (required) - Email -- `password` (optional) - Password -- `reset_password` (optional) - Send user password reset link - true or false(default) -- `username` (required) - Username -- `name` (required) - Name -- `skype` (optional) - Skype ID -- `linkedin` (optional) - LinkedIn -- `twitter` (optional) - Twitter account -- `website_url` (optional) - Website URL -- `organization` (optional) - Organization name -- `projects_limit` (optional) - Number of projects user can create -- `extern_uid` (optional) - External UID -- `provider` (optional) - External provider name -- `group_id_for_saml` (optional) - ID of group where SAML has been configured -- `bio` (optional) - User's biography -- `location` (optional) - User's location -- `public_email` (optional) - The public email of the user -- `admin` (optional) - User is admin - true or false (default) -- `can_create_group` (optional) - User can create groups - true or false -- `skip_confirmation` (optional) - Skip confirmation - true or false (default) -- `external` (optional) - Flags the user as external - true or false(default) -- `avatar` (optional) - Image file for user's avatar -- `private_profile` (optional) - User's profile is private - true or false +- `email` (required) - Email +- `password` (optional) - Password +- `reset_password` (optional) - Send user password reset link - true or false (default) +- `force_random_password` (optional) - Set user password to a random value - true or false (default) +- `username` (required) - Username +- `name` (required) - Name +- `skype` (optional) - Skype ID +- `linkedin` (optional) - LinkedIn +- `twitter` (optional) - Twitter account +- `website_url` (optional) - Website URL +- `organization` (optional) - Organization name +- `projects_limit` (optional) - Number of projects user can create +- `extern_uid` (optional) - External UID +- `provider` (optional) - External provider name +- `group_id_for_saml` (optional) - ID of group where SAML has been configured +- `bio` (optional) - User's biography +- `location` (optional) - User's location +- `public_email` (optional) - The public email of the user +- `admin` (optional) - User is admin - true or false (default) +- `can_create_group` (optional) - User can create groups - true or false +- `skip_confirmation` (optional) - Skip confirmation - true or false (default) +- `external` (optional) - Flags the user as external - true or false(default) +- `avatar` (optional) - Image file for user's avatar +- `private_profile` (optional) - User's profile is private - true or false - `shared_runners_minutes_limit` (optional) - Pipeline minutes quota for this user - `extra_shared_runners_minutes_limit` (optional) - Extra pipeline minutes quota for this user diff --git a/doc/ci/README.md b/doc/ci/README.md index 1743c38eb46..da864a0b3cc 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -63,7 +63,7 @@ Once you're familiar with how GitLab CI/CD works, see the for all the attributes you can set and use. NOTE: **Note:** -GitLab CI/CD and [shared runners](runners/README.md#shared-specific-and-group-runners) are enabled in GitLab.com and available for all users, limited only to the [user's pipelines quota](../user/admin_area/settings/continuous_integration.md#extra-shared-runners-pipeline-minutes-quota). +GitLab CI/CD and [shared runners](runners/README.md#shared-specific-and-group-runners) are enabled in GitLab.com and available for all users, limited only to the [user's pipelines quota](../user/admin_area/settings/continuous_integration.md#extra-shared-runners-pipeline-minutes-quota-free-only). ## Configuration diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index b4c4bea6447..efdcaf5a6f5 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -205,7 +205,14 @@ An example project using this approach can be found here: <https://gitlab.com/gi ### Use Docker socket binding -The third approach is to bind-mount `/var/run/docker.sock` into the container so that docker is available in the context of that image. +The third approach is to bind-mount `/var/run/docker.sock` into the +container so that Docker is available in the context of that image. + +NOTE: **Note:** +If you bind the Docker socket [when using GitLab Runner 11.11 or +newer](https://gitlab.com/gitlab-org/gitlab-runner/merge_requests/1261), +you can no longer use `docker:dind` as a service because volume bindings +are done to the services as well, making these incompatible. In order to do that, follow the steps: diff --git a/doc/development/api_graphql_styleguide.md b/doc/development/api_graphql_styleguide.md index 2ed2a905db7..aeddad14995 100644 --- a/doc/development/api_graphql_styleguide.md +++ b/doc/development/api_graphql_styleguide.md @@ -447,7 +447,7 @@ want to validate the abilities for. Alternatively, we can add a `find_object` method that will load the object on the mutation. This would allow you to use the -`authorized_find!` and `authorized_find!` helper methods. +`authorized_find!` helper method. When a user is not allowed to perform the action, or an object is not found, we should raise a diff --git a/doc/development/chatops_on_gitlabcom.md b/doc/development/chatops_on_gitlabcom.md index c63ec53414c..a7b402c3fb0 100644 --- a/doc/development/chatops_on_gitlabcom.md +++ b/doc/development/chatops_on_gitlabcom.md @@ -1,12 +1,13 @@ # Chatops on GitLab.com -Chatops on GitLab.com allows GitLabbers to run various automation tasks on GitLab.com using Slack. +ChatOps on GitLab.com allows GitLab team members to run various automation tasks on GitLab.com using Slack. ## Requesting access -GitLabbers may need access to Chatops on GitLab.com for administration tasks such as: +GitLab team-members may need access to Chatops on GitLab.com for administration +tasks such as: -- Configuring feature flags on staging. +- Configuring feature flags. - Running `EXPLAIN` queries against the GitLab.com production replica. To request access to Chatops on GitLab.com: @@ -18,4 +19,4 @@ To request access to Chatops on GitLab.com: - [Chatops Usage](https://docs.gitlab.com/ee/ci/chatops/README.html) - [Understanding EXPLAIN plans](understanding_explain_plans.md) - - [Feature Groups](feature_flags.md#feature-groups) + - [Feature Groups](feature_flags/development.md#feature-groups) diff --git a/doc/development/contributing/issue_workflow.md b/doc/development/contributing/issue_workflow.md index 3d36a7bf3b1..db426dec5e4 100644 --- a/doc/development/contributing/issue_workflow.md +++ b/doc/development/contributing/issue_workflow.md @@ -43,6 +43,10 @@ The descriptions on the [labels page][labels-page] explain what falls under each Subject labels are labels that define what area or feature of GitLab this issue hits. They are not always necessary, but very convenient. +Subject labels are now used to infer and apply relevant group and devops stage +labels. Please apply them whenever possible to facilitate accurate matching. +Please refer to [this merge request][inferred-labels] for more information. + Examples of subject labels are ~wiki, ~ldap, ~api, ~issues, ~"merge requests", ~labels, and ~"Container Registry". @@ -65,10 +69,12 @@ The current team labels are: - ~Defend - ~Distribution - ~Documentation +- ~Ecosystem - ~Geo - ~Gitaly - ~Growth - ~Manage +- ~Memory - ~Monitor - ~Plan - ~Quality @@ -442,3 +448,4 @@ A recent example of this was the issue for [labels-page]: https://gitlab.com/gitlab-org/gitlab-ce/labels [ce-tracker]: https://gitlab.com/gitlab-org/gitlab-ce/issues [ee-tracker]: https://gitlab.com/gitlab-org/gitlab-ee/issues +[inferred-labels]: https://gitlab.com/gitlab-org/quality/triage-ops/merge_requests/155 diff --git a/doc/development/contributing/style_guides.md b/doc/development/contributing/style_guides.md index f319d00d7fe..87e61a7476f 100644 --- a/doc/development/contributing/style_guides.md +++ b/doc/development/contributing/style_guides.md @@ -31,8 +31,8 @@ This is also the style used by linting tools such as [Return to Contributing documentation](index.md) -[rss-source]: https://github.com/bbatsov/ruby-style-guide/blob/master/README.md#source-code-layout -[rss-naming]: https://github.com/bbatsov/ruby-style-guide/blob/master/README.md#naming +[rss-source]: https://github.com/rubocop-hq/ruby-style-guide/blob/master/README.adoc#source-code-layout +[rss-naming]: https://github.com/rubocop-hq/ruby-style-guide/blob/master/README.adoc#naming-conventions [doc-guidelines]: ../documentation/index.md "Documentation guidelines" [js-styleguide]: ../fe_guide/style_guide_js.md "JavaScript styleguide" [scss-styleguide]: ../fe_guide/style_guide_scss.md "SCSS styleguide" diff --git a/doc/development/documentation/site_architecture/index.md b/doc/development/documentation/site_architecture/index.md index ee3a9caf9a0..6dd12b5efa7 100644 --- a/doc/development/documentation/site_architecture/index.md +++ b/doc/development/documentation/site_architecture/index.md @@ -11,8 +11,40 @@ and deploy it to <https://docs.gitlab.com>. While the source of the documentation content is stored in GitLab's respective product repositories, the source that is used to build the documentation site _from that content_ -is located at <https://gitlab.com/gitlab-com/gitlab-docs>. See the README there for -detailed information. +is located at <https://gitlab.com/gitlab-com/gitlab-docs>. + +The following diagram illustrates the relationship between the repositories +from where content is sourced, the `gitlab-docs` project, and the published output. + +```mermaid + graph LR + A[gitlab-ce/doc] + B[gitlab-ee/doc] + C[gitlab-runner/docs] + D[omnibus-gitlab/doc] + E[charts/doc] + F[gitlab-docs] + A --> F + B --> F + C --> F + D --> F + E --> F + F -- Build pipeline --> G + G[docs.gitlab.com] + H[/ce/] + I[/ee/] + J[/runner/] + K[/omnibus/] + L[/charts/] + G --> H + G --> I + G --> J + G --> K + G --> L +``` + +See the [README there](https://gitlab.com/gitlab-com/gitlab-docs/blob/master/README.md) +for detailed information. ## Assets @@ -22,9 +54,9 @@ the GitLab Documentation website. ### Libraries -- [Bootstrap 3.3 components](https://getbootstrap.com/docs/3.3/components/) -- [Bootstrap 3.3 JS](https://getbootstrap.com/docs/3.3/javascript/) -- [jQuery](https://jquery.com/) 3.2.1 +- [Bootstrap 4.3.1 components](https://getbootstrap.com/docs/4.3/components/) +- [Bootstrap 4.3.1 JS](https://getbootstrap.com/docs/4.3/getting-started/javascript/) +- [jQuery](https://jquery.com/) 3.3.1 - [Clipboard JS](https://clipboardjs.com/) - [Font Awesome 4.7.0](https://fontawesome.com/v4.7.0/icons/) diff --git a/doc/development/documentation/styleguide.md b/doc/development/documentation/styleguide.md index 23d52a33881..7cd3d82ec4e 100644 --- a/doc/development/documentation/styleguide.md +++ b/doc/development/documentation/styleguide.md @@ -100,6 +100,13 @@ use regular Markdown markup, following the rules in the linked style guide. Note that Kramdown-specific markup (e.g., `{:.class}`) will not render properly on GitLab instances under [`/help`](index.md#gitlab-help). +Hard-coded HTML is valid, although it's discouraged to be used while we have `/help`. HTML is permitted as long as: + +- There's no equivalent markup in markdown. +- Advanced tables are necessary. +- Special styling is required. +- Reviewed and approved by a technical writer. + ## Structure ### Organize by topic, not by type @@ -143,7 +150,8 @@ The table below shows what kind of documentation goes where. a proper naming would be `import_projects_from_github.md`. The same rule applies to images. 1. For image files, do not exceed 100KB. -1. We do not yet support embedded videos. Please link out. +1. Do not upload video files to the product repositories. +[Link or embed videos](#videos) instead. 1. There are four main directories, `user`, `administration`, `api` and `development`. 1. The `doc/user/` directory has five main subdirectories: `project/`, `group/`, `profile/`, `dashboard/` and `admin_area/`. @@ -207,6 +215,7 @@ Do not include the same information in multiple places. [Link to a SSOT instead. ## Text +- [Write in markdown](#markdown). - Splitting long lines (preferably up to 100 characters) can make it easier to provide feedback on small chunks of text. - Insert an empty line for new paragraphs. - Use sentence case for titles, headings, labels, menu items, and buttons. @@ -409,11 +418,20 @@ To indicate the steps of navigation through the UI: ## Images - Place images in a separate directory named `img/` in the same directory where - the `.md` document that you're working on is located. Always prepend their - names with the name of the document that they will be included in. For - example, if there is a document called `twitter.md`, then a valid image name - could be `twitter_login_screen.png`. -- Images should have a specific, non-generic name that will differentiate and describe them properly. + the `.md` document that you're working on is located. +- Images should have a specific, non-generic name that will + differentiate and describe them properly. +- Always add to the end of the file name the GitLab release version + number corresponding to the release milestone the image was added to, + or corresponding to the release the screenshot was taken from, using the + format `image_name_vX_Y.png`. + ([Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/61027) in GitLab 12.1.) +- For example, for a screenshot taken from the pipelines page of + GitLab 11.1, a valid name is `pipelines_v11_1.png`. If you're + adding an illustration that does not include parts of the UI, + add the release number corresponding to the release the image + was added to. Example, for an MR added to 11.1's milestone, + a valid name for an illustration is `devops_diagram_v11_1.png`. - Keep all file names in lower case. - Consider using PNG images instead of JPEG. - Compress all images with <https://tinypng.com/> or similar tool. @@ -421,12 +439,12 @@ To indicate the steps of navigation through the UI: - Images should be used (only when necessary) to _illustrate_ the description of a process, not to _replace_ it. - Max image size: 100KB (gifs included). -- The GitLab docs do not support videos yet. +- See also how to link and embed [videos](#videos) to illustrate the docs. Inside the document: - The Markdown way of using an image inside a document is: - `` + `` - Always use a proper description for what the image is about. That way, when a browser fails to show the image, this text will be used as an alternative description. @@ -446,6 +464,85 @@ directly to an HTML `img` tag: <img src="path/to/image.jpg" alt="Alt text (required)" class="image-noshadow"> ``` +## Videos + +Adding GitLab's existing YouTube video tutorials to the documentation is +highly encouraged, unless the video is outdated. Videos should not +replace documentation, but complement or illustrate it. If content in a video is +fundamental to a feature and its key use cases, but this is not adequately covered in the documentation, +add this detail to the documentation text or create an issue to review the video and do so. + +Do not upload videos to the product repositories. [Link](#link-to-video) or [embed](#embed-videos) them instead. + +### Link to video + +To link out to a video, include a YouTube icon so that readers can +quickly and easily scan the page for videos before reading: + +```md +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For an overview, see [Video Title](link-to-video). +``` + +You can link any up-to-date video that is useful to the GitLab user. + +### Embed videos + +> [Introduced](https://gitlab.com/gitlab-com/gitlab-docs/merge_requests/472) in GitLab 12.1. + +GitLab docs (docs.gitlab.com) support embedded videos. + +You can only embed videos from +[GitLab's official YouTube account](https://www.youtube.com/channel/UCnMGQ8QHMAnVIsI3xJrihhg). +For videos from other sources, [link](#link-to-video) them instead. + +In most cases, it is better to [link to video](#link-to-video) instead, +because an embed takes up a lot of space on the page and can be distracting +to readers. + +To embed a video, follow the instructions below and make sure +you have your MR reviewed and approved by a technical writer. + +1. Copy the code below and paste it into your markdown file. + Leave a blank line above and below it. Do NOT edit the code + (don't remove or add any spaces, etc). +1. On YouTube, visit the video URL you want to display. Copy + the regular URL from your browser (`https://www.youtube.com/watch?v=VIDEO-ID`) + and replace the video title and link in the line under `<div class="video-fallback">`. +1. On YouTube, click **Share**, then **Embed**. +1. Copy the `<iframe>` source (`src`) **URL only** + (`https://www.youtube.com/embed/VIDEO-ID`), + and paste it, replacing the content of the `src` field in the + `iframe` tag. + +```html +leave a blank line here +<div class="video-fallback"> + See the video: [Video title](https://www.youtube.com/watch?v=MqL6BMOySIQ). +</div> +<figure class="video-container"> + <iframe src="https://www.youtube.com/embed/MqL6BMOySIQ" frameborder="0" allowfullscreen="true"> </iframe> +</figure> +leave a blank line here +``` + +This is how it renders on docs.gitlab.com: + +<div class="video-fallback"> + See the video: [What is GitLab](https://www.youtube.com/watch?v=enMumwvLAug). +</div> +<figure class="video-container"> + <iframe src="https://www.youtube.com/embed/MqL6BMOySIQ" frameborder="0" allowfullscreen="true"> </iframe> +</figure> + +> Notes: +> +> - The `figure` tag is required for semantic SEO and the `video_container` +class is necessary to make sure the video is responsive and displays +nicely on different mobile devices. +> - The `<div class="video-fallback">` is a fallback necessary for GitLab's +`/help`, as GitLab's markdown processor does not support iframes. It's hidden on the docs site but will be displayed on GitLab's `/help`. + ## Code blocks - Always wrap code added to a sentence in inline code blocks (``` ` ```). diff --git a/doc/development/fe_guide/accessibility.md b/doc/development/fe_guide/accessibility.md index df32242a522..64c793cfd64 100644 --- a/doc/development/fe_guide/accessibility.md +++ b/doc/development/fe_guide/accessibility.md @@ -5,8 +5,16 @@ [Chrome Accessibility Developer Tools][chrome-accessibility-developer-tools] are useful for testing for potential accessibility problems in GitLab. -Accessibility best-practices and more in-depth information is available on -[the Audit Rules page][audit-rules] for the Chrome Accessibility Developer Tools. +The [axe][axe-website] browser extension (available for [Firefox][axe-firefox-extension] and [Chrome][axe-chrome-extension]) is +also a handy tool for running audits and getting feedback on markup, CSS and even potentially problematic color usages. + +Accessibility best-practices and more in-depth information are available on +[the Audit Rules page][audit-rules] for the Chrome Accessibility Developer Tools. The "[awesome a11y][awesome-a11y]" list is also a +useful compilation of accessibility-related material. [chrome-accessibility-developer-tools]: https://github.com/GoogleChrome/accessibility-developer-tools [audit-rules]: https://github.com/GoogleChrome/accessibility-developer-tools/wiki/Audit-Rules +[axe-website]: https://www.deque.com/axe/ +[axe-firefox-extension]: https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/ +[axe-chrome-extension]: https://chrome.google.com/webstore/detail/axe/lhdoppojpmngadmnindnejefpokejbdd +[awesome-a11y]: https://github.com/brunopulis/awesome-a11y diff --git a/doc/development/feature_flags.md b/doc/development/feature_flags.md index 13f0c5cc33e..6bad91d6287 100644 --- a/doc/development/feature_flags.md +++ b/doc/development/feature_flags.md @@ -1,127 +1 @@ -# Manage feature flags - -Starting from GitLab 9.3 we support feature flags for features in GitLab via -[Flipper](https://github.com/jnunemaker/flipper/). You should use the `Feature` -class (defined in `lib/feature.rb`) in your code to get, set and list feature -flags. - -During runtime you can set the values for the gates via the -[features API](../api/features.md) (accessible to admins only). - -## Feature groups - -Starting from GitLab 9.4 we support feature groups via -[Flipper groups](https://github.com/jnunemaker/flipper/blob/v0.10.2/docs/Gates.md#2-group). - -Feature groups must be defined statically in `lib/feature.rb` (in the -`.register_feature_groups` method), but their implementation can obviously be -dynamic (querying the DB etc.). - -Once defined in `lib/feature.rb`, you will be able to activate a -feature for a given feature group via the [`feature_group` param of the features API](../api/features.md#set-or-create-a-feature) - -For GitLab.com, [team members have access to feature flags through Chatops](chatops_on_gitlabcom.md). Only -percentage gates are supported at this time. Setting a feature to be used 50% of -the time, you should execute `/chatops run feature set my_feature_flag 50`. - -## Feature flags for user applications - -This document only covers feature flags used in the development of GitLab -itself. Feature flags in deployed user applications can be found at -[Feature Flags](../user/project/operations/feature_flags.md) - -## Developing with feature flags - -In general, it's better to have a group- or user-based gate, and you should prefer -it over the use of percentage gates. This would make debugging easier, as you -filter for example logs and errors based on actors too. Furthermore, this allows -for enabling for the `gitlab-org` group first, while the rest of the users -aren't impacted. - -```ruby -# Good -Feature.enabled?(:feature_flag, project) - -# Avoid, if possible -Feature.enabled?(:feature_flag) -``` - -To use feature gates based on actors, the model needs to respond to -`flipper_id`. For example, to enable for the Foo model: - -```ruby -class Foo < ActiveRecord::Base - include FeatureGate -end -``` - -Features that are developed and are intended to be merged behind a feature flag -should not include a changelog entry. The entry should be added in the merge -request removing the feature flags. - -In the rare case that you need the feature flag to be on automatically, use -`default_enabled: true` when checking: - -```ruby -Feature.enabled?(:feature_flag, project, default_enabled: true) -``` - -For more information about rolling out changes using feature flags, refer to the -[Rolling out changes using feature flags](rolling_out_changes_using_feature_flags.md) -guide. - -### Frontend - -For frontend code you can use the method `push_frontend_feature_flag`, which is -available to all controllers that inherit from `ApplicationController`. Using -this method you can expose the state of a feature flag as follows: - -```ruby -before_action do - push_frontend_feature_flag(:vim_bindings) -end - -def index - # ... -end - -def edit - # ... -end -``` - -You can then check for the state of the feature flag in JavaScript as follows: - -```javascript -if ( gon.features.vimBindings ) { - // ... -} -``` - -The name of the feature flag in JavaScript will always be camelCased, meaning -that checking for `gon.features.vim_bindings` would not work. - -### Specs - -In the test environment `Feature.enabled?` is stubbed to always respond to `true`, -so we make sure behavior under feature flag doesn't go untested in some non-specific -contexts. - -Whenever a feature flag is present, make sure to test _both_ states of the -feature flag. - -See the -[testing guide](testing_guide/best_practices.md#feature-flags-in-tests) -for information and examples on how to stub feature flags in tests. - -## Enabling a feature flag (in development) - -In the rails console (`rails c`), enter the following command to enable your feature flag - -```ruby -Feature.enable(:feature_flag_name) -``` - -## Enabling a feature flag (in production) - -Check how to [roll out changes using feature flags](rolling_out_changes_using_feature_flags.md). +This document was moved to [another location](feature_flags/index.md). diff --git a/doc/development/feature_flags/controls.md b/doc/development/feature_flags/controls.md new file mode 100644 index 00000000000..c67467b7c11 --- /dev/null +++ b/doc/development/feature_flags/controls.md @@ -0,0 +1,123 @@ +# Access for enabling a feature flag in production + +In order to be able to turn on/off features behind feature flags in any of the +GitLab Inc. provided environments such as staging and production, you need to +have access to the chatops bot. Chatops bot is currently running on the ops instance, +which is different from GitLab.com or dev.gitlab.org. + +Follow the Chatops document to [request access](https://docs.gitlab.com/ee/development/chatops_on_gitlabcom.html#requesting-access). + +Once you are added to the project test if your access propagated, +run: + +``` +/chatops run feature --help +``` + +## Rolling out changes + +When the changes are deployed to the environments it is time to start +rolling out the feature to our users. The exact procedure of rolling out a +change is unspecified, as this can vary from change to change. However, in +general we recommend rolling out changes incrementally, instead of enabling them +for everybody right away. We also recommend you to _not_ enable a feature +_before_ the code is being deployed. +This allows you to separate rolling out a feature from a deploy, making it +easier to measure the impact of both separately. + +GitLab's feature library (using +[Flipper](https://github.com/jnunemaker/flipper), and covered in the [Feature +Flags process](process.md) guide) supports rolling out changes to a percentage of +users. This in turn can be controlled using [GitLab chatops](../../ci/chatops/README.md). + +For an up to date list of feature flag commands please see [the source +code](https://gitlab.com/gitlab-com/chatops/blob/master/lib/chatops/commands/feature.rb). +Note that all the examples in that file must be preceded by +`/chatops run`. + +If you get an error "Whoops! This action is not allowed. This incident +will be reported." that means your Slack account is not allowed to +change feature flags or you do not [have access](#access-for-enabling-a-feature-flag-in-production). + +### Enabling feature for staging and dev.gitlab.org + +As a first step in a feature rollout, you should enable the feature on <https://staging.gitlab.com> +and <https://dev.gitlab.org>. + +For example, to enable a feature for 25% of all users, run the following in +Slack: + +``` +/chatops run feature set new_navigation_bar 25 --dev +/chatops run feature set new_navigation_bar 25 --staging +``` + +These two environments have different scopes. +`dev.gitlab.org` is a production CE environment that has internal GitLab Inc. +traffic and is used for some development and other related work. +`staging.gitlab.com` has a smaller subset of GitLab.com database and repositories +and does not have regular traffic. Staging is an EE instance and can give you +a (very) rough estimate of how your feature will look/behave on GitLab.com. +Both of these instances are connected to Sentry so make sure you check the projects +there for any exceptions while testing your feature after enabling the feature flag. + +Once you are confident enough that these environments are in a good state with your +feature enabled, you can roll out the change to GitLab.com. + +## Enabling feature for GitLab.com + +Similar to above, to enable a feature for 25% of all users, run the following in +Slack: + +``` +/chatops run feature set new_navigation_bar 25 +``` + +This will enable the feature for GitLab.com, with `new_navigation_bar` being the +name of the feature. + +If you are not certain what percentages to use, simply use the following steps: + +1. 25% +1. 50% +1. 75% +1. 100% + +Between every step you'll want to wait a little while and monitor the +appropriate graphs on <https://dashboards.gitlab.net>. The exact time to wait +may differ. For some features a few minutes is enough, while for others you may +want to wait several hours or even days. This is entirely up to you, just make +sure it is clearly communicated to your team, and the Production team if you +anticipate any potential problems. + +Feature gates can also be actor based, for example a feature could first be +enabled for only the `gitlab-ce` project. The project is passed by supplying a +`--project` flag: + +``` +/chatops run feature set --project=gitlab-org/gitlab-ce some_feature true +``` + +For groups the `--group` flag is available: + +``` +/chatops run feature set --group=gitlab-org some_feature true +``` + +## Cleaning up + +Once the change is deemed stable, submit a new merge request to remove the +feature flag. This ensures the change is available to all users and self-hosted +instances. Make sure to add the ~"feature flag" label to this merge request so +release managers are aware the changes are hidden behind a feature flag. If the +merge request has to be picked into a stable branch, make sure to also add the +appropriate "Pick into X" label (e.g. "Pick into XX.X"). +See [the process document](https://docs.gitlab.com/ee/development/feature_flags/process.html#including-a-feature-behind-feature-flag-in-the-final-release) for further details. + +When a feature gate has been removed from the code base, the value still exists +in the database. +This can be removed through ChatOps: + +``` +/chatops run feature delete some_feature +``` diff --git a/doc/development/feature_flags/development.md b/doc/development/feature_flags/development.md new file mode 100644 index 00000000000..238052529d9 --- /dev/null +++ b/doc/development/feature_flags/development.md @@ -0,0 +1,131 @@ +# Developing with feature flags + +In general, it's better to have a group- or user-based gate, and you should prefer +it over the use of percentage gates. This would make debugging easier, as you +filter for example logs and errors based on actors too. Furthermore, this allows +for enabling for the `gitlab-org` or `gitlab-com` group first, while the rest of +the users aren't impacted. + +```ruby +# Good +Feature.enabled?(:feature_flag, project) + +# Avoid, if possible +Feature.enabled?(:feature_flag) +``` + +To use feature gates based on actors, the model needs to respond to +`flipper_id`. For example, to enable for the Foo model: + +```ruby +class Foo < ActiveRecord::Base + include FeatureGate +end +``` + +Features that are developed and are intended to be merged behind a feature flag +should not include a changelog entry. The entry should be added in the merge +request removing the feature flags. + +In the rare case that you need the feature flag to be on automatically, use +`default_enabled: true` when checking: + +```ruby +Feature.enabled?(:feature_flag, project, default_enabled: true) +``` + +The [`Project#feature_available?`][project-fa], +[`Namespace#feature_available?`][namespace-fa] (EE), and +[`License.feature_available?`][license-fa] (EE) methods all implicitly check for +a feature flag by the same name as the provided argument. + +For example if a feature is license-gated, there's no need to add an additional +explicit feature flag check since the flag will be checked as part of the +`License.feature_available?` call. Similarly, there's no need to "clean up" a +feature flag once the feature has reached general availability. + +You'd still want to use an explicit `Feature.enabled?` check if your new feature +isn't gated by a License or Plan. + +[project-fa]: https://gitlab.com/gitlab-org/gitlab-ee/blob/4cc1c62918aa4c31750cb21dfb1a6c3492d71080/app/models/project_feature.rb#L63-68 +[namespace-fa]: https://gitlab.com/gitlab-org/gitlab-ee/blob/4cc1c62918aa4c31750cb21dfb1a6c3492d71080/ee/app/models/ee/namespace.rb#L71-85 +[license-fa]: https://gitlab.com/gitlab-org/gitlab-ee/blob/4cc1c62918aa4c31750cb21dfb1a6c3492d71080/ee/app/models/license.rb#L293-300 + +An important side-effect of the implicit feature flags mentioned above is that +unless the feature is explicitly disabled or limited to a percentage of users, +the feature flag check will default to `true`. + +As an example, if you were to ship the backend half of a feature behind a flag, +you'd want to explicitly disable that flag until the frontend half is also ready +to be shipped. [You can do this via Chatops](https://docs.gitlab.com/ee/development/feature_flags/controls.html): + +``` +/chatops run feature set some_feature 0 +``` + +Note that you can do this at any time, even before the merge request using the +flag has been merged! + +## Feature groups + +Starting from GitLab 9.4 we support feature groups via +[Flipper groups](https://github.com/jnunemaker/flipper/blob/v0.10.2/docs/Gates.md#2-group). + +Feature groups must be defined statically in `lib/feature.rb` (in the +`.register_feature_groups` method), but their implementation can obviously be +dynamic (querying the DB etc.). + +Once defined in `lib/feature.rb`, you will be able to activate a +feature for a given feature group via the [`feature_group` param of the features API](../../api/features.md#set-or-create-a-feature) + +### Frontend + +For frontend code you can use the method `push_frontend_feature_flag`, which is +available to all controllers that inherit from `ApplicationController`. Using +this method you can expose the state of a feature flag as follows: + +```ruby +before_action do + push_frontend_feature_flag(:vim_bindings) +end + +def index + # ... +end + +def edit + # ... +end +``` + +You can then check for the state of the feature flag in JavaScript as follows: + +```javascript +if ( gon.features.vimBindings ) { + // ... +} +``` + +The name of the feature flag in JavaScript will always be camelCased, meaning +that checking for `gon.features.vim_bindings` would not work. + +### Specs + +In the test environment `Feature.enabled?` is stubbed to always respond to `true`, +so we make sure behavior under feature flag doesn't go untested in some non-specific +contexts. + +Whenever a feature flag is present, make sure to test _both_ states of the +feature flag. + +See the +[testing guide](../testing_guide/best_practices.md#feature-flags-in-tests) +for information and examples on how to stub feature flags in tests. + +### Enabling a feature flag (in development) + +In the rails console (`rails c`), enter the following command to enable your feature flag + +```ruby +Feature.enable(:feature_flag_name) +``` diff --git a/doc/development/feature_flags/index.md b/doc/development/feature_flags/index.md new file mode 100644 index 00000000000..56872f8c075 --- /dev/null +++ b/doc/development/feature_flags/index.md @@ -0,0 +1,12 @@ +# Feature flags in development of GitLab + +Feature flags can be used to gradually roll out changes, be +it a new feature, or a performance improvement. By using feature flags, we can +comfortably measure the impact of our changes, while still being able to easily +disable those changes, without having to revert an entire release. + +Before using feature flags for GitLab's development, read through the following: + +- [Process for using features flags](process.md). +- [Developing with feature flags documentation](development.md). +- [Controlling feature flags documentation](controls.md). diff --git a/doc/development/feature_flags/process.md b/doc/development/feature_flags/process.md new file mode 100644 index 00000000000..ee142b0da66 --- /dev/null +++ b/doc/development/feature_flags/process.md @@ -0,0 +1,130 @@ +# Feature flags process +## Feature flags for user applications + +This document only covers feature flags used in the development of GitLab +itself. Feature flags in deployed user applications can be found at +[Feature Flags feature documentation](../../user/project/operations/feature_flags.md). + +## Feature flags in GitLab development + +The following highlights should be considered when deciding if feature flags +should be leveraged: + +- By default, the feature flags should be **off**. +- Feature flags should remain in the codebase for as short period as possible +to reduce the need for feature flag accounting. +- The person operating with feature flags is responsible for clearly communicating +the status of a feature behind the feature flag with responsible stakeholders. +- Merge requests that make changes hidden behind a feature flag, or remove an +existing feature flag because a feature is deemed stable must have the +~"feature flag" label assigned. + +One might be tempted to think that feature flags will delay the release of a +feature by at least one month (= one release). This is not the case. A feature +flag does not have to stick around for a specific amount of time +(e.g. at least one release), instead they should stick around until the feature +is deemed stable. Stable means it works on GitLab.com without causing any +problems, such as outages. + +### When to use feature flags + +Starting with GitLab 11.4, developers are required to use feature flags for +non-trivial changes. Such changes include: + +- New features (e.g. a new merge request widget, epics, etc). +- Complex performance improvements that may require additional testing in + production, such as rewriting complex queries. +- Invasive changes to the user interface, such as a new navigation bar or the + removal of a sidebar. +- Adding support for importing projects from a third-party service. + +In all cases, those working on the changes can best decide if a feature flag is +necessary. For example, changing the color of a button doesn't need a feature +flag, while changing the navigation bar definitely needs one. In case you are +uncertain if a feature flag is necessary, simply ask about this in the merge +request, and those reviewing the changes will likely provide you with an answer. + +When using a feature flag for UI elements, make sure to _also_ use a feature +flag for the underlying backend code, if there is any. This ensures there is +absolutely no way to use the feature until it is enabled. + +### Including a feature behind feature flag in the final release + +In order to build a final release and present the feature for self-hosted +users, the feature flag should be at least defaulted to **on**. If the feature +is deemed stable and there is confidence that removing the feature flag is safe, +consider removing the feature flag altogether. Take into consideration that such +action can make the feature available on GitLab.com shortly after the change to +the feature flag is merged. + +Changing the default state or removing the feature flag has to be done before +the 22nd of the month, _at least_ 2 working days before, in order for the change +to be included in the final self-managed release. + +In addition to this, the feature behind feature flag should: + +- Run in all GitLab.com environments for a sufficient period of time. This time +period depends on the feature behind the feature flag, but as a general rule of +thumb 2-4 working days should be sufficient to gather enough feedback. +- The feature should be exposed to all users within the GitLab.com plan during +the above mentioned period of time. Exposing the feature to a smaller percentage +or only a group of users might not expose a sufficient amount of information to aid in +making a decision on feature stability. + +While rare, release managers may decide to reject picking or revert a change in +a stable branch, even when feature flags are used. This might be necessary if +the changes are deemed problematic, too invasive, or there simply isn't enough +time to properly measure how the changes behave on GitLab.com. + +### The cost of feature flags + +When reading the above, one might be tempted to think this procedure is going to +add a lot of work. Fortunately, this is not the case, and we'll show why. For +this example we'll specify the cost of the work to do as a number, ranging from +0 to infinity. The greater the number, the more expensive the work is. The cost +does _not_ translate to time, it's just a way of measuring complexity of one +change relative to another. + +Let's say we are building a new feature, and we have determined that the cost of +this is 10. We have also determined that the cost of adding a feature flag check +in a variety of places is 1. If we do not use feature flags, and our feature +works as intended, our total cost is 10. This however is the best case scenario. +Optimizing for the best case scenario is guaranteed to lead to trouble, whereas +optimizing for the worst case scenario is almost always better. + +To illustrate this, let's say our feature causes an outage, and there's no +immediate way to resolve it. This means we'd have to take the following steps to +resolve the outage: + +1. Revert the release. +1. Perform any cleanups that might be necessary, depending on the changes that + were made. +1. Revert the commit, ensuring the "master" branch remains stable. This is + especially necessary if solving the problem can take days or even weeks. +1. Pick the revert commit into the appropriate stable branches, ensuring we + don't block any future releases until the problem is resolved. + +As history has shown, these steps are time consuming, complex, often involve +many developers, and worst of all: our users will have a bad experience using +GitLab.com until the problem is resolved. + +Now let's say that all of this has an associated cost of 10. This means that in +the worst case scenario, which we should optimize for, our total cost is now 20. + +If we had used a feature flag, things would have been very different. We don't +need to revert a release, and because feature flags are disabled by default we +don't need to revert and pick any Git commits. In fact, all we have to do is +disable the feature, and in the worst case, perform cleanup. Let's say that +the cost of this is 2. In this case, our best case cost is 11: 10 to build the +feature, and 1 to add the feature flag. The worst case cost is now 13: 10 to +build the feature, 1 to add the feature flag, and 2 to disable and clean up. + +Here we can see that in the best case scenario the work necessary is only a tiny +bit more compared to not using a feature flag. Meanwhile, the process of +reverting our changes has been made significantly and reliably cheaper. + +In other words, feature flags do not slow down the development process. Instead, +they speed up the process as managing incidents now becomes _much_ easier. Once +continuous deployments are easier to perform, the time to iterate on a feature +is reduced even further, as you no longer need to wait weeks before your changes +are available on GitLab.com. diff --git a/doc/development/i18n/externalization.md b/doc/development/i18n/externalization.md index ce310672dad..17462887162 100644 --- a/doc/development/i18n/externalization.md +++ b/doc/development/i18n/externalization.md @@ -135,7 +135,7 @@ There is also and alternative method to [translate messages from validation erro ### Interpolation Placeholders in translated text should match the code style of the respective source file. -For example use `%{created_at}` in Ruby but `%{createdAt}` in JavaScript. +For example use `%{created_at}` in Ruby but `%{createdAt}` in JavaScript. Make sure to [avoid splitting sentences when adding links](#avoid-splitting-sentences-when-adding-links). - In Ruby/HAML: @@ -267,20 +267,41 @@ should be externalized as follows: This also applies when using links in between translated sentences, otherwise these texts are not translatable in certain languages. -Instead of: +- In Ruby/HAML, instead of: + + ```haml + - zones_link = link_to(s_('ClusterIntegration|zones'), 'https://cloud.google.com/compute/docs/regions-zones/regions-zones', target: '_blank', rel: 'noopener noreferrer') + = s_('ClusterIntegration|Learn more about %{zones_link}').html_safe % { zones_link: zones_link } + ``` + + Set the link starting and ending HTML fragments as variables like so: + + ```haml + - zones_link_url = 'https://cloud.google.com/compute/docs/regions-zones/regions-zones' + - zones_link_start = '<a href="%{url}" target="_blank" rel="noopener noreferrer">'.html_safe % { url: zones_link_url } + = s_('ClusterIntegration|Learn more about %{zones_link_start}zones%{zones_link_end}').html_safe % { zones_link_start: zones_link_start, zones_link_end: '</a>'.html_safe } + ``` -```haml -- zones_link = link_to(s_('ClusterIntegration|zones'), 'https://cloud.google.com/compute/docs/regions-zones/regions-zones', target: '_blank', rel: 'noopener noreferrer') -= s_('ClusterIntegration|Learn more about %{zones_link}').html_safe % { zones_link: zones_link } -``` +- In JavaScript, instead of: -Set the link starting and ending HTML fragments as variables like so: + ```js + {{ + sprintf(s__("ClusterIntegration|Learn more about %{link}"), { + link: '<a href="https://cloud.google.com/compute/docs/regions-zones/regions-zones" target="_blank" rel="noopener noreferrer">zones</a>' + }) + }} + ``` -```haml -- zones_link_url = 'https://cloud.google.com/compute/docs/regions-zones/regions-zones' -- zones_link_start = '<a href="%{url}" target="_blank" rel="noopener noreferrer">'.html_safe % { url: zones_link_url } -= s_('ClusterIntegration|Learn more about %{zones_link_start}zones%{zones_link_end}').html_safe % { zones_link_start: zones_link_start, zones_link_end: '</a>'.html_safe } -``` + Set the link starting and ending HTML fragments as variables like so: + + ```js + {{ + sprintf(s__("ClusterIntegration|Learn more about %{linkStart}zones%{linkEnd}"), { + linkStart: '<a href="https://cloud.google.com/compute/docs/regions-zones/regions-zones" target="_blank" rel="noopener noreferrer">' + linkEnd: '</a>', + }) + }} + ``` The reasoning behind this is that in some languages words change depending on context. For example in Japanese は is added to the subject of a sentence and を to the object. This is impossible to translate correctly if we extract individual words from the sentence. diff --git a/doc/development/new_fe_guide/tips.md b/doc/development/new_fe_guide/tips.md index 4564f678ec0..879b54bd93c 100644 --- a/doc/development/new_fe_guide/tips.md +++ b/doc/development/new_fe_guide/tips.md @@ -10,16 +10,16 @@ yarn clean ## Creating feature flags in development -The process for creating a feature flag is the same as [enabling a feature flag in development](../feature_flags.md#enabling-a-feature-flag-in-development). +The process for creating a feature flag is the same as [enabling a feature flag in development](../feature_flags/development.md#enabling-a-feature-flag-in-development). Your feature flag can now be: -- [made available to the frontend](../feature_flags.md#frontend) via the `gon` -- queried in [tests](../feature_flags.md#specs) -- queried in HAML templates and ruby files via the `Feature.enabled?(:my_shiny_new_feature_flag)` method +- [Made available to the frontend](../feature_flags/development.md#frontend) via the `gon` +- Queried in [tests](../feature_flags/development.md#specs) +- Queried in HAML templates and ruby files via the `Feature.enabled?(:my_shiny_new_feature_flag)` method ### More on feature flags - [Deleting a feature flag](../../api/features.md#delete-a-feature) -- [Manage feature flags](../feature_flags.md) +- [Manage feature flags](../feature_flags/process.md) - [Feature flags API](../../api/features.md) diff --git a/doc/development/rolling_out_changes_using_feature_flags.md b/doc/development/rolling_out_changes_using_feature_flags.md index 84028b1b342..6bad91d6287 100644 --- a/doc/development/rolling_out_changes_using_feature_flags.md +++ b/doc/development/rolling_out_changes_using_feature_flags.md @@ -1,225 +1 @@ -# Rolling out changes using feature flags - -[Feature flags](feature_flags.md) can be used to gradually roll out changes, be -it a new feature, or a performance improvement. By using feature flags, we can -comfortably measure the impact of our changes, while still being able to easily -disable those changes, without having to revert an entire release. - -## When to use feature flags - -Starting with GitLab 11.4, developers are required to use feature flags for -non-trivial changes. Such changes include: - -- New features (e.g. a new merge request widget, epics, etc). -- Complex performance improvements that may require additional testing in - production, such as rewriting complex queries. -- Invasive changes to the user interface, such as a new navigation bar or the - removal of a sidebar. -- Adding support for importing projects from a third-party service. - -In all cases, those working on the changes can best decide if a feature flag is -necessary. For example, changing the color of a button doesn't need a feature -flag, while changing the navigation bar definitely needs one. In case you are -uncertain if a feature flag is necessary, simply ask about this in the merge -request, and those reviewing the changes will likely provide you with an answer. - -When using a feature flag for UI elements, make sure to _also_ use a feature -flag for the underlying backend code, if there is any. This ensures there is -absolutely no way to use the feature until it is enabled. - -## The cost of feature flags - -When reading the above, one might be tempted to think this procedure is going to -add a lot of work. Fortunately, this is not the case, and we'll show why. For -this example we'll specify the cost of the work to do as a number, ranging from -0 to infinity. The greater the number, the more expensive the work is. The cost -does _not_ translate to time, it's just a way of measuring complexity of one -change relative to another. - -Let's say we are building a new feature, and we have determined that the cost of -this is 10. We have also determined that the cost of adding a feature flag check -in a variety of places is 1. If we do not use feature flags, and our feature -works as intended, our total cost is 10. This however is the best case scenario. -Optimising for the best case scenario is guaranteed to lead to trouble, whereas -optimising for the worst case scenario is almost always better. - -To illustrate this, let's say our feature causes an outage, and there's no -immediate way to resolve it. This means we'd have to take the following steps to -resolve the outage: - -1. Revert the release. -1. Perform any cleanups that might be necessary, depending on the changes that - were made. -1. Revert the commit, ensuring the "master" branch remains stable. This is - especially necessary if solving the problem can take days or even weeks. -1. Pick the revert commit into the appropriate stable branches, ensuring we - don't block any future releases until the problem is resolved. - -As history has shown, these steps are time consuming, complex, often involve -many developers, and worst of all: our users will have a bad experience using -GitLab.com until the problem is resolved. - -Now let's say that all of this has an associated cost of 10. This means that in -the worst case scenario, which we should optimise for, our total cost is now 20. - -If we had used a feature flag, things would have been very different. We don't -need to revert a release, and because feature flags are disabled by default we -don't need to revert and pick any Git commits. In fact, all we have to do is -disable the feature, and in the worst case, perform cleanup. Let's say that -the cost of this is 2. In this case, our best case cost is 11: 10 to build the -feature, and 1 to add the feature flag. The worst case cost is now 13: 10 to -build the feature, 1 to add the feature flag, and 2 to disable and clean up. - -Here we can see that in the best case scenario the work necessary is only a tiny -bit more compared to not using a feature flag. Meanwhile, the process of -reverting our changes has been made significantly and reliably cheaper. - -In other words, feature flags do not slow down the development process. Instead, -they speed up the process as managing incidents now becomes _much_ easier. Once -continuous deployments are easier to perform, the time to iterate on a feature -is reduced even further, as you no longer need to wait weeks before your changes -are available on GitLab.com. - -## Rolling out changes - -The procedure of using feature flags is straightforward, and similar to not -using them. You add the necessary tests (make sure to test both the on and off -states of your feature flag(s)), make sure they all pass, have the code -reviewed, etc. You then submit your merge request, and add the ~"feature flag" -label. This label is used to signal to release managers that your changes are -hidden behind a feature flag and that it is safe to pick the MR into a stable -branch, without the need for an exception request. - -When the changes are deployed it is time to start rolling out the feature to our -users. The exact procedure of rolling out a change is unspecified, as this can -vary from change to change. However, in general we recommend rolling out changes -incrementally, instead of enabling them for everybody right away. We also -recommend you to _not_ enable a feature _before_ the code is being deployed. -This allows you to separate rolling out a feature from a deploy, making it -easier to measure the impact of both separately. - -GitLab's feature library (using -[Flipper](https://github.com/jnunemaker/flipper), and covered in the [Feature -Flags](feature_flags.md) guide) supports rolling out changes to a percentage of -users. This in turn can be controlled using [GitLab -chatops](../ci/chatops/README.md). - -For an up to date list of feature flag commands please see [the source -code](https://gitlab.com/gitlab-com/chatops/blob/master/lib/chatops/commands/feature.rb). -Note that all the examples in that file must be preceded by -`/chatops run`. - -If you get an error "Whoops! This action is not allowed. This incident -will be reported." that means your Slack account is not allowed to -change feature flags. To test if you are allowed to do anything at all, -run: - -``` -/chatops run feature --help -``` - -For example, to enable a feature for 25% of all users, run the following in -Slack: - -``` -/chatops run feature set new_navigation_bar 25 -``` - -This will enable the feature for GitLab.com, with `new_navigation_bar` being the -name of the feature. We can also enable the feature for <https://dev.gitlab.org> -or <https://staging.gitlab.com>: - -``` -/chatops run feature set new_navigation_bar 25 --dev -/chatops run feature set new_navigation_bar 25 --staging -``` - -If you are not certain what percentages to use, simply use the following steps: - -1. 25% -1. 50% -1. 75% -1. 100% - -Between every step you'll want to wait a little while and monitor the -appropriate graphs on <https://dashboards.gitlab.net>. The exact time to wait -may differ. For some features a few minutes is enough, while for others you may -want to wait several hours or even days. This is entirely up to you, just make -sure it is clearly communicated to your team, and the Production team if you -anticipate any potential problems. - -Feature gates can also be actor based, for example a feature could first be -enabled for only the `gitlab-ce` project. The project is passed by supplying a -`--project` flag: - -``` -/chatops run feature set --project=gitlab-org/gitlab-ce some_feature true -``` - -For groups the `--group` flag is available: - -``` -/chatops run feature set --group=gitlab-org some_feature true -``` - -Once a change is deemed stable, submit a new merge request to remove the -feature flag. This ensures the change is available to all users and self-hosted -instances. Make sure to add the ~"feature flag" label to this merge request so -release managers are aware the changes are hidden behind a feature flag. If the -merge request has to be picked into a stable branch (e.g. after the 7th), make -sure to also add the appropriate "Pick into X" label (e.g. "Pick into 11.4"). - -One might be tempted to think this will delay the release of a feature by at -least one month (= one release). This is not the case. A feature flag does not -have to stick around for a specific amount of time (e.g. at least one release), -instead they should stick around until the feature is deemed stable. Stable -means it works on GitLab.com without causing any problems, such as outages. In -most cases this will translate to a feature (with a feature flag) being shipped -in RC1, followed by the feature flag being removed in RC2. This in turn means -the feature will be stable by the time we publish a stable package around the -22nd of the month. - -## Implicit feature flags - -The [`Project#feature_available?`][project-fa], -[`Namespace#feature_available?`][namespace-fa] (EE), and -[`License.feature_available?`][license-fa] (EE) methods all implicitly check for -a feature flag by the same name as the provided argument. - -For example if a feature is license-gated, there's no need to add an additional -explicit feature flag check since the flag will be checked as part of the -`License.feature_available?` call. Similarly, there's no need to "clean up" a -feature flag once the feature has reached general availability. - -You'd still want to use an explicit `Feature.enabled?` check if your new feature -isn't gated by a License or Plan. - -[project-fa]: https://gitlab.com/gitlab-org/gitlab-ee/blob/4cc1c62918aa4c31750cb21dfb1a6c3492d71080/app/models/project_feature.rb#L63-68 -[namespace-fa]: https://gitlab.com/gitlab-org/gitlab-ee/blob/4cc1c62918aa4c31750cb21dfb1a6c3492d71080/ee/app/models/ee/namespace.rb#L71-85 -[license-fa]: https://gitlab.com/gitlab-org/gitlab-ee/blob/4cc1c62918aa4c31750cb21dfb1a6c3492d71080/ee/app/models/license.rb#L293-300 - -### Undefined feature flags default to "on" - -An important side-effect of the [implicit feature flags](#implicit-feature-flags) -mentioned above is that unless the feature is explicitly disabled or limited to a -percentage of users, the feature flag check will default to `true`. - -As an example, if you were to ship the backend half of a feature behind a flag, -you'd want to explicitly disable that flag until the frontend half is also ready -to be shipped. You can do this via ChatOps: - -``` -/chatops run feature set some_feature 0 -``` - -Note that you can do this at any time, even before the merge request using the -flag has been merged! - -### Cleaning up - -When a feature gate has been removed from the code base, the value still exists -in the database. This can be removed through ChatOps: - -``` -/chatops run feature delete some_feature -``` +This document was moved to [another location](feature_flags/index.md). diff --git a/doc/development/testing_guide/end_to_end/quick_start_guide.md b/doc/development/testing_guide/end_to_end/quick_start_guide.md index f96c85be1ba..041bdf716b3 100644 --- a/doc/development/testing_guide/end_to_end/quick_start_guide.md +++ b/doc/development/testing_guide/end_to_end/quick_start_guide.md @@ -242,7 +242,7 @@ module QA issue = Resource::Issue.fabricate_via_api! do |issue| issue.title = 'Issue to test the scoped labels' - issue.labels = @initial_label + issue.labels = [@initial_label] end [@new_label_same_scope, @new_label_different_scope].each do |label| @@ -365,6 +365,14 @@ Add the following `attribute :id` and `attribute :labels` right above the [`attr > We add the attributes above the existing attribute to keep them alphabetically organized. +Then, let's initialize an instance variable for labels to allow an empty array as default value when such information is not passed during the resource fabrication, since this optional. [Between the attributes and the `fabricate!` method](https://gitlab.com/gitlab-org/gitlab-ee/blob/1a1f1408728f19b2aa15887cd20bddab7e70c8bd/qa/qa/resource/issue.rb#L18), add the following: + +```ruby +def initialize + @labels = [] +end +``` + Next, add the following code right below the [`fabricate!`](https://gitlab.com/gitlab-org/gitlab-ee/blob/d3584e80b4236acdf393d815d604801573af72cc/qa/qa/resource/issue.rb#L27) method. ```ruby @@ -378,7 +386,7 @@ end def api_post_body { - labels: [labels], + labels: labels, title: title } end diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md index fc9b175bc8a..28ebb6f0f64 100644 --- a/doc/development/testing_guide/frontend_testing.md +++ b/doc/development/testing_guide/frontend_testing.md @@ -27,7 +27,7 @@ we need to solve before being able to use Jest for all our needs. ### Differences to Karma - Jest runs in a Node.js environment, not in a browser. Support for running Jest tests in a browser [is planned](https://gitlab.com/gitlab-org/gitlab-ce/issues/58205). -- Because Jest runs in a Node.js environment, it uses [jsdom](https://github.com/jsdom/jsdom) by default. +- Because Jest runs in a Node.js environment, it uses [jsdom](https://github.com/jsdom/jsdom) by default. See also its [limitations](#limitations-of-jsdom) below. - Jest does not have access to Webpack loaders or aliases. The aliases used by Jest are defined in its [own config](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/jest.config.js). - All calls to `setTimeout` and `setInterval` are mocked away. See also [Jest Timer Mocks](https://jestjs.io/docs/en/timer-mocks). @@ -40,6 +40,17 @@ we need to solve before being able to use Jest for all our needs. - Unhandled Promise rejections. - Calls to `console.warn`, including warnings from libraries like Vue. +### Limitations of jsdom + +As mentioned [above](#differences-to-karma), Jest uses jsdom instead of a browser for running tests. +This comes with a number of limitations, namely: + +- [No scrolling support](https://github.com/jsdom/jsdom/blob/15.1.1/lib/jsdom/browser/Window.js#L623-L625) +- [No element sizes or positions](https://github.com/jsdom/jsdom/blob/15.1.1/lib/jsdom/living/nodes/Element-impl.js#L334-L371) +- [No layout engine](https://github.com/jsdom/jsdom/issues/1322) in general + +See also the issue for [support running Jest tests in browsers](https://gitlab.com/gitlab-org/gitlab-ce/issues/58205). + ### Debugging Jest tests Running `yarn jest-debug` will run Jest in debug mode, allowing you to debug/inspect as described in the [Jest docs](https://jestjs.io/docs/en/troubleshooting#tests-are-failing-and-you-don-t-know-why). diff --git a/doc/gitlab-basics/command-line-commands.md b/doc/gitlab-basics/command-line-commands.md index 1cf883679d7..b7e6844f43a 100644 --- a/doc/gitlab-basics/command-line-commands.md +++ b/doc/gitlab-basics/command-line-commands.md @@ -29,7 +29,9 @@ git clone PASTE HTTPS OR SSH HERE A clone of the project will be created in your computer. ->**Note:** If you clone your project via a URL that contains special characters, make sure that characters are URL-encoded. +NOTE: **Note:** +If you clone your project via a URL that contains special characters, make sure +that characters are URL-encoded. ### Go into a project directory to work in it @@ -86,12 +88,18 @@ cat README.md ### Remove a file +DANGER: **Danger:** +This will permanently delete the file. + ``` rm NAME-OF-FILE ``` ### Remove a directory and all of its contents +DANGER: **Danger:** +This will permanently delete the directory and all of its contents. + ``` rm -r NAME-OF-DIRECTORY ``` @@ -113,9 +121,13 @@ history You will be asked for an administrator’s password. ``` -sudo +sudo COMMAND ``` +CAUTION: **Caution:** +Be careful of the commands you run with `sudo`. Certain commands may cause +damage to your data and system. + ### Show which directory I am in ``` diff --git a/doc/install/requirements.md b/doc/install/requirements.md index 92122fca7dd..68c1bcbc801 100644 --- a/doc/install/requirements.md +++ b/doc/install/requirements.md @@ -9,7 +9,7 @@ as the hardware requirements that are needed to install and use GitLab. ## Operating Systems -### Supported Unix distributions +### Supported Linux distributions - Ubuntu - Debian @@ -21,7 +21,7 @@ as the hardware requirements that are needed to install and use GitLab. For the installations options, see [the main installation page](README.md). -### Unsupported Unix distributions +### Unsupported Linux distributions and Unix-like operating systems - Arch Linux - Fedora @@ -29,13 +29,13 @@ For the installations options, see [the main installation page](README.md). - Gentoo - macOS -On the above unsupported distributions is still possible to install GitLab yourself. +Installation of GitLab on these operating systems is possible, but not supported. Please see the [installation from source guide](installation.md) and the [installation guides](https://about.gitlab.com/installation/) for more information. -### Non-Unix operating systems such as Windows +### Microsoft Windows -GitLab is developed for Unix operating systems. -It does **not** run on Windows, and we have no plans to support it in the near future. For the latest development status view this [issue](https://gitlab.com/gitlab-org/gitlab-ce/issues/46567). +GitLab is developed for Linux-based operating systems. +It does **not** run on Microsoft Windows, and we have no plans to support it in the near future. For the latest development status view this [issue](https://gitlab.com/gitlab-org/gitlab-ce/issues/46567). Please consider using a virtual machine to run GitLab. ## Ruby versions diff --git a/doc/integration/README.md b/doc/integration/README.md index f74da97119a..135952a1b08 100644 --- a/doc/integration/README.md +++ b/doc/integration/README.md @@ -13,10 +13,10 @@ See the documentation below for details on how to configure these services. - [Auth0 OmniAuth](auth0.md) Enable the Auth0 OmniAuth provider - [Bitbucket](bitbucket.md) Import projects from Bitbucket.org and login to your GitLab instance with your Bitbucket.org account - [CAS](cas.md) Configure GitLab to sign in using CAS -- [External issue tracker](external-issue-tracker.md) Redmine, JIRA, etc. +- [External issue tracker](external-issue-tracker.md) Redmine, Jira, etc. - [Gmail actions buttons](gmail_action_buttons_for_gitlab.md) Adds GitLab actions to messages - [Jenkins](jenkins.md) Integrate with the Jenkins CI -- [JIRA](../user/project/integrations/jira.md) Integrate with the JIRA issue tracker +- [Jira](../user/project/integrations/jira.md) Integrate with the Jira issue tracker - [Kerberos](kerberos.md) Integrate with Kerberos - [LDAP](ldap.md) Set up sign in via LDAP - [OAuth2 provider](oauth_provider.md) OAuth2 application creation diff --git a/doc/push_rules/push_rules.md b/doc/push_rules/push_rules.md index b2d626a0a74..2142f5a5f69 100644 --- a/doc/push_rules/push_rules.md +++ b/doc/push_rules/push_rules.md @@ -26,11 +26,11 @@ Every push rule could have its own use case, but let's consider some examples. Let's assume you have the following requirements for your workflow: -- every commit should reference a JIRA issue, for example: `Refactored css. Fixes JIRA-123.` +- every commit should reference a Jira issue, for example: `Refactored css. Fixes JIRA-123.` - users should not be able to remove git tags with `git push` All you need to do is write a simple regular expression that requires the mention -of a JIRA issue in the commit message, like `JIRA\-\d+`. +of a Jira issue in the commit message, like `JIRA\-\d+`. Now when a user tries to push a commit with a message `Bugfix`, their push will be declined. Only pushing commits with messages like `Bugfix according to JIRA-123` diff --git a/doc/security/rack_attack.md b/doc/security/rack_attack.md index fa4b0d1fb09..8695b5d2194 100644 --- a/doc/security/rack_attack.md +++ b/doc/security/rack_attack.md @@ -53,8 +53,9 @@ For more information on how to use these options check out The following settings can be configured: - `enabled`: By default this is set to `false`. Set this to `true` to enable Rack Attack. -- `ip_whitelist`: Whitelist any IPs from being blocked. They must be formatted as strings within a ruby array. - For example, `["127.0.0.1", "127.0.0.2", "127.0.0.3"]`. +- `ip_whitelist`: Whitelist any IPs from being blocked. They must be formatted as strings within a Ruby array. + CIDR notation is supported in GitLab v12.1 and up. + For example, `["127.0.0.1", "127.0.0.2", "127.0.0.3", "192.168.0.1/24"]`. - `maxretry`: The maximum amount of times a request can be made in the specified time. - `findtime`: The maximum amount of time that failed requests can count against an IP diff --git a/doc/topics/application_development_platform/index.md b/doc/topics/application_development_platform/index.md index e8960a76dd9..8742606479d 100644 --- a/doc/topics/application_development_platform/index.md +++ b/doc/topics/application_development_platform/index.md @@ -1,12 +1,21 @@ # Application Development Platform -The GitLab Application Development Platform refers to the set of GitLab features that can be used by operations teams to -provide a full development environment to internal software development teams. +The GitLab Application Development Platform refers to the set of GitLab features used to create, configure, and manage +a complete software development environment. It provides development, operations, and security teams with a robust feature set aimed at supporting best practices out of the box. ## Overview -The GitLab Application Development Platform aims to reduce and even eliminate the time it takes for an Operations team -to provide a full environment for software developers. It comprises the following high-level elements: +The GitLab Application Development Platform aims to: + +- Reduce and even eliminate the time it takes for an Operations team + to provide a full environment for software developers. +- Get developers up and running fast so they can focus on writing + great applications with a robust development feature set. +- Provide best-of-breed security features so that applications developed + with GitLab are not affected by vulnerabilities that may lead to security + problems and unintended use. + +It is comprised of the following high-level elements: 1. Compute 1. Build, test, and deploy a wide range of applications diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index 702245b22a0..41d12128e51 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -723,7 +723,7 @@ also be customized, and you can easily use a [custom buildpack](#custom-buildpac | `AUTO_DEVOPS_CHART_REPOSITORY_USERNAME` | From Gitlab 11.11, this variable can be used to set a username to connect to the helm repository. Defaults to no credentials. (Also set AUTO_DEVOPS_CHART_REPOSITORY_PASSWORD) | | `AUTO_DEVOPS_CHART_REPOSITORY_PASSWORD` | From Gitlab 11.11, this variable can be used to set a password to connect to the helm repository. Defaults to no credentials. (Also set AUTO_DEVOPS_CHART_REPOSITORY_USERNAME) | | `REPLICAS` | The number of replicas to deploy; defaults to 1. | -| `PRODUCTION_REPLICAS` | The number of replicas to deploy in the production environment. This takes precedence over `REPLICAS`; defaults to 1. | +| `PRODUCTION_REPLICAS` | The number of replicas to deploy in the production environment. Takes precedence over `REPLICAS` and defaults to 1. For zero downtime upgrades, set to 2 or greater. | | `CANARY_REPLICAS` | The number of canary replicas to deploy for [Canary Deployments](../../user/project/canary_deployments.md); defaults to 1 | | `CANARY_PRODUCTION_REPLICAS` | The number of canary replicas to deploy for [Canary Deployments](../../user/project/canary_deployments.md) in the production environment. This takes precedence over `CANARY_REPLICAS`; defaults to 1 | | `ADDITIONAL_HOSTS` | Fully qualified domain names specified as a comma-separated list that are added to the ingress hosts. | diff --git a/doc/university/README.md b/doc/university/README.md index 1d2c0f2068a..9d861460618 100644 --- a/doc/university/README.md +++ b/doc/university/README.md @@ -181,7 +181,7 @@ The GitLab University curriculum is composed of GitLab videos, screencasts, pres ### 3.9. Integrations -1. [How to Integrate JIRA and Jenkins with GitLab - Video](https://gitlabmeetings.webex.com/gitlabmeetings/ldr.php?RCID=44b548147a67ab4d8a62274047146415) +1. [How to Integrate Jira and Jenkins with GitLab - Video](https://gitlabmeetings.webex.com/gitlabmeetings/ldr.php?RCID=44b548147a67ab4d8a62274047146415) 1. [How to Integrate Jira with GitLab](../user/project/integrations/jira.md) 1. [How to Integrate Jenkins with GitLab](../integration/jenkins.md) 1. [How to Integrate Bamboo with GitLab](../user/project/integrations/bamboo.md) diff --git a/doc/university/process/README.md b/doc/university/process/README.md index fdf6224f7f6..b278e02ccd5 100644 --- a/doc/university/process/README.md +++ b/doc/university/process/README.md @@ -9,11 +9,11 @@ title: University | Process # Suggesting improvements If you would like to teach a class or participate or help in any way please -submit a merge request and assign it to [Job](https://gitlab.com/u/JobV). +submit a merge request and assign it to [Job](https://gitlab.com/JobV). If you have suggestions for additional courses you would like to see, please submit a merge request to add an upcoming class, assign to -[Chad](https://gitlab.com/u/chadmalchow) and /cc [Job](https://gitlab.com/u/JobV). +[Chad](https://gitlab.com/chadmalchow) and /cc [Job](https://gitlab.com/JobV). ## Adding classes @@ -31,4 +31,4 @@ please submit a merge request to add an upcoming class, assign to 1. Please upload any video recordings to our Youtube channel. We prefer them to be public, if needed they can be unlisted but if so they should be linked from this page. -1. Please create a merge request and assign to [Erica](https://gitlab.com/u/Erica). +1. Please create a merge request and assign to [Erica](https://gitlab.com/Erica). diff --git a/doc/university/support/README.md b/doc/university/support/README.md index 35e65b60768..2c6e52acfde 100644 --- a/doc/university/support/README.md +++ b/doc/university/support/README.md @@ -80,7 +80,7 @@ Our integrations add great value to GitLab. User questions often relate to integ - Learn about our Integrations (specially, not only): - [LDAP](../../integration/ldap.md) - - [JIRA](../../project_services/jira.md) + - [Jira](../../project_services/jira.md) - [Jenkins](../../integration/jenkins.md) - [SAML](../../integration/saml.md) diff --git a/doc/user/admin_area/index.md b/doc/user/admin_area/index.md index fa60ee96cf7..d2947ae3371 100644 --- a/doc/user/admin_area/index.md +++ b/doc/user/admin_area/index.md @@ -21,7 +21,7 @@ The Admin Area is made up of the following sections: | Section | Description | |:------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | [Overview](#overview-section) | View your GitLab [Dashboard](#admin-dashboard), and administer [projects](#administering-projects), [users](#administering-users), [groups](#administering-groups), [jobs](#administering-jobs), [Runners](#administering-runners), and [Gitaly servers](#administering-gitaly-servers). | -| Monitoring | View GitLab system information, and information on background jobs, logs, [health checks](monitoring/health_check.md), request profiles, and audit logs. | +| Monitoring | View GitLab [system information](#system-info), and information on [background jobs](#background-jobs), [logs](#logs), [health checks](monitoring/health_check.md), [requests profiles](#requests-profiles), and [audit logs](#audit-log-premium-only). | | Messages | Send and manage [broadcast messages](broadcast_messages.md) for your users. | | System Hooks | Configure [system hooks](../../system_hooks/system_hooks.md) for many events. | | Applications | Create system [OAuth applications](../../integration/oauth_provider.md) for integrations with other services. | @@ -229,3 +229,66 @@ For each Gitaly server, the following details are listed: | Server version | Gitaly version | | Git version | Version of Git installed on the Gitaly server | | Up to date | Indicates if the Gitaly server version is the latest version available. A green dot indicates the server is up to date. | + +## Monitoring section + +The following topics document the **Monitoring** section of the Admin Area. + +### System Info + +The **System Info** page provides the following statistics: + +| Field | Description | +| :----------- | :---------- | +| CPU | Number of CPU cores available | +| Memory Usage | Memory in use, and total memory available | +| Disk Usage | Disk space in use, and total disk space available | +| Uptime | Approximate uptime of the GitLab instance | + +These statistics are updated only when you navigate to the **System Info** page, or you refresh the page in your browser. + +### Background Jobs + +The **Background Jobs** page displays the Sidekiq dashboard. Sidekiq is used by GitLab to +perform processing in the background. + +The Sidekiq dashboard consists of the following elements: + +- A tab per jobs' status. +- A breakdown of background job statistics. +- A live graph of **Processed** and **Failed** jobs, with a selectable polling interval. +- An historical graph of **Processed** and **Failed** jobs, with a selectable time span. +- Redis statistics, including: + - Version number + - Uptime, measured in days + - Number of connections + - Current memory usage, measured in MB + - Peak memory usage, measured in MB + +### Logs + +The **Logs** page provides access to the following log files: + +| Log file | Contents | +| :---------------------- | :------- | +| `application.log` | GitLab user activity | +| `githost.log` | Failed GitLab interaction with Git repositories | +| `production.log` | Requests received from Unicorn, and the actions taken to serve those requests | +| `sidekiq.log` | Background jobs | +| `repocheck.log` | Repository activity | +| `integrations_json.log` | Activity between GitLab and integrated systems | +| `kubernetes.log` | Kubernetes activity | + +The contents of these log files can be useful when troubleshooting a problem. Access is available to GitLab admins, without requiring direct access to the log files. + +For details of these log files and their contents, see [Log system](../../administration/logs.md). + +The content of each log file is listed in chronological order. To minimize performance issues, a maximum 2000 lines of each log file are shown. + +### Requests Profiles + +The **Requests Profiles** page contains the token required for profiling. For more details, see [Request Profiling](../../administration/monitoring/performance/request_profiling.md). + +### Audit Log **[PREMIUM ONLY]** + +The **Audit Log** page lists changes made within the GitLab server. With this information you can control, analyze, and track every change. diff --git a/doc/user/admin_area/settings/continuous_integration.md b/doc/user/admin_area/settings/continuous_integration.md index 5768a97e727..84596ff6a2c 100644 --- a/doc/user/admin_area/settings/continuous_integration.md +++ b/doc/user/admin_area/settings/continuous_integration.md @@ -94,13 +94,11 @@ a group in the **Usage Quotas** page available to the group page settings list.  -## Extra Shared Runners pipeline minutes quota +## Extra Shared Runners pipeline minutes quota **[FREE ONLY]** -NOTE: **Note:** -Only available on GitLab.com. - -You can purchase additional CI minutes so your pipelines will not be blocked after you have -used all your CI minutes from your main quota. +If you're using GitLab.com, you can purchase additional CI minutes so your +pipelines will not be blocked after you have used all your CI minutes from your +main quota. In order to purchase additional minutes, you should follow these steps: diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md index 4a2fb1d7190..9dfbe326f1d 100644 --- a/doc/user/application_security/container_scanning/index.md +++ b/doc/user/application_security/container_scanning/index.md @@ -206,6 +206,11 @@ vulnerabilities in your groups and projects. Read more about the Once a vulnerability is found, you can interact with it. Read more on how to [interact with the vulnerabilities](../index.md#interacting-with-the-vulnerabilities). +## Vulnerabilities database update + +For more information about the vulnerabilities database update, check the +[maintenance table](../index.md#maintenance-and-update-of-the-vulnerabilities-database). + ## Troubleshooting ### docker: Error response from daemon: failed to copy xattrs diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md index a722aa88f9d..2283efe3a44 100644 --- a/doc/user/application_security/dast/index.md +++ b/doc/user/application_security/dast/index.md @@ -259,3 +259,8 @@ vulnerabilities in your groups and projects. Read more about the Once a vulnerability is found, you can interact with it. Read more on how to [interact with the vulnerabilities](../index.md#interacting-with-the-vulnerabilities). + +## Vulnerabilities database update + +For more information about the vulnerabilities database update, check the +[maintenance table](../index.md#maintenance-and-update-of-the-vulnerabilities-database). diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md index ea8b96eb24d..9145e034dcb 100644 --- a/doc/user/application_security/dependency_scanning/index.md +++ b/doc/user/application_security/dependency_scanning/index.md @@ -404,6 +404,11 @@ vulnerabilities in your groups and projects. Read more about the Once a vulnerability is found, you can interact with it. Read more on how to [interact with the vulnerabilities](../index.md#interacting-with-the-vulnerabilities). +## Vulnerabilities database update + +For more information about the vulnerabilities database update, check the +[maintenance table](../index.md#maintenance-and-update-of-the-vulnerabilities-database). + ## Dependency List > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/10075) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.0. diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md index 679847b76d7..69fa1ec5da6 100644 --- a/doc/user/application_security/index.md +++ b/doc/user/application_security/index.md @@ -10,7 +10,7 @@ high-level view on projects and groups, and start remediation processes when nee GitLab can scan and report any vulnerabilities found in your project. -| Secure scanning tools | Description | +| Secure scanning tool | Description | |:-----------------------------------------------------------------------------|:-----------------------------------------------------------------------| | [Container Scanning](container_scanning/index.md) **[ULTIMATE]** | Scan Docker containers for known vulnerabilities. | | [Dependency Scanning](dependency_scanning/index.md) **[ULTIMATE]** | Analyze your dependencies for known vulnerabilities. | @@ -19,6 +19,29 @@ GitLab can scan and report any vulnerabilities found in your project. | [Security Dashboard](security_dashboard/index.md) **[ULTIMATE]** | View vulnerabilities in all your projects and groups. | | [Static Application Security Testing (SAST)](sast/index.md) **[ULTIMATE]** | Analyze source code for known vulnerabilities. | +## Maintenance and update of the vulnerabilities database + +The various scanning tools and the vulnerabilities database are updated regularly. + +| Secure scanning tool | Vulnerabilities database updates | +|:-------------------------------------------------------------|-------------------------------------------| +| [Container Scanning](container_scanning/index.md) | Uses `clair` underneath and the latest `clair-db` version is used for each job run by running the [`latest` docker image tag](https://gitlab.com/gitlab-org/gitlab-ee/blob/438a0a56dc0882f22bdd82e700554525f552d91b/lib/gitlab/ci/templates/Security/Container-Scanning.gitlab-ci.yml#L37). The `clair-db` database [is updated daily according to the author](https://github.com/arminc/clair-local-scan#clair-server-or-local). | +| [Dependency Scanning](dependency_scanning/index.md) | Relies on `bundler-audit` (for Rubygems), `retire.js` (for NPM packages) and `gemnasium` (GitLab's own tool for all libraries). `bundler-audit` and `retire.js` both fetch their vulnerabilities data from GitHub repositories, so vulnerabilities added to `ruby-advisory-db` and `retire.js` are immediately available. The tools themselves are updated once per month if there's a new version. The [Gemnasium DB](https://gitlab.com/gitlab-org/security-products/gemnasium-db) is updated at least once a week. | +| [Dynamic Application Security Testing (DAST)](dast/index.md) | Updated weekly on Sundays. The underlying tool, `zaproxy`, downloads fresh rules at startup. | +| [Static Application Security Testing (SAST)](sast/index.md) | Relies exclusively on [the tools GitLab is wrapping](sast/index.md#supported-languages-and-frameworks). The underlying analyzers are updated at least once per month if a relevant update is available. The vulnerabilities database is updated by the upstream tools. | + +You don't have to update GitLab to benefit from the latest vulnerabilities definitions, +but you may have to in the future. + +The security tools are released as Docker images, and the vendored job definitions +to enable them are using the `x-y-stable` image tags that get overridden each time a new +release of the tools is pushed. The Docker images are updated to match the +previous GitLab releases, so they automatically get the latest versions of the +scanning tools without the user having to do anything. + +This workflow comes with some drawbacks and there's a +[plan to change this](https://gitlab.com/gitlab-org/gitlab-ee/issues/9725). + ## Interacting with the vulnerabilities > Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing) 10.8. diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index ec3f7fbde76..9074ac3f4a1 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -269,7 +269,7 @@ it highlighted: "url": "https://cwe.mitre.org/data/definitions/330.html" } ] - }, + }, { "category": "sast", "message": "Probable insecure usage of temp file/directory.", @@ -296,7 +296,7 @@ it highlighted: "url": "https://docs.openstack.org/bandit/latest/plugins/b108_hardcoded_tmp_directory.html" } ] - }, + }, ], "remediations": [] } @@ -320,7 +320,7 @@ the report JSON unless stated otherwise. Presence of optional fields depends on | `vulnerabilities[].scanner` | A node that describes the analyzer used to find this vulnerability. | | `vulnerabilities[].scanner.id` | Id of the scanner as a snake_case string. | | `vulnerabilities[].scanner.name` | Name of the scanner, for display purposes. | -| `vulnerabilities[].location` | A node that tells where the vulnerability is located. | +| `vulnerabilities[].location` | A node that tells where the vulnerability is located. | | `vulnerabilities[].location.file` | Path to the file where the vulnerability is located. Optional. | | `vulnerabilities[].location.start_line` | The first line of the code affected by the vulnerability. Optional. | | `vulnerabilities[].location.end_line` | The last line of the code affected by the vulnerability. Optional. | @@ -330,7 +330,7 @@ the report JSON unless stated otherwise. Presence of optional fields depends on | `vulnerabilities[].identifiers[].type` | Type of the identifier. Possible values: common identifier types (among `cve`, `cwe`, `osvdb`, and `usn`) or analyzer-dependent ones (e.g., `bandit_test_id` for [Bandit analyzer](https://wiki.openstack.org/wiki/Security/Projects/Bandit)). | | `vulnerabilities[].identifiers[].name` | Name of the identifier for display purposes. | | `vulnerabilities[].identifiers[].value` | Value of the identifier for matching purposes. | -| `vulnerabilities[].identifiers[].url` | URL to identifier's documentation. Optional. | +| `vulnerabilities[].identifiers[].url` | URL to identifier's documentation. Optional. | ## Secret detection @@ -363,3 +363,8 @@ vulnerabilities in your groups and projects. Read more about the Once a vulnerability is found, you can interact with it. Read more on how to [interact with the vulnerabilities](../index.md#interacting-with-the-vulnerabilities). + +## Vulnerabilities database update + +For more information about the vulnerabilities database update, check the +[maintenance table](../index.md#maintenance-and-update-of-the-vulnerabilities-database). diff --git a/doc/user/group/clusters/index.md b/doc/user/group/clusters/index.md index 26d764fa2cf..8d4ffd93f59 100644 --- a/doc/user/group/clusters/index.md +++ b/doc/user/group/clusters/index.md @@ -138,14 +138,6 @@ The result will then be: - The Staging cluster will be used for the `deploy to staging` job. - The Production cluster will be used for the `deploy to production` job. -## Unavailable features - -The following features are not currently available for group-level clusters: - -1. Terminals (see [related issue](https://gitlab.com/gitlab-org/gitlab-ce/issues/55487)). -1. Pod logs (see [related issue](https://gitlab.com/gitlab-org/gitlab-ce/issues/55488)). -1. Deployment boards (see [related issue](https://gitlab.com/gitlab-org/gitlab-ce/issues/55489)). - <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/img/color_inline_colorchip_render_gfm.png b/doc/user/img/color_inline_colorchip_render_gfm.png Binary files differdeleted file mode 100644 index 6f93dbeeb10..00000000000 --- a/doc/user/img/color_inline_colorchip_render_gfm.png +++ /dev/null diff --git a/doc/user/img/markdown_inline_diffs_tags_rendered.png b/doc/user/img/markdown_inline_diffs_tags_rendered.png Binary files differdeleted file mode 100644 index 4279a20b5a0..00000000000 --- a/doc/user/img/markdown_inline_diffs_tags_rendered.png +++ /dev/null diff --git a/doc/user/img/math_inline_sup_render_gfm.png b/doc/user/img/math_inline_sup_render_gfm.png Binary files differdeleted file mode 100644 index 3ee2abb14df..00000000000 --- a/doc/user/img/math_inline_sup_render_gfm.png +++ /dev/null diff --git a/doc/user/img/task_list_ordered_render_gfm.png b/doc/user/img/task_list_ordered_render_gfm.png Binary files differdeleted file mode 100644 index 98ec791e958..00000000000 --- a/doc/user/img/task_list_ordered_render_gfm.png +++ /dev/null diff --git a/doc/user/index.md b/doc/user/index.md index 1fc4e4c43cf..899026a801f 100644 --- a/doc/user/index.md +++ b/doc/user/index.md @@ -66,7 +66,7 @@ With GitLab Enterprise Edition, you can also: - Leverage continuous delivery method with [Canary Deployments](project/canary_deployments.md). - Scan your code for vulnerabilities and [display them in merge requests](application_security/sast/index.md). -You can also [integrate](project/integrations/project_services.md) GitLab with numerous third-party applications, such as Mattermost, Microsoft Teams, HipChat, Trello, Slack, Bamboo CI, JIRA, and a lot more. +You can also [integrate](project/integrations/project_services.md) GitLab with numerous third-party applications, such as Mattermost, Microsoft Teams, HipChat, Trello, Slack, Bamboo CI, Jira, and a lot more. ## Projects @@ -153,7 +153,7 @@ you have quick access to. You can also gather feedback on them through ## Integrations [Integrate GitLab](../integration/README.md) with your preferred tool, -such as Trello, JIRA, etc. +such as Trello, Jira, etc. ## Webhooks diff --git a/doc/user/markdown.md b/doc/user/markdown.md index 31c8093ced7..16df6d93277 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -1,15 +1,20 @@ # GitLab Markdown -This markdown guide is **valid for GitLab's system markdown entries and files**. -It is not valid for the [GitLab documentation website](https://docs.gitlab.com) -nor [GitLab's main website](https://about.gitlab.com), as they both use -[Kramdown](https://kramdown.gettalong.org) as their markdown engine. -The documentation website uses an extended Kramdown gem, [GitLab Kramdown](https://gitlab.com/gitlab-org/gitlab_kramdown). -Consult the [GitLab Kramdown Guide](https://about.gitlab.com/handbook/product/technical-writing/markdown-guide/) for a complete Kramdown reference. +This markdown guide is **valid only for GitLab's internal markdown rendering system for entries and files**. +It is **not** valid for the [GitLab documentation website](https://docs.gitlab.com) +or [GitLab's main website](https://about.gitlab.com), as they both use +[Kramdown](https://kramdown.gettalong.org) as their markdown engine. The documentation +website uses an extended Kramdown gem, [GitLab Kramdown](https://gitlab.com/gitlab-org/gitlab_kramdown). +Consult the [GitLab Kramdown Guide](https://about.gitlab.com/handbook/product/technical-writing/markdown-guide/) +for a complete Kramdown reference. + +NOTE: **Note:** We encourage you to view this document as [rendered by GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md). ## GitLab Flavored Markdown (GFM) -GitLab uses "GitLab Flavored Markdown" (GFM). It extends the [CommonMark specification][commonmark-spec] (which is based on standard Markdown) in a few significant ways to add additional useful functionality. It was inspired by [GitHub Flavored Markdown](https://help.github.com/articles/basic-writing-and-formatting-syntax/). +GitLab uses "GitLab Flavored Markdown" (GFM). It extends the [CommonMark specification](https://spec.commonmark.org/current/) +(which is based on standard Markdown) in several ways to add additional useful functionality. +It was inspired by [GitHub Flavored Markdown](https://help.github.com/articles/basic-writing-and-formatting-syntax/). You can use GFM in the following areas: @@ -22,35 +27,29 @@ You can use GFM in the following areas: - Markdown documents inside repositories - Epics **[ULTIMATE]** -You can also use other rich text files in GitLab. You might have to install a -dependency to do so. Please see the [`github-markup` gem readme](https://github.com/gitlabhq/markup#markups) for more information. - -> **Notes:** -> -> We encourage you to view this document as [rendered by GitLab itself](markdown.md). -> -> As of 11.1, GitLab uses the [CommonMark Ruby Library][commonmarker] for Markdown -processing of all new issues, merge requests, comments, and other Markdown content -in the GitLab system. As of 11.3, wiki pages and Markdown files (`.md`) in the -repositories are also processed with CommonMark. As of 11.8, the [Redcarpet -Ruby library][redcarpet] has been removed and all issues/comments, including -those from pre-11.1, are now processed using [CommonMark Ruby -Library][commonmarker]. -> -> The documentation website had its [markdown engine migrated from Redcarpet to Kramdown](https://gitlab.com/gitlab-com/gitlab-docs/merge_requests/108) -in October 2018. -> -> _Where there are significant differences, we will try to call them out in this document._ +You can also use other rich text files in GitLab. You might have to install a dependency +to do so. Please see the [`gitlab-markup` gem project](https://gitlab.com/gitlab-org/gitlab-markup) +for more information. + +### Transition from Redcarpet to CommonMark -### Transitioning to CommonMark +Since 11.1, GitLab uses the [CommonMark Ruby Library](https://github.com/gjtorikian/commonmarker) +for Markdown processing of all new issues, merge requests, comments, and other Markdown +content in the GitLab system. Since 11.3, wiki pages and Markdown files (`*.md`) in +repositories are also processed with CommonMark. As of 11.8, the [Redcarpet Ruby library](https://github.com/vmg/redcarpet) +has been removed and all issues and comments, including those from pre-11.1, are now processed +using the [CommonMark Ruby Library](https://github.com/gjtorikian/commonmarker). -You may have older issues/merge requests or Markdown documents in your -repository that were written using some of the nuances of RedCarpet's version +The documentation website had its [markdown engine migrated from Redcarpet to Kramdown](https://gitlab.com/gitlab-com/gitlab-docs/merge_requests/108) +in October 2018. + +You may have older issues, merge requests, or Markdown documents in your +repository that were written using some of the nuances of GitLab's RedCarpet version of Markdown. Since CommonMark uses a slightly stricter syntax, these documents -may now display a little strangely since we've transitioned to CommonMark. -Numbered lists with nested lists in particular can be displayed incorrectly. +may now display a little differently since we've transitioned to CommonMark. -It is usually quite easy to fix. In the case of a nested list such as this: +It is usually quite easy to fix. For example, numbered lists with nested lists may +render incorrectly: ```markdown 1. Chocolate @@ -58,7 +57,14 @@ It is usually quite easy to fix. In the case of a nested list such as this: - milk ``` -simply add a space to each nested item: +1. Chocolate + - dark + - milk + +--- + +Simply add a space to each nested item to align the `-` with the first character of +the top list item (`C` in this case): ```markdown 1. Chocolate @@ -66,515 +72,674 @@ simply add a space to each nested item: - milk ``` -In the documentation below, we try to highlight some of the differences. +1. Chocolate + - dark + - milk + +NOTE: **Note:** We will flag any significant differences between Redcarpet and CommonMark + markdown in this document. If you have a large volume of Markdown files, it can be tedious to determine -if they will be displayed correctly or not. You can use the +if they will display correctly or not. You can use the [diff_redcarpet_cmark](https://gitlab.com/digitalmoksha/diff_redcarpet_cmark) -tool (not an officially supported product) to generate a list of files and +tool (not an officially supported product) to generate a list of files, and the differences between how RedCarpet and CommonMark render the files. It can give -you a great idea if anything needs to be changed - many times nothing will need -to changed. +an indication if anything needs to be changed - often nothing will need +to change. + +### GFM extends standard markdown + +GitLab makes full use of the standard (CommonMark) formatting, but also includes additional +functionality useful for GitLab users. + +It makes use of [new markdown features](#new-GFM-markdown-extensions), +not found in standard markdown: + +- [Color "chips" written in HEX, RGB or HSL](#colors) +- [Diagrams and flowcharts using Mermaid](#diagrams-and-flowcharts-using-mermaid) +- [Emoji](#emoji) +- [Front matter](#front-matter) +- [Inline diffs](#inline-diff) +- [Math equations and symbols written in LaTeX](#math) +- [Special GitLab references](#special-gitlab-references) +- [Task Lists](#task-lists) +- [Wiki specific markdown](#wiki-specific-markdown) + +It also has [extended markdown features](#standard-markdown-and-extensions-in-gitlab), without +changing how standard markdown is used: + +| Standard markdown | Extended markdown in GitLab | +| ------------------------------------- | ------------------------- | +| [blockquotes](#blockquotes) | [multiline blockquotes](#multiline-blockquote) | +| [code blocks](#code-spans-and-blocks) | [colored code and syntax highlighting](#colored-code-and-syntax-highlighting) | +| [emphasis](#emphasis) | [multiple underscores in words](#multiple-underscores-in-words-and-mid-word-emphasis) +| [headers](#headers) | [linkable Header IDs](#header-ids-and-links) | +| [images](#images) | [embedded videos](#videos) | +| [linebreaks](#line-breaks) | [more linebreak control](#newlines) | +| [links](#links) | [automatically linking URLs](#url-auto-linking) | + +## New GFM markdown extensions + +### Colors -### Newlines +> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#colors). -> If this is not rendered correctly, see -https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#newlines +It is possible to have color written in HEX, RGB or HSL format rendered with a color +indicator. -GFM honors the markdown specification in how [paragraphs and line breaks are handled][commonmark-spec]. +Supported formats (named colors are not supported): -A paragraph is simply one or more consecutive lines of text, separated by one or -more blank lines. -Line-breaks, or soft returns, are rendered if you end a line with two or more spaces: +- HEX: `` `#RGB[A]` `` or `` `#RRGGBB[AA]` `` +- RGB: `` `RGB[A](R, G, B[, A])` `` +- HSL: `` `HSL[A](H, S, L[, A])` `` -<!-- (Do *NOT* remove the two ending whitespaces in the following line.) --> -<!-- (They are needed for the Markdown text to render correctly.) --> - Roses are red [followed by two or more spaces] - Violets are blue +Color written inside backticks will be followed by a color "chip": - Sugar is sweet +```markdown +`#F00` +`#F00A` +`#FF0000` +`#FF0000AA` +`RGB(0,255,0)` +`RGB(0%,100%,0%)` +`RGBA(0,255,0,0.3)` +`HSL(540,70%,50%)` +`HSLA(540,70%,50%,0.3)` +``` -<!-- (Do *NOT* remove the two ending whitespaces in the following line.) --> -<!-- (They are needed for the Markdown text to render correctly.) --> -Roses are red -Violets are blue +`#F00` +`#F00A` +`#FF0000` +`#FF0000AA` +`RGB(0,255,0)` +`RGB(0%,100%,0%)` +`RGBA(0,255,0,0.3)` +`HSL(540,70%,50%)` +`HSLA(540,70%,50%,0.3)` -Sugar is sweet +### Diagrams and flowcharts using Mermaid -### Multiple underscores in words +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/15107) in +GitLab 10.3. -> If this is not rendered correctly, see -https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#multiple-underscores-in-words +It is possible to generate diagrams and flowcharts from text using [Mermaid](https://mermaidjs.github.io/). +Visit the official page for more details. -It is not reasonable to italicize just _part_ of a word, especially when you're -dealing with code and names that often appear with multiple underscores. -Therefore, GFM ignores multiple underscores in words: +In order to generate a diagram or flowchart, you should write your text inside the `mermaid` block: - perform_complicated_task +~~~ +```mermaid +graph TD; + A-->B; + A-->C; + B-->D; + C-->D; +``` +~~~ - do_this_and_do_that_and_another_thing +```mermaid +graph TD; + A-->B; + A-->C; + B-->D; + C-->D; +``` -perform_complicated_task +### Emoji -do_this_and_do_that_and_another_thing +> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#emoji). -### URL auto-linking +```md +Sometimes you want to :monkey: around a bit and add some :star2: to your :speech_balloon:. Well we have a gift for you: -> If this is not rendered correctly, see -https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#url-auto-linking +:zap: You can use emoji anywhere GFM is supported. :v: -GFM will autolink almost any URL you copy and paste into your text: +You can use it to point out a :bug: or warn about :speak_no_evil: patches. And if someone improves your really :snail: code, send them some :birthday:. People will :heart: you for that. - * https://www.google.com - * https://google.com/ - * ftp://ftp.us.debian.org/debian/ - * smb://foo/bar/baz - * irc://irc.freenode.net/gitlab - * http://localhost:3000 +If you are new to this, don't be :fearful:. You can easily join the emoji :family:. All you need to do is to look up one of the supported codes. -* https://www.google.com -* https://google.com/ -* ftp://ftp.us.debian.org/debian/ -* <a href="smb://foo/bar/baz">smb://foo/bar/baz</a> -* <a href="irc://irc.freenode.net/gitlab">irc://irc.freenode.net/gitlab</a> -* http://localhost:3000 +Consult the [Emoji Cheat Sheet](https://www.emojicopy.com) for a list of all supported emoji codes. :thumbsup: +``` -### Multiline blockquote +Sometimes you want to <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/monkey.png" width="20px" height="20px" style="display:inline;margin:0"> around a bit and add some <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/star2.png" width="20px" height="20px" style="display:inline;margin:0"> to your <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/speech_balloon.png" width="20px" height="20px" style="display:inline;margin:0">. Well we have a gift for you: -> If this is not rendered correctly, see -https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#multiline-blockquote +<img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/zap.png" width="20px" height="20px" style="display:inline;margin:0">You can use emoji anywhere GFM is supported. <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/v.png" width="20px" height="20px" style="display:inline;margin:0"> -On top of standard Markdown [blockquotes](#blockquotes), which require prepending `>` to quoted lines, -GFM supports multiline blockquotes fenced by <code>>>></code>: +You can use it to point out a <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/bug.png" width="20px" height="20px" style="display:inline;margin:0"> or warn about <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/speak_no_evil.png" width="20px" height="20px" style="display:inline;margin:0"> patches. And if someone improves your really <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/snail.png" width="20px" height="20px" style="display:inline;margin:0"> code, send them some <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/birthday.png" width="20px" height="20px" style="display:inline;margin:0">. People will <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/heart.png" width="20px" height="20px" style="display:inline;margin:0"> you for that. -``` ->>> -If you paste a message from somewhere else +If you are new to this, don't be <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/fearful.png" width="20px" height="20px" style="display:inline;margin:0">. You can easily join the emoji <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/family.png" width="20px" height="20px" style="display:inline;margin:0">. All you need to do is to look up one of the supported codes. -that +Consult the [Emoji Cheat Sheet](https://www.webfx.com/tools/emoji-cheat-sheet/) for a list of all supported emoji codes. <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/thumbsup.png" width="20px" height="20px" style="display:inline;margin:0"> -spans +> **Note:** The emoji example above uses hard-coded images for this documentation. The emoji, +when rendered within GitLab, may appear different depending on the OS and browser used. -multiple lines, +Most emoji are natively supported on macOS, Windows, iOS, Android and will fallback to image-based emoji where there is lack of support. -you can quote that without having to manually prepend `>` to every line! ->>> -``` +NOTE: **Note:** On Linux, you can download [Noto Color Emoji](https://www.google.com/get/noto/help/emoji/) +to get full native emoji support. Ubuntu 18.04 (like many modern Linux distros) has +this font installed by default. -<blockquote dir="auto"> -<p>If you paste a message from somewhere else</p> -<p>that</p> -<p>spans</p> -<p>multiple lines,</p> -<p>you can quote that without having to manually prepend <code>></code> to every line!</p> -</blockquote> +### Front matter -### Code and syntax highlighting +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/23331) + in GitLab 11.6. -> If this is not rendered correctly, see -https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#code-and-syntax-highlighting +Front matter is metadata included at the beginning of a markdown document, preceding +its content. This data can be used by static site generators such as [Jekyll](https://jekyllrb.com/docs/front-matter/), +[Hugo](https://gohugo.io/content-management/front-matter/), and many other applications. -_GitLab uses the [Rouge Ruby library][rouge] for syntax highlighting. For a -list of supported languages visit the Rouge website._ +When you view a Markdown file rendered by GitLab, any front matter is displayed as-is, +in a box at the top of the document, before the rendered HTML content. To view an example, +you can toggle between the source and rendered version of a [GitLab documentation file](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/README.md). -Blocks of code are either fenced by lines with three back-ticks <code>```</code>, -or are indented with four spaces. Only the fenced code blocks support syntax -highlighting: +In GitLab, front matter is only used in Markdown files and wiki pages, not the other +places where Markdown formatting is supported. It must be at the very top of the document, +and must be between delimiters, as explained below. -``` -Inline `code` has `back-ticks around` it. -``` +The following delimeters are supported: -Inline `code` has `back-ticks around` it. +- YAML (`---`): -Example: + ~~~yaml + --- + title: About Front Matter + example: + language: yaml + --- + ~~~ - ```javascript - var s = "JavaScript syntax highlighting"; - alert(s); - ``` +- TOML (`+++`): - ```python - def function(): - #indenting works just fine in the fenced code block - s = "Python syntax highlighting" - print s - ``` + ~~~toml + +++ + title = "About Front Matter" + [example] + language = "toml" + +++ + ~~~ - ```ruby - require 'redcarpet' - markdown = Redcarpet.new("Hello World!") - puts markdown.to_html - ``` +- JSON (`;;;`): - ``` - No language indicated, so no syntax highlighting. - s = "There is no highlighting for this." - But let's throw in a <b>tag</b>. - ``` + ~~~json + ;;; + { + "title": "About Front Matter" + "example": { + "language": "json" + } + } + ;;; + ~~~ -becomes: +Other languages are supported by adding a specifier to any of the existing +delimiters. For example: -```javascript -var s = "JavaScript syntax highlighting"; -alert(s); +```php +---php +$title = "About Front Matter"; +$example = array( + 'language' => "php", +); +--- ``` -```python -def function(): - #indenting works just fine in the fenced code block - s = "Python syntax highlighting" - print s -``` +### Inline diff -```ruby -require 'redcarpet' -markdown = Redcarpet.new("Hello World!") -puts markdown.to_html -``` +> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#inline-diff). -``` -No language indicated, so no syntax highlighting. -s = "There is no highlighting for this." -But let's throw in a <b>tag</b>. -``` +With inline diff tags you can display {+ additions +} or [- deletions -]. -### Inline diff +The wrapping tags can be either curly braces or square brackets: -> If this is not rendered correctly, see -https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#inline-diff +```markdown +- {+ addition 1 +} +- [+ addition 2 +] +- {- deletion 3 -} +- [- deletion 4 -] +``` -With inline diffs tags you can display {+ additions +} or [- deletions -]. +- {+ addition 1 +} +- [+ addition 2 +] +- {- deletion 3 -} +- [- deletion 4 -] -The wrapping tags can be either curly braces or square brackets. +--- -Examples: +However the wrapping tags cannot be mixed: +```markdown +- {+ addition +] +- [+ addition +} +- {- deletion -] +- [- deletion -} ``` -- {+ additions +} -- [+ additions +] -- {- deletions -} -- [- deletions -] -``` -becomes: +### Math + +> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#math). + +It is possible to have math written with LaTeX syntax rendered using [KaTeX](https://github.com/Khan/KaTeX). - +Math written between dollar signs `$` will be rendered inline with the text. Math written +inside a [code block](#code-spans-and-blocks) with the language declared as `math`, will be rendered +on a separate line: -However the wrapping tags cannot be mixed as such: +~~~ +This math is inline $`a^2+b^2=c^2`$. +This is on a separate line + +```math +a^2+b^2=c^2 ``` -- {+ additions +] -- [+ additions +} -- {- deletions -] -- [- deletions -} +~~~ + +This math is inline $`a^2+b^2=c^2`$. + +This is on a separate line + +```math +a^2+b^2=c^2 ``` -### Emoji +_Be advised that KaTeX only supports a [subset](https://katex.org/docs/supported.html) of LaTeX._ -```md -Sometimes you want to :monkey: around a bit and add some :star2: to your :speech_balloon:. Well we have a gift for you: +NOTE: **Note:** This also works for the asciidoctor `:stem: latexmath`. For details see +the [asciidoctor user manual](http://asciidoctor.org/docs/user-manual/#activating-stem-support). -:zap: You can use emoji anywhere GFM is supported. :v: +### Special GitLab references -You can use it to point out a :bug: or warn about :speak_no_evil: patches. And if someone improves your really :snail: code, send them some :birthday:. People will :heart: you for that. +GFM recognizes special GitLab related references. For example, you can easily reference +an issue, a commit, a team member or even the whole team within a project. GFM will turn +that reference into a link so you can navigate between them easily. -If you are new to this, don't be :fearful:. You can easily join the emoji :family:. All you need to do is to look up one of the supported codes. +Additionally, GFM recognizes certain cross-project references, and also has a shorthand +version to reference other projects from the same namespace. -Consult the [Emoji Cheat Sheet](https://www.emojicopy.com) for a list of all supported emoji codes. :thumbsup: +GFM will recognize the following: -Most emoji are natively supported on macOS, Windows, iOS, Android and will fallback to image-based emoji where there is lack of support. +| references | input | cross-project reference | shortcut within same namespace | +| :------------------------------ | :------------------------- | :-------------------------------------- | :----------------------------- | +| specific user | `@user_name` | | | +| specific group | `@group_name` | | | +| entire team | `@all` | | | +| project | `namespace/project>` | | | +| issue | ``#123`` | `namespace/project#123` | `project#123` | +| merge request | `!123` | `namespace/project!123` | `project!123` | +| snippet | `$123` | `namespace/project$123` | `project$123` | +| epic **[ULTIMATE]** | `&123` | `group1/subgroup&123` | | +| label by ID | `~123` | `namespace/project~123` | `project~123` | +| one-word label by name | `~bug` | `namespace/project~bug` | `project~bug` | +| multi-word label by name | `~"feature request"` | `namespace/project~"feature request"` | `project~"feature request"` | +| project milestone by ID | `%123` | `namespace/project%123` | `project%123` | +| one-word milestone by name | `%v1.23` | `namespace/project%v1.23` | `project%v1.23` | +| multi-word milestone by name | `%"release candidate"` | `namespace/project%"release candidate"` | `project%"release candidate"` | +| specific commit | `9ba12248` | `namespace/project@9ba12248` | `project@9ba12248` | +| commit range comparison | `9ba12248...b19a04f5` | `namespace/project@9ba12248...b19a04f5` | `project@9ba12248...b19a04f5` | +| repository file references | `[README](doc/README)` | | | +| repository file line references | `[README](doc/README#L13)` | | | -On Linux, you can download [Noto Color Emoji](https://www.google.com/get/noto/help/emoji/) to get full native emoji support. +### Task lists + +> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#task-lists). + +You can add task lists anywhere markdown is supported, but you can only "click" to +toggle the boxes if they are in issues, merge requests, or comments. In other places +you must edit the markdown manually to change the status by adding or removing the `x`. -Ubuntu 18.04 (like many modern Linux distros) has this font installed by default. +To create a task list, add a specially-formatted Markdown list. You can use either +unordered or ordered lists: + +```markdown +- [x] Completed task +- [ ] Incomplete task + - [ ] Sub-task 1 + - [x] Sub-task 2 + - [ ] Sub-task 3 +1. [x] Completed task +1. [ ] Incomplete task + 1. [ ] Sub-task 1 + 1. [x] Sub-task 2 ``` -Sometimes you want to <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/monkey.png" width="20px" height="20px" style="display:inline;margin:0"> around a bit and add some <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/star2.png" width="20px" height="20px" style="display:inline;margin:0"> to your <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/speech_balloon.png" width="20px" height="20px" style="display:inline;margin:0">. Well we have a gift for you: +- [x] Completed task +- [ ] Incomplete task + - [ ] Sub-task 1 + - [x] Sub-task 2 + - [ ] Sub-task 3 +1. [x] Completed task +1. [ ] Incomplete task + 1. [ ] Sub-task 1 + 1. [x] Sub-task 2 -<img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/zap.png" width="20px" height="20px" style="display:inline;margin:0">You can use emoji anywhere GFM is supported. <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/v.png" width="20px" height="20px" style="display:inline;margin:0"> +### Wiki-specific Markdown -You can use it to point out a <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/bug.png" width="20px" height="20px" style="display:inline;margin:0"> or warn about <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/speak_no_evil.png" width="20px" height="20px" style="display:inline;margin:0"> patches. And if someone improves your really <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/snail.png" width="20px" height="20px" style="display:inline;margin:0"> code, send them some <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/birthday.png" width="20px" height="20px" style="display:inline;margin:0">. People will <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/heart.png" width="20px" height="20px" style="display:inline;margin:0"> you for that. +The following examples show how links inside wikis behave. -If you are new to this, don't be <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/fearful.png" width="20px" height="20px" style="display:inline;margin:0">. You can easily join the emoji <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/family.png" width="20px" height="20px" style="display:inline;margin:0">. All you need to do is to look up one of the supported codes. +#### Wiki - Direct page link -Consult the [Emoji Cheat Sheet](https://www.webfx.com/tools/emoji-cheat-sheet/) for a list of all supported emoji codes. <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/thumbsup.png" width="20px" height="20px" style="display:inline;margin:0"> +A link which just includes the slug for a page will point to that page, +_at the base level of the wiki_. -Most emoji are natively supported on macOS, Windows, iOS, Android and will fallback to image-based emoji where there is lack of support. +This snippet would link to a `documentation` page at the root of your wiki: -On Linux, you can download [Noto Color Emoji](https://www.google.com/get/noto/help/emoji/) to get full native emoji support. +```markdown +[Link to Documentation](documentation) +``` -Ubuntu 18.04 (like many modern Linux distros) has this font installed by default. +#### Wiki - Direct file link -### Special GitLab references +Links with a file extension point to that file, _relative to the current page_. -GFM recognizes special references. +If the snippet below was placed on a page at `<your_wiki>/documentation/related`, +it would link to `<your_wiki>/documentation/file.md`: -You can easily reference e.g. an issue, a commit, a team member or even the whole team within a project. +```markdown +[Link to File](file.md) +``` -GFM will turn that reference into a link so you can navigate between them easily. +#### Wiki - Hierarchical link -GFM will recognize the following: +A link can be constructed relative to the current wiki page using `./<page>`, +`../<page>`, etc. -| input | references | -|:---------------------------|:--------------------------------| -| `@user_name` | specific user | -| `@group_name` | specific group | -| `@all` | entire team | -| `namespace/project>` | project | -| `#12345` | issue | -| `!123` | merge request | -| `$123` | snippet | -| `&123` | epic **[ULTIMATE]** | -| `~123` | label by ID | -| `~bug` | one-word label by name | -| `~"feature request"` | multi-word label by name | -| `%123` | project milestone by ID | -| `%v1.23` | one-word milestone by name | -| `%"release candidate"` | multi-word milestone by name | -| `9ba12248` | specific commit | -| `9ba12248...b19a04f5` | commit range comparison | -| `[README](doc/README)` | repository file references | -| `[README](doc/README#L13)` | repository file line references | - -GFM also recognizes certain cross-project references: - -| input | references | -|:----------------------------------------|:------------------------| -| `namespace/project#123` | issue | -| `namespace/project!123` | merge request | -| `namespace/project%123` | project milestone | -| `namespace/project$123` | snippet | -| `namespace/project@9ba12248` | specific commit | -| `group1/subgroup&123` | epic **[ULTIMATE]** | -| `namespace/project@9ba12248...b19a04f5` | commit range comparison | -| `namespace/project~"Some label"` | issues with given label | - -It also has a shorthand version to reference other projects from the same namespace: - -| input | references | -|:------------------------------|:------------------------| -| `project#123` | issue | -| `project!123` | merge request | -| `project%123` | project milestone | -| `project$123` | snippet | -| `project@9ba12248` | specific commit | -| `project@9ba12248...b19a04f5` | commit range comparison | -| `project~"Some label"` | issues with given label | +If this snippet was placed on a page at `<your_wiki>/documentation/main`, +it would link to `<your_wiki>/documentation/related`: -### Task lists +```markdown +[Link to Related Page](./related) +``` + +If this snippet was placed on a page at `<your_wiki>/documentation/related/content`, +it would link to `<your_wiki>/documentation/main`: -> If this is not rendered correctly, see -https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#task-lists +```markdown +[Link to Related Page](../main) +``` -You can add task lists to issues, merge requests and comments. To create a task list, add a specially-formatted Markdown list, like so: +If this snippet was placed on a page at `<your_wiki>/documentation/main`, +it would link to `<your_wiki>/documentation/related.md`: +```markdown +[Link to Related Page](./related.md) ``` -- [x] Completed task -- [ ] Incomplete task - - [ ] Sub-task 1 - - [x] Sub-task 2 - - [ ] Sub-task 3 + +If this snippet was placed on a page at `<your_wiki>/documentation/related/content`, +it would link to `<your_wiki>/documentation/main.md`: + +```markdown +[Link to Related Page](../main.md) ``` - +#### Wiki - Root link -Tasks formatted as ordered lists are supported as well: +A link starting with a `/` is relative to the wiki root. + +This snippet links to `<wiki_root>/documentation`: +```markdown +[Link to Related Page](/documentation) ``` -1. [x] Completed task -1. [ ] Incomplete task - 1. [ ] Sub-task 1 - 1. [x] Sub-task 2 + +This snippet links to `<wiki_root>/miscellaneous.md`: + +```markdown +[Link to Related Page](/miscellaneous.md) ``` - +## Standard markdown and extensions in GitLab -Task lists can only be created in descriptions, not in titles. Task item state can be managed by editing the description's Markdown or by toggling the rendered check boxes. +All standard markdown formatting should work as expected within GitLab. Some standard +functionality is extended with additional features, without affecting the standard usage. +If a functionality is extended, the new option will be listed as a sub-section. -### Videos +### Blockquotes -> If this is not rendered correctly, see -https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#videos +Blockquotes are an easy way to highlight information, such as a side-note. It is generated +by starting the lines of the blockquote with `>`: -Image tags with a video extension are automatically converted to a video player. +```markdown +> Blockquotes are very handy to emulate reply text. +> This line is part of the same quote. -The valid video extensions are `.mp4`, `.m4v`, `.mov`, `.webm`, and `.ogv`. +Quote break. - Here's a sample video: +> This is a very long line that will still be quoted properly when it wraps. Oh boy let's keep writing to make sure this is long enough to actually wrap for everyone. Oh, you can *put* **Markdown** into a blockquote. +``` -  +> Blockquotes are very handy to emulate reply text. +> This line is part of the same quote. -Here's a sample video: +Quote break. -<div class="video-container"> - <video src="img/markdown_video.mp4" width="400" controls="true" data-setup="{}" data-title="Sample Video"></video> - <p><a href="img/markdown_video.mp4" target="_blank" rel="noopener noreferrer" title="Download 'Sample Video'">Sample Video</a></p> -</div> +> This is a very long line that will still be quoted properly when it wraps. Oh boy let's keep writing to make sure this is long enough to actually wrap for everyone. Oh, you can *put* **Markdown** into a blockquote. -### Math +#### Multiline blockquote -> If this is not rendered correctly, see -https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#math +> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#multiline-blockquote). -It is possible to have math written with the LaTeX syntax rendered using [KaTeX][katex]. +GFM extends the standard markdown standard by also supporting multiline blockquotes +fenced by `>>>`: -Math written inside ```$``$``` will be rendered inline with the text. +``` +>>> +If you paste a message from somewhere else -Math written inside triple back quotes, with the language declared as `math`, will be rendered on a separate line. +that spans multiple lines, -Example: +you can quote that without having to manually prepend `>` to every line! +>>> +``` - This math is inline $`a^2+b^2=c^2`$. +>>> +If you paste a message from somewhere else - This is on a separate line - ```math - a^2+b^2=c^2 - ``` +that spans multiple lines, -Becomes: +you can quote that without having to manually prepend `>` to every line! +>>> -This math is inline . +### Code spans and blocks -This is on a separate line +You can easily highlight anything that should be viewed as code and not simple text. -<img src="./img/math_inline_sup_render_gfm.png" > +Simple inline code is easily highlighted with single backticks `` ` ``: -_Be advised that KaTeX only supports a [subset][katex-subset] of LaTeX._ +```markdown +Inline `code` has `back-ticks around` it. +``` ->**Note:** -This also works for the asciidoctor `:stem: latexmath`. For details see the [asciidoctor user manual][asciidoctor-manual]. +Inline `code` has `back-ticks around` it. -### Colors +--- -> If this is not rendered correctly, see -https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#colors +Similarly, a whole block of code can be fenced with triple backticks ```` ``` ````, +triple tildes (`~~~`), or indended 4 or more spaces to achieve a similar effect for +a larger body of code. test. -It is possible to have color written in HEX, RGB or HSL format rendered with a color indicator. +~~~ +``` +def function(): + #indenting works just fine in the fenced code block + s = "Python code" + print s +``` -Color written inside backticks will be followed by a color "chip". + Using 4 spaces + is like using + 3-backtick fences. +~~~ -Examples: +``` +~~~ +Tildes are OK too. +~~~ +``` - `#F00` - `#F00A` - `#FF0000` - `#FF0000AA` - `RGB(0,255,0)` - `RGB(0%,100%,0%)` - `RGBA(0,255,0,0.7)` - `HSL(540,70%,50%)` - `HSLA(540,70%,50%,0.7)` +The three examples above render as: -Becomes: +``` +def function(): + #indenting works just fine in the fenced code block + s = "Python code" + print s +``` - + Using 4 spaces + is like using + 3-backtick fences. -#### Supported formats: +~~~ +Tildes are OK too. +~~~ -* HEX: `` `#RGB[A]` `` or `` `#RRGGBB[AA]` `` -* RGB: `` `RGB[A](R, G, B[, A])` `` -* HSL: `` `HSL[A](H, S, L[, A])` `` +#### Colored code and syntax highlighting -### Mermaid +> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#colored-code-and-syntax-highlighting). -> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/15107) in -GitLab 10.3. -> -> If this is not rendered correctly, see -https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#mermaid +GitLab uses the [Rouge Ruby library](http://rouge.jneen.net/) for more colorful syntax +highlighting in code blocks. For a list of supported languages visit the +[Rouge project wiki](https://github.com/jneen/rouge/wiki/List-of-supported-languages-and-lexers). +Syntax highlighting is only supported in code blocks, it is not possible to highlight +code when it is inline. -It is possible to generate diagrams and flowcharts from text using [Mermaid](https://mermaidjs.github.io/). +Blocks of code are fenced by lines with three back-ticks ```` ``` ```` or three tildes `~~~`, and have +the language identified at the end of the first fence: -In order to generate a diagram or flowchart, you should write your text inside the `mermaid` block. +~~~ +```javascript +var s = "JavaScript syntax highlighting"; +alert(s); +``` -Example: +```python +def function(): + #indenting works just fine in the fenced code block + s = "Python syntax highlighting" + print s +``` - ```mermaid - graph TD; - A-->B; - A-->C; - B-->D; - C-->D; - ``` +```ruby +require 'redcarpet' +markdown = Redcarpet.new("Hello World!") +puts markdown.to_html +``` -Becomes: +``` +No language indicated, so no syntax highlighting. +s = "There is no highlighting for this." +But let's throw in a <b>tag</b>. +``` +~~~ -```mermaid -graph TD; - A-->B; - A-->C; - B-->D; - C-->D; +The four examples above render as: + +```javascript +var s = "JavaScript syntax highlighting"; +alert(s); ``` -For details see the [Mermaid official page](https://mermaidjs.github.io/). +```python +def function(): + #indenting works just fine in the fenced code block + s = "Python syntax highlighting" + print s +``` -### Front matter +```ruby +require 'redcarpet' +markdown = Redcarpet.new("Hello World!") +puts markdown.to_html +``` -> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/23331) - in GitLab 11.6. +``` +No language indicated, so no syntax highlighting. +s = "There is no highlighting for this." +But let's throw in a <b>tag</b>. +``` -Front matter is metadata included at the beginning of a markdown document, preceding -its content. This data can be used by static site generators such as [Jekyll](https://jekyllrb.com/docs/front-matter/) and [Hugo](https://gohugo.io/content-management/front-matter/), -and many other applications. +### Emphasis -In GitLab, front matter is only used in Markdown files and wiki pages, not the other places where Markdown formatting is supported. -When you view a Markdown file rendered by GitLab, any front matter is displayed as-is, in a box at the top of the document, before the rendered HTML content. -To view an example, you can toggle between the source and rendered version of a [GitLab documentation file](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/README.md). +There are multiple ways to emphasize text in markdown. You can italicize, bold, strikethrough, +as well as combine these emphasis styles together. -The following delimeters are supported: +Examples: -- YAML (`---`): +```markdown +Emphasis, aka italics, with *asterisks* or _underscores_. - ``` - --- - title: About Front Matter - example: - language: yaml - --- - ``` +Strong emphasis, aka bold, with double **asterisks** or __underscores__. -- TOML (`+++`): +Combined emphasis with **asterisks and _underscores_**. - ``` - +++ - title = "About Front Matter" - [example] - language = "toml" - +++ - ``` +Strikethrough uses two tildes. ~~Scratch this.~~ +``` -- JSON (`;;;`): +Emphasis, aka italics, with *asterisks* or _underscores_. - ``` - ;;; - { - "title": "About Front Matter" - "example": { - "language": "json" - } - } - ;;; - ``` +Strong emphasis, aka bold, with double **asterisks** or __underscores__. -Other languages are supported by adding a specifier to any of the existing -delimiters. For example: +Combined emphasis with **asterisks and _underscores_**. + +Strikethrough uses two tildes. ~~Scratch this.~~ +NOTE: **Note:** Strikethrough is not part of the core Markdown standard, but is part of GFM. + +#### Multiple underscores in words and mid-word emphasis + +> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#multiple-underscores-in-words). + +It is not usually useful to italicize just _part_ of a word, especially when you're +dealing with code and names that often appear with multiple underscores. As a result, +GFM extends the standard markdown standard by ignoring multiple underlines in words, +to allow better rendering of markdown documents discussing code: + +```md +perform_complicated_task + +do_this_and_do_that_and_another_thing + +but_emphasis is_desired _here_ ``` ----php -$title = "About Front Matter"; -$example = array( - 'language' => "php", -); + +perform_complicated_task + +do_this_and_do_that_and_another_thing + +but_emphasis is_desired _here_ + --- + +If you wish to emphasize only a part of a word, it can still be done with asterisks: + +```md +perform*complicated*task + +do*this*and*do*that*and*another thing ``` -## Standard Markdown +perform*complicated*task + +do*this*and*do*that*and*another thing -### Headers +### Footnotes + +Footnotes add a link to a note rendered at the end of a markdown file: + +```markdown +You can add footnotes to your text as follows.[^1] +[^1]: This is my awesome footnote (later in file). ``` + +You can add footnotes to your text as follows.[^1] + +[^1]: This is my awesome footnote (later in file). + +### Headers + +```markdown # H1 ## H2 ### H3 @@ -593,9 +758,11 @@ Alt-H2 #### Header IDs and links -All Markdown-rendered headers automatically get IDs, which can be linked to, except in comments. +GFM extends the standard markdown standard so that all Markdown-rendered headers automatically +get IDs, which can be linked to, except in comments. -On hover, a link to those IDs becomes visible to make it easier to copy the link to the header to use it somewhere else. +On hover, a link to those IDs becomes visible to make it easier to copy the link to +the header to use it somewhere else. The IDs are generated from the content of the header according to the following rules: @@ -606,7 +773,7 @@ The IDs are generated from the content of the header according to the following 1. If a header with the same ID has already been generated, a unique incrementing number is appended, starting at 1. -For example: +Example: ``` # This header has spaces in it @@ -626,215 +793,164 @@ Would generate the following link IDs: 1. `this-header-has-spaces-in-it-2` 1. `this-header-has-3-5-in-it-and-parentheses` -Note that the Emoji processing happens before the header IDs are generated, so the Emoji is converted to an image which then gets removed from the ID. - -### Emphasis - -Examples: - -``` -Emphasis, aka italics, with *asterisks* or _underscores_. - -Strong emphasis, aka bold, with **asterisks** or __underscores__. - -Combined emphasis with **asterisks and _underscores_**. - -Strikethrough uses two tildes. ~~Scratch this.~~ -``` - -Becomes: - -Emphasis, aka italics, with *asterisks* or _underscores_. - -Strong emphasis, aka bold, with **asterisks** or __underscores__. - -Combined emphasis with **asterisks and _underscores_**. - -Strikethrough uses two tildes. ~~Scratch this.~~ - -### Lists - -Examples: - -``` -1. First ordered list item -2. Another item - * Unordered sub-list. -1. Actual numbers don't matter, just that it's a number - 1. Ordered sub-list -4. And another item. - -* Unordered list can use asterisks -- Or minuses -+ Or pluses -``` - -Becomes: - -1. First ordered list item -2. Another item - * Unordered sub-list. -1. Actual numbers don't matter, just that it's a number - 1. Ordered sub-list -4. And another item. - -* Unordered list can use asterisks -- Or minuses -+ Or pluses - -If a list item contains multiple paragraphs, -each subsequent paragraph should be indented to the same level as the start of the list item text +Note that the Emoji processing happens before the header IDs are generated, so the +Emoji is converted to an image which is then removed from the ID. -Example: - -``` -1. First ordered list item - - Second paragraph of first item. - -2. Another item -``` +### Horizontal Rule -Becomes: +It's very simple to create a horizontal rule, by using three or more hyphens, asterisks, +or underscores: -1. First ordered list item +```markdown +Three or more hyphens, - Paragraph of first item. +--- -2. Another item +asterisks, -If the paragraph of the first item is not indented with the proper number of spaces, -the paragraph will appear outside the list, instead of properly indented under the list item. +*** -Example: +or underscores +___ ``` -1. First ordered list item - Paragraph of first item. +Three or more hyphens, -2. Another item -``` +--- -Becomes: +asterisks, -1. First ordered list item +*** - Paragraph of first item. +or underscores -2. Another item +___ -### Links +### Images -There are two ways to create links, inline-style and reference-style. +Examples: ```markdown -[I'm an inline-style link](https://www.google.com) -[I'm a link to a repository file in the same directory](index.md) -[I am an absolute reference within the repository](/doc/user/index.md) -[I'm a relative link to the Milestones page](../README.md) +Inline-style (hover to see title text): -[I link to a section on a different markdown page, using a header ID](index.md#overview) -[I link to a different section on the same page, using the header ID](#header-ids-and-links) + -[I'm a reference-style link][Arbitrary case-insensitive reference text] -[You can use numbers for reference-style link definitions][1] -Or leave it empty and use the [link text itself][] +Reference-style (hover to see title text): -Some text to show that the reference links can follow later. +![alt text1][logo] -[arbitrary case-insensitive reference text]: https://www.mozilla.org -[1]: http://slashdot.org -[link text itself]: https://www.reddit.com +[logo]: img/markdown_logo.png "Title Text" ``` ->**Note:** -Relative links do not allow referencing project files in a wiki page or wiki -page in a project file. The reason for this is that, in GitLab, wiki is always -a separate Git repository. For example, `[I'm a reference-style link](style)` -will point the link to `wikis/style` when the link is inside of a wiki markdown file. - -### Images +Inline-style (hover to see title text): -Examples: + - Here's our logo (hover to see the title text): +Reference-style (hover to see title text): - Inline-style: -  +![alt text][logo] - Reference-style: - ![alt text1][logo] +[logo]: img/markdown_logo.png "Title Text" - [logo]: img/markdown_logo.png +#### Videos -Becomes: +> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#videos). -Here's our logo: +Image tags that link to files with a video extension are automatically converted to +a video player. The valid video extensions are `.mp4`, `.m4v`, `.mov`, `.webm`, and `.ogv`: -Inline-style: +```md +Here's a sample video: - + +``` -Reference-style: +Here's a sample video: -![alt text][logo] + -[logo]: img/markdown_logo.png +### Inline HTML -### Blockquotes +> To see the markdown rendered within HTML in the second example, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#inline-html). -Examples: +You can also use raw HTML in your Markdown, and it'll usually work pretty well. -``` -> Blockquotes are very handy in email to emulate reply text. -> This line is part of the same quote. +See the documentation for HTML::Pipeline's [SanitizationFilter](http://www.rubydoc.info/gems/html-pipeline/1.11.0/HTML/Pipeline/SanitizationFilter#WHITELIST-constant) +class for the list of allowed HTML tags and attributes. In addition to the default +`SanitizationFilter` whitelist, GitLab allows `span`, `abbr`, `details` and `summary` elements. -Quote break. +```html +<dl> + <dt>Definition list</dt> + <dd>Is something people use sometimes.</dd> -> This is a very long line that will still be quoted properly when it wraps. Oh boy let's keep writing to make sure this is long enough to actually wrap for everyone. Oh, you can *put* **Markdown** into a blockquote. + <dt>Markdown in HTML</dt> + <dd>Does *not* work **very** well. HTML <em>tags</em> will <b>always</b> work.</dd> +</dl> ``` -Becomes: - -> Blockquotes are very handy in email to emulate reply text. -> This line is part of the same quote. - -Quote break. - -> This is a very long line that will still be quoted properly when it wraps. Oh boy let's keep writing to make sure this is long enough to actually wrap for everyone. Oh, you can *put* **Markdown** into a blockquote. - -### Inline HTML +<dl> + <dt>Definition list</dt> + <dd>Is something people use sometimes.</dd> -You can also use raw HTML in your Markdown, and it'll mostly work pretty well. + <dt>Markdown in HTML</dt> + <dd>Does *not* work **very** well. HTML <em>tags</em> will <b>always</b> work.</dd> +</dl> -See the documentation for HTML::Pipeline's [SanitizationFilter](http://www.rubydoc.info/gems/html-pipeline/1.11.0/HTML/Pipeline/SanitizationFilter#WHITELIST-constant) class for the list of allowed HTML tags and attributes. In addition to the default `SanitizationFilter` whitelist, GitLab allows `span`, `abbr`, `details` and `summary` elements. +--- -Examples: +It is still possible to use markdown inside HTML tags, but only if the lines containing markdown +are separated into their own lines: -``` +```html <dl> - <dt>Definition list</dt> - <dd>Is something people use sometimes.</dd> + <dt>Markdown in HTML</dt> + <dd>Does *not* work **very** well. HTML tags will always work.</dd> <dt>Markdown in HTML</dt> - <dd>Does *not* work **very** well. Use HTML <em>tags</em>.</dd> + <dd> + + Does *not* work **very** well. HTML tags will always work. + + </dd> </dl> ``` -Becomes: +<!-- Note: The example below uses HTML to force correct rendering on docs.gitlab.com, markdown will be fine in GitLab --> <dl> - <dt>Definition list</dt> - <dd>Is something people use sometimes.</dd> + <dt>Markdown in HTML</dt> + <dd>Does *not* work **very** well. HTML tags will always work.</dd> <dt>Markdown in HTML</dt> - <dd>Does *not* work **very** well. Use HTML <em>tags</em>.</dd> + <dd> + + Does <em>not</em> work <b>very</b> well. HTML tags will always work. + + </dd> </dl> #### Details and Summary -Content can be collapsed using HTML's [`<details>`](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/details) and [`<summary>`](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/summary) tags. This is especially useful for collapsing long logs so they take up less screen space. +> To see the markdown rendered within HTML in the second example, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#details-and-summary). + +Content can be collapsed using HTML's [`<details>`](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/details) +and [`<summary>`](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/summary) +tags. This is especially useful for collapsing long logs so they take up less screen space. + +```html +<p> +<details> +<summary>Click me to collapse/fold.</summary> + +These details <em>will</em> remain <strong>hidden</strong> until expanded. + +<pre><code>PASTE LOGS HERE</code></pre> + +</details> +</p> +``` <p> <details> @@ -847,7 +963,10 @@ These details <em>will</em> remain <strong>hidden</strong> until expanded. </details> </p> -**Note:** Markdown inside these tags is supported, as long as you have a blank line after the `</summary>` tag and before the `</details>` tag, as shown in the example. +--- + +Markdown inside these tags is supported as well, as long as you have a blank line +after the `</summary>` tag and before the `</details>` tag, as shown in the example: ```html <details> @@ -860,232 +979,302 @@ These details _will_ remain **hidden** until expanded. </details> ``` -### Horizontal Rule +<!-- Note: The example below uses HTML to force correct rendering on docs.gitlab.com, markdown will be fine in GitLab --> -Examples: +<details> +<summary>Click me to collapse/fold.</summary> -``` -Three or more... +These details <em>will</em> remain <b>hidden</b> until expanded. ---- + PASTE LOGS HERE + +</details> -Hyphens +### Line Breaks -*** +A line break will be inserted (a new paragraph will start) if the previous text is +ended with two newlines, i.e. you hit <kbd>Enter</kbd> twice in a row. If you only +use one newline (hit <kbd>Enter</kbd> once), the next sentence will be part of the +same paragraph. This is useful if you want to keep long lines from wrapping, and keep +them easily editable: -Asterisks +```markdown +Here's a line for us to start with. -___ +This longer line is separated from the one above by two newlines, so it will be a *separate paragraph*. -Underscores +This line is also a separate paragraph, but... +These lines are only separated by single newlines, +so they *do not break* and just follow the previous lines +in the *same paragraph*. ``` -Becomes: +Here's a line for us to start with. -Three or more... +This longer line is separated from the one above by two newlines, so it will be a *separate paragraph*. ---- +This line is also a separate paragraph, but... +These lines are only separated by single newlines, +so they *do not break* and just follow the previous lines +in the *same paragraph*. -Hyphens +#### Newlines -*** +GFM adheres to the markdown specification in how [paragraphs and line breaks are handled](https://spec.commonmark.org/current/). -Asterisks +A paragraph is simply one or more consecutive lines of text, separated by one or +more blank lines (i.e. two newlines at the end of the first paragraph), as [explained above](#line-breaks). -___ +If you need more control over line-breaks or soft returns, you can add a single line-break +by ending a line with a backslash, or two or more spaces. Two newlines in a row will create a new +paragraph, with a blank line in between: -Underscores +```markdown +First paragraph. +Another line in the same paragraph. +A third line in the same paragraph, but this time ending with two spaces.{space}{space} +A new line directly under the first paragraph. + +Second paragraph. +Another line, this time ending with a backslash.\ +A new line due to the previous backslash. +``` -### Line Breaks +<!-- (Do *NOT* remove the two ending whitespaces in the third line) --> +<!-- (They are needed for the Markdown text to render correctly) --> -A good way to learn how line breaks work is to experiment and discover -- hit <kbd>Enter</kbd> once (i.e., insert one newline), then hit it twice (i.e., insert two newlines), see what happens. You'll soon learn to get what you want. The "Preview" tab is your friend. +First paragraph. +Another line in the same paragraph. +A third line in the same paragraph, but this time ending with two spaces. +A new line directly under the first paragraph. -Here are some things to try out: +<!-- (Do *NOT* remove the two ending whitespaces in the second line) --> +<!-- (They are needed for the Markdown text to render correctly on docs.gitlab.com, the backslash works fine inside GitLab itself) --> -Examples: +Second paragraph. +Another line, this time ending with a backslash. +A new line due to the previous backslash. -``` -Here's a line for us to start with. +### Links -This line is separated from the one above by two newlines, so it will be a *separate paragraph*. +There are two ways to create links, inline-style and reference-style: -This line is also a separate paragraph, but... -This line is only separated by a single newline, so it *does not break* and just follows the previous line in the *same paragraph*. +```md +- This is an [inline-style link](https://www.google.com) +- This is a [link to a repository file in the same directory](index.md) +- This is a [relative link to a readme one directory higher](../README.md) +- This is a [link that also has title text](https://www.google.com "This link takes you to Google!") -This line is also a separate paragraph, and... -This line is *on its own line*, because the previous line ends with two spaces. (but still in the *same paragraph*) +Using header ID anchors: -spaces. -``` +- This links to [a section on a different markdown page, using a "#" and the header ID](index.md#overview) +- This links to [a different section on the same page, using a "#" and the header ID](#header-ids-and-links) -Becomes: +Using references: -Here's a line for us to start with. +- This is a [reference-style link, see below][Arbitrary case-insensitive reference text] +- You can [use numbers for reference-style link definitions, see below][1] +- Or leave it empty and use the [link text itself][], see below. -This line is separated from the one above by two newlines, so it will be a *separate paragraph*. +Some text to show that the reference links can follow later. -This line is also a separate paragraph, but... -This line is only separated by a single newline, so it *does not break* and just follows the previous line in the *same paragraph*. +[arbitrary case-insensitive reference text]: https://www.mozilla.org +[1]: http://slashdot.org +[link text itself]: https://www.reddit.com +``` -This line is also a separate paragraph, and... -This line is *on its own line*, because the previous line ends with two spaces. (but still in the *same paragraph*) +- This is an [inline-style link](https://www.google.com) +- This is a [link to a repository file in the same directory](index.md) +- This is a [relative link to a readme one directory higher](../README.md) +- This is a [link that also has title text](https://www.google.com "This link takes you to Google!") -spaces. +Using header ID anchors: -### Tables +- This links to [a section on a different markdown page, using a "#" and the header ID](index.md#overview) +- This links to [a different section on the same page, using a "#" and the header ID](#header-ids-and-links) -Tables aren't part of the core Markdown spec, but they are part of GFM. +Using references: -Example: +- This is a [reference-style link, see below][Arbitrary case-insensitive reference text] +- You can [use numbers for reference-style link definitions, see below][1] +- Or leave it empty and use the [link text itself][], see below. -``` -| header 1 | header 2 | -| -------- | -------- | -| cell 1 | cell 2 | -| cell 3 | cell 4 | -``` - -Becomes: +Some text to show that the reference links can follow later. -| header 1 | header 2 | -| -------- | -------- | -| cell 1 | cell 2 | -| cell 3 | cell 4 | +[arbitrary case-insensitive reference text]: https://www.mozilla.org +[1]: http://slashdot.org +[link text itself]: https://www.reddit.com -**Note:** The row of dashes between the table header and body must have at least three dashes in each column. +NOTE: **Note:** Relative links do not allow the referencing of project files in a wiki +page, or a wiki page in a project file. The reason for this is that a wiki is always +in a separate Git repository in GitLab. For example, `[I'm a reference-style link](style)` +will point the link to `wikis/style` only when the link is inside of a wiki markdown file. -By including colons in the header row, you can align the text within that column. +#### URL auto-linking -Example: +GFM will autolink almost any URL you put into your text: -``` -| Left Aligned | Centered | Right Aligned | Left Aligned | Centered | Right Aligned | -| :----------- | :------: | ------------: | :----------- | :------: | ------------: | -| Cell 1 | Cell 2 | Cell 3 | Cell 4 | Cell 5 | Cell 6 | -| Cell 7 | Cell 8 | Cell 9 | Cell 10 | Cell 11 | Cell 12 | +```markdown +* https://www.google.com +* https://google.com/ +* ftp://ftp.us.debian.org/debian/ +* smb://foo/bar/baz +* irc://irc.freenode.net/gitlab +* http://localhost:3000 ``` -Becomes: +* https://www.google.com +* https://google.com/ +* ftp://ftp.us.debian.org/debian/ +* smb://foo/bar/baz +* irc://irc.freenode.net/gitlab +* http://localhost:3000 -| Left Aligned | Centered | Right Aligned | Left Aligned | Centered | Right Aligned | -| :----------- | :------: | ------------: | :----------- | :------: | ------------: | -| Cell 1 | Cell 2 | Cell 3 | Cell 4 | Cell 5 | Cell 6 | -| Cell 7 | Cell 8 | Cell 9 | Cell 10 | Cell 11 | Cell 12 | +### Lists -### Footnotes +Ordered and unordered lists can be easily created. Add the number you want the list +to start with, like `1. ` (with a space) at the start of each line for ordered lists. +After the first number, it does not matter what number you use, ordered lists will be +numbered automatically by vertical order, so repeating `1. ` for all items in the +same list is common. If you start with a number other than `1. `, it will use that as the first +number, and count up from there. -Example: +Add a `* `, `- ` or `+ ` (with a space) at the start of each line for unordered lists, but +you should not use a mix of them. -``` -You can add footnotes to your text as follows.[^2] -[^2]: This is my awesome footnote. +Examples: + +```md +1. First ordered list item +2. Another item + * Unordered sub-list. +1. Actual numbers don't matter, just that it's a number + 1. Ordered sub-list + 1. Next ordered sub-list item +4. And another item. + +* Unordered lists can use asterisks +- Or minuses ++ Or pluses ``` -Becomes: +1. First ordered list item +2. Another item + * Unordered sub-list. +1. Actual numbers don't matter, just that it's a number + 1. Ordered sub-list + 1. Next ordered sub-list item +4. And another item. -You can add footnotes to your text as follows.[^2] +* Unordered lists can use asterisks +- Or minuses ++ Or pluses -### Superscripts / Subscripts +--- -CommonMark and GFM currently do not support the superscript syntax ( `x^2` ) that Redcarpet does. You can use the standard HTML syntax for superscripts and subscripts. +If a list item contains multiple paragraphs, each subsequent paragraph should be indented +to the same level as the start of the list item text. -``` -The formula for water is H<sub>2</sub>O -while the equation for the theory of relativity is E = mc<sup>2</sup>. -``` +Example: -The formula for water is H<sub>2</sub>O while the equation for the theory of relativity is E = mc<sup>2</sup>. +```markdown +1. First ordered list item -## Wiki-specific Markdown + Second paragraph of first item. -The following examples show how links inside wikis behave. +2. Another item +``` -### Wiki - Direct page link +1. First ordered list item -A link which just includes the slug for a page will point to that page, -_at the base level of the wiki_. + Second paragraph of first item. -This snippet would link to a `documentation` page at the root of your wiki: +2. Another item -```markdown -[Link to Documentation](documentation) -``` +--- -### Wiki - Direct file link +If the paragraph of the first item is not indented with the proper number of spaces, +the paragraph will appear outside the list, instead of properly indented under the list item. -Links with a file extension point to that file, _relative to the current page_. +Example: -If this snippet was placed on a page at `<your_wiki>/documentation/related`, -it would link to `<your_wiki>/documentation/file.md`: +``` +1. First ordered list item -```markdown -[Link to File](file.md) + Paragraph of first item. + +2. Another item ``` -### Wiki - Hierarchical link +1. First ordered list item -A link can be constructed relative to the current wiki page using `./<page>`, -`../<page>`, etc. + Paragraph of first item. -- If this snippet was placed on a page at `<your_wiki>/documentation/main`, - it would link to `<your_wiki>/documentation/related`: +2. Another item - ```markdown - [Link to Related Page](related) - ``` +### Superscripts / Subscripts -- If this snippet was placed on a page at `<your_wiki>/documentation/related/content`, - it would link to `<your_wiki>/documentation/main`: +CommonMark and GFM currently do not support the superscript syntax ( `x^2` ) that +Redcarpet does. You can use the standard HTML syntax for superscripts and subscripts: - ```markdown - [Link to Related Page](../main) - ``` +```html +The formula for water is H<sub>2</sub>O +while the equation for the theory of relativity is E = mc<sup>2</sup>. +``` -- If this snippet was placed on a page at `<your_wiki>/documentation/main`, - it would link to `<your_wiki>/documentation/related.md`: +The formula for water is H<sub>2</sub>O +while the equation for the theory of relativity is E = mc<sup>2</sup>. - ```markdown - [Link to Related Page](related.md) - ``` +### Tables -- If this snippet was placed on a page at `<your_wiki>/documentation/related/content`, - it would link to `<your_wiki>/documentation/main.md`: +Tables aren't part of the core Markdown spec, but they are part of GFM. - ```markdown - [Link to Related Page](../main.md) - ``` +1. The first line contains the headers, separated by "pipes" (`|`). +1. The second line separates the headers from the cells, and must contain three or more dashes. +1. The third, and any following lines, contain the cell values. + - You **can't** have cells separated over many lines in the markdown, they must be kept to single lines, + but they can be very long. You can also include HTML `<br>` tags to force newlines if needed. + - The cell sizes **don't** have to match each other. They are flexible, but must be separated + by pipes (`|`). + - You **can** have blank cells. -### Wiki - Root link +Example: -A link starting with a `/` is relative to the wiki root. +```markdown +| header 1 | header 2 | header 3 | +| --- | ------ |----------| +| cell 1 | cell 2 | cell 3 | +| cell 4 | cell 5 is longer | cell 6 is much longer than the others, but that's ok. It will eventually wrap the text when the cell is too large for the display size. | +| cell 7 | | cell <br> 9 | +``` -- This snippet links to `<wiki_root>/documentation`: +| header 1 | header 2 | header 3 | +| --- | ------ |---------:| +| cell 1 | cell 2 | cell 3 | +| cell 4 | cell 5 is longer | cell 6 is much longer than the others, but that's ok. It will eventually wrap the text when the cell is too large for the display size. | +| cell 7 | | cell <br> 9 | - ```markdown - [Link to Related Page](/documentation) - ``` +Additionally, you can choose the alignment of text within columns by adding colons (`:`) +to the sides of the "dash" lines in the second row. This will affect every cell in the column. -- This snippet links to `<wiki_root>/miscellaneous.md`: +> Note that the headers are always right aligned [within GitLab itself itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#tables). - ```markdown - [Link to Related Page](/miscellaneous.md) - ``` +```markdown +| Left Aligned | Centered | Right Aligned | Left Aligned | Centered | Right Aligned | +| :--- | :---: | ---: | :----------- | :------: | ------------: | +| Cell 1 | Cell 2 | Cell 3 | Cell 4 | Cell 5 | Cell 6 | +| Cell 7 | Cell 8 | Cell 9 | Cell 10 | Cell 11 | Cell 12 | +``` + +| Left Aligned | Centered | Right Aligned | Left Aligned | Centered | Right Aligned | +| :--- | :---: | ---: | :----------- | :------: | ------------: | +| Cell 1 | Cell 2 | Cell 3 | Cell 4 | Cell 5 | Cell 6 | +| Cell 7 | Cell 8 | Cell 9 | Cell 10 | Cell 11 | Cell 12 | ## References - This document leveraged heavily from the [Markdown-Cheatsheet](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet). -- The original [Markdown Syntax Guide](https://daringfireball.net/projects/markdown/syntax) at Daring Fireball is an excellent resource for a detailed explanation of standard markdown. -- The detailed specification for CommonMark can be found in the [CommonMark Spec][commonmark-spec] +- The original [Markdown Syntax Guide](https://daringfireball.net/projects/markdown/syntax) + at Daring Fireball is an excellent resource for a detailed explanation of standard markdown. +- The detailed specification for CommonMark can be found in the [CommonMark Spec](https://spec.commonmark.org/current/) - The [CommonMark Dingus](http://try.commonmark.org) is a handy tool for testing CommonMark syntax. - -[^1]: This link will be broken if you see this document from the Help page or docs.gitlab.com -[^2]: This is my awesome footnote. - -[markdown.md]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md -[rouge]: http://rouge.jneen.net/ "Rouge website" -[redcarpet]: https://github.com/vmg/redcarpet "Redcarpet website" -[katex]: https://github.com/Khan/KaTeX "KaTeX website" -[katex-subset]: https://katex.org/docs/supported.html "Macros supported by KaTeX" -[asciidoctor-manual]: http://asciidoctor.org/docs/user-manual/#activating-stem-support "Asciidoctor user manual" -[commonmarker]: https://github.com/gjtorikian/commonmarker -[commonmark-spec]: https://spec.commonmark.org/current/ diff --git a/doc/user/permissions.md b/doc/user/permissions.md index 7af3d4a0ac3..03abef9fc62 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -365,3 +365,8 @@ for details about the pipelines security model. Since GitLab 8.15, LDAP user permissions can now be manually overridden by an admin user. Read through the documentation on [LDAP users permissions](../administration/auth/how_to_configure_ldap_gitlab_ee/index.html) to learn more. + +## Project aliases + +Project aliases can only be read, created and deleted by a GitLab administrator. +Read through the documentation on [Project aliases](../user/project/index.md#project-aliases-premium-only) to learn more. diff --git a/doc/user/profile/account/two_factor_authentication.md b/doc/user/profile/account/two_factor_authentication.md index df413a11af0..26cacbe5545 100644 --- a/doc/user/profile/account/two_factor_authentication.md +++ b/doc/user/profile/account/two_factor_authentication.md @@ -84,9 +84,8 @@ Click on **Register U2F Device** to complete the process. > **Note:** Recovery codes are not generated for U2F devices. -Should you ever lose access to your one time password authenticator, you can use one of the ten provided -backup codes to login to your account. We suggest copying them, printing them, or downloading them using -the **Download codes** button for storage in a safe place. +Immediately after successfully enabling two-factor authentication, you'll be prompted to download a set of set recovery codes. Should you ever lose access to your one time password authenticator, you can use one of them to log in to your account. We suggest copying them, printing them, or downloading them using +the **Download codes** button for storage in a safe place. If you choose to download them, the file will be called **gitlab-recovery-codes.txt**. CAUTION: **Caution:** Each code can be used only once to log in to your account. diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index 97d2dfc0f7e..c6ee168bad0 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -533,22 +533,20 @@ This job failed because the necessary resources were not successfully created. To find the cause of this error when creating a namespace and service account, check the [logs](../../../administration/logs.md#kuberneteslog). -NOTE: **NOTE:** -As of GitLab 12.1 we require [`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) -tokens for all project level clusters unless you unselect the -[GitLab-managed cluster](#gitlab-managed-clusters) option. If you -want to manage namespaces and service accounts yourself and don't -want to provide a `cluster-admin` token to GitLab you must unselect this -option or you will get the above error. - -Common reasons for failure include: +Reasons for failure include: -- The token you gave GitLab did not have [`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) +- The token you gave GitLab does not have [`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) privileges required by GitLab. - Missing `KUBECONFIG` or `KUBE_TOKEN` variables. To be passed to your job, they must have a matching [`environment:name`](../../../ci/environments.md#defining-environments). If your job has no `environment:name` set, it will not be passed the Kubernetes credentials. +NOTE: **NOTE:** +Project-level clusters upgraded from GitLab 12.0 or older may be configured +in a way that causes this error. Ensure you deselect the +[GitLab-managed cluster](#gitlab-managed-clusters) option if you want to manage +namespaces and service accounts yourself. + ## Monitoring your Kubernetes cluster **[ULTIMATE]** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/4701) in [GitLab Ultimate][ee] 10.6. diff --git a/doc/user/project/clusters/serverless/img/function-endpoint.png b/doc/user/project/clusters/serverless/img/function-endpoint.png Binary files differnew file mode 100644 index 00000000000..f3c7ae7a00d --- /dev/null +++ b/doc/user/project/clusters/serverless/img/function-endpoint.png diff --git a/doc/user/project/clusters/serverless/index.md b/doc/user/project/clusters/serverless/index.md index 3be5bfeaddc..91f0e24b44e 100644 --- a/doc/user/project/clusters/serverless/index.md +++ b/doc/user/project/clusters/serverless/index.md @@ -325,3 +325,275 @@ loading or is not available at this time._ It will appear upon the first access page, but should go away after a few seconds. If the message does not disappear, then it is possible that GitLab is unable to connect to the Prometheus instance running on the cluster. + +## Enabling TLS for Knative services + +By default, a GitLab serverless deployment will be served over `http`. In order to serve over `https` you +must manually obtain and install TLS certificates. + +The simplest way to accomplish this is to +use [Certbot to manually obtain Let's Encrypt certificates](https://knative.dev/docs/serving/using-a-tls-cert/#using-certbot-to-manually-obtain-let-s-encrypt-certificates). Certbot is a free, open source software tool for automatically using Let’s Encrypt certificates on manually-administrated websites to enable HTTPS. + +NOTE: **Note:** +The instructions below relate to installing and running Certbot on a Linux server and may not work on other operating systems. + +1. Install Certbot by running the + [`certbot-auto` wrapper script](https://certbot.eff.org/docs/install.html#certbot-auto). + On the command line of your server, run the following commands: + + ```sh + wget https://dl.eff.org/certbot-auto + sudo mv certbot-auto /usr/local/bin/certbot-auto + sudo chown root /usr/local/bin/certbot-auto + chmod 0755 /usr/local/bin/certbot-auto + /usr/local/bin/certbot-auto --help + ``` + + To check the integrity of the `certbot-auto` script, run: + + ```sh + wget -N https://dl.eff.org/certbot-auto.asc + gpg2 --keyserver ipv4.pool.sks-keyservers.net --recv-key A2CFB51FA275A7286234E7B24D17C995CD9775F2 + gpg2 --trusted-key 4D17C995CD9775F2 --verify certbot-auto.asc /usr/local/bin/certbot-auto + ``` + + The output of the last command should look something like: + + ```sh + gpg: Signature made Mon 10 Jun 2019 06:24:40 PM EDT + gpg: using RSA key A2CFB51FA275A7286234E7B24D17C995CD9775F2 + gpg: key 4D17C995CD9775F2 marked as ultimately trusted + gpg: checking the trustdb + gpg: marginals needed: 3 completes needed: 1 trust model: pgp + gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u + gpg: next trustdb check due at 2027-11-22 + gpg: Good signature from "Let's Encrypt Client Team <letsencrypt-client@eff.org>" [ultimate] + ``` + +1. Run the following command to use Certbot to request a certificate + using DNS challenge during authorization: + + + ```sh + ./certbot-auto certonly --manual --preferred-challenges dns -d '*.<namespace>.example.com' + ``` + + Where `<namespace>` is the namespace created by GitLab for your serverless project (composed of `<projectname+id>`) and + `example.com` is the domain being used for your project. If you are unsure what the namespace of your project is, navigate + to the **Operations > Serverless** page of your project and inspect + the endpoint provided for your function/app. + +  + + In the above image, the namespace for the project is `node-function-11909507` and the domain is `knative.info`, thus + certificate request line would look like this: + + ```sh + ./certbot-auto certonly --manual --preferred-challenges dns -d '*.node-function-11909507.knative.info' + ``` + + The Certbot tool walks you through the steps of validating that you own each domain that you specify by creating TXT records in those domains. + After this process is complete, the output should look something like this: + + ```sh + IMPORTANT NOTES: + - Congratulations! Your certificate and chain have been saved at: + /etc/letsencrypt/live/namespace.example.com/fullchain.pem + Your key file has been saved at: + /etc/letsencrypt/live/namespace.example/privkey.pem + Your cert will expire on 2019-09-19. To obtain a new or tweaked + version of this certificate in the future, simply run certbot-auto + again. To non-interactively renew *all* of your certificates, run + "certbot-auto renew" + -----BEGIN PRIVATE KEY----- + - Your account credentials have been saved in your Certbot + configuration directory at /etc/letsencrypt. You should make a + secure backup of this folder now. This configuration directory will + also contain certificates and private keys obtained by Certbot so + making regular backups of this folder is ideal. + ``` + +1. Create certificate and private key files. Using the contents of the files + returned by Certbot, we'll create two files in order to create the + Kubernetes secret: + + Run the following command to see the contents of `fullchain.pem`: + + ```sh + sudo cat /etc/letsencrypt/live/node-function-11909507.knative.info/fullchain.pem + ``` + + Output should look like this: + + ```sh + -----BEGIN CERTIFICATE----- + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b4ag== + -----END CERTIFICATE----- + -----BEGIN CERTIFICATE----- + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + K2fcb195768c39e9a94cec2c2e30Qg== + -----END CERTIFICATE----- + ``` + + Create a file with the name `cert.pem` with the contents of the entire output. + + Once `cert.pem` is created, run the following command to see the contents of `privkey.pem`: + + ```sh + sudo cat /etc/letsencrypt/live/namespace.example/privkey.pem + ``` + + Output should look like this: + + ```sh + -----BEGIN PRIVATE KEY----- + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + 2fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 04f294d1eaca42b8692017b426d53bbc8fe75f827734f0260710b83a556082df + -----BEGIN CERTIFICATE----- + fcb195768c39e9a94cec2c2e32c59c0aad7a3365c10892e8116b5d83d4096b6 + 4f294d1eaca42b8692017b4262== + -----END PRIVATE KEY----- + ``` + + Create a new file with the name `cert.pk` with the contents of the entire output. + +1. Create a Kubernetes secret to hold your TLS certificate, `cert.pem`, and + the private key `cert.pk`: + + NOTE: **Note:** + Running `kubectl` commands on your cluster requires setting up access to the cluster first. + For clusters created on GKE, see + [GKE Cluster Access](https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-access-for-kubectl). + For other platforms, [install `kubectl`](https://kubernetes.io/docs/tasks/tools/install-kubectl/). + + ```sh + kubectl create --namespace istio-system secret tls istio-ingressgateway-certs \ + --key cert.pk \ + --cert cert.pem + ``` + + Where `cert.pem` and `cert.pk` are your certificate and private key files. Note that the `istio-ingressgateway-certs` secret name is required. + +1. Configure Knative to use the new secret that you created for HTTPS + connections. Run the + following command to open the Knative shared `gateway` in edit mode: + + ```sh + kubectl edit gateway knative-ingress-gateway --namespace knative-serving + ``` + + Update the gateway to include the following tls: section and configuration: + + ```sh + tls: + mode: SIMPLE + privateKey: /etc/istio/ingressgateway-certs/tls.key + serverCertificate: /etc/istio/ingressgateway-certs/tls.crt + ``` + + Example: + + ```sh + apiVersion: networking.istio.io/v1alpha3 + kind: Gateway + metadata: + # ... skipped ... + spec: + selector: + istio: ingressgateway + servers: + - hosts: + - "*" + port: + name: http + number: 80 + protocol: HTTP + - hosts: + - "*" + port: + name: https + number: 443 + protocol: HTTPS + tls: + mode: SIMPLE + privateKey: /etc/istio/ingressgateway-certs/tls.key + serverCertificate: /etc/istio/ingressgateway-certs/tls.crt + ``` + + After your changes are running on your Knative cluster, you can begin using the HTTPS protocol for secure access your deployed Knative services. + In the event a mistake is made during this process and you need to update the cert, you will need to edit the gateway `knative-ingress-gateway` + to switch back to `PASSTHROUGH` mode. Once corrections are made, edit the file again so the gateway will use the new certificates.
\ No newline at end of file diff --git a/doc/user/project/index.md b/doc/user/project/index.md index 587b4121e4e..06286951e20 100644 --- a/doc/user/project/index.md +++ b/doc/user/project/index.md @@ -193,6 +193,28 @@ password <personal_access_token> To quickly access a project from the GitLab UI using the project ID, visit the `/projects/:id` URL in your browser or other tool accessing the project. +## Project aliases **[PREMIUM ONLY]** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/3264) in [GitLab Premium](https://about.gitlab.com/pricing/) 12.1. + +When migrating repositories to GitLab and they are being accessed by other systems, +it's very useful to be able to access them using the same name especially when +they are a lot. It reduces the risk of changing significant number of Git URLs in +a large number of systems. + +GitLab provides a functionality to help with this. In GitLab, repositories are +usually accessed with a namespace and project name. It is also possible to access +them via a project alias. This feature is only available on Git over SSH. + +A project alias can be only created via API and only by GitLab administrators. +Follow the [Project Aliases API documentation](../../api/project_aliases.md) for +more details. + +Once an alias has been created for a project (e.g., an alias `gitlab-ce` for the +project `https://gitlab.com/gitlab-org/gitlab-ce`), the repository can be cloned +using the alias (e.g `git clone git@gitlab.com:gitlab-ce.git` instead of +`git clone git@gitlab.com:gitlab-org/gitlab-ce.git`). + ## Project APIs There are numerous [APIs](../../api/README.md) to use with your projects: @@ -212,3 +234,4 @@ There are numerous [APIs](../../api/README.md) to use with your projects: - [Templates](../../api/project_templates.md) - [Traffic](../../api/project_statistics.md) - [Variables](../../api/project_level_variables.md) +- [Aliases](../../api/project_aliases.md) diff --git a/doc/user/project/integrations/jira.md b/doc/user/project/integrations/jira.md index 234e3ad31cc..8f2e5a55b5f 100644 --- a/doc/user/project/integrations/jira.md +++ b/doc/user/project/integrations/jira.md @@ -39,7 +39,7 @@ a GitLab project with any single Jira project. If you have one Jira instance, you can pre-fill the settings page with a default template. See the [Services Templates][services-templates] docs. -Configuration happens via user name and password. Connecting to a Jira server +Configuration happens via user name and password. Connecting to a Jira Server via CAS is not possible. In order to enable the Jira service in GitLab, you need to first configure the @@ -47,13 +47,13 @@ project in Jira and then enter the correct values in GitLab. ### Configuring Jira -When connecting to **JIRA Server**, which supports basic authentication, a **username and password** are required. Check the link below and proceed to the next step: +When connecting to **Jira Server**, which supports basic authentication, a **username and password** are required. Check the link below and proceed to the next step: -- [Setting up a user in JIRA server](jira_server_configuration.md) +- [Setting up a user in Jira Server](jira_server_configuration.md) -When connecting to **JIRA Cloud**, which supports authentication via API token, an **email and API token**, are required. Check the link below and proceed to the next step: +When connecting to **Jira Cloud**, which supports authentication via API token, an **email and API token**, are required. Check the link below and proceed to the next step: -- [Setting up a user in JIRA cloud](jira_cloud_configuration.md) +- [Setting up a user in Jira Cloud](jira_cloud_configuration.md) ### Configuring GitLab @@ -77,8 +77,8 @@ in the table below. | ----- | ----------- | | `Web URL` | The base URL to the Jira instance web interface which is being linked to this GitLab project. E.g., `https://Jira.example.com`. | | `Jira API URL` | The base URL to the Jira instance API. Web URL value will be used if not set. E.g., `https://jira-api.example.com`. | -| `Username/Email` | Created when [configuring Jira step](#configuring-jira). Use `username` for **JIRA server** or `email` for **JIRA cloud**. | -| `Password/API token` |Created in [configuring Jira step](#configuring-jira). Use `password` for **JIRA server** or `API token` for **JIRA cloud**. | +| `Username/Email` | Created when [configuring Jira step](#configuring-jira). Use `username` for **Jira Server** or `email` for **Jira Cloud**. | +| `Password/API token` |Created in [configuring Jira step](#configuring-jira). Use `password` for **Jira Server** or `API token` for **Jira Cloud**. | | `Transition ID` | This is the ID of a transition that moves issues to the desired state. It is possible to insert transition ids separated by `,` or `;` which means the issue will be moved to each state after another using the given order. **Closing Jira issues via commits or Merge Requests won't work if you don't set the ID correctly.** | ### Obtaining a transition ID diff --git a/doc/user/project/integrations/jira_cloud_configuration.md b/doc/user/project/integrations/jira_cloud_configuration.md index 849df707521..614f05d5b7e 100644 --- a/doc/user/project/integrations/jira_cloud_configuration.md +++ b/doc/user/project/integrations/jira_cloud_configuration.md @@ -1,18 +1,18 @@ -# Creating an API token in JIRA cloud +# Creating an API token in Jira Cloud -An API token is needed when integrating with JIRA Cloud, follow the steps +An API token is needed when integrating with Jira Cloud, follow the steps below to create one: 1. Log in to <https://id.atlassian.com> with your email. 1. **Click API tokens**, then **Create API token**. - + - + 1. Make sure to write down your new API token as you will need it in the next [steps](jira.md#configuring-gitlab). NOTE: **Note** -It is important that the user associated with this email has 'write' access to projects in JIRA. +It is important that the user associated with this email has 'write' access to projects in Jira. -The JIRA configuration is complete. You are going to need this new created token and the email you used to log in when [configuring GitLab in the next section](jira.md#configuring-gitlab). +The Jira configuration is complete. You are going to need this newly created token and the email you used to log in, when [configuring GitLab in the next section](jira.md#configuring-gitlab). diff --git a/doc/user/project/integrations/jira_server_configuration.md b/doc/user/project/integrations/jira_server_configuration.md index 13d65c4d8e4..32991714973 100644 --- a/doc/user/project/integrations/jira_server_configuration.md +++ b/doc/user/project/integrations/jira_server_configuration.md @@ -1,4 +1,4 @@ -# Creating a username and password for JIRA server +# Creating a username and password for Jira Server We need to create a user in Jira which will have access to all projects that need to integrate with GitLab. diff --git a/doc/user/project/integrations/project_services.md b/doc/user/project/integrations/project_services.md index 0bfee3bac99..0e4c71a9d3e 100644 --- a/doc/user/project/integrations/project_services.md +++ b/doc/user/project/integrations/project_services.md @@ -38,7 +38,7 @@ Click on the service links to see further configuration instructions and details | [Hangouts Chat](hangouts_chat.md) | Receive events notifications in Google Hangouts Chat | | [HipChat](hipchat.md) | Private group chat and IM | | [Irker (IRC gateway)](irker.md) | Send IRC messages, on update, to a list of recipients through an Irker gateway | -| [JIRA](jira.md) | JIRA issue tracker | +| [Jira](jira.md) | Jira issue tracker | | [Jenkins](../../../integration/jenkins.md) **[STARTER]** | An extendable open source continuous integration server | | JetBrains TeamCity CI | A continuous integration and build server | | [Mattermost slash commands](mattermost_slash_commands.md) | Mattermost chat and ChatOps slash commands | diff --git a/doc/user/project/pages/getting_started_part_three.md b/doc/user/project/pages/getting_started_part_three.md index d585c19fc5c..bc9a11504cd 100644 --- a/doc/user/project/pages/getting_started_part_three.md +++ b/doc/user/project/pages/getting_started_part_three.md @@ -1,5 +1,5 @@ --- -last_updated: 2019-06-04 +last_updated: 2019-06-25 type: concepts, reference, howto --- @@ -138,9 +138,9 @@ verify your domain's ownership with a TXT record: > - **Do not** add any special chars after the default Pages domain. E.g., **do not** point your `subdomain.domain.com` to `namespace.gitlab.io.` or `namespace.gitlab.io/`. -> - GitLab Pages IP on GitLab.com [was changed](https://about.gitlab.com/2017/03/06/we-are-changing-the-ip-of-gitlab-pages-on-gitlab-com/) in 2017 +> - GitLab Pages IP on GitLab.com [was changed](https://about.gitlab.com/2017/03/06/we-are-changing-the-ip-of-gitlab-pages-on-gitlab-com/) in 2017. > - GitLab Pages IP on GitLab.com [has been changed](https://about.gitlab.com/2018/07/19/gcp-move-update/#gitlab-pages-and-custom-domains) - from `52.167.214.135` to `35.185.44.232` in 2018 + from `52.167.214.135` to `35.185.44.232` in 2018. ### Add your custom domain to GitLab Pages settings @@ -199,7 +199,7 @@ Certificates are NOT required to add to your custom highly recommendable. Let's start with an introduction to the importance of HTTPS. -Alternatively, jump ahead to [adding certificates to your project](#adding-certificates-to-your-project). +Alternatively, jump ahead to [adding certificates to your project](#adding-certificates-to-pages). ### Why should I care about HTTPS? @@ -255,12 +255,12 @@ which also offers a [free CDN service](https://blog.cloudflare.com/cloudflares-f Their certs are valid up to 15 years. See the tutorial on [how to add a CloudFlare Certificate to your GitLab Pages website](https://about.gitlab.com/2017/02/07/setting-up-gitlab-pages-with-cloudflare-certificates/). -### Adding certificates to your project +### Adding certificates to Pages Regardless the CA you choose, the steps to add your certificate to your Pages project are the same. -### What do you need +#### Requirements 1. A PEM certificate 1. An intermediate certificate @@ -270,7 +270,7 @@ your Pages project are the same. These fields are found under your **Project**'s **Settings** > **Pages** > **New Domain**. -### What's what? +#### Certificate types - A PEM certificate is the certificate generated by the CA, which needs to be added to the field **Certificate (PEM)**. @@ -283,21 +283,32 @@ These fields are found under your **Project**'s **Settings** > **Pages** > **New - A private key is an encrypted key which validates your PEM against your domain. -### Now what? +#### Add the certificate to your project -Now that you hopefully understand why you need all -of this, it's simple: +Once you've met the requirements: -- Your PEM certificate needs to be added to the first field +- Your PEM certificate needs to be added to the first field. - If your certificate is missing its intermediate, copy and paste the root certificate (usually available from your CA website) and paste it in the [same field as your PEM certificate](https://about.gitlab.com/2017/02/07/setting-up-gitlab-pages-with-cloudflare-certificates/), just jumping a line between them. -- Copy your private key and paste it in the last field +- Copy your private key and paste it in the last field. ->**Note:** +NOTE: **Note:** **Do not** open certificates or encryption keys in regular text editors. Always use code editors (such as Sublime Text, Atom, Dreamweaver, Brackets, etc). -_Read on about [Creating and Tweaking GitLab CI/CD for GitLab Pages](getting_started_part_four.md)_ +## Force HTTPS for GitLab Pages websites + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/28857) in GitLab 10.7. + +To make your website's visitors even more secure, you can choose to +force HTTPS for GitLab Pages. By doing so, all attempts to visit your +website via HTTP will be automatically redirected to HTTPS via 301. + +It works with both GitLab's default domain and with your custom +domain (as long as you've set a valid certificate for it). + +To enable this setting, navigate to your project's **Settings > Pages** +and tick the checkbox **Force HTTPS (requires valid certificates)**. diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md index 6c430ff7cd9..1281ba561b8 100644 --- a/doc/user/project/quick_actions.md +++ b/doc/user/project/quick_actions.md @@ -57,6 +57,7 @@ discussions, and descriptions: | `/approve` | Approve the merge request | | ✓ | | `/merge` | Merge (when pipeline succeeds) | | ✓ | | `/create_merge_request <branch name>` | Create a new merge request starting from the current issue | ✓ | | +| `/relate #issue1 #issue2` | Mark issues as related **[STARTER]** | ✓ | | ## Quick actions for commit messages diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md index be4215b1934..2bf8d4dfe7b 100644 --- a/doc/user/project/settings/index.md +++ b/doc/user/project/settings/index.md @@ -18,7 +18,7 @@ Adjust your project's name, description, avatar, [default branch](../repository/  -The project description also partially supports [standard markdown](../../markdown.md#standard-markdown). You can use [emphasis](../../markdown.md#emphasis), [links](../../markdown.md#links), and [line-breaks](../../markdown.md#line-breaks) to add more context to the project description. +The project description also partially supports [standard markdown](../../markdown.md#standard-markdown-and-extensions-in-gitlab). You can use [emphasis](../../markdown.md#emphasis), [links](../../markdown.md#links), and [line-breaks](../../markdown.md#line-breaks) to add more context to the project description. ### Sharing and permissions |
