diff options
| author | at.ramya <rauthappan@gitlab.com> | 2019-06-27 13:28:12 +0530 |
|---|---|---|
| committer | at.ramya <rauthappan@gitlab.com> | 2019-06-27 13:28:12 +0530 |
| commit | fb7097c36fb770fab89d8842a35d8305c164d9fa (patch) | |
| tree | 528ded7fc0577284d6d12902a33db9cd7a181088 /doc | |
| parent | 7f721f9ef1e7e6a4858f2215f086ea9e6ea33dfe (diff) | |
| parent | edcd28ca44649b5b3e0729a95a4d6b684e1ff7cb (diff) | |
| download | gitlab-ce-fb7097c36fb770fab89d8842a35d8305c164d9fa.tar.gz | |
Merge branch 'master' into qa-xss
Diffstat (limited to 'doc')
93 files changed, 2166 insertions, 1229 deletions
diff --git a/doc/README.md b/doc/README.md index 3863e17c268..cfda2c9293d 100644 --- a/doc/README.md +++ b/doc/README.md @@ -54,7 +54,7 @@ GitLab provides solutions for [all the stages of the DevOps lifecycle](https://a  GitLab is like a top-of-the-line kitchen for making software. As the executive -chef, you decide what software you want serve. Using your recipe, GitLab handles +chef, you decide what software you want to serve. Using your recipe, GitLab handles all the prep work, cooking, and delivery, so you can turn around orders faster than ever. diff --git a/doc/administration/auth/ldap.md b/doc/administration/auth/ldap.md index 54279897e04..1f2961ea39a 100644 --- a/doc/administration/auth/ldap.md +++ b/doc/administration/auth/ldap.md @@ -396,21 +396,34 @@ omniauth-ldap. ### Escaping special characters -If the `user_filter` DN contains special characters. For example, a comma: +The `user_filter` DN can contain special characters. For example: -``` -OU=GitLab, Inc,DC=gitlab,DC=com -``` +- A comma: -This character needs to be escaped as documented in [RFC 4515](https://tools.ietf.org/search/rfc4515). + ``` + OU=GitLab, Inc,DC=gitlab,DC=com + ``` -Due to the way the string is parsed, the special character needs to be converted -to hex and `\\5C\\` (`5C` = `\` in hex) added before it. -As an example the above DN would look like +- Open and close brackets: -``` -OU=GitLab\\5C\\2C Inc,DC=gitlab,DC=com -``` + ``` + OU=Gitlab (Inc),DC=gitlab,DC=com + ``` + + These characters must be escaped as documented in + [RFC 4515](https://tools.ietf.org/search/rfc4515). + +- Escape commas with `\2C`. For example: + + ``` + OU=GitLab\2C Inc,DC=gitlab,DC=com + ``` + +- Escape open and close brackets with `\28` and `\29`, respectively. For example: + + ``` + OU=Gitlab \28Inc\29,DC=gitlab,DC=com + ``` ## Enabling LDAP sign-in for existing GitLab users diff --git a/doc/administration/auth/oidc.md b/doc/administration/auth/oidc.md index 00422ec347c..6e48add6930 100644 --- a/doc/administration/auth/oidc.md +++ b/doc/administration/auth/oidc.md @@ -144,20 +144,20 @@ for more details: If you're having trouble, here are some tips: 1. Ensure `discovery` is set to `true`. Setting it to `false` requires -specifying all the URLs and keys required to make OpenID work. + specifying all the URLs and keys required to make OpenID work. 1. Check your system clock to ensure the time is synchronized properly. 1. As mentioned in [the -documentation](https://github.com/m0n9oose/omniauth_openid_connect), -make sure `issuer` corresponds to the base URL of the Discovery URL. For -example, `https://accounts.google.com` is used for the URL -`https://accounts.google.com/.well-known/openid-configuration`. + documentation](https://github.com/m0n9oose/omniauth_openid_connect), + make sure `issuer` corresponds to the base URL of the Discovery URL. For + example, `https://accounts.google.com` is used for the URL + `https://accounts.google.com/.well-known/openid-configuration`. 1. The OpenID Connect client uses HTTP Basic Authentication to send the -OAuth2 access token. For example, if you are seeing 401 errors upon -retrieving the `userinfo` endpoint, you may want to check your OpenID -Web server configuration. For example, for -[oauth2-server-php](https://github.com/bshaffer/oauth2-server-php), you -may need to [add a configuration parameter to -Apache](https://github.com/bshaffer/oauth2-server-php/issues/926#issuecomment-387502778). + OAuth2 access token. For example, if you are seeing 401 errors upon + retrieving the `userinfo` endpoint, you may want to check your OpenID + Web server configuration. For example, for + [oauth2-server-php](https://github.com/bshaffer/oauth2-server-php), you + may need to [add a configuration parameter to + Apache](https://github.com/bshaffer/oauth2-server-php/issues/926#issuecomment-387502778). diff --git a/doc/administration/geo/replication/faq.md b/doc/administration/geo/replication/faq.md index dd1af0dbf9c..c527248bc72 100644 --- a/doc/administration/geo/replication/faq.md +++ b/doc/administration/geo/replication/faq.md @@ -6,8 +6,8 @@ The requirements are listed [on the index page](index.md#requirements-for-runnin ## How does Geo know which projects to sync? -On each **secondary** node, there is a read-only replicated copy of the GitLab database. -A **secondary** node also has a tracking database where it stores which projects have been synced. +On each **secondary** node, there is a read-only replicated copy of the GitLab database. +A **secondary** node also has a tracking database where it stores which projects have been synced. Geo compares the two databases to find projects that are not yet tracked. At the start, this tracking database is empty, so Geo will start trying to update from every project that it can see in the GitLab database. @@ -15,19 +15,19 @@ At the start, this tracking database is empty, so Geo will start trying to updat For each project to sync: 1. Geo will issue a `git fetch geo --mirror` to get the latest information from the **primary** node. -If there are no changes, the sync will be fast and end quickly. Otherwise, it will pull the latest commits. + If there are no changes, the sync will be fast and end quickly. Otherwise, it will pull the latest commits. 1. The **secondary** node will update the tracking database to store the fact that it has synced projects A, B, C, etc. 1. Repeat until all projects are synced. -When someone pushes a commit to the **primary** node, it generates an event in the GitLab database that the repository has changed. +When someone pushes a commit to the **primary** node, it generates an event in the GitLab database that the repository has changed. The **secondary** node sees this event, marks the project in question as dirty, and schedules the project to be resynced. To ensure that problems with pipelines (for example, syncs failing too many times or jobs being lost) don't permanently stop projects syncing, Geo also periodically checks the tracking database for projects that are marked as dirty. This check happens when -the number of concurrent syncs falls below `repos_max_capacity` and there are no new projects waiting to be synced. +the number of concurrent syncs falls below `repos_max_capacity` and there are no new projects waiting to be synced. -Geo also has a checksum feature which runs a SHA256 sum across all the Git references to the SHA values. -If the refs don't match between the **primary** node and the **secondary** node, then the **secondary** node will mark that project as dirty and try to resync it. -So even if we have an outdated tracking database, the validation should activate and find discrepancies in the repository state and resync. +Geo also has a checksum feature which runs a SHA256 sum across all the Git references to the SHA values. +If the refs don't match between the **primary** node and the **secondary** node, then the **secondary** node will mark that project as dirty and try to resync it. +So even if we have an outdated tracking database, the validation should activate and find discrepancies in the repository state and resync. ## Can I use Geo in a disaster recovery situation? diff --git a/doc/administration/geo/replication/troubleshooting.md b/doc/administration/geo/replication/troubleshooting.md index c5bdd36ba70..5394e6dd763 100644 --- a/doc/administration/geo/replication/troubleshooting.md +++ b/doc/administration/geo/replication/troubleshooting.md @@ -1,15 +1,23 @@ # Geo Troubleshooting **[PREMIUM ONLY]** -NOTE: **Note:** -This list is an attempt to document all the moving parts that can go wrong. -We are working into getting all this steps verified automatically in a -rake task in the future. - Setting up Geo requires careful attention to details and sometimes it's easy to -miss a step. Here is a list of questions you should ask to try to detect -what you need to fix (all commands and path locations are for Omnibus installs): +miss a step. + +Here is a list of steps you should take to attempt to fix problem: + +- Perform [basic troubleshooting](#basic-troubleshooting). +- Fix any [replication errors](#fixing-replication-errors). +- Fix any [Foreign Data Wrapper](#fixing-foreign-data-wrapper-errors) errors. +- Fix any [common](#fixing-common-errors) errors. -## First check the health of the **secondary** node +## Basic troubleshooting + +Before attempting more advanced troubleshooting: + +- Check [the health of the **secondary** node](#check-the-health-of-the-secondary-node). +- Check [if PostgreSQL replication is working](#check-if-postgresql-replication-is-working). + +### Check the health of the **secondary** node Visit the **primary** node's **Admin Area > Geo** (`/admin/geo/nodes`) in your browser. We perform the following health checks on each **secondary** node @@ -23,10 +31,12 @@ to help identify if something is wrong:  -For information on how to resolve common errors reported from the UI, see [common errors](#common-errors). +For information on how to resolve common errors reported from the UI, see +[Fixing Common Errors](#fixing-common-errors). If the UI is not working, or you are unable to log in, you can run the Geo health check manually to get this information as well as a few more details. + This rake task can be run on an app node in the **primary** or **secondary** Geo nodes: @@ -36,7 +46,7 @@ sudo gitlab-rake gitlab:geo:check Example output: -``` +```text Checking Geo ... GitLab Geo is available ... yes @@ -68,7 +78,7 @@ sudo gitlab-rake geo:status Example output: -``` +```text http://secondary.example.com/ ----------------------------------------------------- GitLab Version: 11.10.4-ee @@ -89,16 +99,21 @@ http://secondary.example.com/ Last status report was: 2 minutes ago ``` -## Is Postgres replication working? +### Check if PostgreSQL replication is working + +To check if PostgreSQL replication is working, check if: + +- [Nodes are pointing to the correct database instance](#are-nodes-pointing-to-the-correct-database-instance). +- [Geo can detect the current node correctly](#can-geo-detect-the-current-node-correctly). -### Are my nodes pointing to the correct database instance? +#### Are nodes pointing to the correct database instance? You should make sure your **primary** Geo node points to the instance with writing permissions. Any **secondary** nodes should point only to read-only instances. -### Can Geo detect my current node correctly? +#### Can Geo detect the current node correctly? Geo uses the defined node from the **Admin Area > Geo** screen, and tries to match it with the value defined in the `/etc/gitlab/gitlab.rb` configuration file. @@ -112,29 +127,38 @@ sudo gitlab-rails runner "puts Gitlab::Geo.current_node.inspect" and expect something like: -``` +```ruby #<GeoNode id: 2, schema: "https", host: "gitlab.example.com", port: 443, relative_url_root: "", primary: false, ...> ``` By running the command above, `primary` should be `true` when executed in the **primary** node, and `false` on any **secondary** node. -## How do I fix the message, "ERROR: replication slots can only be used if max_replication_slots > 0"? +## Fixing replication errors + +The following sections outline troubleshooting steps for fixing replication +errors. + +### Message: "ERROR: replication slots can only be used if max_replication_slots > 0"? This means that the `max_replication_slots` PostgreSQL variable needs to be set on the **primary** database. In GitLab 9.4, we have made this setting default to 1. You may need to increase this value if you have more -**secondary** nodes. Be sure to restart PostgreSQL for this to take +**secondary** nodes. + +Be sure to restart PostgreSQL for this to take effect. See the [PostgreSQL replication setup][database-pg-replication] guide for more details. -## How do I fix the message, "FATAL: could not start WAL streaming: ERROR: replication slot "geo_secondary_my_domain_com" does not exist"? +### Message: "FATAL: could not start WAL streaming: ERROR: replication slot "geo_secondary_my_domain_com" does not exist"? This occurs when PostgreSQL does not have a replication slot for the -**secondary** node by that name. You may want to rerun the [replication +**secondary** node by that name. + +You may want to rerun the [replication process](database.md) on the **secondary** node . -## How do I fix the message, "Command exceeded allowed execution time" when setting up replication? +### Message: "Command exceeded allowed execution time" when setting up replication? This may happen while [initiating the replication process][database-start-replication] on the **secondary** node, and indicates that your initial dataset is too large to be replicated in the default timeout (30 minutes). @@ -153,7 +177,7 @@ sudo gitlab-ctl \ This will give the initial replication up to six hours to complete, rather than the default thirty minutes. Adjust as required for your installation. -## How do I fix the message, "PANIC: could not write to file 'pg_xlog/xlogtemp.123': No space left on device" +### Message: "PANIC: could not write to file 'pg_xlog/xlogtemp.123': No space left on device" Determine if you have any unused replication slots in the **primary** database. This can cause large amounts of log data to build up in `pg_xlog`. Removing the unused slots can reduce the amount of space used in the `pg_xlog`. @@ -184,11 +208,12 @@ Slots where `active` is `f` are not active. SELECT pg_drop_replication_slot('<name_of_extra_slot>'); ``` -## Very large repositories never successfully synchronize on the **secondary** node +### Very large repositories never successfully synchronize on the **secondary** node GitLab places a timeout on all repository clones, including project imports and Geo synchronization operations. If a fresh `git clone` of a repository on the primary takes more than a few minutes, you may be affected by this. + To increase the timeout, add the following line to `/etc/gitlab/gitlab.rb` on the **secondary** node: @@ -205,7 +230,7 @@ sudo gitlab-ctl reconfigure This will increase the timeout to three hours (10800 seconds). Choose a time long enough to accommodate a full clone of your largest repositories. -## How to reset Geo **secondary** node replication +### Reseting Geo **secondary** node replication If you get a **secondary** node in a broken state and want to reset the replication state, to start again from scratch, there are a few steps that can help you: @@ -289,12 +314,16 @@ to start again from scratch, there are a few steps that can help you: gitlab-ctl start ``` -## How do I fix a "Foreign Data Wrapper (FDW) is not configured" error? +## Fixing Foreign Data Wrapper errors + +This section documents ways to fix potential Foreign Data Wrapper errors. + +### "Foreign Data Wrapper (FDW) is not configured" error When setting up Geo, you might see this warning in the `gitlab-rake gitlab:geo:check` output: -``` +```text GitLab Geo tracking database Foreign Data Wrapper schema is up-to-date? ... foreign data wrapper is not configured ``` @@ -302,12 +331,12 @@ There are a few key points to remember: 1. The FDW settings are configured on the Geo **tracking** database. 1. The configured foreign server enables a login to the Geo -**secondary**, read-only database. + **secondary**, read-only database. By default, the Geo secondary and tracking database are running on the same host on different ports. That is, 5432 and 5431 respectively. -### Checking configuration +#### Checking configuration NOTE: **Note:** The following steps are for Omnibus installs only. Using Geo with source-based installs was **deprecated** in GitLab 11.5. @@ -321,7 +350,7 @@ To check the configuration: ``` 1. Check whether any tables are present. If everything is working, you -should see something like this: + should see something like this: ```sql gitlabhq_geo_production=# SELECT * from information_schema.foreign_tables; @@ -419,7 +448,7 @@ should see something like this: - `geo_postgresql['fdw_external_user']` - `geo_postgresql['fdw_external_password']` -### Manual reload of FDW schema +#### Manual reload of FDW schema If you're still unable to get FDW working, you may want to try a manual reload of the FDW schema. To manually reload the FDW schema: @@ -459,9 +488,25 @@ reload of the FDW schema. To manually reload the FDW schema: [database-start-replication]: database.md#step-3-initiate-the-replication-process [database-pg-replication]: database.md#postgresql-replication -## Common errors +### "Geo database has an outdated FDW remote schema" error + +GitLab can error with a `Geo database has an outdated FDW remote schema` message. + +For example: -This section documents common errors reported in the admin UI and how to fix them. +```text +Geo database has an outdated FDW remote schema. It contains 229 of 236 expected tables. Please refer to Geo Troubleshooting. +``` + +To resolve this, run the following command: + +```sh +sudo gitlab-rake geo:db:refresh_foreign_tables +``` + +## Fixing common errors + +This section documents common errors reported in the Admin UI and how to fix them. ### Geo database configuration file is missing @@ -470,7 +515,6 @@ GitLab cannot find or doesn't have permission to access the `database_geo.yml` c In an Omnibus GitLab installation, the file should be in `/var/opt/gitlab/gitlab-rails/etc`. If it doesn't exist or inadvertent changes have been made to it, run `sudo gitlab-ctl reconfigure` to restore it to its correct state. - If this path is mounted on a remote volume, please check your volume configuration and that it has correct permissions. ### Geo node has a database that is writable which is an indication it is not configured for replication with the primary node. @@ -503,7 +547,7 @@ Make sure you follow the [Geo database replication](database.md) instructions fo If you are using GitLab Omnibus installation, something might have failed during upgrade. You can: -- Run `sudo gitlab-ctl reconfigure`. +- Run `sudo gitlab-ctl reconfigure`. - Manually trigger the database migration by running: `sudo gitlab-rake geo:db:migrate` as root on the **secondary** node. ### Geo database is not configured to use Foreign Data Wrapper @@ -511,4 +555,4 @@ If you are using GitLab Omnibus installation, something might have failed during This error means the Geo Tracking Database doesn't have the FDW server and credentials configured. -See [How do I fix a "Foreign Data Wrapper (FDW) is not configured" error?](#how-do-i-fix-a-foreign-data-wrapper-fdw-is-not-configured-error). +See ["Foreign Data Wrapper (FDW) is not configured" error?](#foreign-data-wrapper-fdw-is-not-configured-error). diff --git a/doc/administration/high_availability/README.md b/doc/administration/high_availability/README.md index d9c80b1ec59..0c4f926c579 100644 --- a/doc/administration/high_availability/README.md +++ b/doc/administration/high_availability/README.md @@ -65,6 +65,7 @@ larger one. - 1 Redis node - 1 NFS/Gitaly storage server - 2 or more GitLab application nodes (Unicorn, Workhorse, Sidekiq) +- 1 Monitoring node (Prometheus, Grafana) #### Installation Instructions @@ -76,6 +77,7 @@ you can continue with the next step. 1. [Redis](redis.md#redis-in-a-scaled-environment) 1. [Gitaly](gitaly.md) (recommended) or [NFS](nfs.md) 1. [GitLab application nodes](gitlab.md) +1. [Monitoring node (Prometheus and Grafana)](monitoring_node.md) ### Full Scaling @@ -90,6 +92,7 @@ in size, indicating that there is contention or not enough resources. - 2 or more NFS/Gitaly storage servers - 2 or more Sidekiq nodes - 2 or more GitLab application nodes (Unicorn, Workhorse) +- 1 Monitoring node (Prometheus, Grafana) ## High Availability Architecture Examples @@ -133,6 +136,7 @@ the contention. - 3 Consul/Sentinel nodes - 2 or more GitLab application nodes (Unicorn, Workhorse, Sidekiq, PGBouncer) - 1 NFS/Gitaly server +- 1 Monitoring node (Prometheus, Grafana)  @@ -192,6 +196,7 @@ with the added complexity of many more nodes to configure, manage and monitor. - 2 or more API nodes (All requests to `/api`) - 2 or more Web nodes (All other web requests) - 2 or more NFS/Gitaly servers +- 1 Monitoring node (Prometheus, Grafana)  @@ -205,4 +210,5 @@ separately: 1. [NFS Client and Host setup](nfs_host_client_setup.md) 1. [Configure the GitLab application servers](gitlab.md) 1. [Configure the load balancers](load_balancer.md) +1. [Monitoring node (Prometheus and Grafana)](monitoring_node.md) diff --git a/doc/administration/high_availability/database.md b/doc/administration/high_availability/database.md index 2c051b660ee..20bbfdb2603 100644 --- a/doc/administration/high_availability/database.md +++ b/doc/administration/high_availability/database.md @@ -82,6 +82,7 @@ deploy the bundled PostgreSQL. 1. Note the PostgreSQL node's IP address or hostname, port, and plain text password. These will be necessary when configuring the GitLab application servers later. +1. [Enable monitoring](#enable-monitoring) Advanced configuration options are supported and can be added if needed. @@ -203,9 +204,9 @@ Few notes on the service itself: - The service runs under a system account, by default `gitlab-consul`. - If you are using a different username, you will have to specify it. We -will refer to it with `CONSUL_USERNAME`, + will refer to it with `CONSUL_USERNAME`, - There will be a database user created with read only access to the repmgr -database + database - Passwords will be stored in the following locations: - `/etc/gitlab/gitlab.rb`: hashed - `/var/opt/gitlab/pgbouncer/pg_auth`: hashed @@ -399,6 +400,7 @@ check the [Troubleshooting section](#troubleshooting) before proceeding. ``` 1. [Reconfigure GitLab] for the changes to take effect. +1. [Enable Monitoring](#enable-monitoring) > Please note: > @@ -1086,6 +1088,25 @@ the previous section: the `gitlab` database user 1. [Reconfigure GitLab] for the changes to take effect +## Enable Monitoring + +> [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/3786) in GitLab 12.0. + +If you enable Monitoring, it must be enabled on **all** database servers. + +1. Create/edit `/etc/gitlab/gitlab.rb` and add the following configuration: + + ```ruby + # Enable service discovery for Prometheus + consul['monitoring_service_discovery'] = true + + # Set the network addresses that the exporters will listen on + node_exporter['listen_address'] = '0.0.0.0:9100' + postgres_exporter['listen_address'] = '0.0.0.0:9187' + ``` + +1. Run `sudo gitlab-ctl reconfigure` to compile the configuration. + ## Troubleshooting #### Consul and PostgreSQL changes not taking effect. diff --git a/doc/administration/high_availability/gitaly.md b/doc/administration/high_availability/gitaly.md index 1d8e6c999cb..b7eaa4ce105 100644 --- a/doc/administration/high_availability/gitaly.md +++ b/doc/administration/high_availability/gitaly.md @@ -2,13 +2,13 @@ Gitaly does not yet support full high availability. However, Gitaly is quite stable and is in use on GitLab.com. Scaled and highly available GitLab environments -should consider using Gitaly on a separate node. +should consider using Gitaly on a separate node. -See the [Gitaly HA Epic](https://gitlab.com/groups/gitlab-org/-/epics/289) to -track plans and progress toward high availability support. +See the [Gitaly HA Epic](https://gitlab.com/groups/gitlab-org/-/epics/289) to +track plans and progress toward high availability support. This document is relevant for [Scaled Architecture](README.md#scalable-architecture-examples) -environments and [High Availability Architecture](README.md#high-availability-architecture-examples). +environments and [High Availability Architecture](README.md#high-availability-architecture-examples). ## Running Gitaly on its own server @@ -19,3 +19,30 @@ Continue configuration of other components by going back to: - [Scaled Architectures](README.md#scalable-architecture-examples) - [High Availability Architectures](README.md#high-availability-architecture-examples) + +## Enable Monitoring + +> [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/3786) in GitLab 12.0. + +1. Make sure to collect [`CONSUL_SERVER_NODES`](database.md#consul-information), which are the IP addresses or DNS records of the Consul server nodes, for the next step. Note they are presented as `Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z` + +1. Create/edit `/etc/gitlab/gitlab.rb` and add the following configuration: + + ```ruby + # Enable service discovery for Prometheus + consul['enable'] = true + consul['monitoring_service_discovery'] = true + + # Replace placeholders + # Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z + # with the addresses of the Consul server nodes + consul['configuration'] = { + retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z), + } + + # Set the network addresses that the exporters will listen on + node_exporter['listen_address'] = '0.0.0.0:9100' + gitaly['prometheus_listen_addr'] = "0.0.0.0:9236" + ``` + +1. Run `sudo gitlab-ctl reconfigure` to compile the configuration. diff --git a/doc/administration/high_availability/gitlab.md b/doc/administration/high_availability/gitlab.md index 7e3ff741071..3045be616a6 100644 --- a/doc/administration/high_availability/gitlab.md +++ b/doc/administration/high_availability/gitlab.md @@ -76,6 +76,8 @@ registry['gid'] = 9002 ``` +1. [Enable monitoring](#enable-monitoring) + > **Note:** To maintain uniformity of links across HA clusters, the `external_url` on the first application server as well as the additional application servers should point to the external url that users will use to access GitLab. @@ -88,7 +90,8 @@ [Nginx documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https) for more information. > - > **Note:** It is best to set the `uid` and `gid`s prior to the initial reconfigure of GitLab. Omnibus will not recursively `chown` directories if set after the initial reconfigure. + > **Note:** It is best to set the `uid` and `gid`s prior to the initial reconfigure + of GitLab. Omnibus will not recursively `chown` directories if set after the initial reconfigure. ## First GitLab application server @@ -129,6 +132,47 @@ need some extra configuration. 1. Run `sudo gitlab-ctl reconfigure` to compile the configuration. +## Enable Monitoring + +> [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/3786) in GitLab 12.0. + +If you enable Monitoring, it must be enabled on **all** GitLab servers. + +1. Make sure to collect [`CONSUL_SERVER_NODES`](database.md#consul-information), which are the IP addresses or DNS records of the Consul server nodes, for the next step. Note they are presented as `Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z` + +1. Create/edit `/etc/gitlab/gitlab.rb` and add the following configuration: + + ```ruby + # Enable service discovery for Prometheus + consul['enable'] = true + consul['monitoring_service_discovery'] = true + + # Replace placeholders + # Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z + # with the addresses of the Consul server nodes + consul['configuration'] = { + retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z), + } + + # Set the network addresses that the exporters will listen on + node_exporter['listen_address'] = '0.0.0.0:9100' + gitlab_workhorse['prometheus_listen_addr'] = '0.0.0.0:9229' + sidekiq['listen_address'] = "0.0.0.0" + unicorn['listen'] = '0.0.0.0' + + # Add the monitoring node's IP address to the monitoring whitelist and allow it to + # scrape the NGINX metrics. Replace placeholder `monitoring.gitlab.example.com` with + # the address and/or subnets gathered from the monitoring node(s). + gitlab_rails['monitoring_whitelist'] = ['monitoring.gitlab.example.com', '127.0.0.0/8'] + nginx['status']['options']['allow'] = ['monitoring.gitlab.example.com', '127.0.0.0/8'] + ``` + +1. Run `sudo gitlab-ctl reconfigure` to compile the configuration. + +> **Warning:** After changing `unicorn['listen']` in `gitlab.rb`, and running `sudo gitlab-ctl reconfigure`, + it can take an extended period of time for unicorn to complete reloading after receiving a `HUP`. + For more information, see the [issue](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/4401). + ## Troubleshooting - `mount: wrong fs type, bad option, bad superblock on` diff --git a/doc/administration/high_availability/monitoring_node.md b/doc/administration/high_availability/monitoring_node.md new file mode 100644 index 00000000000..ef415dde10a --- /dev/null +++ b/doc/administration/high_availability/monitoring_node.md @@ -0,0 +1,69 @@ +# Configuring a Monitoring node for Scaling and High Availability + +> [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/3786) in GitLab 12.0. + +## Standalone Monitoring node using GitLab Omnibus + +The GitLab Omnibus package can be used to configure a standalone Monitoring node running Prometheus and Grafana. +The monitoring node is not highly available. See [Scaling and High Availability](README.md) +for an overview of GitLab scaling and high availability options. + +The steps below are the minimum necessary to configure a Monitoring node running Prometheus and Grafana with +Omnibus: + +1. SSH into the Monitoring node. +1. [Download/install](https://about.gitlab.com/installation) the Omnibus GitLab + package you want using **steps 1 and 2** from the GitLab downloads page. + - Do not complete any other steps on the download page. + +1. Make sure to collect [`CONSUL_SERVER_NODES`](database.md#consul-information), which are the IP addresses or DNS records of the Consul server nodes, for the next step. Note they are presented as `Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z` + +1. Edit `/etc/gitlab/gitlab.rb` and add the contents: + + ```ruby + external_url 'http://gitlab.example.com' + + # Enable Prometheus + prometheus['enable'] = true + prometheus['listen_address'] = '0.0.0.0:9090' + prometheus['monitor_kubernetes'] = false + + # Enable Grafana + grafana['enable'] = true + grafana['admin_password'] = 'toomanysecrets' + + # Enable service discovery for Prometheus + consul['enable'] = true + consul['monitoring_service_discovery'] = true + + # Replace placeholders + # Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z + # with the addresses of the Consul server nodes + consul['configuration'] = { + retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z), + } + + # Disable all other services + gitlab_rails['auto_migrate'] = false + alertmanager['enable'] = false + gitaly['enable'] = false + gitlab_monitor['enable'] = false + gitlab_workhorse['enable'] = false + nginx['enable'] = true + postgres_exporter['enable'] = false + postgresql['enable'] = false + redis['enable'] = false + redis_exporter['enable'] = false + sidekiq['enable'] = false + unicorn['enable'] = false + node_exporter['enable'] = false + ``` + +1. Run `sudo gitlab-ctl reconfigure` to compile the configuration. + +## Migrating to Service Discovery + +Once monitoring using Service Discovery is enabled with `consul['monitoring_service_discovery'] = true`, +ensure that `prometheus['scrape_configs']` is not set in `/etc/gitlab/gitlab.rb`. Setting both +`consul['monitoring_service_discovery'] = true` and `prometheus['scrape_configs']` in `/etc/gitlab/gitlab.rb` +will result in errors. diff --git a/doc/administration/high_availability/pgbouncer.md b/doc/administration/high_availability/pgbouncer.md index 762179cf756..053dae25823 100644 --- a/doc/administration/high_availability/pgbouncer.md +++ b/doc/administration/high_availability/pgbouncer.md @@ -62,6 +62,33 @@ See our [HA documentation for PostgreSQL](database.md) for information on runnin 1. At this point, your instance should connect to the database through pgbouncer. If you are having issues, see the [Troubleshooting](#troubleshooting) section +## Enable Monitoring + +> [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/3786) in GitLab 12.0. + + If you enable Monitoring, it must be enabled on **all** pgbouncer servers. + + 1. Create/edit `/etc/gitlab/gitlab.rb` and add the following configuration: + + ```ruby + # Enable service discovery for Prometheus + consul['enable'] = true + consul['monitoring_service_discovery'] = true + + # Replace placeholders + # Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z + # with the addresses of the Consul server nodes + consul['configuration'] = { + retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z), + } + + # Set the network addresses that the exporters will listen on + node_exporter['listen_address'] = '0.0.0.0:9100' + pgbouncer_exporter['listen_address'] = '0.0.0.0:9188' + ``` + + 1. Run `sudo gitlab-ctl reconfigure` to compile the configuration. + ### Interacting with pgbouncer #### Administrative console diff --git a/doc/administration/high_availability/redis.md b/doc/administration/high_availability/redis.md index 1aaa709fc8f..8621224272c 100644 --- a/doc/administration/high_availability/redis.md +++ b/doc/administration/high_availability/redis.md @@ -22,10 +22,10 @@ environments including [Basic Scaling](README.md#basic-scaling) and ### Provide your own Redis instance **[CORE ONLY]** -If you want to use your own deployed Redis instance(s), -see [Provide your own Redis instance](#provide-your-own-redis-instance-core-only) -for more details. However, you can use the GitLab Omnibus package to easily -deploy the bundled Redis. +If you want to use your own deployed Redis instance(s), +see [Provide your own Redis instance](#provide-your-own-redis-instance-core-only) +for more details. However, you can use the GitLab Omnibus package to easily +deploy the bundled Redis. ### Standalone Redis using GitLab Omnibus **[CORE ONLY]** @@ -62,11 +62,11 @@ Omnibus: pgbouncer_exporter['enable'] = false gitlab_monitor['enable'] = false gitaly['enable'] = false - + redis['bind'] = '0.0.0.0' redis['port'] = '6379' redis['password'] = 'SECRET_PASSWORD_HERE' - + gitlab_rails['auto_migrate'] = false ``` @@ -74,6 +74,7 @@ Omnibus: 1. Note the Redis node's IP address or hostname, port, and Redis password. These will be necessary when configuring the GitLab application servers later. +1. [Enable Monitoring](#enable-monitoring) Advanced configuration options are supported and can be added if needed. @@ -90,10 +91,10 @@ environments including [Horizontal](README.md#horizontal), ### Provide your own Redis instance **[CORE ONLY]** -If you want to use your own deployed Redis instance(s), -see [Provide your own Redis instance](#provide-your-own-redis-instance-core-only) -for more details. However, you can use the GitLab Omnibus package to easily -deploy the bundled Redis. +If you want to use your own deployed Redis instance(s), +see [Provide your own Redis instance](#provide-your-own-redis-instance-core-only) +for more details. However, you can use the GitLab Omnibus package to easily +deploy the bundled Redis. ### High Availability with GitLab Omnibus **[PREMIUM ONLY]** @@ -367,7 +368,7 @@ The prerequisites for a HA Redis setup are the following: ```ruby # Specify server role as 'redis_master_role' roles ['redis_master_role'] - + # IP address pointing to a local IP that the other machines can reach to. # You can also set bind to '0.0.0.0' which listen in all interfaces. # If you really need to bind to an external accessible IP, make @@ -381,7 +382,7 @@ The prerequisites for a HA Redis setup are the following: # Set up password authentication for Redis (use the same password in all nodes). redis['password'] = 'redis-password-goes-here' ``` - + 1. Only the primary GitLab application server should handle migrations. To prevent database migrations from running on upgrade, add the following @@ -393,8 +394,8 @@ The prerequisites for a HA Redis setup are the following: 1. [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect. -> Note: You can specify multiple roles like sentinel and redis as: -> roles ['redis_sentinel_role', 'redis_master_role']. Read more about high +> Note: You can specify multiple roles like sentinel and redis as: +> roles ['redis_sentinel_role', 'redis_master_role']. Read more about high > availability roles at https://docs.gitlab.com/omnibus/roles/ ### Step 2. Configuring the slave Redis instances @@ -411,7 +412,7 @@ The prerequisites for a HA Redis setup are the following: ```ruby # Specify server role as 'redis_slave_role' roles ['redis_slave_role'] - + # IP address pointing to a local IP that the other machines can reach to. # You can also set bind to '0.0.0.0' which listen in all interfaces. # If you really need to bind to an external accessible IP, make @@ -442,8 +443,8 @@ The prerequisites for a HA Redis setup are the following: 1. [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect. 1. Go through the steps again for all the other slave nodes. -> Note: You can specify multiple roles like sentinel and redis as: -> roles ['redis_sentinel_role', 'redis_slave_role']. Read more about high +> Note: You can specify multiple roles like sentinel and redis as: +> roles ['redis_sentinel_role', 'redis_slave_role']. Read more about high > availability roles at https://docs.gitlab.com/omnibus/roles/ --- @@ -749,6 +750,35 @@ gitlab_rails['redis_sentinels'] = [ [Reconfigure Omnibus GitLab][reconfigure] for the changes to take effect. +## Enable Monitoring + +> [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/3786) in GitLab 12.0. + +If you enable Monitoring, it must be enabled on **all** Redis servers. + +1. Make sure to collect [`CONSUL_SERVER_NODES`](database.md#consul-information), which are the IP addresses or DNS records of the Consul server nodes, for the next step. Note they are presented as `Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z` + +1. Create/edit `/etc/gitlab/gitlab.rb` and add the following configuration: + + ```ruby + # Enable service discovery for Prometheus + consul['enable'] = true + consul['monitoring_service_discovery'] = true + + # Replace placeholders + # Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z + # with the addresses of the Consul server nodes + consul['configuration'] = { + retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z), + } + + # Set the network addresses that the exporters will listen on + node_exporter['listen_address'] = '0.0.0.0:9100' + redis_exporter['listen_address'] = '0.0.0.0:9121' + ``` + +1. Run `sudo gitlab-ctl reconfigure` to compile the configuration. + ## Advanced configuration Omnibus GitLab configures some things behind the curtains to make the sysadmins' diff --git a/doc/administration/operations/extra_sidekiq_processes.md b/doc/administration/operations/extra_sidekiq_processes.md index 286b99aceb5..7297507f599 100644 --- a/doc/administration/operations/extra_sidekiq_processes.md +++ b/doc/administration/operations/extra_sidekiq_processes.md @@ -1,70 +1,132 @@ # Extra Sidekiq processes **[STARTER ONLY]** -GitLab Enterprise Edition allows one to start an extra set of Sidekiq processes +NOTE: **Note:** +The information in this page applies only to Omnibus GitLab. + +GitLab Starter allows one to start an extra set of Sidekiq processes besides the default one. These processes can be used to consume a dedicated set of queues. This can be used to ensure certain queues always have dedicated workers, no matter the number of jobs that need to be processed. -## Starting extra processes via Omnibus GitLab +## Available Sidekiq queues -To enable `sidekiq-cluster`, you must apply the `sidekiq_cluster['enable'] = true` -setting `/etc/gitlab/gitlab.rb`: +For a list of the existing Sidekiq queues, check the following files: -```ruby -sidekiq_cluster['enable'] = true -``` +- [Queues for both GitLab Community and Enterprise Editions](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/app/workers/all_queues.yml) +- [Queues for GitLab Enterprise Editions only](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/app/workers/all_queues.yml) -You will then specify how many additional processes to create via `sidekiq-cluster` -as well as which queues for them to handle. This is done via the -`sidekiq_cluster['queue_groups']` setting. This is an array whose items contain -which queues to process. Each item in the array will equate to one additional -sidekiq process. +Each entry in the above files represents a queue on which extra Sidekiq processes +can be started. -As an example, to make additional sidekiq processes that process the -`elastic_indexer` and `mailers` queues, you would apply the following: +## Starting extra processes -```ruby -sidekiq_cluster['queue_groups'] = [ - "elastic_indexer", - "mailers" -] -``` +To start extra Sidekiq processes, you must enable `sidekiq-cluster`: -To have an additional sidekiq process handle multiple queues, you simply put a -comma after the first queue name and then put the next queue name: +1. Edit `/etc/gitlab/gitlab.rb` and add: -```ruby -sidekiq_cluster['queue_groups'] = [ - "elastic_indexer,elastic_commit_indexer", - "mailers" -] -``` + ```ruby + sidekiq_cluster['enable'] = true + ``` -Keep in mind, all changes must be followed by reconfiguring your GitLab -application via `sudo gitlab-ctl reconfigure`. +1. You will then need to specify how many additional processes to create via `sidekiq-cluster` + and which queue they should handle via the `sidekiq_cluster['queue_groups']` + array setting. Each item in the array equates to one additional Sidekiq + process, and values in each item determine the queues it works on. -### Monitoring + For example, the following setting adds additional Sidekiq processes to two + queues, one to `elastic_indexer` and one to `mailers`: -Once the Sidekiq processes are added, you can visit the "Background Jobs" + ```ruby + sidekiq_cluster['queue_groups'] = [ + "elastic_indexer", + "mailers" + ] + ``` + + To have an additional Sidekiq process handle multiple queues, add multiple + queue names to its item delimited by commas. For example: + + ```ruby + sidekiq_cluster['queue_groups'] = [ + "elastic_indexer, elastic_commit_indexer", + "mailers" + ] + ``` + +1. Save the file and reconfigure GitLab for the changes to take effect: + + ```sh + sudo gitlab-ctl reconfigure + ``` + +Once the extra Sidekiq processes are added, you can visit the "Background Jobs" section under the admin area in GitLab (`/admin/background_jobs`). - + -### All queues with exceptions +## Negating settings -To have the additional sidekiq processes work on every queue EXCEPT the ones +To have the additional Sidekiq processes work on every queue **except** the ones you list: +1. After you follow the steps for [starting extra processes](#starting-extra-processes), + edit `/etc/gitlab/gitlab.rb` and add: + + ```ruby + sidekiq_cluster['negate'] = true + ``` + +1. Save the file and reconfigure GitLab for the changes to take effect: + + ```sh + sudo gitlab-ctl reconfigure + ``` + +## Ignore all GitHub import queues + +When [importing from GitHub](../../user/project/import/github.md), Sidekiq might +use all of its resources to perform those operations. To set up a separate +`sidekiq-cluster` process to ignore all GitHub import-related queues: + 1. Edit `/etc/gitlab/gitlab.rb` and add: ```ruby + sidekiq_cluster['enable'] = true sidekiq_cluster['negate'] = true + sidekiq_cluster['queue_groups'] = [ + "github_import_advance_stage", + "github_importer:github_import_import_diff_note", + "github_importer:github_import_import_issue", + "github_importer:github_import_import_note", + "github_importer:github_import_import_lfs_object", + "github_importer:github_import_import_pull_request", + "github_importer:github_import_refresh_import_jid", + "github_importer:github_import_stage_finish_import", + "github_importer:github_import_stage_import_base_data", + "github_importer:github_import_stage_import_issues_and_diff_notes", + "github_importer:github_import_stage_import_notes", + "github_importer:github_import_stage_import_lfs_objects", + "github_importer:github_import_stage_import_pull_requests", + "github_importer:github_import_stage_import_repository" + ] ``` -1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. +1. Save the file and reconfigure GitLab for the changes to take effect: + ```sh + sudo gitlab-ctl reconfigure + ``` -### Limiting concurrency +## Number of threads + +Each process defined under `sidekiq_cluster` starts with a +number of threads that equals the number of queues, plus one spare thread. +For example, a process that handles the `process_commit` and `post_receive` +queues will use three threads in total. + +## Limiting concurrency + +To limit the concurrency of the Sidekiq processes: 1. Edit `/etc/gitlab/gitlab.rb` and add: @@ -72,11 +134,22 @@ you list: sidekiq_cluster['concurrency'] = 25 ``` -1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. +1. Save the file and reconfigure GitLab for the changes to take effect: -Keep in mind, this normally would not exceed the number of CPU cores available. + ```sh + sudo gitlab-ctl reconfigure + ``` -### Modifying the check interval +For each queue group, the concurrency factor will be set to `min(number of queues, N)`. +Setting the value to 0 will disable the limit. Keep in mind this normally would +not exceed the number of CPU cores available. + +Each thread requires a Redis connection, so adding threads may +increase Redis latency and potentially cause client timeouts. See the [Sidekiq +documentation about Redis](https://github.com/mperham/sidekiq/wiki/Using-Redis) +for more details. + +## Modifying the check interval To modify the check interval for the additional Sidekiq processes: @@ -90,9 +163,14 @@ To modify the check interval for the additional Sidekiq processes: This tells the additional processes how often to check for enqueued jobs. -## Starting extra processes via command line +## Troubleshooting using the CLI -Starting extra Sidekiq processes can be done using the command +CAUTION: **Warning:** +It's recommended to use `/etc/gitlab/gitlab.rb` to configure the Sidekiq processes. +If you experience a problem, you should contact GitLab support. Use the command +line at your own risk. + +For debugging purposes, you can start extra Sidekiq processes by using the command `/opt/gitlab/embedded/service/gitlab-rails/ee/bin/sidekiq-cluster`. This command takes arguments using the following syntax: @@ -111,29 +189,29 @@ see the relevant section in the [Sidekiq style guide](../../development/sidekiq_style_guide.md#queue-namespaces). For example, say you want to start 2 extra processes: one to process the -"process_commit" queue, and one to process the "post_receive" queue. This can be +`process_commit` queue, and one to process the `post_receive` queue. This can be done as follows: ```bash /opt/gitlab/embedded/service/gitlab-rails/ee/bin/sidekiq-cluster process_commit post_receive ``` -If you instead want to start one process processing both queues you'd use the +If you instead want to start one process processing both queues, you'd use the following syntax: ```bash /opt/gitlab/embedded/service/gitlab-rails/ee/bin/sidekiq-cluster process_commit,post_receive ``` -If you want to have one Sidekiq process process the "process_commit" and -"post_receive" queues, and one process to process the "gitlab_shell" queue, +If you want to have one Sidekiq process dealing with the `process_commit` and +`post_receive` queues, and one process to process the `gitlab_shell` queue, you'd use the following: ```bash /opt/gitlab/embedded/service/gitlab-rails/ee/bin/sidekiq-cluster process_commit,post_receive gitlab_shell ``` -### Monitoring +### Monitoring the `sidekiq-cluster` command The `sidekiq-cluster` command will not terminate once it has started the desired amount of Sidekiq processes. Instead, the process will continue running and @@ -172,24 +250,24 @@ command and not the PID(s) of the started Sidekiq processes. The Rails environment can be set by passing the `--environment` flag to the `sidekiq-cluster` command, or by setting `RAILS_ENV` to a non-empty value. The -default value is "development". +default value can be found in `/opt/gitlab/etc/gitlab-rails/env/RAILS_ENV`. -### All queues with exceptions +### Using negation You're able to run all queues in `sidekiq_queues.yml` file on a single or multiple processes with exceptions using the `--negate` flag. For example, say you want to run a single process for all queues, -except "process_commit" and "post_receive". You can do so by executing: +except `process_commit` and `post_receive`: ```bash -sidekiq-cluster process_commit,post_receive --negate +/opt/gitlab/embedded/service/gitlab-rails/ee/bin/sidekiq-cluster process_commit,post_receive --negate ``` -For multiple processes of all queues (except "process_commit" and "post_receive"): +For multiple processes of all queues (except `process_commit` and `post_receive`): ```bash -sidekiq-cluster process_commit,post_receive process_commit,post_receive --negate +/opt/gitlab/embedded/service/gitlab-rails/ee/bin/sidekiq-cluster process_commit,post_receive process_commit,post_receive --negate ``` ### Limiting concurrency @@ -201,18 +279,3 @@ the `-m N` option. For example, this would cap the maximum number of threads to ```bash /opt/gitlab/embedded/service/gitlab-rails/ee/bin/sidekiq-cluster process_commit,post_receive -m 1 ``` - -For each queue group, the concurrency factor will be set to min(number of -queues, N). Setting the value to 0 will disable the limit. - -Note that each thread requires a Redis connection, so adding threads may -increase Redis latency and potentially cause client timeouts. See the [Sidekiq -documentation about Redis](https://github.com/mperham/sidekiq/wiki/Using-Redis) -for more details. - -## Number of threads - -Each process started using `sidekiq-cluster` (whether it be via command line or -via the gitlab.rb file) starts with a number of threads that equals the number -of queues, plus one spare thread. For example, a process that handles the -"process_commit" and "post_receive" queues will use 3 threads in total. diff --git a/doc/api/README.md b/doc/api/README.md index 3a1064b787e..23c69deef23 100644 --- a/doc/api/README.md +++ b/doc/api/README.md @@ -35,6 +35,7 @@ The following API resources are available in the project context: | [Environments](environments.md) | `/projects/:id/environments` | | [Events](events.md) | `/projects/:id/events` (also available for users and standalone) | | [Issues](issues.md) | `/projects/:id/issues` (also available for groups and standalone) | +| [Issues Statistics](issues_statistics.md) | `/projects/:id/issues_statistics` (also available for groups and standalone) | | [Issue boards](boards.md) | `/projects/:id/boards` | | [Issue links](issue_links.md) **[STARTER]** | `/projects/:id/issues/.../links` | | [Jobs](jobs.md) | `/projects/:id/jobs`, `/projects/:id/pipelines/.../jobs` | @@ -92,6 +93,7 @@ The following API resources are available in the group context: | [Group-level variables](group_level_variables.md) | `/groups/:id/variables` | | [Group milestones](group_milestones.md) | `/groups/:id/milestones` | | [Issues](issues.md) | `/groups/:id/issues` (also available for projects and standalone) | +| [Issues Statistics](issues_statistics.md) | `/groups/:id/issues_statistics` (also available for projects and standalone) | | [Members](members.md) | `/groups/:id/members` (also available for projects) | | [Merge requests](merge_requests.md) | `/groups/:id/merge_requests` (also available for projects and standalone) | | [Notes](notes.md) (comments) | `/groups/:id/epics/.../notes` (also available for projects) | @@ -116,6 +118,7 @@ The following API resources are available outside of project and group contexts | [Geo Nodes](geo_nodes.md) **[PREMIUM ONLY]** | `/geo_nodes` | | [Import repository from GitHub](import.md) | `/import/github` | | [Issues](issues.md) | `/issues` (also available for groups and projects) | +| [Issues Statistics](issues_statistics.md) | `/issues_statistics` (also available for groups and projects) | | [Keys](keys.md) | `/keys` | | [License](license.md) **[CORE ONLY]** | `/license` | | [Markdown](markdown.md) | `/markdown` | diff --git a/doc/api/container_registry.md b/doc/api/container_registry.md index 1f17af1f1e9..64ea15bca93 100644 --- a/doc/api/container_registry.md +++ b/doc/api/container_registry.md @@ -145,6 +145,9 @@ DELETE /projects/:id/registry/repositories/:repository_id/tags/:tag_name curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/registry/repositories/2/tags/v10.0.0" ``` +This action does not delete blobs. In order to delete them and recycle disk space, +[run the garbage collection](https://docs.gitlab.com/omnibus/maintenance/README.html#removing-unused-layers-not-referenced-by-manifests). + ## Delete repository tags in bulk Delete repository tags in bulk based on given criteria. @@ -174,6 +177,8 @@ This API call performs the following operations: These operations are executed asynchronously and it might take time to get executed. You can run this at most once an hour for a given container repository. +This action does not delete blobs. In order to delete them and recycle disk space, +[run the garbage collection](https://docs.gitlab.com/omnibus/maintenance/README.html#removing-unused-layers-not-referenced-by-manifests). NOTE: **Note:** Due to a [Docker Distribution deficiency](https://gitlab.com/gitlab-org/gitlab-ce/issues/21405), diff --git a/doc/api/environments.md b/doc/api/environments.md index ebcdc546d08..44f86861ff7 100644 --- a/doc/api/environments.md +++ b/doc/api/environments.md @@ -11,9 +11,11 @@ GET /projects/:id/environments | Attribute | Type | Required | Description | | --------- | ------- | -------- | --------------------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `name` | string | no | Return the environment with this name. Mutually exclusive with `search` | +| `search` | string | no | Return list of environments matching the search criteria. Mutually exclusive with `name` | ```bash -curl --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/1/environments +curl --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/1/environments?name=review%2Ffix-foo ``` Example response: diff --git a/doc/api/merge_request_approvals.md b/doc/api/merge_request_approvals.md index ddac81328b9..49aaac06b46 100644 --- a/doc/api/merge_request_approvals.md +++ b/doc/api/merge_request_approvals.md @@ -222,7 +222,7 @@ GET /projects/:id/merge_requests/:merge_request_iid/approvals "id": 1, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80\u0026d=identicon", - "web_url": "http://localhost:3000/u/root" + "web_url": "http://localhost:3000/root" } } ], @@ -314,7 +314,7 @@ PUT /projects/:id/merge_requests/:merge_request_iid/approvers "id": 1, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80\u0026d=identicon", - "web_url": "http://localhost:3000/u/root" + "web_url": "http://localhost:3000/root" } } ], @@ -387,7 +387,7 @@ does not match, the response code will be `409`. "id": 1, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80\u0026d=identicon", - "web_url": "http://localhost:3000/u/root" + "web_url": "http://localhost:3000/root" } }, { @@ -397,7 +397,7 @@ does not match, the response code will be `409`. "id": 2, "state": "active", "avatar_url": "http://www.gravatar.com/avatar/cf7ad14b34162a76d593e3affca2adca?s=80\u0026d=identicon", - "web_url": "http://localhost:3000/u/ryley" + "web_url": "http://localhost:3000/ryley" } } ], diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index dd7810c3403..7b58aa3100e 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -1191,33 +1191,29 @@ Parameters: } ``` -## Merge to default merge ref path +## Returns the up to date merge-ref HEAD commit Merge the changes between the merge request source and target branches into `refs/merge-requests/:iid/merge` -ref, of the target project repository. This ref will have the state the target branch would have if +ref, of the target project repository, if possible. This ref will have the state the target branch would have if a regular merge action was taken. -This is not a regular merge action given it doesn't change the merge request state in any manner. +This is not a regular merge action given it doesn't change the merge request target branch state in any manner. -This ref (`refs/merge-requests/:iid/merge`) is **always** overwritten when submitting -requests to this API, so none of its state is kept or used in the process. +This ref (`refs/merge-requests/:iid/merge`) isn't necessarily overwritten when submitting +requests to this API, though it'll make sure the ref has the latest possible state. -If the merge request has conflicts, is empty or already merged, -you'll get a `400` and a descriptive error message. If you don't have permissions to do so, -you'll get a `403`. +If the merge request has conflicts, is empty or already merged, you'll get a `400` and a descriptive error message. -It returns the HEAD commit of `refs/merge-requests/:iid/merge` in the response body in -case of `200`. +It returns the HEAD commit of `refs/merge-requests/:iid/merge` in the response body in case of `200`. ``` -PUT /projects/:id/merge_requests/:merge_request_iid/merge_to_ref +GET /projects/:id/merge_requests/:merge_request_iid/merge_ref ``` Parameters: - `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user - `merge_request_iid` (required) - Internal ID of MR -- `merge_commit_message` (optional) - Custom merge commit message ```json { diff --git a/doc/api/search.md b/doc/api/search.md index da81c8321c9..abb77ae05dc 100644 --- a/doc/api/search.md +++ b/doc/api/search.md @@ -285,7 +285,7 @@ Example response: ### Scope: wiki_blobs **[STARTER]** -This scope is available only if [Elasticsearch](../integration/elasticsearch.md) is enabled. +This scope is available only if [Elasticsearch](../integration/elasticsearch.md) is enabled. ```bash curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/search?scope=wiki_blobs&search=bye @@ -346,6 +346,7 @@ Example response: This scope is available only if [Elasticsearch](../integration/elasticsearch.md) is enabled. Filters are available for this scope: + - filename - path - extension @@ -679,6 +680,7 @@ Example response: This scope is available only if [Elasticsearch](../integration/elasticsearch.md) is enabled. Filters are available for this scope: + - filename - path - extension diff --git a/doc/api/settings.md b/doc/api/settings.md index c2a1f7feefd..b01cec64837 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -142,8 +142,6 @@ are listed in the descriptions of the relevant settings. | `authorized_keys_enabled` | boolean | no | By default, we write to the `authorized_keys` file to support Git over SSH without additional configuration. GitLab can be optimized to authenticate SSH keys via the database file. Only disable this if you have configured your OpenSSH server to use the AuthorizedKeysCommand. | | `auto_devops_domain` | string | no | Specify a domain to use by default for every project's Auto Review Apps and Auto Deploy stages. | | `auto_devops_enabled` | boolean | no | Enable Auto DevOps for projects by default. It will automatically build, test, and deploy applications based on a predefined CI/CD configuration. | -| `clientside_sentry_dsn` | string | required by: `clientside_sentry_enabled` | Clientside Sentry Data Source Name. | -| `clientside_sentry_enabled` | boolean | no | (**If enabled, requires:** `clientside_sentry_dsn`) Enable Sentry error reporting for the client side. | | `container_registry_token_expire_delay` | integer | no | Container Registry token duration in minutes. | | `default_artifacts_expire_in` | string | no | Set the default expiration time for each job's artifacts. | | `default_branch_protection` | integer | no | Determine if developers can push to master. Can take: `0` _(not protected, both developers and maintainers can push new commits, force push, or delete the branch)_, `1` _(partially protected, developers and maintainers can push new commits, but cannot force push or delete the branch)_ or `2` _(fully protected, developers cannot push new commits, but maintainers can; no-one can force push or delete the branch)_ as a parameter. Default is `2`. | @@ -212,8 +210,6 @@ are listed in the descriptions of the relevant settings. | `restricted_visibility_levels` | array of strings | no | Selected levels cannot be used by non-admin users for groups, projects or snippets. Can take `private`, `internal` and `public` as a parameter. Default is `null` which means there is no restriction. | | `rsa_key_restriction` | integer | no | The minimum allowed bit length of an uploaded RSA key. Default is `0` (no restriction). `-1` disables RSA keys. | | `send_user_confirmation_email` | boolean | no | Send confirmation email on sign-up. | -| `sentry_dsn` | string | required by: `sentry_enabled` | Sentry Data Source Name. | -| `sentry_enabled` | boolean | no | (**If enabled, requires:** `sentry_dsn`) Sentry is an error reporting and logging tool which is currently not shipped with GitLab, available at <https://sentry.io>. | | `session_expire_delay` | integer | no | Session duration in minutes. GitLab restart is required to apply changes | | `shared_runners_enabled` | boolean | no | (**If enabled, requires:** `shared_runners_text`) Enable shared runners for new projects. | | `shared_runners_text` | string | required by: `shared_runners_enabled` | Shared runners text. | @@ -231,6 +227,7 @@ are listed in the descriptions of the relevant settings. | `throttle_unauthenticated_enabled` | boolean | no | (**If enabled, requires:** `throttle_unauthenticated_period_in_seconds` and `throttle_unauthenticated_requests_per_period`) Enable unauthenticated request rate limit. Helps reduce request volume (e.g. from crawlers or abusive bots). | | `throttle_unauthenticated_period_in_seconds` | integer | required by: `throttle_unauthenticated_enabled` | Rate limit period in seconds. | | `throttle_unauthenticated_requests_per_period` | integer | required by: `throttle_unauthenticated_enabled` | Max requests per period per IP. | +| `time_tracking_limit_to_hours` | boolean | no | Limit display of time tracking units to hours. Default is `false`. | | `two_factor_grace_period` | integer | required by: `require_two_factor_authentication` | Amount of time (in hours) that users are allowed to skip forced configuration of two-factor authentication. | | `unique_ips_limit_enabled` | boolean | no | (**If enabled, requires:** `unique_ips_limit_per_user` and `unique_ips_limit_time_window`) Limit sign in from multiple ips. | | `unique_ips_limit_per_user` | integer | required by: `unique_ips_limit_enabled` | Maximum number of ips per user. | diff --git a/doc/ci/README.md b/doc/ci/README.md index 1743c38eb46..da864a0b3cc 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -63,7 +63,7 @@ Once you're familiar with how GitLab CI/CD works, see the for all the attributes you can set and use. NOTE: **Note:** -GitLab CI/CD and [shared runners](runners/README.md#shared-specific-and-group-runners) are enabled in GitLab.com and available for all users, limited only to the [user's pipelines quota](../user/admin_area/settings/continuous_integration.md#extra-shared-runners-pipeline-minutes-quota). +GitLab CI/CD and [shared runners](runners/README.md#shared-specific-and-group-runners) are enabled in GitLab.com and available for all users, limited only to the [user's pipelines quota](../user/admin_area/settings/continuous_integration.md#extra-shared-runners-pipeline-minutes-quota-free-only). ## Configuration diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index b4c4bea6447..efdcaf5a6f5 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -205,7 +205,14 @@ An example project using this approach can be found here: <https://gitlab.com/gi ### Use Docker socket binding -The third approach is to bind-mount `/var/run/docker.sock` into the container so that docker is available in the context of that image. +The third approach is to bind-mount `/var/run/docker.sock` into the +container so that Docker is available in the context of that image. + +NOTE: **Note:** +If you bind the Docker socket [when using GitLab Runner 11.11 or +newer](https://gitlab.com/gitlab-org/gitlab-runner/merge_requests/1261), +you can no longer use `docker:dind` as a service because volume bindings +are done to the services as well, making these incompatible. In order to do that, follow the steps: diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index f6b84dca734..f752f942e24 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -478,7 +478,7 @@ To define which should be used, the GitLab Runner process reads the configuratio - A [variable](../variables/README.md#gitlab-cicd-environment-variables) in `.gitlab-ci.yml`. - A project's variables stored on the projects **Settings > CI/CD** page. - `DOCKER_AUTH_CONFIG` variable provided as environment variable in `config.toml` of the Runner. -- `config.json` file placed in `$HOME/docker` directory of the user running GitLab Runner process. +- `config.json` file placed in `$HOME/.docker` directory of the user running GitLab Runner process. If the `--user` flag is provided to run the GitLab Runner child processes as unprivileged user, the home directory of the main GitLab Runner process user will be used. @@ -489,6 +489,7 @@ it's provided as an environment variable. This is because GitLab Runnner uses ** runtime. ### Using statically-defined credentials + As an example, let's assume that you want to use the `registry.example.com:5000/private/image:latest` image which is private and requires you to login into a private container registry. @@ -566,7 +567,6 @@ for the Runner to match the `DOCKER_AUTH_CONFIG`. For example, if then the `DOCKER_AUTH_CONFIG` must also specify `registry.example.com:5000`. Specifying only `registry.example.com` will not work. - ### Using Credentials Store > Support for using Credentials Store was added in GitLab Runner 9.5. @@ -574,7 +574,7 @@ Specifying only `registry.example.com` will not work. To configure credentials store, follow these steps: 1. To use a credentials store, you need an external helper program to interact with a specific keychain or external store. -Make sure helper program is available in GitLab Runner `$PATH`. + Make sure helper program is available in GitLab Runner `$PATH`. 1. Make GitLab Runner use it. There are two ways to accomplish this. Either: - Create a diff --git a/doc/ci/examples/test-scala-application.md b/doc/ci/examples/test-scala-application.md index 0e33a1ba060..bd899240307 100644 --- a/doc/ci/examples/test-scala-application.md +++ b/doc/ci/examples/test-scala-application.md @@ -47,10 +47,10 @@ deploy: In the above configuration: - The `before_script` installs [SBT](http://www.scala-sbt.org/) and -displays the version that is being used. + displays the version that is being used. - The `test` stage executes SBT to compile and test the project. - - [sbt-scoverage](https://github.com/scoverage/sbt-scoverage) is used as an SBT -plugin to measure test coverage. + - [sbt-scoverage](https://github.com/scoverage/sbt-scoverage) is used as an SBT + plugin to measure test coverage. - The `deploy` stage automatically deploys the project to Heroku using dpl. You can use other versions of Scala and SBT by defining them in diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index df455857dee..1b50273eca2 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -27,8 +27,7 @@ CI/CD's pipelines. Using variables means no hardcoded values. ### Predefined environment variables -GitLab CI/CD has a default set of -[predefined variables](predefined_variables.md) +GitLab CI/CD has a [default set of predefined variables](predefined_variables.md) which can be used without any specification needed. You can call issues numbers, user names, branch names, pipeline and commit IDs, and much more. @@ -36,7 +35,7 @@ pipeline and commit IDs, and much more. Predefined environment variables are the ones that GitLab provides out of the box for the local environment of the Runner. -GitLab reads the .gitlab-ci.yml file, sends the information +GitLab reads the `.gitlab-ci.yml` file, sends the information to the Runner (which runs the script commands), under which the variables are exposed. @@ -44,6 +43,9 @@ For example, two jobs under the same pipeline can share the same `CI_PIPELINE_ID` variable, but each one has its own `CI_JOB_ID` variable. +NOTE: **Note:** +Find here the full [**predefined variables reference table**](predefined_variables.md). + ### Custom environment variables When your use case requires a specific variable, you can @@ -337,7 +339,7 @@ Group-level variables can be added by: 1. Navigating to your group's **Settings > CI/CD** page. 1. Inputing variable types, keys, and values in the **Variables** section. -Any variables of [subgroups](../../user/group/subgroups/index.md) will be inherited recursively. + Any variables of [subgroups](../../user/group/subgroups/index.md) will be inherited recursively. Once you set them, they will be available for all subsequent pipelines. @@ -480,7 +482,7 @@ Below you can find supported syntax reference: > Example: `$VARIABLE == "some value"` - > Example: `$VARIABLE != "some value"` _(added in 11.11)_ + > Example: `$VARIABLE != "some value"` (introduced in GitLab 11.11) You can use equality operator `==` or `!=` to compare a variable content to a string. We support both, double quotes and single quotes to define a string @@ -491,7 +493,7 @@ Below you can find supported syntax reference: > Example: `$VARIABLE == null` - > Example: `$VARIABLE != null` _(added in 11.11)_ + > Example: `$VARIABLE != null` (introduced in GitLab 11.11) It sometimes happens that you want to check whether a variable is defined or not. To do that, you can compare a variable to `null` keyword, like @@ -502,7 +504,7 @@ Below you can find supported syntax reference: > Example: `$VARIABLE == ""` - > Example: `$VARIABLE != ""` _(added in 11.11)_ + > Example: `$VARIABLE != ""` (introduced in GitLab 11.11) If you want to check whether a variable is defined, but is empty, you can simply compare it against an empty string, like `$VAR == ''` or non-empty @@ -512,7 +514,7 @@ Below you can find supported syntax reference: > Example: `$VARIABLE_1 == $VARIABLE_2` - > Example: `$VARIABLE_1 != $VARIABLE_2` _(added in 11.11)_ + > Example: `$VARIABLE_1 != $VARIABLE_2` (introduced in GitLab 11.11) It is possible to compare two variables. This is going to compare values of these variables. @@ -528,11 +530,11 @@ Below you can find supported syntax reference: `$STAGING` value needs to a string, with length higher than zero. Variable that contains only whitespace characters is not an empty variable. -1. Pattern matching _(added in 11.0)_ +1. Pattern matching (introduced in GitLab 11.0) > Example: `$VARIABLE =~ /^content.*/` - > Example: `$VARIABLE_1 !~ /^content.*/` _(added in 11.11)_ + > Example: `$VARIABLE_1 !~ /^content.*/` (introduced in GitLab 11.11) It is possible perform pattern matching against a variable and regular expression. Expression like this evaluates to truth if matches are found @@ -541,7 +543,7 @@ Below you can find supported syntax reference: Pattern matching is case-sensitive by default. Use `i` flag modifier, like `/pattern/i` to make a pattern case-insensitive. -1. Conjunction / Disjunction +1. Conjunction / Disjunction ([introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/27925) in GitLab 12.0) > Example: `$VARIABLE1 =~ /^content.*/ && $VARIABLE2 == "something"` diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md index 4655eec51de..7dbb9af2869 100644 --- a/doc/ci/variables/predefined_variables.md +++ b/doc/ci/variables/predefined_variables.md @@ -30,7 +30,7 @@ future GitLab releases.** | `CI_BUILDS_DIR` | all | 11.10 | Top-level directory where builds are executed. | | `CI_CONCURRENT_ID` | all | 11.10 | Unique ID of build execution within a single executor. | | `CI_CONCURRENT_PROJECT_ID` | all | 11.10 | Unique ID of build execution within a single executor and project. | -| `CI_COMMIT_BEFORE_SHA` | 11.2 | all | The previous latest commit present on a branch before a push request. | +| `CI_COMMIT_BEFORE_SHA` | 11.2 | all | The previous latest commit present on a branch before a push request. Only populated when there is a merge request associated with the pipeline. | | `CI_COMMIT_DESCRIPTION` | 10.8 | all | The description of the commit: the message without first line, if the title is shorter than 100 characters; full message in other case. | | `CI_COMMIT_MESSAGE` | 10.8 | all | The full commit message. | | `CI_COMMIT_REF_NAME` | 9.0 | all | The branch or tag name for which project is built | diff --git a/doc/development/README.md b/doc/development/README.md index af3207671e6..5df6ec5fd56 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -108,6 +108,7 @@ description: 'Learn how to contribute to GitLab.' - [Database Debugging and Troubleshooting](database_debugging.md) - [Query Count Limits](query_count_limits.md) - [Database helper modules](database_helpers.md) +- [Code comments](code_comments.md) ## Integration guides diff --git a/doc/development/api_graphql_styleguide.md b/doc/development/api_graphql_styleguide.md index f7f36901fe1..aeddad14995 100644 --- a/doc/development/api_graphql_styleguide.md +++ b/doc/development/api_graphql_styleguide.md @@ -198,9 +198,9 @@ abilities as in the Rails app. If the: - Currently authenticated user fails the authorization, the authorized -resource will be returned as `null`. + resource will be returned as `null`. - Resource is part of a collection, the collection will be filtered to -exclude the objects that the user's authorization checks failed against. + exclude the objects that the user's authorization checks failed against. TIP: **Tip:** Try to load only what the currently authenticated user is allowed to @@ -447,7 +447,7 @@ want to validate the abilities for. Alternatively, we can add a `find_object` method that will load the object on the mutation. This would allow you to use the -`authorized_find!` and `authorized_find!` helper methods. +`authorized_find!` helper method. When a user is not allowed to perform the action, or an object is not found, we should raise a @@ -496,4 +496,4 @@ it 'returns a successful response' do expect(response).to have_gitlab_http_status(:success) expect(graphql_mutation_response(:merge_request_set_wip)['errors']).to be_empty end -```
\ No newline at end of file +``` diff --git a/doc/development/architecture.md b/doc/development/architecture.md index a43a40152db..d14a760f972 100644 --- a/doc/development/architecture.md +++ b/doc/development/architecture.md @@ -141,13 +141,13 @@ Component statuses are linked to configuration documentation for each component. | [Consul](#consul) | Database node discovery, failover | [⚙][consul-omnibus] | [❌][consul-charts] | [❌][consul-charts] | [✅](../user/gitlab_com/index.md#consul) | ❌ | ❌ | EE Only | | [GitLab self-monitoring: Prometheus](#prometheus) | Time-series database, metrics collection, and query service | [✅][prometheus-omnibus] | [✅][prometheus-charts] | [⚙][prometheus-charts] | [✅](../user/gitlab_com/index.md#prometheus) | ❌ | ❌ | CE & EE | | [GitLab self-monitoring: Alertmanager](#alertmanager) | Deduplicates, groups, and routes alerts from Prometheus | [⚙][alertmanager-omnibus] | [✅][alertmanager-charts] | [⚙][alertmanager-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | -| [GitLab self-monitoring: Grafana](#grafana) | Metrics dashboard | [⚙][grafana-omnibus] | [⤓][grafana-charts] | [⤓][grafana-charts] | [✅](https://dashboards.gitlab.com/d/RZmbBr7mk/gitlab-triage?refresh=30s) | ❌ | ❌ | CE & EE | +| [GitLab self-monitoring: Grafana](#grafana) | Metrics dashboard | [✅][grafana-omnibus] | [⤓][grafana-charts] | [⤓][grafana-charts] | [✅](https://dashboards.gitlab.com/d/RZmbBr7mk/gitlab-triage?refresh=30s) | ❌ | ❌ | CE & EE | | [GitLab self-monitoring: Sentry](#sentry) | Track errors generated by the GitLab instance | [⤓][sentry-omnibus] | [❌][sentry-charts] | [❌][sentry-charts] | [✅](https://about.gitlab.com/handbook/support/workflows/services/gitlab_com/500_errors.html#searching-sentry) | [⤓][gitlab-yml] | [⤓][gitlab-yml] | CE & EE | | [GitLab self-monitoring: Jaeger](#jaeger) | View traces generated by the GitLab instance | [❌][jaeger-omnibus] | [❌][jaeger-charts] | [❌][jaeger-charts] | [❌](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/4104) | [⤓][jaeger-source] | [⚙][jaeger-gdk] | CE & EE | | [Redis Exporter](#redis-exporter) | Prometheus endpoint with Redis metrics | [✅][redis-exporter-omnibus] | [✅][redis-exporter-charts] | [✅][redis-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | | [Postgres Exporter](#postgres-exporter) | Prometheus endpoint with PostgreSQL metrics | [✅][postgres-exporter-omnibus] | [✅][postgres-exporter-charts] | [✅][postgres-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | | [PgBouncer Exporter](#pgbouncer-exporter) | Prometheus endpoint with PgBouncer metrics | [⚙][pgbouncer-exporter-omnibus] | [❌][pgbouncer-exporter-charts] | [❌][pgbouncer-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | -| [GitLab Monitor](#gitlab-monitor) | Generates a variety of GitLab metrics | [✅][gitlab-monitor-omnibus] | [❌][gitab-monitor-charts] | [❌][gitab-monitor-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | +| [GitLab Monitor](#gitlab-monitor) | Generates a variety of GitLab metrics | [✅][gitlab-monitor-omnibus] | [✅][gitab-monitor-charts] | [✅][gitab-monitor-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | | [Node Exporter](#node-exporter) | Prometheus endpoint with system metrics | [✅][node-exporter-omnibus] | [❌][node-exporter-charts] | [❌][node-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | | [Mattermost](#mattermost) | Open-source Slack alternative | [⚙][mattermost-omnibus] | [⤓][mattermost-charts] | [⤓][mattermost-charts] | [⤓](../user/project/integrations/mattermost.md) | ❌ | ❌ | CE & EE | | [MinIO](#minio) | Object storage service | [⤓][minio-omnibus] | [✅][minio-charts] | [✅][minio-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#storage-architecture) | ❌ | [⚙][minio-gdk] | CE & EE | @@ -681,7 +681,7 @@ We've also detailed [our architecture of GitLab.com](https://about.gitlab.com/ha [pgbouncer-exporter-omnibus]: ../administration/monitoring/prometheus/pgbouncer_exporter.md [pgbouncer-exporter-charts]: https://docs.gitlab.com/charts/installation/deployment.html#postgresql [gitlab-monitor-omnibus]: ../administration/monitoring/prometheus/gitlab_monitor_exporter.md -[gitab-monitor-charts]: https://gitlab.com/charts/gitlab/issues/319 +[gitab-monitor-charts]: https://docs.gitlab.com/charts/charts/gitlab/gitlab-monitor/index.html [node-exporter-omnibus]: ../administration/monitoring/prometheus/node_exporter.md [node-exporter-charts]: https://gitlab.com/charts/gitlab/issues/1332 [mattermost-omnibus]: https://docs.gitlab.com/omnibus/gitlab-mattermost/ diff --git a/doc/development/code_comments.md b/doc/development/code_comments.md new file mode 100644 index 00000000000..36962eb46d4 --- /dev/null +++ b/doc/development/code_comments.md @@ -0,0 +1,14 @@ +# Code comments + +Whenever you add comment to the code that is expected to be addressed at any time +in future, please create a technical debt issue for it. Then put a link to it +to the code comment you've created. This will allow other developers to quickly +check if a comment is still relevant and what needs to be done to address it. + +Examples: + +```rb +# Deprecated scope until code_owner column has been migrated to rule_type. +# To be removed with https://gitlab.com/gitlab-org/gitlab-ee/issues/11834. +scope :code_owner, -> { where(code_owner: true).or(where(rule_type: :code_owner)) } +``` diff --git a/doc/development/code_review.md b/doc/development/code_review.md index 29e2aa1a581..6123f9f845a 100644 --- a/doc/development/code_review.md +++ b/doc/development/code_review.md @@ -60,6 +60,8 @@ from teams other than your own. 1. If your merge request includes backend changes [^1], it must be **approved by a [backend maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab-ce_maintainers_backend)**. + 1. If your merge request includes database migrations or changes to expensive queries [^2], it must be + **approved by a [database maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab-ce_maintainers_database)**. 1. If your merge request includes frontend changes [^1], it must be **approved by a [frontend maintainer](https://about.gitlab.com/handbook/engineering/projects/#gitlab-ce_maintainers_frontend)**. 1. If your merge request includes UX changes [^1], it must be @@ -377,3 +379,4 @@ Largely based on the [thoughtbot code review guide]. [team]: https://about.gitlab.com/team/ [build handbook]: https://about.gitlab.com/handbook/build/handbook/build#how-to-work-with-build [^1]: Please note that specs other than JavaScript specs are considered backend code. +[^2]: We encourage you to seek guidance from a database maintainer if your merge request is potentially introducing expensive queries. It is most efficient to comment on the line of code in question with the SQL queries so they can give their advice. diff --git a/doc/development/contributing/issue_workflow.md b/doc/development/contributing/issue_workflow.md index 0396f7ebc45..3d36a7bf3b1 100644 --- a/doc/development/contributing/issue_workflow.md +++ b/doc/development/contributing/issue_workflow.md @@ -172,35 +172,35 @@ This label documents the planned timeline & urgency which is used to measure aga | ~P3 | Medium Priority | Within the next 3 releases (approx one quarter or 90 days) | | ~P4 | Low Priority | Anything outside the next 3 releases (more than one quarter or 120 days) | -If an issue seems to fall between two priority labels, assign it to the higher- -priority label. - ## Severity labels Severity labels help us clearly communicate the impact of a ~bug on users. - -| Label | Meaning | Impact on Functionality | Example | -|-------|-------------------|-------------------------------------------------------|---------| -| ~S1 | Blocker | Outage, broken feature with no workaround | Unable to create an issue. Data corruption/loss. Security breach. | -| ~S2 | Critical Severity | Broken Feature, workaround too complex & unacceptable | Can push commits, but only via the command line. | -| ~S3 | Major Severity | Broken Feature, workaround acceptable | Can create merge requests only from the Merge Requests page, not through the Issue. | -| ~S4 | Low Severity | Functionality inconvenience or cosmetic issue | Label colors are incorrect / not being displayed. | - -If an issue seems to fall between two severity labels, even taking the -[severity impact guidance](#severity-impact-guidance) into account, assign -it to the higher-severity label. - -### Severity impact guidance - -Severity levels can be applied further depending on the facet of the impact; e.g. Affected customers, GitLab.com availability, performance and etc. The below is a guideline. - -| Severity | Affected Customers/Users | GitLab.com Availability | Performance Degradation | -|----------|---------------------------------------------------------------------|----------------------------------------------------|------------------------------| -| ~S1 | >50% users affected (possible company extinction level event) | Significant impact on all of GitLab.com | | -| ~S2 | Many users or multiple paid customers affected (but not apocalyptic)| Significant impact on large portions of GitLab.com | Degradation is guaranteed to occur in the near future | -| ~S3 | A few users or a single paid customer affected | Limited impact on important portions of GitLab.com | Degradation is likely to occur in the near future | -| ~S4 | No paid users/customer affected, or expected to in the near future | Minor impact on GitLab.com | Degradation _may_ occur but it's not likely | - +There can be multiple facets of the impact. The below is a guideline. + +| Label | Meaning | Functionality | Affected Users | GitLab.com Availability | Performance Degradation | +|-------|-------------------|-------------------------------------------------------|----------------------------------|----------------------------------------------------|------------------------------| +| ~S1 | Blocker | Unusable feature with no workaround, user is blocked | Impacts 50% or more of users | Outage, Significant impact on all of GitLab.com | | +| ~S2 | Critical Severity | Broken Feature, workaround too complex & unacceptable | Impacts between 25%-50% of users | Significant impact on large portions of GitLab.com | Degradation is guaranteed to occur in the near future | +| ~S3 | Major Severity | Broken feature with an acceptable workaround | Impacts up to 25% of users | Limited impact on important portions of GitLab.com | Degradation is likely to occur in the near future | +| ~S4 | Low Severity | Functionality inconvenience or cosmetic issue | Impacts less than 5% of users | Minor impact on GitLab.com | Degradation _may_ occur but it's not likely | + +If a bug seems to fall between two severity labels, assign it to the higher-severity label. + +* Example(s) of ~S1 + * Data corruption/loss. + * Security breach. + * Unable to create an issue or merge request. + * Unable to add a comment or discussion to the issue or merge request. +* Example(s) of ~S2 + * Cannot submit changes through the web IDE but the commandline works. + * A status widget on the merge request page is not working but information can be seen in the test pipeline page. +* Example(s) of ~S3 + * Can create merge requests only from the Merge Requests list view, not from an Issue page. + * Status is not updated in real time and needs a page refresh. +* Example(s) of ~S4 + * Label colors are incorrect. + * UI elements are not fully aligned. + ## Label for community contributors Issues that are beneficial to our users, 'nice to haves', that we currently do diff --git a/doc/development/contributing/merge_request_workflow.md b/doc/development/contributing/merge_request_workflow.md index 8a4aa5dfa7f..6064f59ed10 100644 --- a/doc/development/contributing/merge_request_workflow.md +++ b/doc/development/contributing/merge_request_workflow.md @@ -126,16 +126,16 @@ When writing commit messages, please follow the guidelines below: - The commit subject must contain at least 3 words. - The commit subject should ideally contain up to 50 characters, -and must not be longer than 72 characters. + and must not be longer than 72 characters. - The commit subject must start with a capital letter. - The commit subject must not end with a period. - The commit subject and body must be separated by a blank line. - The commit body must not contain more than 72 characters per line. - Commits that change 30 or more lines across at least 3 files must -describe these changes in the commit body. + describe these changes in the commit body. - The commit subject or body must not contain Emojis. - Use issues and merge requests' full URLs instead of short references, -as they are displayed as plain text outside of GitLab. + as they are displayed as plain text outside of GitLab. - The merge request must not contain more than 10 commit messages. If the guidelines are not met, the MR will not pass the diff --git a/doc/development/documentation/index.md b/doc/development/documentation/index.md index c7fa40af930..418e58b22d5 100644 --- a/doc/development/documentation/index.md +++ b/doc/development/documentation/index.md @@ -76,10 +76,10 @@ After a given documentation path is aligned across CE and EE, all merge requests affecting that path must be submitted to CE, regardless of the content it has. This means that: -* For **EE-only docs changes**, you only have to submit a CE MR. -* For **EE-only features** that touch both the code and the docs, you have to submit -an EE MR containing all changes, and a CE MR containing only the docs changes -and without a changelog entry. +- For **EE-only docs changes**, you only have to submit a CE MR. +- For **EE-only features** that touch both the code and the docs, you have to submit + an EE MR containing all changes, and a CE MR containing only the docs changes + and without a changelog entry. This might seem like a duplicate effort, but it's only for the short term. A list of the already aligned docs can be found in diff --git a/doc/development/documentation/styleguide.md b/doc/development/documentation/styleguide.md index b8506a72666..ff6dc16d1a0 100644 --- a/doc/development/documentation/styleguide.md +++ b/doc/development/documentation/styleguide.md @@ -165,8 +165,8 @@ The table below shows what kind of documentation goes where. `doc/topics/topic-name/subtopic-name/index.md` when subtopics become necessary. General user- and admin- related documentation, should be placed accordingly. 1. The directories `/workflow/`, `/university/`, and `/articles/` have -been **deprecated** and the majority their docs have been moved to their correct location -in small iterations. + been **deprecated** and the majority their docs have been moved to their correct location + in small iterations. If you are unsure where a document or a content addition should live, this should not stop you from authoring and contributing. You can use your best judgment and @@ -409,11 +409,20 @@ To indicate the steps of navigation through the UI: ## Images - Place images in a separate directory named `img/` in the same directory where - the `.md` document that you're working on is located. Always prepend their - names with the name of the document that they will be included in. For - example, if there is a document called `twitter.md`, then a valid image name - could be `twitter_login_screen.png`. -- Images should have a specific, non-generic name that will differentiate and describe them properly. + the `.md` document that you're working on is located. +- Images should have a specific, non-generic name that will + differentiate and describe them properly. +- Always add to the end of the file name the GitLab release version + number corresponding to the release milestone the image was added to, + or corresponding to the release the screenshot was taken from, using the + format `image_name_vX_Y.png`. + ([Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/61027) in GitLab 12.1.) +- For example, for a screenshot taken from the pipelines page of + GitLab 11.1, a valid name is `pipelines_v11_1.png`. If you're + adding an illustration that does not include parts of the UI, + add the release number corresponding to the release the image + was added to. Example, for an MR added to 11.1's milestone, + a valid name for an illustration is `devops_diagram_v11_1.png`. - Keep all file names in lower case. - Consider using PNG images instead of JPEG. - Compress all images with <https://tinypng.com/> or similar tool. @@ -426,7 +435,7 @@ To indicate the steps of navigation through the UI: Inside the document: - The Markdown way of using an image inside a document is: - `` + `` - Always use a proper description for what the image is about. That way, when a browser fails to show the image, this text will be used as an alternative description. diff --git a/doc/development/ee_features.md b/doc/development/ee_features.md index cca52706ddc..6b416cf588c 100644 --- a/doc/development/ee_features.md +++ b/doc/development/ee_features.md @@ -909,11 +909,12 @@ import bundle from 'ee_else_ce/protected_branches/protected_branches_bundle.js'; See the frontend guide [performance section](fe_guide/performance.md) for information on managing page-specific javascript within EE. - ## Vue code in `assets/javascript` + ### script tag #### Child Component only used in EE + To separate Vue template differences we should [async import the components](https://vuejs.org/v2/guide/components-dynamic-async.html#Async-Components). Doing this allows for us to load the correct component in EE whilst in CE @@ -937,10 +938,12 @@ export default { ``` #### For JS code that is EE only, like props, computed properties, methods, etc, we will keep the current approach - - Since we [can't async load a mixin](https://github.com/vuejs/vue-loader/issues/418#issuecomment-254032223) we will use the [`ee_else_ce`](../development/ee_features.md#javascript-code-in-assetsjavascripts) alias we already have for webpack. + +- Since we [can't async load a mixin](https://github.com/vuejs/vue-loader/issues/418#issuecomment-254032223) we will use the [`ee_else_ce`](../development/ee_features.md#javascript-code-in-assetsjavascripts) alias we already have for webpack. - This means all the EE specific props, computed properties, methods, etc that are EE only should be in a mixin in the `ee/` folder and we need to create a CE counterpart of the mixin ##### Example: + ```javascript import mixin from 'ee_else_ce/path/mixin'; @@ -955,6 +958,7 @@ import mixin from 'ee_else_ce/path/mixin'; - You can see an MR with an example [here](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/9762) #### `template` tag + * **EE Child components** - Since we are using the async loading to check which component to load, we'd still use the component's name, check [this example](#child-component-only-used-in-ee). @@ -962,11 +966,12 @@ import mixin from 'ee_else_ce/path/mixin'; - For the templates that have extra HTML in EE we should move it into a new component and use the `ee_else_ce` dynamic import ### Non Vue Files + For regular JS files, the approach is similar. 1. We will keep using the [`ee_else_ce`](../development/ee_features.md#javascript-code-in-assetsjavascripts) helper, this means that EE only code should be inside the `ee/` folder. - 1. An EE file should be created with the EE only code, and it should extend the CE counterpart. - 1. For code inside functions that can't be extended, the code should be moved into a new file and we should use `ee_else_ce` helper: + 1. An EE file should be created with the EE only code, and it should extend the CE counterpart. + 1. For code inside functions that can't be extended, the code should be moved into a new file and we should use `ee_else_ce` helper: ##### Example: @@ -996,6 +1001,7 @@ to isolate such ruleset from rest of CE rules (along with adding comment describ to avoid conflicts during CE to EE merge. #### Bad + ```scss .section-body { .section-title { @@ -1011,6 +1017,7 @@ to avoid conflicts during CE to EE merge. ``` #### Good + ```scss .section-body { .section-title { diff --git a/doc/development/elasticsearch.md b/doc/development/elasticsearch.md index c8c70fa7216..94b3796f6e9 100644 --- a/doc/development/elasticsearch.md +++ b/doc/development/elasticsearch.md @@ -64,20 +64,25 @@ All indexing after the initial one is done via `ElasticIndexerWorker` (sidekiq j Search queries are generated by the concerns found in [ee/app/models/concerns/elastic](https://gitlab.com/gitlab-org/gitlab-ee/tree/master/ee/app/models/concerns/elastic). These concerns are also in charge of access control, and have been a historic source of security bugs so please pay close attention to them! ## Existing Analyzers/Tokenizers/Filters + These are all defined in https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/lib/elasticsearch/git/model.rb ### Analyzers + #### `path_analyzer` + Used when indexing blobs' paths. Uses the `path_tokenizer` and the `lowercase` and `asciifolding` filters. Please see the `path_tokenizer` explanation below for an example. #### `sha_analyzer` + Used in blobs and commits. Uses the `sha_tokenizer` and the `lowercase` and `asciifolding` filters. Please see the `sha_tokenizer` explanation later below for an example. #### `code_analyzer` + Used when indexing a blob's filename and content. Uses the `whitespace` tokenizer and the filters: `code`, `edgeNGram_filter`, `lowercase`, and `asciifolding` The `whitespace` tokenizer was selected in order to have more control over how tokens are split. For example the string `Foo::bar(4)` needs to generate tokens like `Foo` and `bar(4)` in order to be properly searched. @@ -85,15 +90,19 @@ The `whitespace` tokenizer was selected in order to have more control over how t Please see the `code` filter for an explanation on how tokens are split. #### `code_search_analyzer` + Not directly used for indexing, but rather used to transform a search input. Uses the `whitespace` tokenizer and the `lowercase` and `asciifolding` filters. ### Tokenizers + #### `sha_tokenizer` + This is a custom tokenizer that uses the [`edgeNGram` tokenizer](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/analysis-edgengram-tokenizer.html) to allow SHAs to be searcheable by any sub-set of it (minimum of 5 chars). -example: +Example: `240c29dc7e` becomes: + - `240c2` - `240c29` - `240c29d` @@ -102,21 +111,26 @@ example: - `240c29dc7e` #### `path_tokenizer` + This is a custom tokenizer that uses the [`path_hierarchy` tokenizer](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/analysis-pathhierarchy-tokenizer.html) with `reverse: true` in order to allow searches to find paths no matter how much or how little of the path is given as input. -example: +Example: `'/some/path/application.js'` becomes: + - `'/some/path/application.js'` - `'some/path/application.js'` - `'path/application.js'` - `'application.js'` ### Filters + #### `code` -Uses a [Pattern Capture token filter](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/analysis-pattern-capture-tokenfilter.html) to split tokens into more easily searched versions of themselves. + +Uses a [Pattern Capture token filter](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/analysis-pattern-capture-tokenfilter.html) to split tokens into more easily searched versions of themselves. Patterns: + - `"(\\p{Ll}+|\\p{Lu}\\p{Ll}+|\\p{Lu}+)"`: captures CamelCased and lowedCameCased strings as separate tokens - `"(\\d+)"`: extracts digits - `"(?=([\\p{Lu}]+[\\p{L}]+))"`: captures CamelCased strings recursively. Ex: `ThisIsATest` => `[ThisIsATest, IsATest, ATest, Test]` @@ -126,6 +140,7 @@ Patterns: - `'\/?([^\/]+)(?=\/|\b)'`: separate path terms `like/this/one` #### `edgeNGram_filter` + Uses an [Edge NGram token filter](https://www.elastic.co/guide/en/elasticsearch/reference/5.5/analysis-edgengram-tokenfilter.html) to allow inputs with only parts of a token to find the token. For example it would turn `glasses` into permutations starting with `gl` and ending with `glasses`, which would allow a search for "`glass`" to find the original token `glasses` ## Gotchas @@ -140,13 +155,13 @@ Uses an [Edge NGram token filter](https://www.elastic.co/guide/en/elasticsearch/ You might get an error such as ``` -[2018-10-31T15:54:19,762][WARN ][o.e.c.r.a.DiskThresholdMonitor] [pval5Ct] - flood stage disk watermark [95%] exceeded on - [pval5Ct7SieH90t5MykM5w][pval5Ct][/usr/local/var/lib/elasticsearch/nodes/0] free: 56.2gb[3%], +[2018-10-31T15:54:19,762][WARN ][o.e.c.r.a.DiskThresholdMonitor] [pval5Ct] + flood stage disk watermark [95%] exceeded on + [pval5Ct7SieH90t5MykM5w][pval5Ct][/usr/local/var/lib/elasticsearch/nodes/0] free: 56.2gb[3%], all indices on this node will be marked read-only ``` -This is because you've exceeded the disk space threshold - it thinks you don't have enough disk space left, based on the default 95% threshold. +This is because you've exceeded the disk space threshold - it thinks you don't have enough disk space left, based on the default 95% threshold. In addition, the `read_only_allow_delete` setting will be set to `true`. It will block indexing, `forcemerge`, etc @@ -158,16 +173,16 @@ Add this to your `elasticsearch.yml` file: ``` # turn off the disk allocator -cluster.routing.allocation.disk.threshold_enabled: false +cluster.routing.allocation.disk.threshold_enabled: false ``` _or_ ``` # set your own limits -cluster.routing.allocation.disk.threshold_enabled: true +cluster.routing.allocation.disk.threshold_enabled: true cluster.routing.allocation.disk.watermark.flood_stage: 5gb # ES 6.x only -cluster.routing.allocation.disk.watermark.low: 15gb +cluster.routing.allocation.disk.watermark.low: 15gb cluster.routing.allocation.disk.watermark.high: 10gb ``` diff --git a/doc/development/geo.md b/doc/development/geo.md index 6e59fab34c7..a10f13b069f 100644 --- a/doc/development/geo.md +++ b/doc/development/geo.md @@ -14,10 +14,10 @@ Geo handles replication for different components: - [Database](#database-replication): includes the entire application, except cache and jobs. - [Git repositories](#repository-replication): includes both projects and wikis. - [Uploaded blobs](#uploads-replication): includes anything from images attached on issues -to raw logs and assets from CI. + to raw logs and assets from CI. With the exception of the Database replication, on a *secondary* node, everything is coordinated -by the [Geo Log Cursor](#geo-log-cursor). +by the [Geo Log Cursor](#geo-log-cursor). ### Geo Log Cursor daemon @@ -31,8 +31,8 @@ picks the event up and schedules a `Geo::ProjectSyncWorker` job which will use the `Geo::RepositorySyncService` and `Geo::WikiSyncService` classes to update the repository and the wiki respectively. -The Geo Log Cursor daemon can operate in High Availability mode automatically. -The daemon will try to acquire a lock from time to time and once acquired, it +The Geo Log Cursor daemon can operate in High Availability mode automatically. +The daemon will try to acquire a lock from time to time and once acquired, it will behave as the *active* daemon. Any additional running daemons on the same node, will be in standby @@ -164,20 +164,20 @@ The Git Push Proxy exists as a functionality built inside the `gitlab-shell` com It is active on a **secondary** node only. It allows the user that has cloned a repository from the secondary node to push to the same URL. -Git `push` requests directed to a **secondary** node will be sent over to the **primary** node, +Git `push` requests directed to a **secondary** node will be sent over to the **primary** node, while `pull` requests will continue to be served by the **secondary** node for maximum efficiency. HTTPS and SSH requests are handled differently: - With HTTPS, we will give the user a `HTTP 302 Redirect` pointing to the project on the **primary** node. -The git client is wise enough to understand that status code and process the redirection. + The git client is wise enough to understand that status code and process the redirection. - With SSH, because there is no equivalent way to perform a redirect, we have to proxy the request. -This is done inside [`gitlab-shell`](https://gitlab.com/gitlab-org/gitlab-shell), by first translating the request -to the HTTP protocol, and then proxying it to the **primary** node. + This is done inside [`gitlab-shell`](https://gitlab.com/gitlab-org/gitlab-shell), by first translating the request + to the HTTP protocol, and then proxying it to the **primary** node. -The [`gitlab-shell`](https://gitlab.com/gitlab-org/gitlab-shell) daemon knows when to proxy based on the response -from `/api/v4/allowed`. A special `HTTP 300` status code is returned and we execute a "custom action", -specified in the response body. The response contains additional data that allows the proxied `push` operation +The [`gitlab-shell`](https://gitlab.com/gitlab-org/gitlab-shell) daemon knows when to proxy based on the response +from `/api/v4/allowed`. A special `HTTP 300` status code is returned and we execute a "custom action", +specified in the response body. The response contains additional data that allows the proxied `push` operation to happen on the **primary** node. ## Using the Tracking Database @@ -229,17 +229,17 @@ named `gitlab_secondary`. This configuration exists within the database's user context only. To access the `gitlab_secondary`, GitLab needs to use the same database user that had previously been configured. -The Geo Tracking Database accesses the readonly database replica via FDW as a regular user, -limited by its own restrictions. The credentials are configured as a -`USER MAPPING` associated with the `SERVER` mapped previously +The Geo Tracking Database accesses the readonly database replica via FDW as a regular user, +limited by its own restrictions. The credentials are configured as a +`USER MAPPING` associated with the `SERVER` mapped previously (`gitlab_secondary`). FDW configuration and credentials definition are managed automatically by the -Omnibus GitLab `gitlab-ctl reconfigure` command. +Omnibus GitLab `gitlab-ctl reconfigure` command. #### Refeshing the Foreign Tables -Whenever a new Geo node is configured or the database schema changes on the +Whenever a new Geo node is configured or the database schema changes on the **primary** node, you must refresh the foreign tables on the **secondary** node by running the following: @@ -279,11 +279,11 @@ on the Tracking Database: SELECT project_registry.* FROM project_registry JOIN gitlab_secondary.projects - ON (project_registry.project_id = gitlab_secondary.projects.id + ON (project_registry.project_id = gitlab_secondary.projects.id AND gitlab_secondary.projects.archived IS FALSE) ``` -At the ActiveRecord level, we have additional Models that represent the +At the ActiveRecord level, we have additional Models that represent the foreign tables. They must be mapped in a slightly different way, and they are read-only. Check the existing FDW models in `ee/app/models/geo/fdw` for reference. diff --git a/doc/development/profiling.md b/doc/development/profiling.md index b2f3a105b23..795523b82aa 100644 --- a/doc/development/profiling.md +++ b/doc/development/profiling.md @@ -38,10 +38,6 @@ For routes that require authorization you will need to provide a user to Gitlab::Profiler.profile('/gitlab-org/gitlab-test', user: User.first) ``` -The user you provide will need to have a [personal access -token](https://docs.gitlab.com/ce/user/profile/personal_access_tokens.html) in -the GitLab instance. - Passing a `logger:` keyword argument to `Gitlab::Profiler.profile` will send ActiveRecord and ActionController log output to that logger. Further options are documented with the method source. diff --git a/doc/development/reusing_abstractions.md b/doc/development/reusing_abstractions.md index 01cedf734fb..59da02ed6fd 100644 --- a/doc/development/reusing_abstractions.md +++ b/doc/development/reusing_abstractions.md @@ -127,6 +127,42 @@ Everything in `lib/api`. Everything that resides in `app/services`. +#### ServiceResponse + +Service classes usually have an `execute` method, which can return a +`ServiceResponse`. You can use `ServiceResponse.success` and +`ServiceResponse.error` to return a response in `execute` method. + +In a successful case: + +``` ruby +response = ServiceResponse.success(message: 'Branch was deleted') + +response.success? # => true +response.error? # => false +response.status # => :success +response.message # => 'Branch was deleted' +``` + +In a failed case: + +``` ruby +response = ServiceResponse.error(message: 'Unsupported operation') + +response.success? # => false +response.error? # => true +response.status # => :error +response.message # => 'Unsupported operation' +``` + +An additional payload can also be attached: + +``` ruby +response = ServiceResponse.success(payload: { issue: issue }) + +response.payload[:issue] # => issue +``` + ### Finders Everything in `app/finders`, typically used for retrieving data from a database. diff --git a/doc/development/testing_guide/end_to_end/dynamic_element_validation.md b/doc/development/testing_guide/end_to_end/dynamic_element_validation.md index f7b3ca8bc89..aec0a3ede5a 100644 --- a/doc/development/testing_guide/end_to_end/dynamic_element_validation.md +++ b/doc/development/testing_guide/end_to_end/dynamic_element_validation.md @@ -5,8 +5,8 @@ We devised a solution to solve common test automation problems such as the dread Other problems that dynamic element validations solve are... - When we perform an action with the mouse, we expect something to occur. -- When our test is navigating to (or from) a page, we ensure that we are on the page we expect before -test continuation. +- When our test is navigating to (or from) a page, we ensure that we are on the page we expect before + test continuation. ## How it works @@ -19,7 +19,7 @@ We interpret user actions on the page to have some sort of effect. These actions When a page is navigated to, there are elements that will always appear on the page unconditionally. -Dynamic element validation is instituted when using +Dynamic element validation is instituted when using ```ruby Runtime::Browser.visit(:gitlab, Some::Page) @@ -27,7 +27,7 @@ Runtime::Browser.visit(:gitlab, Some::Page) ### Clicks -When we perform a click within our tests, we expect something to occur. That something could be a component to now +When we perform a click within our tests, we expect something to occur. That something could be a component to now appear on the webpage, or the test to navigate away from the page entirely. Dynamic element validation is instituted when using @@ -71,7 +71,7 @@ class MyPage < Page::Base element :another_element, required: true element :conditional_element end - + def open_layer click_element :my_element, Layer::MyLayer end @@ -95,7 +95,7 @@ execute_stuff ``` will invoke GitLab QA to scan `MyPage` for `my_element` and `another_element` to be on the page before continuing to -`execute_stuff` +`execute_stuff` ### Clicking diff --git a/doc/development/testing_guide/end_to_end/index.md b/doc/development/testing_guide/end_to_end/index.md index afd81ff00b2..527cd350633 100644 --- a/doc/development/testing_guide/end_to_end/index.md +++ b/doc/development/testing_guide/end_to_end/index.md @@ -126,6 +126,18 @@ See [Review Apps][review-apps] for more details about Review Apps. [helm-chart]: https://gitlab.com/charts/gitlab/ [cng]: https://gitlab.com/gitlab-org/build/CNG +## How do I run the tests? + +There are two main options for running the tests. If you simply want to run the +existing tests against a live GitLab instance or against a pre-built docker image +you can use the [GitLab QA orchestrator][gitlab-qa-readme]. See also [examples +of the test scenarios you can run via the orchestrator](https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/what_tests_can_be_run.md#examples). + +On the other hand, if you would like to run against a local development GitLab +environment, you can use the [GitLab Development Kit (GDK)](https://gitlab.com/gitlab-org/gitlab-development-kit/). +Please refer to the instructions in the [QA README](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa/README.md#how-can-i-use-it) +and the section below. + ## How do I write tests? In order to write new tests, you first need to learn more about GitLab QA diff --git a/doc/development/testing_guide/end_to_end/page_objects.md b/doc/development/testing_guide/end_to_end/page_objects.md index 73e1fd862c1..05cb03eb4bd 100644 --- a/doc/development/testing_guide/end_to_end/page_objects.md +++ b/doc/development/testing_guide/end_to_end/page_objects.md @@ -82,7 +82,7 @@ module Page end # ... - end + end end end ``` @@ -134,7 +134,7 @@ for each element defined. In our case, `qa-login-field`, `qa-password-field` and `qa-sign-in-button` -**app/views/my/view.html.haml** +**app/views/my/view.html.haml** ```haml = f.text_field :login, class: "form-control top qa-login-field", autofocus: "autofocus", autocapitalize: "off", autocorrect: "off", required: true, title: "This field is required." @@ -146,7 +146,7 @@ Things to note: - The CSS class must be `kebab-cased` (separated with hyphens "`-`") - If the element appears on the page unconditionally, add `required: true` to the element. See -[Dynamic element validation](dynamic_element_validation.md) + [Dynamic element validation](dynamic_element_validation.md) ## Running the test locally diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md index 4c9d1684c00..28ebb6f0f64 100644 --- a/doc/development/testing_guide/frontend_testing.md +++ b/doc/development/testing_guide/frontend_testing.md @@ -27,14 +27,30 @@ we need to solve before being able to use Jest for all our needs. ### Differences to Karma - Jest runs in a Node.js environment, not in a browser. Support for running Jest tests in a browser [is planned](https://gitlab.com/gitlab-org/gitlab-ce/issues/58205). -- Because Jest runs in a Node.js environment, it uses [jsdom](https://github.com/jsdom/jsdom) by default. +- Because Jest runs in a Node.js environment, it uses [jsdom](https://github.com/jsdom/jsdom) by default. See also its [limitations](#limitations-of-jsdom) below. +- Jest does not have access to Webpack loaders or aliases. + The aliases used by Jest are defined in its [own config](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/jest.config.js). - All calls to `setTimeout` and `setInterval` are mocked away. See also [Jest Timer Mocks](https://jestjs.io/docs/en/timer-mocks). - `rewire` is not required because Jest supports mocking modules. See also [Manual Mocks](https://jestjs.io/docs/en/manual-mocks). +- No [context object](https://jasmine.github.io/tutorials/your_first_suite#section-The_%3Ccode%3Ethis%3C/code%3E_keyword) is passed to tests in Jest. + This means sharing `this.something` between `beforeEach()` and `it()` for example does not work. + Instead you should declare shared variables in the context that they are needed (via `const` / `let`). - The following will cause tests to fail in Jest: - Unmocked requests. - Unhandled Promise rejections. - Calls to `console.warn`, including warnings from libraries like Vue. +### Limitations of jsdom + +As mentioned [above](#differences-to-karma), Jest uses jsdom instead of a browser for running tests. +This comes with a number of limitations, namely: + +- [No scrolling support](https://github.com/jsdom/jsdom/blob/15.1.1/lib/jsdom/browser/Window.js#L623-L625) +- [No element sizes or positions](https://github.com/jsdom/jsdom/blob/15.1.1/lib/jsdom/living/nodes/Element-impl.js#L334-L371) +- [No layout engine](https://github.com/jsdom/jsdom/issues/1322) in general + +See also the issue for [support running Jest tests in browsers](https://gitlab.com/gitlab-org/gitlab-ce/issues/58205). + ### Debugging Jest tests Running `yarn jest-debug` will run Jest in debug mode, allowing you to debug/inspect as described in the [Jest docs](https://jestjs.io/docs/en/troubleshooting#tests-are-failing-and-you-don-t-know-why). diff --git a/doc/install/requirements.md b/doc/install/requirements.md index 92122fca7dd..68c1bcbc801 100644 --- a/doc/install/requirements.md +++ b/doc/install/requirements.md @@ -9,7 +9,7 @@ as the hardware requirements that are needed to install and use GitLab. ## Operating Systems -### Supported Unix distributions +### Supported Linux distributions - Ubuntu - Debian @@ -21,7 +21,7 @@ as the hardware requirements that are needed to install and use GitLab. For the installations options, see [the main installation page](README.md). -### Unsupported Unix distributions +### Unsupported Linux distributions and Unix-like operating systems - Arch Linux - Fedora @@ -29,13 +29,13 @@ For the installations options, see [the main installation page](README.md). - Gentoo - macOS -On the above unsupported distributions is still possible to install GitLab yourself. +Installation of GitLab on these operating systems is possible, but not supported. Please see the [installation from source guide](installation.md) and the [installation guides](https://about.gitlab.com/installation/) for more information. -### Non-Unix operating systems such as Windows +### Microsoft Windows -GitLab is developed for Unix operating systems. -It does **not** run on Windows, and we have no plans to support it in the near future. For the latest development status view this [issue](https://gitlab.com/gitlab-org/gitlab-ce/issues/46567). +GitLab is developed for Linux-based operating systems. +It does **not** run on Microsoft Windows, and we have no plans to support it in the near future. For the latest development status view this [issue](https://gitlab.com/gitlab-org/gitlab-ce/issues/46567). Please consider using a virtual machine to run GitLab. ## Ruby versions diff --git a/doc/integration/jenkins.md b/doc/integration/jenkins.md index 5950737b964..e6496ae3a2e 100644 --- a/doc/integration/jenkins.md +++ b/doc/integration/jenkins.md @@ -25,22 +25,22 @@ and [Migrating from Jenkins to GitLab](https://www.youtube.com/watch?v=RlEVGOpYF ## Use cases - Suppose you are new to GitLab, and want to keep using Jenkins until you prepare -your projects to build with [GitLab CI/CD](../ci/README.md). You set up the -integration between GitLab and Jenkins, then you migrate to GitLab CI later. While -you organize yourself and your team to onboard GitLab, you keep your pipelines -running with Jenkins, but view the results in your project's repository in GitLab. + your projects to build with [GitLab CI/CD](../ci/README.md). You set up the + integration between GitLab and Jenkins, then you migrate to GitLab CI later. While + you organize yourself and your team to onboard GitLab, you keep your pipelines + running with Jenkins, but view the results in your project's repository in GitLab. - Your team uses [Jenkins Plugins](https://plugins.jenkins.io/) for other proceedings, -therefore, you opt for keep using Jenkins to build your apps. Show the results of your -pipelines directly in GitLab. + therefore, you opt for keep using Jenkins to build your apps. Show the results of your + pipelines directly in GitLab. For a real use case, read the blog post [Continuous integration: From Jenkins to GitLab using Docker](https://about.gitlab.com/2017/07/27/docker-my-precious/). ## Requirements -* [Jenkins GitLab Plugin](https://wiki.jenkins.io/display/JENKINS/GitLab+Plugin) -* [Jenkins Git Plugin](https://wiki.jenkins.io/display/JENKINS/Git+Plugin) -* Git clone access for Jenkins from the GitLab repository -* GitLab API access to report build status +- [Jenkins GitLab Plugin](https://wiki.jenkins.io/display/JENKINS/GitLab+Plugin) +- [Jenkins Git Plugin](https://wiki.jenkins.io/display/JENKINS/Git+Plugin) +- Git clone access for Jenkins from the GitLab repository +- GitLab API access to report build status ## Configure GitLab users @@ -65,7 +65,7 @@ Go to Manage Jenkins -> Configure System and scroll down to the 'GitLab' section Enter the GitLab server URL in the 'GitLab host URL' field and paste the API token copied earlier in the 'API Token' field. -For more information, see GitLab Plugin documentation about +For more information, see GitLab Plugin documentation about [Jenkins-to-GitLab authentication](https://github.com/jenkinsci/gitlab-plugin#jenkins-to-gitlab-authentication)  @@ -76,8 +76,8 @@ Follow the GitLab Plugin documentation about [Jenkins Job Configuration](https:/ NOTE: **Note:** Be sure to include the steps about [Build status configuration](https://github.com/jenkinsci/gitlab-plugin#build-status-configuration). -The 'Publish build status to GitLab' post-build step is required to view -Jenkins build status in GitLab Merge Requests. +The 'Publish build status to GitLab' post-build step is required to view +Jenkins build status in GitLab Merge Requests. ## Configure a GitLab project @@ -114,21 +114,21 @@ and storing build status for Commits and Merge Requests. All steps are implemented using AJAX requests on the merge request page. 1. In order to display the build status in a merge request you must create a project service in GitLab. -2. Your project service will do a (JSON) query to a URL of the CI tool with the SHA1 of the commit. -3. The project service builds this URL and payload based on project service settings and knowledge of the CI tool. -4. The response is parsed to give a response in GitLab (success/failed/pending). +1. Your project service will do a (JSON) query to a URL of the CI tool with the SHA1 of the commit. +1. The project service builds this URL and payload based on project service settings and knowledge of the CI tool. +1. The response is parsed to give a response in GitLab (success/failed/pending). ## Troubleshooting ### Error in merge requests - "Could not connect to the CI server" This integration relies on Jenkins reporting the build status back to GitLab via -the [Commit Status API](../api/commits.md#commit-status). +the [Commit Status API](../api/commits.md#commit-status). The error 'Could not connect to the CI server' usually means that GitLab did not receive a build status update via the API. Either Jenkins was not properly -configured or there was an error reporting the status via the API. +configured or there was an error reporting the status via the API. 1. [Configure the Jenkins server](#configure-the-jenkins-server) for GitLab API access -2. [Configure a Jenkins project](#configure-a-jenkins-project), including the - 'Publish build status to GitLab' post-build action. +1. [Configure a Jenkins project](#configure-a-jenkins-project), including the + 'Publish build status to GitLab' post-build action. diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index 764916ca82d..c7aa22b11f8 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -936,5 +936,36 @@ A similar strategy can be employed for the remaining features - by removing the data that cannot be decrypted, GitLab can be brought back into working order, and the lost data can be manually replaced. +### Container Registry push failures after restoring from a backup + +If you use the [Container Registry](../user/project/container_registry.md), you +may see pushes to the registry fail after restoring your backup on an Omnibus +GitLab instance after restoring the registry data. + +These failures will mention permission issues in the registry logs, like: + +``` +level=error +msg="response completed with error" +err.code=unknown +err.detail="filesystem: mkdir /var/opt/gitlab/gitlab-rails/shared/registry/docker/registry/v2/repositories/...: permission denied" +err.message="unknown error" +``` + +This is caused by the restore being run as the unprivileged user `git` which was +unable to assign the correct ownership to the registry files during the restore +([issue 62759](https://gitlab.com/gitlab-org/gitlab-ce/issues/62759 "Incorrect permissions on registry filesystem after restore")). + +To get your registry working again: + +```bash +sudo chown -R registry:registry /var/opt/gitlab/gitlab-rails/shared/registry/docker +``` + +NOTE: **Note:** +If you have changed the default filesystem location for the registry, you will +want to run the chown against your custom location instead of +`/var/opt/gitlab/gitlab-rails/shared/registry/docker`. + [reconfigure GitLab]: ../administration/restart_gitlab.md#omnibus-gitlab-reconfigure [restart GitLab]: ../administration/restart_gitlab.md#installations-from-source diff --git a/doc/subscriptions/index.md b/doc/subscriptions/index.md index 37051f6b10f..1e9c53b9811 100644 --- a/doc/subscriptions/index.md +++ b/doc/subscriptions/index.md @@ -14,7 +14,7 @@ Learn how GitLab helps you in the stages of the DevOps lifecycle by learning mor ### Self-managed: Install GitLab -Take a look at [installing GitLab](https://about.gitlab.com/install/) and our [administrator documentation](../administration/index.md). Then, follow the instructions below under [Your subscription](#your-subscription) to apply your license file. +Take a look at [installing GitLab](https://about.gitlab.com/install/) and our [administrator documentation](../administration/index.md). Then, follow the instructions below under [Your subscription](#your-subscription) to apply your license file. ### GitLab.com: Create a user and group @@ -74,11 +74,11 @@ Please note that you need to be a group owner to associate a group to your subsc To see the status of your GitLab.com subscription, you can click on the Billings section of the relevant namespace: -* For individuals, this is located at https://gitlab.com/profile/billings under -in your Settings, -* For groups, this is located under the group's Settings dropdown, under Billing. +- For individuals, this is located at https://gitlab.com/profile/billings under + in your Settings, +- For groups, this is located under the group's Settings dropdown, under Billing. -For groups, you can see details of your subscription - including your current +For groups, you can see details of your subscription - including your current plan - in the included table:  @@ -86,11 +86,11 @@ plan - in the included table: | Field | Description | | ------ | ------ | | Seats in subscription | If this is a paid plan, this represents the number of seats you've paid to support in your group. | -| Seats currently in use | The number of active seats currently in use. | -| Max seats used | The highest number of seats you've used. If this exceeds the seats in subscription, you may owe an additional fee for the additional users. | -| Seats owed | If your max seats used exceeds the seats in your subscription, you'll owe an additional fee for the users you've added. | -| Subscription start date | The date your subscription started. If this is for a Free plan, this is the date you transitioned off your group's paid plan. | -| Subscription end date | The date your current subscription will end. This does not apply to Free plans. | +| Seats currently in use | The number of active seats currently in use. | +| Max seats used | The highest number of seats you've used. If this exceeds the seats in subscription, you may owe an additional fee for the additional users. | +| Seats owed | If your max seats used exceeds the seats in your subscription, you'll owe an additional fee for the users you've added. | +| Subscription start date | The date your subscription started. If this is for a Free plan, this is the date you transitioned off your group's paid plan. | +| Subscription end date | The date your current subscription will end. This does not apply to Free plans. | ### Subscription changes and your data diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index 702245b22a0..41d12128e51 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -723,7 +723,7 @@ also be customized, and you can easily use a [custom buildpack](#custom-buildpac | `AUTO_DEVOPS_CHART_REPOSITORY_USERNAME` | From Gitlab 11.11, this variable can be used to set a username to connect to the helm repository. Defaults to no credentials. (Also set AUTO_DEVOPS_CHART_REPOSITORY_PASSWORD) | | `AUTO_DEVOPS_CHART_REPOSITORY_PASSWORD` | From Gitlab 11.11, this variable can be used to set a password to connect to the helm repository. Defaults to no credentials. (Also set AUTO_DEVOPS_CHART_REPOSITORY_USERNAME) | | `REPLICAS` | The number of replicas to deploy; defaults to 1. | -| `PRODUCTION_REPLICAS` | The number of replicas to deploy in the production environment. This takes precedence over `REPLICAS`; defaults to 1. | +| `PRODUCTION_REPLICAS` | The number of replicas to deploy in the production environment. Takes precedence over `REPLICAS` and defaults to 1. For zero downtime upgrades, set to 2 or greater. | | `CANARY_REPLICAS` | The number of canary replicas to deploy for [Canary Deployments](../../user/project/canary_deployments.md); defaults to 1 | | `CANARY_PRODUCTION_REPLICAS` | The number of canary replicas to deploy for [Canary Deployments](../../user/project/canary_deployments.md) in the production environment. This takes precedence over `CANARY_REPLICAS`; defaults to 1 | | `ADDITIONAL_HOSTS` | Fully qualified domain names specified as a comma-separated list that are added to the ingress hosts. | diff --git a/doc/university/process/README.md b/doc/university/process/README.md index fdf6224f7f6..b278e02ccd5 100644 --- a/doc/university/process/README.md +++ b/doc/university/process/README.md @@ -9,11 +9,11 @@ title: University | Process # Suggesting improvements If you would like to teach a class or participate or help in any way please -submit a merge request and assign it to [Job](https://gitlab.com/u/JobV). +submit a merge request and assign it to [Job](https://gitlab.com/JobV). If you have suggestions for additional courses you would like to see, please submit a merge request to add an upcoming class, assign to -[Chad](https://gitlab.com/u/chadmalchow) and /cc [Job](https://gitlab.com/u/JobV). +[Chad](https://gitlab.com/chadmalchow) and /cc [Job](https://gitlab.com/JobV). ## Adding classes @@ -31,4 +31,4 @@ please submit a merge request to add an upcoming class, assign to 1. Please upload any video recordings to our Youtube channel. We prefer them to be public, if needed they can be unlisted but if so they should be linked from this page. -1. Please create a merge request and assign to [Erica](https://gitlab.com/u/Erica). +1. Please create a merge request and assign to [Erica](https://gitlab.com/Erica). diff --git a/doc/user/admin_area/index.md b/doc/user/admin_area/index.md index add5971e01c..fa60ee96cf7 100644 --- a/doc/user/admin_area/index.md +++ b/doc/user/admin_area/index.md @@ -186,7 +186,7 @@ the sort order to *Last Contacted* from the dropdown beside the search field. To search Runners' descriptions: 1. In the **Search or filter results...** field, type the description of the Runner you want to -find. + find. 1. Press Enter. You can also filter Runners by status, type, and tag. To filter: diff --git a/doc/user/admin_area/settings/continuous_integration.md b/doc/user/admin_area/settings/continuous_integration.md index fde7d1aeaf7..84596ff6a2c 100644 --- a/doc/user/admin_area/settings/continuous_integration.md +++ b/doc/user/admin_area/settings/continuous_integration.md @@ -94,14 +94,11 @@ a group in the **Usage Quotas** page available to the group page settings list.  +## Extra Shared Runners pipeline minutes quota **[FREE ONLY]** -## Extra Shared Runners pipeline minutes quota - -NOTE: **Note:** -Only available on GitLab.com. - -You can purchase additional CI minutes so your pipelines will not be blocked after you have -used all your CI minutes from your main quota. +If you're using GitLab.com, you can purchase additional CI minutes so your +pipelines will not be blocked after you have used all your CI minutes from your +main quota. In order to purchase additional minutes, you should follow these steps: @@ -110,27 +107,27 @@ In order to purchase additional minutes, you should follow these steps:  1. Locate the subscription card that is linked to your group on GitLab.com, -click on **Buy more CI minutes**, and complete the details about the transaction. + click on **Buy more CI minutes**, and complete the details about the transaction.  1. Once we have processed your payment, the extra CI minutes -will be synced to your Group and you can visualize it from the -**Group > Settings > Pipelines quota** page: + will be synced to your Group and you can visualize it from the + **Group > Settings > Pipelines quota** page:  Be aware that: 1. If you have purchased extra CI minutes before the purchase of a paid plan, -we will calculate a pro-rated charge for your paid plan. That means you may -be charged for less than one year since your subscription was previously -created with the extra CI minutes. + we will calculate a pro-rated charge for your paid plan. That means you may + be charged for less than one year since your subscription was previously + created with the extra CI minutes. 1. Once the extra CI minutes has been assigned to a Group they cannot be transferred -to a different Group. + to a different Group. 1. If you have some minutes used over your default quota, these minutes will -be deducted from your Additional Minutes quota immediately after your purchase of additional -minutes. + be deducted from your Additional Minutes quota immediately after your purchase of additional + minutes. ## What happens when my CI minutes quota run out diff --git a/doc/user/admin_area/settings/terms.md b/doc/user/admin_area/settings/terms.md index a5f8d05f662..a1bce5a6c69 100644 --- a/doc/user/admin_area/settings/terms.md +++ b/doc/user/admin_area/settings/terms.md @@ -17,7 +17,7 @@ To enforce acceptance of a Terms of Service and Privacy Policy: 1. Go to **Admin Area > Settings > General**. 1. Expand the **Terms of Service and Privacy Policy** section. 1. Check the **Require all users to accept Terms of Service and Privacy Policy when they access -GitLab.** checkbox. + GitLab.** checkbox. 1. Input the text of the **Terms of Service and Privacy Policy**. Markdown formatting can be used in this input box. 1. Click **Save changes**. 1. When you are presented with the **Terms of Service** statement, click **Accept terms**. diff --git a/doc/user/admin_area/settings/visibility_and_access_controls.md b/doc/user/admin_area/settings/visibility_and_access_controls.md index a1229484388..63879935fd8 100644 --- a/doc/user/admin_area/settings/visibility_and_access_controls.md +++ b/doc/user/admin_area/settings/visibility_and_access_controls.md @@ -4,12 +4,15 @@ type: reference # Visibility and access controls -GitLab allows admins to: +GitLab allows administrators to: - Control access and visibility to GitLab resources including branches and projects. - Select from which hosting sites code can be imported into GitLab. - Select the protocols permitted to access GitLab. - Enable or disable repository mirroring. +- Prevent non-administrators from deleting projects + ([introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/5615) in GitLab 12.0). + **[PREMIUM ONLY]** To access the visibility and access control options: diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md index a24374dff1d..4a2fb1d7190 100644 --- a/doc/user/application_security/container_scanning/index.md +++ b/doc/user/application_security/container_scanning/index.md @@ -40,6 +40,9 @@ To enable Container Scanning in your pipeline, you need: [`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html#running-privileged-containers-for-the-runners) executor running in privileged mode. If you're using the shared Runners on GitLab.com, this is enabled by default. +- Docker `18.09.03` or higher installed on the machine where the Runners are + running. If you're using the shared Runners on GitLab.com, this is already + the case. - To [build and push](../../../ci/docker/using_docker_build.md#container-registry-examples) your Docker image to your project's [Container Registry](../../project/container_registry.md). The name of the Docker image should match the following scheme: @@ -202,3 +205,20 @@ vulnerabilities in your groups and projects. Read more about the Once a vulnerability is found, you can interact with it. Read more on how to [interact with the vulnerabilities](../index.md#interacting-with-the-vulnerabilities). + +## Troubleshooting + +### docker: Error response from daemon: failed to copy xattrs + +When the GitLab Runner uses the Docker executor and NFS is used +(e.g., `/var/lib/docker` is on an NFS mount), Container Scanning might fail with +an error like the following: + +``` +docker: Error response from daemon: failed to copy xattrs: failed to set xattr "security.selinux" on /path/to/file: operation not supported. +``` + +This is a result of a bug in Docker which is now [fixed](https://github.com/containerd/continuity/pull/138 "fs: add WithAllowXAttrErrors CopyOpt"). +To prevent the error, ensure the Docker version that the Runner is using is +`18.09.03` or higher. For more information, see +[issue #10241](https://gitlab.com/gitlab-org/gitlab-ee/issues/10241 "Investigate why Container Scanning is not working with NFS mounts"). diff --git a/doc/user/application_security/dependency_scanning/analyzers.md b/doc/user/application_security/dependency_scanning/analyzers.md new file mode 100644 index 00000000000..937ded287e5 --- /dev/null +++ b/doc/user/application_security/dependency_scanning/analyzers.md @@ -0,0 +1,133 @@ +# Dependency Scanning Analyzers **[ULTIMATE]** + +Dependency Scanning relies on underlying third party tools that are wrapped into +what we call "Analyzers". An analyzer is a +[dedicated project](https://gitlab.com/gitlab-org/security-products/analyzers) +that wraps a particular tool to: + +- Expose its detection logic. +- Handle its execution. +- Convert its output to the common format. + +This is achieved by implementing the [common API](https://gitlab.com/gitlab-org/security-products/analyzers/common). + +Dependency Scanning supports the following official analyzers: + +- [`bundler-audit`](https://gitlab.com/gitlab-org/security-products/analyzers/bundler-audit) +- [`gemnasium`](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium) +- [`gemnasium-maven`](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven) +- [`gemnasium-python`](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python) +- [`retire.js`](https://gitlab.com/gitlab-org/security-products/analyzers/retire.js) + +The analyzers are published as Docker images that Dependency Scanning will use +to launch dedicated containers for each analysis. + +Dependency Scanning is pre-configured with a set of **default images** that are +maintained by GitLab, but users can also integrate their own **custom images**. + +## Official default analyzers + +Any custom change to the official analyzers can be achieved by using an +[environment variable in your `.gitlab-ci.yml`](index.md#customizing-the-dependency-scanning-settings). + +### Using a custom Docker mirror + +You can switch to a custom Docker registry that provides the official analyzer +images under a different prefix. For instance, the following instructs Dependency +Scanning to pull `my-docker-registry/gl-images/gemnasium` +instead of `registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium`. +In `.gitlab-ci.yml` define: + +```yaml +include: + template: Dependency-Scanning.gitlab-ci.yml + +variables: + DS_ANALYZER_IMAGE_PREFIX: my-docker-registry/gl-images +``` + +This configuration requires that your custom registry provides images for all +the official analyzers. + +### Selecting specific analyzers + +You can select the official analyzers you want to run. Here's how to enable +`bundler-audit` and `gemnasium` while disabling all the other default ones. +In `.gitlab-ci.yml` define: + +```yaml +include: + template: Dependency-Scanning.gitlab-ci.yml + +variables: + DS_DEFAULT_ANALYZERS: "bundler-audit,gemnasium" +``` + +`bundler-audit` runs first. When merging the reports, Dependency Scanning will +remove the duplicates and will keep the `bundler-audit` entries. + +### Disabling default analyzers + +Setting `DS_DEFAULT_ANALYZERS` to an empty string will disable all the official +default analyzers. In `.gitlab-ci.yml` define: + +```yaml +include: + template: Dependency-Scanning.gitlab-ci.yml + +variables: + DS_DEFAULT_ANALYZERS: "" +``` + +That's needed when one totally relies on [custom analyzers](#custom-analyzers). + +## Custom analyzers + +You can provide your own analyzers as a comma separated list of Docker images. +Here's how to add `analyzers/nugget` and `analyzers/perl` to the default images. +In `.gitlab-ci.yml` define: + +```yaml +include: + template: Dependency-Scanning.gitlab-ci.yml + +variables: + DS_ANALYZER_IMAGES: "my-docker-registry/analyzers/nugget,amy-docker-registry/nalyzers/perl" +``` + +The values must be the full path to the container registry images, +like what you would feed to the `docker pull` command. + +NOTE: **Note:** +This configuration doesn't benefit from the integrated detection step. Dependency +Scanning has to fetch and spawn each Docker image to establish whether the +custom analyzer can scan the source code. + +## Analyzers data + +The following table lists the data available for each official analyzer. + +| Property \ Tool | Gemnasium | bundler-audit | Retire.js | +|---------------------------------------|:------------------:|:------------------:|:------------------:| +| Severity | 𐄂 | ✓ | ✓ | +| Title | ✓ | ✓ | ✓ | +| File | ✓ | ⚠ | ✓ | +| Start line | 𐄂 | 𐄂 | 𐄂 | +| End line | 𐄂 | 𐄂 | 𐄂 | +| External ID (e.g., CVE) | ✓ | ✓ | ⚠ | +| URLs | ✓ | ✓ | ✓ | +| Internal doc/explanation | ✓ | 𐄂 | 𐄂 | +| Solution | ✓ | ✓ | 𐄂 | +| Confidence | 𐄂 | 𐄂 | 𐄂 | +| Affected item (e.g. class or package) | ✓ | ✓ | ✓ | +| Source code extract | 𐄂 | 𐄂 | 𐄂 | +| Internal ID | ✓ | 𐄂 | 𐄂 | +| Date | ✓ | 𐄂 | 𐄂 | +| Credits | ✓ | 𐄂 | 𐄂 | + +- ✓ => we have that data +- ⚠ => we have that data but it's partially reliable, or we need to extract that data from unstructured content +- 𐄂 => we don't have that data or it would need to develop specific or inefficient/unreliable logic to obtain it. + +The values provided by these tools are heterogeneous so they are sometimes +normalized into common values (e.g., `severity`, `confidence`, etc). diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md index 34d4507210e..ea8b96eb24d 100644 --- a/doc/user/application_security/dependency_scanning/index.md +++ b/doc/user/application_security/dependency_scanning/index.md @@ -46,17 +46,33 @@ this is enabled by default. The following languages and dependency managers are supported. -| Language (package managers) | Scan tool | -|-----------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------| -| JavaScript ([npm](https://www.npmjs.com/), [yarn](https://yarnpkg.com/en/)) | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium/general), [Retire.js](https://retirejs.github.io/retire.js) | -| Python ([pip](https://pip.pypa.io/en/stable/)) (only `requirements.txt` supported) | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium/general) | -| Ruby ([gem](https://rubygems.org/)) | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium/general), [bundler-audit](https://github.com/rubysec/bundler-audit) | -| Java ([Maven](https://maven.apache.org/)) | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium/general) | -| PHP ([Composer](https://getcomposer.org/)) | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium/general) | - -Some scanners require to send a list of project dependencies to GitLab's central -servers to check for vulnerabilities. To learn more about this or to disable it, -refer to the [GitLab Dependency Scanning tool documentation](https://gitlab.com/gitlab-org/security-products/dependency-scanning#remote-checks). +| Language (package managers) | Supported | Scan tool(s) | +|----------------------------- | --------- | ------------ | +| JavaScript ([npm](https://www.npmjs.com/), [yarn](https://yarnpkg.com/en/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium), [Retire.js](https://retirejs.github.io/retire.js) | +| Python ([pip](https://pip.pypa.io/en/stable/)) (only `requirements.txt` supported) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | +| Ruby ([gem](https://rubygems.org/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium), [bundler-audit](https://github.com/rubysec/bundler-audit) | +| Java ([Maven](https://maven.apache.org/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | +| PHP ([Composer](https://getcomposer.org/)) | yes | [gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium) | +| Python ([poetry](https://poetry.eustace.io/)) | no ([issue](https://gitlab.com/gitlab-org/gitlab-ee/issues/7006 "Support Poetry in Dependency Scanning")) | not available | +| Python ([Pipfile](https://docs.pipenv.org/en/latest/basics/)) | no ([issue](https://gitlab.com/gitlab-org/gitlab-ee/issues/11756 "Pipfile.lock support for Dependency Scanning"))| not available | +| Go ([Golang](https://golang.org/)) | no ([issue](https://gitlab.com/gitlab-org/gitlab-ee/issues/7132 "Dependency Scanning for Go")) | not available | + +## Remote checks + +While some tools pull a local database to check vulnerabilities, some others +like Gemnasium require sending data to GitLab central servers to analyze them: + +1. Gemnasium scans the dependencies of your project locally and sends a list of + packages to GitLab central servers. +1. The servers return the list of known vulnerabilities for all versions of + these packages. +1. The client picks up the relevant vulnerabilities by comparing with the versions + of the packages that are used by the project. + +The Gemnasium client does **NOT** send the exact package versions your project relies on. + +You can disable the remote checks by [using](#customizing-the-dependency-scanning-settings) +the `DS_DISABLE_REMOTE_CHECKS` environment variable and setting it to `true`. ## Configuring Dependency Scanning @@ -97,17 +113,10 @@ The report will be saved as a that you can later download and analyze. Due to implementation limitations, we always take the latest Dependency Scanning artifact available. -Some security scanners require to send a list of project dependencies to GitLab -central servers to check for vulnerabilities. To learn more about this or to -disable it, check the -[GitLab Dependency Scanning tool documentation](https://gitlab.com/gitlab-org/security-products/dependency-scanning#remote-checks). - #### Customizing the Dependency Scanning settings -The Dependency Scanning settings can be changed through environment variables by using the +The Dependency Scanning settings can be changed through [environment variables](#available-variables) by using the [`variables`](../../../ci/yaml/README.md#variables) parameter in `.gitlab-ci.yml`. -These variables are documented in the -[Dependency Scanning tool documentation](https://gitlab.com/gitlab-org/security-products/dependency-scanning#settings). For example: @@ -116,7 +125,7 @@ include: template: Dependency-Scanning.gitlab-ci.yml variables: - DEP_SCAN_DISABLE_REMOTE_CHECKS: true + DS_DISABLE_REMOTE_CHECKS: true ``` Because template is [evaluated before](../../../ci/yaml/README.md#include) the pipeline @@ -137,6 +146,24 @@ dependency_scanning: CI_DEBUG_TRACE: "true" ``` +#### Available variables + +Dependency Scanning can be [configured](#customizing-the-dependency-scanning-settings) +using environment variables. + +| Environment variable | Function | +|-------------------------------- |----------| +| `DS_ANALYZER_IMAGES` | Comma separated list of custom images. The official default images are still enabled. Read more about [customizing analyzers](analyzers.md). | +| `DS_ANALYZER_IMAGE_PREFIX` | Override the name of the Docker registry providing the official default images (proxy). Read more about [customizing analyzers](analyzers.md). | +| `DS_ANALYZER_IMAGE_TAG` | Override the Docker tag of the official default images. Read more about [customizing analyzers](analyzers.md). | +| `DS_DEFAULT_ANALYZERS` | Override the names of the official default images. Read more about [customizing analyzers](analyzers.md). | +| `DS_DISABLE_REMOTE_CHECKS` | Do not send any data to GitLab. Used in the [Gemnasium analyzer](#remote-checks). | +| `DS_PULL_ANALYZER_IMAGES` | Pull the images from the Docker registry (set to `0` to disable). | +| `DS_EXCLUDED_PATHS` | Exclude vulnerabilities from output based on the paths. A comma-separated list of patterns. Patterns can be globs, file or folder paths. Parent directories will also match patterns. | +| `DS_DOCKER_CLIENT_NEGOTIATION_TIMEOUT` | Time limit for Docker client negotiation. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`. For example, `300ms`, `1.5h`, or `2h45m`. | +| `DS_PULL_ANALYZER_IMAGE_TIMEOUT` | Time limit when pulling the image of an analyzer. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`. For example, `300ms`, `1.5h`, or `2h45m`. | +| `DS_RUN_ANALYZER_TIMEOUT` | Time limit when running an analyzer. Timeouts are parsed using Go's [`ParseDuration`](https://golang.org/pkg/time/#ParseDuration). Valid time units are `ns`, `us` (or `µs`), `ms`, `s`, `m`, `h`. For example, `300ms`, `1.5h`, or `2h45m`. | + ### Manual job definition for GitLab 11.5 and later For GitLab 11.5 and GitLab Runner 11.5 and later, the following `dependency_scanning` @@ -171,7 +198,7 @@ dependency_scanning: dependency_scanning: gl-dependency-scanning-report.json ``` -You can supply many other [settings variables](https://gitlab.com/gitlab-org/security-products/dependency-scanning#settings) +You can supply many other [settings variables](#available-variables) via `docker run --env` to customize your job execution. ### Manual job definition for GitLab 11.4 and earlier (deprecated) @@ -346,7 +373,7 @@ the report JSON unless stated otherwise. Presence of optional fields depends on | `vulnerabilities[].scanner` | A node that describes the analyzer used to find this vulnerability. | | `vulnerabilities[].scanner.id` | Id of the scanner as a snake_case string. | | `vulnerabilities[].scanner.name` | Name of the scanner, for display purposes. | -| `vulnerabilities[].location` | A node that tells where the vulnerability is located. | +| `vulnerabilities[].location` | A node that tells where the vulnerability is located. | | `vulnerabilities[].location.file` | Path to the dependencies file (e.g., `yarn.lock`). Optional. | | `vulnerabilities[].location.dependency` | A node that describes the dependency of a project where the vulnerability is located. | | `vulnerabilities[].location.dependency.package` | A node that provides the information on the package where the vulnerability is located. | @@ -379,17 +406,21 @@ Once a vulnerability is found, you can interact with it. Read more on how to ## Dependency List -> [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/10075) -in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.0. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/10075) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.0. + +An additional benefit of Dependency Scanning is the ability to get a list of your +project's dependencies with their versions. This list can be generated only for +[languages and package managers](#supported-languages-and-package-managers) +supported by Gemnasium. -An additional benefit of Dependency Scanning is the ability to get a list of your project's dependencies with their versions. +To see the generated dependency list, navigate to your project's **Project > Dependency List**. -This list can be generated only for [languages and package managers](#supported-languages-and-package-managers) supported by [Gemnasium](https://gitlab.com/gitlab-org/security-products/gemnasium/general). +## Versioning and release process -To see the generated dependency list, navigate to the Dependency List page under your project's left sidebar menu **Project > Dependency List**. +Please check the [Release Process documentation](https://gitlab.com/gitlab-org/security-products/release/blob/master/docs/release_process.md). ## Contributing to the vulnerability database You can search the [gemnasium-db](https://gitlab.com/gitlab-org/security-products/gemnasium-db) project to find a vulnerability in the Gemnasium database. -You can also [submit new vulnerabilities](https://gitlab.com/gitlab-org/security-products/gemnasium-db/blob/master/CONTRIBUTING.md).
\ No newline at end of file +You can also [submit new vulnerabilities](https://gitlab.com/gitlab-org/security-products/gemnasium-db/blob/master/CONTRIBUTING.md). diff --git a/doc/user/application_security/license_management/img/license_management_add_license.png b/doc/user/application_security/license_management/img/license_management_add_license.png Binary files differnew file mode 100644 index 00000000000..1e1a698515b --- /dev/null +++ b/doc/user/application_security/license_management/img/license_management_add_license.png diff --git a/doc/user/application_security/license_management/img/license_management_search.png b/doc/user/application_security/license_management/img/license_management_search.png Binary files differnew file mode 100644 index 00000000000..7b6006cef9d --- /dev/null +++ b/doc/user/application_security/license_management/img/license_management_search.png diff --git a/doc/user/application_security/license_management/img/license_management_settings.png b/doc/user/application_security/license_management/img/license_management_settings.png Binary files differindex b5490e59074..1a2bfa78a03 100644 --- a/doc/user/application_security/license_management/img/license_management_settings.png +++ b/doc/user/application_security/license_management/img/license_management_settings.png diff --git a/doc/user/application_security/license_management/index.md b/doc/user/application_security/license_management/index.md index 7a583016586..957c4ede981 100644 --- a/doc/user/application_security/license_management/index.md +++ b/doc/user/application_security/license_management/index.md @@ -262,6 +262,8 @@ To approve or blacklist a license: navigate to the project's **Settings > CI/CD** and expand the **License Management** section. 1. Click the **Add a license** button. + +  1. In the **License name** dropdown, either: - Select one of the available licenses. You can search for licenses in the field at the top of the list. @@ -270,8 +272,22 @@ To approve or blacklist a license: 1. Select the **Approve** or **Blacklist** radio button to approve or blacklist respectively the selected license. + + +To modify an existing license: + +1. In the **License Management** list, click the **Approved/Declined** dropdown to change it to the desired status. +  +Searching for Licenses: + +1. Use the **Search** box to search for a specific license. + +  + + + ## License Management report under pipelines > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/5491) diff --git a/doc/user/group/clusters/index.md b/doc/user/group/clusters/index.md index 26d764fa2cf..8d4ffd93f59 100644 --- a/doc/user/group/clusters/index.md +++ b/doc/user/group/clusters/index.md @@ -138,14 +138,6 @@ The result will then be: - The Staging cluster will be used for the `deploy to staging` job. - The Production cluster will be used for the `deploy to production` job. -## Unavailable features - -The following features are not currently available for group-level clusters: - -1. Terminals (see [related issue](https://gitlab.com/gitlab-org/gitlab-ce/issues/55487)). -1. Pod logs (see [related issue](https://gitlab.com/gitlab-org/gitlab-ce/issues/55488)). -1. Deployment boards (see [related issue](https://gitlab.com/gitlab-org/gitlab-ce/issues/55489)). - <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/group/contribution_analytics/index.md b/doc/user/group/contribution_analytics/index.md index 7e6cb24a51e..0da131ab7bd 100644 --- a/doc/user/group/contribution_analytics/index.md +++ b/doc/user/group/contribution_analytics/index.md @@ -21,9 +21,9 @@ page. ## Use cases - Analyze your team's contributions over a period of time, and offer a bonus for the top -contributors. + contributors. - Identify opportunities for improvement with group members who may benefit from additional -support. + support. ## Using Contribution Analytics diff --git a/doc/user/group/epics/index.md b/doc/user/group/epics/index.md index 2e4106f55e5..f53c1dd95d7 100644 --- a/doc/user/group/epics/index.md +++ b/doc/user/group/epics/index.md @@ -202,8 +202,8 @@ You may also consult the [group permissions table][permissions]. ## Thread - Comments: collaborate on that epic by posting comments in its thread. -These text fields also fully support -[GitLab Flavored Markdown](../../markdown.md#gitlab-flavored-markdown-gfm). + These text fields also fully support + [GitLab Flavored Markdown](../../markdown.md#gitlab-flavored-markdown-gfm). ## Comment, or start a discussion @@ -216,7 +216,7 @@ Once you wrote your comment, you can either: - You can [award an emoji](../../award_emojis.md) to that epic or its comments. -## Notifications +## Notifications - [Receive notifications](../../../workflow/notifications.md) for epic events. diff --git a/doc/user/group/index.md b/doc/user/group/index.md index abd95eddf63..183b12b1c73 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -41,12 +41,13 @@ You can create groups for numerous reasons. To name a couple: - Make it easier to `@mention` all of your team at once in issues and merge requests by creating a group and including the appropriate members. For example, you could create a group for your company members, and create a [subgroup](subgroups/index.md) for each individual team. Let's say you create a group called `company-team`, and you create subgroups in this group for the individual teams `backend-team`, `frontend-team`, and `production-team`. - - When you start a new implementation from an issue, you add a comment: - _"`@company-team`, let's do it! `@company-team/backend-team` you're good to go!"_ - - When your backend team needs help from frontend, they add a comment: - _"`@company-team/frontend-team` could you help us here please?"_ - - When the frontend team completes their implementation, they comment: - _"`@company-team/backend-team`, it's done! Let's ship it `@company-team/production-team`!"_ + +- When you start a new implementation from an issue, you add a comment: + _"`@company-team`, let's do it! `@company-team/backend-team` you're good to go!"_ +- When your backend team needs help from frontend, they add a comment: + _"`@company-team/frontend-team` could you help us here please?"_ +- When the frontend team completes their implementation, they comment: + _"`@company-team/backend-team`, it's done! Let's ship it `@company-team/production-team`!"_ ## Namespaces @@ -268,9 +269,10 @@ be unique. To change your group path: -1. Navigate to your group's **Settings > General**. -1. Enter a new name under **Group path**. -1. Click **Save group**. +1. Navigate to your group's **Settings > General** page. +1. Expand the **Path, transfer, remove** section. +1. Enter a new name under **Change group path**. +1. Click **Change group path**. CAUTION: **Caution:** It is currently not possible to rename a namespace if it contains a diff --git a/doc/user/group/saml_sso/scim_setup.md b/doc/user/group/saml_sso/scim_setup.md index 96cc523f4ec..5aef463d782 100644 --- a/doc/user/group/saml_sso/scim_setup.md +++ b/doc/user/group/saml_sso/scim_setup.md @@ -24,27 +24,27 @@ The following identity providers are supported: ## Requirements -- [Group SSO](index.md) needs to be configured. +- [Group SSO](index.md) needs to be configured. - The `scim_group` feature flag must be enabled: Run the following commands in a Rails console: - + ```sh # Omnibus GitLab gitlab-rails console - + # Installation from source cd /home/git/gitlab sudo -u git -H bin/rails console RAILS_ENV=production ``` - + To enable SCIM for a group named `group_name`: - + ```ruby group = Group.find_by_full_path('group_name') Feature.enable(:group_scim, group) ``` - + ### GitLab configuration Once [Single sign-on](index.md) has been configured, we can: @@ -53,7 +53,7 @@ Once [Single sign-on](index.md) has been configured, we can: 1. Click on the **Generate a SCIM token** button. 1. Save the token and URL so they can be used in the next step. - + ## SCIM IdP configuration @@ -63,15 +63,15 @@ In the [Single sign-on](index.md) configuration for the group, make sure that the **Name identifier value** (NameID) points to a unique identifier, such as the `user.objectid`. This will match the `extern_uid` used on GitLab. -The GitLab app in Azure needs to be configured following +The GitLab app in Azure needs to be configured following [Azure's SCIM setup](https://docs.microsoft.com/en-us/azure/active-directory/manage-apps/use-scim-to-provision-users-and-groups#getting-started). Note the following: - The `Tenant URL` and `secret token` are the ones retrieved in the -[previous step](#gitlab-configuration). + [previous step](#gitlab-configuration). - Should there be any problems with the availability of GitLab or similar -errors, the notification email set will get those. + errors, the notification email set will get those. - For mappings, we will only leave `Synchronize Azure Active Directory Users to AppName` enabled. You can then test the connection clicking on `Test Connection`. @@ -79,14 +79,14 @@ You can then test the connection clicking on `Test Connection`. ### Synchronize Azure Active Directory users 1. Click on `Synchronize Azure Active Directory Users to AppName`, to configure -the attribute mapping. + the attribute mapping. 1. Select the unique identifier (in the example `objectId`) as the `id` and `externalId`, -and enable the `Create`, `Update`, and `Delete` actions. + and enable the `Create`, `Update`, and `Delete` actions. 1. Map the `userPricipalName` to `emails[type eq "work"].value` and `mailNickname` to -`userName`. + `userName`. Example configuration: - +  1. Click on **Show advanced options > Edit attribute list for AppName**. @@ -95,11 +95,11 @@ and enable the `Create`, `Update`, and `Delete` actions. NOTE: **Note:** `username` should neither be primary nor required as we don't support that field on GitLab SCIM yet. - +  1. Save all the screens and, in the **Provisioning** step, set -the `Provisioning Status` to `ON`. + the `Provisioning Status` to `ON`. NOTE: **Note:** You can control what is actually synced by selecting the `Scope`. For example, diff --git a/doc/user/img/color_inline_colorchip_render_gfm.png b/doc/user/img/color_inline_colorchip_render_gfm.png Binary files differdeleted file mode 100644 index 6f93dbeeb10..00000000000 --- a/doc/user/img/color_inline_colorchip_render_gfm.png +++ /dev/null diff --git a/doc/user/img/markdown_inline_diffs_tags_rendered.png b/doc/user/img/markdown_inline_diffs_tags_rendered.png Binary files differdeleted file mode 100644 index 4279a20b5a0..00000000000 --- a/doc/user/img/markdown_inline_diffs_tags_rendered.png +++ /dev/null diff --git a/doc/user/img/math_inline_sup_render_gfm.png b/doc/user/img/math_inline_sup_render_gfm.png Binary files differdeleted file mode 100644 index 3ee2abb14df..00000000000 --- a/doc/user/img/math_inline_sup_render_gfm.png +++ /dev/null diff --git a/doc/user/img/task_list_ordered_render_gfm.png b/doc/user/img/task_list_ordered_render_gfm.png Binary files differdeleted file mode 100644 index 98ec791e958..00000000000 --- a/doc/user/img/task_list_ordered_render_gfm.png +++ /dev/null diff --git a/doc/user/markdown.md b/doc/user/markdown.md index 31c8093ced7..16df6d93277 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -1,15 +1,20 @@ # GitLab Markdown -This markdown guide is **valid for GitLab's system markdown entries and files**. -It is not valid for the [GitLab documentation website](https://docs.gitlab.com) -nor [GitLab's main website](https://about.gitlab.com), as they both use -[Kramdown](https://kramdown.gettalong.org) as their markdown engine. -The documentation website uses an extended Kramdown gem, [GitLab Kramdown](https://gitlab.com/gitlab-org/gitlab_kramdown). -Consult the [GitLab Kramdown Guide](https://about.gitlab.com/handbook/product/technical-writing/markdown-guide/) for a complete Kramdown reference. +This markdown guide is **valid only for GitLab's internal markdown rendering system for entries and files**. +It is **not** valid for the [GitLab documentation website](https://docs.gitlab.com) +or [GitLab's main website](https://about.gitlab.com), as they both use +[Kramdown](https://kramdown.gettalong.org) as their markdown engine. The documentation +website uses an extended Kramdown gem, [GitLab Kramdown](https://gitlab.com/gitlab-org/gitlab_kramdown). +Consult the [GitLab Kramdown Guide](https://about.gitlab.com/handbook/product/technical-writing/markdown-guide/) +for a complete Kramdown reference. + +NOTE: **Note:** We encourage you to view this document as [rendered by GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md). ## GitLab Flavored Markdown (GFM) -GitLab uses "GitLab Flavored Markdown" (GFM). It extends the [CommonMark specification][commonmark-spec] (which is based on standard Markdown) in a few significant ways to add additional useful functionality. It was inspired by [GitHub Flavored Markdown](https://help.github.com/articles/basic-writing-and-formatting-syntax/). +GitLab uses "GitLab Flavored Markdown" (GFM). It extends the [CommonMark specification](https://spec.commonmark.org/current/) +(which is based on standard Markdown) in several ways to add additional useful functionality. +It was inspired by [GitHub Flavored Markdown](https://help.github.com/articles/basic-writing-and-formatting-syntax/). You can use GFM in the following areas: @@ -22,35 +27,29 @@ You can use GFM in the following areas: - Markdown documents inside repositories - Epics **[ULTIMATE]** -You can also use other rich text files in GitLab. You might have to install a -dependency to do so. Please see the [`github-markup` gem readme](https://github.com/gitlabhq/markup#markups) for more information. - -> **Notes:** -> -> We encourage you to view this document as [rendered by GitLab itself](markdown.md). -> -> As of 11.1, GitLab uses the [CommonMark Ruby Library][commonmarker] for Markdown -processing of all new issues, merge requests, comments, and other Markdown content -in the GitLab system. As of 11.3, wiki pages and Markdown files (`.md`) in the -repositories are also processed with CommonMark. As of 11.8, the [Redcarpet -Ruby library][redcarpet] has been removed and all issues/comments, including -those from pre-11.1, are now processed using [CommonMark Ruby -Library][commonmarker]. -> -> The documentation website had its [markdown engine migrated from Redcarpet to Kramdown](https://gitlab.com/gitlab-com/gitlab-docs/merge_requests/108) -in October 2018. -> -> _Where there are significant differences, we will try to call them out in this document._ +You can also use other rich text files in GitLab. You might have to install a dependency +to do so. Please see the [`gitlab-markup` gem project](https://gitlab.com/gitlab-org/gitlab-markup) +for more information. + +### Transition from Redcarpet to CommonMark -### Transitioning to CommonMark +Since 11.1, GitLab uses the [CommonMark Ruby Library](https://github.com/gjtorikian/commonmarker) +for Markdown processing of all new issues, merge requests, comments, and other Markdown +content in the GitLab system. Since 11.3, wiki pages and Markdown files (`*.md`) in +repositories are also processed with CommonMark. As of 11.8, the [Redcarpet Ruby library](https://github.com/vmg/redcarpet) +has been removed and all issues and comments, including those from pre-11.1, are now processed +using the [CommonMark Ruby Library](https://github.com/gjtorikian/commonmarker). -You may have older issues/merge requests or Markdown documents in your -repository that were written using some of the nuances of RedCarpet's version +The documentation website had its [markdown engine migrated from Redcarpet to Kramdown](https://gitlab.com/gitlab-com/gitlab-docs/merge_requests/108) +in October 2018. + +You may have older issues, merge requests, or Markdown documents in your +repository that were written using some of the nuances of GitLab's RedCarpet version of Markdown. Since CommonMark uses a slightly stricter syntax, these documents -may now display a little strangely since we've transitioned to CommonMark. -Numbered lists with nested lists in particular can be displayed incorrectly. +may now display a little differently since we've transitioned to CommonMark. -It is usually quite easy to fix. In the case of a nested list such as this: +It is usually quite easy to fix. For example, numbered lists with nested lists may +render incorrectly: ```markdown 1. Chocolate @@ -58,7 +57,14 @@ It is usually quite easy to fix. In the case of a nested list such as this: - milk ``` -simply add a space to each nested item: +1. Chocolate + - dark + - milk + +--- + +Simply add a space to each nested item to align the `-` with the first character of +the top list item (`C` in this case): ```markdown 1. Chocolate @@ -66,515 +72,674 @@ simply add a space to each nested item: - milk ``` -In the documentation below, we try to highlight some of the differences. +1. Chocolate + - dark + - milk + +NOTE: **Note:** We will flag any significant differences between Redcarpet and CommonMark + markdown in this document. If you have a large volume of Markdown files, it can be tedious to determine -if they will be displayed correctly or not. You can use the +if they will display correctly or not. You can use the [diff_redcarpet_cmark](https://gitlab.com/digitalmoksha/diff_redcarpet_cmark) -tool (not an officially supported product) to generate a list of files and +tool (not an officially supported product) to generate a list of files, and the differences between how RedCarpet and CommonMark render the files. It can give -you a great idea if anything needs to be changed - many times nothing will need -to changed. +an indication if anything needs to be changed - often nothing will need +to change. + +### GFM extends standard markdown + +GitLab makes full use of the standard (CommonMark) formatting, but also includes additional +functionality useful for GitLab users. + +It makes use of [new markdown features](#new-GFM-markdown-extensions), +not found in standard markdown: + +- [Color "chips" written in HEX, RGB or HSL](#colors) +- [Diagrams and flowcharts using Mermaid](#diagrams-and-flowcharts-using-mermaid) +- [Emoji](#emoji) +- [Front matter](#front-matter) +- [Inline diffs](#inline-diff) +- [Math equations and symbols written in LaTeX](#math) +- [Special GitLab references](#special-gitlab-references) +- [Task Lists](#task-lists) +- [Wiki specific markdown](#wiki-specific-markdown) + +It also has [extended markdown features](#standard-markdown-and-extensions-in-gitlab), without +changing how standard markdown is used: + +| Standard markdown | Extended markdown in GitLab | +| ------------------------------------- | ------------------------- | +| [blockquotes](#blockquotes) | [multiline blockquotes](#multiline-blockquote) | +| [code blocks](#code-spans-and-blocks) | [colored code and syntax highlighting](#colored-code-and-syntax-highlighting) | +| [emphasis](#emphasis) | [multiple underscores in words](#multiple-underscores-in-words-and-mid-word-emphasis) +| [headers](#headers) | [linkable Header IDs](#header-ids-and-links) | +| [images](#images) | [embedded videos](#videos) | +| [linebreaks](#line-breaks) | [more linebreak control](#newlines) | +| [links](#links) | [automatically linking URLs](#url-auto-linking) | + +## New GFM markdown extensions + +### Colors -### Newlines +> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#colors). -> If this is not rendered correctly, see -https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#newlines +It is possible to have color written in HEX, RGB or HSL format rendered with a color +indicator. -GFM honors the markdown specification in how [paragraphs and line breaks are handled][commonmark-spec]. +Supported formats (named colors are not supported): -A paragraph is simply one or more consecutive lines of text, separated by one or -more blank lines. -Line-breaks, or soft returns, are rendered if you end a line with two or more spaces: +- HEX: `` `#RGB[A]` `` or `` `#RRGGBB[AA]` `` +- RGB: `` `RGB[A](R, G, B[, A])` `` +- HSL: `` `HSL[A](H, S, L[, A])` `` -<!-- (Do *NOT* remove the two ending whitespaces in the following line.) --> -<!-- (They are needed for the Markdown text to render correctly.) --> - Roses are red [followed by two or more spaces] - Violets are blue +Color written inside backticks will be followed by a color "chip": - Sugar is sweet +```markdown +`#F00` +`#F00A` +`#FF0000` +`#FF0000AA` +`RGB(0,255,0)` +`RGB(0%,100%,0%)` +`RGBA(0,255,0,0.3)` +`HSL(540,70%,50%)` +`HSLA(540,70%,50%,0.3)` +``` -<!-- (Do *NOT* remove the two ending whitespaces in the following line.) --> -<!-- (They are needed for the Markdown text to render correctly.) --> -Roses are red -Violets are blue +`#F00` +`#F00A` +`#FF0000` +`#FF0000AA` +`RGB(0,255,0)` +`RGB(0%,100%,0%)` +`RGBA(0,255,0,0.3)` +`HSL(540,70%,50%)` +`HSLA(540,70%,50%,0.3)` -Sugar is sweet +### Diagrams and flowcharts using Mermaid -### Multiple underscores in words +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/15107) in +GitLab 10.3. -> If this is not rendered correctly, see -https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#multiple-underscores-in-words +It is possible to generate diagrams and flowcharts from text using [Mermaid](https://mermaidjs.github.io/). +Visit the official page for more details. -It is not reasonable to italicize just _part_ of a word, especially when you're -dealing with code and names that often appear with multiple underscores. -Therefore, GFM ignores multiple underscores in words: +In order to generate a diagram or flowchart, you should write your text inside the `mermaid` block: - perform_complicated_task +~~~ +```mermaid +graph TD; + A-->B; + A-->C; + B-->D; + C-->D; +``` +~~~ - do_this_and_do_that_and_another_thing +```mermaid +graph TD; + A-->B; + A-->C; + B-->D; + C-->D; +``` -perform_complicated_task +### Emoji -do_this_and_do_that_and_another_thing +> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#emoji). -### URL auto-linking +```md +Sometimes you want to :monkey: around a bit and add some :star2: to your :speech_balloon:. Well we have a gift for you: -> If this is not rendered correctly, see -https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#url-auto-linking +:zap: You can use emoji anywhere GFM is supported. :v: -GFM will autolink almost any URL you copy and paste into your text: +You can use it to point out a :bug: or warn about :speak_no_evil: patches. And if someone improves your really :snail: code, send them some :birthday:. People will :heart: you for that. - * https://www.google.com - * https://google.com/ - * ftp://ftp.us.debian.org/debian/ - * smb://foo/bar/baz - * irc://irc.freenode.net/gitlab - * http://localhost:3000 +If you are new to this, don't be :fearful:. You can easily join the emoji :family:. All you need to do is to look up one of the supported codes. -* https://www.google.com -* https://google.com/ -* ftp://ftp.us.debian.org/debian/ -* <a href="smb://foo/bar/baz">smb://foo/bar/baz</a> -* <a href="irc://irc.freenode.net/gitlab">irc://irc.freenode.net/gitlab</a> -* http://localhost:3000 +Consult the [Emoji Cheat Sheet](https://www.emojicopy.com) for a list of all supported emoji codes. :thumbsup: +``` -### Multiline blockquote +Sometimes you want to <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/monkey.png" width="20px" height="20px" style="display:inline;margin:0"> around a bit and add some <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/star2.png" width="20px" height="20px" style="display:inline;margin:0"> to your <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/speech_balloon.png" width="20px" height="20px" style="display:inline;margin:0">. Well we have a gift for you: -> If this is not rendered correctly, see -https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#multiline-blockquote +<img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/zap.png" width="20px" height="20px" style="display:inline;margin:0">You can use emoji anywhere GFM is supported. <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/v.png" width="20px" height="20px" style="display:inline;margin:0"> -On top of standard Markdown [blockquotes](#blockquotes), which require prepending `>` to quoted lines, -GFM supports multiline blockquotes fenced by <code>>>></code>: +You can use it to point out a <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/bug.png" width="20px" height="20px" style="display:inline;margin:0"> or warn about <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/speak_no_evil.png" width="20px" height="20px" style="display:inline;margin:0"> patches. And if someone improves your really <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/snail.png" width="20px" height="20px" style="display:inline;margin:0"> code, send them some <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/birthday.png" width="20px" height="20px" style="display:inline;margin:0">. People will <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/heart.png" width="20px" height="20px" style="display:inline;margin:0"> you for that. -``` ->>> -If you paste a message from somewhere else +If you are new to this, don't be <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/fearful.png" width="20px" height="20px" style="display:inline;margin:0">. You can easily join the emoji <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/family.png" width="20px" height="20px" style="display:inline;margin:0">. All you need to do is to look up one of the supported codes. -that +Consult the [Emoji Cheat Sheet](https://www.webfx.com/tools/emoji-cheat-sheet/) for a list of all supported emoji codes. <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/thumbsup.png" width="20px" height="20px" style="display:inline;margin:0"> -spans +> **Note:** The emoji example above uses hard-coded images for this documentation. The emoji, +when rendered within GitLab, may appear different depending on the OS and browser used. -multiple lines, +Most emoji are natively supported on macOS, Windows, iOS, Android and will fallback to image-based emoji where there is lack of support. -you can quote that without having to manually prepend `>` to every line! ->>> -``` +NOTE: **Note:** On Linux, you can download [Noto Color Emoji](https://www.google.com/get/noto/help/emoji/) +to get full native emoji support. Ubuntu 18.04 (like many modern Linux distros) has +this font installed by default. -<blockquote dir="auto"> -<p>If you paste a message from somewhere else</p> -<p>that</p> -<p>spans</p> -<p>multiple lines,</p> -<p>you can quote that without having to manually prepend <code>></code> to every line!</p> -</blockquote> +### Front matter -### Code and syntax highlighting +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/23331) + in GitLab 11.6. -> If this is not rendered correctly, see -https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#code-and-syntax-highlighting +Front matter is metadata included at the beginning of a markdown document, preceding +its content. This data can be used by static site generators such as [Jekyll](https://jekyllrb.com/docs/front-matter/), +[Hugo](https://gohugo.io/content-management/front-matter/), and many other applications. -_GitLab uses the [Rouge Ruby library][rouge] for syntax highlighting. For a -list of supported languages visit the Rouge website._ +When you view a Markdown file rendered by GitLab, any front matter is displayed as-is, +in a box at the top of the document, before the rendered HTML content. To view an example, +you can toggle between the source and rendered version of a [GitLab documentation file](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/README.md). -Blocks of code are either fenced by lines with three back-ticks <code>```</code>, -or are indented with four spaces. Only the fenced code blocks support syntax -highlighting: +In GitLab, front matter is only used in Markdown files and wiki pages, not the other +places where Markdown formatting is supported. It must be at the very top of the document, +and must be between delimiters, as explained below. -``` -Inline `code` has `back-ticks around` it. -``` +The following delimeters are supported: -Inline `code` has `back-ticks around` it. +- YAML (`---`): -Example: + ~~~yaml + --- + title: About Front Matter + example: + language: yaml + --- + ~~~ - ```javascript - var s = "JavaScript syntax highlighting"; - alert(s); - ``` +- TOML (`+++`): - ```python - def function(): - #indenting works just fine in the fenced code block - s = "Python syntax highlighting" - print s - ``` + ~~~toml + +++ + title = "About Front Matter" + [example] + language = "toml" + +++ + ~~~ - ```ruby - require 'redcarpet' - markdown = Redcarpet.new("Hello World!") - puts markdown.to_html - ``` +- JSON (`;;;`): - ``` - No language indicated, so no syntax highlighting. - s = "There is no highlighting for this." - But let's throw in a <b>tag</b>. - ``` + ~~~json + ;;; + { + "title": "About Front Matter" + "example": { + "language": "json" + } + } + ;;; + ~~~ -becomes: +Other languages are supported by adding a specifier to any of the existing +delimiters. For example: -```javascript -var s = "JavaScript syntax highlighting"; -alert(s); +```php +---php +$title = "About Front Matter"; +$example = array( + 'language' => "php", +); +--- ``` -```python -def function(): - #indenting works just fine in the fenced code block - s = "Python syntax highlighting" - print s -``` +### Inline diff -```ruby -require 'redcarpet' -markdown = Redcarpet.new("Hello World!") -puts markdown.to_html -``` +> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#inline-diff). -``` -No language indicated, so no syntax highlighting. -s = "There is no highlighting for this." -But let's throw in a <b>tag</b>. -``` +With inline diff tags you can display {+ additions +} or [- deletions -]. -### Inline diff +The wrapping tags can be either curly braces or square brackets: -> If this is not rendered correctly, see -https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#inline-diff +```markdown +- {+ addition 1 +} +- [+ addition 2 +] +- {- deletion 3 -} +- [- deletion 4 -] +``` -With inline diffs tags you can display {+ additions +} or [- deletions -]. +- {+ addition 1 +} +- [+ addition 2 +] +- {- deletion 3 -} +- [- deletion 4 -] -The wrapping tags can be either curly braces or square brackets. +--- -Examples: +However the wrapping tags cannot be mixed: +```markdown +- {+ addition +] +- [+ addition +} +- {- deletion -] +- [- deletion -} ``` -- {+ additions +} -- [+ additions +] -- {- deletions -} -- [- deletions -] -``` -becomes: +### Math + +> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#math). + +It is possible to have math written with LaTeX syntax rendered using [KaTeX](https://github.com/Khan/KaTeX). - +Math written between dollar signs `$` will be rendered inline with the text. Math written +inside a [code block](#code-spans-and-blocks) with the language declared as `math`, will be rendered +on a separate line: -However the wrapping tags cannot be mixed as such: +~~~ +This math is inline $`a^2+b^2=c^2`$. +This is on a separate line + +```math +a^2+b^2=c^2 ``` -- {+ additions +] -- [+ additions +} -- {- deletions -] -- [- deletions -} +~~~ + +This math is inline $`a^2+b^2=c^2`$. + +This is on a separate line + +```math +a^2+b^2=c^2 ``` -### Emoji +_Be advised that KaTeX only supports a [subset](https://katex.org/docs/supported.html) of LaTeX._ -```md -Sometimes you want to :monkey: around a bit and add some :star2: to your :speech_balloon:. Well we have a gift for you: +NOTE: **Note:** This also works for the asciidoctor `:stem: latexmath`. For details see +the [asciidoctor user manual](http://asciidoctor.org/docs/user-manual/#activating-stem-support). -:zap: You can use emoji anywhere GFM is supported. :v: +### Special GitLab references -You can use it to point out a :bug: or warn about :speak_no_evil: patches. And if someone improves your really :snail: code, send them some :birthday:. People will :heart: you for that. +GFM recognizes special GitLab related references. For example, you can easily reference +an issue, a commit, a team member or even the whole team within a project. GFM will turn +that reference into a link so you can navigate between them easily. -If you are new to this, don't be :fearful:. You can easily join the emoji :family:. All you need to do is to look up one of the supported codes. +Additionally, GFM recognizes certain cross-project references, and also has a shorthand +version to reference other projects from the same namespace. -Consult the [Emoji Cheat Sheet](https://www.emojicopy.com) for a list of all supported emoji codes. :thumbsup: +GFM will recognize the following: -Most emoji are natively supported on macOS, Windows, iOS, Android and will fallback to image-based emoji where there is lack of support. +| references | input | cross-project reference | shortcut within same namespace | +| :------------------------------ | :------------------------- | :-------------------------------------- | :----------------------------- | +| specific user | `@user_name` | | | +| specific group | `@group_name` | | | +| entire team | `@all` | | | +| project | `namespace/project>` | | | +| issue | ``#123`` | `namespace/project#123` | `project#123` | +| merge request | `!123` | `namespace/project!123` | `project!123` | +| snippet | `$123` | `namespace/project$123` | `project$123` | +| epic **[ULTIMATE]** | `&123` | `group1/subgroup&123` | | +| label by ID | `~123` | `namespace/project~123` | `project~123` | +| one-word label by name | `~bug` | `namespace/project~bug` | `project~bug` | +| multi-word label by name | `~"feature request"` | `namespace/project~"feature request"` | `project~"feature request"` | +| project milestone by ID | `%123` | `namespace/project%123` | `project%123` | +| one-word milestone by name | `%v1.23` | `namespace/project%v1.23` | `project%v1.23` | +| multi-word milestone by name | `%"release candidate"` | `namespace/project%"release candidate"` | `project%"release candidate"` | +| specific commit | `9ba12248` | `namespace/project@9ba12248` | `project@9ba12248` | +| commit range comparison | `9ba12248...b19a04f5` | `namespace/project@9ba12248...b19a04f5` | `project@9ba12248...b19a04f5` | +| repository file references | `[README](doc/README)` | | | +| repository file line references | `[README](doc/README#L13)` | | | -On Linux, you can download [Noto Color Emoji](https://www.google.com/get/noto/help/emoji/) to get full native emoji support. +### Task lists + +> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#task-lists). + +You can add task lists anywhere markdown is supported, but you can only "click" to +toggle the boxes if they are in issues, merge requests, or comments. In other places +you must edit the markdown manually to change the status by adding or removing the `x`. -Ubuntu 18.04 (like many modern Linux distros) has this font installed by default. +To create a task list, add a specially-formatted Markdown list. You can use either +unordered or ordered lists: + +```markdown +- [x] Completed task +- [ ] Incomplete task + - [ ] Sub-task 1 + - [x] Sub-task 2 + - [ ] Sub-task 3 +1. [x] Completed task +1. [ ] Incomplete task + 1. [ ] Sub-task 1 + 1. [x] Sub-task 2 ``` -Sometimes you want to <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/monkey.png" width="20px" height="20px" style="display:inline;margin:0"> around a bit and add some <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/star2.png" width="20px" height="20px" style="display:inline;margin:0"> to your <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/speech_balloon.png" width="20px" height="20px" style="display:inline;margin:0">. Well we have a gift for you: +- [x] Completed task +- [ ] Incomplete task + - [ ] Sub-task 1 + - [x] Sub-task 2 + - [ ] Sub-task 3 +1. [x] Completed task +1. [ ] Incomplete task + 1. [ ] Sub-task 1 + 1. [x] Sub-task 2 -<img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/zap.png" width="20px" height="20px" style="display:inline;margin:0">You can use emoji anywhere GFM is supported. <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/v.png" width="20px" height="20px" style="display:inline;margin:0"> +### Wiki-specific Markdown -You can use it to point out a <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/bug.png" width="20px" height="20px" style="display:inline;margin:0"> or warn about <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/speak_no_evil.png" width="20px" height="20px" style="display:inline;margin:0"> patches. And if someone improves your really <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/snail.png" width="20px" height="20px" style="display:inline;margin:0"> code, send them some <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/birthday.png" width="20px" height="20px" style="display:inline;margin:0">. People will <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/heart.png" width="20px" height="20px" style="display:inline;margin:0"> you for that. +The following examples show how links inside wikis behave. -If you are new to this, don't be <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/fearful.png" width="20px" height="20px" style="display:inline;margin:0">. You can easily join the emoji <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/family.png" width="20px" height="20px" style="display:inline;margin:0">. All you need to do is to look up one of the supported codes. +#### Wiki - Direct page link -Consult the [Emoji Cheat Sheet](https://www.webfx.com/tools/emoji-cheat-sheet/) for a list of all supported emoji codes. <img src="https://gitlab.com/gitlab-org/gitlab-ce/raw/master/app/assets/images/emoji/thumbsup.png" width="20px" height="20px" style="display:inline;margin:0"> +A link which just includes the slug for a page will point to that page, +_at the base level of the wiki_. -Most emoji are natively supported on macOS, Windows, iOS, Android and will fallback to image-based emoji where there is lack of support. +This snippet would link to a `documentation` page at the root of your wiki: -On Linux, you can download [Noto Color Emoji](https://www.google.com/get/noto/help/emoji/) to get full native emoji support. +```markdown +[Link to Documentation](documentation) +``` -Ubuntu 18.04 (like many modern Linux distros) has this font installed by default. +#### Wiki - Direct file link -### Special GitLab references +Links with a file extension point to that file, _relative to the current page_. -GFM recognizes special references. +If the snippet below was placed on a page at `<your_wiki>/documentation/related`, +it would link to `<your_wiki>/documentation/file.md`: -You can easily reference e.g. an issue, a commit, a team member or even the whole team within a project. +```markdown +[Link to File](file.md) +``` -GFM will turn that reference into a link so you can navigate between them easily. +#### Wiki - Hierarchical link -GFM will recognize the following: +A link can be constructed relative to the current wiki page using `./<page>`, +`../<page>`, etc. -| input | references | -|:---------------------------|:--------------------------------| -| `@user_name` | specific user | -| `@group_name` | specific group | -| `@all` | entire team | -| `namespace/project>` | project | -| `#12345` | issue | -| `!123` | merge request | -| `$123` | snippet | -| `&123` | epic **[ULTIMATE]** | -| `~123` | label by ID | -| `~bug` | one-word label by name | -| `~"feature request"` | multi-word label by name | -| `%123` | project milestone by ID | -| `%v1.23` | one-word milestone by name | -| `%"release candidate"` | multi-word milestone by name | -| `9ba12248` | specific commit | -| `9ba12248...b19a04f5` | commit range comparison | -| `[README](doc/README)` | repository file references | -| `[README](doc/README#L13)` | repository file line references | - -GFM also recognizes certain cross-project references: - -| input | references | -|:----------------------------------------|:------------------------| -| `namespace/project#123` | issue | -| `namespace/project!123` | merge request | -| `namespace/project%123` | project milestone | -| `namespace/project$123` | snippet | -| `namespace/project@9ba12248` | specific commit | -| `group1/subgroup&123` | epic **[ULTIMATE]** | -| `namespace/project@9ba12248...b19a04f5` | commit range comparison | -| `namespace/project~"Some label"` | issues with given label | - -It also has a shorthand version to reference other projects from the same namespace: - -| input | references | -|:------------------------------|:------------------------| -| `project#123` | issue | -| `project!123` | merge request | -| `project%123` | project milestone | -| `project$123` | snippet | -| `project@9ba12248` | specific commit | -| `project@9ba12248...b19a04f5` | commit range comparison | -| `project~"Some label"` | issues with given label | +If this snippet was placed on a page at `<your_wiki>/documentation/main`, +it would link to `<your_wiki>/documentation/related`: -### Task lists +```markdown +[Link to Related Page](./related) +``` + +If this snippet was placed on a page at `<your_wiki>/documentation/related/content`, +it would link to `<your_wiki>/documentation/main`: -> If this is not rendered correctly, see -https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#task-lists +```markdown +[Link to Related Page](../main) +``` -You can add task lists to issues, merge requests and comments. To create a task list, add a specially-formatted Markdown list, like so: +If this snippet was placed on a page at `<your_wiki>/documentation/main`, +it would link to `<your_wiki>/documentation/related.md`: +```markdown +[Link to Related Page](./related.md) ``` -- [x] Completed task -- [ ] Incomplete task - - [ ] Sub-task 1 - - [x] Sub-task 2 - - [ ] Sub-task 3 + +If this snippet was placed on a page at `<your_wiki>/documentation/related/content`, +it would link to `<your_wiki>/documentation/main.md`: + +```markdown +[Link to Related Page](../main.md) ``` - +#### Wiki - Root link -Tasks formatted as ordered lists are supported as well: +A link starting with a `/` is relative to the wiki root. + +This snippet links to `<wiki_root>/documentation`: +```markdown +[Link to Related Page](/documentation) ``` -1. [x] Completed task -1. [ ] Incomplete task - 1. [ ] Sub-task 1 - 1. [x] Sub-task 2 + +This snippet links to `<wiki_root>/miscellaneous.md`: + +```markdown +[Link to Related Page](/miscellaneous.md) ``` - +## Standard markdown and extensions in GitLab -Task lists can only be created in descriptions, not in titles. Task item state can be managed by editing the description's Markdown or by toggling the rendered check boxes. +All standard markdown formatting should work as expected within GitLab. Some standard +functionality is extended with additional features, without affecting the standard usage. +If a functionality is extended, the new option will be listed as a sub-section. -### Videos +### Blockquotes -> If this is not rendered correctly, see -https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#videos +Blockquotes are an easy way to highlight information, such as a side-note. It is generated +by starting the lines of the blockquote with `>`: -Image tags with a video extension are automatically converted to a video player. +```markdown +> Blockquotes are very handy to emulate reply text. +> This line is part of the same quote. -The valid video extensions are `.mp4`, `.m4v`, `.mov`, `.webm`, and `.ogv`. +Quote break. - Here's a sample video: +> This is a very long line that will still be quoted properly when it wraps. Oh boy let's keep writing to make sure this is long enough to actually wrap for everyone. Oh, you can *put* **Markdown** into a blockquote. +``` -  +> Blockquotes are very handy to emulate reply text. +> This line is part of the same quote. -Here's a sample video: +Quote break. -<div class="video-container"> - <video src="img/markdown_video.mp4" width="400" controls="true" data-setup="{}" data-title="Sample Video"></video> - <p><a href="img/markdown_video.mp4" target="_blank" rel="noopener noreferrer" title="Download 'Sample Video'">Sample Video</a></p> -</div> +> This is a very long line that will still be quoted properly when it wraps. Oh boy let's keep writing to make sure this is long enough to actually wrap for everyone. Oh, you can *put* **Markdown** into a blockquote. -### Math +#### Multiline blockquote -> If this is not rendered correctly, see -https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#math +> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#multiline-blockquote). -It is possible to have math written with the LaTeX syntax rendered using [KaTeX][katex]. +GFM extends the standard markdown standard by also supporting multiline blockquotes +fenced by `>>>`: -Math written inside ```$``$``` will be rendered inline with the text. +``` +>>> +If you paste a message from somewhere else -Math written inside triple back quotes, with the language declared as `math`, will be rendered on a separate line. +that spans multiple lines, -Example: +you can quote that without having to manually prepend `>` to every line! +>>> +``` - This math is inline $`a^2+b^2=c^2`$. +>>> +If you paste a message from somewhere else - This is on a separate line - ```math - a^2+b^2=c^2 - ``` +that spans multiple lines, -Becomes: +you can quote that without having to manually prepend `>` to every line! +>>> -This math is inline . +### Code spans and blocks -This is on a separate line +You can easily highlight anything that should be viewed as code and not simple text. -<img src="./img/math_inline_sup_render_gfm.png" > +Simple inline code is easily highlighted with single backticks `` ` ``: -_Be advised that KaTeX only supports a [subset][katex-subset] of LaTeX._ +```markdown +Inline `code` has `back-ticks around` it. +``` ->**Note:** -This also works for the asciidoctor `:stem: latexmath`. For details see the [asciidoctor user manual][asciidoctor-manual]. +Inline `code` has `back-ticks around` it. -### Colors +--- -> If this is not rendered correctly, see -https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#colors +Similarly, a whole block of code can be fenced with triple backticks ```` ``` ````, +triple tildes (`~~~`), or indended 4 or more spaces to achieve a similar effect for +a larger body of code. test. -It is possible to have color written in HEX, RGB or HSL format rendered with a color indicator. +~~~ +``` +def function(): + #indenting works just fine in the fenced code block + s = "Python code" + print s +``` -Color written inside backticks will be followed by a color "chip". + Using 4 spaces + is like using + 3-backtick fences. +~~~ -Examples: +``` +~~~ +Tildes are OK too. +~~~ +``` - `#F00` - `#F00A` - `#FF0000` - `#FF0000AA` - `RGB(0,255,0)` - `RGB(0%,100%,0%)` - `RGBA(0,255,0,0.7)` - `HSL(540,70%,50%)` - `HSLA(540,70%,50%,0.7)` +The three examples above render as: -Becomes: +``` +def function(): + #indenting works just fine in the fenced code block + s = "Python code" + print s +``` - + Using 4 spaces + is like using + 3-backtick fences. -#### Supported formats: +~~~ +Tildes are OK too. +~~~ -* HEX: `` `#RGB[A]` `` or `` `#RRGGBB[AA]` `` -* RGB: `` `RGB[A](R, G, B[, A])` `` -* HSL: `` `HSL[A](H, S, L[, A])` `` +#### Colored code and syntax highlighting -### Mermaid +> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#colored-code-and-syntax-highlighting). -> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/15107) in -GitLab 10.3. -> -> If this is not rendered correctly, see -https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#mermaid +GitLab uses the [Rouge Ruby library](http://rouge.jneen.net/) for more colorful syntax +highlighting in code blocks. For a list of supported languages visit the +[Rouge project wiki](https://github.com/jneen/rouge/wiki/List-of-supported-languages-and-lexers). +Syntax highlighting is only supported in code blocks, it is not possible to highlight +code when it is inline. -It is possible to generate diagrams and flowcharts from text using [Mermaid](https://mermaidjs.github.io/). +Blocks of code are fenced by lines with three back-ticks ```` ``` ```` or three tildes `~~~`, and have +the language identified at the end of the first fence: -In order to generate a diagram or flowchart, you should write your text inside the `mermaid` block. +~~~ +```javascript +var s = "JavaScript syntax highlighting"; +alert(s); +``` -Example: +```python +def function(): + #indenting works just fine in the fenced code block + s = "Python syntax highlighting" + print s +``` - ```mermaid - graph TD; - A-->B; - A-->C; - B-->D; - C-->D; - ``` +```ruby +require 'redcarpet' +markdown = Redcarpet.new("Hello World!") +puts markdown.to_html +``` -Becomes: +``` +No language indicated, so no syntax highlighting. +s = "There is no highlighting for this." +But let's throw in a <b>tag</b>. +``` +~~~ -```mermaid -graph TD; - A-->B; - A-->C; - B-->D; - C-->D; +The four examples above render as: + +```javascript +var s = "JavaScript syntax highlighting"; +alert(s); ``` -For details see the [Mermaid official page](https://mermaidjs.github.io/). +```python +def function(): + #indenting works just fine in the fenced code block + s = "Python syntax highlighting" + print s +``` -### Front matter +```ruby +require 'redcarpet' +markdown = Redcarpet.new("Hello World!") +puts markdown.to_html +``` -> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/23331) - in GitLab 11.6. +``` +No language indicated, so no syntax highlighting. +s = "There is no highlighting for this." +But let's throw in a <b>tag</b>. +``` -Front matter is metadata included at the beginning of a markdown document, preceding -its content. This data can be used by static site generators such as [Jekyll](https://jekyllrb.com/docs/front-matter/) and [Hugo](https://gohugo.io/content-management/front-matter/), -and many other applications. +### Emphasis -In GitLab, front matter is only used in Markdown files and wiki pages, not the other places where Markdown formatting is supported. -When you view a Markdown file rendered by GitLab, any front matter is displayed as-is, in a box at the top of the document, before the rendered HTML content. -To view an example, you can toggle between the source and rendered version of a [GitLab documentation file](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/README.md). +There are multiple ways to emphasize text in markdown. You can italicize, bold, strikethrough, +as well as combine these emphasis styles together. -The following delimeters are supported: +Examples: -- YAML (`---`): +```markdown +Emphasis, aka italics, with *asterisks* or _underscores_. - ``` - --- - title: About Front Matter - example: - language: yaml - --- - ``` +Strong emphasis, aka bold, with double **asterisks** or __underscores__. -- TOML (`+++`): +Combined emphasis with **asterisks and _underscores_**. - ``` - +++ - title = "About Front Matter" - [example] - language = "toml" - +++ - ``` +Strikethrough uses two tildes. ~~Scratch this.~~ +``` -- JSON (`;;;`): +Emphasis, aka italics, with *asterisks* or _underscores_. - ``` - ;;; - { - "title": "About Front Matter" - "example": { - "language": "json" - } - } - ;;; - ``` +Strong emphasis, aka bold, with double **asterisks** or __underscores__. -Other languages are supported by adding a specifier to any of the existing -delimiters. For example: +Combined emphasis with **asterisks and _underscores_**. + +Strikethrough uses two tildes. ~~Scratch this.~~ +NOTE: **Note:** Strikethrough is not part of the core Markdown standard, but is part of GFM. + +#### Multiple underscores in words and mid-word emphasis + +> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#multiple-underscores-in-words). + +It is not usually useful to italicize just _part_ of a word, especially when you're +dealing with code and names that often appear with multiple underscores. As a result, +GFM extends the standard markdown standard by ignoring multiple underlines in words, +to allow better rendering of markdown documents discussing code: + +```md +perform_complicated_task + +do_this_and_do_that_and_another_thing + +but_emphasis is_desired _here_ ``` ----php -$title = "About Front Matter"; -$example = array( - 'language' => "php", -); + +perform_complicated_task + +do_this_and_do_that_and_another_thing + +but_emphasis is_desired _here_ + --- + +If you wish to emphasize only a part of a word, it can still be done with asterisks: + +```md +perform*complicated*task + +do*this*and*do*that*and*another thing ``` -## Standard Markdown +perform*complicated*task + +do*this*and*do*that*and*another thing -### Headers +### Footnotes + +Footnotes add a link to a note rendered at the end of a markdown file: + +```markdown +You can add footnotes to your text as follows.[^1] +[^1]: This is my awesome footnote (later in file). ``` + +You can add footnotes to your text as follows.[^1] + +[^1]: This is my awesome footnote (later in file). + +### Headers + +```markdown # H1 ## H2 ### H3 @@ -593,9 +758,11 @@ Alt-H2 #### Header IDs and links -All Markdown-rendered headers automatically get IDs, which can be linked to, except in comments. +GFM extends the standard markdown standard so that all Markdown-rendered headers automatically +get IDs, which can be linked to, except in comments. -On hover, a link to those IDs becomes visible to make it easier to copy the link to the header to use it somewhere else. +On hover, a link to those IDs becomes visible to make it easier to copy the link to +the header to use it somewhere else. The IDs are generated from the content of the header according to the following rules: @@ -606,7 +773,7 @@ The IDs are generated from the content of the header according to the following 1. If a header with the same ID has already been generated, a unique incrementing number is appended, starting at 1. -For example: +Example: ``` # This header has spaces in it @@ -626,215 +793,164 @@ Would generate the following link IDs: 1. `this-header-has-spaces-in-it-2` 1. `this-header-has-3-5-in-it-and-parentheses` -Note that the Emoji processing happens before the header IDs are generated, so the Emoji is converted to an image which then gets removed from the ID. - -### Emphasis - -Examples: - -``` -Emphasis, aka italics, with *asterisks* or _underscores_. - -Strong emphasis, aka bold, with **asterisks** or __underscores__. - -Combined emphasis with **asterisks and _underscores_**. - -Strikethrough uses two tildes. ~~Scratch this.~~ -``` - -Becomes: - -Emphasis, aka italics, with *asterisks* or _underscores_. - -Strong emphasis, aka bold, with **asterisks** or __underscores__. - -Combined emphasis with **asterisks and _underscores_**. - -Strikethrough uses two tildes. ~~Scratch this.~~ - -### Lists - -Examples: - -``` -1. First ordered list item -2. Another item - * Unordered sub-list. -1. Actual numbers don't matter, just that it's a number - 1. Ordered sub-list -4. And another item. - -* Unordered list can use asterisks -- Or minuses -+ Or pluses -``` - -Becomes: - -1. First ordered list item -2. Another item - * Unordered sub-list. -1. Actual numbers don't matter, just that it's a number - 1. Ordered sub-list -4. And another item. - -* Unordered list can use asterisks -- Or minuses -+ Or pluses - -If a list item contains multiple paragraphs, -each subsequent paragraph should be indented to the same level as the start of the list item text +Note that the Emoji processing happens before the header IDs are generated, so the +Emoji is converted to an image which is then removed from the ID. -Example: - -``` -1. First ordered list item - - Second paragraph of first item. - -2. Another item -``` +### Horizontal Rule -Becomes: +It's very simple to create a horizontal rule, by using three or more hyphens, asterisks, +or underscores: -1. First ordered list item +```markdown +Three or more hyphens, - Paragraph of first item. +--- -2. Another item +asterisks, -If the paragraph of the first item is not indented with the proper number of spaces, -the paragraph will appear outside the list, instead of properly indented under the list item. +*** -Example: +or underscores +___ ``` -1. First ordered list item - Paragraph of first item. +Three or more hyphens, -2. Another item -``` +--- -Becomes: +asterisks, -1. First ordered list item +*** - Paragraph of first item. +or underscores -2. Another item +___ -### Links +### Images -There are two ways to create links, inline-style and reference-style. +Examples: ```markdown -[I'm an inline-style link](https://www.google.com) -[I'm a link to a repository file in the same directory](index.md) -[I am an absolute reference within the repository](/doc/user/index.md) -[I'm a relative link to the Milestones page](../README.md) +Inline-style (hover to see title text): -[I link to a section on a different markdown page, using a header ID](index.md#overview) -[I link to a different section on the same page, using the header ID](#header-ids-and-links) + -[I'm a reference-style link][Arbitrary case-insensitive reference text] -[You can use numbers for reference-style link definitions][1] -Or leave it empty and use the [link text itself][] +Reference-style (hover to see title text): -Some text to show that the reference links can follow later. +![alt text1][logo] -[arbitrary case-insensitive reference text]: https://www.mozilla.org -[1]: http://slashdot.org -[link text itself]: https://www.reddit.com +[logo]: img/markdown_logo.png "Title Text" ``` ->**Note:** -Relative links do not allow referencing project files in a wiki page or wiki -page in a project file. The reason for this is that, in GitLab, wiki is always -a separate Git repository. For example, `[I'm a reference-style link](style)` -will point the link to `wikis/style` when the link is inside of a wiki markdown file. - -### Images +Inline-style (hover to see title text): -Examples: + - Here's our logo (hover to see the title text): +Reference-style (hover to see title text): - Inline-style: -  +![alt text][logo] - Reference-style: - ![alt text1][logo] +[logo]: img/markdown_logo.png "Title Text" - [logo]: img/markdown_logo.png +#### Videos -Becomes: +> If this is not rendered correctly, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#videos). -Here's our logo: +Image tags that link to files with a video extension are automatically converted to +a video player. The valid video extensions are `.mp4`, `.m4v`, `.mov`, `.webm`, and `.ogv`: -Inline-style: +```md +Here's a sample video: - + +``` -Reference-style: +Here's a sample video: -![alt text][logo] + -[logo]: img/markdown_logo.png +### Inline HTML -### Blockquotes +> To see the markdown rendered within HTML in the second example, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#inline-html). -Examples: +You can also use raw HTML in your Markdown, and it'll usually work pretty well. -``` -> Blockquotes are very handy in email to emulate reply text. -> This line is part of the same quote. +See the documentation for HTML::Pipeline's [SanitizationFilter](http://www.rubydoc.info/gems/html-pipeline/1.11.0/HTML/Pipeline/SanitizationFilter#WHITELIST-constant) +class for the list of allowed HTML tags and attributes. In addition to the default +`SanitizationFilter` whitelist, GitLab allows `span`, `abbr`, `details` and `summary` elements. -Quote break. +```html +<dl> + <dt>Definition list</dt> + <dd>Is something people use sometimes.</dd> -> This is a very long line that will still be quoted properly when it wraps. Oh boy let's keep writing to make sure this is long enough to actually wrap for everyone. Oh, you can *put* **Markdown** into a blockquote. + <dt>Markdown in HTML</dt> + <dd>Does *not* work **very** well. HTML <em>tags</em> will <b>always</b> work.</dd> +</dl> ``` -Becomes: - -> Blockquotes are very handy in email to emulate reply text. -> This line is part of the same quote. - -Quote break. - -> This is a very long line that will still be quoted properly when it wraps. Oh boy let's keep writing to make sure this is long enough to actually wrap for everyone. Oh, you can *put* **Markdown** into a blockquote. - -### Inline HTML +<dl> + <dt>Definition list</dt> + <dd>Is something people use sometimes.</dd> -You can also use raw HTML in your Markdown, and it'll mostly work pretty well. + <dt>Markdown in HTML</dt> + <dd>Does *not* work **very** well. HTML <em>tags</em> will <b>always</b> work.</dd> +</dl> -See the documentation for HTML::Pipeline's [SanitizationFilter](http://www.rubydoc.info/gems/html-pipeline/1.11.0/HTML/Pipeline/SanitizationFilter#WHITELIST-constant) class for the list of allowed HTML tags and attributes. In addition to the default `SanitizationFilter` whitelist, GitLab allows `span`, `abbr`, `details` and `summary` elements. +--- -Examples: +It is still possible to use markdown inside HTML tags, but only if the lines containing markdown +are separated into their own lines: -``` +```html <dl> - <dt>Definition list</dt> - <dd>Is something people use sometimes.</dd> + <dt>Markdown in HTML</dt> + <dd>Does *not* work **very** well. HTML tags will always work.</dd> <dt>Markdown in HTML</dt> - <dd>Does *not* work **very** well. Use HTML <em>tags</em>.</dd> + <dd> + + Does *not* work **very** well. HTML tags will always work. + + </dd> </dl> ``` -Becomes: +<!-- Note: The example below uses HTML to force correct rendering on docs.gitlab.com, markdown will be fine in GitLab --> <dl> - <dt>Definition list</dt> - <dd>Is something people use sometimes.</dd> + <dt>Markdown in HTML</dt> + <dd>Does *not* work **very** well. HTML tags will always work.</dd> <dt>Markdown in HTML</dt> - <dd>Does *not* work **very** well. Use HTML <em>tags</em>.</dd> + <dd> + + Does <em>not</em> work <b>very</b> well. HTML tags will always work. + + </dd> </dl> #### Details and Summary -Content can be collapsed using HTML's [`<details>`](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/details) and [`<summary>`](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/summary) tags. This is especially useful for collapsing long logs so they take up less screen space. +> To see the markdown rendered within HTML in the second example, [view it in GitLab itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#details-and-summary). + +Content can be collapsed using HTML's [`<details>`](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/details) +and [`<summary>`](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/summary) +tags. This is especially useful for collapsing long logs so they take up less screen space. + +```html +<p> +<details> +<summary>Click me to collapse/fold.</summary> + +These details <em>will</em> remain <strong>hidden</strong> until expanded. + +<pre><code>PASTE LOGS HERE</code></pre> + +</details> +</p> +``` <p> <details> @@ -847,7 +963,10 @@ These details <em>will</em> remain <strong>hidden</strong> until expanded. </details> </p> -**Note:** Markdown inside these tags is supported, as long as you have a blank line after the `</summary>` tag and before the `</details>` tag, as shown in the example. +--- + +Markdown inside these tags is supported as well, as long as you have a blank line +after the `</summary>` tag and before the `</details>` tag, as shown in the example: ```html <details> @@ -860,232 +979,302 @@ These details _will_ remain **hidden** until expanded. </details> ``` -### Horizontal Rule +<!-- Note: The example below uses HTML to force correct rendering on docs.gitlab.com, markdown will be fine in GitLab --> -Examples: +<details> +<summary>Click me to collapse/fold.</summary> -``` -Three or more... +These details <em>will</em> remain <b>hidden</b> until expanded. ---- + PASTE LOGS HERE + +</details> -Hyphens +### Line Breaks -*** +A line break will be inserted (a new paragraph will start) if the previous text is +ended with two newlines, i.e. you hit <kbd>Enter</kbd> twice in a row. If you only +use one newline (hit <kbd>Enter</kbd> once), the next sentence will be part of the +same paragraph. This is useful if you want to keep long lines from wrapping, and keep +them easily editable: -Asterisks +```markdown +Here's a line for us to start with. -___ +This longer line is separated from the one above by two newlines, so it will be a *separate paragraph*. -Underscores +This line is also a separate paragraph, but... +These lines are only separated by single newlines, +so they *do not break* and just follow the previous lines +in the *same paragraph*. ``` -Becomes: +Here's a line for us to start with. -Three or more... +This longer line is separated from the one above by two newlines, so it will be a *separate paragraph*. ---- +This line is also a separate paragraph, but... +These lines are only separated by single newlines, +so they *do not break* and just follow the previous lines +in the *same paragraph*. -Hyphens +#### Newlines -*** +GFM adheres to the markdown specification in how [paragraphs and line breaks are handled](https://spec.commonmark.org/current/). -Asterisks +A paragraph is simply one or more consecutive lines of text, separated by one or +more blank lines (i.e. two newlines at the end of the first paragraph), as [explained above](#line-breaks). -___ +If you need more control over line-breaks or soft returns, you can add a single line-break +by ending a line with a backslash, or two or more spaces. Two newlines in a row will create a new +paragraph, with a blank line in between: -Underscores +```markdown +First paragraph. +Another line in the same paragraph. +A third line in the same paragraph, but this time ending with two spaces.{space}{space} +A new line directly under the first paragraph. + +Second paragraph. +Another line, this time ending with a backslash.\ +A new line due to the previous backslash. +``` -### Line Breaks +<!-- (Do *NOT* remove the two ending whitespaces in the third line) --> +<!-- (They are needed for the Markdown text to render correctly) --> -A good way to learn how line breaks work is to experiment and discover -- hit <kbd>Enter</kbd> once (i.e., insert one newline), then hit it twice (i.e., insert two newlines), see what happens. You'll soon learn to get what you want. The "Preview" tab is your friend. +First paragraph. +Another line in the same paragraph. +A third line in the same paragraph, but this time ending with two spaces. +A new line directly under the first paragraph. -Here are some things to try out: +<!-- (Do *NOT* remove the two ending whitespaces in the second line) --> +<!-- (They are needed for the Markdown text to render correctly on docs.gitlab.com, the backslash works fine inside GitLab itself) --> -Examples: +Second paragraph. +Another line, this time ending with a backslash. +A new line due to the previous backslash. -``` -Here's a line for us to start with. +### Links -This line is separated from the one above by two newlines, so it will be a *separate paragraph*. +There are two ways to create links, inline-style and reference-style: -This line is also a separate paragraph, but... -This line is only separated by a single newline, so it *does not break* and just follows the previous line in the *same paragraph*. +```md +- This is an [inline-style link](https://www.google.com) +- This is a [link to a repository file in the same directory](index.md) +- This is a [relative link to a readme one directory higher](../README.md) +- This is a [link that also has title text](https://www.google.com "This link takes you to Google!") -This line is also a separate paragraph, and... -This line is *on its own line*, because the previous line ends with two spaces. (but still in the *same paragraph*) +Using header ID anchors: -spaces. -``` +- This links to [a section on a different markdown page, using a "#" and the header ID](index.md#overview) +- This links to [a different section on the same page, using a "#" and the header ID](#header-ids-and-links) -Becomes: +Using references: -Here's a line for us to start with. +- This is a [reference-style link, see below][Arbitrary case-insensitive reference text] +- You can [use numbers for reference-style link definitions, see below][1] +- Or leave it empty and use the [link text itself][], see below. -This line is separated from the one above by two newlines, so it will be a *separate paragraph*. +Some text to show that the reference links can follow later. -This line is also a separate paragraph, but... -This line is only separated by a single newline, so it *does not break* and just follows the previous line in the *same paragraph*. +[arbitrary case-insensitive reference text]: https://www.mozilla.org +[1]: http://slashdot.org +[link text itself]: https://www.reddit.com +``` -This line is also a separate paragraph, and... -This line is *on its own line*, because the previous line ends with two spaces. (but still in the *same paragraph*) +- This is an [inline-style link](https://www.google.com) +- This is a [link to a repository file in the same directory](index.md) +- This is a [relative link to a readme one directory higher](../README.md) +- This is a [link that also has title text](https://www.google.com "This link takes you to Google!") -spaces. +Using header ID anchors: -### Tables +- This links to [a section on a different markdown page, using a "#" and the header ID](index.md#overview) +- This links to [a different section on the same page, using a "#" and the header ID](#header-ids-and-links) -Tables aren't part of the core Markdown spec, but they are part of GFM. +Using references: -Example: +- This is a [reference-style link, see below][Arbitrary case-insensitive reference text] +- You can [use numbers for reference-style link definitions, see below][1] +- Or leave it empty and use the [link text itself][], see below. -``` -| header 1 | header 2 | -| -------- | -------- | -| cell 1 | cell 2 | -| cell 3 | cell 4 | -``` - -Becomes: +Some text to show that the reference links can follow later. -| header 1 | header 2 | -| -------- | -------- | -| cell 1 | cell 2 | -| cell 3 | cell 4 | +[arbitrary case-insensitive reference text]: https://www.mozilla.org +[1]: http://slashdot.org +[link text itself]: https://www.reddit.com -**Note:** The row of dashes between the table header and body must have at least three dashes in each column. +NOTE: **Note:** Relative links do not allow the referencing of project files in a wiki +page, or a wiki page in a project file. The reason for this is that a wiki is always +in a separate Git repository in GitLab. For example, `[I'm a reference-style link](style)` +will point the link to `wikis/style` only when the link is inside of a wiki markdown file. -By including colons in the header row, you can align the text within that column. +#### URL auto-linking -Example: +GFM will autolink almost any URL you put into your text: -``` -| Left Aligned | Centered | Right Aligned | Left Aligned | Centered | Right Aligned | -| :----------- | :------: | ------------: | :----------- | :------: | ------------: | -| Cell 1 | Cell 2 | Cell 3 | Cell 4 | Cell 5 | Cell 6 | -| Cell 7 | Cell 8 | Cell 9 | Cell 10 | Cell 11 | Cell 12 | +```markdown +* https://www.google.com +* https://google.com/ +* ftp://ftp.us.debian.org/debian/ +* smb://foo/bar/baz +* irc://irc.freenode.net/gitlab +* http://localhost:3000 ``` -Becomes: +* https://www.google.com +* https://google.com/ +* ftp://ftp.us.debian.org/debian/ +* smb://foo/bar/baz +* irc://irc.freenode.net/gitlab +* http://localhost:3000 -| Left Aligned | Centered | Right Aligned | Left Aligned | Centered | Right Aligned | -| :----------- | :------: | ------------: | :----------- | :------: | ------------: | -| Cell 1 | Cell 2 | Cell 3 | Cell 4 | Cell 5 | Cell 6 | -| Cell 7 | Cell 8 | Cell 9 | Cell 10 | Cell 11 | Cell 12 | +### Lists -### Footnotes +Ordered and unordered lists can be easily created. Add the number you want the list +to start with, like `1. ` (with a space) at the start of each line for ordered lists. +After the first number, it does not matter what number you use, ordered lists will be +numbered automatically by vertical order, so repeating `1. ` for all items in the +same list is common. If you start with a number other than `1. `, it will use that as the first +number, and count up from there. -Example: +Add a `* `, `- ` or `+ ` (with a space) at the start of each line for unordered lists, but +you should not use a mix of them. -``` -You can add footnotes to your text as follows.[^2] -[^2]: This is my awesome footnote. +Examples: + +```md +1. First ordered list item +2. Another item + * Unordered sub-list. +1. Actual numbers don't matter, just that it's a number + 1. Ordered sub-list + 1. Next ordered sub-list item +4. And another item. + +* Unordered lists can use asterisks +- Or minuses ++ Or pluses ``` -Becomes: +1. First ordered list item +2. Another item + * Unordered sub-list. +1. Actual numbers don't matter, just that it's a number + 1. Ordered sub-list + 1. Next ordered sub-list item +4. And another item. -You can add footnotes to your text as follows.[^2] +* Unordered lists can use asterisks +- Or minuses ++ Or pluses -### Superscripts / Subscripts +--- -CommonMark and GFM currently do not support the superscript syntax ( `x^2` ) that Redcarpet does. You can use the standard HTML syntax for superscripts and subscripts. +If a list item contains multiple paragraphs, each subsequent paragraph should be indented +to the same level as the start of the list item text. -``` -The formula for water is H<sub>2</sub>O -while the equation for the theory of relativity is E = mc<sup>2</sup>. -``` +Example: -The formula for water is H<sub>2</sub>O while the equation for the theory of relativity is E = mc<sup>2</sup>. +```markdown +1. First ordered list item -## Wiki-specific Markdown + Second paragraph of first item. -The following examples show how links inside wikis behave. +2. Another item +``` -### Wiki - Direct page link +1. First ordered list item -A link which just includes the slug for a page will point to that page, -_at the base level of the wiki_. + Second paragraph of first item. -This snippet would link to a `documentation` page at the root of your wiki: +2. Another item -```markdown -[Link to Documentation](documentation) -``` +--- -### Wiki - Direct file link +If the paragraph of the first item is not indented with the proper number of spaces, +the paragraph will appear outside the list, instead of properly indented under the list item. -Links with a file extension point to that file, _relative to the current page_. +Example: -If this snippet was placed on a page at `<your_wiki>/documentation/related`, -it would link to `<your_wiki>/documentation/file.md`: +``` +1. First ordered list item -```markdown -[Link to File](file.md) + Paragraph of first item. + +2. Another item ``` -### Wiki - Hierarchical link +1. First ordered list item -A link can be constructed relative to the current wiki page using `./<page>`, -`../<page>`, etc. + Paragraph of first item. -- If this snippet was placed on a page at `<your_wiki>/documentation/main`, - it would link to `<your_wiki>/documentation/related`: +2. Another item - ```markdown - [Link to Related Page](related) - ``` +### Superscripts / Subscripts -- If this snippet was placed on a page at `<your_wiki>/documentation/related/content`, - it would link to `<your_wiki>/documentation/main`: +CommonMark and GFM currently do not support the superscript syntax ( `x^2` ) that +Redcarpet does. You can use the standard HTML syntax for superscripts and subscripts: - ```markdown - [Link to Related Page](../main) - ``` +```html +The formula for water is H<sub>2</sub>O +while the equation for the theory of relativity is E = mc<sup>2</sup>. +``` -- If this snippet was placed on a page at `<your_wiki>/documentation/main`, - it would link to `<your_wiki>/documentation/related.md`: +The formula for water is H<sub>2</sub>O +while the equation for the theory of relativity is E = mc<sup>2</sup>. - ```markdown - [Link to Related Page](related.md) - ``` +### Tables -- If this snippet was placed on a page at `<your_wiki>/documentation/related/content`, - it would link to `<your_wiki>/documentation/main.md`: +Tables aren't part of the core Markdown spec, but they are part of GFM. - ```markdown - [Link to Related Page](../main.md) - ``` +1. The first line contains the headers, separated by "pipes" (`|`). +1. The second line separates the headers from the cells, and must contain three or more dashes. +1. The third, and any following lines, contain the cell values. + - You **can't** have cells separated over many lines in the markdown, they must be kept to single lines, + but they can be very long. You can also include HTML `<br>` tags to force newlines if needed. + - The cell sizes **don't** have to match each other. They are flexible, but must be separated + by pipes (`|`). + - You **can** have blank cells. -### Wiki - Root link +Example: -A link starting with a `/` is relative to the wiki root. +```markdown +| header 1 | header 2 | header 3 | +| --- | ------ |----------| +| cell 1 | cell 2 | cell 3 | +| cell 4 | cell 5 is longer | cell 6 is much longer than the others, but that's ok. It will eventually wrap the text when the cell is too large for the display size. | +| cell 7 | | cell <br> 9 | +``` -- This snippet links to `<wiki_root>/documentation`: +| header 1 | header 2 | header 3 | +| --- | ------ |---------:| +| cell 1 | cell 2 | cell 3 | +| cell 4 | cell 5 is longer | cell 6 is much longer than the others, but that's ok. It will eventually wrap the text when the cell is too large for the display size. | +| cell 7 | | cell <br> 9 | - ```markdown - [Link to Related Page](/documentation) - ``` +Additionally, you can choose the alignment of text within columns by adding colons (`:`) +to the sides of the "dash" lines in the second row. This will affect every cell in the column. -- This snippet links to `<wiki_root>/miscellaneous.md`: +> Note that the headers are always right aligned [within GitLab itself itself](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md#tables). - ```markdown - [Link to Related Page](/miscellaneous.md) - ``` +```markdown +| Left Aligned | Centered | Right Aligned | Left Aligned | Centered | Right Aligned | +| :--- | :---: | ---: | :----------- | :------: | ------------: | +| Cell 1 | Cell 2 | Cell 3 | Cell 4 | Cell 5 | Cell 6 | +| Cell 7 | Cell 8 | Cell 9 | Cell 10 | Cell 11 | Cell 12 | +``` + +| Left Aligned | Centered | Right Aligned | Left Aligned | Centered | Right Aligned | +| :--- | :---: | ---: | :----------- | :------: | ------------: | +| Cell 1 | Cell 2 | Cell 3 | Cell 4 | Cell 5 | Cell 6 | +| Cell 7 | Cell 8 | Cell 9 | Cell 10 | Cell 11 | Cell 12 | ## References - This document leveraged heavily from the [Markdown-Cheatsheet](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet). -- The original [Markdown Syntax Guide](https://daringfireball.net/projects/markdown/syntax) at Daring Fireball is an excellent resource for a detailed explanation of standard markdown. -- The detailed specification for CommonMark can be found in the [CommonMark Spec][commonmark-spec] +- The original [Markdown Syntax Guide](https://daringfireball.net/projects/markdown/syntax) + at Daring Fireball is an excellent resource for a detailed explanation of standard markdown. +- The detailed specification for CommonMark can be found in the [CommonMark Spec](https://spec.commonmark.org/current/) - The [CommonMark Dingus](http://try.commonmark.org) is a handy tool for testing CommonMark syntax. - -[^1]: This link will be broken if you see this document from the Help page or docs.gitlab.com -[^2]: This is my awesome footnote. - -[markdown.md]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/user/markdown.md -[rouge]: http://rouge.jneen.net/ "Rouge website" -[redcarpet]: https://github.com/vmg/redcarpet "Redcarpet website" -[katex]: https://github.com/Khan/KaTeX "KaTeX website" -[katex-subset]: https://katex.org/docs/supported.html "Macros supported by KaTeX" -[asciidoctor-manual]: http://asciidoctor.org/docs/user-manual/#activating-stem-support "Asciidoctor user manual" -[commonmarker]: https://github.com/gjtorikian/commonmarker -[commonmark-spec]: https://spec.commonmark.org/current/ diff --git a/doc/user/operations_dashboard/index.md b/doc/user/operations_dashboard/index.md index 66362f27299..54bf3ff8a40 100644 --- a/doc/user/operations_dashboard/index.md +++ b/doc/user/operations_dashboard/index.md @@ -1,9 +1,6 @@ # Operations Dashboard **[PREMIUM]** -> [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/5781) -in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.5. -[Moved](https://gitlab.com/gitlab-org/gitlab-ee/issues/9218) to -[GitLab Premium](https://about.gitlab.com/pricing/) in 11.10. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/5781) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.5. [Moved](https://gitlab.com/gitlab-org/gitlab-ee/issues/9218) to [GitLab Premium](https://about.gitlab.com/pricing/) in 11.10. The Operations Dashboard provides a summary of each project's operational health, including pipeline and alert status. @@ -16,9 +13,9 @@ dashboard icon: ## Adding a project to the dashboard NOTE: **Note:** -For GitLab.com, the Operations Dashboard is available for free for public projects. -If your project is private, the group it belongs to must have a -[Gold](https://about.gitlab.com/pricing/) plan. +For GitLab.com, you can add your project to the Operations Dashboard for free if +your project is public. If your project is private, the group it belongs to must +have a [Silver](https://about.gitlab.com/pricing/) plan. To add a project to the dashboard: diff --git a/doc/user/profile/account/two_factor_authentication.md b/doc/user/profile/account/two_factor_authentication.md index df413a11af0..26cacbe5545 100644 --- a/doc/user/profile/account/two_factor_authentication.md +++ b/doc/user/profile/account/two_factor_authentication.md @@ -84,9 +84,8 @@ Click on **Register U2F Device** to complete the process. > **Note:** Recovery codes are not generated for U2F devices. -Should you ever lose access to your one time password authenticator, you can use one of the ten provided -backup codes to login to your account. We suggest copying them, printing them, or downloading them using -the **Download codes** button for storage in a safe place. +Immediately after successfully enabling two-factor authentication, you'll be prompted to download a set of set recovery codes. Should you ever lose access to your one time password authenticator, you can use one of them to log in to your account. We suggest copying them, printing them, or downloading them using +the **Download codes** button for storage in a safe place. If you choose to download them, the file will be called **gitlab-recovery-codes.txt**. CAUTION: **Caution:** Each code can be used only once to log in to your account. diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index a4d4fb91f71..97d2dfc0f7e 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -519,10 +519,11 @@ service account of the cluster integration. ### Troubleshooting failed deployment jobs -GitLab will create a namespace and service account specifically for your -deployment jobs. On project level clusters, this happens when the cluster -is created. On group level clusters, resources are created immediately -before the deployment job starts. +Before the deployment jobs starts, GitLab creates the following specifically for +the deployment job: + +- A namespace. +- A service account. However, sometimes GitLab can not create them. In such instances, your job will fail with the message: @@ -532,6 +533,14 @@ This job failed because the necessary resources were not successfully created. To find the cause of this error when creating a namespace and service account, check the [logs](../../../administration/logs.md#kuberneteslog). +NOTE: **NOTE:** +As of GitLab 12.1 we require [`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) +tokens for all project level clusters unless you unselect the +[GitLab-managed cluster](#gitlab-managed-clusters) option. If you +want to manage namespaces and service accounts yourself and don't +want to provide a `cluster-admin` token to GitLab you must unselect this +option or you will get the above error. + Common reasons for failure include: - The token you gave GitLab did not have [`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) diff --git a/doc/user/project/clusters/kubernetes_pod_logs.md b/doc/user/project/clusters/kubernetes_pod_logs.md index 368031070c1..25d8abebf07 100644 --- a/doc/user/project/clusters/kubernetes_pod_logs.md +++ b/doc/user/project/clusters/kubernetes_pod_logs.md @@ -12,9 +12,9 @@ By displaying the logs directly in GitLab, developers can avoid having to manage 1. Go to **Operations > Environments** and find the environment which contains the desired pod, like `production`. 1. On the **Environments** page, you should see the status of the environment's pods with [Deploy Boards](../deploy_boards.md). 1. When mousing over the list of pods, a tooltip will appear with the exact pod name and status. - +  1. Click on the desired pod to bring up the logs view, which will contain the last 500 lines for that pod. Support for pods with multiple containers is coming [in a future release](https://gitlab.com/gitlab-org/gitlab-ee/issues/6502). - +  ## Requirements diff --git a/doc/user/project/clusters/serverless/index.md b/doc/user/project/clusters/serverless/index.md index 0ab6d8e112f..3be5bfeaddc 100644 --- a/doc/user/project/clusters/serverless/index.md +++ b/doc/user/project/clusters/serverless/index.md @@ -85,12 +85,22 @@ on a given project but not both. The current implementation makes use of a `serv ## Using an existing installation of Knative +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/58941) in GitLab 12.0. + NOTE: **Note:** -The "invocations" monitoring feature of GitLab serverless will not work when adding an existing installation of Knative. +The "invocations" monitoring feature of GitLab serverless will not work when +adding an existing installation of Knative. + +It is also possible to use GitLab Serverless with an existing Kubernetes +cluster which already has Knative installed. + +Simply: -It is also possible to use GitLab Serverless with an existing Kubernetes cluster which already has Knative installed. -Simply follow the steps to [add an existing Kubernetes cluster](../index.md#adding-an-existing-kubernetes-cluster) -and then follow the steps to deploy [functions](#deploying-functions) or [serverless applications](#deploying-serverless-applications) onto your cluster. +1. Follow the steps to + [add an existing Kubernetes cluster](../index.md#adding-an-existing-kubernetes-cluster). +1. Follow the steps to deploy [functions](#deploying-functions) + or [serverless applications](#deploying-serverless-applications) onto your + cluster. ## Deploying functions diff --git a/doc/user/project/container_registry.md b/doc/user/project/container_registry.md index 58b7fe33906..fdf9ce3e225 100644 --- a/doc/user/project/container_registry.md +++ b/doc/user/project/container_registry.md @@ -113,6 +113,7 @@ This feature requires GitLab 8.8 and GitLab Runner 1.2. Make sure that your GitLab Runner is configured to allow building Docker images by following the [Using Docker Build](../../ci/docker/using_docker_build.md) and [Using the GitLab Container Registry documentation](../../ci/docker/using_docker_build.md#using-the-gitlab-container-registry). +Alternatively, you can [build images with Kaniko](../../ci/docker/using_kaniko.md) if the Docker builds are not an option for you. ## Using with private projects diff --git a/doc/user/project/deploy_boards.md b/doc/user/project/deploy_boards.md index b7f6a855cb6..175384bc985 100644 --- a/doc/user/project/deploy_boards.md +++ b/doc/user/project/deploy_boards.md @@ -59,6 +59,8 @@ specific environment, there are lot of uses cases. To name a few: To display the Deploy Boards for a specific [environment] you should: +1. Have [defined an environment](../../ci/environments.md#defining-environments) with a deploy stage. + 1. Have a Kubernetes cluster up and running. NOTE: **Running on OpenShift:** @@ -86,7 +88,10 @@ To display the Deploy Boards for a specific [environment] you should: Kubernetes. NOTE: **Note:** - Matching based on the Kubernetes `app` label was removed in [GitLab 12.1](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/14020) + Matching based on the Kubernetes `app` label was removed in [GitLab + 12.1](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/14020). + To migrate, please apply the required annotations (see above) and + re-deploy your application.  diff --git a/doc/user/project/description_templates.md b/doc/user/project/description_templates.md index 05ad15476ab..7520237251a 100644 --- a/doc/user/project/description_templates.md +++ b/doc/user/project/description_templates.md @@ -25,14 +25,14 @@ templates of the default branch will be taken into account. ## Use-cases - Add a template to be used in every issue for a specific project, -giving instructions and guidelines, requiring for information specific to that subject. -For example, if you have a project for tracking new blog posts, you can require the -title, outlines, author name, author social media information, etc. + giving instructions and guidelines, requiring for information specific to that subject. + For example, if you have a project for tracking new blog posts, you can require the + title, outlines, author name, author social media information, etc. - Following the previous example, you can make a template for every MR submitted -with a new blog post, requiring information about the post date, frontmatter data, -images guidelines, link to the related issue, reviewer name, etc. + with a new blog post, requiring information about the post date, frontmatter data, + images guidelines, link to the related issue, reviewer name, etc. - You can also create issues and merge request templates for different -stages of your workflow, e.g., feature proposal, feature improvement, bug report, etc. + stages of your workflow, e.g., feature proposal, feature improvement, bug report, etc. ## Creating issue templates @@ -58,6 +58,7 @@ changes you made after picking the template and return it to its initial status. ## Setting a default template for issues and merge requests **[STARTER]** > **Notes:** +> > - This feature was introduced before [description templates](#overview) and is > available in [GitLab Starter][products]. It can be enabled > in the project's settings. @@ -65,7 +66,7 @@ changes you made after picking the template and return it to its initial status. > - Templates for merge requests were [introduced][ee-7478ece] in GitLab EE 6.9. The visibility of issues and/or merge requests should be set to either "Everyone -with access" or "Only team members" in your project's **Settings** otherwise the +with access" or "Only Project Members" in your project's **Settings** otherwise the template text areas won't show. This is the default behavior so in most cases you should be fine. diff --git a/doc/user/project/integrations/hipchat.md b/doc/user/project/integrations/hipchat.md index 0fd847d415f..7a0540aa9e3 100644 --- a/doc/user/project/integrations/hipchat.md +++ b/doc/user/project/integrations/hipchat.md @@ -23,7 +23,7 @@ allow GitLab to send messages only to *one* room. 1. Find "Build Your Own!" and click "Create". 1. Select the desired room, name the integration "GitLab", and click "Create". 1. In the "Send messages to this room by posting this URL" column, you should -see a URL in the format: + see a URL in the format: ``` https://api.hipchat.com/v2/room/<room>/notification?auth_token=<token> diff --git a/doc/user/project/integrations/prometheus.md b/doc/user/project/integrations/prometheus.md index 751e8e44e60..aab7131e353 100644 --- a/doc/user/project/integrations/prometheus.md +++ b/doc/user/project/integrations/prometheus.md @@ -160,7 +160,7 @@ receivers: > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/4925) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.11. -Alerts can be used to trigger actions, like open an issue automatically. To configure the actions: +Alerts can be used to trigger actions, like open an issue automatically (enabled by default since `12.1`). To configure the actions: 1. Navigate to your project's **Settings > Operations > Incidents**. 1. Enable the option to create issues. diff --git a/doc/user/project/issues/index.md b/doc/user/project/issues/index.md index 76dc6e49bce..4acbb4cc3f6 100644 --- a/doc/user/project/issues/index.md +++ b/doc/user/project/issues/index.md @@ -134,4 +134,4 @@ For more information, see [Crosslinking issues](crosslinking_issues.md). - [Export issues](csv_export.md) **[STARTER]** - [Issues API](../../../api/issues.md) - Configure an [external issue tracker](../../../integration/external-issue-tracker.md) such as Jira, Redmine, -or Bugzilla. + or Bugzilla. diff --git a/doc/user/project/issues/issue_data_and_actions.md b/doc/user/project/issues/issue_data_and_actions.md index ac26b672d99..2103f331aa2 100644 --- a/doc/user/project/issues/issue_data_and_actions.md +++ b/doc/user/project/issues/issue_data_and_actions.md @@ -149,12 +149,19 @@ The plain text title and description of the issue fill the top center of the iss The description fully supports [GitLab Flavored Markdown](../../markdown.md#gitlab-flavored-markdown-gfm), allowing many formatting options. -##### 16.1 Zoom Call Links +##### Zoom call links -Including a link to a Zoom call in the description of an issue will result in a "Join Zoom meeting" button at the top of the issue, just under the header. To remove the button, edit the description and remove the Zoom call link. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/62966) in GitLab 12.0. + +Including a link to a [Zoom](https://zoom.us) call in the description of an issue +results in a **Join Zoom meeting** button at the top of the issue, just under the header. + +For example:  +To remove the button, edit the description and remove the Zoom call link. + #### 17. Mentions You can mention a user or a group present in your GitLab instance with `@username` or diff --git a/doc/user/project/merge_requests/merge_request_approvals.md b/doc/user/project/merge_requests/merge_request_approvals.md index fd151a6df45..8e8ec26daf2 100644 --- a/doc/user/project/merge_requests/merge_request_approvals.md +++ b/doc/user/project/merge_requests/merge_request_approvals.md @@ -12,9 +12,9 @@ to approve a merge request before it can be unblocked for merging. ## Use cases 1. Enforcing review of all code that gets merged into a repository. -2. Specifying code maintainers for an entire repository. -3. Specifying reviewers for a given proposed code change. -4. Specifying categories of reviewers, such as BE, FE, QA, DB, etc., for all proposed code changes. +1. Specifying code maintainers for an entire repository. +1. Specifying reviewers for a given proposed code change. +1. Specifying categories of reviewers, such as BE, FE, QA, DB, etc., for all proposed code changes. ## Enabling the new approvals interface @@ -246,7 +246,7 @@ restrictions (compared to [GitLab Starter](#overriding-the-merge-request-approva - Approval rules can be added to an MR with no restriction. - For project sourced approval rules, editing and removing approvers is not allowed. - The approvals required of all approval rules is configurable, but if a rule is backed by a project rule, then it is restricted -to the minimum approvals required set in the project's corresponding rule. + to the minimum approvals required set in the project's corresponding rule. ## Resetting approvals on push diff --git a/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md b/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md index c93c7a5fe08..0dd60d84c42 100644 --- a/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md +++ b/doc/user/project/merge_requests/merge_when_pipeline_succeeds.md @@ -42,6 +42,8 @@ Navigate to your project's settings page and expand the **Merge requests** secti In the **Merge checks** subsection, select the **Pipelines must succeed** check box and hit **Save** for the changes to take effect. +NOTE: **Note:** This setting also prevents merge requests from being merged if there is no pipeline. +  From now on, every time the pipeline fails you will not be able to merge the @@ -49,6 +51,21 @@ merge request from the UI, until you make all relevant jobs pass.  +### Limitations + +When this setting is enabled, a merge request is prevented from being merged if there is no pipeline. This may conflict with some use cases where [`only/except`](../../../ci/yaml/README.md#onlyexcept-advanced) rules are used and they don't generate any pipelines. + +Users that expect to be able to merge a merge request in this scenario should ensure that [there is always a pipeline](https://gitlab.com/gitlab-org/gitlab-ce/issues/54226) and that it's succesful. + +For example, to that on merge requests there is always a passing job even though `only/except` rules may not generate any other jobs: + +```yaml +enable_merge: + only: merge_requests + script: + - echo true +``` + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/user/project/pages/getting_started_part_three.md b/doc/user/project/pages/getting_started_part_three.md index d585c19fc5c..bc9a11504cd 100644 --- a/doc/user/project/pages/getting_started_part_three.md +++ b/doc/user/project/pages/getting_started_part_three.md @@ -1,5 +1,5 @@ --- -last_updated: 2019-06-04 +last_updated: 2019-06-25 type: concepts, reference, howto --- @@ -138,9 +138,9 @@ verify your domain's ownership with a TXT record: > - **Do not** add any special chars after the default Pages domain. E.g., **do not** point your `subdomain.domain.com` to `namespace.gitlab.io.` or `namespace.gitlab.io/`. -> - GitLab Pages IP on GitLab.com [was changed](https://about.gitlab.com/2017/03/06/we-are-changing-the-ip-of-gitlab-pages-on-gitlab-com/) in 2017 +> - GitLab Pages IP on GitLab.com [was changed](https://about.gitlab.com/2017/03/06/we-are-changing-the-ip-of-gitlab-pages-on-gitlab-com/) in 2017. > - GitLab Pages IP on GitLab.com [has been changed](https://about.gitlab.com/2018/07/19/gcp-move-update/#gitlab-pages-and-custom-domains) - from `52.167.214.135` to `35.185.44.232` in 2018 + from `52.167.214.135` to `35.185.44.232` in 2018. ### Add your custom domain to GitLab Pages settings @@ -199,7 +199,7 @@ Certificates are NOT required to add to your custom highly recommendable. Let's start with an introduction to the importance of HTTPS. -Alternatively, jump ahead to [adding certificates to your project](#adding-certificates-to-your-project). +Alternatively, jump ahead to [adding certificates to your project](#adding-certificates-to-pages). ### Why should I care about HTTPS? @@ -255,12 +255,12 @@ which also offers a [free CDN service](https://blog.cloudflare.com/cloudflares-f Their certs are valid up to 15 years. See the tutorial on [how to add a CloudFlare Certificate to your GitLab Pages website](https://about.gitlab.com/2017/02/07/setting-up-gitlab-pages-with-cloudflare-certificates/). -### Adding certificates to your project +### Adding certificates to Pages Regardless the CA you choose, the steps to add your certificate to your Pages project are the same. -### What do you need +#### Requirements 1. A PEM certificate 1. An intermediate certificate @@ -270,7 +270,7 @@ your Pages project are the same. These fields are found under your **Project**'s **Settings** > **Pages** > **New Domain**. -### What's what? +#### Certificate types - A PEM certificate is the certificate generated by the CA, which needs to be added to the field **Certificate (PEM)**. @@ -283,21 +283,32 @@ These fields are found under your **Project**'s **Settings** > **Pages** > **New - A private key is an encrypted key which validates your PEM against your domain. -### Now what? +#### Add the certificate to your project -Now that you hopefully understand why you need all -of this, it's simple: +Once you've met the requirements: -- Your PEM certificate needs to be added to the first field +- Your PEM certificate needs to be added to the first field. - If your certificate is missing its intermediate, copy and paste the root certificate (usually available from your CA website) and paste it in the [same field as your PEM certificate](https://about.gitlab.com/2017/02/07/setting-up-gitlab-pages-with-cloudflare-certificates/), just jumping a line between them. -- Copy your private key and paste it in the last field +- Copy your private key and paste it in the last field. ->**Note:** +NOTE: **Note:** **Do not** open certificates or encryption keys in regular text editors. Always use code editors (such as Sublime Text, Atom, Dreamweaver, Brackets, etc). -_Read on about [Creating and Tweaking GitLab CI/CD for GitLab Pages](getting_started_part_four.md)_ +## Force HTTPS for GitLab Pages websites + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/28857) in GitLab 10.7. + +To make your website's visitors even more secure, you can choose to +force HTTPS for GitLab Pages. By doing so, all attempts to visit your +website via HTTP will be automatically redirected to HTTPS via 301. + +It works with both GitLab's default domain and with your custom +domain (as long as you've set a valid certificate for it). + +To enable this setting, navigate to your project's **Settings > Pages** +and tick the checkbox **Force HTTPS (requires valid certificates)**. diff --git a/doc/user/project/pages/getting_started_part_two.md b/doc/user/project/pages/getting_started_part_two.md index 3e50cd4887c..fe92d19567d 100644 --- a/doc/user/project/pages/getting_started_part_two.md +++ b/doc/user/project/pages/getting_started_part_two.md @@ -77,10 +77,10 @@ containing the most popular SSGs templates to get you started. 1. [Fork](../../../gitlab-basics/fork-project.md) a sample project from the [GitLab Pages examples](https://gitlab.com/pages) group. 1. From the left sidebar, navigate to your project's **CI/CD > Pipelines** -and click **Run pipeline** to trigger GitLab CI/CD to build and deploy your -site to the server. + and click **Run pipeline** to trigger GitLab CI/CD to build and deploy your + site to the server. 1. Once the pipeline has finished successfully, find the link to visit your -website from your project's **Settings > Pages**. + website from your project's **Settings > Pages**. You can also take some **optional** further steps: @@ -89,14 +89,14 @@ You can also take some **optional** further steps:  - _Make it a user or group website._ To turn a **project website** forked -from the Pages group into a **user/group** website, you'll need to: + from the Pages group into a **user/group** website, you'll need to: - Rename it to `namespace.gitlab.io`: go to your project's - **Settings > General** and expand **Advanced**. Scroll down to - **Rename repository** and change the path to `namespace.gitlab.io`. + **Settings > General** and expand **Advanced**. Scroll down to + **Rename repository** and change the path to `namespace.gitlab.io`. - Adjust your SSG's [base URL](#urls-and-baseurls) from `"project-name"` to - `""`. This setting will be at a different place for each SSG, as each of them - have their own structure and file tree. Most likely, it will be in the SSG's - config file. + `""`. This setting will be at a different place for each SSG, as each of them + have their own structure and file tree. Most likely, it will be in the SSG's + config file. ### Create a project from scratch diff --git a/doc/user/project/pages/index.md b/doc/user/project/pages/index.md index 04bda212128..fa79c393b72 100644 --- a/doc/user/project/pages/index.md +++ b/doc/user/project/pages/index.md @@ -12,7 +12,6 @@ type: index, reference > - Support for subgroup project's websites was [introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/30548) in GitLab 11.8. > - Bundled project templates were [introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/47857) in GitLab 11.8. - **GitLab Pages is a feature that allows you to publish static websites directly from a repository in GitLab.** @@ -105,10 +104,10 @@ To get started with GitLab Pages, you can either:  1. From the left sidebar, navigate to your project's **CI/CD > Pipelines** -and click **Run pipeline** to trigger GitLab CI/CD to build and deploy your -site to the server. + and click **Run pipeline** to trigger GitLab CI/CD to build and deploy your + site to the server. 1. Once the pipeline has finished successfully, find the link to visit your -website from your project's **Settings > Pages**. + website from your project's **Settings > Pages**. Your website is then visible on your domain, and you can modify yourfiles as you wish. For every modification pushed to your repository, GitLab CI/CD diff --git a/doc/user/project/pages/introduction.md b/doc/user/project/pages/introduction.md index 4fab7f79e0c..4ea3bd9be9b 100644 --- a/doc/user/project/pages/introduction.md +++ b/doc/user/project/pages/introduction.md @@ -13,17 +13,17 @@ To familiarize yourself with GitLab Pages first: - Read an [introduction to GitLab Pages](index.md#overview). - Learn [how to get started with Pages](index.md#getting-started). - Learn how to enable GitLab Pages -across your GitLab instance on the [administrator documentation](../../../administration/pages/index.md). + across your GitLab instance on the [administrator documentation](../../../administration/pages/index.md). ## GitLab Pages requirements In brief, this is what you need to upload your website in GitLab Pages: 1. Domain of the instance: domain name that is used for GitLab Pages -(ask your administrator). + (ask your administrator). 1. GitLab CI/CD: a `.gitlab-ci.yml` file with a specific job named [`pages`][pages] in the root directory of your repository. 1. A directory called `public` in your site's repo containing the content -to be published. + to be published. 1. GitLab Runner enabled for the project. ## GitLab Pages on GitLab.com diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md index 1d640966013..6c430ff7cd9 100644 --- a/doc/user/project/quick_actions.md +++ b/doc/user/project/quick_actions.md @@ -35,7 +35,7 @@ discussions, and descriptions: | `/label ~label1 ~label2` | Add label(s). Label names can also start without ~ but mixed syntax is not supported. | ✓ | ✓ | | `/unlabel ~label1 ~label2` | Remove all or specific label(s)| ✓ | ✓ | | `/relabel ~label1 ~label2` | Replace label | ✓ | ✓ | -| <code>/copy_metadata #issue | !merge_request</code> | Copy labels and milestone from other issue or merge request in the project | ✓ | ✓ | +| <code>/copy_metadata <#issue | !merge_request></code> | Copy labels and milestone from other issue or merge request in the project | ✓ | ✓ | | <code>/estimate <1w 3d 2h 14m></code> | Set time estimate | ✓ | ✓ | | `/remove_estimate` | Remove time estimate | ✓ | ✓ | | <code>/spend <time(1h 30m | -1h 5m)> <date(YYYY-MM-DD)></code> | Add or subtract spent time; optionally, specify the date that time was spent on | ✓ | ✓ | @@ -44,14 +44,14 @@ discussions, and descriptions: | `/unlock` | Unlock the discussion | ✓ | ✓ | | <code>/due <in 2 days | this Friday | December 31st></code>| Set due date | ✓ | | | `/remove_due_date` | Remove due date | ✓ | | -| `/weight 0,1,2, ...` | Set weight **[STARTER]** | ✓ | | +| <code>/weight <0 | 1 | 2 | ...></code> | Set weight **[STARTER]** | ✓ | | | `/clear_weight` | Clears weight **[STARTER]** | ✓ | | -| `/epic <&epic | group&epic | Epic URL>` | Add to epic **[ULTIMATE]** | ✓ | | +| <code>/epic <&epic | group&epic | Epic URL></code> | Add to epic **[ULTIMATE]** | ✓ | | | `/remove_epic` | Removes from epic **[ULTIMATE]** | ✓ | | | `/promote` | Promote issue to epic **[ULTIMATE]** | ✓ | | | `/confidential` | Make confidential | ✓ | | -| `/duplicate #issue` | Mark this issue as a duplicate of another issue | ✓ | -| `/move path/to/project` | Move this issue to another project | ✓ | | +| `/duplicate <#issue>` | Mark this issue as a duplicate of another issue | ✓ | +| `/move <path/to/project>` | Move this issue to another project | ✓ | | | `/target_branch <Local branch Name>` | Set target branch | | ✓ | | `/wip` | Toggle the Work In Progress status | | ✓ | | `/approve` | Approve the merge request | | ✓ | @@ -85,3 +85,5 @@ The following quick actions are applicable for epics threads and description: | `/label ~label1 ~label2` | Add label(s) | | `/unlabel ~label1 ~label2` | Remove all or specific label(s) | | `/relabel ~label1 ~label2` | Replace label | +| <code>/child_epic <&epic | group&epic | Epic URL></code> | Adds child epic to epic ([introduced in GitLab 12.0](https://gitlab.com/gitlab-org/gitlab-ee/issues/7330)) | +| <code>/remove_child_epic <&epic | group&epic | Epic URL></code> | Removes child epic from epic ([introduced in GitLab 12.0](https://gitlab.com/gitlab-org/gitlab-ee/issues/7330)) | diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md index ba890c5ac01..2bf8d4dfe7b 100644 --- a/doc/user/project/settings/index.md +++ b/doc/user/project/settings/index.md @@ -18,7 +18,7 @@ Adjust your project's name, description, avatar, [default branch](../repository/  -The project description also partially supports [standard markdown](../../markdown.md#standard-markdown). You can use [emphasis](../../markdown.md#emphasis), [links](../../markdown.md#links), and [line-breaks](../../markdown.md#line-breaks) to add more context to the project description. +The project description also partially supports [standard markdown](../../markdown.md#standard-markdown-and-extensions-in-gitlab). You can use [emphasis](../../markdown.md#emphasis), [links](../../markdown.md#links), and [line-breaks](../../markdown.md#line-breaks) to add more context to the project description. ### Sharing and permissions @@ -26,10 +26,10 @@ Set up your project's access, [visibility](../../../public_access/public_access.  -If Issues are disabled, or you can't access Issues because you're not a project member, then Lables and Milestones +If Issues are disabled, or you can't access Issues because you're not a project member, then Lables and Milestones links will be missing from the sidebar UI. -You can still access them with direct links if you can access Merge Requests. This is deliberate, if you can see +You can still access them with direct links if you can access Merge Requests. This is deliberate, if you can see Issues or Merge Requests, both of which use Labels and Milestones, then you shouldn't be denied access to Labels and Milestones pages. ### Issue settings @@ -109,8 +109,8 @@ You can transfer an existing project into a [group](../../group/index.md) if: 1. You have at least **Maintainer** [permissions] to that group. 1. The project is in a subgroup you own. 1. You are at least a **Maintainer** of the project under your personal namespace. -Similarly, if you are an owner of a group, you can transfer any of its projects -under your own user. + Similarly, if you are an owner of a group, you can transfer any of its projects + under your own user. To transfer a project: diff --git a/doc/user/project/web_ide/index.md b/doc/user/project/web_ide/index.md index ef4eb45de66..7d85f4adfed 100644 --- a/doc/user/project/web_ide/index.md +++ b/doc/user/project/web_ide/index.md @@ -67,8 +67,8 @@ shows you a preview of the merge request diff if you commit your changes. > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/19279) in [GitLab Core][ce] 11.0. -You can use the Web IDE to quickly fix failing tests by opening -the branch or merge request in the Web IDE and opening the logs of the failed +You can use the Web IDE to quickly fix failing tests by opening +the branch or merge request in the Web IDE and opening the logs of the failed job. You can access the status of all jobs for the most recent pipeline and job traces for the current commit by clicking the **Pipelines** button in the top right. @@ -80,8 +80,8 @@ left. > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/19318) in [GitLab Core][ce] 11.0. -To switch between your authored and assigned merge requests, click the -dropdown in the top of the sidebar to open a list of merge requests. You will +To switch between your authored and assigned merge requests, click the +dropdown in the top of the sidebar to open a list of merge requests. You will need to commit or discard all your changes before switching to a different merge request. @@ -89,9 +89,9 @@ request. > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/20850) in [GitLab Core][ce] 11.2. -To switch between branches of the current project repository, click the dropdown -in the top of the sidebar to open a list of branches. -You will need to commit or discard all your changes before switching to a +To switch between branches of the current project repository, click the dropdown +in the top of the sidebar to open a list of branches. +You will need to commit or discard all your changes before switching to a different branch. ## Client Side Evaluation @@ -117,7 +117,7 @@ GitLab.com  Once you have done that, you can preview projects with a `package.json` file and -a `main` entry point inside the Web IDE. An example `package.json` is shown +a `main` entry point inside the Web IDE. An example `package.json` is shown below. ```json @@ -135,24 +135,20 @@ below. CAUTION: **Warning:** Interactive Web Terminals for the Web IDE is currently in **Beta**. +Shared Runners [do not yet support Interactive Web Terminals](https://gitlab.com/gitlab-org/gitlab-ce/issues/52611), +so you would need to use your own private Runner(s) to make use of this feature. -[Interactive web terminals](../../../ci/interactive_web_terminal/index.md) -give the user access to a terminal to interact with the Runner directly from +[Interactive Web Terminals](../../../ci/interactive_web_terminal/index.md) +give the project [Maintainers](../../permissions.md#project-members-permissions) +user access to a terminal to interact with the Runner directly from GitLab, including through the Web IDE. -Only project [**maintainers**](../../permissions.md#project-members-permissions) -can run Interactive Web Terminals through the Web IDE. - -CAUTION: **Warning:** -GitLab.com [does not support Interactive Web Terminals yet](https://gitlab.com/gitlab-org/gitlab-ce/issues/52611). -Shared Runners in private instances are not supported either. - ### Runner configuration Some things need to be configured in the runner for the interactive web terminal to work: -- The Runner needs to have +- The Runner needs to have [`[session_server]` configured properly](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-session_server-section). - If you are using a reverse proxy with your GitLab instance, web terminals need to be [enabled](../../../administration/integration/terminal.md#enabling-and-disabling-terminal-support). **[ULTIMATE ONLY]** @@ -175,13 +171,13 @@ syntax but with some restrictions: - No global blocks can be defined (ie: `before_script` or `after_script`) - Only one job named `terminal` can be added to this file. - Only the keywords `image`, `services`, `tags`, `before_script`, `script`, and -`variables` are allowed to be used to configure the job. + `variables` are allowed to be used to configure the job. - To connect to the interactive terminal, the `terminal` job must be still alive -and running, otherwise the terminal won't be able to connect to the job's session. -By default the `script` keyword has the value `sleep 60` to prevent -the job from ending and giving the Web IDE enough time to connect. This means -that, if you override the default `script` value, you'll have to add a command -which would keep the job running, like `sleep`. + and running, otherwise the terminal won't be able to connect to the job's session. + By default the `script` keyword has the value `sleep 60` to prevent + the job from ending and giving the Web IDE enough time to connect. This means + that, if you override the default `script` value, you'll have to add a command + which would keep the job running, like `sleep`. In the code below there is an example of this configuration file: @@ -204,7 +200,7 @@ the selected branch of the Web IDE. If there is no configuration file in a branch, an error message will be shown. -### Running Interactive Terminals in the Web IDE +### Running interactive terminals in the Web IDE If Interactive Terminals are available for the current user, the **Terminal** button will be visible in the right sidebar of the Web IDE. Click this button to open @@ -212,7 +208,7 @@ or close the terminal tab. Once open, the tab will show the **Start Web Terminal** button. This button may be disabled if the environment is not configured correctly. If so, a status -message will describe the issue. Here are some reasons why **Start Web Terminal** +message will describe the issue. Here are some reasons why **Start Web Terminal** may be disabled: - `.gitlab/.gitlab-webide.yml` does not exist or is set up incorrectly. @@ -231,45 +227,27 @@ While the terminal is running, it can be stopped by clicking **Stop Terminal**. This will disconnect the terminal and stop the runner's terminal job. From here, click **Restart Terminal** to start a new terminal session. -### File Syncing to Web Terminal +### File syncing to web terminal > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/5276) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.0. -File changes in the Web IDE can be synced to a running Web Terminal. +File changes in the Web IDE can be synced to a running web terminal. This enables users to test their code changes in a preconfigured terminal environment. NOTE: **Note:** Only file changes in the Web IDE are synced to the terminal. Changes made in the terminal are **not** synced to the Web IDE. +This feature is only available for Kubernetes Runners. -Once you have [configured the Web Terminal for File Syncing](#configuring-file-syncing), -then when the Web terminal is started, a **Terminal** status will be visible -in the status bar. - - - -Changes made to your files via the Web IDE will sync to the running terminal -when: - -- <kbd>Ctrl</kbd> + <kbd>S</kbd> (or <kbd>Cmd</kbd> + <kbd>S</kbd> on Mac) - is pressed while editing a file. -- Anything outside the file editor is clicked after editing a file. -- A file or folder is created, deleted, or renamed. - -### Configuring File Syncing - -NOTE: **Note:** -This feature is only available for Kubernetes runners. - -To enable file syncing to the Web Terminal, the `.gitlab/.gitlab-webide.yml` -file needs to have a `webide-file-sync` service configured. Here is an example +To enable file syncing to the web terminal, the `.gitlab/.gitlab-webide.yml` +file needs to have a `webide-file-sync` service configured. Here is an example configuration for a Node JS project which uses this service: ```yaml terminal: # This can be any image that has the necessary runtime environment for your project. - image: + image: name: node:10-alpine services: - name: registry.gitlab.com/gitlab-org/webide-file-sync:latest @@ -281,18 +259,30 @@ terminal: - number: 3000 ``` -> **Notes:** -> - For now, the `webide-file-sync` executable must start **after** the project -> directory is available. This is why we need to add `sleep 5` to the `command`. -> See [this issue](https://gitlab.com/gitlab-org/webide-file-sync/issues/7) for -> more info. -> - `$CI_PROJECT_DIR` is a -> [predefined environment variable](../../../ci/variables/predefined_variables.md) -> for GitLab Runners. This is where your project's repository will be. +- The `webide-file-sync` executable must start **after** the project + directory is available. This is why we need to add `sleep 5` to the `command`. + See [this issue](https://gitlab.com/gitlab-org/webide-file-sync/issues/7) for + more info. +- `$CI_PROJECT_DIR` is a + [predefined environment variable](../../../ci/variables/predefined_variables.md) + for GitLab Runners. This is where your project's repository will be. + +Once you have configured the web terminal for file syncing, then when the web +terminal is started, a **Terminal** status will be visible in the status bar. + + + +Changes made to your files via the Web IDE will sync to the running terminal +when: + +- <kbd>Ctrl</kbd> + <kbd>S</kbd> (or <kbd>Cmd</kbd> + <kbd>S</kbd> on Mac) + is pressed while editing a file. +- Anything outside the file editor is clicked after editing a file. +- A file or folder is created, deleted, or renamed. ### Limitations -Interactive Terminals is in a beta phase and will continue to be improved upon in upcoming +Interactive Terminals is in a beta phase and will continue to be improved upon in upcoming releases. In the meantime, please note that the user is limited to having only one active terminal at a time. diff --git a/doc/workflow/time_tracking.md b/doc/workflow/time_tracking.md index c03dffa967d..4286a3625a2 100644 --- a/doc/workflow/time_tracking.md +++ b/doc/workflow/time_tracking.md @@ -73,7 +73,15 @@ The following time units are available: Default conversion rates are 1mo = 4w, 1w = 5d and 1d = 8h. -Other interesting links: +### Limit displayed units to hours + +> Introduced in GitLab 12.0. + +The display of time units can be limited to hours through the option in **Admin Area > Settings > Preferences** under 'Localization'. + +With this option enabled, `75h` is displayed instead of `1w 4d 3h`. + +## Other interesting links - [Time Tracking landing page on about.gitlab.com](https://about.gitlab.com/solutions/time-tracking/) |
