diff options
Diffstat (limited to 'doc')
350 files changed, 7201 insertions, 3204 deletions
diff --git a/doc/administration/auth/google_secure_ldap.md b/doc/administration/auth/google_secure_ldap.md index 65a51fc4aa0..760af0cfd1a 100644 --- a/doc/administration/auth/google_secure_ldap.md +++ b/doc/administration/auth/google_secure_ldap.md @@ -27,7 +27,7 @@ The steps below cover: 'Entire domain (GitLab)' or 'Selected organizational units' for both 'Verify user credentials' and 'Read user information'. Select 'Add LDAP Client' - TIP: **Tip:** If you plan to use GitLab [LDAP Group Sync](https://docs.gitlab.com/ee/administration/auth/ldap-ee.html#group-sync) + TIP: **Tip:** If you plan to use GitLab [LDAP Group Sync](ldap-ee.md#group-sync) , turn on 'Read group information'.  diff --git a/doc/administration/auth/oidc.md b/doc/administration/auth/oidc.md index e55f7dbb4df..df4f22aa3e7 100644 --- a/doc/administration/auth/oidc.md +++ b/doc/administration/auth/oidc.md @@ -31,6 +31,7 @@ The OpenID Connect will provide you with a client details and secret for you to { 'name' => 'openid_connect', 'label' => '<your_oidc_label>', 'args' => { + "name' => 'openid_connect', 'scope' => ['openid','profile'], 'response_type' => 'code', 'issuer' => '<your_oidc_url>', @@ -53,6 +54,7 @@ The OpenID Connect will provide you with a client details and secret for you to - { name: 'openid_connect', label: '<your_oidc_label>', args: { + name: 'openid_connect', scope: ['openid','profile'], response_type: 'code', issuer: '<your_oidc_url>', @@ -103,3 +105,36 @@ On the sign in page, there should now be an OpenID Connect icon below the regula Click the icon to begin the authentication process. The OpenID Connect provider will ask the user to sign in and authorize the GitLab application (if confirmation required by the client). If everything goes well, the user will be redirected to GitLab and will be signed in. + +## Example configurations + +The following configurations illustrate how to set up OpenID with +different providers with Omnibus GitLab. + +### Google + +See the [Google +documentation](https://developers.google.com/identity/protocols/OpenIDConnect) +for more details: + +```ruby + gitlab_rails['omniauth_providers'] = [ + { + 'name' => 'openid_connect', + 'label' => 'Google OpenID', + 'args' => { + 'name' => 'openid_connect', + 'scope' => ['openid', 'profile', 'email'], + 'response_type' => 'code', + 'issuer' => 'https://accounts.google.com', + 'client_auth_method' => 'query', + 'discovery' => true, + 'uid_field' => 'preferred_username', + 'client_options' => { + 'identifier' => '<YOUR PROJECT CLIENT ID>', + 'secret' => '<YOUR PROJECT CLIENT SECRET>', + 'redirect_uri' => 'https://example.com/users/auth/openid_connect/callback', + } + } + } +``` diff --git a/doc/administration/compliance.md b/doc/administration/compliance.md index 9c13ff772b3..246addb6dc9 100644 --- a/doc/administration/compliance.md +++ b/doc/administration/compliance.md @@ -12,7 +12,7 @@ GitLab’s [security features](../security/README.md) may also help you meet rel |**[Email all users of a project, group, or entire server](../user/admin_area/settings/terms.md)**<br>An admin can email groups of users based on project or group membership, or email everyone using the GitLab instance. This is great for scheduled maintenance or upgrades.|Starter+|| |**[Omnibus package supports log forwarding](https://docs.gitlab.com/omnibus/settings/logs.html#udp-log-forwarding)**<br>Forward your logs to a central system.|Starter+|| |**[Lock project membership to group](../user/group/index.md#member-lock-starter)**<br>Group owners can prevent new members from being added to projects within a group.|Starter+|✓| -|**[LDAP group sync](https://docs.gitlab.com/ee/administration/auth/ldap-ee.html#group-sync)**<br>GitLab Enterprise Edition gives admins the ability to automatically sync groups and manage SSH keys, permissions, and authentication, so you can focus on building your product, not configuring your tools.|Starter+|| -|**[LDAP group sync filters](https://docs.gitlab.com/ee/administration/auth/ldap-ee.html#group-sync)**<br>GitLab Enterprise Edition Premium gives more flexibility to synchronize with LDAP based on filters, meaning you can leverage LDAP attributes to map GitLab permissions.|Premium+|| +|**[LDAP group sync](auth/ldap-ee.md#group-sync)**<br>GitLab Enterprise Edition gives admins the ability to automatically sync groups and manage SSH keys, permissions, and authentication, so you can focus on building your product, not configuring your tools.|Starter+|| +|**[LDAP group sync filters](auth/ldap-ee.md#group-sync)**<br>GitLab Enterprise Edition Premium gives more flexibility to synchronize with LDAP based on filters, meaning you can leverage LDAP attributes to map GitLab permissions.|Premium+|| |**[Audit logs](audit_events.md)**<br>To maintain the integrity of your code, GitLab Enterprise Edition Premium gives admins the ability to view any modifications made within the GitLab server in an advanced audit log system, so you can control, analyze and track every change.|Premium+|| |**[Auditor users](auditor_users.md)**<br>Auditor users are users who are given read-only access to all projects, groups, and other resources on the GitLab instance.|Premium+|| diff --git a/doc/administration/custom_hooks.md b/doc/administration/custom_hooks.md index 288cb1bf0bb..113514e1ee8 100644 --- a/doc/administration/custom_hooks.md +++ b/doc/administration/custom_hooks.md @@ -5,11 +5,11 @@ Custom Git hooks must be configured on the filesystem of the GitLab server. Only GitLab server administrators will be able to complete these tasks. Please explore [webhooks] and [CI] as an option if you do not have filesystem access. For a user configurable Git hook interface, see -[Push Rules](https://docs.gitlab.com/ee/push_rules/push_rules.html), +[Push Rules](../push_rules/push_rules.md), available in GitLab Enterprise Edition. NOTE: **Note:** -Custom Git hooks won't be replicated to secondary nodes if you use [GitLab Geo][gitlab-geo] +Custom Git hooks won't be replicated to secondary nodes if you use [GitLab Geo](geo/replication/index.md) Git natively supports hooks that are executed on different actions. Examples of server-side git hooks include pre-receive, post-receive, and update. @@ -123,6 +123,5 @@ exit 1 [CI]: ../ci/README.md [hooks]: https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks#Server-Side-Hooks [webhooks]: ../user/project/integrations/webhooks.md -[gitlab-geo]: https://docs.gitlab.com/ee/administration/geo/replication/index.html [5073]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5073 [93]: https://gitlab.com/gitlab-org/gitlab-shell/merge_requests/93 diff --git a/doc/administration/geo/disaster_recovery/background_verification.md b/doc/administration/geo/disaster_recovery/background_verification.md index c7299b6e196..d4c8c2d3624 100644 --- a/doc/administration/geo/disaster_recovery/background_verification.md +++ b/doc/administration/geo/disaster_recovery/background_verification.md @@ -29,12 +29,7 @@ the node more time before scheduling a planned failover. Run the following commands in a Rails console on the **primary** node: ```sh -# Omnibus GitLab gitlab-rails console - -# Installation from source -cd /home/git/gitlab -sudo -u git -H bin/rails console RAILS_ENV=production ``` To check if automatic background verification is enabled: @@ -102,12 +97,7 @@ disable if you need. Run the following commands in a Rails console on the **primary** node: ```sh -# Omnibus GitLab gitlab-rails console - -# Installation from source -cd /home/git/gitlab -sudo -u git -H bin/rails console RAILS_ENV=production ``` To disable automatic background re-verification: @@ -131,31 +121,15 @@ to be resynced without the backoff period: For repositories: -- Omnibus Installation - - ```sh - sudo gitlab-rake geo:verification:repository:reset - ``` - -- Source Installation - - ```sh - sudo -u git -H bundle exec rake geo:verification:repository:reset RAILS_ENV=production - ``` +```sh +sudo gitlab-rake geo:verification:repository:reset +``` For wikis: -- Omnibus Installation - - ```sh - sudo gitlab-rake geo:verification:wiki:reset - ``` - -- Source Installation - - ```sh - sudo -u git -H bundle exec rake geo:verification:wiki:reset RAILS_ENV=production - ``` +```sh +sudo gitlab-rake geo:verification:wiki:reset +``` ## Reconcile differences with checksum mismatches @@ -169,7 +143,7 @@ If the **primary** and **secondary** nodes have a checksum verification mismatch 1. On the project admin page get the **Gitaly storage name**, and **Gitaly relative path**:  -1. Navigate to the project's repository directory on both **primary** and **secondary** nodes. For an installation from source, the path is usually `/home/git/repositories`. For Omnibus installs, the path is usually `/var/opt/gitlab/git-data/repositories`. Note that if `git_data_dirs` is customized, check the directory layout on your server to be sure. +1. Navigate to the project's repository directory on both **primary** and **secondary** nodes (the path is usually `/var/opt/gitlab/git-data/repositories`). Note that if `git_data_dirs` is customized, check the directory layout on your server to be sure. ```sh cd /var/opt/gitlab/git-data/repositories diff --git a/doc/administration/geo/disaster_recovery/planned_failover.md b/doc/administration/geo/disaster_recovery/planned_failover.md index 88ab12d910a..b8071b5993f 100644 --- a/doc/administration/geo/disaster_recovery/planned_failover.md +++ b/doc/administration/geo/disaster_recovery/planned_failover.md @@ -113,7 +113,7 @@ If any objects are failing to replicate, this should be investigated before scheduling the maintenance window. Following a planned failover, anything that failed to replicate will be **lost**. -You can use the [Geo status API](https://docs.gitlab.com/ee/api/geo_nodes.html#retrieve-project-sync-or-verification-failures-that-occurred-on-the-current-node) to review failed objects and +You can use the [Geo status API](../../../api/geo_nodes.md#retrieve-project-sync-or-verification-failures-that-occurred-on-the-current-node) to review failed objects and the reasons for failure. A common cause of replication failures is the data being missing on the diff --git a/doc/administration/geo/replication/configuration.md b/doc/administration/geo/replication/configuration.md index 57735b21cda..3d4f69d3abe 100644 --- a/doc/administration/geo/replication/configuration.md +++ b/doc/administration/geo/replication/configuration.md @@ -1,9 +1,4 @@ -# Geo configuration (GitLab Omnibus) **[PREMIUM ONLY]** - -NOTE: **Note:** -This is the documentation for the Omnibus GitLab packages. For installations -from source, follow the [**Geo nodes configuration for installations -from source**][configuration-source] guide. +# Geo configuration **[PREMIUM ONLY]** ## Configuring a new **secondary** node @@ -303,7 +298,6 @@ See the [updating the Geo nodes document](updating_the_geo_nodes.md). See the [troubleshooting document](troubleshooting.md). -[configuration-source]: configuration_source.md [setup-geo-omnibus]: index.md#using-omnibus-gitlab [Hashed Storage]: ../../repository_storage_types.md [Disaster Recovery]: ../disaster_recovery/index.md diff --git a/doc/administration/geo/replication/configuration_source.md b/doc/administration/geo/replication/configuration_source.md deleted file mode 100644 index 10dd9405b4b..00000000000 --- a/doc/administration/geo/replication/configuration_source.md +++ /dev/null @@ -1,172 +0,0 @@ -# Geo configuration (source) **[PREMIUM ONLY]** - -NOTE: **Note:** -This documentation applies to GitLab source installations. In GitLab 11.5, this documentation was deprecated and will be removed in a future release. -Please consider [migrating to GitLab Omnibus install](https://docs.gitlab.com/omnibus/update/convert_to_omnibus.html). For installations -using the Omnibus GitLab packages, follow the -[**Omnibus Geo nodes configuration**][configuration] guide. - -## Configuring a new **secondary** node - -NOTE: **Note:** -This is the final step in setting up a **secondary** node. Stages of the setup -process must be completed in the documented order. Before attempting the steps -in this stage, [complete all prior stages](index.md#using-gitlab-installed-from-source-deprecated). - -The basic steps of configuring a **secondary** node are to: - -- Replicate required configurations between the **primary** and **secondary** nodes. -- Configure a tracking database on each **secondary** node. -- Start GitLab on the **secondary** node. - -You are encouraged to first read through all the steps before executing them -in your testing/production environment. - -NOTE: **Note:** -**Do not** set up any custom authentication on **secondary** nodes, this will be handled by the **primary** node. - -NOTE: **Note:** -**Do not** add anything in the **secondary** node's admin area (**Admin Area > Geo**). This is handled solely by the **primary** node. - -### Step 1. Manually replicate secret GitLab values - -GitLab stores a number of secret values in the `/home/git/gitlab/config/secrets.yml` -file which *must* match between the **primary** and **secondary** nodes. Until there is -a means of automatically replicating these between nodes (see [gitlab-org/gitlab-ee#3789]), they must -be manually replicated to **secondary** nodes. - -1. SSH into the **primary** node, and execute the command below: - - ```sh - sudo cat /home/git/gitlab/config/secrets.yml - ``` - - This will display the secrets that need to be replicated, in YAML format. - -1. SSH into the **secondary** node and login as the `git` user: - - ```sh - sudo -i -u git - ``` - -1. Make a backup of any existing secrets: - - ```sh - mv /home/git/gitlab/config/secrets.yml /home/git/gitlab/config/secrets.yml.`date +%F` - ``` - -1. Copy `/home/git/gitlab/config/secrets.yml` from the **primary** node to the **secondary** node, or - copy-and-paste the file contents between nodes: - - ```sh - sudo editor /home/git/gitlab/config/secrets.yml - - # paste the output of the `cat` command you ran on the primary - # save and exit - ``` - -1. Ensure the file permissions are correct: - - ```sh - chown git:git /home/git/gitlab/config/secrets.yml - chmod 0600 /home/git/gitlab/config/secrets.yml - ``` - -1. Restart GitLab - - ```sh - service gitlab restart - ``` - -Once restarted, the **secondary** node will automatically start replicating missing data -from the **primary** node in a process known as backfill. Meanwhile, the **primary** node -will start to notify the **secondary** node of any changes, so that the **secondary** node can -act on those notifications immediately. - -Make sure the **secondary** node is running and accessible. You can login to -the **secondary** node with the same credentials as used for the **primary** node. - -### Step 2. Manually replicate the **primary** node's SSH host keys - -Read [Manually replicate the **primary** node's SSH host keys](configuration.md#step-2-manually-replicate-the-primary-nodes-ssh-host-keys) - -### Step 3. Add the **secondary** GitLab node - -1. Navigate to the **primary** node's **Admin Area > Geo** - (`/admin/geo/nodes`) in your browser. -1. Add the **secondary** node by providing its full URL. **Do NOT** check the - **This is a primary node** checkbox. -1. Optionally, choose which namespaces should be replicated by the - **secondary** node. Leave blank to replicate all. Read more in - [selective synchronization](#selective-synchronization). -1. Click the **Add node** button. -1. SSH into your GitLab **secondary** server and restart the services: - - ```sh - service gitlab restart - ``` - - Check if there are any common issue with your Geo setup by running: - - ```sh - bundle exec rake gitlab:geo:check - ``` - -1. SSH into your GitLab **primary** server and login as root to verify the - **secondary** node is reachable or there are any common issue with your Geo setup: - - ```sh - bundle exec rake gitlab:geo:check - ``` - -Once reconfigured, the **secondary** node will automatically start -replicating missing data from the **primary** node in a process known as backfill. -Meanwhile, the **primary** node will start to notify the **secondary** node of any changes, so -that the **secondary** node can act on those notifications immediately. - -Make sure the **secondary** node is running and accessible. -You can log in to the **secondary** node with the same credentials as used for the **primary** node. - -### Step 4. Enabling Hashed Storage - -Read [Enabling Hashed Storage](configuration.md#step-4-enabling-hashed-storage). - -### Step 5. (Optional) Configuring the secondary to trust the primary - -You can safely skip this step if your **primary** node uses a CA-issued HTTPS certificate. - -If your **primary** node is using a self-signed certificate for *HTTPS* support, you will -need to add that certificate to the **secondary** node's trust store. Retrieve the -certificate from the **primary** node and follow your distribution's instructions for -adding it to the **secondary** node's trust store. In Debian/Ubuntu, you would follow these steps: - -```sh -sudo -i -cp <primary_node_certification_file> /usr/local/share/ca-certificates -update-ca-certificates -``` - -### Step 6. Enable Git access over HTTP/HTTPS - -Geo synchronizes repositories over HTTP/HTTPS, and therefore requires this clone -method to be enabled. Navigate to **Admin Area > Settings** -(`/admin/application_settings`) on the **primary** node, and set -`Enabled Git access protocols` to `Both SSH and HTTP(S)` or `Only HTTP(S)`. - -### Step 7. Verify proper functioning of the secondary node - -Read [Verify proper functioning of the secondary node][configuration-verify-node]. - -## Selective synchronization - -Read [Selective synchronization][configuration-selective-replication]. - -## Troubleshooting - -Read the [troubleshooting document][troubleshooting]. - -[gitlab-org/gitlab-ee#3789]: https://gitlab.com/gitlab-org/gitlab-ee/issues/3789 -[configuration]: configuration.md -[configuration-selective-replication]: configuration.md#selective-synchronization -[configuration-verify-node]: configuration.md#step-7-verify-proper-functioning-of-the-secondary-node -[troubleshooting]: troubleshooting.md diff --git a/doc/administration/geo/replication/database.md b/doc/administration/geo/replication/database.md index e57583a3bf9..1e5a56c3f4e 100644 --- a/doc/administration/geo/replication/database.md +++ b/doc/administration/geo/replication/database.md @@ -1,9 +1,7 @@ -# Geo database replication (GitLab Omnibus) **[PREMIUM ONLY]** +# Geo database replication **[PREMIUM ONLY]** NOTE: **Note:** -This is the documentation for the Omnibus GitLab packages. For installations -from source, follow the -[Geo database replication (source)](database_source.md) guide. +The following steps are for Omnibus installs only. Using Geo with source-based installs was **deprecated** in GitLab 11.5. NOTE: **Note:** If your GitLab installation uses external (not managed by Omnibus) PostgreSQL @@ -102,10 +100,15 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o else. If you are using an external database not managed by Omnibus GitLab, you need - to create the replicator user and define a password to it manually. - For information on how to create a replication user, refer to the - [appropriate step](database_source.md#step-1-configure-the-primary-server) - in [Geo database replication (source)](database_source.md). + to create the replicator user and define a password to it manually: + + ```sql + --- Create a new user 'replicator' + CREATE USER gitlab_replicator; + + --- Set/change a password and grants replication privilege + ALTER USER gitlab_replicator WITH REPLICATION ENCRYPTED PASSWORD '<replication_password>'; + ``` 1. Configure PostgreSQL to listen on network interfaces: @@ -340,7 +343,7 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o ## ## Secondary address - ## - replace '<secondary_node_ip>' with the public or VPC address of your Geo secondary node + ## - replace '<secondary_node_ip>' with the public or VPC address of your Geo secondary node ## postgresql['listen_address'] = '<secondary_node_ip>' postgresql['md5_auth_cidr_addresses'] = ['<secondary_node_ip>/32'] @@ -383,8 +386,7 @@ the database on the **primary** node, replicates the database, and creates the needed files for streaming replication. The directories used are the defaults that are set up in Omnibus. If you have -changed any defaults or are using a source installation, configure it as you -see fit replacing the directories and paths. +changed any defaults, configure it as you see fit replacing the directories and paths. CAUTION: **Warning:** Make sure to run this on the **secondary** server as it removes all PostgreSQL's @@ -443,8 +445,7 @@ The replication process is now complete. PostgreSQL connections. We recommend using PGBouncer if you use GitLab in a high-availability configuration with a cluster of nodes supporting a Geo **primary** node and another cluster of nodes supporting a Geo **secondary** node. For more -information, see the [Omnibus HA](https://docs.gitlab.com/ee/administration/high_availability/database.html#configure-using-omnibus-for-high-availability) -documentation. +information, see [High Availability with GitLab Omnibus](../../high_availability/database.md#high-availability-with-gitlab-omnibus-premium-only). For a Geo **secondary** node to work properly with PGBouncer in front of the database, it will need a separate read-only user to make [PostgreSQL FDW queries][FDW] diff --git a/doc/administration/geo/replication/database_source.md b/doc/administration/geo/replication/database_source.md deleted file mode 100644 index 67cf8b6535f..00000000000 --- a/doc/administration/geo/replication/database_source.md +++ /dev/null @@ -1,439 +0,0 @@ -# Geo database replication (source) **[PREMIUM ONLY]** - -NOTE: **Note:** -This documentation applies to GitLab source installations. In GitLab 11.5, this documentation was deprecated and will be removed in a future release. -Please consider [migrating to GitLab Omnibus install](https://docs.gitlab.com/omnibus/update/convert_to_omnibus.html). For installations -using the Omnibus GitLab packages, follow the -[**database replication for Omnibus GitLab**][database] guide. - -NOTE: **Note:** -The stages of the setup process must be completed in the documented order. -Before attempting the steps in this stage, [complete all prior stages](index.md#using-gitlab-installed-from-source-deprecated). - -This document describes the minimal steps you have to take in order to -replicate your **primary** GitLab database to a **secondary** node's database. You may -have to change some values according to your database setup, how big it is, etc. - -You are encouraged to first read through all the steps before executing them -in your testing/production environment. - -## PostgreSQL replication - -The GitLab **primary** node where the write operations happen will connect to -**primary** database server, and the **secondary** ones which are read-only will -connect to **secondary** database servers (which are read-only too). - -NOTE: **Note:** -In many databases' documentation, you will see "**primary**" being referenced as "master" -and "**secondary**" as either "slave" or "standby" server (read-only). - -We recommend using [PostgreSQL replication slots][replication-slots-article] -to ensure the **primary** node retains all the data necessary for the secondaries to -recover. See below for more details. - -The following guide assumes that: - -- You are using PostgreSQL 9.6 or later which includes the - [`pg_basebackup` tool][pgback] and improved [Foreign Data Wrapper][FDW] support. -- You have a **primary** node already set up (the GitLab server you are - replicating from), running PostgreSQL 9.6 or later, and - you have a new **secondary** server set up with the same versions of the OS, - PostgreSQL, and GitLab on all nodes. -- The IP of the **primary** server for our examples is `198.51.100.1`, whereas the - **secondary** node's IP is `198.51.100.2`. Note that the **primary** and **secondary** servers - **must** be able to communicate over these addresses. These IP addresses can either - be public or private. - -CAUTION: **Warning:** -Geo works with streaming replication. Logical replication is not supported at this time. -There is an [issue where support is being discussed](https://gitlab.com/gitlab-org/gitlab-ee/issues/7420). - -### Step 1. Configure the **primary** server - -1. SSH into your GitLab **primary** server and login as root: - - ```sh - sudo -i - ``` - -1. Add this node as the Geo **primary** by running: - - ```sh - bundle exec rake geo:set_primary_node - ``` - -1. Create a [replication user] named `gitlab_replicator`: - - ```sql - --- Create a new user 'replicator' - CREATE USER gitlab_replicator; - - --- Set/change a password and grants replication privilege - ALTER USER gitlab_replicator WITH REPLICATION ENCRYPTED PASSWORD '<replication_password>'; - ``` - -1. Make sure your the `gitlab` database user has a password defined: - - ```sh - sudo \ - -u postgres psql \ - -d template1 \ - -c "ALTER USER gitlab WITH ENCRYPTED PASSWORD '<database_password>';" - ``` - -1. Edit the content of `database.yml` in `production:` and add the password like the example below: - - ```yaml - # - # PRODUCTION - # - production: - adapter: postgresql - encoding: unicode - database: gitlabhq_production - pool: 10 - username: gitlab - password: <database_password> - host: /var/opt/gitlab/geo-postgresql - ``` - -1. Set up TLS support for the PostgreSQL **primary** server: - - CAUTION: **Warning**: - Only skip this step if you **know** that PostgreSQL traffic - between the **primary** and **secondary** nodes will be secured through some other - means, e.g., a known-safe physical network path or a site-to-site VPN that - you have configured. - - If you are replicating your database across the open Internet, it is - **essential** that the connection is TLS-secured. Correctly configured, this - provides protection against both passive eavesdroppers and active - "man-in-the-middle" attackers. - - To generate a self-signed certificate and key, run this command: - - ```sh - openssl req \ - -nodes \ - -batch \ - -x509 \ - -newkey rsa:4096 \ - -keyout server.key \ - -out server.crt \ - -days 3650 - ``` - - This will create two files - `server.key` and `server.crt` - that you can - use for authentication. - - Copy them to the correct location for your PostgreSQL installation: - - ```sh - # Copying a self-signed certificate and key - install -o postgres -g postgres -m 0400 -T server.crt ~postgres/9.x/main/data/server.crt - install -o postgres -g postgres -m 0400 -T server.key ~postgres/9.x/main/data/server.key - ``` - - Add this configuration to `postgresql.conf`, removing any existing - configuration for `ssl_cert_file` or `ssl_key_file`: - - ``` - ssl = on - ssl_cert_file='server.crt' - ssl_key_file='server.key' - ``` - -1. Edit `postgresql.conf` to configure the **primary** server for streaming replication - (for Debian/Ubuntu that would be `/etc/postgresql/9.x/main/postgresql.conf`): - - ``` - listen_address = '<primary_node_ip>' - wal_level = hot_standby - max_wal_senders = 5 - min_wal_size = 80MB - max_wal_size = 1GB - max_replicaton_slots = 1 # Number of Geo secondary nodes - wal_keep_segments = 10 - hot_standby = on - ``` - - NOTE: **Note**: - Be sure to set `max_replication_slots` to the number of Geo **secondary** - nodes that you may potentially have (at least 1). - - For security reasons, PostgreSQL by default only listens on the local - interface (e.g. 127.0.0.1). However, Geo needs to communicate - between the **primary** and **secondary** nodes over a common network, such as a - corporate LAN or the public Internet. For this reason, we need to - configure PostgreSQL to listen on more interfaces. - - The `listen_address` option opens PostgreSQL up to external connections - with the interface corresponding to the given IP. See [the PostgreSQL - documentation][pg-docs-runtime-conn] for more details. - - You may also want to edit the `wal_keep_segments` and `max_wal_senders` to - match your database replication requirements. Consult the - [PostgreSQL - Replication documentation][pg-docs-runtime-replication] for more information. - -1. Set the access control on the **primary** node to allow TCP connections using the - server's public IP and set the connection from the **secondary** node to require a - password. Edit `pg_hba.conf` (for Debian/Ubuntu that would be - `/etc/postgresql/9.x/main/pg_hba.conf`): - - ```sh - host all all <primary_node_ip>/32 md5 - host replication gitlab_replicator <secondary_node_ip>/32 md5 - ``` - - If you want to add another secondary, add one more row like the replication - one and change the IP address: - - ```sh - host all all <primary_node_ip>/32 md5 - host replication gitlab_replicator <secondary_node_ip>/32 md5 - host replication gitlab_replicator <another_secondary_node_ip>/32 md5 - ``` - -1. Restart PostgreSQL for the changes to take effect. - -1. Choose a database-friendly name to use for your secondary to use as the - replication slot name. For example, if your domain is - `secondary.geo.example.com`, you may use `secondary_example` as the slot - name. - -1. Create the replication slot on the **primary** node: - - ```sh - $ sudo -u postgres psql -c "SELECT * FROM pg_create_physical_replication_slot('secondary_example');" - slot_name | xlog_position - ------------------+--------------- - secondary_example | - (1 row) - ``` - -1. Now that the PostgreSQL server is set up to accept remote connections, run - `netstat -plnt` to make sure that PostgreSQL is listening to the server's - public IP. - -### Step 2. Configure the secondary server - -Follow the first steps in ["configure the secondary server"][database-replication] and note that since you are installing from source, the username and -group listed as `gitlab-psql` in those steps should be replaced by `postgres` -instead. After completing the "Test that the `gitlab-psql` user can connect to -the **primary** node's database" step, continue here: - -1. Edit `postgresql.conf` to configure the secondary for streaming replication - (for Debian/Ubuntu that would be `/etc/postgresql/9.*/main/postgresql.conf`): - - ```sh - wal_level = hot_standby - max_wal_senders = 5 - checkpoint_segments = 10 - wal_keep_segments = 10 - hot_standby = on - ``` - -1. Restart PostgreSQL for the changes to take effect. - -#### Enable tracking database on the secondary server - -Geo secondary nodes use a tracking database to keep track of replication status -and recover automatically from some replication issues. Follow the steps below to create -the tracking database. - -1. On the secondary node, run the following command to create `database_geo.yml` with the - information of your secondary PostgreSQL instance: - - ```sh - sudo cp /home/git/gitlab/config/database_geo.yml.postgresql /home/git/gitlab/config/database_geo.yml - ``` - -1. Edit the content of `database_geo.yml` in `production:` as in the example below: - - ```yaml - # - # PRODUCTION - # - production: - adapter: postgresql - encoding: unicode - database: gitlabhq_geo_production - pool: 10 - username: gitlab_geo - # password: - host: /var/opt/gitlab/geo-postgresql - ``` - -1. Create the database `gitlabhq_geo_production` on the PostgreSQL instance of the **secondary** node. - -1. Set up the Geo tracking database: - - ```sh - bundle exec rake geo:db:migrate - ``` - -1. Configure the [PostgreSQL FDW][FDW] connection and credentials: - - Save the script below in a file, ex. `/tmp/geo_fdw.sh` and modify the connection - params to match your environment. Execute it to set up the FDW connection. - - ```sh - #!/bin/bash - - # Secondary Database connection params: - DB_HOST="/var/opt/gitlab/postgresql" # change to the public IP or VPC private IP if its an external server - DB_NAME="gitlabhq_production" - DB_USER="gitlab" - DB_PORT="5432" - - # Tracking Database connection params: - GEO_DB_HOST="/var/opt/gitlab/geo-postgresql" # change to the public IP or VPC private IP if its an external server - GEO_DB_NAME="gitlabhq_geo_production" - GEO_DB_USER="gitlab_geo" - GEO_DB_PORT="5432" - - query_exec () { - gitlab-psql -h $GEO_DB_HOST -d $GEO_DB_NAME -p $GEO_DB_PORT -c "${1}" - } - - query_exec "CREATE EXTENSION postgres_fdw;" - query_exec "CREATE SERVER gitlab_secondary FOREIGN DATA WRAPPER postgres_fdw OPTIONS (host '${DB_HOST}', dbname '${DB_NAME}', port '${DB_PORT}');" - query_exec "CREATE USER MAPPING FOR ${GEO_DB_USER} SERVER gitlab_secondary OPTIONS (user '${DB_USER}');" - query_exec "CREATE SCHEMA gitlab_secondary;" - query_exec "GRANT USAGE ON FOREIGN SERVER gitlab_secondary TO ${GEO_DB_USER};" - ``` - - And edit the content of `database_geo.yml` and to add `fdw: true` to - the `production:` block. - -### Step 3. Initiate the replication process - -Below we provide a script that connects the database on the **secondary** node to -the database on the **primary** node, replicates the database, and creates the -needed files for streaming replication. - -The directories used are the defaults for Debian/Ubuntu. If you have changed -any defaults, configure it as you see fit replacing the directories and paths. - -CAUTION: **Warning:** -Make sure to run this on the **secondary** server as it removes all PostgreSQL's -data before running `pg_basebackup`. - -1. SSH into your GitLab **secondary** server and login as root: - - ```sh - sudo -i - ``` - -1. Save the snippet below in a file, let's say `/tmp/replica.sh`. Modify the - embedded paths if necessary: - - ``` - #!/bin/bash - - PORT="5432" - USER="gitlab_replicator" - echo --------------------------------------------------------------- - echo WARNING: Make sure this script is run from the secondary server - echo --------------------------------------------------------------- - echo - echo Enter the IP or FQDN of the primary PostgreSQL server - read HOST - echo Enter the password for $USER@$HOST - read -s PASSWORD - echo Enter the required sslmode - read SSLMODE - - echo Stopping PostgreSQL and all GitLab services - sudo service gitlab stop - sudo service postgresql stop - - echo Backing up postgresql.conf - sudo -u postgres mv /var/opt/gitlab/postgresql/data/postgresql.conf /var/opt/gitlab/postgresql/ - - echo Cleaning up old cluster directory - sudo -u postgres rm -rf /var/opt/gitlab/postgresql/data - - echo Starting base backup as the replicator user - echo Enter the password for $USER@$HOST - sudo -u postgres /opt/gitlab/embedded/bin/pg_basebackup -h $HOST -D /var/opt/gitlab/postgresql/data -U gitlab_replicator -v -x -P - - echo Writing recovery.conf file - sudo -u postgres bash -c "cat > /var/opt/gitlab/postgresql/data/recovery.conf <<- _EOF1_ - standby_mode = 'on' - primary_conninfo = 'host=$HOST port=$PORT user=$USER password=$PASSWORD sslmode=$SSLMODE' - _EOF1_ - " - - echo Restoring postgresql.conf - sudo -u postgres mv /var/opt/gitlab/postgresql/postgresql.conf /var/opt/gitlab/postgresql/data/ - - echo Starting PostgreSQL - sudo service postgresql start - ``` - -1. Run it with: - - ```sh - bash /tmp/replica.sh - ``` - - When prompted, enter the IP/FQDN of the **primary** node, and the password you set up - for the `gitlab_replicator` user in the first step. - - You should use `verify-ca` for the `sslmode`. You can use `disable` if you - are happy to skip PostgreSQL TLS authentication altogether (e.g., you know - the network path is secure, or you are using a site-to-site VPN). This is - **not** safe over the public Internet! - - You can read more details about each `sslmode` in the - [PostgreSQL documentation][pg-docs-ssl]; - the instructions above are carefully written to ensure protection against - both passive eavesdroppers and active "man-in-the-middle" attackers. - -The replication process is now over. - -## PGBouncer support (optional) - -1. First, enter the PostgreSQL console as an admin user. - -1. Then create the read-only user: - - ```sql - -- NOTE: Use the password defined earlier - CREATE USER gitlab_geo_fdw WITH password '<your_password_here>'; - GRANT CONNECT ON DATABASE gitlabhq_production to gitlab_geo_fdw; - GRANT USAGE ON SCHEMA public TO gitlab_geo_fdw; - GRANT SELECT ON ALL TABLES IN SCHEMA public TO gitlab_geo_fdw; - GRANT SELECT ON ALL SEQUENCES IN SCHEMA public TO gitlab_geo_fdw; - - -- Tables created by "gitlab" should be made read-only for "gitlab_geo_fdw" - -- automatically. - ALTER DEFAULT PRIVILEGES FOR USER gitlab IN SCHEMA public GRANT SELECT ON TABLES TO gitlab_geo_fdw; - ALTER DEFAULT PRIVILEGES FOR USER gitlab IN SCHEMA public GRANT SELECT ON SEQUENCES TO gitlab_geo_fdw; - ``` - -1. Enter the PostgreSQL console on the **secondary** tracking database and change the user mapping to this new user: - - ``` - ALTER USER MAPPING FOR gitlab_geo SERVER gitlab_secondary OPTIONS (SET user 'gitlab_geo_fdw') - ``` - -## MySQL replication - -MySQL replication is not supported for Geo. - -## Troubleshooting - -Read the [troubleshooting document](troubleshooting.md). - -[replication-slots-article]: https://medium.com/@tk512/replication-slots-in-postgresql-b4b03d277c75 -[pgback]: http://www.postgresql.org/docs/9.6/static/app-pgbasebackup.html -[replication user]:https://wiki.postgresql.org/wiki/Streaming_Replication -[FDW]: https://www.postgresql.org/docs/9.6/static/postgres-fdw.html -[database]: database.md -[add-geo-node]: configuration.md#step-3-add-the-secondary-gitlab-node -[database-replication]: database.md#step-2-configure-the-secondary-server -[pg-docs-ssl]: https://www.postgresql.org/docs/9.6/static/libpq-ssl.html#LIBPQ-SSL-PROTECTION -[pg-docs-runtime-conn]: https://www.postgresql.org/docs/9.6/static/runtime-config-connection.html -[pg-docs-runtime-replication]: https://www.postgresql.org/docs/9.6/static/runtime-config-replication.html diff --git a/doc/administration/geo/replication/external_database.md b/doc/administration/geo/replication/external_database.md index dae5ed911b0..177ca68613e 100644 --- a/doc/administration/geo/replication/external_database.md +++ b/doc/administration/geo/replication/external_database.md @@ -129,7 +129,7 @@ To configure the connection to the external read-replica database and enable Log database to keep track of replication status and automatically recover from potential replication issues. Omnibus automatically configures a tracking database when `roles ['geo_secondary_role']` is set. For high availability, -refer to [Geo High Availability](https://docs.gitlab.com/ee/administration/high_availability). +refer to [Geo High Availability](../../high_availability/README.md). If you want to run this database external to Omnibus, please follow the instructions below. The tracking database requires an [FDW](https://www.postgresql.org/docs/9.6/static/postgres-fdw.html) diff --git a/doc/administration/geo/replication/high_availability.md b/doc/administration/geo/replication/high_availability.md index 715a83a9ff3..921a3ef1c7a 100644 --- a/doc/administration/geo/replication/high_availability.md +++ b/doc/administration/geo/replication/high_availability.md @@ -6,7 +6,7 @@ described, it is possible to adapt these instructions to your needs. ## Architecture overview - + _[diagram source - gitlab employees only][diagram-source]_ @@ -68,7 +68,7 @@ NOTE: **Note:** PostgreSQL and Redis should have already been disabled on the application servers, and connections from the application servers to those services on the backend servers configured, during normal GitLab HA set up. See high availability configuration documentation for -[PostgreSQL](https://docs.gitlab.com/ee/administration/high_availability/database.html#configuring-the-application-nodes) +[PostgreSQL](../../high_availability/database.md#configuring-the-application-nodes) and [Redis](../../high_availability/redis.md#example-configuration-for-the-gitlab-application). The **primary** database will require modification later, as part of diff --git a/doc/administration/geo/replication/index.md b/doc/administration/geo/replication/index.md index 6abea2cf271..54377f7ae4e 100644 --- a/doc/administration/geo/replication/index.md +++ b/doc/administration/geo/replication/index.md @@ -179,37 +179,20 @@ The steps below should be followed in the order they appear. **Make sure the Git If you installed GitLab using the Omnibus packages (highly recommended): 1. [Install GitLab Enterprise Edition](https://about.gitlab.com/installation/) on the server that will serve as the **secondary** node. Do not create an account or log in to the new **secondary** node. -1. [Upload the GitLab License](https://docs.gitlab.com/ee/user/admin_area/license.html) on the **primary** node to unlock Geo. The license must be for [GitLab Premium](https://about.gitlab.com/pricing/) or higher. +1. [Upload the GitLab License](../../../user/admin_area/license.md) on the **primary** node to unlock Geo. The license must be for [GitLab Premium](https://about.gitlab.com/pricing/) or higher. 1. [Set up the database replication](database.md) (`primary (read-write) <-> secondary (read-only)` topology). 1. [Configure fast lookup of authorized SSH keys in the database](../../operations/fast_ssh_key_lookup.md). This step is required and needs to be done on **both** the **primary** and **secondary** nodes. 1. [Configure GitLab](configuration.md) to set the **primary** and **secondary** nodes. 1. Optional: [Configure a secondary LDAP server](../../auth/ldap.md) for the **secondary** node. See [notes on LDAP](#ldap). 1. [Follow the "Using a Geo Server" guide](using_a_geo_server.md). -### Using GitLab installed from source (Deprecated) - -NOTE: **Note:** -In GitLab 11.5, support for using Geo in GitLab source installations was deprecated and will be removed in a future release. Please consider [migrating to GitLab Omnibus install](https://docs.gitlab.com/omnibus/update/convert_to_omnibus.html). - -If you installed GitLab from source: - -1. [Install GitLab Enterprise Edition](../../../install/installation.md) on the server that will serve as the **secondary** node. Do not create an account or log in to the new **secondary** node. -1. [Upload the GitLab License](https://docs.gitlab.com/ee/user/admin_area/license.html) on the **primary** node to unlock Geo. The license must be for [GitLab Premium](https://about.gitlab.com/pricing/) or higher. -1. [Set up the database replication](database_source.md) (`primary (read-write) <-> secondary (read-only)` topology). -1. [Configure fast lookup of authorized SSH keys in the database](../../operations/fast_ssh_key_lookup.md). Do this step for **both** **primary** and **secondary** nodes. -1. [Configure GitLab](configuration_source.md) to set the **primary** and **secondary** nodes. -1. [Follow the "Using a Geo Server" guide](using_a_geo_server.md). - ## Post-installation documentation After installing GitLab on the **secondary** nodes and performing the initial configuration, see the following documentation for post-installation information. ### Configuring Geo -For information on configuring Geo, see: - -- [Geo configuration (GitLab Omnibus)](configuration.md). -- [Geo configuration (source)](configuration_source.md). Configuring Geo in GitLab source installations was **deprecated** in GitLab 11.5. +For information on configuring Geo, see [Geo configuration](configuration.md). ### Updating Geo @@ -278,7 +261,7 @@ Take special note that these examples of GitLab features are both: Examples include: -- [Elasticsearch integration](https://docs.gitlab.com/ee/integration/elasticsearch.html). +- [Elasticsearch integration](../../../integration/elasticsearch.md). - [Container Registry](../../container_registry.md). [Object Storage](object_storage.md) can mitigate this. - [GitLab Pages](../../pages/index.md). - [Mattermost integration](https://docs.gitlab.com/omnibus/gitlab-mattermost/). diff --git a/doc/administration/geo/replication/security_review.md b/doc/administration/geo/replication/security_review.md index 46d3e68ab63..cd54e2dc8c4 100644 --- a/doc/administration/geo/replication/security_review.md +++ b/doc/administration/geo/replication/security_review.md @@ -120,9 +120,7 @@ questions from [owasp.org](https://www.owasp.org). ### What details regarding required OS components and lock‐down needs have been defined? -- The recommended installation method (Omnibus) packages most components itself. - A from-source installation method exists. Both are documented at - <https://docs.gitlab.com/ee/administration/geo/replication/index.html> +- The supported installation method (Omnibus) packages most components itself. - There are significant dependencies on the system-installed OpenSSH daemon (Geo requires users to set up custom authentication methods) and the omnibus or system-provided PostgreSQL daemon (it must be configured to listen on TCP, diff --git a/doc/administration/geo/replication/troubleshooting.md b/doc/administration/geo/replication/troubleshooting.md index 9c95720487d..c5bdd36ba70 100644 --- a/doc/administration/geo/replication/troubleshooting.md +++ b/doc/administration/geo/replication/troubleshooting.md @@ -23,6 +23,8 @@ to help identify if something is wrong:  +For information on how to resolve common errors reported from the UI, see [common errors](#common-errors). + If the UI is not working, or you are unable to log in, you can run the Geo health check manually to get this information as well as a few more details. This rake task can be run on an app node in the **primary** or **secondary** @@ -40,7 +42,8 @@ Checking Geo ... GitLab Geo is available ... yes GitLab Geo is enabled ... yes GitLab Geo secondary database is correctly configured ... yes -Using database streaming replication? ... yes +Database replication enabled? ... yes +Database replication working? ... yes GitLab Geo tracking database is configured to use Foreign Data Wrapper? ... yes GitLab Geo tracking database Foreign Data Wrapper schema is up-to-date? ... yes GitLab Geo HTTP(S) connectivity ... @@ -68,22 +71,22 @@ Example output: ``` http://secondary.example.com/ ----------------------------------------------------- - GitLab Version: 11.8.1-ee + GitLab Version: 11.10.4-ee Geo Role: Secondary Health Status: Healthy - Repositories: 190/190 (100%) - Verified Repositories: 190/190 (100%) - Wikis: 190/190 (100%) - Verified Wikis: 190/190 (100%) - LFS Objects: 35/35 (100%) - Attachments: 528/528 (100%) - CI job artifacts: 477/477 (100%) - Repositories Checked: 0/190 (0%) + Repositories: 289/289 (100%) + Verified Repositories: 289/289 (100%) + Wikis: 289/289 (100%) + Verified Wikis: 289/289 (100%) + LFS Objects: 8/8 (100%) + Attachments: 5/5 (100%) + CI job artifacts: 0/0 (0%) + Repositories Checked: 0/289 (0%) Sync Settings: Full Database replication lag: 0 seconds - Last event ID seen from primary: 2158 (about 2 minute ago) - Last event ID processed by cursor: 2158 (about 2 minute ago) - Last status report was: 4 minutes ago + Last event ID seen from primary: 10215 (about 2 minutes ago) + Last event ID processed by cursor: 10215 (about 2 minutes ago) + Last status report was: 2 minutes ago ``` ## Is Postgres replication working? @@ -307,7 +310,7 @@ same host on different ports. That is, 5432 and 5431 respectively. ### Checking configuration NOTE: **Note:** -The following steps are for Omnibus installs only. Using Geo with source-based installs [is deprecated](index.md#using-gitlab-installed-from-source-deprecated). +The following steps are for Omnibus installs only. Using Geo with source-based installs was **deprecated** in GitLab 11.5. To check the configuration: @@ -455,3 +458,57 @@ reload of the FDW schema. To manually reload the FDW schema: [database-start-replication]: database.md#step-3-initiate-the-replication-process [database-pg-replication]: database.md#postgresql-replication + +## Common errors + +This section documents common errors reported in the admin UI and how to fix them. + +### Geo database configuration file is missing + +GitLab cannot find or doesn't have permission to access the `database_geo.yml` configuration file. + +In an Omnibus GitLab installation, the file should be in `/var/opt/gitlab/gitlab-rails/etc`. +If it doesn't exist or inadvertent changes have been made to it, run `sudo gitlab-ctl reconfigure` to restore it to its correct state. + + +If this path is mounted on a remote volume, please check your volume configuration and that it has correct permissions. + +### Geo node has a database that is writable which is an indication it is not configured for replication with the primary node. + +This error refers to a problem with the database replica on a **secondary** node, +which Geo expects to have access to. It usually means, either: + +- An unsupported replication method was used (for example, logical replication). +- The instructions to setup a [Geo database replication](database.md) were not followed correctly. + +A common source of confusion with **secondary** nodes is that it requires two separate +PostgreSQL instances: + +- A read-only replica of the **primary** node. +- A regular, writable instance that holds replication metadata. That is, the Geo tracking database. + +### Geo node does not appear to be replicating the database from the primary node. + +The most common problems that prevent the database from replicating correctly are: + +- **Secondary** nodes cannot reach the **primary** node. Check credentials, firewall rules, etc. +- SSL certificate problems. Make sure you copied `/etc/gitlab/gitlab-secrets.json` from the **primary** node. +- Database storage disk is full. +- Database replication slot is misconfigured. +- Database is not using a replication slot or another alternative and cannot catch-up because WAL files were purged. + +Make sure you follow the [Geo database replication](database.md) instructions for supported configuration. + +### Geo database version (...) does not match latest migration (...) + +If you are using GitLab Omnibus installation, something might have failed during upgrade. You can: + +- Run `sudo gitlab-ctl reconfigure`. +- Manually trigger the database migration by running: `sudo gitlab-rake geo:db:migrate` as root on the **secondary** node. + +### Geo database is not configured to use Foreign Data Wrapper + +This error means the Geo Tracking Database doesn't have the FDW server and credentials +configured. + +See [How do I fix a "Foreign Data Wrapper (FDW) is not configured" error?](#how-do-i-fix-a-foreign-data-wrapper-fdw-is-not-configured-error). diff --git a/doc/administration/geo/replication/tuning.md b/doc/administration/geo/replication/tuning.md index b9921b2e69f..1943f2230df 100644 --- a/doc/administration/geo/replication/tuning.md +++ b/doc/administration/geo/replication/tuning.md @@ -13,5 +13,5 @@ However, this may not lead to more downloads in parallel unless the number of available Sidekiq threads is also increased. For example, if repository sync capacity is increased from 25 to 50, you may also want to increase the number of Sidekiq threads from 25 to 50. See the -[Sidekiq concurrency documentation](https://docs.gitlab.com/ee/administration/operations/extra_sidekiq_processes.html#number-of-threads) +[Sidekiq concurrency documentation](../../operations/extra_sidekiq_processes.md#number-of-threads) for more details. diff --git a/doc/administration/geo/replication/updating_the_geo_nodes.md b/doc/administration/geo/replication/updating_the_geo_nodes.md index 66728366e24..933a75c47d8 100644 --- a/doc/administration/geo/replication/updating_the_geo_nodes.md +++ b/doc/administration/geo/replication/updating_the_geo_nodes.md @@ -337,6 +337,53 @@ is prepended with the relevant node for better clarity: 1. **[secondary]** Create the `replica.sh` script as described in the [database configuration document][database-source-replication]. + 1. 1. **[secondary]** Save the snippet below in a file, let's say `/tmp/replica.sh`. Modify the + embedded paths if necessary: + + ``` + #!/bin/bash + + PORT="5432" + USER="gitlab_replicator" + echo --------------------------------------------------------------- + echo WARNING: Make sure this script is run from the secondary server + echo --------------------------------------------------------------- + echo + echo Enter the IP or FQDN of the primary PostgreSQL server + read HOST + echo Enter the password for $USER@$HOST + read -s PASSWORD + echo Enter the required sslmode + read SSLMODE + + echo Stopping PostgreSQL and all GitLab services + sudo service gitlab stop + sudo service postgresql stop + + echo Backing up postgresql.conf + sudo -u postgres mv /var/opt/gitlab/postgresql/data/postgresql.conf /var/opt/gitlab/postgresql/ + + echo Cleaning up old cluster directory + sudo -u postgres rm -rf /var/opt/gitlab/postgresql/data + + echo Starting base backup as the replicator user + echo Enter the password for $USER@$HOST + sudo -u postgres /opt/gitlab/embedded/bin/pg_basebackup -h $HOST -D /var/opt/gitlab/postgresql/data -U gitlab_replicator -v -x -P + + echo Writing recovery.conf file + sudo -u postgres bash -c "cat > /var/opt/gitlab/postgresql/data/recovery.conf <<- _EOF1_ + standby_mode = 'on' + primary_conninfo = 'host=$HOST port=$PORT user=$USER password=$PASSWORD sslmode=$SSLMODE' + _EOF1_ + " + + echo Restoring postgresql.conf + sudo -u postgres mv /var/opt/gitlab/postgresql/postgresql.conf /var/opt/gitlab/postgresql/data/ + + echo Starting PostgreSQL + sudo service postgresql start + ``` + 1. **[secondary]** Run the recovery script using the credentials from the previous step: @@ -396,8 +443,6 @@ and it is required since 10.0. [update]: ../../../update/README.md [database]: database.md -[database-replication]: database.md#step-3-initiate-the-replication-process -[database-source-replication]: database_source.md#step-3-initiate-the-replication-process [Hashed Storage]: ../../repository_storage_types.md [hashed-migration]: ../../raketasks/storage.md [ssh-fast-lookup]: ../../operations/fast_ssh_key_lookup.md diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md index f1cedb85455..53a85dfad6c 100644 --- a/doc/administration/gitaly/index.md +++ b/doc/administration/gitaly/index.md @@ -53,6 +53,10 @@ But since 11.8 the indexer uses Gitaly for data access as well. NFS can still be leveraged for redudancy on block level of the Git data. But only has to be mounted on the Gitaly server. +NOTE: **Note:** While Gitaly can be used as a replacement for NFS, we do not recommend +using EFS as it may impact GitLab's performance. Please review the [relevant documentation](../high_availability/nfs.md#avoid-using-awss-elastic-file-system-efs) +for more details. + ### Network architecture - gitlab-rails shards repositories into "repository storages" @@ -73,18 +77,29 @@ be mounted on the Gitaly server. - Gitaly servers must not be exposed to the public internet Gitaly network traffic is unencrypted by default, but supports -[TLS](#tls-support). Authentication is done through a static token. For -security in depth, its recommended to use a firewall to restrict access -to your Gitaly server. +[TLS](#tls-support). Authentication is done through a static token. + +NOTE: **Note:** Gitaly network traffic is unencrypted so we recommend a firewall to +restrict access to your Gitaly server. Below we describe how to configure a Gitaly server at address `gitaly.internal:8075` with secret token `abc123secret`. We assume your GitLab installation has two repository storages, `default` and `storage1`. +### Installation + +First install Gitaly using either Omnibus or from source. + +Omnibus: [Download/install](https://about.gitlab.com/installation) the Omnibus GitLab +package you want using **steps 1 and 2** from the GitLab downloads page but +**_do not_** provide the `EXTERNAL_URL=` value. + +Source: [Install Gitaly](../../install/installation.md#install-gitaly) + ### Client side token configuration -Start by configuring a token on the client side. +Configure a token on the client side. Omnibus installations: @@ -110,7 +125,7 @@ changes to be picked up. Next, on the Gitaly server, we need to configure storage paths, enable the network listener and configure the token. -Note: if you want to reduce the risk of downtime when you enable +NOTE: **Note:** if you want to reduce the risk of downtime when you enable authentication you can temporarily disable enforcement, see [the documentation on configuring Gitaly authentication](https://gitlab.com/gitlab-org/gitaly/blob/master/doc/configuration/README.md#authentication) @@ -122,12 +137,17 @@ the Gitaly server. The easiest way to accomplish this is to copy `/etc/gitlab/gi from an existing GitLab server to the Gitaly server. Without this shared secret, Git operations in GitLab will result in an API error. -> **NOTE:** In most or all cases the storage paths below end in `/repositories` which is +NOTE: **Note:** In most or all cases the storage paths below end in `/repositories` which is different than `path` in `git_data_dirs` of Omnibus installations. Check the directory layout on your Gitaly server to be sure. Omnibus installations: +<!-- +updates to following example must also be made at +https://gitlab.com/charts/gitlab/blob/master/doc/advanced/external-gitaly/external-omnibus-gitaly.md#configure-omnibus-gitlab +--> + ```ruby # /etc/gitlab/gitlab.rb @@ -147,6 +167,7 @@ gitlab_rails['auto_migrate'] = false # Configure the gitlab-shell API callback URL. Without this, `git push` will # fail. This can be your 'front door' GitLab URL or an internal load # balancer. +# Don't forget to copy `/etc/gitlab/gitlab-secrets.json` from web server to Gitaly server. gitlab_rails['internal_api_url'] = 'https://gitlab.example.com' # Make Gitaly accept connections on all network interfaces. You must use @@ -199,6 +220,9 @@ network, firewall, or name resolution problem preventing your GitLab server from reaching the Gitaly server then all Gitaly requests will fail. +Additionally, you need to +[disable Rugged if previously manually enabled](../high_availability/nfs.md#improving-nfs-performance-with-gitlab). + We assume that your Gitaly server can be reached at `gitaly.internal:8075` from your GitLab server, and that Gitaly can read and write to `/mnt/gitlab/default` and `/mnt/gitlab/storage1` respectively. diff --git a/doc/administration/high_availability/README.md b/doc/administration/high_availability/README.md index 24db1c28778..002deeaf945 100644 --- a/doc/administration/high_availability/README.md +++ b/doc/administration/high_availability/README.md @@ -64,8 +64,8 @@ larger one. - 1 PostgreSQL node - 1 Redis node -- 2 or more GitLab application nodes (Unicorn, Workhorse, Sidekiq) - 1 NFS/Gitaly storage server +- 2 or more GitLab application nodes (Unicorn, Workhorse, Sidekiq) #### Installation Instructions @@ -73,10 +73,10 @@ Complete the following installation steps in order. A link at the end of each section will bring you back to the Scalable Architecture Examples section so you can continue with the next step. -1. [PostgreSQL](./database.md#postgresql-in-a-scaled-environment) -1. [Redis](./redis.md#redis-in-a-scaled-environment) -1. [Gitaly](./gitaly.md) (recommended) or [NFS](./nfs.md) -1. [GitLab application nodes](./gitlab.md) +1. [PostgreSQL](database.md#postgresql-in-a-scaled-environment) +1. [Redis](redis.md#redis-in-a-scaled-environment) +1. [Gitaly](gitaly.md) (recommended) or [NFS](nfs.md) +1. [GitLab application nodes](gitlab.md) ### Full Scaling @@ -88,9 +88,9 @@ in size, indicating that there is contention or not enough resources. - 1 PostgreSQL node - 1 Redis node -- 2 or more GitLab application nodes (Unicorn, Workhorse) -- 2 or more Sidekiq nodes - 2 or more NFS/Gitaly storage servers +- 2 or more Sidekiq nodes +- 2 or more GitLab application nodes (Unicorn, Workhorse) ## High Availability Architecture Examples @@ -135,7 +135,7 @@ the contention. - 2 or more GitLab application nodes (Unicorn, Workhorse, Sidekiq, PGBouncer) - 1 NFS/Gitaly server - + ### Hybrid @@ -153,7 +153,7 @@ contention due to certain workloads. - 1 or more NFS/Gitaly servers - 1 Monitoring node (Prometheus, Grafana) - + #### Reference Architecture @@ -194,7 +194,7 @@ with the added complexity of many more nodes to configure, manage and monitor. - 2 or more Web nodes (All other web requests) - 2 or more NFS/Gitaly servers - + The following pages outline the steps necessary to configure each component separately: diff --git a/doc/administration/high_availability/database.md b/doc/administration/high_availability/database.md index 1648b6b848a..3b874e5d312 100644 --- a/doc/administration/high_availability/database.md +++ b/doc/administration/high_availability/database.md @@ -17,16 +17,16 @@ If you use a cloud-managed service, or provide your own PostgreSQL: ## PostgreSQL in a Scaled Environment -This section is relevant for [Scaled Architecture](./README.md#scalable-architecture-examples) -environments including [Basic Scaling](./README.md#basic-scaling) and -[Full Scaling](./README.md#full-scaling). +This section is relevant for [Scaled Architecture](README.md#scalable-architecture-examples) +environments including [Basic Scaling](README.md#basic-scaling) and +[Full Scaling](README.md#full-scaling). ### Provide your own PostgreSQL instance **[CORE ONLY]** -If you want to use your own deployed PostgreSQL instance(s), +If you want to use your own deployed PostgreSQL instance(s), see [Provide your own PostgreSQL instance](#provide-your-own-postgresql-instance-core-only) -for more details. However, you can use the GitLab Omnibus package to easily -deploy the bundled PostgreSQL. +for more details. However, you can use the GitLab Omnibus package to easily +deploy the bundled PostgreSQL. ### Standalone PostgreSQL using GitLab Omnibus **[CORE ONLY]** @@ -36,19 +36,19 @@ deploy the bundled PostgreSQL. - Do not complete any other steps on the download page. 1. Generate a password hash for PostgreSQL. This assumes you will use the default username of `gitlab` (recommended). The command will request a password - and confirmation. Use the value that is output by this command in the next + and confirmation. Use the value that is output by this command in the next step as the value of `POSTGRESQL_PASSWORD_HASH`. ```sh sudo gitlab-ctl pg-password-md5 gitlab ``` - + 1. Edit `/etc/gitlab/gitlab.rb` and add the contents below, updating placeholder - values appropriately. - + values appropriately. + - `POSTGRESQL_PASSWORD_HASH` - The value output from the previous step - `APPLICATION_SERVER_IP_BLOCKS` - A space delimited list of IP subnets or IP - addresses of the GitLab application servers that will connect to the + addresses of the GitLab application servers that will connect to the database. Example: `%w(123.123.123.123/32 123.123.123.234/32)` ```ruby @@ -65,11 +65,11 @@ deploy the bundled PostgreSQL. postgresql['listen_address'] = '0.0.0.0' postgresql['port'] = 5432 - # Replace POSTGRESQL_PASSWORD_HASH with a generated md5 value + # Replace POSTGRESQL_PASSWORD_HASH with a generated md5 value postgresql['sql_user_password'] = 'POSTGRESQL_PASSWORD_HASH' # Replace XXX.XXX.XXX.XXX/YY with Network Address - # ???? + # ???? postgresql['trust_auth_cidr_addresses'] = %w(APPLICATION_SERVER_IP_BLOCKS) # Disable automatic database migrations @@ -77,42 +77,43 @@ deploy the bundled PostgreSQL. ``` NOTE: **Note:** The role `postgres_role` was introduced with GitLab 10.3 - + 1. [Reconfigure GitLab] for the changes to take effect. 1. Note the PostgreSQL node's IP address or hostname, port, and plain text password. These will be necessary when configuring the GitLab application servers later. - + Advanced configuration options are supported and can be added if needed. Continue configuration of other components by going -[back to Scaled Architectures](./README.md#scalable-architecture-examples) +[back to Scaled Architectures](README.md#scalable-architecture-examples) ## PostgreSQL with High Availability -This section is relevant for [High Availability Architecture](./README.md#high-availability-architecture-examples) -environments including [Horizontal](./README.md#horizontal), -[Hybrid](./README.md#hybrid), and -[Fully Distributed](./README.md#fully-distributed). +This section is relevant for [High Availability Architecture](README.md#high-availability-architecture-examples) +environments including [Horizontal](README.md#horizontal), +[Hybrid](README.md#hybrid), and +[Fully Distributed](README.md#fully-distributed). ### Provide your own PostgreSQL instance **[CORE ONLY]** If you want to use your own deployed PostgreSQL instance(s), see [Provide your own PostgreSQL instance](#provide-your-own-postgresql-instance-core-only) for more details. However, you can use the GitLab Omnibus package to easily -deploy the bundled PostgreSQL. +deploy the bundled PostgreSQL. ### High Availability with GitLab Omnibus **[PREMIUM ONLY]** > Important notes: +> > - This document will focus only on configuration supported with [GitLab Premium](https://about.gitlab.com/pricing/), using the Omnibus GitLab package. > - If you are a Community Edition or Starter user, consider using a cloud hosted solution. > - This document will not cover installations from source. > > - If HA setup is not what you were looking for, see the [database configuration document](http://docs.gitlab.com/omnibus/settings/database.html) > for the Omnibus GitLab packages. - +> > Please read this document fully before attempting to configure PostgreSQL HA > for GitLab. > @@ -122,9 +123,9 @@ The recommended configuration for a PostgreSQL HA requires: - A minimum of three database nodes - Each node will run the following services: - - `PostgreSQL` - The database itself - - `repmgrd` - A service to monitor, and handle failover in case of a failure - - `Consul` agent - Used for service discovery, to alert other nodes when failover occurs + - `PostgreSQL` - The database itself + - `repmgrd` - A service to monitor, and handle failover in case of a failure + - `Consul` agent - Used for service discovery, to alert other nodes when failover occurs - A minimum of three `Consul` server nodes - A minimum of one `pgbouncer` service node @@ -134,7 +135,7 @@ otherwise the networks will become a single point of failure. #### Architecture - + Database nodes run two services with PostgreSQL: @@ -142,7 +143,7 @@ Database nodes run two services with PostgreSQL: - Selecting a new master for the cluster. - Promoting the new node to master. - Instructing remaining servers to follow the new master node. - + On failure, the old master node is automatically evicted from the cluster, and should be rejoined manually once recovered. - Consul. Monitors the status of each node in the database cluster and tracks its health in a service definition on the consul cluster. @@ -171,13 +172,10 @@ Similarly, PostgreSQL access is controlled based on the network source. This is why you will need: -> IP address of each nodes network interface -> - This can be set to `0.0.0.0` to listen on all interfaces. It cannot -> be set to the loopack address `127.0.0.1` -> -> Network Address -> - This can be in subnet (i.e. `192.168.0.0/255.255.255.0`) or CIDR (i.e. -> `192.168.0.0/24`) form. +- IP address of each nodes network interface. This can be set to `0.0.0.0` to + listen on all interfaces. It cannot be set to the loopack address `127.0.0.1`. +- Network Address. This can be in subnet (i.e. `192.168.0.0/255.255.255.0`) + or CIDR (i.e. `192.168.0.0/24`) form. ##### User information @@ -199,7 +197,7 @@ When using default setup, minimum configuration requires: sudo gitlab-ctl pg-password-md5 CONSUL_USERNAME ``` -- `CONSUL_SERVER_NODES`. The IP addresses or DNS records of the Consul server nodes. +- `CONSUL_SERVER_NODES`. The IP addresses or DNS records of the Consul server nodes. Few notes on the service itself: @@ -220,8 +218,7 @@ the number of database nodes in the cluster. This is used to prevent replication from using up all of the available database connections. -> Note: -> - In this document we are assuming 3 database nodes, which makes this configuration: +In this document we are assuming 3 database nodes, which makes this configuration: ``` postgresql['max_wal_senders'] = 4 @@ -277,7 +274,7 @@ be allowed to authenticate with the service. Few notes on the service itself: - The service runs under the same system account as the database - - In the package, this is by default `gitlab-psql` + - In the package, this is by default `gitlab-psql` - The service will have a superuser database user account generated for it - This defaults to `gitlab_repmgr` @@ -327,7 +324,7 @@ On each Consul node perform the following: Before moving on, make sure Consul is configured correctly. Run the following command to verify all server nodes are communicating: -``` +```sh /opt/gitlab/embedded/bin/consul members ``` @@ -401,14 +398,15 @@ check the [Troubleshooting section](#troubleshooting) before proceeding. repmgr['master_on_initialization'] = false ``` -1. [Reconfigure GitLab] for te changes to take effect. +1. [Reconfigure GitLab] for the changes to take effect. > Please note: +> > - If you want your database to listen on a specific interface, change the config: -> `postgresql['listen_address'] = '0.0.0.0'` +> `postgresql['listen_address'] = '0.0.0.0'`. > - If your Pgbouncer service runs under a different user account, > you also need to specify: `postgresql['pgbouncer_user'] = PGBOUNCER_USERNAME` in -> your configuration +> your configuration. ##### Database nodes post-configuration @@ -449,7 +447,6 @@ Select one node as a primary node. is not an IP address, it will need to be a resolvable name (via DNS or `/etc/hosts`) - ###### Secondary nodes 1. Set up the repmgr standby: @@ -500,7 +497,7 @@ Before moving on, make sure the databases are configured correctly. Run the following command on the **primary** node to verify that replication is working properly: -``` +```sh gitlab-ctl repmgr cluster show ``` @@ -518,7 +515,7 @@ If the 'Role' column for any node says "FAILED", check the Also, check that the check master command works successfully on each node: -``` +```sh su - gitlab-consul gitlab-ctl repmgr-check-master || echo 'This node is a standby repmgr node' ``` @@ -649,7 +646,7 @@ in the Troubleshooting section before proceeding. ##### Ensure GitLab is running At this point, your GitLab instance should be up and running. Verify you are -able to login, and create issues and merge requests. If you have troubles check +able to login, and create issues and merge requests. If you have troubles check the [Troubleshooting section](#troubleshooting). #### Example configuration @@ -665,13 +662,13 @@ can connect to each freely other on those addresses. Here is a list and description of each machine and the assigned IP: -* `10.6.0.11`: Consul 1 -* `10.6.0.12`: Consul 2 -* `10.6.0.13`: Consul 3 -* `10.6.0.21`: PostgreSQL master -* `10.6.0.22`: PostgreSQL secondary -* `10.6.0.23`: PostgreSQL secondary -* `10.6.0.31`: GitLab application +- `10.6.0.11`: Consul 1 +- `10.6.0.12`: Consul 2 +- `10.6.0.13`: Consul 3 +- `10.6.0.21`: PostgreSQL master +- `10.6.0.22`: PostgreSQL secondary +- `10.6.0.23`: PostgreSQL secondary +- `10.6.0.31`: GitLab application All passwords are set to `toomanysecrets`, please do not use this password or derived hashes. @@ -735,7 +732,7 @@ consul['configuration'] = { On secondary nodes, edit `/etc/gitlab/gitlab.rb` and add all the configuration added to primary node, noted above. In addition, append the following -configuration +configuration: ``` # HA setting to specify if a node should attempt to be master on initialization @@ -839,10 +836,10 @@ In this example we start with all servers on the same 10.6.0.0/16 private networ Here is a list and description of each machine and the assigned IP: -* `10.6.0.21`: PostgreSQL master -* `10.6.0.22`: PostgreSQL secondary -* `10.6.0.23`: PostgreSQL secondary -* `10.6.0.31`: GitLab application +- `10.6.0.21`: PostgreSQL master +- `10.6.0.22`: PostgreSQL secondary +- `10.6.0.23`: PostgreSQL secondary +- `10.6.0.31`: GitLab application All passwords are set to `toomanysecrets`, please do not use this password or derived hashes. @@ -853,6 +850,7 @@ Please note that after the initial configuration, if a failover occurs, the Post ##### Example minimal configuration for database servers ##### Primary node + On primary database node edit `/etc/gitlab/gitlab.rb`: ```ruby @@ -1047,7 +1045,6 @@ For example: repmgr['trust_auth_cidr_addresses'] = %w(192.168.1.44/32 db2.example.com) ``` - ##### MD5 Authentication If you are running on an untrusted network, repmgr can use md5 authentication @@ -1114,7 +1111,7 @@ steps to fix the problem: 1. Change to the `gitlab-consul` user - `su - gitlab-consul` 1. Try the check command again - `gitlab-ctl repmgr-check-master`. -Now there should not be errors. If errors still occur then there is another problem. +Now there should not be errors. If errors still occur then there is another problem. #### PGBouncer error `ERROR: pgbouncer cannot connect to server` @@ -1157,8 +1154,6 @@ If you're running into an issue with a component not outlined here, be sure to c **Note**: We recommend that you follow the instructions here for a full [PostgreSQL cluster](#high-availability-with-gitlab-omnibus-premium-only). If you are reading this section due to an old bookmark, you can find that old documentation [in the repository](https://gitlab.com/gitlab-org/gitlab-ce/blob/v10.1.4/doc/administration/high_availability/database.md#configure-using-omnibus). ---- - Read more on high-availability configuration: 1. [Configure Redis](redis.md) diff --git a/doc/administration/high_availability/gitaly.md b/doc/administration/high_availability/gitaly.md index d44744f2af8..1d8e6c999cb 100644 --- a/doc/administration/high_availability/gitaly.md +++ b/doc/administration/high_availability/gitaly.md @@ -7,84 +7,15 @@ should consider using Gitaly on a separate node. See the [Gitaly HA Epic](https://gitlab.com/groups/gitlab-org/-/epics/289) to track plans and progress toward high availability support. -This document is relevant for [Scaled Architecture](./README.md#scalable-architecture-examples) -environments and [High Availability Architecture](./README.md#high-availability-architecture-examples). +This document is relevant for [Scaled Architecture](README.md#scalable-architecture-examples) +environments and [High Availability Architecture](README.md#high-availability-architecture-examples). ## Running Gitaly on its own server -Starting with GitLab 11.4, Gitaly is a replacement for NFS except -when the [Elastic Search indexer](https://gitlab.com/gitlab-org/gitlab-elasticsearch-indexer) -is used. - -NOTE: **Note:** While Gitaly can be used as a replacement for NFS, we do not recommend using EFS as it may impact GitLab's performance. Please review the [relevant documentation](nfs.md#avoid-using-awss-elastic-file-system-efs) for more details. - -NOTE: **Note:** Gitaly network traffic is unencrypted so we recommend a firewall to -restrict access to your Gitaly server. - -The steps below are the minimum necessary to configure a Gitaly server with -Omnibus: - -1. SSH into the Gitaly server. -1. [Download/install](https://about.gitlab.com/installation) the Omnibus GitLab - package you want using **steps 1 and 2** from the GitLab downloads page. - - Do not complete any other steps on the download page. - -1. Edit `/etc/gitlab/gitlab.rb` and add the contents: - - Gitaly must trigger some callbacks to GitLab via GitLab Shell. As a result, - the GitLab Shell secret must be the same between the other GitLab servers and - the Gitaly server. The easiest way to accomplish this is to copy `/etc/gitlab/gitlab-secrets.json` - from an existing GitLab server to the Gitaly server. Without this shared secret, - Git operations in GitLab will result in an API error. - - > **NOTE:** In most or all cases the storage paths below end in `repositories` which is - different than `path` in `git_data_dirs` of Omnibus installations. Check the - directory layout on your Gitaly server to be sure. - - ```ruby - # Enable Gitaly - gitaly['enable'] = true - - ## Disable all other services - sidekiq['enable'] = false - gitlab_workhorse['enable'] = false - unicorn['enable'] = false - postgresql['enable'] = false - nginx['enable'] = false - prometheus['enable'] = false - alertmanager['enable'] = false - pgbouncer_exporter['enable'] = false - redis_exporter['enable'] = false - gitlab_monitor['enable'] = false - - # Prevent database connections during 'gitlab-ctl reconfigure' - gitlab_rails['rake_cache_clear'] = false - gitlab_rails['auto_migrate'] = false - - # Configure the gitlab-shell API callback URL. Without this, `git push` will - # fail. This can be your 'front door' GitLab URL or an internal load - # balancer. - gitlab_rails['internal_api_url'] = 'https://gitlab.example.com' - - # Make Gitaly accept connections on all network interfaces. You must use - # firewalls to restrict access to this address/port. - gitaly['listen_addr'] = "0.0.0.0:8075" - gitaly['auth_token'] = 'abc123secret' - - gitaly['storage'] = [ - { 'name' => 'default', 'path' => '/mnt/gitlab/default/repositories' }, - { 'name' => 'storage1', 'path' => '/mnt/gitlab/storage1/repositories' }, - ] - - # To use tls for gitaly you need to add - gitaly['tls_listen_addr'] = "0.0.0.0:9999" - gitaly['certificate_path'] = "path/to/cert.pem" - gitaly['key_path'] = "path/to/key.pem" - ``` - -Again, reconfigure (Omnibus) or restart (source). +See [Running Gitaly on its own server](../gitaly/index.md#running-gitaly-on-its-own-server) +in Gitaly documentation. Continue configuration of other components by going back to: -- [Scaled Architectures](./README.md#scalable-architecture-examples) -- [High Availability Architectures](./README.md#high-availability-architecture-examples) +- [Scaled Architectures](README.md#scalable-architecture-examples) +- [High Availability Architectures](README.md#high-availability-architecture-examples) diff --git a/doc/administration/high_availability/img/fully-distributed.png b/doc/administration/high_availability/img/fully-distributed.png Binary files differnew file mode 100644 index 00000000000..ad23207134e --- /dev/null +++ b/doc/administration/high_availability/img/fully-distributed.png diff --git a/doc/administration/high_availability/img/geo-ha-diagram.png b/doc/administration/high_availability/img/geo-ha-diagram.png Binary files differnew file mode 100644 index 00000000000..da5d612827c --- /dev/null +++ b/doc/administration/high_availability/img/geo-ha-diagram.png diff --git a/doc/administration/high_availability/img/horizontal.png b/doc/administration/high_availability/img/horizontal.png Binary files differnew file mode 100644 index 00000000000..c3bd489d96f --- /dev/null +++ b/doc/administration/high_availability/img/horizontal.png diff --git a/doc/administration/high_availability/img/hybrid.png b/doc/administration/high_availability/img/hybrid.png Binary files differnew file mode 100644 index 00000000000..7d4a56bf0ea --- /dev/null +++ b/doc/administration/high_availability/img/hybrid.png diff --git a/doc/administration/high_availability/pg_ha_architecture.png b/doc/administration/high_availability/img/pg_ha_architecture.png Binary files differindex ef870f652ae..ef870f652ae 100644 --- a/doc/administration/high_availability/pg_ha_architecture.png +++ b/doc/administration/high_availability/img/pg_ha_architecture.png diff --git a/doc/administration/high_availability/nfs.md b/doc/administration/high_availability/nfs.md index caadec3ac4e..490c2699458 100644 --- a/doc/administration/high_availability/nfs.md +++ b/doc/administration/high_availability/nfs.md @@ -39,13 +39,8 @@ options: ### Improving NFS performance with GitLab -NOTE: **Note:** -This is only available starting in certain versions of GitLab: - * 11.5.11 - * 11.6.11 - * 11.7.12 - * 11.8.8 - * 11.9.0 and up (e.g. 11.10, 11.11, etc.) +NOTE: **Note:** This is only available starting in certain versions of GitLab: 11.5.11, +11.6.11, 11.7.12, 11.8.8, 11.9.0 and up (e.g. 11.10, 11.11, etc.) If you are using NFS to share Git data, we recommend that you enable a number of feature flags that will allow GitLab application processes to @@ -61,7 +56,8 @@ To do this, run the Rake task: sudo gitlab-rake gitlab:features:enable_rugged ``` -If you need to undo this setting for some reason, run: +If you need to undo this setting for some reason such as switching to [Gitaly without NFS](gitaly.md) +(recommended), run: ```sh sudo gitlab-rake gitlab:features:disable_rugged @@ -107,6 +103,11 @@ stored on a local volume. For more details on another person's experience with EFS, see [Amazon's Elastic File System: Burst Credits](https://rawkode.com/2017/04/16/amazons-elastic-file-system-burst-credits/) +## Avoid using CephFS and GlusterFS + +GitLab strongly recommends against using CephFS and GlusterFS. +These distributed file systems are not well-suited for GitLab's input/output access patterns because git uses many small files and access times and file locking times to propagate will make git activity very slow. + ## Avoid using PostgreSQL with NFS GitLab strongly recommends against running your PostgreSQL database @@ -122,7 +123,7 @@ Additionally, this configuration is specifically warned against in the >to the NFS server can cause data corruption problems. For supported database architecture, please see our documentation on -[Configuring a Database for GitLab HA](https://docs.gitlab.com/ee/administration/high_availability/database.html). +[Configuring a Database for GitLab HA](database.md). ## NFS Client mount options diff --git a/doc/administration/high_availability/nfs_host_client_setup.md b/doc/administration/high_availability/nfs_host_client_setup.md index a8bc101dee6..a8d69b9ab0a 100644 --- a/doc/administration/high_availability/nfs_host_client_setup.md +++ b/doc/administration/high_availability/nfs_host_client_setup.md @@ -67,7 +67,7 @@ apt-get install nfs-common ### Step 2 - Create Mount Points on Client -Create a directroy on the client that we can mount the shared directory from the host. +Create a directory on the client that we can mount the shared directory from the host. Please note that if your mount point directory contains any files they will be hidden once the remote shares are mounted. An empty/new directory on the client is recommended for this purpose. diff --git a/doc/administration/high_availability/redis.md b/doc/administration/high_availability/redis.md index 46ad3ecd9bb..1aaa709fc8f 100644 --- a/doc/administration/high_availability/redis.md +++ b/doc/administration/high_availability/redis.md @@ -16,9 +16,9 @@ These will be necessary when configuring the GitLab application servers later. ## Redis in a Scaled Environment -This section is relevant for [Scaled Architecture](./README.md#scalable-architecture-examples) -environments including [Basic Scaling](./README.md#basic-scaling) and -[Full Scaling](./README.md#full-scaling). +This section is relevant for [Scaled Architecture](README.md#scalable-architecture-examples) +environments including [Basic Scaling](README.md#basic-scaling) and +[Full Scaling](README.md#full-scaling). ### Provide your own Redis instance **[CORE ONLY]** @@ -34,7 +34,7 @@ In this configuration Redis is not highly available, and represents a single point of failure. However, in a scaled environment the objective is to allow the environment to handle more users or to increase throughput. Redis itself is generally stable and can handle many requests so it is an acceptable -trade off to have only a single instance. See [Scaling and High Availability](./README.md) +trade off to have only a single instance. See [Scaling and High Availability](README.md) for an overview of GitLab scaling and high availability options. The steps below are the minimum necessary to configure a Redis server with @@ -79,14 +79,14 @@ Advanced configuration options are supported and can be added if needed. Continue configuration of other components by going -[back to Scaled Architectures](./README.md#scalable-architecture-examples) +[back to Scaled Architectures](README.md#scalable-architecture-examples) ## Redis with High Availability -This section is relevant for [High Availability Architecture](./README.md#high-availability-architecture-examples) -environments including [Horizontal](./README.md#horizontal), -[Hybrid](./README.md#hybrid), and -[Fully Distributed](./README.md#fully-distributed). +This section is relevant for [High Availability Architecture](README.md#high-availability-architecture-examples) +environments including [Horizontal](README.md#horizontal), +[Hybrid](README.md#hybrid), and +[Fully Distributed](README.md#fully-distributed). ### Provide your own Redis instance **[CORE ONLY]** @@ -863,7 +863,7 @@ You can check if everything is correct by connecting to each server using `redis-cli` application, and sending the `info replication` command as below. ``` -/opt/gitlab/embedded/bin/redis-cli -a <redis-password> info replication +/opt/gitlab/embedded/bin/redis-cli -h <redis-host-or-ip> -a '<redis-password>' info replication ``` When connected to a `master` redis, you will see the number of connected diff --git a/doc/administration/incoming_email.md b/doc/administration/incoming_email.md index 9ac310af248..8271c579f5b 100644 --- a/doc/administration/incoming_email.md +++ b/doc/administration/incoming_email.md @@ -10,7 +10,7 @@ GitLab has several features based on receiving incoming emails: - [New merge request by email](../user/project/merge_requests/index.md#create-new-merge-requests-by-email): allow GitLab users to create a new merge request by sending an email to a user-specific email address. -- [Service Desk](https://docs.gitlab.com/ee/user/project/service_desk.html): provide e-mail support to +- [Service Desk](../user/project/service_desk.md): provide e-mail support to your customers through GitLab. **[PREMIUM]** ## Requirements diff --git a/doc/administration/index.md b/doc/administration/index.md index 797a7242bd0..95a0e84deb6 100644 --- a/doc/administration/index.md +++ b/doc/administration/index.md @@ -20,7 +20,7 @@ GitLab Community Edition installations only have access to Core features. GitLab.com is administered by GitLab, Inc., therefore, only GitLab team members have access to its admin configurations. If you're a GitLab.com user, please check the -[user documentation](../user/index.html). +[user documentation](../user/index.md). NOTE: **Note:** Non-administrator users don’t have access to GitLab administration tools and settings. @@ -37,9 +37,9 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Omnibus support for log forwarding](https://docs.gitlab.com/omnibus/settings/logs.html#udp-log-shipping-gitlab-enterprise-edition-only) **[STARTER ONLY]** - [High Availability](high_availability/README.md): Configure multiple servers for scaling or high availability. - [High Availability on AWS](../university/high-availability/aws/README.md): Set up GitLab HA on Amazon AWS. -- [Geo](https://docs.gitlab.com/ee/administration/geo/replication/index.html): Replicate your GitLab instance to other geographic locations as a read-only fully operational version. **[PREMIUM ONLY]** -- [Disaster Recovery](https://docs.gitlab.com/ee/administration/geo/disaster_recovery/index.html): Quickly fail-over to a different site with minimal effort in a disaster situation. **[PREMIUM ONLY]** -- [Pivotal Tile](https://docs.gitlab.com/ee/install/pivotal/index.html): Deploy GitLab as a pre-configured appliance using Ops Manager (BOSH) for Pivotal Cloud Foundry. **[PREMIUM ONLY]** +- [Geo](geo/replication/index.md): Replicate your GitLab instance to other geographic locations as a read-only fully operational version. **[PREMIUM ONLY]** +- [Disaster Recovery](geo/disaster_recovery/index.md): Quickly fail-over to a different site with minimal effort in a disaster situation. **[PREMIUM ONLY]** +- [Pivotal Tile](../install/pivotal/index.md): Deploy GitLab as a pre-configured appliance using Ops Manager (BOSH) for Pivotal Cloud Foundry. **[PREMIUM ONLY]** - [Add License](../user/admin_area/license.md): Upload a license at install time to unlock features that are in paid tiers of GitLab. **[STARTER ONLY]** ### Configuring GitLab @@ -61,9 +61,9 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Diff limits](../user/admin_area/diff_limits.md): Configure the diff rendering size limits of branch comparison pages. - [Merge request diffs storage](merge_request_diffs.md): Configure merge requests diffs external storage. - [Broadcast Messages](../user/admin_area/broadcast_messages.md): Send messages to GitLab users through the UI. -- [Elasticsearch](https://docs.gitlab.com/ee/integration/elasticsearch.html): Enable Elasticsearch to empower GitLab's Advanced Global Search. Useful when you deal with a huge amount of data. **[STARTER ONLY]** +- [Elasticsearch](../integration/elasticsearch.md): Enable Elasticsearch to empower GitLab's Advanced Global Search. Useful when you deal with a huge amount of data. **[STARTER ONLY]** - [External Classification Policy Authorization](../user/admin_area/settings/external_authorization.md) **[PREMIUM ONLY]** -- [Upload a license](https://docs.gitlab.com/ee/user/admin_area/license.html): Upload a license to unlock features that are in paid tiers of GitLab. **[STARTER ONLY]** +- [Upload a license](../user/admin_area/license.md): Upload a license to unlock features that are in paid tiers of GitLab. **[STARTER ONLY]** - [Admin Area](../user/admin_area/index.md): for self-managed instance-wide configuration and maintenance. #### Customizing GitLab's appearance @@ -73,7 +73,7 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Branded login page](../customization/branded_login_page.md): Customize the login page with your own logo, title, and description. - [Welcome message](../customization/welcome_message.md): Add a custom welcome message to the sign-in page. - ["New Project" page](../customization/new_project_page.md): Customize the text to be displayed on the page that opens whenever your users create a new project. -- [Additional custom email text](https://docs.gitlab.com/ee/user/admin_area/settings/email.html#custom-additional-text-premium-only): Add additional custom text to emails sent from GitLab. **[PREMIUM ONLY]** +- [Additional custom email text](../user/admin_area/settings/email.md#custom-additional-text-premium-only): Add additional custom text to emails sent from GitLab. **[PREMIUM ONLY]** ### Maintaining GitLab @@ -108,34 +108,29 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Sign-up restrictions](../user/admin_area/settings/sign_up_restrictions.md): block email addresses of specific domains, or whitelist only specific domains. - [Access restrictions](../user/admin_area/settings/visibility_and_access_controls.md#enabled-git-access-protocols): Define which Git access protocols can be used to talk to GitLab (SSH, HTTP, HTTPS). - [Authentication and Authorization](auth/README.md): Configure external authentication with LDAP, SAML, CAS and additional providers. - - [Sync LDAP](https://docs.gitlab.com/ee/administration/auth/ldap-ee.html) **[STARTER ONLY]** - - [Kerberos authentication](https://docs.gitlab.com/ee/integration/kerberos.html) **[STARTER ONLY]** + - [Sync LDAP](auth/ldap-ee.md) **[STARTER ONLY]** + - [Kerberos authentication](../integration/kerberos.md) **[STARTER ONLY]** - See also other [authentication](../topics/authentication/index.md#gitlab-administrators) topics (for example, enforcing 2FA). -- [Email users](https://docs.gitlab.com/ee/tools/email.html): Email GitLab users from within GitLab. **[STARTER ONLY]** +- [Email users](../tools/email.md): Email GitLab users from within GitLab. **[STARTER ONLY]** - [User Cohorts](../user/admin_area/user_cohorts.md): Display the monthly cohorts of new users and their activities over time. - [Audit logs and events](audit_events.md): View the changes made within the GitLab server for: - Groups and projects. **[STARTER]** - Instances. **[PREMIUM ONLY]** - [Auditor users](auditor_users.md): Users with read-only access to all projects, groups, and other resources on the GitLab instance. **[PREMIUM ONLY]** - [Incoming email](incoming_email.md): Configure incoming emails to allow - users to [reply by email], create [issues by email] and - [merge requests by email], and to enable [Service Desk]. + users to [reply by email](reply_by_email.md), create [issues by email](../user/project/issues/create_new_issue.md#new-issue-via-email) and + [merge requests by email](../user/project/merge_requests/index.md#create-new-merge-requests-by-email), and to enable [Service Desk](../user/project/service_desk.md). - [Postfix for incoming email](reply_by_email_postfix_setup.md): Set up a basic Postfix mail server with IMAP authentication on Ubuntu for incoming emails. - [Abuse reports](../user/admin_area/abuse_reports.md): View and resolve abuse reports from your users. -[reply by email]: reply_by_email.md -[issues by email]: ../user/project/issues/create_new_issue.md#new-issue-via-email -[merge requests by email]: ../user/project/merge_requests/index.md#create-new-merge-requests-by-email -[Service Desk]: https://docs.gitlab.com/ee/user/project/service_desk.html - ## Project settings - [Container Registry](container_registry.md): Configure Container Registry with GitLab. - [Issue closing pattern](issue_closing_pattern.md): Customize how to close an issue from commit messages. - [Gitaly](gitaly/index.md): Configuring Gitaly, GitLab's Git repository storage service. -- [Default labels](../user/admin_area/labels.html): Create labels that will be automatically added to every new project. +- [Default labels](../user/admin_area/labels.md): Create labels that will be automatically added to every new project. - [Restrict the use of public or internal projects](../public_access/public_access.md#restricting-the-use-of-public-or-internal-projects): Restrict the use of visibility levels for users when they create a project or a snippet. - [Custom project templates](../user/admin_area/custom_project_templates.md): Configure a set of projects to be used as custom templates when creating a new project. **[PREMIUM ONLY]** - [Packages](packages.md): Enable GitLab to act as a Maven repository or NPM registry. **[PREMIUM ONLY]** @@ -145,7 +140,7 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Repository checks](repository_checks.md): Periodic Git repository checks. - [Repository storage paths](repository_storage_paths.md): Manage the paths used to store repositories. - [Repository storage rake tasks](raketasks/storage.md): A collection of rake tasks to list and migrate existing projects and attachments associated with it from Legacy storage to Hashed storage. -- [Limit repository size](https://docs.gitlab.com/ee/user/admin_area/settings/account_and_limit_settings.html): Set a hard limit for your repositories' size. **[STARTER ONLY]** +- [Limit repository size](../user/admin_area/settings/account_and_limit_settings.md): Set a hard limit for your repositories' size. **[STARTER ONLY]** ## Continuous Integration settings @@ -154,7 +149,7 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Job artifacts](job_artifacts.md): Enable, disable, and configure job artifacts (a set of files and directories which are outputted by a job when it completes successfully). - [Job traces](job_traces.md): Information about the job traces (logs). - [Register Shared and specific Runners](../ci/runners/README.md#registering-a-shared-runner): Learn how to register and configure Shared and specific Runners to your own instance. -- [Shared Runners pipelines quota](https://docs.gitlab.com/ee/user/admin_area/settings/continuous_integration.html#shared-runners-pipeline-minutes-quota-starter-only): Limit the usage of pipeline minutes for Shared Runners. **[STARTER ONLY]** +- [Shared Runners pipelines quota](../user/admin_area/settings/continuous_integration.md#shared-runners-pipeline-minutes-quota-starter-only): Limit the usage of pipeline minutes for Shared Runners. **[STARTER ONLY]** - [Enable/disable Auto DevOps](../topics/autodevops/index.md#enablingdisabling-auto-devops): Enable or disable Auto DevOps for your instance. ## Git configuration options diff --git a/doc/administration/integration/plantuml.md b/doc/administration/integration/plantuml.md index b7b820abb40..82e0c14ffc2 100644 --- a/doc/administration/integration/plantuml.md +++ b/doc/administration/integration/plantuml.md @@ -27,7 +27,7 @@ own PlantUML server is easy in Debian/Ubuntu distributions using Tomcat. First you need to create a `plantuml.war` file from the source code: ``` -sudo apt-get install graphviz openjdk-7-jdk git-core maven +sudo apt-get install graphviz openjdk-8-jdk git-core maven git clone https://github.com/plantuml/plantuml-server.git cd plantuml-server mvn package diff --git a/doc/administration/integration/terminal.md b/doc/administration/integration/terminal.md index 2596e3fe68b..c34858cd0db 100644 --- a/doc/administration/integration/terminal.md +++ b/doc/administration/integration/terminal.md @@ -43,6 +43,11 @@ detail below. ## Enabling and disabling terminal support +NOTE: **Note:** AWS Elastic Load Balancers (ELBs) do not support web sockets. +AWS Application Load Balancers (ALBs) must be used if you want web terminals +to work. See [AWS Elastic Load Balancing Product Comparison](https://aws.amazon.com/elasticloadbalancing/features/#compare) +for more information. + As web terminals use WebSockets, every HTTP/HTTPS reverse proxy in front of Workhorse needs to be configured to pass the `Connection` and `Upgrade` headers through to the next one in the chain. If you installed GitLab using Omnibus, or diff --git a/doc/administration/job_artifacts.md b/doc/administration/job_artifacts.md index e7792106f81..05e15fc303b 100644 --- a/doc/administration/job_artifacts.md +++ b/doc/administration/job_artifacts.md @@ -7,9 +7,9 @@ > - Starting with GitLab 8.17, builds are renamed to jobs. > - This is the administration documentation. For the user guide see [pipelines/job_artifacts](../user/project/pipelines/job_artifacts.md). -Artifacts is a list of files and directories which are attached to a job -after it completes successfully. This feature is enabled by default in all -GitLab installations. Keep reading if you want to know how to disable it. +Artifacts is a list of files and directories which are attached to a job after it +finishes. This feature is enabled by default in all GitLab installations. Keep reading +if you want to know how to disable it. ## Disabling job artifacts @@ -42,8 +42,9 @@ To disable artifacts site-wide, follow the steps below. ## Storing job artifacts -After a successful job, GitLab Runner uploads an archive containing the job -artifacts to GitLab. +GitLab Runner can upload an archive containing the job artifacts to GitLab. By default, +this is done when the job succeeds, but can also be done on failure, or always, via the +[`artifacts:when`](../ci/yaml/README.md#artifactswhen) parameter. ### Using local storage @@ -100,6 +101,9 @@ artifacts, you can use an object storage like AWS S3 instead. This configuration relies on valid AWS credentials to be configured already. Use an object storage option like AWS S3 to store job artifacts. +DANGER: **Danger:** +If you're enabling S3 in [GitLab HA](high_availability/README.md), you will need to have an [NFS mount set up for CI traces and artifacts](high_availability/nfs.md#a-single-nfs-mount) or enable [live tracing](job_traces.md#new-live-trace-architecture). If these settings are not set, you will risk job traces disappearing or not being saved. + ### Object Storage Settings For source installations the following settings are nested under `artifacts:` and then `object_store:`. On omnibus installs they are prefixed by `artifacts_object_store_`. diff --git a/doc/administration/logs.md b/doc/administration/logs.md index 3d40cda491a..ac41f9177dd 100644 --- a/doc/administration/logs.md +++ b/doc/administration/logs.md @@ -280,6 +280,28 @@ installations from source. Currently it logs the progress of project imports from the Bitbucket Server importer. Future importers may use this file. +## `auth.log` + +Introduced in GitLab 12.0. This file lives in `/var/log/gitlab/gitlab-rails/auth.log` for +Omnibus GitLab packages or in `/home/git/gitlab/log/auth.log` for +installations from source. + +It logs information whenever [Rack Attack] registers an abusive request. + +## `graphql_json.log` + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/59587) in GitLab 12.0. + +This file lives in `/var/log/gitlab/gitlab-rails/graphql_json.log` for +Omnibus GitLab packages or in `/home/git/gitlab/log/graphql_json.log` for +installations from source. + +GraphQL queries are recorded in that file. For example: + +```json +{"query_string":"query IntrospectionQuery{__schema {queryType { name },mutationType { name }}}...(etc)","variables":{"a":1,"b":2},"complexity":181,"depth":1,"duration":7} +``` + ## Reconfigure Logs Reconfigure log files live in `/var/log/gitlab/reconfigure` for Omnibus GitLab @@ -298,3 +320,4 @@ Omnibus GitLab packages or in `/home/git/gitlab/log/sidekiq_exporter.log` for installations from source. [repocheck]: repository_checks.md +[Rack Attack]: ../security/rack_attack.md diff --git a/doc/administration/monitoring/performance/grafana_configuration.md b/doc/administration/monitoring/performance/grafana_configuration.md index ab43ec2cc4f..187fb2f73a1 100644 --- a/doc/administration/monitoring/performance/grafana_configuration.md +++ b/doc/administration/monitoring/performance/grafana_configuration.md @@ -3,7 +3,7 @@ [Grafana](http://grafana.org/) is a tool that allows you to visualize time series metrics through graphs and dashboards. It supports several backend data stores, including InfluxDB. GitLab writes performance data to InfluxDB -and Grafana will allow you to query InfluxDB to display useful graphs. +and Grafana will allow you to query to display useful graphs. For the easiest installation and configuration, install Grafana on the same server as InfluxDB. For larger installations, you may want to split out these @@ -11,11 +11,13 @@ services. ## Installation -Grafana supplies package repositories (Yum/Apt) for easy installation. +[GitLab Omnibus can help you install Grafana (recommended)](https://docs.gitlab.com/omnibus/settings/grafana.html) +or Grafana supplies package repositories (Yum/Apt) for easy installation. See [Grafana installation documentation](http://docs.grafana.org/installation/) for detailed steps. -> **Note**: Before starting Grafana for the first time, set the admin user +NOTE: **Note:** +Before starting Grafana for the first time, set the admin user and password in `/etc/grafana/grafana.ini`. Otherwise, the default password will be `admin`. diff --git a/doc/administration/monitoring/prometheus/gitlab_metrics.md b/doc/administration/monitoring/prometheus/gitlab_metrics.md index 48bd709a2b7..3dcd1593099 100644 --- a/doc/administration/monitoring/prometheus/gitlab_metrics.md +++ b/doc/administration/monitoring/prometheus/gitlab_metrics.md @@ -43,11 +43,11 @@ The following metrics are available: | redis_ping_latency_seconds | Gauge | 9.4 | Round trip time of the redis ping | | user_session_logins_total | Counter | 9.4 | Counter of how many users have logged in | | upload_file_does_not_exist | Counter | 10.7 in EE, 11.5 in CE | Number of times an upload record could not find its file | -| failed_login_captcha_total | Gauge | 11.0 | Counter of failed CAPTCHA attempts during login | -| successful_login_captcha_total | Gauge | 11.0 | Counter of successful CAPTCHA attempts during login | -| unicorn_active_connections | Gauge | 11.0 | The number of active Unicorn connections (workers) | -| unicorn_queued_connections | Gauge | 11.0 | The number of queued Unicorn connections | -| unicorn_workers | Gauge | 11.11 | The number of Unicorn workers | +| failed_login_captcha_total | Gauge | 11.0 | Counter of failed CAPTCHA attempts during login | +| successful_login_captcha_total | Gauge | 11.0 | Counter of successful CAPTCHA attempts during login | +| unicorn_active_connections | Gauge | 11.0 | The number of active Unicorn connections (workers) | +| unicorn_queued_connections | Gauge | 11.0 | The number of queued Unicorn connections | +| unicorn_workers | Gauge | 12.0 | The number of Unicorn workers | ## Sidekiq Metrics available for Geo **[PREMIUM]** @@ -87,8 +87,8 @@ the `monitoring.sidekiq_exporter` configuration option in `gitlab.yml`. | geo_wikis_checksum_mismatch_count | Gauge | 10.7 | Number of wikis that checksum mismatch on secondary | url | geo_repositories_checked_count | Gauge | 11.1 | Number of repositories that have been checked via `git fsck` | url | geo_repositories_checked_failed_count | Gauge | 11.1 | Number of repositories that have a failure from `git fsck` | url -| geo_repositories_retrying_verification_count | Gauge | 11.2 | Number of repositories verification failures that Geo is actively trying to correct on secondary | url -| geo_wikis_retrying_verification_count | Gauge | 11.2 | Number of wikis verification failures that Geo is actively trying to correct on secondary | url +| geo_repositories_retrying_verification_count | Gauge | 11.2 | Number of repositories verification failures that Geo is actively trying to correct on secondary | url +| geo_wikis_retrying_verification_count | Gauge | 11.2 | Number of wikis verification failures that Geo is actively trying to correct on secondary | url ### Ruby metrics @@ -101,13 +101,31 @@ Some basic Ruby runtime metrics are available: | ruby_file_descriptors | Gauge | 11.1 | File descriptors per process | | ruby_memory_bytes | Gauge | 11.1 | Memory usage by process | | ruby_sampler_duration_seconds_total | Counter | 11.1 | Time spent collecting stats | -| ruby_process_cpu_seconds_total | Gauge | 11.11 | Total amount of CPU time per process | -| ruby_process_max_fds | Gauge | 11.11 | Maximum number of open file descriptors per process | -| ruby_process_resident_memory_bytes | Gauge | 11.11 | Memory usage by process, measured in bytes | -| ruby_process_start_time_seconds | Gauge | 11.11 | The elapsed time between system boot and the process started, measured in seconds | +| ruby_process_cpu_seconds_total | Gauge | 12.0 | Total amount of CPU time per process | +| ruby_process_max_fds | Gauge | 12.0 | Maximum number of open file descriptors per process | +| ruby_process_resident_memory_bytes | Gauge | 12.0 | Memory usage by process, measured in bytes | +| ruby_process_start_time_seconds | Gauge | 12.0 | The elapsed time between system boot and the process started, measured in seconds | [GC.stat]: https://ruby-doc.org/core-2.3.0/GC.html#method-c-stat +## Puma Metrics **[EXPERIMENTAL]** + +When Puma is used instead of Unicorn, following metrics are available: + +| Metric | Type | Since | Description | +|:-------------------------------------------- |:------- |:----- |:----------- | +| puma_workers | Gauge | 12.0 | Total number of workers | +| puma_running_workers | Gauge | 12.0 | Number of booted workers | +| puma_stale_workers | Gauge | 12.0 | Number of old workers | +| puma_phase | Gauge | 12.0 | Phase number (increased during phased restarts) | +| puma_running | Gauge | 12.0 | Number of running threads | +| puma_queued_connections | Gauge | 12.0 | Number of connections in that worker's "todo" set waiting for a worker thread | +| puma_active_connections | Gauge | 12.0 | Number of threads processing a request | +| puma_pool_capacity | Gauge | 12.0 | Number of requests the worker is capable of taking right now | +| puma_max_threads | Gauge | 12.0 | Maximum number of worker threads | +| puma_idle_threads | Gauge | 12.0 | Number of spawned threads which are not processing a request | +| rack_state_total | Gauge | 12.0 | Number of requests in a given rack state | + ## Metrics shared directory GitLab's Prometheus client requires a directory to store metrics data shared between multi-process services. diff --git a/doc/administration/operations/fast_ssh_key_lookup.md b/doc/administration/operations/fast_ssh_key_lookup.md index 69f110805b7..3631ea0822f 100644 --- a/doc/administration/operations/fast_ssh_key_lookup.md +++ b/doc/administration/operations/fast_ssh_key_lookup.md @@ -34,10 +34,10 @@ feature for CentOS 6, follow [the instructions on how to build and install a cus By default, GitLab manages an `authorized_keys` file, which contains all the public SSH keys for users allowed to access GitLab. However, to maintain a -single source of truth, [Geo](https://docs.gitlab.com/ee/administration/geo/replication/index.html) needs to be configured to perform SSH fingerprint +single source of truth, [Geo](../geo/replication/index.md) needs to be configured to perform SSH fingerprint lookups via database lookup. -As part of [setting up Geo](https://docs.gitlab.com/ee/administration/geo/replication/index.html#setup-instructions), +As part of [setting up Geo](../geo/replication/index.md#setup-instructions), you will be required to follow the steps outlined below for both the primary and secondary nodes, but note that the `Write to "authorized keys" file` checkbox only needs to be unchecked on the primary node since it will be reflected diff --git a/doc/administration/packages.md b/doc/administration/packages.md index 5b9a13e3859..0d5f784b71e 100644 --- a/doc/administration/packages.md +++ b/doc/administration/packages.md @@ -8,12 +8,12 @@ The Packages feature allows GitLab to act as a repository for the following: | Software repository | Description | Available in GitLab version | | ------------------- | ----------- | --------------------------- | -| [Maven Repository](https://docs.gitlab.com/ee/user/project/packages/maven_repository.html) | The GitLab Maven Repository enables every project in GitLab to have its own space to store [Maven](https://maven.apache.org/) packages. | 11.3+ | -| [NPM Registry](https://docs.gitlab.com/ee/user/project/packages/npm_registry.html) | The GitLab NPM Registry enables every project in GitLab to have its own space to store [NPM](https://www.npmjs.com/) packages. | 11.7+ | +| [Maven Repository](../user/project/packages/maven_repository.md) | The GitLab Maven Repository enables every project in GitLab to have its own space to store [Maven](https://maven.apache.org/) packages. | 11.3+ | +| [NPM Registry](../user/project/packages/npm_registry.md) | The GitLab NPM Registry enables every project in GitLab to have its own space to store [NPM](https://www.npmjs.com/) packages. | 11.7+ | Don't you see your package management system supported yet? Please consider contributing -to GitLab. This [development documentation](https://docs.gitlab.com/ee/development/packages.html) will guide you through the process. +to GitLab. This [development documentation](../development/packages.md) will guide you through the process. ## Enabling the Packages feature diff --git a/doc/administration/pages/index.md b/doc/administration/pages/index.md index 1373bd56fe3..3a7ca517d56 100644 --- a/doc/administration/pages/index.md +++ b/doc/administration/pages/index.md @@ -291,6 +291,20 @@ Pages access control is disabled by default. To enable it: 1. [Reconfigure GitLab][reconfigure]. 1. Users can now configure it in their [projects' settings](../../user/project/pages/introduction.md#gitlab-pages-access-control-core-only). +### Running behind a proxy + +Like the rest of GitLab, Pages can be used in those environments where external +internet connectivity is gated by a proxy. In order to use a proxy for GitLab +pages: + +1. Configure in `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_pages['http_proxy'] = 'http://example:8080' + ``` + +1. [Reconfigure Gitlab][reconfigure] for the changes to take effect. + ## Activate verbose logging for daemon Verbose logging was [introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/2533) in @@ -376,6 +390,7 @@ Follow the steps below to configure GitLab Pages in a separate server. gitaly['enable'] = false alertmanager['enable'] = false node_exporter['enable'] = false + gitlab_rails['auto_migrate'] = false ``` 1. Run `sudo gitlab-ctl reconfigure`. 1. On `app1` apply the following changes to `/etc/gitlab/gitlab.rb`: diff --git a/doc/administration/raketasks/maintenance.md b/doc/administration/raketasks/maintenance.md index b295b7d5dc4..0b4c1ae15d6 100644 --- a/doc/administration/raketasks/maintenance.md +++ b/doc/administration/raketasks/maintenance.md @@ -205,25 +205,6 @@ cd /home/git/gitlab sudo -u git -H bundle exec rake gitlab:track_deployment RAILS_ENV=production ``` -## Create or repair repository hooks symlink - -If the GitLab shell hooks directory location changes or another circumstance -leads to the hooks symlink becoming missing or invalid, run this Rake task -to create or repair the symlinks. - -**Omnibus Installation** - -``` -sudo gitlab-rake gitlab:shell:create_hooks -``` - -**Source Installation** - -``` -cd /home/git/gitlab -sudo -u git -H bundle exec rake gitlab:shell:create_hooks RAILS_ENV=production -``` - ## Check TCP connectivity to a remote site Sometimes you need to know if your GitLab installation can connect to a TCP diff --git a/doc/administration/raketasks/project_import_export.md b/doc/administration/raketasks/project_import_export.md index f43bba0a7a7..6ca23aabdec 100644 --- a/doc/administration/raketasks/project_import_export.md +++ b/doc/administration/raketasks/project_import_export.md @@ -32,5 +32,4 @@ bundle exec rake gitlab:import_export:data RAILS_ENV=production ``` [ce-3050]: https://gitlab.com/gitlab-org/gitlab-ce/issues/3050 -[feature-flags]: https://docs.gitlab.com/ee/api/features.html [tmp]: ../../development/shared_files.md diff --git a/doc/administration/repository_storage_paths.md b/doc/administration/repository_storage_paths.md index 1689b0a57d6..4aafc06cfdc 100644 --- a/doc/administration/repository_storage_paths.md +++ b/doc/administration/repository_storage_paths.md @@ -62,7 +62,7 @@ files and add the full paths of the alternative repository storage paths. In the example below, we add two more mountpoints that are named `nfs` and `cephfs` respectively. -NOTE: **Note:** This example uses NFS and CephFS. We do not recommend using EFS for storage as it may impact GitLab's performance. See the [relevant documentation](./high_availability/nfs.md#avoid-using-awss-elastic-file-system-efs) for more details. +NOTE: **Note:** This example uses NFS and CephFS. We do not recommend using EFS for storage as it may impact GitLab's performance. See the [relevant documentation](high_availability/nfs.md#avoid-using-awss-elastic-file-system-efs) for more details. **For installations from source** diff --git a/doc/administration/troubleshooting/migration.md b/doc/administration/troubleshooting/migration.md new file mode 100644 index 00000000000..4d2d268b9df --- /dev/null +++ b/doc/administration/troubleshooting/migration.md @@ -0,0 +1,82 @@ +# Migrations problems + +## Legacy upload migration + +> Introduced in GitLab 12.0. + + The migration takes all attachments uploaded by legacy `AttachmentUploader` and + migrate them to the path that current uploaders expect. + +Although it should not usually happen there could possibly be some attachments belonging to +LegacyDiffNotes. These attachments can't be seen before running the migration by users and +they should not be present in your instance. + +However, if you have some of them, you will need to handle them manually. +You can find the ids of failed notes in logs as "MigrateLegacyUploads: LegacyDiffNote" + +1. Run a Rails console: + + ```sh + sudo gitlab-rails console production + ``` + + or for source installs: + + ```sh + bundle exec rails console production + ``` + + 1. Check the failed upload and find the note (you can see their ids in the logs) + + ```ruby + upload = Upload.find(upload_id) + note = Note.find(note_id) + ``` + + + 1. Check the path - it should contain `system/note/attachment` + + ```ruby + upload.absolut_path + ``` + + 1. Check the path in the uploader - it should differ from the upload path and should contain `system/legacy_diff_note` + + ```ruby + uploader = upload.build_uploader + uploader.file + ``` + + 1. First, you need to move the file to the path that is expected from the uploader + + ```ruby + old_path = upload.absolute_path + new_path = upload.absolute_path.sub('-/system/note/attachment', '-/system/legacy_diff_note') + new_dir = File.dirname(new_path) + FileUtils.mkdir_p(new_dir) + + FileUtils.mv(old_path, new_path) + ``` + + 1. You then need to move the file to the `FileUploader` and create a new `Upload` object + + ```ruby + file_uploader = UploadService.new(note.project, File.read(new_path)).execute + ``` + + 1. And update the legacy note to contain the file. + + ```ruby + new_text = "#{note.note} \n #{file_uploader.markdown_link}" + note.update!( + note: new_text + ) + ``` + + 1. And finally, you can remove the old upload + + ```ruby + upload.destroy + ``` + +If you have any problems feel free to contact [GitLab Support](https://about.gitlab.com/support/). diff --git a/doc/analytics/README.md b/doc/analytics/README.md index 6b63edb5174..bfb15f6c4f3 100644 --- a/doc/analytics/README.md +++ b/doc/analytics/README.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/group/index.html#user-contribution-analysis-starter' +redirect_to: '../user/group/index.md#user-contribution-analysis-starter' --- -This document was moved to [another location](https://docs.gitlab.com/ee/user/group/index.html#user-contribution-analysis-starter) +This document was moved to [another location](../user/group/index.md#user-contribution-analysis-starter) diff --git a/doc/analytics/contribution_analytics.md b/doc/analytics/contribution_analytics.md index 38d71263bc1..e36f55071a4 100644 --- a/doc/analytics/contribution_analytics.md +++ b/doc/analytics/contribution_analytics.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/group/contribution_analytics/index.html' +redirect_to: '../user/group/contribution_analytics/index.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/user/group/contribution_analytics/index.html). +This document was moved to [another location](../user/group/contribution_analytics/index.md). diff --git a/doc/api/README.md b/doc/api/README.md index 7ec7955c596..3a1064b787e 100644 --- a/doc/api/README.md +++ b/doc/api/README.md @@ -21,72 +21,83 @@ See also: The following API resources are available in the project context: -| Resource | Available endpoints | -|:------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| [Access requests](access_requests.md) | `/projects/:id/access_requests` (also available for groups) | -| [Award emoji](award_emoji.md) | `/projects/:id/issues/.../award_emoji`, `/projects/:id/merge_requests/.../award_emoji`, `/projects/:id/snippets/.../award_emoji` | -| [Branches](branches.md) | `/projects/:id/repository/branches/`, `/projects/:id/repository/merged_branches` | -| [Commits](commits.md) | `/projects/:id/repository/commits`, `/projects/:id/statuses` | -| [Container Registry](container_registry.md) | `/projects/:id/registry/repositories` | -| [Custom attributes](custom_attributes.md) | `/projects/:id/custom_attributes` (also available for groups and users) | -| [Deploy keys](deploy_keys.md) | `/projects/:id/deploy_keys` (also available standalone) | -| [Deployments](deployments.md) | `/projects/:id/deployments` | -| [Discussions](discussions.md) (threaded comments) | `/projects/:id/issues/.../discussions`, `/projects/:id/snippets/.../discussions`, `/projects/:id/merge_requests/.../discussions`, `/projects/:id/commits/.../discussions` | -| [Environments](environments.md) | `/projects/:id/environments` | -| [Events](events.md) | `/projects/:id/events` (also available for users and standalone) | -| [Issues](issues.md) | `/projects/:id/issues` (also available for groups and standalone) | -| [Issue boards](boards.md) | `/projects/:id/boards` | -| [Jobs](jobs.md) | `/projects/:id/jobs`, `/projects/:id/pipelines/.../jobs` | -| [Labels](labels.md) | `/projects/:id/labels` | -| [Members](members.md) | `/projects/:id/members` (also available for groups) | -| [Merge requests](merge_requests.md) | `/projects/:id/merge_requests` (also available for groups and standalone) | -| [Notes](notes.md) (comments) | `/projects/:id/issues/.../notes`, `/projects/:id/snippets/.../notes`, `/projects/:id/merge_requests/.../notes` | -| [Notification settings](notification_settings.md) | `/projects/:id/notification_settings` (also available for groups and standalone) | -| [Pages domains](pages_domains.md) | `/projects/:id/pages` (also available standalone) | -| [Pipelines](pipelines.md) | `/projects/:id/pipelines` | -| [Pipeline schedules](pipeline_schedules.md) | `/projects/:id/pipeline_schedules` | -| [Pipeline triggers](pipeline_triggers.md) | `/projects/:id/triggers` | -| [Projects](projects.md) including setting Webhooks | `/projects`, `/projects/:id/hooks` (also available for users) | -| [Project badges](project_badges.md) | `/projects/:id/badges` | -| [Project clusters](project_clusters.md) | `/projects/:id/clusters` | -| [Project-level variables](project_level_variables.md) | `/projects/:id/variables` | -| [Project import/export](project_import_export.md) | `/projects/:id/export`, `/projects/import`, `/projects/:id/import` | -| [Project milestones](milestones.md) | `/projects/:id/milestones` | -| [Project snippets](project_snippets.md) | `/projects/:id/snippets` | -| [Project templates](project_templates.md) | `/projects/:id/templates` | -| [Protected branches](protected_branches.md) | `/projects/:id/protected_branches` | -| [Protected tags](protected_tags.md) | `/projects/:id/protected_tags` | -| [Releases](releases/index.md) | `/projects/:id/releases` | -| [Release links](releases/links.md) | `/projects/:id/releases/.../assets/links` | -| [Repositories](repositories.md) | `/projects/:id/repository` | -| [Repository files](repository_files.md) | `/projects/:id/repository/files` | -| [Repository submodules](repository_submodules.md) | `/projects/:id/repository/submodules` | -| [Resource label events](resource_label_events.md) | `/projects/:id/issues/.../resource_label_events`, `/projects/:id/merge_requests/.../resource_label_events` | -| [Runners](runners.md) | `/projects/:id/runners` (also available standalone) | -| [Search](search.md) | `/projects/:id/search` (also available for groups and standalone) | -| [Services](services.md) | `/projects/:id/services` | -| [Tags](tags.md) | `/projects/:id/repository/tags` | -| [Wikis](wikis.md) | `/projects/:id/wikis` | +| Resource | Available endpoints | +|:--------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| [Access requests](access_requests.md) | `/projects/:id/access_requests` (also available for groups) | +| [Award emoji](award_emoji.md) | `/projects/:id/issues/.../award_emoji`, `/projects/:id/merge_requests/.../award_emoji`, `/projects/:id/snippets/.../award_emoji` | +| [Branches](branches.md) | `/projects/:id/repository/branches/`, `/projects/:id/repository/merged_branches` | +| [Commits](commits.md) | `/projects/:id/repository/commits`, `/projects/:id/statuses` | +| [Container Registry](container_registry.md) | `/projects/:id/registry/repositories` | +| [Custom attributes](custom_attributes.md) | `/projects/:id/custom_attributes` (also available for groups and users) | +| [Deploy keys](deploy_keys.md) | `/projects/:id/deploy_keys` (also available standalone) | +| [Deployments](deployments.md) | `/projects/:id/deployments` | +| [Discussions](discussions.md) (threaded comments) | `/projects/:id/issues/.../discussions`, `/projects/:id/snippets/.../discussions`, `/projects/:id/merge_requests/.../discussions`, `/projects/:id/commits/.../discussions` (also available for groups) | +| [Environments](environments.md) | `/projects/:id/environments` | +| [Events](events.md) | `/projects/:id/events` (also available for users and standalone) | +| [Issues](issues.md) | `/projects/:id/issues` (also available for groups and standalone) | +| [Issue boards](boards.md) | `/projects/:id/boards` | +| [Issue links](issue_links.md) **[STARTER]** | `/projects/:id/issues/.../links` | +| [Jobs](jobs.md) | `/projects/:id/jobs`, `/projects/:id/pipelines/.../jobs` | +| [Labels](labels.md) | `/projects/:id/labels` | +| [Managed licenses](managed_licenses.md) **[ULTIMATE]** | `/projects/:id/managed_licenses` | +| [Members](members.md) | `/projects/:id/members` (also available for groups) | +| [Merge request approvals](merge_request_approvals.md) **[STARTER]** | `/projects/:id/approvals`, `/projects/:id/merge_requests/.../approvals` | +| [Merge requests](merge_requests.md) | `/projects/:id/merge_requests` (also available for groups and standalone) | +| [Notes](notes.md) (comments) | `/projects/:id/issues/.../notes`, `/projects/:id/snippets/.../notes`, `/projects/:id/merge_requests/.../notes` (also available for groups) | +| [Notification settings](notification_settings.md) | `/projects/:id/notification_settings` (also available for groups and standalone) | +| [Packages](packages.md) **[PREMIUM]** | `/projects/:id/packages` | +| [Pages domains](pages_domains.md) | `/projects/:id/pages` (also available standalone) | +| [Pipelines](pipelines.md) | `/projects/:id/pipelines` | +| [Pipeline schedules](pipeline_schedules.md) | `/projects/:id/pipeline_schedules` | +| [Pipeline triggers](pipeline_triggers.md) | `/projects/:id/triggers` | +| [Projects](projects.md) including setting Webhooks | `/projects`, `/projects/:id/hooks` (also available for users) | +| [Project badges](project_badges.md) | `/projects/:id/badges` | +| [Project clusters](project_clusters.md) | `/projects/:id/clusters` | +| [Project-level variables](project_level_variables.md) | `/projects/:id/variables` | +| [Project import/export](project_import_export.md) | `/projects/:id/export`, `/projects/import`, `/projects/:id/import` | +| [Project milestones](milestones.md) | `/projects/:id/milestones` | +| [Project snippets](project_snippets.md) | `/projects/:id/snippets` | +| [Project templates](project_templates.md) | `/projects/:id/templates` | +| [Protected branches](protected_branches.md) | `/projects/:id/protected_branches` | +| [Protected tags](protected_tags.md) | `/projects/:id/protected_tags` | +| [Releases](releases/index.md) | `/projects/:id/releases` | +| [Release links](releases/links.md) | `/projects/:id/releases/.../assets/links` | +| [Repositories](repositories.md) | `/projects/:id/repository` | +| [Repository files](repository_files.md) | `/projects/:id/repository/files` | +| [Repository submodules](repository_submodules.md) | `/projects/:id/repository/submodules` | +| [Resource label events](resource_label_events.md) | `/projects/:id/issues/.../resource_label_events`, `/projects/:id/merge_requests/.../resource_label_events` (also available for groups) | +| [Runners](runners.md) | `/projects/:id/runners` (also available standalone) | +| [Search](search.md) | `/projects/:id/search` (also available for groups and standalone) | +| [Services](services.md) | `/projects/:id/services` | +| [Tags](tags.md) | `/projects/:id/repository/tags` | +| [Vulnerabilities](vulnerabilities.md) **[ULTIMATE]** | `/projects/:id/vulnerabilities` (also available for groups) | +| [Wikis](wikis.md) | `/projects/:id/wikis` | ### Group resources The following API resources are available in the group context: -| Resource | Available endpoints | -|:--------------------------------------------------|:---------------------------------------------------------------------------------| -| [Access requests](access_requests.md) | `/groups/:id/access_requests/` (also available for projects) | -| [Custom attributes](custom_attributes.md) | `/groups/:id/custom_attributes` (also available for projects and users) | -| [Groups](groups.md) | `/groups`, `/groups/.../subgroups` | -| [Group badges](group_badges.md) | `/groups/:id/badges` | -| [Group issue boards](group_boards.md) | `/groups/:id/boards` | -| [Group labels](group_labels.md) | `/groups/:id/labels` | -| [Group-level variables](group_level_variables.md) | `/groups/:id/variables` | -| [Group milestones](group_milestones.md) | `/groups/:id/milestones` | -| [Issues](issues.md) | `/groups/:id/issues` (also available for projects and standalone) | -| [Members](members.md) | `/groups/:id/members` (also available for projects) | -| [Merge requests](merge_requests.md) | `/groups/:id/merge_requests` (also available for projects and standalone) | -| [Notification settings](notification_settings.md) | `/groups/:id/notification_settings` (also available for projects and standalone) | -| [Search](search.md) | `/groups/:id/search` (also available for projects and standalone) | +| Resource | Available endpoints | +|:-----------------------------------------------------------------|:---------------------------------------------------------------------------------| +| [Access requests](access_requests.md) | `/groups/:id/access_requests/` (also available for projects) | +| [Custom attributes](custom_attributes.md) | `/groups/:id/custom_attributes` (also available for projects and users) | +| [Discussions](discussions.md) (threaded comments) **[ULTIMATE]** | `/groups/:id/epics/.../discussions` (also available for projects) | +| [Epic issues](epic_issues.md) **[ULTIMATE]** | `/groups/:id/epics/.../issues` | +| [Epic links](epic_links.md) **[ULTIMATE]** | `/groups/:id/epics/.../epics` | +| [Epics](epics.md) **[ULTIMATE]** | `/groups/:id/epics` | +| [Groups](groups.md) | `/groups`, `/groups/.../subgroups` | +| [Group badges](group_badges.md) | `/groups/:id/badges` | +| [Group issue boards](group_boards.md) | `/groups/:id/boards` | +| [Group labels](group_labels.md) | `/groups/:id/labels` | +| [Group-level variables](group_level_variables.md) | `/groups/:id/variables` | +| [Group milestones](group_milestones.md) | `/groups/:id/milestones` | +| [Issues](issues.md) | `/groups/:id/issues` (also available for projects and standalone) | +| [Members](members.md) | `/groups/:id/members` (also available for projects) | +| [Merge requests](merge_requests.md) | `/groups/:id/merge_requests` (also available for projects and standalone) | +| [Notes](notes.md) (comments) | `/groups/:id/epics/.../notes` (also available for projects) | +| [Notification settings](notification_settings.md) | `/groups/:id/notification_settings` (also available for projects and standalone) | +| [Resource label events](resource_label_events.md) | `/groups/:id/epics/.../resource_label_events` (also available for projects) | +| [Search](search.md) | `/groups/:id/search` (also available for projects and standalone) | ### Standalone resources @@ -102,9 +113,11 @@ The following API resources are available outside of project and group contexts | [Deploy keys](deploy_keys.md) | `/deploy_keys` (also available for projects) | | [Events](events.md) | `/events`, `/users/:id/events` (also available for projects) | | [Feature flags](features.md) | `/features` | +| [Geo Nodes](geo_nodes.md) **[PREMIUM ONLY]** | `/geo_nodes` | | [Import repository from GitHub](import.md) | `/import/github` | | [Issues](issues.md) | `/issues` (also available for groups and projects) | | [Keys](keys.md) | `/keys` | +| [License](license.md) **[CORE ONLY]** | `/license` | | [Markdown](markdown.md) | `/markdown` | | [Merge requests](merge_requests.md) | `/merge_requests` (also available for groups and projects) | | [Namespaces](namespaces.md) | `/namespaces` | @@ -131,6 +144,11 @@ Endpoints are available for: - [GitLab CI YAML templates](templates/gitlab_ci_ymls.md). - [Open source license templates](templates/licenses.md). +## SCIM **[SILVER ONLY]** + +[GitLab.com Silver and above](https://about.gitlab.com/pricing/) provides an [SCIM API](scim.md) that implements [the RFC7644 protocol](https://tools.ietf.org/html/rfc7644) and provides +the `/Users` endpoint. The base URL is: `/api/scim/v2/groups/:group_path/Users/`. + ## Road to GraphQL Going forward, we will start on moving to diff --git a/doc/api/boards.md b/doc/api/boards.md index 28c73db6b98..a96206f5df3 100644 --- a/doc/api/boards.md +++ b/doc/api/boards.md @@ -141,6 +141,173 @@ Example response: } ``` +## Create a board **[STARTER]** + +Creates a board. + +``` +POST /projects/:id/boards +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `name` | string | yes | The name of the new board | + +```bash +curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/5/boards?name=newboard +``` + +Example response: + +```json + { + "id": 1, + "project": { + "id": 5, + "name": "Diaspora Project Site", + "name_with_namespace": "Diaspora / Diaspora Project Site", + "path": "diaspora-project-site", + "path_with_namespace": "diaspora/diaspora-project-site", + "http_url_to_repo": "http://example.com/diaspora/diaspora-project-site.git", + "web_url": "http://example.com/diaspora/diaspora-project-site" + }, + "name": "newboard", + "milestone": { + "id": 12 + "title": "10.0" + }, + "lists" : [ + { + "id" : 1, + "label" : { + "name" : "Testing", + "color" : "#F0AD4E", + "description" : null + }, + "position" : 1 + }, + { + "id" : 2, + "label" : { + "name" : "Ready", + "color" : "#FF0000", + "description" : null + }, + "position" : 2 + }, + { + "id" : 3, + "label" : { + "name" : "Production", + "color" : "#FF5F00", + "description" : null + }, + "position" : 3 + } + ] + } +``` + +## Update a board **[STARTER]** + +> [Introduced][ee-5954] in [GitLab Starter](https://about.gitlab.com/pricing/) 11.1. + +Updates a board. + +``` +PUT /projects/:id/boards/:board_id +``` + +| Attribute | Type | Required | Description | +| ------------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `board_id` | integer | yes | The ID of a board | +| `name` | string | no | The new name of the board | +| `assignee_id` | integer | no | The assignee the board should be scoped to | +| `milestone_id` | integer | no | The milestone the board should be scoped to | +| `labels` | string | no | Comma-separated list of label names which the board should be scoped to | +| `weight` | integer | no | The weight range from 0 to 9, to which the board should be scoped to | + + +```bash +curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/5/boards/1?name=new_name&milestone_id=43&assignee_id=1&labels=Doing&weight=4 +``` + +Example response: + +```json + { + "id": 1, + "project": { + "id": 5, + "name": "Diaspora Project Site", + "name_with_namespace": "Diaspora / Diaspora Project Site", + "path": "diaspora-project-site", + "path_with_namespace": "diaspora/diaspora-project-site", + "created_at": "2018-07-03T05:48:49.982Z", + "default_branch": null, + "tag_list": [], + "ssh_url_to_repo": "ssh://user@example.com/diaspora/diaspora-project-site.git", + "http_url_to_repo": "http://example.com/diaspora/diaspora-project-site.git", + "web_url": "http://example.com/diaspora/diaspora-project-site", + "readme_url": null, + "avatar_url": null, + "star_count": 0, + "forks_count": 0, + "last_activity_at": "2018-07-03T05:48:49.982Z" + }, + "lists": [], + "name": "new_name", + "group": null, + "milestone": { + "id": 43, + "iid": 1, + "project_id": 15, + "title": "Milestone 1", + "description": "Milestone 1 desc", + "state": "active", + "created_at": "2018-07-03T06:36:42.618Z", + "updated_at": "2018-07-03T06:36:42.618Z", + "due_date": null, + "start_date": null, + "web_url": "http://example.com/root/board1/milestones/1" + }, + "assignee": { + "id": 1, + "name": "Administrator", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", + "web_url": "http://example.com/root" + }, + "labels": [{ + "id": 10, + "name": "Doing", + "color": "#5CB85C", + "description": null + }], + "weight": 4 + } +``` + +## Delete a board **[STARTER]** + +Deletes a board. + +``` +DELETE /projects/:id/boards/:board_id +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `board_id` | integer | yes | The ID of a board | + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/5/boards/1 +``` + ## List board lists Get a list of the board's lists. @@ -237,7 +404,15 @@ POST /projects/:id/boards/:board_id/lists | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | | `board_id` | integer | yes | The ID of a board | -| `label_id` | integer | yes | The ID of a label | +| `label_id` | integer | no | The ID of a label | +| `assignee_id` **[PREMIUM]** | integer | no | The ID of a user | +| `milestone_id` **[PREMIUM]** | integer | no | The ID of a milestone | + +NOTE: **Note**: +Label, assignee and milestone arguments are mutually exclusive, +that is, only one of them are accepted in a request. +Check the [Issue Board docs](../user/project/issue_board.md#summary-of-features-per-tier) +for more information regarding the required license for each list type. ```bash curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/5/boards/1/lists?label_id=5 @@ -307,3 +482,5 @@ DELETE /projects/:id/boards/:board_id/lists/:list_id ```bash curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/5/boards/1/lists/1 ``` + +[ee-5954]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/5954 diff --git a/doc/api/commits.md b/doc/api/commits.md index 92f53c7b5e6..25015fad9e3 100644 --- a/doc/api/commits.md +++ b/doc/api/commits.md @@ -75,6 +75,7 @@ POST /projects/:id/repository/commits | `branch` | string | yes | Name of the branch to commit into. To create a new branch, also provide `start_branch`. | | `commit_message` | string | yes | Commit message | | `start_branch` | string | no | Name of the branch to start the new commit from | +| `start_project` | integer/string | no | The project ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) to start the commit from. Defaults to the value of `id`. | | `actions[]` | array | yes | An array of action hashes to commit as a batch. See the next table for what attributes it can take. | | `author_email` | string | no | Specify the commit author's email address | | `author_name` | string | no | Specify the commit author's name | diff --git a/doc/api/discussions.md b/doc/api/discussions.md index 07a6201b10b..9defef4fd53 100644 --- a/doc/api/discussions.md +++ b/doc/api/discussions.md @@ -1,6 +1,12 @@ # Discussions API -Discussions are set of related notes on snippets, issues, merge requests or commits. +Discussions are a set of related notes on: + +- Snippets +- Issues +- Epics **[ULTIMATE]** +- Merge requests +- Commits This includes system notes, which are notes about changes to the object (for example, when a milestone changes, there will be a corresponding system note). Label notes are not part of this API, but recorded as separate events in [resource label events](resource_label_events.md). @@ -424,6 +430,214 @@ Parameters: curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/5/snippets/11/discussions/636 ``` +## Epics **[ULTIMATE]** + +### List group epic discussions + +Gets a list of all discussions for a single epic. + +``` +GET /groups/:id/epics/:epic_id/discussions +``` + +| Attribute | Type | Required | Description | +| ------------------- | ---------------- | ---------- | ------------ | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `epic_id` | integer | yes | The ID of an epic | + +```json +[ + { + "id": "6a9c1750b37d513a43987b574953fceb50b03ce7", + "individual_note": false, + "notes": [ + { + "id": 1126, + "type": "DiscussionNote", + "body": "discussion text", + "attachment": null, + "author": { + "id": 1, + "name": "root", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/00afb8fb6ab07c3ee3e9c1f38777e2f4?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "created_at": "2018-03-03T21:54:39.668Z", + "updated_at": "2018-03-03T21:54:39.668Z", + "system": false, + "noteable_id": 3, + "noteable_type": "Epic", + "noteable_id": null, + "resolvable": false + }, + { + "id": 1129, + "type": "DiscussionNote", + "body": "reply to the discussion", + "attachment": null, + "author": { + "id": 1, + "name": "root", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/00afb8fb6ab07c3ee3e9c1f38777e2f4?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "created_at": "2018-03-04T13:38:02.127Z", + "updated_at": "2018-03-04T13:38:02.127Z", + "system": false, + "noteable_id": 3, + "noteable_type": "Epic", + "noteable_id": null, + "resolvable": false + } + ] + }, + { + "id": "87805b7c09016a7058e91bdbe7b29d1f284a39e6", + "individual_note": true, + "notes": [ + { + "id": 1128, + "type": null, + "body": "a single comment", + "attachment": null, + "author": { + "id": 1, + "name": "root", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/00afb8fb6ab07c3ee3e9c1f38777e2f4?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "created_at": "2018-03-04T09:17:22.520Z", + "updated_at": "2018-03-04T09:17:22.520Z", + "system": false, + "noteable_id": 3, + "noteable_type": "Epic", + "noteable_id": null, + "resolvable": false + } + ] + } +] +``` + +```bash +curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/5/epics/11/discussions +``` + +### Get single epic discussion + +Returns a single discussion for a specific group epic + +``` +GET /groups/:id/epics/:epic_id/discussions/:discussion_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `epic_id` | integer | yes | The ID of an epic | +| `discussion_id` | integer | yes | The ID of a discussion | + +```bash +curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/5/epics/11/discussions/6a9c1750b37d513a43987b574953fceb50b03ce7 +``` + +### Create new epic discussion + +Creates a new discussion to a single group epic. This is similar to creating +a note but but another comments (replies) can be added to it later. + +``` +POST /groups/:id/epics/:epic_id/discussions +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `epic_id` | integer | yes | The ID of an epic | +| `body` | string | yes | The content of a discussion | +| `created_at` | string | no | Date time string, ISO 8601 formatted, e.g. 2016-03-11T03:45:40Z (requires admin or project/group owner rights) | + +```bash +curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/5/epics/11/discussions?body=comment +``` + +### Add note to existing epic discussion + +Adds a new note to the discussion. This can also +[create a discussion from a single comment](../user/discussions/#start-a-discussion-by-replying-to-a-standard-comment). + +``` +POST /groups/:id/epics/:epic_id/discussions/:discussion_id/notes +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `epic_id` | integer | yes | The ID of an epic | +| `discussion_id` | integer | yes | The ID of a discussion | +| `note_id` | integer | yes | The ID of a discussion note | +| `body` | string | yes | The content of a discussion | +| `created_at` | string | no | Date time string, ISO 8601 formatted, e.g. 2016-03-11T03:45:40Z (requires admin or project/group owner rights) | + +```bash +curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/5/epics/11/discussions/6a9c1750b37d513a43987b574953fceb50b03ce7/notes?body=comment +``` + +### Modify existing epic discussion note + +Modify existing discussion note of an epic. + +``` +PUT /groups/:id/epics/:epic_id/discussions/:discussion_id/notes/:note_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `epic_id` | integer | yes | The ID of an epic | +| `discussion_id` | integer | yes | The ID of a discussion | +| `note_id` | integer | yes | The ID of a discussion note | +| `body` | string | yes | The content of a discussion | + +```bash +curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/5/epics/11/discussions/6a9c1750b37d513a43987b574953fceb50b03ce7/notes/1108?body=comment +``` + +### Delete an epic discussion note + +Deletes an existing discussion note of an epic. + +``` +DELETE /groups/:id/epics/:epic_id/discussions/:discussion_id/notes/:note_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `epic_id` | integer | yes | The ID of an epic | +| `discussion_id` | integer | yes | The ID of a discussion | +| `note_id` | integer | yes | The ID of a discussion note | + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/5/epics/11/discussions/636 +``` + ## Merge requests ### List project merge request discussions diff --git a/doc/api/epic_issues.md b/doc/api/epic_issues.md index 438a3361dcc..ec59ea7068e 100644 --- a/doc/api/epic_issues.md +++ b/doc/api/epic_issues.md @@ -7,6 +7,7 @@ If a user is not a member of a group and the group is private, a `GET` request o Epics are available only in Ultimate. If epics feature is not available a `403` status code will be returned. ## List issues for an epic + Gets all issues that are assigned to an epic and the authenticated user has access to. ``` diff --git a/doc/api/epic_links.md b/doc/api/epic_links.md index 619ae6ea2dc..9ad90a6d0f1 100644 --- a/doc/api/epic_links.md +++ b/doc/api/epic_links.md @@ -12,6 +12,7 @@ If a user is not a member of a group and the group is private, a `GET` request o Epics are available only in the [Ultimate/Gold tier](https://about.gitlab.com/pricing/). If the epics feature is not available, a `403` status code will be returned. ## List epics related to a given epic + Gets all child epics of an epic. ``` @@ -125,7 +126,7 @@ POST /groups/:id/epics/:epic_iid/epics | --------------- | -------------- | -------- | ------------------------------------------------------------------------------------------------------------------ | | `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | | `epic_iid` | integer | yes | The internal ID of the (future parent) epic. | -| `title` | integer | yes | The global ID of the child epic. Internal ID can't be used because they can conflict with epics from other groups. | +| `title` | string | yes | The title of a newly created epic. | ```bash curl --header POST "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/1/epics/5/epics?title=Newpic @@ -141,6 +142,7 @@ Example response: "group_id": 49, "parent_id": 23, "has_children": false, + "has_issues": false, "reference": "&2", "url": "http://localhost/groups/group16/-/epics/2", "relation_url": "http://localhost/groups/group16/-/epics/1/links/24" diff --git a/doc/api/geo_nodes.md b/doc/api/geo_nodes.md index a1cb524499f..ea31abdd87e 100644 --- a/doc/api/geo_nodes.md +++ b/doc/api/geo_nodes.md @@ -192,12 +192,10 @@ Example response: "job_artifacts_failed_count": nil, "job_artifacts_synced_missing_on_primary_count": 0, "job_artifacts_synced_in_percentage": "0.00%", - "repositories_count": 41, "projects_count": 41, "repositories_failed_count": nil, "repositories_synced_count": nil, "repositories_synced_in_percentage": "0.00%", - "wikis_count": 41, "wikis_failed_count": nil, "wikis_synced_count": nil, "wikis_synced_in_percentage": "0.00%", @@ -257,12 +255,10 @@ Example response: "job_artifacts_failed_count": 1, "job_artifacts_synced_missing_on_primary_count": 0, "job_artifacts_synced_in_percentage": "50.00%", - "repositories_count": 41, "projects_count": 41, "repositories_failed_count": 1, "repositories_synced_count": 40, "repositories_synced_in_percentage": "97.56%", - "wikis_count": 41, "wikis_failed_count": 0, "wikis_synced_count": 41, "wikis_synced_in_percentage": "100.00%", @@ -300,7 +296,8 @@ Example response: ] ``` -Note: fields `wikis_count` and `repositories_count` are deprecated and will be deleted soon. Please use `projects_count` instead. +NOTE: **Note:** +In GitLab 12.0, deprecated fields `wikis_count` and `repositories_count` were removed. Use `projects_count` instead. ## Retrieve status about a specific Geo node @@ -337,12 +334,10 @@ Example response: "job_artifacts_failed_count": 1, "job_artifacts_synced_missing_on_primary_count": 0, "job_artifacts_synced_in_percentage": "50.00%", - "repositories_count": 41, "projects_count": 41, "repositories_failed_count": 1, "repositories_synced_count": 40, "repositories_synced_in_percentage": "97.56%", - "wikis_count": 41, "wikis_failed_count": 0, "wikis_synced_count": 41, "wikis_synced_in_percentage": "100.00%", @@ -362,7 +357,8 @@ Example response: Note: The `health_status` parameter can only be in an "Healthy" or "Unhealthy" state, while the `health` parameter can be empty, "Healthy", or contain the actual error message. -Note: Fields `wikis_count` and `repositories_count` are deprecated and will be deleted soon. Please use `projects_count` instead. +NOTE: **Note:** +In GitLab 12.0, deprecated fields `wikis_count` and `repositories_count` were removed. Use `projects_count` instead. ## Retrieve project sync or verification failures that occurred on the current node diff --git a/doc/api/graphql/index.md b/doc/api/graphql/index.md index cf02bbd9c92..88e657a5d2f 100644 --- a/doc/api/graphql/index.md +++ b/doc/api/graphql/index.md @@ -16,6 +16,20 @@ added to the API without creating breaking changes. This allows us to have a versionless API as described in [the GraphQL documentation](https://graphql.org/learn/best-practices/#versioning). +## Vision + +We want the GraphQL API to be the **primary** means of interacting +programmatically with GitLab. To achieve this, it needs full coverage - anything +possible in the REST API should also be possible in the GraphQL API. + +To help us meet this vision, the frontend should use GraphQL in preference to +the REST API for new features, although the alpha status of GraphQL may prevent +this from being a possibility at times. + +There are no plans to deprecate the REST API. To reduce the technical burden of +supporting two APIs in parallel, they should share implementations as much as +possible. + ## Enabling the GraphQL feature The GraphQL API itself is currently in Alpha, and therefore hidden behind a @@ -32,8 +46,16 @@ curl --data "value=100" --header "PRIVATE-TOKEN: <your_access_token>" https://gi A first iteration of a GraphQL API includes the following queries 1. `project` : Within a project it is also possible to fetch a `mergeRequest` by IID. - 1. `group` : Only basic group information is currently supported. +1. `namespace` : Within a namespace it is also possible to fetch `projects`. + +### Multiplex queries + +GitLab supports batching queries into a single request using +[apollo-link-batch-http](https://www.apollographql.com/docs/link/links/batch-http). More +info about multiplexed queries is also available for +[graphql-ruby](https://graphql-ruby.org/queries/multiplex.html) the +library GitLab uses on the backend. ## GraphiQL diff --git a/doc/api/groups.md b/doc/api/groups.md index 907b443d355..20789a1d4a4 100644 --- a/doc/api/groups.md +++ b/doc/api/groups.md @@ -68,6 +68,7 @@ GET /groups?statistics=true "statistics": { "storage_size" : 212, "repository_size" : 33, + "wiki_size" : 100, "lfs_objects_size" : 123, "job_artifacts_size" : 57 diff --git a/doc/api/issues.md b/doc/api/issues.md index cb5789e76b7..0d96cfa1b21 100644 --- a/doc/api/issues.md +++ b/doc/api/issues.md @@ -37,23 +37,26 @@ GET /issues?confidential=true | Attribute | Type | Required | Description | | ------------------- | ---------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | -| `state` | string | no | Return all issues or just those that are `opened` or `closed` | +| `state` | string | no | Return `all` issues or just those that are `opened` or `closed` | | `labels` | string | no | Comma-separated list of label names, issues must have all labels to be returned. `None` lists all issues with no labels. `Any` lists all issues with at least one label. `No+Label` (Deprecated) lists all issues with no labels. Predefined names are case-insensitive. | +| `with_labels_details`| Boolean | no | If `true`, response will return more details for each label in labels field: `:name`, `:color`, `:description`, `:text_color`. Default is `false`. | | `milestone` | string | no | The milestone title. `None` lists all issues with no milestone. `Any` lists all issues that have an assigned milestone. | | `scope` | string | no | Return issues for the given scope: `created_by_me`, `assigned_to_me` or `all`. Defaults to `created_by_me`<br> For versions before 11.0, use the now deprecated `created-by-me` or `assigned-to-me` scopes instead.<br> _([Introduced][ce-13004] in GitLab 9.5. [Changed to snake_case][ce-18935] in GitLab 11.0)_ | -| `author_id` | integer | no | Return issues created by the given user `id`. Combine with `scope=all` or `scope=assigned_to_me`. _([Introduced][ce-13004] in GitLab 9.5)_ | -| `assignee_id` | integer | no | Return issues assigned to the given user `id`. `None` returns unassigned issues. `Any` returns issues with an assignee. _([Introduced][ce-13004] in GitLab 9.5)_ | +| `author_id` | integer | no | Return issues created by the given user `id`. Mutually exclusive with `author_username`. Combine with `scope=all` or `scope=assigned_to_me`. _([Introduced][ce-13004] in GitLab 9.5)_ | +| `author_username` | string | no | Return issues created by the given `username`. Simillar to `author_id` and mutually exclusive with `author_id`. | +| `assignee_id` | integer | no | Return issues assigned to the given user `id`. Mutually exclusive with `assignee_username`. `None` returns unassigned issues. `Any` returns issues with an assignee. _([Introduced][ce-13004] in GitLab 9.5)_ | +| `assignee_username` | Array[String] | no | Return issues assigned to the given `username`. Simillar to `assignee_id` and mutually exclusive with `assignee_id`. In CE version `assignee_username` array should only contain a single value or an invalid param error will be returned otherwise. | | `my_reaction_emoji` | string | no | Return issues reacted by the authenticated user by the given `emoji`. `None` returns issues not given a reaction. `Any` returns issues given at least one reaction. _([Introduced][ce-14016] in GitLab 10.0)_ | | `iids[]` | Array[integer] | no | Return only the issues having the given `iid` | | `order_by` | string | no | Return issues ordered by `created_at` or `updated_at` fields. Default is `created_at` | | `sort` | string | no | Return issues sorted in `asc` or `desc` order. Default is `desc` | | `search` | string | no | Search issues against their `title` and `description` | -| `in` | string | no | Modify the scope of the `search` attribute. `title`, `description`, or a string joining them with comma. Default is `title,description` | +| `in` | string | no | Modify the scope of the `search` attribute. `title`, `description`, or a string joining them with comma. Default is `title,description` | | `created_after` | datetime | no | Return issues created on or after the given time | | `created_before` | datetime | no | Return issues created on or before the given time | | `updated_after` | datetime | no | Return issues updated on or after the given time | | `updated_before` | datetime | no | Return issues updated on or before the given time | -| `confidential ` | Boolean | no | Filter confidential or public issues. | +| `confidential ` | Boolean | no | Filter confidential or public issues. | ```bash curl --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/issues @@ -109,7 +112,7 @@ Example response: "title" : "Consequatur vero maxime deserunt laboriosam est voluptas dolorem.", "created_at" : "2016-01-04T15:31:51.081Z", "iid" : 6, - "labels" : [], + "labels" : ["foo", "bar"], "upvotes": 4, "downvotes": 0, "merge_requests_count": 0, @@ -122,8 +125,21 @@ Example response: "human_time_estimate": null, "human_total_time_spent": null }, + "has_tasks": true, + "task_status": "10 of 15 tasks completed", "confidential": false, - "discussion_locked": false + "discussion_locked": false, + "_links":{ + "self":"http://example.com/api/v4/projects/1/issues/76", + "notes":"`http://example.com/`api/v4/projects/1/issues/76/notes", + "award_emoji":"http://example.com/api/v4/projects/1/issues/76/award_emoji", + "project":"http://example.com/api/v4/projects/1" + }, + "subscribed": false, + "task_completion_status":{ + "count":0, + "completed_count":0 + } } ] ``` @@ -158,11 +174,14 @@ GET /groups/:id/issues?confidential=true | `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | | `state` | string | no | Return all issues or just those that are `opened` or `closed` | | `labels` | string | no | Comma-separated list of label names, issues must have all labels to be returned. `None` lists all issues with no labels. `Any` lists all issues with at least one label. `No+Label` (Deprecated) lists all issues with no labels. Predefined names are case-insensitive. | +| `with_labels_details`| Boolean | no | If `true`, response will return more details for each label in labels field: `:name`, `:color`, `:description`, `:text_color`. Default is `false`. | | `iids[]` | Array[integer] | no | Return only the issues having the given `iid` | | `milestone` | string | no | The milestone title. `None` lists all issues with no milestone. `Any` lists all issues that have an assigned milestone. | | `scope` | string | no | Return issues for the given scope: `created_by_me`, `assigned_to_me` or `all`.<br> For versions before 11.0, use the now deprecated `created-by-me` or `assigned-to-me` scopes instead.<br> _([Introduced][ce-13004] in GitLab 9.5. [Changed to snake_case][ce-18935] in GitLab 11.0)_ | -| `author_id` | integer | no | Return issues created by the given user `id` _([Introduced][ce-13004] in GitLab 9.5)_ | -| `assignee_id` | integer | no | Return issues assigned to the given user `id`. `None` returns unassigned issues. `Any` returns issues with an assignee. _([Introduced][ce-13004] in GitLab 9.5)_ | +| `author_id` | integer | no | Return issues created by the given user `id`. Mutually exclusive with `author_username`. Combine with `scope=all` or `scope=assigned_to_me`. _([Introduced][ce-13004] in GitLab 9.5)_ | +| `author_username` | string | no | Return issues created by the given `username`. Simillar to `author_id` and mutually exclusive with `author_id`. | +| `assignee_id` | integer | no | Return issues assigned to the given user `id`. Mutually exclusive with `assignee_username`. `None` returns unassigned issues. `Any` returns issues with an assignee. _([Introduced][ce-13004] in GitLab 9.5)_ | +| `assignee_username` | Array[String] | no | Return issues assigned to the given `username`. Simillar to `assignee_id` and mutually exclusive with `assignee_id`. In CE version `assignee_username` array should only contain a single value or an invalid param error will be returned otherwise. | | `my_reaction_emoji` | string | no | Return issues reacted by the authenticated user by the given `emoji`. `None` returns issues not given a reaction. `Any` returns issues given at least one reaction. _([Introduced][ce-14016] in GitLab 10.0)_ | | `order_by` | string | no | Return issues ordered by `created_at` or `updated_at` fields. Default is `created_at` | | `sort` | string | no | Return issues sorted in `asc` or `desc` order. Default is `desc` | @@ -221,7 +240,7 @@ Example response: "id" : 9, "name" : "Dr. Luella Kovacek" }, - "labels" : [], + "labels" : ["foo", "bar"], "upvotes": 4, "downvotes": 0, "merge_requests_count": 0, @@ -240,8 +259,21 @@ Example response: "human_time_estimate": null, "human_total_time_spent": null }, + "has_tasks": true, + "task_status": "10 of 15 tasks completed", "confidential": false, - "discussion_locked": false + "discussion_locked": false, + "_links":{ + "self":"http://example.com/api/v4/projects/4/issues/41", + "notes":"`http://example.com/`api/v4/projects/4/issues/41/notes", + "award_emoji":"http://example.com/api/v4/projects/4/issues/41/award_emoji", + "project":"http://example.com/api/v4/projects/4" + }, + "subscribed": false, + "task_completion_status":{ + "count":0, + "completed_count":0 + } } ] ``` @@ -277,10 +309,13 @@ GET /projects/:id/issues?confidential=true | `iids[]` | Array[integer] | no | Return only the milestone having the given `iid` | | `state` | string | no | Return all issues or just those that are `opened` or `closed` | | `labels` | string | no | Comma-separated list of label names, issues must have all labels to be returned. `None` lists all issues with no labels. `Any` lists all issues with at least one label. `No+Label` (Deprecated) lists all issues with no labels. Predefined names are case-insensitive. | +| `with_labels_details`| Boolean | no | If `true`, response will return more details for each label in labels field: `:name`, `:color`, `:description`, `:text_color`. Default is `false`. | | `milestone` | string | no | The milestone title. `None` lists all issues with no milestone. `Any` lists all issues that have an assigned milestone. | | `scope` | string | no | Return issues for the given scope: `created_by_me`, `assigned_to_me` or `all`.<br> For versions before 11.0, use the now deprecated `created-by-me` or `assigned-to-me` scopes instead.<br> _([Introduced][ce-13004] in GitLab 9.5. [Changed to snake_case][ce-18935] in GitLab 11.0)_ | -| `author_id` | integer | no | Return issues created by the given user `id` _([Introduced][ce-13004] in GitLab 9.5)_ | -| `assignee_id` | integer | no | Return issues assigned to the given user `id`. `None` returns unassigned issues. `Any` returns issues with an assignee. _([Introduced][ce-13004] in GitLab 9.5)_ | +| `author_id` | integer | no | Return issues created by the given user `id`. Mutually exclusive with `author_username`. Combine with `scope=all` or `scope=assigned_to_me`. _([Introduced][ce-13004] in GitLab 9.5)_ | +| `author_username` | string | no | Return issues created by the given `username`. Simillar to `author_id` and mutually exclusive with `author_id`. | +| `assignee_id` | integer | no | Return issues assigned to the given user `id`. Mutually exclusive with `assignee_username`. `None` returns unassigned issues. `Any` returns issues with an assignee. _([Introduced][ce-13004] in GitLab 9.5)_ | +| `assignee_username` | Array[String] | no | Return issues assigned to the given `username`. Simillar to `assignee_id` and mutually exclusive with `assignee_id`. In CE version `assignee_username` array should only contain a single value or an invalid param error will be returned otherwise. | | `my_reaction_emoji` | string | no | Return issues reacted by the authenticated user by the given `emoji`. `None` returns issues not given a reaction. `Any` returns issues given at least one reaction. _([Introduced][ce-14016] in GitLab 10.0)_ | | `order_by` | string | no | Return issues ordered by `created_at` or `updated_at` fields. Default is `created_at` | | `sort` | string | no | Return issues sorted in `asc` or `desc` order. Default is `desc` | @@ -340,7 +375,7 @@ Example response: "id" : 9, "name" : "Dr. Luella Kovacek" }, - "labels" : [], + "labels" : ["foo", "bar"], "upvotes": 4, "downvotes": 0, "merge_requests_count": 0, @@ -366,8 +401,21 @@ Example response: "human_time_estimate": null, "human_total_time_spent": null }, + "has_tasks": true, + "task_status": "10 of 15 tasks completed", "confidential": false, - "discussion_locked": false + "discussion_locked": false, + "_links":{ + "self":"http://example.com/api/v4/projects/4/issues/41", + "notes":"`http://example.com/`api/v4/projects/4/issues/41/notes", + "award_emoji":"http://example.com/api/v4/projects/4/issues/41/award_emoji", + "project":"http://example.com/api/v4/projects/4" + }, + "subscribed": false, + "task_completion_status":{ + "count":0, + "completed_count":0 + } } ] ``` @@ -464,6 +512,10 @@ Example response: "notes": "http://example.com/api/v4/projects/1/issues/2/notes", "award_emoji": "http://example.com/api/v4/projects/1/issues/2/award_emoji", "project": "http://example.com/api/v4/projects/1" + }, + "task_completion_status":{ + "count":0, + "completed_count":0 } } ``` @@ -547,6 +599,10 @@ Example response: "notes": "http://example.com/api/v4/projects/1/issues/2/notes", "award_emoji": "http://example.com/api/v4/projects/1/issues/2/award_emoji", "project": "http://example.com/api/v4/projects/1" + }, + "task_completion_status":{ + "count":0, + "completed_count":0 } } ``` @@ -638,6 +694,10 @@ Example response: "notes": "http://example.com/api/v4/projects/1/issues/2/notes", "award_emoji": "http://example.com/api/v4/projects/1/issues/2/award_emoji", "project": "http://example.com/api/v4/projects/1" + }, + "task_completion_status":{ + "count":0, + "completed_count":0 } } ``` @@ -744,6 +804,10 @@ Example response: "notes": "http://example.com/api/v4/projects/1/issues/2/notes", "award_emoji": "http://example.com/api/v4/projects/1/issues/2/award_emoji", "project": "http://example.com/api/v4/projects/1" + }, + "task_completion_status":{ + "count":0, + "completed_count":0 } } ``` @@ -829,6 +893,10 @@ Example response: "notes": "http://example.com/api/v4/projects/1/issues/2/notes", "award_emoji": "http://example.com/api/v4/projects/1/issues/2/award_emoji", "project": "http://example.com/api/v4/projects/1" + }, + "task_completion_status":{ + "count":0, + "completed_count":0 } } ``` @@ -895,7 +963,11 @@ Example response: "due_date": null, "web_url": "http://example.com/example/example/issues/12", "confidential": false, - "discussion_locked": false + "discussion_locked": false, + "task_completion_status":{ + "count":0, + "completed_count":0 + } } ``` @@ -993,7 +1065,11 @@ Example response: "due_date": null, "web_url": "http://example.com/example/example/issues/110", "confidential": false, - "discussion_locked": false + "discussion_locked": false, + "task_completion_status":{ + "count":0, + "completed_count":0 + } }, "target_url": "https://gitlab.example.com/gitlab-org/gitlab-ci/issues/10", "body": "Vel voluptas atque dicta mollitia adipisci qui at.", diff --git a/doc/api/issues_statistics.md b/doc/api/issues_statistics.md new file mode 100644 index 00000000000..82bc9c142cc --- /dev/null +++ b/doc/api/issues_statistics.md @@ -0,0 +1,177 @@ +# Issues Statistics API + +Every API call to issues_statistics must be authenticated. + +If a user is not a member of a project and the project is private, a `GET` +request on that project will result to a `404` status code. + +## Get issues statistics + +Gets issues count statistics on all issues the authenticated user has access to. By default it +returns only issues created by the current user. To get all issues, +use parameter `scope=all`. + +``` +GET /issues_statistics +GET /issues_statistics?labels=foo +GET /issues_statistics?labels=foo,bar +GET /issues_statistics?labels=foo,bar&state=opened +GET /issues_statistics?milestone=1.0.0 +GET /issues_statistics?milestone=1.0.0&state=opened +GET /issues_statistics?iids[]=42&iids[]=43 +GET /issues_statistics?author_id=5 +GET /issues_statistics?assignee_id=5 +GET /issues_statistics?my_reaction_emoji=star +GET /issues_statistics?search=foo&in=title +GET /issues_statistics?confidential=true +``` + +| Attribute | Type | Required | Description | +| ------------------- | ---------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | +| `labels` | string | no | Comma-separated list of label names, issues must have all labels to be returned. `None` lists all issues with no labels. `Any` lists all issues with at least one label. | +| `milestone` | string | no | The milestone title. `None` lists all issues with no milestone. `Any` lists all issues that have an assigned milestone. | +| `scope` | string | no | Return issues for the given scope: `created_by_me`, `assigned_to_me` or `all`. Defaults to `created_by_me` | +| `author_id` | integer | no | Return issues created by the given user `id`. Mutually exclusive with `author_username`. Combine with `scope=all` or `scope=assigned_to_me`. | +| `author_username` | string | no | Return issues created by the given `username`. Similar to `author_id` and mutually exclusive with `author_id`. | +| `assignee_id` | integer | no | Return issues assigned to the given user `id`. Mutually exclusive with `assignee_username`. `None` returns unassigned issues. `Any` returns issues with an assignee. | +| `assignee_username` | Array[String] | no | Return issues assigned to the given `username`. Similar to `assignee_id` and mutually exclusive with `assignee_id`. In CE version `assignee_username` array should only contain a single value or an invalid param error will be returned otherwise. | +| `my_reaction_emoji` | string | no | Return issues reacted by the authenticated user by the given `emoji`. `None` returns issues not given a reaction. `Any` returns issues given at least one reaction. | +| `iids[]` | Array[integer] | no | Return only the issues having the given `iid` | +| `search` | string | no | Search issues against their `title` and `description` | +| `in` | string | no | Modify the scope of the `search` attribute. `title`, `description`, or a string joining them with comma. Default is `title,description` | +| `created_after` | datetime | no | Return issues created on or after the given time | +| `created_before` | datetime | no | Return issues created on or before the given time | +| `updated_after` | datetime | no | Return issues updated on or after the given time | +| `updated_before` | datetime | no | Return issues updated on or before the given time | +| `confidential ` | Boolean | no | Filter confidential or public issues. | + +```bash +curl --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/issues_statistics +``` + +Example response: + +```json +{ + "statistics": { + "counts": { + "all": 20, + "closed": 5, + "opened": 15 + } + } +} +``` + +## Get group issues statistics + +Gets issues count statistics for given group. + +``` +GET /groups/:id/issues_statistics +GET /groups/:id/issues_statistics?labels=foo +GET /groups/:id/issues_statistics?labels=foo,bar +GET /groups/:id/issues_statistics?labels=foo,bar&state=opened +GET /groups/:id/issues_statistics?milestone=1.0.0 +GET /groups/:id/issues_statistics?milestone=1.0.0&state=opened +GET /groups/:id/issues_statistics?iids[]=42&iids[]=43 +GET /groups/:id/issues_statistics?search=issue+title+or+description +GET /groups/:id/issues_statistics?author_id=5 +GET /groups/:id/issues_statistics?assignee_id=5 +GET /groups/:id/issues_statistics?my_reaction_emoji=star +GET /groups/:id/issues_statistics?confidential=true +``` + +| Attribute | Type | Required | Description | +| ------------------- | ---------------- | ---------- | ----------------------------------------------------------------------------------------------------------------------------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | +| `labels` | string | no | Comma-separated list of label names, issues must have all labels to be returned. `None` lists all issues with no labels. `Any` lists all issues with at least one label. | +| `iids[]` | Array[integer] | no | Return only the issues having the given `iid` | +| `milestone` | string | no | The milestone title. `None` lists all issues with no milestone. `Any` lists all issues that have an assigned milestone. | +| `scope` | string | no | Return issues for the given scope: `created_by_me`, `assigned_to_me` or `all`. | +| `author_id` | integer | no | Return issues created by the given user `id`. Mutually exclusive with `author_username`. Combine with `scope=all` or `scope=assigned_to_me`. | +| `author_username` | string | no | Return issues created by the given `username`. Similar to `author_id` and mutually exclusive with `author_id`. | +| `assignee_id` | integer | no | Return issues assigned to the given user `id`. Mutually exclusive with `assignee_username`. `None` returns unassigned issues. `Any` returns issues with an assignee. | +| `assignee_username` | Array[String] | no | Return issues assigned to the given `username`. Similar to `assignee_id` and mutually exclusive with `assignee_id`. In CE version `assignee_username` array should only contain a single value or an invalid param error will be returned otherwise. | +| `my_reaction_emoji` | string | no | Return issues reacted by the authenticated user by the given `emoji`. `None` returns issues not given a reaction. `Any` returns issues given at least one reaction. | +| `search` | string | no | Search group issues against their `title` and `description` | +| `created_after` | datetime | no | Return issues created on or after the given time | +| `created_before` | datetime | no | Return issues created on or before the given time | +| `updated_after` | datetime | no | Return issues updated on or after the given time | +| `updated_before` | datetime | no | Return issues updated on or before the given time | +| `confidential ` | Boolean | no | Filter confidential or public issues. | + +```bash +curl --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/4/issues_statistics +``` + +Example response: + +```json +{ + "statistics": { + "counts": { + "all": 20, + "closed": 5, + "opened": 15 + } + } +} +``` + +## Get project issues statistics + +Gets issues count statistics for given project. + +``` +GET /projects/:id/issues_statistics +GET /projects/:id/issues_statistics?labels=foo +GET /projects/:id/issues_statistics?labels=foo,bar +GET /projects/:id/issues_statistics?labels=foo,bar&state=opened +GET /projects/:id/issues_statistics?milestone=1.0.0 +GET /projects/:id/issues_statistics?milestone=1.0.0&state=opened +GET /projects/:id/issues_statistics?iids[]=42&iids[]=43 +GET /projects/:id/issues_statistics?search=issue+title+or+description +GET /projects/:id/issues_statistics?author_id=5 +GET /projects/:id/issues_statistics?assignee_id=5 +GET /projects/:id/issues_statistics?my_reaction_emoji=star +GET /projects/:id/issues_statistics?confidential=true +``` + +| Attribute | Type | Required | Description | +| ------------------- | ---------------- | ---------- | ----------------------------------------------------------------------------------------------------------------------------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `iids[]` | Array[integer] | no | Return only the milestone having the given `iid` | +| `labels` | string | no | Comma-separated list of label names, issues must have all labels to be returned. `None` lists all issues with no labels. `Any` lists all issues with at least one label. | +| `milestone` | string | no | The milestone title. `None` lists all issues with no milestone. `Any` lists all issues that have an assigned milestone. | +| `scope` | string | no | Return issues for the given scope: `created_by_me`, `assigned_to_me` or `all`. | +| `author_id` | integer | no | Return issues created by the given user `id`. Mutually exclusive with `author_username`. Combine with `scope=all` or `scope=assigned_to_me`. | +| `author_username` | string | no | Return issues created by the given `username`. Similar to `author_id` and mutually exclusive with `author_id`. | +| `assignee_id` | integer | no | Return issues assigned to the given user `id`. Mutually exclusive with `assignee_username`. `None` returns unassigned issues. `Any` returns issues with an assignee. | +| `assignee_username` | Array[String] | no | Return issues assigned to the given `username`. Similar to `assignee_id` and mutually exclusive with `assignee_id`. In CE version `assignee_username` array should only contain a single value or an invalid param error will be returned otherwise. | +| `my_reaction_emoji` | string | no | Return issues reacted by the authenticated user by the given `emoji`. `None` returns issues not given a reaction. `Any` returns issues given at least one reaction. | +| `search` | string | no | Search project issues against their `title` and `description` | +| `created_after` | datetime | no | Return issues created on or after the given time | +| `created_before` | datetime | no | Return issues created on or before the given time | +| `updated_after` | datetime | no | Return issues updated on or after the given time | +| `updated_before` | datetime | no | Return issues updated on or before the given time | +| `confidential ` | Boolean | no | Filter confidential or public issues. | + + +```bash +curl --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/4/issues_statistics +``` + +Example response: + +```json +{ + "statistics": { + "counts": { + "all": 20, + "closed": 5, + "opened": 15 + } + } +} +``` diff --git a/doc/api/jobs.md b/doc/api/jobs.md index 877cd99723a..72973b69117 100644 --- a/doc/api/jobs.md +++ b/doc/api/jobs.md @@ -32,6 +32,7 @@ Example of response "title": "Test the CI integration." }, "coverage": null, + "allow_failure": false, "created_at": "2015-12-24T15:51:21.727Z", "started_at": "2015-12-24T17:54:24.729Z", "finished_at": "2015-12-24T17:54:24.921Z", @@ -81,6 +82,7 @@ Example of response "title": "Test the CI integration." }, "coverage": null, + "allow_failure": false, "created_at": "2015-12-24T15:51:21.802Z", "started_at": "2015-12-24T17:54:27.722Z", "finished_at": "2015-12-24T17:54:27.895Z", @@ -165,6 +167,7 @@ Example of response "title": "Test the CI integration." }, "coverage": null, + "allow_failure": false, "created_at": "2015-12-24T15:51:21.727Z", "started_at": "2015-12-24T17:54:24.729Z", "finished_at": "2015-12-24T17:54:24.921Z", @@ -214,6 +217,7 @@ Example of response "title": "Test the CI integration." }, "coverage": null, + "allow_failure": false, "created_at": "2015-12-24T15:51:21.802Z", "started_at": "2015-12-24T17:54:27.722Z", "finished_at": "2015-12-24T17:54:27.895Z", @@ -296,6 +300,7 @@ Example of response "title": "Test the CI integration." }, "coverage": null, + "allow_failure": false, "created_at": "2015-12-24T15:51:21.880Z", "started_at": "2015-12-24T17:54:30.733Z", "finished_at": "2015-12-24T17:54:31.198Z", @@ -341,30 +346,58 @@ Example of response > **Notes**: > > - [Introduced][ce-2893] in GitLab 8.5. +> - The use of `CI_JOB_TOKEN` in the artifacts download API was [introduced][ee-2346] +> in [GitLab Premium][ee] 9.5. -Get job artifacts of a project. +Get the job's artifacts zipped archive of a project. ``` GET /projects/:id/jobs/:job_id/artifacts ``` -| Attribute | Type | Required | Description | -|-----------|----------------|----------|------------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user. | -| `job_id` | integer | yes | ID of a job. | +| Attribute | Type | Required | Description | +|-------------|----------------|----------|-------------------------------------------------------------------------------------------------------------------------------------------------| +| `id` | integer/string | yes | ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user. | +| `job_id` | integer | yes | ID of a job. | +| `job_token` **[PREMIUM]** | string | no | To be used with [triggers] for multi-project pipelines. It should be invoked only inside `.gitlab-ci.yml`. Its value is always `$CI_JOB_TOKEN`. | -Example requests: +Example request using the `PRIVATE-TOKEN` header: ```sh -curl --location --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/jobs/8/artifacts" +curl --output artifacts.zip --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/jobs/42/artifacts" ``` +To use this in a [`script` definition](../ci/yaml/README.md#script) inside +`.gitlab-ci.yml` **[PREMIUM]**, you can use either: + +- The `JOB-TOKEN` header with the GitLab-provided `CI_JOB_TOKEN` variable. + For example, the following job will download the artifacts of the job with ID + `42`. Note that the command is wrapped into single quotes since it contains a + colon (`:`): + + ```yaml + artifact_download: + stage: test + script: + - 'curl --location --output artifacts.zip --header "JOB-TOKEN: $CI_JOB_TOKEN" "https://gitlab.example.com/api/v4/projects/1/jobs/42/artifacts"' + ``` + +- Or the `job_token` attribute with the GitLab-provided `CI_JOB_TOKEN` variable. + For example, the following job will download the artifacts of the job with ID `42`: + + ```yaml + artifact_download: + stage: test + script: + - 'curl --location --output artifacts.zip "https://gitlab.example.com/api/v4/projects/1/jobs/42/artifacts?job_token=$CI_JOB_TOKEN"' + ``` + Possible response status codes: | Status | Description | |-----------|---------------------------------| -| 200 | Serves the artifacts file | -| 404 | Build not found or no artifacts | +| 200 | Serves the artifacts file. | +| 404 | Build not found or no artifacts.| [ce-2893]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/2893 @@ -373,9 +406,13 @@ Possible response status codes: > **Notes**: > > - [Introduced][ce-5347] in GitLab 8.10. +> - The use of `CI_JOB_TOKEN` in the artifacts download API was [introduced][ee-2346] +> in [GitLab Premium][ee] 9.5. -Download the artifacts archive from the given reference name and job provided the -job finished successfully. +Download the artifacts zipped archive from the given reference name and job, +provided the job finished successfully. This is the same as +[getting the job's artifacts](#get-job-artifacts), but by defining the job's +name instead of its ID. ``` GET /projects/:id/jobs/artifacts/:ref_name/download?job=name @@ -383,24 +420,51 @@ GET /projects/:id/jobs/artifacts/:ref_name/download?job=name Parameters -| Attribute | Type | Required | Description | -|------------|----------------|----------|------------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user. | -| `ref_name` | string | yes | Branch or tag name in repository. HEAD or SHA references are not supported. | -| `job` | string | yes | The name of the job. | +| Attribute | Type | Required | Description | +|-------------|----------------|----------|-------------------------------------------------------------------------------------------------------------------------------------------------| +| `id` | integer/string | yes | ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user. | +| `ref_name` | string | yes | Branch or tag name in repository. HEAD or SHA references are not supported. | +| `job` | string | yes | The name of the job. | +| `job_token` **[PREMIUM]** | string | no | To be used with [triggers] for multi-project pipelines. It should be invoked only inside `.gitlab-ci.yml`. Its value is always `$CI_JOB_TOKEN`. | -Example requests: +Example request using the `PRIVATE-TOKEN` header: ```sh curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/jobs/artifacts/master/download?job=test" ``` +To use this in a [`script` definition](../ci/yaml/README.md#script) inside +`.gitlab-ci.yml` **[PREMIUM]**, you can use either: + +- The `JOB-TOKEN` header with the GitLab-provided `CI_JOB_TOKEN` variable. + For example, the following job will download the artifacts of the `test` job + of the `master` branch. Note that the command is wrapped into single quotes + since it contains a colon (`:`): + + ```yaml + artifact_download: + stage: test + script: + - 'curl --location --output artifacts.zip --header "JOB-TOKEN: $CI_JOB_TOKEN" "https://gitlab.example.com/api/v4/projects/$CI_PROJECT_ID/jobs/artifacts/master/download?job=test"' + ``` + +- Or the `job_token` attribute with the GitLab-provided `CI_JOB_TOKEN` variable. + For example, the following job will download the artifacts of the `test` job + of the `master` branch: + + ```yaml + artifact_download: + stage: test + script: + - 'curl --location --output artifacts.zip "https://gitlab.example.com/api/v4/projects/$CI_PROJECT_ID/jobs/artifacts/master/download?job=test&job_token=$CI_JOB_TOKEN"' + ``` + Possible response status codes: | Status | Description | |-----------|---------------------------------| -| 200 | Serves the artifacts file | -| 404 | Build not found or no artifacts | +| 200 | Serves the artifacts file. | +| 404 | Build not found or no artifacts.| [ce-5347]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5347 @@ -409,7 +473,7 @@ Possible response status codes: > Introduced in GitLab 10.0 Download a single artifact file from a job with a specified ID from within -the job's artifacts archive. The file is extracted from the archive and +the job's artifacts zipped archive. The file is extracted from the archive and streamed to the client. ``` @@ -528,6 +592,7 @@ Example of response "title": "Test the CI integration." }, "coverage": null, + "allow_failure": false, "created_at": "2016-01-11T10:13:33.506Z", "started_at": "2016-01-11T10:14:09.526Z", "finished_at": null, @@ -576,6 +641,7 @@ Example of response "title": "Test the CI integration." }, "coverage": null, + "allow_failure": false, "created_at": "2016-01-11T10:13:33.506Z", "started_at": null, "finished_at": null, @@ -628,6 +694,7 @@ Example of response "title": "Test the CI integration." }, "coverage": null, + "allow_failure": false, "download_url": null, "id": 42, "name": "rubocop", @@ -681,6 +748,7 @@ Example response: "title": "Test the CI integration." }, "coverage": null, + "allow_failure": false, "download_url": null, "id": 42, "name": "rubocop", @@ -757,6 +825,7 @@ Example of response "title": "Test the CI integration." }, "coverage": null, + "allow_failure": false, "created_at": "2016-01-11T10:13:33.506Z", "started_at": null, "finished_at": null, @@ -773,3 +842,7 @@ Example of response "user": null } ``` + +[ee]: https://about.gitlab.com/pricing/ +[ee-2346]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/2346 +[triggers]: ../ci/triggers/README.md#when-a-pipeline-depends-on-the-artifacts-of-another-pipeline-premium diff --git a/doc/api/members.md b/doc/api/members.md index 0593d2c20ea..8784d577f99 100644 --- a/doc/api/members.md +++ b/doc/api/members.md @@ -62,7 +62,9 @@ Example response: ## List all members of a group or project including inherited members Gets a list of group or project members viewable by the authenticated user, including inherited members through ancestor groups. -Returns multiple times the same user (with different member attributes) when the user is a member of the project/group and of one or more ancestor group. +When a user is a member of the project/group and of one or more ancestor groups the user is returned only once with the project access_level (if exists) +or the access_level for the user in the first group which he belongs to in the project groups ancestors chain. +**Note:** We plan to [change](https://gitlab.com/gitlab-org/gitlab-ce/issues/62284) this behavior to return highest access_level instead. ``` GET /groups/:id/members/all diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index 7992af15448..96a956ad03a 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -138,7 +138,11 @@ Parameters: "human_time_estimate": null, "human_total_time_spent": null }, - "squash": false + "squash": false, + "task_completion_status":{ + "count":0, + "completed_count":0 + } } ] ``` @@ -280,7 +284,11 @@ Parameters: "human_time_estimate": null, "human_total_time_spent": null }, - "squash": false + "squash": false, + "task_completion_status":{ + "count":0, + "completed_count":0 + } } ] ``` @@ -410,7 +418,11 @@ Parameters: "human_time_estimate": null, "human_total_time_spent": null }, - "squash": false + "squash": false, + "task_completion_status":{ + "count":0, + "completed_count":0 + } } ] ``` @@ -545,7 +557,11 @@ Parameters: "start_sha": "c380d3acebd181f13629a25d2e2acca46ffe1e00" }, "diverged_commits_count": 2, - "rebase_in_progress": false + "rebase_in_progress": false, + "task_completion_status":{ + "count":0, + "completed_count":0 + } } ``` @@ -579,7 +595,7 @@ Parameters: "state": "active", "avatar_url": "http://www.gravatar.com/avatar/10fc7f102be8de7657fb4d80898bbfe3?s=80&d=identicon", "web_url": "http://localhost/user2" - }, + } ] ``` @@ -702,7 +718,11 @@ Parameters: "total_time_spent": 0, "human_time_estimate": null, "human_total_time_spent": null - } + }, + "task_completion_status":{ + "count":0, + "completed_count":0 + }, "changes": [ { "old_path": "VERSION", @@ -865,7 +885,11 @@ POST /projects/:id/merge_requests "head_sha": "2be7ddb704c7b6b83732fdd5b9f09d5a397b5f8f", "start_sha": "c380d3acebd181f13629a25d2e2acca46ffe1e00" }, - "diverged_commits_count": 2 + "diverged_commits_count": 2, + "task_completion_status":{ + "count":0, + "completed_count":0 + } } ``` @@ -1002,7 +1026,11 @@ Must include at least one non-required attribute from above. "head_sha": "2be7ddb704c7b6b83732fdd5b9f09d5a397b5f8f", "start_sha": "c380d3acebd181f13629a25d2e2acca46ffe1e00" }, - "diverged_commits_count": 2 + "diverged_commits_count": 2, + "task_completion_status":{ + "count":0, + "completed_count":0 + } } ``` @@ -1155,37 +1183,37 @@ Parameters: "head_sha": "2be7ddb704c7b6b83732fdd5b9f09d5a397b5f8f", "start_sha": "c380d3acebd181f13629a25d2e2acca46ffe1e00" }, - "diverged_commits_count": 2 + "diverged_commits_count": 2, + "task_completion_status":{ + "count":0, + "completed_count":0 + } } ``` -## Merge to default merge ref path +## Returns the up to date merge-ref HEAD commit -Merge the changes between the merge request source and target branches into `refs/merge-requests/:iid/merge` -ref, of the target project repository. This ref will have the state the target branch would have if +Merge the changes between the merge request source and target branches into `refs/merge-requests/:iid/merge` +ref, of the target project repository, if possible. This ref will have the state the target branch would have if a regular merge action was taken. -This is not a regular merge action given it doesn't change the merge request state in any manner. +This is not a regular merge action given it doesn't change the merge request target branch state in any manner. -This ref (`refs/merge-requests/:iid/merge`) is **always** overwritten when submitting -requests to this API, so none of its state is kept or used in the process. +This ref (`refs/merge-requests/:iid/merge`) isn't necessarily overwritten when submitting +requests to this API, though it'll make sure the ref has the latest possible state. -If the merge request has conflicts, is empty or already merged, -you'll get a `400` and a descriptive error message. If you don't have permissions to do so, -you'll get a `403`. +If the merge request has conflicts, is empty or already merged, you'll get a `400` and a descriptive error message. -It returns the HEAD commit of `refs/merge-requests/:iid/merge` in the response body in -case of `200`. +It returns the HEAD commit of `refs/merge-requests/:iid/merge` in the response body in case of `200`. ``` -PUT /projects/:id/merge_requests/:merge_request_iid/merge_to_ref +GET /projects/:id/merge_requests/:merge_request_iid/merge_ref ``` Parameters: - `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user - `merge_request_iid` (required) - Internal ID of MR -- `merge_commit_message` (optional) - Custom merge commit message ```json { @@ -1313,7 +1341,11 @@ Parameters: "head_sha": "2be7ddb704c7b6b83732fdd5b9f09d5a397b5f8f", "start_sha": "c380d3acebd181f13629a25d2e2acca46ffe1e00" }, - "diverged_commits_count": 2 + "diverged_commits_count": 2, + "task_completion_status":{ + "count":0, + "completed_count":0 + } } ``` @@ -1349,7 +1381,7 @@ If the rebase operation is ongoing, the response will include the following: ```json { - "rebase_in_progress": true + "rebase_in_progress": true, "merge_error": null } ``` @@ -1360,7 +1392,7 @@ the following: ```json { "rebase_in_progress": false, - "merge_error": null, + "merge_error": null } ``` @@ -1369,7 +1401,7 @@ If the rebase operation fails, the response will include the following: ```json { "rebase_in_progress": false, - "merge_error": "Rebase failed. Please rebase locally", + "merge_error": "Rebase failed. Please rebase locally" } ``` @@ -1576,7 +1608,11 @@ Example response: "head_sha": "2be7ddb704c7b6b83732fdd5b9f09d5a397b5f8f", "start_sha": "c380d3acebd181f13629a25d2e2acca46ffe1e00" }, - "diverged_commits_count": 2 + "diverged_commits_count": 2, + "task_completion_status":{ + "count":0, + "completed_count":0 + } } ``` @@ -1705,7 +1741,11 @@ Example response: "head_sha": "2be7ddb704c7b6b83732fdd5b9f09d5a397b5f8f", "start_sha": "c380d3acebd181f13629a25d2e2acca46ffe1e00" }, - "diverged_commits_count": 2 + "diverged_commits_count": 2, + "task_completion_status":{ + "count":0, + "completed_count":0 + } } ``` diff --git a/doc/api/notes.md b/doc/api/notes.md index dfde80c6441..c09129c22d4 100644 --- a/doc/api/notes.md +++ b/doc/api/notes.md @@ -1,6 +1,11 @@ # Notes API -Notes are comments on snippets, issues or merge requests. +Notes are comments on: + +- Snippets +- Issues +- Merge requests +- Epics **[ULTIMATE]** This includes system notes, which are notes about changes to the object (for example, when a milestone changes, there will be a corresponding system note). Label notes are not part of this API, but recorded as separate events in [resource label events](resource_label_events.md). @@ -390,3 +395,126 @@ Parameters: ```bash curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/5/merge_requests/7/notes/1602 ``` + +## Epics **[ULTIMATE]** + +### List all epic notes + +Gets a list of all notes for a single epic. Epic notes are comments users can post to an epic. + +``` +GET /groups/:id/epics/:epic_id/notes +GET /groups/:id/epics/:epic_id/notes?sort=asc&order_by=updated_at +``` + +| Attribute | Type | Required | Description | +| ------------------- | ---------------- | ---------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `epic_id` | integer | yes | The ID of a group epic | +| `sort` | string | no | Return epic notes sorted in `asc` or `desc` order. Default is `desc` | +| `order_by` | string | no | Return epic notes ordered by `created_at` or `updated_at` fields. Default is `created_at` | + +```bash +curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/5/epics/11/notes +``` + +### Get single epic note + +Returns a single note for a given epic. + +``` +GET /groups/:id/epics/:epic_id/notes/:note_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `epic_id` | integer | yes | The ID of an epic | +| `note_id` | integer | yes | The ID of a note | + +```json +{ + "id": 52, + "title": "Epic", + "file_name": "epic.rb", + "author": { + "id": 1, + "username": "pipin", + "email": "admin@example.com", + "name": "Pip", + "state": "active", + "created_at": "2013-09-30T13:46:01Z" + }, + "expires_at": null, + "updated_at": "2013-10-02T07:34:20Z", + "created_at": "2013-10-02T07:34:20Z" +} +``` + +```bash +curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/5/epics/11/notes/1 +``` + +### Create new epic note + +Creates a new note for a single epic. Epic notes are comments users can post to an epic. +If you create a note where the body only contains an Award Emoji, you'll receive this object back. + +``` +POST /groups/:id/epics/:epic_id/notes +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `epic_id` | integer | yes | The ID of an epic | +| `body` | string | yes | The content of a note | + +```bash +curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/5/snippet/11/notes?body=note +``` + +### Modify existing epic note + +Modify existing note of an epic. + +``` +PUT /groups/:id/epics/:epic_id/notes/:note_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `epic_id` | integer | yes | The ID of an epic | +| `note_id` | integer | yes | The ID of a note | +| `body` | string | yes | The content of a note | + +```bash +curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/5/snippet/11/notes?body=note +``` + +### Delete an epic note + +Deletes an existing note of an epic. + +``` +DELETE /groups/:id/epics/:epic_id/notes/:note_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `epic_id` | integer | yes | The ID of an epic | +| `note_id` | integer | yes | The ID of a note | + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/5/epics/52/notes/1659 +``` diff --git a/doc/api/project_clusters.md b/doc/api/project_clusters.md index c831cc52a93..327781f6c93 100644 --- a/doc/api/project_clusters.md +++ b/doc/api/project_clusters.md @@ -167,6 +167,7 @@ Parameters: | `platform_kubernetes_attributes[ca_cert]` | String | no | TLS certificate (needed if API is using a self-signed TLS certificate | | `platform_kubernetes_attributes[namespace]` | String | no | The unique namespace related to the project | | `platform_kubernetes_attributes[authorization_type]` | String | no | The cluster authorization type: `rbac`, `abac` or `unknown_authorization`. Defaults to `rbac`. | +| `environment_scope` | String | no | The associated environment to the cluster. Defaults to `*` **[PREMIUM]** | Example request: @@ -256,6 +257,7 @@ Parameters: | `platform_kubernetes_attributes[token]` | String | no | The token to authenticate against Kubernetes | | `platform_kubernetes_attributes[ca_cert]` | String | no | TLS certificate (needed if API is using a self-signed TLS certificate | | `platform_kubernetes_attributes[namespace]` | String | no | The unique namespace related to the project | +| `environment_scope` | String | no | The associated environment to the cluster **[PREMIUM]** | NOTE: **Note:** `name`, `api_url`, `ca_cert` and `token` can only be updated if the cluster was added diff --git a/doc/api/project_level_variables.md b/doc/api/project_level_variables.md index 4a6f5624394..66a749e4811 100644 --- a/doc/api/project_level_variables.md +++ b/doc/api/project_level_variables.md @@ -1,4 +1,4 @@ -# Project-level Variables API +# Project-level Variables API ## List project variables @@ -52,7 +52,9 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a { "key": "TEST_VARIABLE_1", "variable_type": "env_var", - "value": "TEST_1" + "value": "TEST_1", + "protected": false, + "masked": true } ``` @@ -64,13 +66,15 @@ Create a new variable. POST /projects/:id/variables ``` -| Attribute | Type | required | Description | -|-----------------|---------|----------|-----------------------| -| `id` | integer/string | yes | The ID of a project or [urlencoded NAMESPACE/PROJECT_NAME of the project](README.md#namespaced-path-encoding) owned by the authenticated user | -| `key` | string | yes | The `key` of a variable; must have no more than 255 characters; only `A-Z`, `a-z`, `0-9`, and `_` are allowed | -| `value` | string | yes | The `value` of a variable | -| `variable_type` | string | no | The type of a variable. Available types are: `env_var` (default) and `file` | -| `protected` | boolean | no | Whether the variable is protected | +| Attribute | Type | required | Description | +|---------------------|---------|----------|-----------------------| +| `id` | integer/string | yes | The ID of a project or [urlencoded NAMESPACE/PROJECT_NAME of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `key` | string | yes | The `key` of a variable; must have no more than 255 characters; only `A-Z`, `a-z`, `0-9`, and `_` are allowed | +| `value` | string | yes | The `value` of a variable | +| `variable_type` | string | no | The type of a variable. Available types are: `env_var` (default) and `file` | +| `protected` | boolean | no | Whether the variable is protected | +| `masked` | boolean | no | Whether the variable is masked | +| `environment_scope` | string | no | The `environment_scope` of the variable **[PREMIUM]** | ``` curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/variables" --form "key=NEW_VARIABLE" --form "value=new value" @@ -80,8 +84,11 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitla { "key": "NEW_VARIABLE", "value": "new value", + "protected": false, "variable_type": "env_var", - "protected": false + "protected": false, + "masked": false, + "environment_scope": "*" } ``` @@ -93,13 +100,15 @@ Update a project's variable. PUT /projects/:id/variables/:key ``` -| Attribute | Type | required | Description | -|-----------------|---------|----------|-------------------------| -| `id` | integer/string | yes | The ID of a project or [urlencoded NAMESPACE/PROJECT_NAME of the project](README.md#namespaced-path-encoding) owned by the authenticated user | -| `key` | string | yes | The `key` of a variable | -| `value` | string | yes | The `value` of a variable | -| `variable_type` | string | no | The type of a variable. Available types are: `env_var` (default) and `file` | -| `protected` | boolean | no | Whether the variable is protected | +| Attribute | Type | required | Description | +|---------------------|---------|----------|-------------------------| +| `id` | integer/string | yes | The ID of a project or [urlencoded NAMESPACE/PROJECT_NAME of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `key` | string | yes | The `key` of a variable | +| `value` | string | yes | The `value` of a variable | +| `variable_type` | string | no | The type of a variable. Available types are: `env_var` (default) and `file` | +| `protected` | boolean | no | Whether the variable is protected | +| `masked` | boolean | no | Whether the variable is masked | +| `environment_scope` | string | no | The `environment_scope` of the variable **[PREMIUM]** | ``` curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/variables/NEW_VARIABLE" --form "value=updated value" @@ -110,7 +119,9 @@ curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab "key": "NEW_VARIABLE", "value": "updated value", "variable_type": "env_var", - "protected": true + "protected": true, + "masked": false, + "environment_scope": "*" } ``` diff --git a/doc/api/projects.md b/doc/api/projects.md index 951961e45ff..75669d85803 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -153,6 +153,7 @@ When the user is authenticated and `simple` is not set this returns something li "commit_count": 37, "storage_size": 1038090, "repository_size": 1038090, + "wiki_size" : 0, "lfs_objects_size": 0, "job_artifacts_size": 0 }, @@ -234,6 +235,7 @@ When the user is authenticated and `simple` is not set this returns something li "commit_count": 12, "storage_size": 2066080, "repository_size": 2066080, + "wiki_size" : 0, "lfs_objects_size": 0, "job_artifacts_size": 0 }, @@ -342,6 +344,7 @@ GET /users/:user_id/projects "commit_count": 37, "storage_size": 1038090, "repository_size": 1038090, + "wiki_size" : 0, "lfs_objects_size": 0, "job_artifacts_size": 0 }, @@ -423,6 +426,7 @@ GET /users/:user_id/projects "commit_count": 12, "storage_size": 2066080, "repository_size": 2066080, + "wiki_size" : 0, "lfs_objects_size": 0, "job_artifacts_size": 0 }, @@ -548,6 +552,7 @@ GET /projects/:id "commit_count": 37, "storage_size": 1038090, "repository_size": 1038090, + "wiki_size" : 0, "lfs_objects_size": 0, "job_artifacts_size": 0 }, diff --git a/doc/api/repository_storage_health.md b/doc/api/repository_storage_health.md deleted file mode 100644 index edf4b04acea..00000000000 --- a/doc/api/repository_storage_health.md +++ /dev/null @@ -1,5 +0,0 @@ -# Circuitbreaker API - -NOTE: **Deprecated:** -Support of the circuit breaker is removed, as Gitaly can be configured to -to work without NFS and [communicate solely over HTTP](../administration/gitaly/index.md). diff --git a/doc/api/resource_label_events.md b/doc/api/resource_label_events.md index e1f9ffa9472..f0a7ac4e41d 100644 --- a/doc/api/resource_label_events.md +++ b/doc/api/resource_label_events.md @@ -88,6 +88,92 @@ Parameters: curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/5/issues/11/resource_label_events/1 ``` +## Epics **[ULTIMATE]** + +### List group epic label events + +Gets a list of all label events for a single epic. + +``` +GET /groups/:id/epics/:epic_id/resource_label_events +``` + +| Attribute | Type | Required | Description | +| ------------------- | ---------------- | ---------- | ------------ | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `epic_id` | integer | yes | The ID of an epic | + +```json +[ + { + "id": 106, + "user": { + "id": 1, + "name": "Administrator", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", + "web_url": "http://gitlab.example.com/root" + }, + "created_at": "2018-08-19T11:43:01.746Z", + "resource_type": "Epic", + "resource_id": 33, + "label": { + "id": 73, + "name": "a1", + "color": "#34495E", + "description": "" + }, + "action": "add" + }, + { + "id": 107, + "user": { + "id": 1, + "name": "Administrator", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", + "web_url": "http://gitlab.example.com/root" + }, + "created_at": "2018-08-19T11:43:01.746Z", + "resource_type": "Epic", + "resource_id": 33, + "label": { + "id": 37, + "name": "glabel2", + "color": "#A8D695", + "description": "" + }, + "action": "add" + } +] +``` + +```bash +curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/1/epics/11/resource_label_events +``` + +### Get single epic label event + +Returns a single label event for a specific group epic + +``` +GET /groups/:id/epics/:epic_id/resource_label_events/:resource_label_event_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `epic_id` | integer | yes | The ID of an epic | +| `resource_label_event_id` | integer | yes | The ID of a label event | + +```bash +curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/1/epics/11/resource_label_events/107 +``` + ## Merge requests ### List project merge request label events diff --git a/doc/api/scim.md b/doc/api/scim.md index 4595c6f2ed3..3870ea788e7 100644 --- a/doc/api/scim.md +++ b/doc/api/scim.md @@ -1,4 +1,4 @@ -# SCIM API +# SCIM API **[SILVER ONLY]** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/9388) in [GitLab Silver](https://about.gitlab.com/pricing/) 11.10. diff --git a/doc/api/search.md b/doc/api/search.md index c2dd72479c1..da81c8321c9 100644 --- a/doc/api/search.md +++ b/doc/api/search.md @@ -19,6 +19,8 @@ GET /search Search the expression within the specified scope. Currently these scopes are supported: projects, issues, merge_requests, milestones, snippet_titles, snippet_blobs, users. +If Elasticsearch is enabled additional scopes available are blobs, wiki_blobs and commits. Find more about [the feature](../integration/elasticsearch.md). **[STARTER]** + The response depends on the requested scope. ### Scope: projects @@ -281,6 +283,98 @@ Example response: ] ``` +### Scope: wiki_blobs **[STARTER]** + +This scope is available only if [Elasticsearch](../integration/elasticsearch.md) is enabled. + +```bash +curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/search?scope=wiki_blobs&search=bye +``` + +Example response: + +```json + +[ + { + "basename": "home", + "data": "hello\n\nand bye\n\nend", + "filename": "home.md", + "id": null, + "ref": "master", + "startline": 5, + "project_id": 6 + } +] +``` + +### Scope: commits **[STARTER]** + +This scope is available only if [Elasticsearch](../integration/elasticsearch.md) is enabled. + +```bash +curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/search?scope=commits&search=bye +``` + +Example response: + +```json + +[ + { + "id": "4109c2d872d5fdb1ed057400d103766aaea97f98", + "short_id": "4109c2d8", + "title": "goodbye $.browser", + "created_at": "2013-02-18T22:02:54.000Z", + "parent_ids": [ + "59d05353ab575bcc2aa958fe1782e93297de64c9" + ], + "message": "goodbye $.browser\n", + "author_name": "angus croll", + "author_email": "anguscroll@gmail.com", + "authored_date": "2013-02-18T22:02:54.000Z", + "committer_name": "angus croll", + "committer_email": "anguscroll@gmail.com", + "committed_date": "2013-02-18T22:02:54.000Z", + "project_id": 6 + } +] +``` + +### Scope: blobs **[STARTER]** + +This scope is available only if [Elasticsearch](../integration/elasticsearch.md) is enabled. + +Filters are available for this scope: +- filename +- path +- extension + +to use a filter simply include it in your query like so: `a query filename:some_name*`. + +You may use wildcards (`*`) to use glob matching. + +```bash +curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/search?scope=blobs&search=installation +``` + +Example response: + +```json + +[ + { + "basename": "README", + "data": "```\n\n## Installation\n\nQuick start using the [pre-built", + "filename": "README.md", + "id": null, + "ref": "master", + "startline": 46, + "project_id": 6 + } +] +``` + ### Scope: users ```bash @@ -320,6 +414,8 @@ GET /groups/:id/search Search the expression within the specified scope. Currently these scopes are supported: projects, issues, merge_requests, milestones, users. +If Elasticsearch is enabled additional scopes available are blobs, wiki_blobs and commits. Find more about [the feature](../integration/elasticsearch.md). **[STARTER]** + The response depends on the requested scope. ### Scope: projects @@ -520,6 +616,98 @@ Example response: ] ``` +### Scope: wiki_blobs **[STARTER]** + +This scope is available only if [Elasticsearch](../integration/elasticsearch.md) is enabled. + +```bash +curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/6/search?scope=wiki_blobs&search=bye +``` + +Example response: + +```json + +[ + { + "basename": "home", + "data": "hello\n\nand bye\n\nend", + "filename": "home.md", + "id": null, + "ref": "master", + "startline": 5, + "project_id": 6 + } +] +``` + +### Scope: commits **[STARTER]** + +This scope is available only if [Elasticsearch](../integration/elasticsearch.md) is enabled. + +```bash +curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/6/search?scope=commits&search=bye +``` + +Example response: + +```json + +[ + { + "id": "4109c2d872d5fdb1ed057400d103766aaea97f98", + "short_id": "4109c2d8", + "title": "goodbye $.browser", + "created_at": "2013-02-18T22:02:54.000Z", + "parent_ids": [ + "59d05353ab575bcc2aa958fe1782e93297de64c9" + ], + "message": "goodbye $.browser\n", + "author_name": "angus croll", + "author_email": "anguscroll@gmail.com", + "authored_date": "2013-02-18T22:02:54.000Z", + "committer_name": "angus croll", + "committer_email": "anguscroll@gmail.com", + "committed_date": "2013-02-18T22:02:54.000Z", + "project_id": 6 + } +] +``` + +### Scope: blobs **[STARTER]** + +This scope is available only if [Elasticsearch](../integration/elasticsearch.md) is enabled. + +Filters are available for this scope: +- filename +- path +- extension + +to use a filter simply include it in your query like so: `a query filename:some_name*`. + +You may use wildcards (`*`) to use glob matching. + +```bash +curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/6/search?scope=blobs&search=installation +``` + +Example response: + +```json + +[ + { + "basename": "README", + "data": "```\n\n## Installation\n\nQuick start using the [pre-built", + "filename": "README.md", + "id": null, + "ref": "master", + "startline": 46, + "project_id": 6 + } +] +``` + ### Scope: users ```bash @@ -556,7 +744,6 @@ GET /projects/:id/search | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | | `scope` | string | yes | The scope to search in | | `search` | string | yes | The search query | -| `ref` | string | no | The name of a repository branch or tag to search on. The project's default branch is used by default. This is only applicable for scopes: commits, blobs, and wiki_blobs. | Search the expression within the specified scope. Currently these scopes are supported: issues, merge_requests, milestones, notes, wiki_blobs, commits, blobs, users. @@ -851,7 +1038,7 @@ Blobs searches are performed on both filenames and contents. Search results: times in the content. ```bash -curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/6/search?scope=blobs&search=installation&ref=feature +curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/6/search?scope=blobs&search=installation ``` Example response: @@ -864,7 +1051,7 @@ Example response: "data": "```\n\n## Installation\n\nQuick start using the [pre-built", "filename": "README.md", "id": null, - "ref": "feature", + "ref": "master", "startline": 46, "project_id": 6 } diff --git a/doc/api/services.md b/doc/api/services.md index 742abccb69e..898cfad7254 100644 --- a/doc/api/services.md +++ b/doc/api/services.md @@ -754,6 +754,7 @@ Parameters: | `recipients` | string | yes | Comma-separated list of recipient email addresses | | `add_pusher` | boolean | no | Add pusher to recipients list | | `notify_only_broken_pipelines` | boolean | no | Notify only broken pipelines | +| `notify_only_default_branch` | boolean | no | Send notifications only for the default branch ([introduced in GitLab 12.0](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/28271)) | ### Delete Pipeline-Emails service @@ -1100,6 +1101,75 @@ Get JetBrains TeamCity CI service settings for a project. GET /projects/:id/services/teamcity ``` +## Jenkins CI **[STARTER]** + +A continuous integration and build server + +### Create/Edit Jenkins CI service + +Set Jenkins CI service for a project. + +``` +PUT /projects/:id/services/jenkins +``` + +Parameters: + +- `jenkins_url` (**required**) - Jenkins URL like http://jenkins.example.com +- `project_name` (**required**) - The URL-friendly project name. Example: my_project_name +- `username` (optional) - A user with access to the Jenkins server, if applicable +- `password` (optional) - The password of the user + +### Delete Jenkins CI service + +Delete Jenkins CI service for a project. + +``` +DELETE /projects/:id/services/jenkins +``` + +### Get Jenkins CI service settings + +Get Jenkins CI service settings for a project. + +``` +GET /projects/:id/services/jenkins +``` + +## Jenkins CI (Deprecated) Service + +A continuous integration and build server + +### Create/Edit Jenkins CI (Deprecated) service + +Set Jenkins CI (Deprecated) service for a project. + +``` +PUT /projects/:id/services/jenkins-deprecated +``` + +Parameters: + +- `project_url` (**required**) - Jenkins project URL like http://jenkins.example.com/job/my-project/ +- `multiproject_enabled` (optional) - Multi-project mode is configured in Jenkins GitLab Hook plugin +- `pass_unstable` (optional) - Unstable builds will be treated as passing + +### Delete Jenkins CI (Deprecated) service + +Delete Jenkins CI (Deprecated) service for a project. + +``` +DELETE /projects/:id/services/jenkins-deprecated +``` + +### Get Jenkins CI (Deprecated) service settings + +Get Jenkins CI (Deprecated) service settings for a project. + +``` +GET /projects/:id/services/jenkins-deprecated +``` + [jira-doc]: ../user/project/integrations/jira.md [old-jira-api]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-13-stable/doc/api/services.md#jira diff --git a/doc/api/snippets.md b/doc/api/snippets.md index f90447e124e..1ce0b1e7a62 100644 --- a/doc/api/snippets.md +++ b/doc/api/snippets.md @@ -165,15 +165,15 @@ Parameters: |:--------------|:-------|:---------|:---------------------------------------------------| | `title` | string | yes | Title of a snippet. | | `file_name` | string | yes | Name of a snippet file. | -| `content` | string | yes | Content of a snippet. | +| `code` | string | yes | Content of a snippet. | | `description` | string | no | Description of a snippet. | -| `visibility` | string | no | Snippet's [visibility](#snippet-visibility-level). | +| `visibility` | string | yes | Snippet's [visibility](#snippet-visibility-level). | Example request: ```sh curl --request POST \ - --data '{"title": "This is a snippet", "content": "Hello world", "description": "Hello World snippet", "file_name": "test.txt", "visibility": "internal" }' \ + --data '{"title": "This is a snippet", "code": "Hello world", "description": "Hello World snippet", "file_name": "test.txt", "visibility": "internal" }' \ --header 'Content-Type: application/json' \ --header "PRIVATE-TOKEN: valid_api_token" \ https://gitlab.example.com/api/v4/snippets @@ -222,14 +222,14 @@ Parameters: | `title` | string | no | Title of a snippet. | | `file_name` | string | no | Name of a snippet file. | | `description` | string | no | Description of a snippet. | -| `content` | string | no | Content of a snippet. | +| `code` | string | no | Content of a snippet. | | `visibility` | string | no | Snippet's [visibility](#snippet-visibility-level). | Example request: ```sh curl --request PUT \ - --data '{"title": "foo", "content": "bar"}' \ + --data '{"title": "foo", "code": "bar"}' \ --header 'Content-Type: application/json' \ --header "PRIVATE-TOKEN: valid_api_token" \ https://gitlab.example.com/api/v4/snippets/1 diff --git a/doc/api/users.md b/doc/api/users.md index d3e67d3d510..47028c679b8 100644 --- a/doc/api/users.md +++ b/doc/api/users.md @@ -140,8 +140,7 @@ GET /users "can_create_project": true, "two_factor_enabled": true, "external": false, - "private_profile": false, - "highest_role":10 + "private_profile": false } ] ``` @@ -257,7 +256,8 @@ Parameters: "can_create_project": true, "two_factor_enabled": true, "external": false, - "private_profile": false + "private_profile": false, + "highest_role":10 } ``` diff --git a/doc/api/vulnerabilities.md b/doc/api/vulnerabilities.md index 87f77613ad3..390d0966244 100644 --- a/doc/api/vulnerabilities.md +++ b/doc/api/vulnerabilities.md @@ -1,4 +1,4 @@ -# Vulnerabilities API +# Vulnerabilities API **[ULTIMATE]** Every API call to vulnerabilities must be authenticated. diff --git a/doc/articles/how_to_configure_ldap_gitlab_ee/index.md b/doc/articles/how_to_configure_ldap_gitlab_ee/index.md index 4ce96fcb230..3e6f3130437 100644 --- a/doc/articles/how_to_configure_ldap_gitlab_ee/index.md +++ b/doc/articles/how_to_configure_ldap_gitlab_ee/index.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/administration/auth/how_to_configure_ldap_gitlab_ee/index.html' +redirect_to: '../../administration/auth/how_to_configure_ldap_gitlab_ee/index.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/administration/auth/how_to_configure_ldap_gitlab_ee/index.html). +This document was moved to [another location](../../administration/auth/how_to_configure_ldap_gitlab_ee/index.md). diff --git a/doc/ci/README.md b/doc/ci/README.md index 27bde2ac0f1..635cce13b4e 100644 --- a/doc/ci/README.md +++ b/doc/ci/README.md @@ -1,6 +1,7 @@ --- comments: false description: "Learn how to use GitLab CI/CD, the GitLab built-in Continuous Integration, Continuous Deployment, and Continuous Delivery toolset to build, test, and deploy your application." +type: index --- # GitLab CI/CD @@ -160,6 +161,33 @@ See also the [Why CI/CD?](https://docs.google.com/presentation/d/1OGgk2Tcxbpl7DJ As GitLab CI/CD has evolved, certain breaking changes have been necessary. These are: +#### 12.0 + +- [Use refspec to clone/fetch git + repository](https://gitlab.com/gitlab-org/gitlab-runner/issues/4069). +- [Old cache + configuration](https://gitlab.com/gitlab-org/gitlab-runner/issues/4070). +- [Old metrics server + configuration](https://gitlab.com/gitlab-org/gitlab-runner/issues/4072). +- [Remove + `FF_K8S_USE_ENTRYPOINT_OVER_COMMAND`](https://gitlab.com/gitlab-org/gitlab-runner/issues/4073). +- [Remove Linux distributions that reach + EOL](https://gitlab.com/gitlab-org/gitlab-runner/merge_requests/1130). +- [Update command line API for helper + images](https://gitlab.com/gitlab-org/gitlab-runner/issues/4013). +- [Remove old `git clean` + flow](https://gitlab.com/gitlab-org/gitlab-runner/issues/4175). + +#### 11.0 + +- No breaking changes. + +#### 10.0 + +- No breaking changes. + +#### 9.0 + - [CI variables renaming for GitLab 9.0](variables/deprecated_variables.md#gitlab-90-renamed-variables). Read about the deprecated CI variables and what you should use for GitLab 9.0+. - [New CI job permissions model](../user/project/new_ci_build_permissions_model.md). diff --git a/doc/ci/caching/index.md b/doc/ci/caching/index.md index 3f72fe3e9eb..9a5a3624c73 100644 --- a/doc/ci/caching/index.md +++ b/doc/ci/caching/index.md @@ -1,3 +1,7 @@ +--- +type: index, concepts, howto +--- + # Cache dependencies in GitLab CI/CD GitLab CI/CD provides a caching mechanism that can be used to save time @@ -60,7 +64,7 @@ In summary: - Caches are disabled if not defined globally or per job (using `cache:`). - Caches are available for all jobs in your `.gitlab-ci.yml` if enabled globally. -- Caches can be used by subsequent pipelines of that very same job (a script in +- Caches can be used by subsequent pipelines of that same job (a script in a stage) in which the cache was created (if not defined globally). - Caches are stored where the Runner is installed **and** uploaded to S3 if [distributed cache is enabled](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching). @@ -87,7 +91,7 @@ cache, when declaring `cache` in your jobs, use one or a mix of the following: that share their cache. - [Use sticky Runners](../runners/README.md#locking-a-specific-runner-from-being-enabled-for-other-projects) that will be only available to a particular project. -- [Use a `key`](../yaml/README.md#cachekey) that fits your workflow (e.g., +- [Use a `key`](../yaml/README.md#cachekey) that fits your workflow (for example, different caches on each branch). For that, you can take advantage of the [CI/CD predefined variables](../variables/README.md#predefined-environment-variables). @@ -100,7 +104,7 @@ From the perspective of the Runner, in order for cache to work effectively, one of the following must be true: - Use a single Runner for all your jobs. -- Use multiple Runners (in autoscale mode or not) that use. +- Use multiple Runners (in autoscale mode or not) that use [distributed caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching), where the cache is stored in S3 buckets (like shared Runners on GitLab.com). - Use multiple Runners (not in autoscale mode) of the same architecture that @@ -420,7 +424,7 @@ mismatch and a few ideas how to fix it. Let's explore some examples. ---- +#### Examples Let's assume you have only one Runner assigned to your project, so the cache will be stored in the Runner's machine by default. If two jobs, A and B, @@ -462,8 +466,6 @@ job B: To fix that, use different `keys` for each job. ---- - In another case, let's assume you have more than one Runners assigned to your project, but the distributed cache is not enabled. We want the second time the pipeline is run, `job A` and `job B` to re-use their cache (which in this case @@ -526,3 +528,15 @@ Behind the scenes, this works by increasing a counter in the database, and the value of that counter is used to create the key for the cache by appending an integer to it: `-1`, `-2`, etc. After a push, a new key is generated and the old cache is not valid anymore. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/ci/chatops/README.md b/doc/ci/chatops/README.md index 5ecdf0f8c54..241134783da 100644 --- a/doc/ci/chatops/README.md +++ b/doc/ci/chatops/README.md @@ -1,3 +1,7 @@ +--- +type: index, concepts, howto +--- + # GitLab ChatOps > **Notes:** @@ -10,7 +14,10 @@ GitLab ChatOps provides a method to interact with CI/CD jobs through chat servic ## How it works -GitLab ChatOps is built upon two existing features, [GitLab CI/CD](../README.md) and [Slack Slash Commmands](../../user/project/integrations/slack_slash_commands.md). +GitLab ChatOps is built upon two existing features: + +- [GitLab CI/CD](../README.md). +- [Slack Slash Commands](../../user/project/integrations/slack_slash_commands.md). A new `run` action has been added to the [slash commands](../../integration/slash_commands.md), which takes two arguments: a `<job name>` to execute and the `<job arguments>`. When executed, ChatOps will look up the specified job name and attempt to match it to a corresponding job in [.gitlab-ci.yml](../yaml/README.md). If a matching job is found on `master`, a pipeline containing just that job is scheduled. Two additional [CI/CD variables](../variables/README.md#predefined-environment-variables) are passed to the job: `CHAT_INPUT` contains any additional arguments, and `CHAT_CHANNEL` is set to the name of channel the action was triggered in. @@ -22,9 +29,9 @@ After the job has finished, its output is sent back to Slack provided it has com Since ChatOps is built upon GitLab CI/CD, the job has all the same features and functions available. There a few best practices to consider however when creating ChatOps jobs: -* It is strongly recommended to set `only: [chat]` so the job does not run as part of the standard CI pipeline. -* If the job is set to `when: manual`, the pipeline will be created however the job will wait to be started. -* It is important to keep in mind that there is very limited support for access control. If the user who triggered the slash command is a developer in the project, the job will run. The job itself can utilize existing [CI/CD variables](../variables/README.html#predefined-environment-variables) like `GITLAB_USER_ID` to perform additional rights validation, however these variables can be [overridden](../variables/README.html#priority-of-environment-variables). +- It is strongly recommended to set `only: [chat]` so the job does not run as part of the standard CI pipeline. +- If the job is set to `when: manual`, the pipeline will be created however the job will wait to be started. +- It is important to keep in mind that there is limited support for access control. If the user who triggered the slash command is a developer in the project, the job will run. The job itself can utilize existing [CI/CD variables](../variables/README.html#predefined-environment-variables) like `GITLAB_USER_ID` to perform additional rights validation, however these variables can be [overridden](../variables/README.html#priority-of-environment-variables). ### Controlling the ChatOps reply @@ -59,3 +66,15 @@ You can find and download the official GitLab ChatOps icon here.  [Download bigger image](img/gitlab-chatops-icon.png) + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md index 1c8e12d229c..9126eb65f07 100644 --- a/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md +++ b/doc/ci/ci_cd_for_external_repos/bitbucket_integration.md @@ -1,8 +1,15 @@ +--- +type: howto +--- + # Using GitLab CI/CD with a Bitbucket Cloud repository **[PREMIUM]** -GitLab CI/CD can be used with Bitbucket Cloud by creating a -[CI/CD project](https://docs.gitlab.com/ee/user/project/ci_cd_for_external_repo.html) and connecting -your Git repository via URL. +GitLab CI/CD can be used with Bitbucket Cloud by: + +1. Creating a [CI/CD project](https://docs.gitlab.com/ee/user/project/ci_cd_for_external_repo.html). +1. Connecting your Git repository via URL. + +To use GitLab CI/CD with a Bitbucket Cloud repository: 1. In GitLab create a **CI/CD for external repo**, select **Repo by URL** and create the project. @@ -16,13 +23,13 @@ your Git repository via URL. with `api` scope. This will be used to authenticate requests from the web hook that will be created in Bitbucket to notify GitLab of new commits. -1. In Bitbucket from **Settings > Webhooks** create a new web hook to notify +1. In Bitbucket, from **Settings > Webhooks**, create a new web hook to notify GitLab of new commits. The web hook URL should be set to the GitLab API to trigger pull mirroring, using the Personal Access Token we just generated for authentication. - ``` + ```text https://gitlab.com/api/v4/projects/<NAMESPACE>%2F<PROJECT>/mirror/pull?private_token=<PERSONAL_ACCESS_TOKEN> ``` @@ -33,27 +40,27 @@ your Git repository via URL. After saving, test the web hook by pushing a change to your Bitbucket repository. -1. In Bitbucket create an **App Password** from **Bitbucket Settings > App +1. In Bitbucket, create an **App Password** from **Bitbucket Settings > App Passwords** to authenticate the build status script setting commit build statuses in Bitbucket. Repository write permissions are required.  -1. In GitLab from **Settings > CI/CD > Environment variables** add variables to allow - communication with Bitbucket via the Bitbucket API. +1. In GitLab, from **Settings > CI/CD > Environment variables**, add variables to allow + communication with Bitbucket via the Bitbucket API: - `BITBUCKET_ACCESS_TOKEN`: the Bitbucket app password created above + `BITBUCKET_ACCESS_TOKEN`: the Bitbucket app password created above. - `BITBUCKET_USERNAME`: the username of the Bitbucket account + `BITBUCKET_USERNAME`: the username of the Bitbucket account. - `BITBUCKET_NAMESPACE`: set this if your GitLab and Bitbucket namespaces differ + `BITBUCKET_NAMESPACE`: set this if your GitLab and Bitbucket namespaces differ. - `BITBUCKET_REPOSITORY`: set this if your GitLab and Bitbucket project names differ + `BITBUCKET_REPOSITORY`: set this if your GitLab and Bitbucket project names differ. -1. In Bitbucket add a script to push the pipeline status to Bitbucket. +1. In Bitbucket, add a script to push the pipeline status to Bitbucket. > Note: changes made in GitLab will be overwritten by any changes made - upstream in Bitbucket. + > upstream in Bitbucket. Create a file `build_status` and insert the script below and run `chmod +x build_status` in your terminal to make the script executable. @@ -111,7 +118,7 @@ your Git repository via URL. 1. Still in Bitbucket, create a `.gitlab-ci.yml` file to use the script to push pipeline success and failures to Bitbucket. - ``` + ```yaml stages: - test - ci_status @@ -145,3 +152,15 @@ GitLab is now configured to mirror changes from Bitbucket, run CI/CD pipelines configured in `.gitlab-ci.yml` and push the status to Bitbucket. [pull-mirroring]: ../../workflow/repository_mirroring.md#pulling-from-a-remote-repository-starter + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/ci/ci_cd_for_external_repos/github_integration.md b/doc/ci/ci_cd_for_external_repos/github_integration.md index 0e2acf957e0..612dcc93bc1 100644 --- a/doc/ci/ci_cd_for_external_repos/github_integration.md +++ b/doc/ci/ci_cd_for_external_repos/github_integration.md @@ -1,3 +1,7 @@ +--- +type: howto +--- + # Using GitLab CI/CD with a GitHub repository **[PREMIUM]** GitLab CI/CD can be used with **GitHub.com** and **GitHub Enterprise** by @@ -15,7 +19,7 @@ administrator: NOTE: **Note:** Due to a 10-token limitation on the [GitHub OAuth Implementation](https://developer.github.com/apps/building-oauth-apps/authorizing-oauth-apps/#creating-multiple-tokens-for-oauth-apps), -if you import more than 10 times, your oldest imported project's token will be +if you import more than 10 times, your oldest imported project's token will be revoked. See issue [#9147](https://gitlab.com/gitlab-org/gitlab-ee/issues/9147) for more information. @@ -31,23 +35,27 @@ for more information. 1. In GitHub, add a `.gitlab-ci.yml` to [configure GitLab CI/CD](../quick_start/README.md). -GitLab will import the project, enable [Pull Mirroring](../../workflow/repository_mirroring.md#pulling-from-a-remote-repository-starter), enable -[GitHub project integration](https://docs.gitlab.com/ee/user/project/integrations/github.html), and create a web hook -on GitHub to notify GitLab of new commits. +GitLab will: + +1. Import the project. +1. Enable [Pull Mirroring](../../workflow/repository_mirroring.md#pulling-from-a-remote-repository-starter). +1. Enable [GitHub project integration](https://docs.gitlab.com/ee/user/project/integrations/github.html). +1. Create a web hook on GitHub to notify GitLab of new commits. ## Connect with Personal Access Token -NOTE: **Note:** Personal access tokens can only be used to connect GitHub.com +NOTE: **Note:** +Personal access tokens can only be used to connect GitHub.com repositories to GitLab. If you are not using the [GitHub integration](../../integration/github.md), you can still perform a one-off authorization with GitHub to grant GitLab access your repositories: -1. Open https://github.com/settings/tokens/new to create a **Personal Access +1. Open <https://github.com/settings/tokens/new> to create a **Personal Access Token**. This token with be used to access your repository and push commit statuses to GitHub. - + The `repo` and `admin:repo_hook` should be enable to allow GitLab access to your project, update commit statuses, and create a web hook to notify GitLab of new commits. @@ -62,20 +70,23 @@ repositories: 1. In GitHub, add a `.gitlab-ci.yml` to [configure GitLab CI/CD](../quick_start/README.md). -GitLab will import the project, enable [Pull Mirroring](../../workflow/repository_mirroring.md#pulling-from-a-remote-repository-starter), enable -[GitHub project integration](https://docs.gitlab.com/ee/user/project/integrations/github.html), and create a web hook -on GitHub to notify GitLab of new commits. +GitLab will: + +1. Import the project. +1. Enable [Pull Mirroring](../../workflow/repository_mirroring.md#pulling-from-a-remote-repository-starter). +1. Enable [GitHub project integration](https://docs.gitlab.com/ee/user/project/integrations/github.html). +1. Create a web hook on GitHub to notify GitLab of new commits. ## Connect manually If the [GitHub integration](../../integration/github.md) is not enabled, or is enabled for a different GitHub instance, you GitLab CI/CD can be manually enabled for -your repository. +your repository: -1. In GitHub open https://github.com/settings/tokens/new create a **Personal +1. In GitHub open <https://github.com/settings/tokens/new> create a **Personal Access Token.** GitLab will use this token to access your repository and push commit statuses. - + Enter a **Token description** and update the scope to allow: `repo` so that GitLab can access your project and update commit statuses @@ -109,3 +120,15 @@ your repository.  1. In GitHub add a `.gitlab-ci.yml` to configure GitLab CI/CD. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/ci/ci_cd_for_external_repos/index.md b/doc/ci/ci_cd_for_external_repos/index.md index 450ffe512dc..5de88412121 100644 --- a/doc/ci/ci_cd_for_external_repos/index.md +++ b/doc/ci/ci_cd_for_external_repos/index.md @@ -1,3 +1,7 @@ +--- +type: index, howto +--- + # GitLab CI/CD for external repositories **[PREMIUM]** NOTE: **Note:** @@ -6,22 +10,26 @@ GitLab.com users until September 22nd, 2019. >[Introduced][ee-4642] in [GitLab Premium][eep] 10.6. -GitLab CI/CD can be used with GitHub or any other Git server. +GitLab CI/CD can be used with: + +- [GitHub](github_integration.md). +- [Bitbucket Cloud](bitbucket_integration.md). +- Any other Git server. + Instead of moving your entire project to GitLab, you can connect your external repository to get the benefits of GitLab CI/CD. -- [GitHub](github_integration.md) -- [Bitbucket Cloud](bitbucket_integration.md) - Connecting an external repository will set up [repository mirroring][mirroring] and create a lightweight project where issues, merge requests, wiki, and snippets disabled. These features [can be re-enabled later][settings]. -1. From your GitLab dashboard click **New project** -1. Switch to the **CI/CD for external repo** tab -1. Choose **GitHub** or **Repo by URL** -1. The next steps are similar to the [import flow](../../user/project/import/index.md) +To connect to an external repository: + +1. From your GitLab dashboard, click **New project**. +1. Switch to the **CI/CD for external repo** tab. +1. Choose **GitHub** or **Repo by URL**. +1. The next steps are similar to the [import flow](../../user/project/import/index.md).  diff --git a/doc/ci/docker/README.md b/doc/ci/docker/README.md index 446f5b54f0c..f76471b50f2 100644 --- a/doc/ci/docker/README.md +++ b/doc/ci/docker/README.md @@ -1,9 +1,13 @@ --- comments: false +type: index --- # Docker integration +GitLab CI/CD can be combined with [Docker](https://www.docker.com) to enable +integration between the two. + The following documentation is available for using GitLab CI/CD with Docker: - [Using Docker images](using_docker_images.md). diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 5222cc45bc4..d8068bbb7f0 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -1,16 +1,18 @@ +--- +type: concepts, howto +--- + # Building Docker images with GitLab CI/CD GitLab CI/CD allows you to use Docker Engine to build and test docker-based projects. -TIP: **Tip:** -This also allows to you to use `docker-compose` and other docker-enabled tools. One of the new trends in Continuous Integration/Deployment is to: -1. Create an application image -1. Run tests against the created image -1. Push image to a remote registry -1. Deploy to a server from the pushed image +1. Create an application image. +1. Run tests against the created image. +1. Push image to a remote registry. +1. Deploy to a server from the pushed image. It's also useful when your application already has the `Dockerfile` that can be used to create and test an image: @@ -75,7 +77,7 @@ GitLab Runner then executes job scripts as the `gitlab-runner` user. - docker run my-docker-image /script/to/run/tests ``` -1. You can now use `docker` command and install `docker-compose` if needed. +1. You can now use `docker` command (and **install** `docker-compose` if needed). NOTE: **Note:** By adding `gitlab-runner` to the `docker` group you are effectively granting `gitlab-runner` full root permissions. @@ -85,8 +87,10 @@ For more information please read [On Docker security: `docker` group considered The second approach is to use the special docker-in-docker (dind) [Docker image](https://hub.docker.com/_/docker/) with all tools installed -(`docker` and `docker-compose`) and run the job script in context of that -image in privileged mode. +(`docker`) and run the job script in context of that +image in privileged mode. + +NOTE: **Note:** `docker-compose` is not part of docker-in-docker (dind). In case you'd like to use `docker-compose` in your CI builds, please follow the (installation instructions for docker-compose)[https://docs.docker.com/compose/install/] provided by docker. In order to do that, follow the steps: @@ -113,7 +117,7 @@ In order to do that, follow the steps: The above command will create a `config.toml` entry similar to this: - ``` + ```toml [[runners]] url = "https://gitlab.com/" token = TOKEN @@ -142,9 +146,12 @@ In order to do that, follow the steps: # The 'docker' hostname is the alias of the service container as described at # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services # - # Note that if you're using Kubernetes executor, the variable should be set to - # tcp://localhost:2375 because of how Kubernetes executor connects services + # Note that if you're using the Kubernetes executor, the variable should be set to + # tcp://localhost:2375/ because of how the Kubernetes executor connects services # to the job container + # DOCKER_HOST: tcp://localhost:2375/ + # + # For non-Kubernetes executors, we use tcp://docker:2375/ DOCKER_HOST: tcp://docker:2375/ # When using dind, it's wise to use the overlayfs driver for # improved performance. @@ -224,7 +231,7 @@ In order to do that, follow the steps: The above command will create a `config.toml` entry similar to this: - ``` + ```toml [[runners]] url = "https://gitlab.com/" token = REGISTRATION_TOKEN @@ -267,9 +274,9 @@ aware of the following implications: create containers with specific names, they may conflict with each other. - Sharing files and directories from the source repo into containers may not work as expected since volume mounting is done in the context of the host - machine, not the build container, e.g.: + machine, not the build container. For example: - ``` + ```sh docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests ``` @@ -334,7 +341,7 @@ NOTE: **Note:** The shared Runners on GitLab.com use the `overlay2` driver by default. By default, when using `docker:dind`, Docker uses the `vfs` storage driver which -copies the filesystem on every run. This is a very disk-intensive operation +copies the filesystem on every run. This is a disk-intensive operation which can be avoided if a different driver is used, for example `overlay2`. ### Requirements @@ -342,13 +349,13 @@ which can be avoided if a different driver is used, for example `overlay2`. 1. Make sure a recent kernel is used, preferably `>= 4.2`. 1. Check whether the `overlay` module is loaded: - ``` + ```sh sudo lsmod | grep overlay ``` If you see no result, then it isn't loaded. To load it use: - ``` + ```sh sudo modprobe overlay ``` @@ -356,7 +363,7 @@ which can be avoided if a different driver is used, for example `overlay2`. On Ubuntu systems, this is done by editing `/etc/modules`. Just add the following line into it: - ``` + ```text overlay ``` @@ -364,7 +371,7 @@ which can be avoided if a different driver is used, for example `overlay2`. You can enable the driver for each project individually by editing the project's `.gitlab-ci.yml`: -``` +```yaml variables: DOCKER_DRIVER: overlay2 ``` @@ -568,3 +575,15 @@ deploy: [docker-cap]: https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities [2fa]: ../../user/profile/account/two_factor_authentication.md [pat]: ../../user/profile/personal_access_tokens.md + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/ci/docker/using_docker_images.md b/doc/ci/docker/using_docker_images.md index 13c26bc5f47..d09efce7cc1 100644 --- a/doc/ci/docker/using_docker_images.md +++ b/doc/ci/docker/using_docker_images.md @@ -1,3 +1,7 @@ +--- +type: concepts, howto +--- + # Using Docker images GitLab CI in conjunction with [GitLab Runner](../runners/README.md) can use @@ -45,10 +49,10 @@ The `image` keyword is the name of the Docker image the Docker executor will run to perform the CI tasks. By default, the executor will only pull images from [Docker Hub][hub], -but this can be configured in the `gitlab-runner/config.toml` by setting +however this can be configured in the `gitlab-runner/config.toml` by setting the [Docker pull policy][] to allow using local images. -For more information about images and Docker Hub please read +For more information about images and Docker Hub, please read the [Docker Fundamentals][] documentation. ## What is a service @@ -95,8 +99,8 @@ required for the CI/CD job to proceed and is accessed by network. To make sure this works, the Runner: -1. checks which ports are exposed from the container by default -1. starts a special container that waits for these ports to be accessible +1. Checks which ports are exposed from the container by default. +1. Starts a special container that waits for these ports to be accessible. When the second stage of the check fails, either because there is no opened port in the service, or the service was not started properly before the timeout and the port is not @@ -106,7 +110,7 @@ In most cases it will affect the job, but there may be situations when the job will still succeed even if that warning was printed. For example: - The service was started a little after the warning was raised, and the job is - not using the linked service from the very beginning. In that case, when the + not using the linked service from the beginning. In that case, when the job needed to access the service, it may have been already there waiting for connections. - The service container is not providing any networking service, but it's doing @@ -143,9 +147,9 @@ job: If you need to have `php`, `node` and `go` available for your script, you should either: -- choose an existing Docker image that contains all required tools, or -- create your own Docker image, which will have all the required tools included - and use that in your job +- Choose an existing Docker image that contains all required tools. +- Create your own Docker image, which will have all the required tools included + and use that in your job. ### Accessing the services @@ -167,18 +171,18 @@ access to it from your build container under two hostnames to choose from: - `tutum-wordpress` - `tutum__wordpress` ->**Note:** +NOTE: **Note:** Hostnames with underscores are not RFC valid and may cause problems in 3rd party applications. The default aliases for the service's hostname are created from its image name following these rules: -- Everything after the colon (`:`) is stripped +- Everything after the colon (`:`) is stripped. - Slash (`/`) is replaced with double underscores (`__`) and the primary alias - is created + is created. - Slash (`/`) is replaced with a single dash (`-`) and the secondary alias is - created (requires GitLab Runner v1.1.0 or higher) + created (requires GitLab Runner v1.1.0 or higher). To override the default behavior, you can [specify a service alias](#available-settings-for-services). @@ -333,7 +337,7 @@ services: ``` The Runner will still start two containers using the `mysql:latest` image, -but now each of them will also be accessible with the alias configured +however now each of them will also be accessible with the alias configured in `.gitlab-ci.yml` file. ### Setting a command for the service @@ -408,8 +412,6 @@ you should check which one your Runner is using. Specifically: The syntax of `image:entrypoint` is similar to [Dockerfile's `ENTRYPOINT`][entrypoint]. ----- - Let's assume you have a `super/sql:experimental` image with some SQL database inside it and you would like to use it as a base image for your job because you want to execute some tests with this database binary. Let's also assume that @@ -443,7 +445,7 @@ image: Look for the `[runners.docker]` section: -``` +```toml [runners.docker] image = "ruby:2.1" services = ["mysql:latest", "postgres:latest"] @@ -469,11 +471,11 @@ image which is private and requires you to login into a private container regist Let's also assume that these are the login credentials: -| Key | Value | -|----------|---------------------------| -| registry | registry.example.com:5000 | -| username | my_username | -| password | my_password | +| Key | Value | +|:---------|:----------------------------| +| registry | `registry.example.com:5000` | +| username | `my_username` | +| password | `my_password` | To configure access for `registry.example.com:5000`, follow these steps: @@ -534,7 +536,8 @@ To configure access for `registry.example.com:5000`, follow these steps: You can add configuration for as many registries as you want, adding more registries to the `"auths"` hash as described above. -NOTE: **Note:** The full `hostname:port` combination is required everywhere +NOTE: **Note:** +The full `hostname:port` combination is required everywhere for the Runner to match the `DOCKER_AUTH_CONFIG`. For example, if `registry.example.com:5000/namespace/image:tag` is specified in `.gitlab-ci.yml`, then the `DOCKER_AUTH_CONFIG` must also specify `registry.example.com:5000`. @@ -551,8 +554,9 @@ service containers. For all possible configuration variables check the documentation of each image provided in their corresponding Docker hub page. -*Note: All variables will be passed to all services containers. It's not -designed to distinguish which variable should go where.* +NOTE: **Note:** +All variables will be passed to all services containers. It's not +designed to distinguish which variable should go where. ### PostgreSQL service example @@ -582,8 +586,9 @@ time. ## How to debug a job locally -*Note: The following commands are run without root privileges. You should be -able to run Docker with your regular user account.* +NOTE: **Note:** +The following commands are run without root privileges. You should be +able to run Docker with your regular user account. First start with creating a file named `build_script`: @@ -602,7 +607,7 @@ is specific to your project. Then create some service containers: -``` +```sh docker run -d --name service-mysql mysql:latest docker run -d --name service-postgres postgres:latest ``` @@ -614,7 +619,7 @@ respectively. They will both run in the background (`-d`). Finally, create a build container by executing the `build_script` file we created earlier: -``` +```sh docker run --name build -i --link=service-mysql:mysql --link=service-postgres:postgres ruby:2.1 /bin/bash < build_script ``` @@ -626,7 +631,7 @@ piped using STDIN to the bash interpreter which in turn executes the When you finish testing and no longer need the containers, you can remove them with: -``` +```sh docker rm -f -v build service-mysql service-postgres ``` diff --git a/doc/ci/docker/using_kaniko.md b/doc/ci/docker/using_kaniko.md index f354cdb398e..50f1ac3d54a 100644 --- a/doc/ci/docker/using_kaniko.md +++ b/doc/ci/docker/using_kaniko.md @@ -1,3 +1,7 @@ +--- +type: howto +--- + # Building images with kaniko and GitLab CI/CD > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/45512) in GitLab 11.2. @@ -9,17 +13,18 @@ container images from a Dockerfile, inside a container or Kubernetes cluster. kaniko solves two problems with using the [docker-in-docker build](using_docker_build.md#use-docker-in-docker-executor) method: -1. Docker-in-docker requires [privileged mode](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities) - in order to function, which is a significant security concern. -1. Docker-in-docker generally incurs a performance penalty and can be quite slow. +- Docker-in-docker requires [privileged mode](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities) + in order to function, which is a significant security concern. +- Docker-in-docker generally incurs a performance penalty and can be quite slow. ## Requirements In order to utilize kaniko with GitLab, a [GitLab Runner](https://docs.gitlab.com/runner/) -using either the [Kubernetes](https://docs.gitlab.com/runner/executors/kubernetes.html), -[Docker](https://docs.gitlab.com/runner/executors/docker.html), or -[Docker Machine](https://docs.gitlab.com/runner/executors/docker_machine.html) -executors is required. +using one of the following executors is required: + +- [Kubernetes](https://docs.gitlab.com/runner/executors/kubernetes.html). +- [Docker](https://docs.gitlab.com/runner/executors/docker.html). +- [Docker Machine](https://docs.gitlab.com/runner/executors/docker_machine.html). ## Building a Docker image with kaniko @@ -34,14 +39,17 @@ few important details: - A Docker `config.json` file needs to be created with the authentication information for the desired container registry. ---- +In the following example, kaniko is used to: + +1. Build a Docker image. +1. Then push it to [GitLab Container Registry](../../user/project/container_registry.md). -In the following example, kaniko is used to build a Docker image and then push -it to [GitLab Container Registry](../../user/project/container_registry.md). The job will run only when a tag is pushed. A `config.json` file is created under `/kaniko/.docker` with the needed GitLab Container Registry credentials taken from the [environment variables](../variables/README.md#predefined-environment-variables) -GitLab CI/CD provides. In the last step, kaniko uses the `Dockerfile` under the +GitLab CI/CD provides. + +In the last step, kaniko uses the `Dockerfile` under the root directory of the project, builds the Docker image and pushes it to the project's Container Registry while tagging it with the Git tag: @@ -80,3 +88,15 @@ store: ... -----END CERTIFICATE-----" >> /kaniko/ssl/certs/ca-certificates.crt ``` + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/ci/enable_or_disable_ci.md b/doc/ci/enable_or_disable_ci.md index 7aa7de97c43..56200142055 100644 --- a/doc/ci/enable_or_disable_ci.md +++ b/doc/ci/enable_or_disable_ci.md @@ -1,25 +1,32 @@ +--- +type: howto +--- + # How to enable or disable GitLab CI/CD -To effectively use GitLab CI/CD, you need a valid [`.gitlab-ci.yml`](yaml/README.md) -file present at the root directory of your project and a -[runner](runners/README.md) properly set up. You can read our -[quick start guide](quick_start/README.md) to get you started. +To effectively use GitLab CI/CD, you need: + +- A valid [`.gitlab-ci.yml`](yaml/README.md) file present at the root directory + of your project. +- A [runner](runners/README.md) properly set up. + +You can read our [quick start guide](quick_start/README.md) to get you started. If you are using an external CI/CD server like Jenkins or Drone CI, it is advised to disable GitLab CI/CD in order to not have any conflicts with the commits status API. ---- - GitLab CI/CD is exposed via the `/pipelines` and `/jobs` pages of a project. Disabling GitLab CI/CD in a project does not delete any previous jobs. In fact, the `/pipelines` and `/jobs` pages can still be accessed, although it's hidden from the left sidebar menu. -GitLab CI/CD is enabled by default on new installations and can be disabled either -individually under each project's settings, or site-wide by modifying the -settings in `gitlab.yml` and `gitlab.rb` for source and Omnibus installations -respectively. +GitLab CI/CD is enabled by default on new installations and can be disabled +either: + +- Individually under each project's settings. +- Site-wide by modifying the settings in `gitlab.yml` and `gitlab.rb` for source + and Omnibus installations respectively. ## Per-project user setting @@ -36,10 +43,10 @@ and `gitlab.rb` for source and Omnibus installations respectively. Two things to note: -1. Disabling GitLab CI/CD, will affect only newly-created projects. Projects that - had it enabled prior to this modification, will work as before. -1. Even if you disable GitLab CI/CD, users will still be able to enable it in the - project's settings. +- Disabling GitLab CI/CD, will affect only newly-created projects. Projects that + had it enabled prior to this modification, will work as before. +- Even if you disable GitLab CI/CD, users will still be able to enable it in the + project's settings. For installations from source, open `gitlab.yml` with your editor and set `builds` to `false`: @@ -54,12 +61,32 @@ default_projects_features: builds: false ``` -Save the file and restart GitLab: `sudo service gitlab restart`. +Save the file and restart GitLab: + +```sh +sudo service gitlab restart +``` For Omnibus installations, edit `/etc/gitlab/gitlab.rb` and add the line: -``` +```ruby gitlab_rails['gitlab_default_projects_features_builds'] = false ``` -Save the file and reconfigure GitLab: `sudo gitlab-ctl reconfigure`. +Save the file and reconfigure GitLab: + +```sh +sudo gitlab-ctl reconfigure +``` + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/ci/environments.md b/doc/ci/environments.md index 5a14ac17aec..f2661c4bafd 100644 --- a/doc/ci/environments.md +++ b/doc/ci/environments.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Environments and deployments > Introduced in GitLab 8.9. @@ -88,7 +92,7 @@ deploy_staging: - master ``` -We have defined 3 [stages](yaml/README.md#stages): +We have defined three [stages](yaml/README.md#stages): - `test` - `build` @@ -131,7 +135,7 @@ In summary, with the above `.gitlab-ci.yml` we have achieved the following: > the name given in `.gitlab-ci.yml` (with any variables expanded), while the > second is a "cleaned-up" version of the name, suitable for use in URLs, DNS, > etc. - +> > Starting with GitLab 9.3, the environment URL is exposed to the Runner via > `$CI_ENVIRONMENT_URL`. The URL is expanded from `.gitlab-ci.yml`, or if > the URL was not defined there, the external URL from the environment is used. @@ -667,9 +671,24 @@ fetch = +refs/environments/*:refs/remotes/origin/environments/* ### Scoping environments with specs **[PREMIUM]** -Some GitLab [Enterprise Edition](https://about.gitlab.com/pricing/) features can -behave differently for each environment. For example, you can -[create a secret variable to be injected only into a production environment](variables/README.md#limiting-environment-scopes-of-environment-variables-premium). +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/2112) in [GitLab Premium](https://about.gitlab.com/pricing/) 9.4. + +You can limit the environment scope of a variable by +defining which environments it can be available for. + +Wildcards can be used, and the default environment scope is `*`, which means +any jobs will have this variable, not matter if an environment is defined or +not. + +For example, if the environment scope is `production`, then only the jobs +having the environment `production` defined would have this specific variable. +Wildcards (`*`) can be used along with the environment name, therefore if the +environment scope is `review/*` then any jobs with environment names starting +with `review/` would have that particular variable. + +Some GitLab features can behave differently for each environment. +For example, you can +[create a secret variable to be injected only into a production environment](variables/README.md#limiting-environment-scopes-of-environment-variables-premium). **[PREMIUM]** In most cases, these features use the _environment specs_ mechanism, which offers an efficient way to implement scoping within each environment group. @@ -693,7 +712,7 @@ Each environment can be matched with the following environment spec: As you can see, you can use specific matching for selecting a particular environment, and also use wildcard matching (`*`) for selecting a particular environment group, -such as [Review apps](review_apps/index.md) (`review/*`). +such as [Review Apps](review_apps/index.md) (`review/*`). NOTE: **Note:** The most _specific_ spec takes precedence over the other wildcard matching. @@ -712,3 +731,15 @@ Below are some links you may find interesting: - [A blog post on Deployments & Environments](https://about.gitlab.com/2016/08/26/ci-deployment-and-environments/) - [Review Apps - Use dynamic environments to deploy your code for every branch](review_apps/index.md) - [Deploy Boards for your applications running on Kubernetes](https://docs.gitlab.com/ee/user/project/deploy_boards.html) **[PREMIUM]** + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/ci/environments/protected_environments.md b/doc/ci/environments/protected_environments.md index ab5c0e2dbad..b72ebe838b8 100644 --- a/doc/ci/environments/protected_environments.md +++ b/doc/ci/environments/protected_environments.md @@ -1,3 +1,7 @@ +--- +type: concepts, howto +--- + # Protected Environments **[PREMIUM]** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/6303) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.3. @@ -47,3 +51,15 @@ Maintainers can: - Update existing protected environments at any time by changing the access in the **Allowed to Deploy** dropdown menu. - Unprotect a protected environment by clicking the **Unprotect** button for that environment. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/ci/git_submodules.md b/doc/ci/git_submodules.md index 37078230b34..551044dd76f 100644 --- a/doc/ci/git_submodules.md +++ b/doc/ci/git_submodules.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Using Git submodules with GitLab CI > **Notes:** diff --git a/doc/ci/interactive_web_terminal/index.md b/doc/ci/interactive_web_terminal/index.md index 7109b2ec583..1387d4df500 100644 --- a/doc/ci/interactive_web_terminal/index.md +++ b/doc/ci/interactive_web_terminal/index.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Interactive Web Terminals > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/50144) in GitLab 11.3. diff --git a/doc/ci/introduction/index.md b/doc/ci/introduction/index.md index 0045fa2fb1f..bd2b9b099f2 100644 --- a/doc/ci/introduction/index.md +++ b/doc/ci/introduction/index.md @@ -1,5 +1,6 @@ --- description: "An overview of Continuous Integration, Continuous Delivery, and Continuous Deployment, as well as an introduction to GitLab CI/CD." +type: concepts --- # Introduction to CI/CD with GitLab diff --git a/doc/ci/junit_test_reports.md b/doc/ci/junit_test_reports.md index 799217c9a08..fa78f53f563 100644 --- a/doc/ci/junit_test_reports.md +++ b/doc/ci/junit_test_reports.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # JUnit test reports > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/45318) in GitLab 11.2. diff --git a/doc/ci/large_repositories/index.md b/doc/ci/large_repositories/index.md index 244ccbb92b0..29d649ad717 100644 --- a/doc/ci/large_repositories/index.md +++ b/doc/ci/large_repositories/index.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Optimizing GitLab for large repositories Large repositories consisting of more than 50k files in a worktree diff --git a/doc/ci/merge_request_pipelines/index.md b/doc/ci/merge_request_pipelines/index.md index 3c26a38e3de..fe2fc790505 100644 --- a/doc/ci/merge_request_pipelines/index.md +++ b/doc/ci/merge_request_pipelines/index.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Pipelines for merge requests NOTE: **Note**: @@ -70,15 +74,18 @@ when a merge request was created or updated. For example: ## Pipelines for Merged Results **[PREMIUM]** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/7380) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.10. +> This feature is disabled by default until we resolve issues with [contention handling](https://gitlab.com/gitlab-org/gitlab-ee/issues/9186), but [can be enabled manually](#enabling-pipelines-for-merged-results). It's possible for your source and target branches to diverge, which can result in the scenario that source branch's pipeline was green, the target's pipeline was green, -but the combined output fails. By having your merge request pipeline automatically +but the combined output fails. + +By having your merge request pipeline automatically create a new ref that contains the merge result of the source and target branch (then running a pipeline on that ref), we can better test that the combined result is also valid. -From GitLab 11.10, pipelines for merge requests run by default +GitLab can run pipelines for merge requests on this merged result. That is, where the source and target branches are combined into a new ref and a pipeline for this ref validates the result prior to merging. @@ -95,7 +102,7 @@ get out of WIP status or resolve merge conflicts as soon as possible. ### Enabling Pipelines for Merged Results -This feature disabled by default until we resolve issues with [contention handling](https://gitlab.com/gitlab-org/gitlab-ee/issues/9186). It can be enabled at the project level: +To enable pipelines on merged results at the project level: 1. Visit your project's **Settings > General** and expand **Merge requests**. 1. Check **Merge pipelines will try to validate the post-merge result prior to merging**. @@ -103,6 +110,10 @@ This feature disabled by default until we resolve issues with [contention handli  +CAUTION: **Warning:** +Make sure your `gitlab-ci.yml` file is [configured properly for pipelines for merge requests](#configuring-pipelines-for-merge-requests), +otherwise pipelines for merged results won't run and your merge requests will be stuck in an unresolved state. + ### Pipelines for Merged Result's limitations - This feature requires [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-runner) 11.9 or newer. @@ -193,5 +204,5 @@ By using pipelines for merge requests, GitLab exposes additional predefined vari Those variables contain information of the associated merge request, so that it's useful to integrate your job with [GitLab Merge Request API](../../api/merge_requests.md). -You can find the list of avilable variables in [the reference sheet](../variables/predefined_variables.md). +You can find the list of available variables in [the reference sheet](../variables/predefined_variables.md). The variable names begin with the `CI_MERGE_REQUEST_` prefix. diff --git a/doc/ci/metrics_reports.md b/doc/ci/metrics_reports.md index 83a7094faaa..b7824402d45 100644 --- a/doc/ci/metrics_reports.md +++ b/doc/ci/metrics_reports.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Metrics Reports **[PREMIUM]** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/9788) in [GitLab Premium](https://about.gitlab.com/pricing) 11.10. diff --git a/doc/ci/multi_project_pipelines.md b/doc/ci/multi_project_pipelines.md index 556059c01b6..50c8d82602b 100644 --- a/doc/ci/multi_project_pipelines.md +++ b/doc/ci/multi_project_pipelines.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Multi-project pipelines **[PREMIUM]** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/2121) in @@ -22,6 +26,10 @@ and when hovering or tapping (on touchscreen devices) they will expand and be sh Multi-project pipelines are useful for larger products that require cross-project inter-dependencies, such as those adopting a [microservices architecture](https://about.gitlab.com/2016/08/16/trends-in-version-control-land-microservices/). +For a demonstration of how cross-functional development teams can use cross-pipeline +triggering to trigger multiple pipelines for different microservices projects, see +[Cross-project Pipeline Triggering and Visualization](https://about.gitlab.com/handbook/marketing/product-marketing/demo/#cross-project-pipeline-triggering-and-visualization-may-2019---1110). + ## Use cases Let's assume you deploy your web app from different projects in GitLab: @@ -134,6 +142,35 @@ staging: The `ENVIRONMENT` variable will be passed to every job defined in a downstream pipeline. It will be available as an environment variable when GitLab Runner picks a job. +In the following configuration, the `MY_VARIABLE` variable will be passed to the downstream pipeline +that is created when the `trigger-downstream` job is queued. This is because `trigger-downstream` +job inherits variables declared in global variables blocks, and then we pass these variables to a downstream pipeline. + +```yaml +variables: + MY_VARIABLE: my-value + +trigger-downstream: + variables: + ENVIRONMENT: something + trigger: my/project +``` + +You might want to pass some information about the upstream pipeline using, for +example, predefined variables. In order to do that, you can use interpolation +to pass any variable. For example: + +```yaml +downstream-job: + variables: + UPSTREAM_BRANCH: $CI_COMMIT_REF_NAME + trigger: my/project +``` + +In this scenario, the `UPSTREAM_BRANCH` variable with a value related to the +upstream pipeline will be passed to the `downstream-job` job, and will be available +within the context of all downstream builds. + ### Limitations Because bridge jobs are a little different to regular jobs, it is not diff --git a/doc/ci/pipelines.md b/doc/ci/pipelines.md index 342c2ab972a..1ad3d516df9 100644 --- a/doc/ci/pipelines.md +++ b/doc/ci/pipelines.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Creating and using CI/CD pipelines > Introduced in GitLab 8.8. @@ -266,9 +270,6 @@ Clicking on an individual job will show you its job trace, and allow you to: - Retry the job. - Erase the job trace. -NOTE: **Note:** -To prevent jobs from being bypassed or run out of order, canceled jobs can only be retried when the whole pipeline they belong to is retried. - ### Seeing the failure reason for jobs > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/17782) in GitLab 10.7. @@ -330,6 +331,19 @@ GitLab provides API endpoints to: - [Triggering pipelines through the API](triggers/README.md). - [Pipeline triggers API](../api/pipeline_triggers.md). +### Start multiple manual actions in a stage + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/27188) in GitLab 11.11. + +Multiple manual actions in a single stage can be started at the same time using the "Play all manual" button. +Once the user clicks this button, each individual manual action will be triggered and refreshed +to an updated status. + +This functionality is only available: + +- For users with at least Developer access. +- If the the stage contains [manual actions](#manual-actions-from-pipeline-graphs). + ## Security on protected branches A strict security model is enforced when pipelines are executed on diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index 015f1c0dc0f..11bcfd5dc2c 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -1,9 +1,18 @@ +--- +type: reference +--- + # Getting started with GitLab CI/CD ->**Note:** Starting from version 8.0, GitLab [Continuous Integration][ci] (CI) +NOTE: **Note:** +Starting from version 8.0, GitLab [Continuous Integration][ci] (CI) is fully integrated into GitLab itself and is [enabled] by default on all projects. +NOTE: **Note:** +Please keep in mind that only project Maintainers and Admin users have +the permissions to access a project's settings. + GitLab offers a [continuous integration][ci] service. If you [add a `.gitlab-ci.yml` file][yaml] to the root directory of your repository, and configure your GitLab project to use a [Runner], then each commit or @@ -35,11 +44,12 @@ project's **Pipelines** page. --- -This guide assumes that you: +This guide assumes that you have: -- have a working GitLab instance of version 8.0+r or are using - [GitLab.com](https://gitlab.com) -- have a project in GitLab that you would like to use CI for +- A working GitLab instance of version 8.0+r or are using + [GitLab.com](https://gitlab.com). +- A project in GitLab that you would like to use CI for. +- Maintainer or owner access to the project Let's break it down to pieces and work on solving the GitLab CI puzzle. @@ -73,6 +83,8 @@ You need to create a file named `.gitlab-ci.yml` in the root directory of your repository. Below is an example for a Ruby on Rails project. ```yaml +image: "ruby:2.5" + before_script: - apt-get update -qq && apt-get install -y -qq sqlite3 libsqlite3-dev nodejs - ruby -v diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md index 1a71c5fd258..7b039fe6654 100644 --- a/doc/ci/review_apps/index.md +++ b/doc/ci/review_apps/index.md @@ -1,9 +1,13 @@ +--- +type: reference +--- + # Review Apps > - [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/21971) in GitLab 8.12. Further additions were made in GitLab 8.13 and 8.14. > - Inspired by [Heroku's Review Apps](https://devcenter.heroku.com/articles/github-integration-review-apps), which itself was inspired by [Fourchette](https://github.com/rainforestapp/fourchette). -Review Apps are a collaboration tool that takes the hard work out of providing an environment to showcase product changes. +Review Apps is a collaboration tool that takes the hard work out of providing an environment to showcase product changes. ## Introduction @@ -18,7 +22,7 @@ Review Apps: In the above example: -- A Review App is built every time a commit is pushed to`topic branch`. +- A Review App is built every time a commit is pushed to `topic branch`. - The reviewer fails two reviews before passing the third review. - Once the review as passed, `topic branch` is merged into `master` where it's deploy to staging. - After been approved in staging, the changes that were merged into `master` are deployed in to production. diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index ce55b231666..b089229ab58 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Configuring GitLab Runners In GitLab CI, Runners run the code defined in [`.gitlab-ci.yml`](../yaml/README.md). diff --git a/doc/ci/services/README.md b/doc/ci/services/README.md index 2eda5d23976..7fe12eb53e7 100644 --- a/doc/ci/services/README.md +++ b/doc/ci/services/README.md @@ -1,13 +1,18 @@ --- comments: false +type: index --- -# GitLab CI Services +# GitLab CI services examples -GitLab CI uses the `services` keyword to define what docker containers should -be linked with your base image. Below is a list of examples you may use. +The [`services`](../docker/using_docker_images.md#what-is-a-service) +keyword defines a Docker image that runs during a `job` linked to the +Docker image that the image keyword defines. This allows you to access +the service image during build time. + +The service image can run any application, but the most common use +case is to run a database container, for example: - [Using MySQL](mysql.md) - [Using PostgreSQL](postgres.md) - [Using Redis](redis.md) -- [Using Other Services](../docker/using_docker_images.md#what-is-a-service) diff --git a/doc/ci/services/mysql.md b/doc/ci/services/mysql.md index 5fa378fc4c2..697452cee83 100644 --- a/doc/ci/services/mysql.md +++ b/doc/ci/services/mysql.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Using MySQL As many applications depend on MySQL as their database, you will eventually diff --git a/doc/ci/services/postgres.md b/doc/ci/services/postgres.md index 2e6d7ae94d2..211eea26eb0 100644 --- a/doc/ci/services/postgres.md +++ b/doc/ci/services/postgres.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Using PostgreSQL As many applications depend on PostgreSQL as their database, you will diff --git a/doc/ci/services/redis.md b/doc/ci/services/redis.md index 36f71427ae7..8b227154b06 100644 --- a/doc/ci/services/redis.md +++ b/doc/ci/services/redis.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Using Redis As many applications depend on Redis as their key-value store, you will diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md index 9ed1ec5aa5c..69591ed605c 100644 --- a/doc/ci/ssh_keys/README.md +++ b/doc/ci/ssh_keys/README.md @@ -1,5 +1,6 @@ --- last_updated: 2017-12-13 +type: tutorial --- # Using SSH keys with GitLab CI/CD diff --git a/doc/ci/triggers/README.md b/doc/ci/triggers/README.md index ad80b5d8818..04c541fefe7 100644 --- a/doc/ci/triggers/README.md +++ b/doc/ci/triggers/README.md @@ -1,3 +1,7 @@ +--- +type: tutorial +--- + # Triggering pipelines through the API > **Notes**: diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index 1ba22070abe..df455857dee 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -1,5 +1,6 @@ --- table_display_block: true +type: reference --- # GitLab CI/CD environment variables @@ -58,22 +59,42 @@ the need to specify the value itself. There are two types of variables supported by GitLab: -- `env_var`: the runner will create environment variable named same as the variable key and set its value to the variable value. -- `file`: the runner will write the variable value to a temporary file and set the path to this file as the value of an environment variable named same as the variable key. +- "Variable": the Runner will create an environment variable named same as the variable key and set its value to the variable value. +- "File": the Runner will write the variable value to a temporary file and set the path to this file as the value of an environment variable named same as the variable key. + +Many tools (like [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html) and [kubectl](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/#the-kubeconfig-environment-variable)) provide the ability to customise configuration using files by either providing the file path as a command line argument or an environment variable. Prior to the introduction of variable types, the common pattern was to use the value of a CI variable, save it in a file, and then use the newly created file in your script: + +```bash +# Save the content of variable in a file +echo "$KUBE_CA_PEM" > "$(pwd)/kube.ca.pem" + # Use the newly created file +kubectl config set-cluster e2e --server="$KUBE_URL" --certificate-authority="$(pwd)/kube.ca.pem" +``` + +This can be simplified by creating a variable of type "File" and using it directly. For example, let's say we have the following variables. + + + +We can then call them from `.gitlab-ci.yml` like this: + +```bash +kubectl config set-cluster e2e --server="$KUBE_URL" --certificate-authority="$KUBE_CA_PEM" +``` + +Variable types can be set via the [UI](#via-the-ui) or the [API](../../api/project_level_variables.md#create-variable), but not in `.gitlab-ci.yml`. #### Masked variables > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/13784) in GitLab 11.10 -By default, variables will be created as masked variables. +Variables can be created as masked variables. This means that the value of the variable will be hidden in job logs, though it must match certain requirements to do so: - The value must be in a single line. -- The value must not have escape characters. -- The value must not use variables. -- The value must not have any whitespace. +- The value must only consist of characters from the Base64 alphabet, defined in [RFC4648](https://tools.ietf.org/html/rfc4648). - The value must be at least 8 characters long. +- The value must not use variables. If the value does not meet the requirements above, then the CI variable will fail to save. In order to save, either alter the value to meet the masking requirements @@ -368,21 +389,9 @@ Once you set them, they will be available for all subsequent pipelines. ### Limiting environment scopes of environment variables **[PREMIUM]** -> [Introduced][ee-2112] in [GitLab Premium](https://about.gitlab.com/pricing/) 9.4. - You can limit the environment scope of a variable by [defining which environments][envs] it can be available for. -Wildcards can be used, and the default environment scope is `*` which means -any jobs will have this variable, not matter if an environment is defined or -not. - -For example, if the environment scope is `production`, then only the jobs -having the environment `production` defined would have this specific variable. -Wildcards (`*`) can be used along with the environment name, therefore if the -environment scope is `review/*` then any jobs with environment names starting -with `review/` would have that particular variable. - To learn more about about scoping environments, see [Scoping environments with specs](../environments.md#scoping-environments-with-specs-premium). ### Deployment environment variables @@ -470,6 +479,7 @@ Below you can find supported syntax reference: 1. Equality matching using a string > Example: `$VARIABLE == "some value"` + > Example: `$VARIABLE != "some value"` _(added in 11.11)_ You can use equality operator `==` or `!=` to compare a variable content to a @@ -480,6 +490,7 @@ Below you can find supported syntax reference: 1. Checking for an undefined value > Example: `$VARIABLE == null` + > Example: `$VARIABLE != null` _(added in 11.11)_ It sometimes happens that you want to check whether a variable is defined @@ -490,6 +501,7 @@ Below you can find supported syntax reference: 1. Checking for an empty variable > Example: `$VARIABLE == ""` + > Example: `$VARIABLE != ""` _(added in 11.11)_ If you want to check whether a variable is defined, but is empty, you can @@ -499,6 +511,7 @@ Below you can find supported syntax reference: 1. Comparing two variables > Example: `$VARIABLE_1 == $VARIABLE_2` + > Example: `$VARIABLE_1 != $VARIABLE_2` _(added in 11.11)_ It is possible to compare two variables. This is going to compare values @@ -518,6 +531,7 @@ Below you can find supported syntax reference: 1. Pattern matching _(added in 11.0)_ > Example: `$VARIABLE =~ /^content.*/` + > Example: `$VARIABLE_1 !~ /^content.*/` _(added in 11.11)_ It is possible perform pattern matching against a variable and regular @@ -527,6 +541,19 @@ Below you can find supported syntax reference: Pattern matching is case-sensitive by default. Use `i` flag modifier, like `/pattern/i` to make a pattern case-insensitive. +1. Conjunction / Disjunction + + > Example: `$VARIABLE1 =~ /^content.*/ && $VARIABLE2 == "something"` + + > Example: `$VARIABLE1 =~ /^content.*/ && $VARIABLE2 =~ /thing$/ && $VARIABLE3` + + > Example: `$VARIABLE1 =~ /^content.*/ || $VARIABLE2 =~ /thing$/ && $VARIABLE3` + + It is possible to join multiple conditions using `&&` or `||`. Any of the otherwise + supported syntax may be used in a conjunctive or disjunctive statement. + Precedence of operators follows standard Ruby 2.5 operation + [precedence](https://ruby-doc.org/core-2.5.0/doc/syntax/precedence_rdoc.html). + ## Debug tracing > Introduced in GitLab Runner 1.7. @@ -591,8 +618,8 @@ $'\''git'\'' "checkout" "-f" "-q" "dd648b2e48ce6518303b0bb580b2ee32fadaf045" Running on runner-8a2f473d-project-1796893-concurrent-0 via runner-8a2f473d-machine-1480971377-317a7d0f-digital-ocean-4gb... ++ export CI=true ++ CI=true -++ export CI_API_V4_API_URL=https://example.com:3000/api/v4 -++ CI_API_V4_API_URL=https://example.com:3000/api/v4 +++ export CI_API_V4_URL=https://example.com:3000/api/v4 +++ CI_API_V4_URL=https://example.com:3000/api/v4 ++ export CI_DEBUG_TRACE=false ++ CI_DEBUG_TRACE=false ++ export CI_COMMIT_SHA=dd648b2e48ce6518303b0bb580b2ee32fadaf045 @@ -631,8 +658,8 @@ Running on runner-8a2f473d-project-1796893-concurrent-0 via runner-8a2f473d-mach ++ GITLAB_CI=true ++ export CI=true ++ CI=true -++ export CI_API_V4_API_URL=https://example.com:3000/api/v4 -++ CI_API_V4_API_URL=https://example.com:3000/api/v4 +++ export CI_API_V4_URL=https://example.com:3000/api/v4 +++ CI_API_V4_URL=https://example.com:3000/api/v4 ++ export GITLAB_CI=true ++ GITLAB_CI=true ++ export CI_JOB_ID=7046507 @@ -696,7 +723,6 @@ MIIFQzCCBCugAwIBAgIRAL/ElDjuf15xwja1ZnCocWAwDQYJKoZIhvcNAQELBQAw' ... ``` -[ee-2112]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/2112 [ce-13784]: https://gitlab.com/gitlab-org/gitlab-ce/issues/13784 "Simple protection of CI variables" [envs]: ../environments.md [protected branches]: ../../user/project/protected_branches.md diff --git a/doc/ci/variables/deprecated_variables.md b/doc/ci/variables/deprecated_variables.md index 2642c9b0eb4..cdca5bf27fc 100644 --- a/doc/ci/variables/deprecated_variables.md +++ b/doc/ci/variables/deprecated_variables.md @@ -1,5 +1,12 @@ +--- +type: reference +--- + # Deprecated GitLab CI/CD variables +Read through this document to learn what predefined variables +were deprecated and their new references. + ## GitLab 9.0 renamed variables To follow conventions of naming across GitLab, and to further move away from the diff --git a/doc/ci/variables/img/new_custom_variables_example.png b/doc/ci/variables/img/new_custom_variables_example.png Binary files differindex 4b78e0ff587..efe104efe4c 100644 --- a/doc/ci/variables/img/new_custom_variables_example.png +++ b/doc/ci/variables/img/new_custom_variables_example.png diff --git a/doc/ci/variables/img/variable_types_usage_example.png b/doc/ci/variables/img/variable_types_usage_example.png Binary files differnew file mode 100644 index 00000000000..0e8bde891fe --- /dev/null +++ b/doc/ci/variables/img/variable_types_usage_example.png diff --git a/doc/ci/variables/predefined_variables.md b/doc/ci/variables/predefined_variables.md index 4e902c042e6..4655eec51de 100644 --- a/doc/ci/variables/predefined_variables.md +++ b/doc/ci/variables/predefined_variables.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Predefined environment variables reference For an introduction on this subject, read through the diff --git a/doc/ci/variables/where_variables_can_be_used.md b/doc/ci/variables/where_variables_can_be_used.md index 0470cf52654..8009b1d5e8a 100644 --- a/doc/ci/variables/where_variables_can_be_used.md +++ b/doc/ci/variables/where_variables_can_be_used.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Where variables can be used As it's described in the [CI/CD variables](README.md) docs, you can diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 31ff56e06f8..fa4b0378f61 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # GitLab CI/CD Pipeline Configuration Reference GitLab CI/CD [pipelines](../pipelines.md) are configured using a YAML file called `.gitlab-ci.yml` within each project. @@ -108,7 +112,7 @@ The following table lists available parameters for jobs: | [`parallel`](#parallel) | How many instances of a job should be run in parallel. | | [`trigger`](#trigger-premium) | Defines a downstream pipeline trigger. | | [`include`](#include) | Allows this job to include external YAML files. Also available: `include:local`, `include:file`, `include:template`, and `include:remote`. | -| [`extends`](#extends) | Configuration entry that this job is going to inherit from. | +| [`extends`](#extends) | Configuration entries that this job is going to inherit from. | | [`pages`](#pages) | Upload the result of a job to use with GitLab Pages. | | [`variables`](#variables) | Define job variables on a job level. | @@ -386,17 +390,12 @@ job: - branches@gitlab-org/gitlab-ce except: - master@gitlab-org/gitlab-ce - - release/.*@gitlab-org/gitlab-ce + - /^release/.*$/@gitlab-org/gitlab-ce ``` The above example will run `job` for all branches on `gitlab-org/gitlab-ce`, except `master` and those with names prefixed with `release/`. -NOTE: **Note:** -Because `@` is used to denote the beginning of a ref's repository path, -matching a ref name containing the `@` character in a regular expression -requires the use of the hex character code match `\x40`. - If a job does not have an `only` rule, `only: ['branches', 'tags']` is set by default. If it doesn't have an `except` rule, it is empty. @@ -415,6 +414,28 @@ job: only: ['branches', 'tags'] ``` +#### Regular expressions + +Because `@` is used to denote the beginning of a ref's repository path, +matching a ref name containing the `@` character in a regular expression +requires the use of the hex character code match `\x40`. + +Only the tag or branch name can be matched by a regular expression. +The repository path, if given, is always matched literally. + +If a regular expression shall be used to match the tag or branch name, +the entire ref name part of the pattern has to be a regular expression, +and must be surrounded by `/`. +(With regular expression flags appended after the closing `/`.) +So `issue-/.*/` won't work to match all tag names or branch names +that begin with `issue-`. + +TIP: **Tip** +Use anchors `^` and `$` to avoid the regular expression +matching only a substring of the tag name or branch name. +For example, `/^issue-.*$/` is equivalent to `/^issue-/`, +while just `/issue/` would also match a branch called `severe-issues`. + ### Supported `only`/`except` regexp syntax CAUTION: **Warning:** @@ -1168,9 +1189,9 @@ skip the download step. > - Job artifacts are only collected for successful jobs by default. `artifacts` is used to specify a list of files and directories which should be -attached to the job after success. +attached to the job when it [succeeds, fails, or always](#artifactswhen). -The artifacts will be sent to GitLab after the job finishes successfully and will +The artifacts will be sent to GitLab after the job finishes and will be available for download in the GitLab UI. [Read more about artifacts](../../user/project/pipelines/job_artifacts.md). @@ -1986,7 +2007,7 @@ production: - deploy environment: name: production - url: https://$CI_PROJECT_PATH_SLUG.$AUTO_DEVOPS_DOMAIN + url: https://$CI_PROJECT_PATH_SLUG.$KUBE_INGRESS_BASE_DOMAIN only: - master ``` @@ -2100,7 +2121,7 @@ docker-test: > Introduced in GitLab 11.3. -`extends` defines an entry name that a job that uses `extends` is going to +`extends` defines entry names that a job that uses `extends` is going to inherit from. It is an alternative to using [YAML anchors](#anchors) and is a little @@ -2177,6 +2198,46 @@ spinach: script: rake spinach ``` +It's also possible to use multiple parents for `extends`. +The algorithm used for merge is "closest scope wins", so keys +from the last member will always shadow anything defined on other levels. +For example: + +```yaml +.only-important: + only: + - master + - stable + tags: + - production + +.in-docker: + tags: + - docker + image: alpine + +rspec: + extends: + - .only-important + - .in-docker + script: + - rake rspec +``` + +This results in the following `rspec` job: + +```yaml +rspec: + only: + - master + - stable + tags: + - docker + image: alpine + script: + - rake rspec +``` + ### Using `extends` and `include` together `extends` works across configuration files combined with `include`. @@ -2730,6 +2791,18 @@ using Git 2.10 or newer: git push -o ci.skip ``` +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> + [ce-6323]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6323 [ce-6669]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6669 [ce-7983]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7983 diff --git a/doc/customization/issue_and_merge_request_template.md b/doc/customization/issue_and_merge_request_template.md index 01c31728c21..adaa120a37e 100644 --- a/doc/customization/issue_and_merge_request_template.md +++ b/doc/customization/issue_and_merge_request_template.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/project/description_templates.html#setting-a-default-template-for-issues-and-merge-requests--starter' +redirect_to: '../user/project/description_templates.md#setting-a-default-template-for-issues-and-merge-requests--starter' --- -This document was moved to [description_templates](https://docs.gitlab.com/ee/user/project/description_templates.html#setting-a-default-template-for-issues-and-merge-requests--starter). +This document was moved to [description_templates](../user/project/description_templates.md#setting-a-default-template-for-issues-and-merge-requests--starter). diff --git a/doc/development/README.md b/doc/development/README.md index 2ff38d68a47..d2f09fc01de 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -20,6 +20,7 @@ description: 'Learn how to contribute to GitLab.' - [Automatic CE->EE merge](automatic_ce_ee_merge.md) - [Guidelines for implementing Enterprise Edition features](ee_features.md) - [Security process for developers](https://gitlab.com/gitlab-org/release/docs/blob/master/general/security/developer.md#security-releases-critical-non-critical-as-a-developer) +- [Requesting access to Chatops on GitLab.com](chatops_on_gitlabcom.md#requesting-access) (for GitLabbers) ## UX and frontend guides @@ -59,6 +60,7 @@ description: 'Learn how to contribute to GitLab.' - [DeclarativePolicy framework](policies.md) - [How Git object deduplication works in GitLab](git_object_deduplication.md) - [Geo development](geo.md) +- [Routing](routing.md) ## Performance guides diff --git a/doc/development/api_graphql_styleguide.md b/doc/development/api_graphql_styleguide.md index 8d2bfff3a5d..38270af682e 100644 --- a/doc/development/api_graphql_styleguide.md +++ b/doc/development/api_graphql_styleguide.md @@ -32,6 +32,21 @@ a new presenter specifically for GraphQL. The presenter is initialized using the object resolved by a field, and the context. +### Exposing Global ids + +When exposing an `id` field on a type, we will by default try to +expose a global id by calling `to_global_id` on the resource being +rendered. + +To override this behaviour, you can implement an `id` method on the +type for which you are exposing an id. Please make sure that when +exposing a `GraphQL::ID_TYPE` using a custom method that it is +globally unique. + +The records that are exposing a `full_path` as an `ID_TYPE` are one of +these exceptions. Since the full path is a unique identifier for a +`Project` or `Namespace`. + ### Connection Types GraphQL uses [cursor based @@ -79,14 +94,14 @@ look like this: { "cursor": "Nzc=", "node": { - "id": "77", + "id": "gid://gitlab/Pipeline/77", "status": "FAILED" } }, { "cursor": "Njc=", "node": { - "id": "67", + "id": "gid://gitlab/Pipeline/67", "status": "FAILED" } } @@ -330,7 +345,7 @@ argument :project_path, GraphQL::ID_TYPE, required: true, description: "The project the merge request to mutate is in" -argument :iid, GraphQL::ID_TYPE, +argument :iid, GraphQL::STRING_TYPE, required: true, description: "The iid of the merge request to mutate" diff --git a/doc/development/architecture.md b/doc/development/architecture.md index 9a012f4299b..650325121b2 100644 --- a/doc/development/architecture.md +++ b/doc/development/architecture.md @@ -59,7 +59,7 @@ graph TB PgBouncerExporter[PgBouncer Exporter] --> PgBouncer Prometheus -- TCP 9187 --> PostgreSQLExporter Prometheus -- TCP 9100 --> NodeExporter[Node Exporter] - Prometheus -- TCP 9168 --> GitLabMonito[GitLab Monitor] + Prometheus -- TCP 9168 --> GitLabMonitor[GitLab Monitor] Prometheus -- TCP 9127 --> PgBouncerExporter GitLabMonitor --> PostgreSQL GitLabMonitor --> GitLabShell @@ -106,43 +106,43 @@ Component statuses are linked to configuration documentation for each component. ### Component list -| Component | Description | [Omnibus GitLab](https://docs.gitlab.com/omnibus/README.html) | [GitLab chart](https://docs.gitlab.com/charts/) | [Minikube Minimal](https://docs.gitlab.com/charts/development/minikube/#deploying-gitlab-with-minimal-settings) | [GitLab.com](https://gitlab.com) | CE/EE | -| --------- | ----------- |:--------------------:|:------------------:|:-----:|:--------:|:--------:| -| [NGINX](#nginx) | Routes requests to appropriate components, terminates SSL | [✅][nginx-omnibus] | [✅][nginx-charts] | [⚙][nginx-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | CE & EE | -| [Unicorn (GitLab Rails)](#unicorn) | Handles requests for the web interface and API | [✅][unicorn-omnibus] | [✅][unicorn-charts] | [✅][unicorn-charts] | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#unicorn) | CE & EE | -| [Sidekiq](#sidekiq) | Background jobs processor | [✅][sidekiq-omnibus] | [✅][sidekiq-charts] | [✅](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/index.html) | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#sidekiq) | CE & EE | -| [Gitaly](#gitaly) | Git RPC service for handling all git calls made by GitLab | [✅][gitaly-omnibus] | [✅][gitaly-charts] | [✅][gitaly-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | CE & EE | -| [GitLab Workhorse](#gitlab-workhorse) | Smart reverse proxy, handles large HTTP requests | [✅][workhorse-omnibus] | [✅][workhorse-charts] | [✅][workhorse-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | CE & EE | -| [GitLab Shell](#gitlab-shell) | Handles `git` over SSH sessions | [✅][shell-omnibus] | [✅][shell-charts] | [✅][shell-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | CE & EE | -| [GitLab Pages](#gitlab-pages) | Hosts static websites | [⚙][pages-omnibus] | [❌][pages-charts] | [❌][pages-charts] | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#gitlab-pages) | CE & EE | -| [Registry](#registry) | Container registry, allows pushing and pulling of images | [⚙][registry-omnibus] | [✅][registry-charts] | [✅][registry-charts] | [✅](https://docs.gitlab.com/ee/user/project/container_registry.html#build-and-push-images) | CE & EE | -| [Redis](#redis) | Caching service | [✅][redis-omnibus] | [✅][redis-omnibus] | [✅][redis-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | CE & EE | -| [PostgreSQL](#postgresql) | Database | [✅][postgres-omnibus] | [✅][postgres-charts] | [✅][postgres-charts] | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#postgresql) | CE & EE | -| [PgBouncer](#pgbouncer) | Database connection pooling, failover | [⚙][pgbouncer-omnibus] | [❌][pgbouncer-charts] | [❌][pgbouncer-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#database-architecture) | EE Only | -| [Consul](#consul) | Database node discovery, failover | [⚙][consul-omnibus] | [❌][consul-charts] | [❌][consul-charts] | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#consul) | EE Only | -| [GitLab self-monitoring: Prometheus](#prometheus) | Time-series database, metrics collection, and query service | [✅][prometheus-omnibus] | [✅][prometheus-charts] | [⚙][prometheus-charts] | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#prometheus) | CE & EE | -| [GitLab self-monitoring: Alertmanager](#alertmanager) | Deduplicates, groups, and routes alerts from Prometheus | [✅][alertmanager-omnibus] | [✅][alertmanager-charts] | [⚙][alertmanager-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | CE & EE | -| [GitLab self-monitoring: Grafana](#grafana) | Metrics dashboard | [⚙][grafana-omnibus] | [⤓][grafana-charts] | [⤓][grafana-charts] | [✅](https://dashboards.gitlab.com/d/RZmbBr7mk/gitlab-triage?refresh=30s) | CE & EE | -| [GitLab self-monitoring: Sentry](#sentry) | Track errors generated by the GitLab instance | [⤓][sentry-omnibus] | [❌][sentry-charts] | [❌][sentry-charts] | [✅](https://about.gitlab.com/handbook/support/workflows/services/gitlab_com/500_errors.html#searching-sentry) | CE & EE | -| [GitLab self-monitoring: Jaeger](#jaeger) | View traces generated by the GitLab instance | [❌][jaeger-omnibus] | [❌][jaeger-charts] | [❌][jaeger-charts] | [❌](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/4104) | CE & EE | -| [Redis Exporter](#redis-exporter) | Prometheus endpoint with Redis metrics | [✅][redis-exporter-omnibus] | [✅][redis-exporter-charts] | [✅][redis-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | CE & EE | -| [Postgres Exporter](#postgres-exporter) | Prometheus endpoint with PostgreSQL metrics | [✅][postgres-exporter-omnibus] | [✅][postgres-exporter-charts] | [✅][postgres-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | CE & EE | -| [PgBouncer Exporter](#pgbouncer-exporter) | Prometheus endpoint with PgBouncer metrics | [⚙][pgbouncer-exporter-omnibus] | [❌][pgbouncer-exporter-charts] | [❌][pgbouncer-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | CE & EE | -| [GitLab Monitor](#gitlab-monitor) | Generates a variety of GitLab metrics | [✅][gitlab-monitor-omnibus] | [❌][gitab-monitor-charts] | [❌][gitab-monitor-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | CE & EE | -| [Node Exporter](#node-exporter) | Prometheus endpoint with system metrics | [✅][node-exporter-omnibus] | [❌][node-exporter-charts] | [❌][node-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | CE & EE | -| [Mattermost](#mattermost) | Open-source Slack alternative | [⚙][mattermost-omnibus] | [⤓][mattermost-charts] | [⤓][mattermost-charts] | [⤓](https://docs.gitlab.com/ee/user/project/integrations/mattermost_slash_commands.html#manual-configuration), [⤓](https://docs.gitlab.com/ee/user/project/integrations/mattermost.html) | CE & EE | -| [Minio](#minio) | Object storage service | [⤓][minio-omnibus] | [✅][minio-charts] | [✅][minio-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#storage-architecture) | CE & EE | -| [Runner](#gitlab-runner) | Executes GitLab CI jobs | [⤓][runner-omnibus] | [✅][runner-charts] | [⚙][runner-charts] | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#shared-runners) | CE & EE | -| [Database Migrations](#database-migrations) | Database migrations | [✅][database-migrations-omnibus] | [✅]() | [✅][database-migrations-charts] | [✅][database-migrations-charts] | CE & EE | -| [Certificate Management](#certificate-management) | TLS Settings, Let's Encrypt | [✅][certificate-management-omnibus] | [✅][certificate-management-charts] | [⚙][certificate-management-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#secrets-management) | CE & EE | -| [GitLab Geo Node](#gitlab-geo) | Geographically distributed GitLab nodes | [⚙][geo-omnibus] | [❌][geo-charts] | [❌][geo-charts] | ✅ | EE Only | -| [LDAP Authentication](#ldap-authentication) | Authenticate users against centralized LDAP directory | [⤓][ldap-omnibus] | [⤓][ldap-charts] | [⤓][ldap-charts] | [❌](https://about.gitlab.com/pricing/#gitlab-com) | CE & EE | -| [Outbound email (SMTP)](#outbound-email) | Send email messages to users | [⤓][outbound-email-omnibus] | [⤓][outbound-email-charts] | [⤓][outbound-email-charts] | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#mail-configuration) | CE & EE | -| [Inbound email (SMTP)](#inbound-email) | Receive messages to update issues | [⤓][inbound-email-omnibus] | [⤓][inbound-email-charts] | [⤓][inbound-email-charts] | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#mail-configuration) | CE & EE | -| [ElasticSearch](#elasticsearch) | Improved search within GitLab | [⤓][elasticsearch-omnibus] | [⤓][elasticsearch-charts] | [⤓][elasticsearch-charts] | [❌](https://gitlab.com/groups/gitlab-org/-/epics/153) | EE Only | -| [Sentry integration](#sentry) | Error tracking for deployed apps | [⤓][sentry-integration] | [⤓][sentry-integration] | [⤓][sentry-integration] | [⤓][sentry-integration] | CE & EE | -| [Jaeger integration](#jaeger) | Distributed tracing for deployed apps | [⤓][jaeger-integration] | [⤓][jaeger-integration] | [⤓][jaeger-integration] | [⤓][jaeger-integration] | EE Only | -| [Kubernetes cluster apps](#kubernetes-cluster-apps) | Deploy [Helm](https://docs.helm.sh/), [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/), [Cert-Manager](https://docs.cert-manager.io/en/latest/), [Prometheus](https://prometheus.io/docs/introduction/overview/), a [Runner](https://docs.gitlab.com/runner/), [JupyterHub](http://jupyter.org/), [Knative](https://cloud.google.com/knative) to a cluster | [⤓][managed-k8s-apps] | [⤓][managed-k8s-apps] | [⤓][managed-k8s-apps] | [⤓][managed-k8s-apps] | CE & EE | +| Component | Description | [Omnibus GitLab](https://docs.gitlab.com/omnibus/) | [GitLab chart](https://docs.gitlab.com/charts/) | [Minikube Minimal](https://docs.gitlab.com/charts/development/minikube/#deploying-gitlab-with-minimal-settings) | [GitLab.com](https://gitlab.com) | [Source](../install/installation.md) | [GDK](https://gitlab.com/gitlab-org/gitlab-development-kit) | CE/EE | +| --------- | ----------- |:--------------------:|:------------------:|:-----:|:--------:|:--------:|:-------:|:-------:| +| [NGINX](#nginx) | Routes requests to appropriate components, terminates SSL | [✅][nginx-omnibus] | [✅][nginx-charts] | [⚙][nginx-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | [⤓][nginx-source] | ❌ | CE & EE | +| [Unicorn (GitLab Rails)](#unicorn) | Handles requests for the web interface and API | [✅][unicorn-omnibus] | [✅][unicorn-charts] | [✅][unicorn-charts] | [✅](../user/gitlab_com/index.md#unicorn) | [⚙][unicorn-source] | [✅][gitlab-yml] | CE & EE | +| [Sidekiq](#sidekiq) | Background jobs processor | [✅][sidekiq-omnibus] | [✅][sidekiq-charts] | [✅](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/index.html) | [✅](../user/gitlab_com/index.md#sidekiq) | [✅][gitlab-yml] | [✅][gitlab-yml] | CE & EE | +| [Gitaly](#gitaly) | Git RPC service for handling all git calls made by GitLab | [✅][gitaly-omnibus] | [✅][gitaly-charts] | [✅][gitaly-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | [⚙][gitaly-source] | ✅ | CE & EE | +| [GitLab Workhorse](#gitlab-workhorse) | Smart reverse proxy, handles large HTTP requests | [✅][workhorse-omnibus] | [✅][workhorse-charts] | [✅][workhorse-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | [⚙][workhorse-source] | ✅ | CE & EE | +| [GitLab Shell](#gitlab-shell) | Handles `git` over SSH sessions | [✅][shell-omnibus] | [✅][shell-charts] | [✅][shell-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | [⚙][shell-source] | [✅][gitlab-yml] | CE & EE | +| [GitLab Pages](#gitlab-pages) | Hosts static websites | [⚙][pages-omnibus] | [❌][pages-charts] | [❌][pages-charts] | [✅](../user/gitlab_com/index.md#gitlab-pages) | [⚙][pages-source] | [⚙][pages-gdk] | CE & EE | +| [Registry](#registry) | Container registry, allows pushing and pulling of images | [⚙][registry-omnibus] | [✅][registry-charts] | [✅][registry-charts] | [✅](../user/project/container_registry.md#build-and-push-images) | [⤓][registry-source] | [⚙][registry-gdk] | CE & EE | +| [Redis](#redis) | Caching service | [✅][redis-omnibus] | [✅][redis-omnibus] | [✅][redis-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | [⤓][redis-source] | ✅ | CE & EE | +| [PostgreSQL](#postgresql) | Database | [✅][postgres-omnibus] | [✅][postgres-charts] | [✅][postgres-charts] | [✅](../user/gitlab_com/index.md#postgresql) | [⤓][postgres-source] | ✅ | CE & EE | +| [PgBouncer](#pgbouncer) | Database connection pooling, failover | [⚙][pgbouncer-omnibus] | [❌][pgbouncer-charts] | [❌][pgbouncer-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#database-architecture) | ❌ | ❌ | EE Only | +| [Consul](#consul) | Database node discovery, failover | [⚙][consul-omnibus] | [❌][consul-charts] | [❌][consul-charts] | [✅](../user/gitlab_com/index.md#consul) | ❌ | ❌ | EE Only | +| [GitLab self-monitoring: Prometheus](#prometheus) | Time-series database, metrics collection, and query service | [✅][prometheus-omnibus] | [✅][prometheus-charts] | [⚙][prometheus-charts] | [✅](../user/gitlab_com/index.md#prometheus) | ❌ | ❌ | CE & EE | +| [GitLab self-monitoring: Alertmanager](#alertmanager) | Deduplicates, groups, and routes alerts from Prometheus | [✅][alertmanager-omnibus] | [✅][alertmanager-charts] | [⚙][alertmanager-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | +| [GitLab self-monitoring: Grafana](#grafana) | Metrics dashboard | [⚙][grafana-omnibus] | [⤓][grafana-charts] | [⤓][grafana-charts] | [✅](https://dashboards.gitlab.com/d/RZmbBr7mk/gitlab-triage?refresh=30s) | ❌ | ❌ | CE & EE | +| [GitLab self-monitoring: Sentry](#sentry) | Track errors generated by the GitLab instance | [⤓][sentry-omnibus] | [❌][sentry-charts] | [❌][sentry-charts] | [✅](https://about.gitlab.com/handbook/support/workflows/services/gitlab_com/500_errors.html#searching-sentry) | [⤓][gitlab-yml] | [⤓][gitlab-yml] | CE & EE | +| [GitLab self-monitoring: Jaeger](#jaeger) | View traces generated by the GitLab instance | [❌][jaeger-omnibus] | [❌][jaeger-charts] | [❌][jaeger-charts] | [❌](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/4104) | [⤓][jaeger-source] | [⚙][jaeger-gdk] | CE & EE | +| [Redis Exporter](#redis-exporter) | Prometheus endpoint with Redis metrics | [✅][redis-exporter-omnibus] | [✅][redis-exporter-charts] | [✅][redis-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | +| [Postgres Exporter](#postgres-exporter) | Prometheus endpoint with PostgreSQL metrics | [✅][postgres-exporter-omnibus] | [✅][postgres-exporter-charts] | [✅][postgres-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | +| [PgBouncer Exporter](#pgbouncer-exporter) | Prometheus endpoint with PgBouncer metrics | [⚙][pgbouncer-exporter-omnibus] | [❌][pgbouncer-exporter-charts] | [❌][pgbouncer-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | +| [GitLab Monitor](#gitlab-monitor) | Generates a variety of GitLab metrics | [✅][gitlab-monitor-omnibus] | [❌][gitab-monitor-charts] | [❌][gitab-monitor-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | +| [Node Exporter](#node-exporter) | Prometheus endpoint with system metrics | [✅][node-exporter-omnibus] | [❌][node-exporter-charts] | [❌][node-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | +| [Mattermost](#mattermost) | Open-source Slack alternative | [⚙][mattermost-omnibus] | [⤓][mattermost-charts] | [⤓][mattermost-charts] | [⤓](../user/project/integrations/mattermost_slash_commands.md#manual-configuration), [⤓](../user/project/integrations/mattermost.html) | ❌ | ❌ | CE & EE | +| [MinIO](#minio) | Object storage service | [⤓][minio-omnibus] | [✅][minio-charts] | [✅][minio-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#storage-architecture) | ❌ | [⚙][minio-gdk] | CE & EE | +| [Runner](#gitlab-runner) | Executes GitLab CI jobs | [⤓][runner-omnibus] | [✅][runner-charts] | [⚙][runner-charts] | [✅](../user/gitlab_com/index.md#shared-runners) | [⚙][runner-source] | [⚙][runner-gdk] | CE & EE | +| [Database Migrations](#database-migrations) | Database migrations | [✅][database-migrations-omnibus] | [✅][database-migrations-charts] | [✅][database-migrations-charts] | ✅ | [⚙][database-migrations-source] | ✅ | CE & EE | +| [Certificate Management](#certificate-management) | TLS Settings, Let's Encrypt | [✅][certificate-management-omnibus] | [✅][certificate-management-charts] | [⚙][certificate-management-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#secrets-management) | [⚙][certificate-management-source] | [⚙][certificate-management-gdk] | CE & EE | +| [GitLab Geo Node](#gitlab-geo) | Geographically distributed GitLab nodes | [⚙][geo-omnibus] | [❌][geo-charts] | [❌][geo-charts] | ✅ | ❌ | [⚙][geo-gdk] | EE Only | +| [LDAP Authentication](#ldap-authentication) | Authenticate users against centralized LDAP directory | [⤓][ldap-omnibus] | [⤓][ldap-charts] | [⤓][ldap-charts] | [❌](https://about.gitlab.com/pricing/#gitlab-com) | [⤓][gitlab-yml] | [⤓][ldap-gdk] | CE & EE | +| [Outbound email (SMTP)](#outbound-email) | Send email messages to users | [⤓][outbound-email-omnibus] | [⤓][outbound-email-charts] | [⤓][outbound-email-charts] | [✅](../user/gitlab_com/index.md#mail-configuration) | [⤓][gitlab-yml] | [⤓][gitlab-yml] | CE & EE | +| [Inbound email (SMTP)](#inbound-email) | Receive messages to update issues | [⤓][inbound-email-omnibus] | [⤓][inbound-email-charts] | [⤓][inbound-email-charts] | [✅](../user/gitlab_com/index.md#mail-configuration) | [⤓][gitlab-yml] | [⤓][gitlab-yml] | CE & EE | +| [ElasticSearch](#elasticsearch) | Improved search within GitLab | [⤓][elasticsearch-omnibus] | [⤓][elasticsearch-charts] | [⤓][elasticsearch-charts] | [❌](https://gitlab.com/groups/gitlab-org/-/epics/153) | [⤓][elasticsearch-source] | [⤓][elasticsearch-gdk] | EE Only | +| [Sentry integration](#sentry) | Error tracking for deployed apps | [⤓][sentry-integration] | [⤓][sentry-integration] | [⤓][sentry-integration] | [⤓][sentry-integration] | [⤓][sentry-integration] | [⤓][sentry-integration] | CE & EE | +| [Jaeger integration](#jaeger) | Distributed tracing for deployed apps | [⤓][jaeger-integration] | [⤓][jaeger-integration] | [⤓][jaeger-integration] | [⤓][jaeger-integration] | [⤓][jaeger-integration] | [⤓][jaeger-integration] | EE Only | +| [GitLab Managed Apps](#gitlab-managed-apps) | Deploy [Helm](https://docs.helm.sh/), [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/), [Cert-Manager](https://docs.cert-manager.io/en/latest/), [Prometheus](https://prometheus.io/docs/introduction/overview/), a [Runner](https://docs.gitlab.com/runner/), [JupyterHub](http://jupyter.org/), [Knative](https://cloud.google.com/knative) to a cluster | [⤓][managed-k8s-apps] | [⤓][managed-k8s-apps] | [⤓][managed-k8s-apps] | [⤓][managed-k8s-apps] | [⤓][managed-k8s-apps] | [⤓][managed-k8s-apps] | CE & EE | ### Component details @@ -164,13 +164,14 @@ GitLab can be considered to have two layers from a process perspective: - [Project page](https://github.com/prometheus/alertmanager/blob/master/README.md) - Configuration: [Omnibus][alertmanager-omnibus], [Charts][alertmanager-charts] - Layer: Monitoring +- Process: `alertmanager` [Alert manager](https://prometheus.io/docs/alerting/alertmanager/) is a tool provided by Prometheus that _"handles alerts sent by client applications such as the Prometheus server. It takes care of deduplicating, grouping, and routing them to the correct receiver integration such as email, PagerDuty, or OpsGenie. It also takes care of silencing and inhibition of alerts."_ You can read more in [issue gitlab-ce#45740](https://gitlab.com/gitlab-org/gitlab-ce/issues/45740) about what we will be alerting on. #### Certificate management - Project page: [Omnibus](https://github.com/certbot/certbot/blob/master/README.rst), [Charts](https://github.com/jetstack/cert-manager/blob/master/README.md) -- Configuration: [Omnibus][certificate-management-omnibus], [Charts][certificate-management-charts] +- Configuration: [Omnibus][certificate-management-omnibus], [Charts][certificate-management-charts], [Source][certificate-management-source], [GDK][certificate-management-gdk] - Layer: Core Service (Processor) #### Consul @@ -183,13 +184,13 @@ Consul is a tool for service discovery and configuration. Consul is distributed, #### Database migrations -- Configuration: [Omnibus][registry-omnibus], [Charts][registry-charts] +- Configuration: [Omnibus][registry-omnibus], [Charts][registry-charts], [Source][database-migrations-source] - Layer: Core Service (Data) #### Elasticsearch - [Project page](https://github.com/elastic/elasticsearch/blob/master/README.textile) -- Configuration: [Omnibus][elasticsearch-omnibus], [Charts][elasticsearch-charts] +- Configuration: [Omnibus][elasticsearch-omnibus], [Charts][elasticsearch-charts], [Source][elasticsearch-source], [GDK][elasticsearch-gdk] - Layer: Core Service (Data) Elasticsearch is a distributed RESTful search engine built for the cloud. @@ -197,14 +198,15 @@ Elasticsearch is a distributed RESTful search engine built for the cloud. #### Gitaly - [Project page](https://gitlab.com/gitlab-org/gitaly/blob/master/README.md) -- Configuration: [Omnibus][gitaly-omnibus], [Charts][gitaly-charts] +- Configuration: [Omnibus][gitaly-omnibus], [Charts][gitaly-charts], [Source][gitaly-source] - Layer: Core Service (Data) +- Process: `gitaly` Gitaly is a service designed by GitLab to remove our need for NFS for Git storage in distributed deployments of GitLab (think GitLab.com or High Availability Deployments). As of 11.3.0, this service handles all Git level access in GitLab. You can read more about the project [in the project's readme](https://gitlab.com/gitlab-org/gitaly). #### Gitlab Geo -- Configuration: [Omnibus][geo-omnibus], [Charts][geo-charts] +- Configuration: [Omnibus][geo-omnibus], [Charts][geo-charts], [GDK][geo-gdk] - Layer: Core Service (Processor) #### Gitlab Monitor @@ -212,12 +214,13 @@ Gitaly is a service designed by GitLab to remove our need for NFS for Git storag - [Project page](https://gitlab.com/gitlab-org/gitlab-monitor) - Configuration: [Omnibus][gitlab-monitor-omnibus], [Charts][gitlab-monitor-charts] - Layer: Monitoring +- Process: `gitlab-monitor` GitLab Monitor is a process designed in house that allows us to export metrics about GitLab application internals to Prometheus. You can read more [in the project's readme](https://gitlab.com/gitlab-org/gitlab-monitor). #### Gitlab Pages -- Configuration: [Omnibus][pages-omnibus], [Charts][pages-charts] +- Configuration: [Omnibus][pages-omnibus], [Charts][pages-charts], [Source][pages-source], [GDK][pages-gdk] - Layer: Core Service (Processor) GitLab Pages is a feature that allows you to publish static websites directly from a repository in GitLab. @@ -227,7 +230,7 @@ You can use it either for personal or business websites, such as portfolios, doc #### Gitlab Runner - [Project page](https://gitlab.com/gitlab-org/gitlab-runner/blob/master/README.md) -- Configuration: [Omnibus][runner-omnibus], [Charts][runner-charts] +- Configuration: [Omnibus][runner-omnibus], [Charts][runner-charts], [Source][runner-source], [GDK][runner-gdk] - Layer: Core Service (Processor) GitLab Runner runs tests and sends the results to GitLab. @@ -237,7 +240,7 @@ GitLab CI is the open-source continuous integration service included with GitLab #### Gitlab Shell - [Project page](https://gitlab.com/gitlab-org/gitlab-shell/blob/master/README.md) -- Configuration: [Omnibus][shell-omnibus], [Charts][shell-charts] +- Configuration: [Omnibus][shell-omnibus], [Charts][shell-charts], [Source][shell-source], [GDK][gitlab-yml] - Layer: Core Service (Processor) [GitLab Shell](https://gitlab.com/gitlab-org/gitlab-shell) is a program designed at GitLab to handle ssh-based `git` sessions, and modifies the list of authorized keys. GitLab Shell is not a Unix shell nor a replacement for Bash or Zsh. @@ -245,8 +248,9 @@ GitLab CI is the open-source continuous integration service included with GitLab #### Gitlab Workhorse - [Project page](https://gitlab.com/gitlab-org/gitlab-workhorse/blob/master/README.md) -- Configuration: [Omnibus][gitlab-workhorse-omnibus], [Charts][gitlab-workhorse-charts] +- Configuration: [Omnibus][gitlab-workhorse-omnibus], [Charts][gitlab-workhorse-charts], [Source][workhorse-source] - Layer: Core Service (Processor) +- Process: `gitlab-workhorse` [GitLab Workhorse](https://gitlab.com/gitlab-org/gitlab-workhorse) is a program designed at GitLab to help alleviate pressure from Unicorn. You can read more about the [historical reasons for developing](https://about.gitlab.com/2016/04/12/a-brief-history-of-gitlab-workhorse/). It's designed to act as a smart reverse proxy to help speed up GitLab as a whole. @@ -261,7 +265,7 @@ Grafana is an open source, feature rich metrics dashboard and graph editor for G #### Jaeger - [Project page](https://github.com/jaegertracing/jaeger/blob/master/README.md) -- Configuration: [Omnibus][jaeger-omnibus], [Charts][jaeger-charts] +- Configuration: [Omnibus][jaeger-omnibus], [Charts][jaeger-charts], [Source][jaeger-source], [GDK][jaeger-gdk] - Layer: Monitoring Jaeger, inspired by Dapper and OpenZipkin, is a distributed tracing system. It can be used for monitoring microservices-based distributed systems. @@ -271,6 +275,7 @@ Jaeger, inspired by Dapper and OpenZipkin, is a distributed tracing system. It c - [Project page](https://github.com/logrotate/logrotate/blob/master/README.md) - Configuration: [Omnibus](https://docs.gitlab.com/omnibus/settings/logs.html#logrotate) - Layer: Core Service +- Process: `logrotate` GitLab is comprised of a large number of services that all log. We started bundling our own logrotate as of 7.4 to make sure we were logging responsibly. This is just a packaged version of the common open source offering. @@ -285,7 +290,7 @@ Mattermost is an open source, private cloud, Slack-alternative from https://matt #### MinIO - [Project page](https://github.com/minio/minio/blob/master/README.md) -- Configuration: [Omnibus][minio-omnibus], [Charts][minio-charts] +- Configuration: [Omnibus][minio-omnibus], [Charts][minio-charts], [GDK][minio-gdk] - Layer: Core Service (Data) MinIO is an object storage server released under Apache License v2.0. It is compatible with Amazon S3 cloud storage service. It is best suited for storing unstructured data such as photos, videos, log files, backups and container / VM images. Size of an object can range from a few KBs to a maximum of 5TB. @@ -293,8 +298,9 @@ MinIO is an object storage server released under Apache License v2.0. It is comp #### NGINX - Project page: [Omnibus](https://github.com/nginx/nginx), [Charts](https://github.com/kubernetes/ingress-nginx/blob/master/README.md) -- Configuration: [Omnibus][nginx-omnibus], [Charts][nginx-charts] +- Configuration: [Omnibus][nginx-omnibus], [Charts][nginx-charts], [Source][nginx-source] - Layer: Core Service (Processor) +- Process: `nginx` Nginx as an ingress port for all HTTP requests and routes them to the approriate sub-systems within GitLab. We are bundling an unmodified version of the popular open source webserver. @@ -303,6 +309,7 @@ Nginx as an ingress port for all HTTP requests and routes them to the approriate - [Project page](https://github.com/prometheus/node_exporter/blob/master/README.md) - Configuration: [Omnibus][node-exporter-omnibus], [Charts][node-exporter-charts] - Layer: Monitoring +- Process: `node-exporter` [Node Exporter](https://github.com/prometheus/node_exporter) is a Prometheus tool that gives us metrics on the underlying machine (think CPU/Disk/Load). It's just a packaged version of the common open source offering from the Prometheus project. @@ -325,8 +332,9 @@ Prometheus exporter for PgBouncer. Exports metrics at 9127/metrics. #### Postgresql - [Project page](https://github.com/postgres/postgres/blob/master/README) -- Configuration: [Omnibus][postgres-omnibus], [Charts][postgres-charts] +- Configuration: [Omnibus][postgres-omnibus], [Charts][postgres-charts], [Source][postgres-source] - Layer: Core Service (Data) +- Process: `postgresql` GitLab packages the popular Database to provide storage for Application meta data and user information. @@ -335,6 +343,7 @@ GitLab packages the popular Database to provide storage for Application meta dat - [Project page](https://github.com/wrouesnel/postgres_exporter/blob/master/README.md) - Configuration: [Omnibus][postgres-exporter-omnibus], [Charts][postgres-exporter-charts] - Layer: Monitoring +- Process: `postgres-exporter` [Postgres-exporter](https://github.com/wrouesnel/postgres_exporter) is the community provided Prometheus exporter that will deliver data about Postgres to Prometheus for use in Grafana Dashboards. @@ -343,14 +352,16 @@ GitLab packages the popular Database to provide storage for Application meta dat - [Project page](https://github.com/prometheus/prometheus/blob/master/README.md) - Configuration: [Omnibus][prometheus-omnibus], [Charts][prometheus-charts] - Layer: Monitoring +- Process: `prometheus` Prometheus is a time-series tool that helps GitLab administrators expose metrics about the individual processes used to provide GitLab the service. #### Redis - [Project page](https://github.com/antirez/redis/blob/unstable/README.md) -- Configuration: [Omnibus][redis-omnibus], [Charts][redis-charts] +- Configuration: [Omnibus][redis-omnibus], [Charts][redis-charts], [Source][redis-source] - Layer: Core Service (Data) +- Process: `redis` Redis is packaged to provide a place to store: @@ -363,13 +374,14 @@ Redis is packaged to provide a place to store: - [Project page](https://github.com/oliver006/redis_exporter/blob/master/README.md) - Configuration: [Omnibus][redis-exporter-omnibus], [Charts][redis-exporter-charts] - Layer: Monitoring +- Process: `redis-exporter` [Redis Exporter](https://github.com/oliver006/redis_exporter) is designed to give specific metrics about the Redis process to Prometheus so that we can graph these metrics in Grafana. #### Registry - [Project page](https://github.com/docker/distribution/blob/master/README.md) -- Configuration: [Omnibus][registry-omnibus], [Charts][registry-charts] +- Configuration: [Omnibus][registry-omnibus], [Charts][registry-charts], [Source][registry-source], [GDK][registry-gdk] - Layer: Core Service (Processor) The registry is what users use to store their own Docker images. The bundled @@ -385,7 +397,7 @@ An external registry can also be configured to use GitLab as an auth endpoint. #### Sentry - [Project page](https://github.com/getsentry/sentry/blob/master/README.rst) -- Configuration: [Omnibus][sentry-omnibus], [Charts][sentry-charts] +- Configuration: [Omnibus][sentry-omnibus], [Charts][sentry-charts], [Source][gitlab-yml], [GDK][gitlab-yml] - Layer: Monitoring Sentry fundamentally is a service that helps you monitor and fix crashes in realtime. The server is in Python, but it contains a full API for sending events from any language, in any application. @@ -393,40 +405,42 @@ Sentry fundamentally is a service that helps you monitor and fix crashes in real #### Sidekiq - [Project page](https://github.com/mperham/sidekiq/blob/master/README.md) -- Configuration: [Omnibus][sidekiq-omnibus], [Charts][sidekiq-charts] +- Configuration: [Omnibus][sidekiq-omnibus], [Charts][sidekiq-charts], [Source][gitlab-yml], [GDK][gitlab-yml] - Layer: Core Service (Processor) +- Process: `sidekiq` Sidekiq is a Ruby background job processor that pulls jobs from the redis queue and processes them. Background jobs allow GitLab to provide a faster request/response cycle by moving work into the background. #### Unicorn - [Project page](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/README.md) -- Configuration: [Omnibus][unicorn-omnibus], [Charts][unicorn-charts] +- Configuration: [Omnibus][unicorn-omnibus], [Charts][unicorn-charts], [Source][unicorn-source], [GDK][gitlab-yml] - Layer: Core Service (Processor) +- Process: `unicorn` [Unicorn](https://bogomips.org/unicorn/) is a Ruby application server that is used to run the core Rails Application that provides the user facing features in GitLab. Often process output you will see this as `bundle` or `config.ru` depending on the GitLab version. #### LDAP Authentication -- Configuration: [Omnibus][ldap-omnibus], [Charts][ldap-charts] +- Configuration: [Omnibus][ldap-omnibus], [Charts][ldap-charts], [Source][gitlab-yml], [GDK][ldap-gdk] - Layer: Core Service (Processor) #### Outbound Email -- Configuration: [Omnibus][outbound-email-omnibus], [Charts][outbound-email-charts] +- Configuration: [Omnibus][outbound-email-omnibus], [Charts][outbound-email-charts], [Source][gitlab-yml], [GDK][gitlab-yml] - Layer: Core Service (Processor) #### Inbound Email -- Configuration: [Omnibus][inbound-email-omnibus], [Charts][inbound-email-charts] +- Configuration: [Omnibus][inbound-email-omnibus], [Charts][inbound-email-charts], [Source][gitlab-yml], [GDK][gitlab-yml] - Layer: Core Service (Processor) -#### Kubernetes Cluster Apps +#### GitLab Managed Apps -- Configuration: [Omnibus][managed-k8s-apps], [Charts][managed-k8s-apps] +- Configuration: [Omnibus][managed-k8s-apps], [Charts][managed-k8s-apps], [Source][managed-k8s-apps], [GDK][managed-k8s-apps] - Layer: Core Service (Processor) -GitLab provides [GitLab Managed Apps](https://docs.gitlab.com/ee/user/project/clusters/#installing-applications), a one-click install for various applications which can be added directly to your configured cluster. These applications are needed for Review Apps and deployments when using Auto DevOps. You can install them after you create a cluster. +GitLab provides [GitLab Managed Apps](../user/project/clusters/index.md#installing-applications), a one-click install for various applications which can be added directly to your configured cluster. These applications are needed for Review Apps and deployments when using Auto DevOps. You can install them after you create a cluster. ## GitLab by Request Type @@ -595,68 +609,92 @@ We've also detailed [our architecture of GitLab.com](https://about.gitlab.com/ha [alertmanager-omnibus]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template [alertmanager-charts]: https://github.com/helm/charts/tree/master/stable/prometheus -[nginx-omnibus]: https://docs.gitlab.com/omnibus/settings/nginx.html -[nginx-charts]: https://docs.gitlab.com/charts/charts/nginx/index.html +[nginx-omnibus]: https://docs.gitlab.com/omnibus/settings/ +[nginx-charts]: https://docs.gitlab.com/charts/charts/nginx/ +[nginx-source]: ../install/installation.md#9-nginx [unicorn-omnibus]: https://docs.gitlab.com/omnibus/settings/unicorn.html -[unicorn-charts]: https://docs.gitlab.com/charts/charts/gitlab/unicorn/index.html +[unicorn-charts]: https://docs.gitlab.com/charts/charts/gitlab/unicorn/ +[unicorn-source]: ../install/installation.md#configure-it +[gitlab-yml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/gitlab.yml.example [sidekiq-omnibus]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template -[sidekiq-charts]: https://docs.gitlab.com/charts/charts/gitlab/sidekiq/index.html -[gitaly-omnibus]: https://docs.gitlab.com/ee/administration/gitaly/ -[gitaly-charts]: https://docs.gitlab.com/charts/charts/gitlab/gitaly/index.html +[sidekiq-charts]: https://docs.gitlab.com/charts/charts/gitlab/sidekiq/ +[gitaly-omnibus]: ../administration/gitaly/index.md +[gitaly-charts]: https://docs.gitlab.com/charts/charts/gitlab/gitaly/ +[gitaly-source]: ../install/installation.md#install-gitaly [workhorse-omnibus]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template -[workhorse-charts]: https://docs.gitlab.com/charts/charts/gitlab/unicorn/index.html +[workhorse-charts]: https://docs.gitlab.com/charts/charts/gitlab/unicorn/ +[workhorse-source]: ../install/installation.md#install-gitlab-workhorse [shell-omnibus]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template -[shell-charts]: https://docs.gitlab.com/charts/charts/gitlab/gitlab-shell/index.html -[pages-omnibus]: https://docs.gitlab.com/ee/administration/pages/ +[shell-charts]: https://docs.gitlab.com/charts/charts/gitlab/gitlab-shell/ +[shell-source]: ../install/installation.md#install-gitlab-shell +[pages-omnibus]: ../administration/pages/index.md [pages-charts]: https://gitlab.com/charts/gitlab/issues/37 -[registry-omnibus]: https://docs.gitlab.com/ee/administration/container_registry.html#container-registry-domain-configuration -[registry-charts]: https://docs.gitlab.com/charts/charts/registry/index.html +[pages-source]: ../install/installation.md#install-gitlab-pages +[pages-gdk]: https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/pages.md +[registry-omnibus]: ../administration/container_registry.md#container-registry-domain-configuration +[registry-charts]: https://docs.gitlab.com/charts/charts/registry/ +[registry-source]: ../administration/container_registry.md#enable-the-container-registry +[registry-gdk]: https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/registry.md [redis-omnibus]: https://docs.gitlab.com/omnibus/settings/redis.html -[redis-charts]: https://docs.gitlab.com/charts/charts/redis/index.html +[redis-charts]: https://docs.gitlab.com/charts/charts/redis/ +[redis-source]: ../install/installation.md#7-redis [postgres-omnibus]: https://docs.gitlab.com/omnibus/settings/database.html [postgres-charts]: https://github.com/helm/charts/tree/master/stable/postgresql -[pgbouncer-omnibus]: https://docs.gitlab.com/ee/administration/high_availability/pgbouncer.html +[postgres-source]: ../install/installation.md#6-database +[pgbouncer-omnibus]: ../administration/high_availability/pgbouncer.md [pgbouncer-charts]: https://docs.gitlab.com/charts/installation/deployment.html#postgresql -[consul-omnibus]: https://docs.gitlab.com/ee/administration/high_availability/consul.html +[consul-omnibus]: ../administration/high_availability/consul.md [consul-charts]: https://docs.gitlab.com/charts/installation/deployment.html#postgresql -[prometheus-omnibus]: https://docs.gitlab.com/ee/administration/monitoring/prometheus/ +[prometheus-omnibus]: ../administration/monitoring/prometheus/index.md [prometheus-charts]: https://github.com/helm/charts/tree/master/stable/prometheus -[grafana-omnibus]: https://docs.gitlab.com/ee/administration/monitoring/performance/grafana_configuration.html +[grafana-omnibus]: ../administration/monitoring/performance/grafana_configuration.md [grafana-charts]: https://github.com/helm/charts/tree/master/stable/grafana [sentry-omnibus]: https://docs.gitlab.com/omnibus/settings/configuration.html#error-reporting-and-logging-with-sentry [sentry-charts]: https://gitlab.com/charts/gitlab/issues/1319 [jaeger-omnibus]: https://gitlab.com/gitlab-org/omnibus-gitlab/issues/4104 [jaeger-charts]: https://gitlab.com/charts/gitlab/issues/1320 -[redis-exporter-omnibus]: https://docs.gitlab.com/ee/administration/monitoring/prometheus/redis_exporter.html -[redis-exporter-charts]: https://docs.gitlab.com/charts/charts/redis/index.html -[postgres-exporter-omnibus]: https://docs.gitlab.com/ee/administration/monitoring/prometheus/postgres_exporter.html +[jaeger-source]: ../development/distributed_tracing.md#enabling-distributed-tracing +[jaeger-gdk]: ../development/distributed_tracing.html#using-jaeger-in-the-gitlab-development-kit +[redis-exporter-omnibus]: ../administration/monitoring/prometheus/redis_exporter.md +[redis-exporter-charts]: https://docs.gitlab.com/charts/charts/redis/ +[postgres-exporter-omnibus]: ../administration/monitoring/prometheus/postgres_exporter.md [postgres-exporter-charts]: https://github.com/helm/charts/tree/master/stable/postgresql -[pgbouncer-exporter-omnibus]: https://docs.gitlab.com/ee/administration/monitoring/prometheus/pgbouncer_exporter.html +[pgbouncer-exporter-omnibus]: ../administration/monitoring/prometheus/pgbouncer_exporter.md [pgbouncer-exporter-charts]: https://docs.gitlab.com/charts/installation/deployment.html#postgresql -[gitlab-monitor-omnibus]: https://docs.gitlab.com/ee/administration/monitoring/prometheus/gitlab_monitor_exporter.html +[gitlab-monitor-omnibus]: ../administration/monitoring/prometheus/gitlab_monitor_exporter.md [gitab-monitor-charts]: https://gitlab.com/charts/gitlab/issues/319 -[node-exporter-omnibus]: https://docs.gitlab.com/ee/administration/monitoring/prometheus/node_exporter.html +[node-exporter-omnibus]: ../administration/monitoring/prometheus/node_exporter.md [node-exporter-charts]: https://gitlab.com/charts/gitlab/issues/1332 [mattermost-omnibus]: https://docs.gitlab.com/omnibus/gitlab-mattermost/ [mattermost-charts]: https://docs.mattermost.com/install/install-mmte-helm-gitlab-helm.html [minio-omnibus]: https://min.io/download -[minio-charts]: https://docs.gitlab.com/charts/charts/minio/index.html +[minio-charts]: https://docs.gitlab.com/charts/charts/minio/ +[minio-gdk]: https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/object_storage.md [runner-omnibus]: https://docs.gitlab.com/runner/ [runner-charts]: https://docs.gitlab.com/runner/install/kubernetes.html +[runner-source]: https://docs.gitlab.com/runner/ +[runner-gdk]: https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/runner.md [database-migrations-omnibus]: https://docs.gitlab.com/omnibus/settings/database.html#disabling-automatic-database-migration -[database-migrations-charts]: https://docs.gitlab.com/charts/charts/gitlab/migrations/index.html +[database-migrations-charts]: https://docs.gitlab.com/charts/charts/gitlab/migrations/ +[database-migrations-source]: ../update/upgrading_from_source.md#14-install-libs-migrations-etc [certificate-management-omnibus]: https://docs.gitlab.com/omnibus/settings/ssl.html [certificate-management-charts]: https://docs.gitlab.com/charts/installation/tls.html -[geo-omnibus]: https://docs.gitlab.com/ee/administration/geo/replication/index.html#setup-instructions +[certificate-management-source]: ../install/installation.md#using-https +[certificate-management-gdk]: https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/https.md +[geo-omnibus]: ../administration/geo/replication/index.md#setup-instructions [geo-charts]: https://gitlab.com/charts/gitlab/issues/8 -[ldap-omnibus]: https://docs.gitlab.com/ee/administration/auth/ldap.html +[geo-gdk]: https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/geo.md +[ldap-omnibus]: ../administration/auth/ldap.md [ldap-charts]: https://docs.gitlab.com/charts/charts/globals.html#ldap +[ldap-gdk]: https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/ldap.md [outbound-email-omnibus]: https://docs.gitlab.com/omnibus/settings/smtp.html [outbound-email-charts]: https://docs.gitlab.com/charts/installation/command-line-options.html#outgoing-email-configuration -[inbound-email-omnibus]: https://docs.gitlab.com/ee/administration/incoming_email.html +[inbound-email-omnibus]: ../administration/incoming_email.md [inbound-email-charts]: https://docs.gitlab.com/charts/installation/command-line-options.html#incoming-email-configuration -[elasticsearch-omnibus]: https://docs.gitlab.com/ee/integration/elasticsearch.html -[elasticsearch-charts]: https://docs.gitlab.com/ee/integration/elasticsearch.html -[sentry-integration]: https://docs.gitlab.com/ee/user/project/operations/error_tracking.html -[jaeger-integration]: https://docs.gitlab.com/ee/user/project/operations/tracing.html -[managed-k8s-apps]: https://docs.gitlab.com/ee/user/project/clusters/#installing-applications +[elasticsearch-omnibus]: ../integration/elasticsearch.md +[elasticsearch-charts]: ../integration/elasticsearch.md +[elasticsearch-source]: ../integration/elasticsearch.md +[elasticsearch-gdk]: https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/elasticsearch.md +[sentry-integration]: ../user/project/operations/error_tracking.md +[jaeger-integration]: ../user/project/operations/tracing.md +[managed-k8s-apps]: ../user/project/clusters/index.md#installing-applications diff --git a/doc/development/changelog.md b/doc/development/changelog.md index 273a7fceaf5..45b3d5a23a1 100644 --- a/doc/development/changelog.md +++ b/doc/development/changelog.md @@ -35,6 +35,7 @@ the `author` field. GitLab team members **should not**. - Any user-facing change **should** have a changelog entry. Example: "GitLab now uses system fonts for all text." +- Any change behind a feature flag **should not** have a changelog entry. The entry should be added [in the merge request removing the feature flags](https://docs.gitlab.com/ee/development/feature_flags.html#developing-with-feature-flags). - A fix for a regression introduced and then fixed in the same release (i.e., fixing a bug introduced during a monthly release candidate) **should not** have a changelog entry. diff --git a/doc/development/chatops_on_gitlabcom.md b/doc/development/chatops_on_gitlabcom.md new file mode 100644 index 00000000000..c63ec53414c --- /dev/null +++ b/doc/development/chatops_on_gitlabcom.md @@ -0,0 +1,21 @@ +# Chatops on GitLab.com + +Chatops on GitLab.com allows GitLabbers to run various automation tasks on GitLab.com using Slack. + +## Requesting access + +GitLabbers may need access to Chatops on GitLab.com for administration tasks such as: + +- Configuring feature flags on staging. +- Running `EXPLAIN` queries against the GitLab.com production replica. + +To request access to Chatops on GitLab.com: + +1. Log into <https://ops.gitlab.net/users/sign_in> using the same username as for GitLab.com. +1. Ask [anyone in the `chatops` project](https://gitlab.com/gitlab-com/chatops/project_members) to add you by running `/chatops run member add <username> gitlab-com/chatops --ops`. + +## See also + + - [Chatops Usage](https://docs.gitlab.com/ee/ci/chatops/README.html) + - [Understanding EXPLAIN plans](understanding_explain_plans.md) + - [Feature Groups](feature_flags.md#feature-groups) diff --git a/doc/development/code_review.md b/doc/development/code_review.md index c4e5995714d..29e2aa1a581 100644 --- a/doc/development/code_review.md +++ b/doc/development/code_review.md @@ -27,7 +27,7 @@ Depending on the areas your merge request touches, it must be **approved** by on or more [maintainers](https://about.gitlab.com/handbook/engineering/workflow/code-review/#maintainer): For approvals, we use the approval functionality found in the merge request -widget. Reviewers can add their approval by [approving additionally](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html#adding-or-removing-an-approval). +widget. Reviewers can add their approval by [approving additionally](../user/project/merge_requests/merge_request_approvals.md#adding-or-removing-an-approval). Getting your merge request **merged** also requires a maintainer. If it requires more than one approval, the last maintainer to review and approve it will also merge it. @@ -43,7 +43,7 @@ It picks reviewers and maintainers from the list at the [engineering projects](https://about.gitlab.com/handbook/engineering/projects/) page, with these behaviours: -1. It will not pick people whose [GitLab status](../user/profile/#current-status) +1. It will not pick people whose [GitLab status](../user/profile/index.md#current-status) contains the string 'OOO'. 2. [Trainee maintainers](https://about.gitlab.com/handbook/engineering/workflow/code-review/#trainee-maintainer) are three times as likely to be picked as other reviewers. @@ -152,7 +152,7 @@ required approvers. Maintainers must check before merging if the merge request is introducing new vulnerabilities, by inspecting the list in the Merge Request [Security -Widget](https://docs.gitlab.com/ee/user/project/merge_requests/#security-reports-ultimate). +Widget](../user/project/merge_requests/index.md#security-reports-ultimate). When in doubt, a [Security Engineer][team] can be involved. The list of detected vulnerabilities must be either empty or containing: @@ -286,7 +286,7 @@ experience, refactors the existing code). Then: author has already set this option or if the merge request clearly contains a messy commit history that is intended to be squashed. -[squash-and-merge]: https://docs.gitlab.com/ee/user/project/merge_requests/squash_and_merge.html#squash-and-merge +[squash-and-merge]: ../user/project/merge_requests/squash_and_merge.md#squash-and-merge ### The right balance @@ -319,7 +319,7 @@ reviewee. GitLab is used in a lot of places. Many users use our [Omnibus packages](https://about.gitlab.com/installation/), but some use the [Docker images](https://docs.gitlab.com/omnibus/docker/), some are -[installed from source](https://docs.gitlab.com/ce/install/installation.html), +[installed from source](../install/installation.md), and there are other installation methods available. GitLab.com itself is a large Enterprise Edition instance. This has some implications: diff --git a/doc/development/contributing/index.md b/doc/development/contributing/index.md index 8b1d014e101..59cf5014da4 100644 --- a/doc/development/contributing/index.md +++ b/doc/development/contributing/index.md @@ -87,7 +87,7 @@ Sometimes style guides will be followed but the code will lack structural integr GitLab will do its best to review community contributions as quickly as possible. Specially appointed developers review community contributions daily. You may take a look at the [team page](https://about.gitlab.com/team/) for the merge request coach who specializes in the type of code you have written and mention them in the merge request. For example, if you have written some JavaScript in your code then you should mention the frontend merge request coach. If your code has multiple disciplines you may mention multiple merge request coaches. -GitLab receives a lot of community contributions, so if your code has not been reviewed within 4 days of its initial submission feel free to re-mention the appropriate merge request coach. +GitLab receives a lot of community contributions, so if your code has not been reviewed within two days (excluding weekend and public holidays) of its initial submission feel free to re-mention the appropriate merge request coach. When submitting code to GitLab, you may feel that your contribution requires the aid of an external library. If your code includes an external library please provide a link to the library, as well as reasons for including it. diff --git a/doc/development/contributing/issue_workflow.md b/doc/development/contributing/issue_workflow.md index 0e1ab8663ed..e3a1dc711fd 100644 --- a/doc/development/contributing/issue_workflow.md +++ b/doc/development/contributing/issue_workflow.md @@ -7,7 +7,7 @@ scheduling into milestones. Labelling is a task for everyone. Most issues will have labels for at least one of the following: - Type: ~feature, ~bug, ~customer, etc. -- Subject: ~wiki, ~"container registry", ~ldap, ~api, ~frontend, etc. +- Subject: ~wiki, ~"Container Registry", ~ldap, ~api, ~frontend, etc. - Team: ~Plan, ~Manage, ~Quality, etc. - Stage: ~"devops:plan", ~"devops:create", etc. - Release Scoping: ~Deliverable, ~Stretch, ~"Next Patch Release" @@ -44,7 +44,7 @@ Subject labels are labels that define what area or feature of GitLab this issue hits. They are not always necessary, but very convenient. Examples of subject labels are ~wiki, ~ldap, ~api, -~issues, ~"merge requests", ~labels, and ~"container registry". +~issues, ~"merge requests", ~labels, and ~"Container Registry". If you are an expert in a particular area, it makes it easier to find issues to work on. You can also subscribe to those labels to receive an email each time an @@ -92,20 +92,21 @@ Stage labels specify which [DevOps stage][devops-stages] the issue belongs to. The current stage labels are: -- ~"devops:manage" -- ~"devops:plan" -- ~"devops:create" -- ~"devops:verify" -- ~"devops:package" -- ~"devops:release" -- ~"devops:configure" -- ~"devops:monitor" -- ~"devops:secure" -- ~"devops:defend" -- ~"devops:enablement" - -These labels should be mutually exclusive. If an issue belongs to multiple -stages, the most relevant should be used. +- ~"devops::manage" +- ~"devops::plan" +- ~"devops::create" +- ~"devops::verify" +- ~"devops::package" +- ~"devops::release" +- ~"devops::configure" +- ~"devops::monitor" +- ~"devops::secure" +- ~"devops::defend" +- ~"devops::growth" +- ~"devops::enablement" + +These labels are [scoped labels](../../user/project/labels.md#scoped-labels-premium) +and thus are mutually exclusive. They differ from the [Team labels](#team-labels) because teams may work on issues outside their stage. @@ -121,6 +122,25 @@ The Stage labels are used to generate the [direction pages][direction-pages] aut [devops-stages]: https://about.gitlab.com/direction/#devops-stages [direction-pages]: https://about.gitlab.com/direction/ +## Group labels + +Group labels specify which [groups][structure-groups] the issue belongs to. + +Examples include: + +- ~"group::control" +- ~"group::editor" + +These labels are [scoped labels](../../user/project/labels.md#scoped-labels-premium) +and thus are mutually exclusive. + +Groups are nested beneath a particular stage, so only one stage label and one group label +can be applied to a single issue. You can find the groups listed in the +[Product Categories pages][product-categories]. + +[structure-groups]: https://about.gitlab.com/company/team/structure/#groups +[product-categories]: https://about.gitlab.com/handbook/product/categories/ + ## Release Scoping labels Release Scoping labels help us clearly communicate expectations of the work for the diff --git a/doc/development/database_debugging.md b/doc/development/database_debugging.md index f00c5ccb9e9..68d33a9d8e0 100644 --- a/doc/development/database_debugging.md +++ b/doc/development/database_debugging.md @@ -56,7 +56,7 @@ bundle exec rails db RAILS_ENV=development ### `ActiveRecord::PendingMigrationError` with Spring -When running specs with the [Spring preloader](./rake_tasks.md#speed-up-tests-rake-tasks-and-migrations), +When running specs with the [Spring preloader](rake_tasks.md#speed-up-tests-rake-tasks-and-migrations), the test database can get into a corrupted state. Trying to run the migration or dropping/resetting the test database has no effect. diff --git a/doc/development/distributed_tracing.md b/doc/development/distributed_tracing.md index 038e3de10d7..bfce7488a8d 100644 --- a/doc/development/distributed_tracing.md +++ b/doc/development/distributed_tracing.md @@ -34,7 +34,7 @@ GITLAB_TRACING=opentracing://<driver>?<param_name>=<param_value>&<param_name_2>= In this example, we have the following hypothetical values: - `driver`: the driver. [GitLab supports - `jaeger`](https://docs.gitlab.com/ee/user/project/operations/tracing.html). In future, other + `jaeger`](../user/project/operations/tracing.md). In future, other tracing implementations may also be supported. - `param_name`, `param_value`: these are driver specific configuration values. Configuration parameters for Jaeger are documented [further on in this diff --git a/doc/development/documentation/feature-change-workflow.md b/doc/development/documentation/feature-change-workflow.md index 1f68b6a6a70..ca29353ecbe 100644 --- a/doc/development/documentation/feature-change-workflow.md +++ b/doc/development/documentation/feature-change-workflow.md @@ -86,7 +86,7 @@ Everyone is encouraged to draft the requirements in the issue, but a product man do the following: - When the issue is assigned a release milestone, review and update the Documentation details. -- By the kickoff, finalizie the Documentation details. +- By the kickoff, finalize the Documentation details. ### Developer and maintainer roles @@ -117,7 +117,7 @@ Follow the process below unless otherwise agreed with the product manager and te #### Reviews and merging -All reviewers can help ensure accuracy, clarity, completeness, and adherence to the plans in the issue, as well as the [Documentation Guidelines](https://docs.gitlab.com/ee/development/documentation/) and [Style Guide](https://docs.gitlab.com/ee/development/documentation/styleguide.html). +All reviewers can help ensure accuracy, clarity, completeness, and adherence to the plans in the issue, as well as the [Documentation Guidelines](index.md) and [Style Guide](styleguide.md). - **Prior to merging**, documentation changes committed by the developer must be reviewed by: @@ -136,7 +136,7 @@ All reviewers can help ensure accuracy, clarity, completeness, and adherence to 1. **The maintainer** who is assigned to merge the MR, to verify clarity, completeness, and quality, to the best of their ability. - Upon merging, if a technical writer review has not been performed and there is not yet a linked issue for a follow-up review, the maintainer should [create an issue using the Doc Review template](https://gitlab.com/gitlab-org/gitlab-ce/issues/new?issuable_template=Doc%20Review), link it from the MR, and - mention the original MR author in the new issue. Alternatively, the mainitainer can ask the MR author to create and link this issue before the MR is merged. + mention the original MR author in the new issue. Alternatively, the maintainer can ask the MR author to create and link this issue before the MR is merged. - After merging, documentation changes are reviewed by: @@ -157,14 +157,14 @@ All reviewers can help ensure accuracy, clarity, completeness, and adherence to #### Collaboration By default, the developer will work on documentation changes independently, but -the developer, PM, or technicial writer can propose a broader collaboration for +the developer, PM, or technical writer can propose a broader collaboration for any given issue. Additionally, technical writers are available for questions at any time. #### Review -- Techncial writers provide non-blocking reviews of all documentation changes, +- Technical writers provide non-blocking reviews of all documentation changes, before or after the change is merged. However, if the docs are ready in the MR while there's time before the freeze, the technical writer's review can commence early, on request. - The technical writer will confirm that the doc is clear, grammatically correct, @@ -173,7 +173,7 @@ Additionally, technical writers are available for questions at any time. the developer and code reviewer should have already made a good-faith effort to ensure: - Clarity. - Adherence to the plans and goals in the issue. - - Location (make sure the docs are in the correct directorkes and has the correct name). + - Location (make sure the docs are in the correct directories and has the correct name). - Syntax, typos, and broken links. - Improvements to the content. - Accordance with the [Documentation Style Guide](styleguide.md), and [Structure and Template](structure.md) doc. diff --git a/doc/development/documentation/index.md b/doc/development/documentation/index.md index 4bf8401c0e8..c7fa40af930 100644 --- a/doc/development/documentation/index.md +++ b/doc/development/documentation/index.md @@ -74,9 +74,14 @@ Here are some links to get you up to speed with the current effort: After a given documentation path is aligned across CE and EE, all merge requests affecting that path must be submitted to CE, regardless of the content it has. -This means that for EE-only features which are being added only to the EE codebase, -you have to submit a separate merge request in CE that contains the docs. -This might seem like a duplicate effort, but it's for the short term. +This means that: + +* For **EE-only docs changes**, you only have to submit a CE MR. +* For **EE-only features** that touch both the code and the docs, you have to submit +an EE MR containing all changes, and a CE MR containing only the docs changes +and without a changelog entry. + +This might seem like a duplicate effort, but it's only for the short term. A list of the already aligned docs can be found in [the epic description](https://gitlab.com/groups/gitlab-org/-/epics/199#ee-specific-lines-check). @@ -466,7 +471,7 @@ If you want to know the in-depth details, here's what's really happening: The following GitLab features are used among others: - [Manual actions](../../ci/yaml/README.md#whenmanual) -- [Multi project pipelines](https://docs.gitlab.com/ee/ci/multi_project_pipeline_graphs.html) +- [Multi project pipelines](../../ci/multi_project_pipeline_graphs.md) - [Review Apps](../../ci/review_apps/index.md) - [Artifacts](../../ci/yaml/README.md#artifacts) - [Specific Runner](../../ci/runners/README.md#locking-a-specific-runner-from-being-enabled-for-other-projects) diff --git a/doc/development/documentation/structure.md b/doc/development/documentation/structure.md index 95b5fcd99a1..fe676efa94d 100644 --- a/doc/development/documentation/structure.md +++ b/doc/development/documentation/structure.md @@ -77,8 +77,8 @@ for use (e.g. variations on the main use case), but if that's not applicable, th Examples of use cases on feature pages: - CE and EE: [Issues](../../user/project/issues/index.md#use-cases) - CE and EE: [Merge Requests](../../user/project/merge_requests/index.md) -- EE-only: [Geo](https://docs.gitlab.com/ee/administration/geo/replication/index.html) -- EE-only: [Jenkins integration](https://docs.gitlab.com/ee/integration/jenkins.html) +- EE-only: [Geo](../../administration/geo/replication/index.md) +- EE-only: [Jenkins integration](../../integration/jenkins.md) ## Requirements @@ -121,8 +121,8 @@ but commented out to help encourage others to add to it in the future. --> Notes: -- (1): Apply the [tier badges](https://docs.gitlab.com/ee/development/documentation/styleguide.html#product-badges) accordingly -- (2): Apply the correct format for the [GitLab version introducing the feature](https://docs.gitlab.com/ee/development/documentation/styleguide.html#gitlab-versions-and-tiers) +- (1): Apply the [tier badges](styleguide.md#product-badges) accordingly +- (2): Apply the correct format for the [GitLab version introducing the feature](styleguide.md#gitlab-versions-and-tiers) ``` ## Help and feedback section diff --git a/doc/development/ee_features.md b/doc/development/ee_features.md index 9db28bcddf8..cca52706ddc 100644 --- a/doc/development/ee_features.md +++ b/doc/development/ee_features.md @@ -459,15 +459,6 @@ Resolving an EE template path that is relative to the CE view path will not work = render_if_exists 'projects/button' # Will render `ee/app/views/projects/_button` ``` -You should not explicitly set render options like `partial` or provide a `locals` hash. -The first argument should be a path string and the second can be a hash replacing `locals`. - -```ruby -render partial: 'projects/button', locals: { project: project } -# becomes -render_if_exists 'projects/button', project: project -``` - #### Using `render_ce` For `render` and `render_if_exists`, they search for the EE partial first, @@ -557,40 +548,56 @@ due to `prepend`, but Grape is complex internally and we couldn't easily do that, so we'll follow regular object-oriented practices that we define the interface first here. -For example, suppose we have a few more optional params for EE, given this CE -API code: +For example, suppose we have a few more optional params for EE. We can move the +params out of the `Grape::API` class to a helper module, so we can `prepend` it +before it would be used in the class. ```ruby module API - class MergeRequests < Grape::API - # EE::API::MergeRequests would override the following helpers - helpers do - params :optional_params_ee do + class Projects < Grape::API + helpers Helpers::ProjectsHelpers + end +end +``` + +Given this CE API `params`: + +```ruby +module API + module Helpers + module ProjectsHelpers + extend ActiveSupport::Concern + extend Grape::API::Helpers + + params :optional_project_params_ce do + # CE specific params go here... end - end - params :optional_params do - # CE specific params go here... + params :optional_project_params_ee do + end - use :optional_params_ee + params :optional_project_params do + use :optional_project_params_ce + use :optional_project_params_ee + end end end end -API::MergeRequests.prepend(EE::API::MergeRequests) +API::Helpers::ProjectsHelpers.prepend(EE::API::Helpers::ProjectsHelpers) ``` -And then we could override it in EE module: +We could override it in EE module: ```ruby module EE module API - module MergeRequests - extend ActiveSupport::Concern + module Helpers + module ProjectsHelpers + extend ActiveSupport::Concern - prepended do - helpers do - params :optional_params_ee do + prepended do + params :optional_project_params_ee do # EE specific params go here... end end @@ -600,9 +607,6 @@ module EE end ``` -This way, the only difference between CE and EE for that API file would be -`prepend EE::API::MergeRequests`. - #### EE helpers To make it easy for an EE module to override the CE helpers, we need to define @@ -902,7 +906,7 @@ import bundle from 'ee/protected_branches/protected_branches_bundle.js'; import bundle from 'ee_else_ce/protected_branches/protected_branches_bundle.js'; ``` -See the frontend guide [performance section](./fe_guide/performance.md) for +See the frontend guide [performance section](fe_guide/performance.md) for information on managing page-specific javascript within EE. @@ -910,7 +914,7 @@ information on managing page-specific javascript within EE. ### script tag #### Child Component only used in EE -To seperate Vue template differences we should [async import the components](https://vuejs.org/v2/guide/components-dynamic-async.html#Async-Components). +To separate Vue template differences we should [async import the components](https://vuejs.org/v2/guide/components-dynamic-async.html#Async-Components). Doing this allows for us to load the correct component in EE whilst in CE we can load a empty component that renders nothing. This code **should** @@ -933,7 +937,7 @@ export default { ``` #### For JS code that is EE only, like props, computed properties, methods, etc, we will keep the current approach - - Since we [can't async load a mixin](https://github.com/vuejs/vue-loader/issues/418#issuecomment-254032223) we will use the [`ee_else_ce`](https://docs.gitlab.com/ee/development/ee_features.html#javascript-code-in-assetsjavascripts) alias we already have for webpack. + - Since we [can't async load a mixin](https://github.com/vuejs/vue-loader/issues/418#issuecomment-254032223) we will use the [`ee_else_ce`](../development/ee_features.md#javascript-code-in-assetsjavascripts) alias we already have for webpack. - This means all the EE specific props, computed properties, methods, etc that are EE only should be in a mixin in the `ee/` folder and we need to create a CE counterpart of the mixin ##### Example: @@ -960,7 +964,7 @@ import mixin from 'ee_else_ce/path/mixin'; ### Non Vue Files For regular JS files, the approach is similar. -1. We will keep using the [`ee_else_ce`](https://docs.gitlab.com/ee/development/ee_features.html#javascript-code-in-assetsjavascripts) helper, this means that EE only code should be inside the `ee/` folder. +1. We will keep using the [`ee_else_ce`](../development/ee_features.md#javascript-code-in-assetsjavascripts) helper, this means that EE only code should be inside the `ee/` folder. 1. An EE file should be created with the EE only code, and it should extend the CE counterpart. 1. For code inside functions that can't be extended, the code should be moved into a new file and we should use `ee_else_ce` helper: diff --git a/doc/development/elasticsearch.md b/doc/development/elasticsearch.md index 0c9e7908713..8b0f4f02d19 100644 --- a/doc/development/elasticsearch.md +++ b/doc/development/elasticsearch.md @@ -2,7 +2,7 @@ This area is to maintain a compendium of useful information when working with elasticsearch. -Information on how to enable ElasticSearch and perform the initial indexing is kept in https://docs.gitlab.com/ee/integration/elasticsearch.html#enabling-elasticsearch +Information on how to enable ElasticSearch and perform the initial indexing is kept in ../integration/elasticsearch.md#enabling-elasticsearch ## Initial installation on OS X @@ -16,7 +16,7 @@ and use `docker stop elastic56` and `docker start elastic56` to stop/start it. ### Installing on the host -We currently only support Elasticsearch [5.6 to 6.x](https://docs.gitlab.com/ee/integration/elasticsearch.html#requirements) +We currently only support Elasticsearch [5.6 to 6.x](../integration/elasticsearch.md#version-requirements) Version 5.6 is available on homebrew and is the recommended version to use in order to test compatibility. @@ -43,7 +43,7 @@ this adds `gitlab-elasticsearch-indexer` to `$GOPATH/bin`, please make sure that - `gitlab:elastic:test:index_size`: Tells you how much space the current index is using, as well as how many documents are in the index. - `gitlab:elastic:test:index_size_change`: Outputs index size, reindexes, and outputs index size again. Useful when testing improvements to indexing size. -Additionally, if you need large repos or multiple forks for testing, please consider [following these instructions](https://docs.gitlab.com/ee/development/rake_tasks.html#extra-project-seed-options) +Additionally, if you need large repos or multiple forks for testing, please consider [following these instructions](rake_tasks.md#extra-project-seed-options) ## How does it work? diff --git a/doc/development/fe_guide/frontend_faq.md b/doc/development/fe_guide/frontend_faq.md index 77f064a21a9..e4225f2bc39 100644 --- a/doc/development/fe_guide/frontend_faq.md +++ b/doc/development/fe_guide/frontend_faq.md @@ -25,3 +25,17 @@ document.body.dataset.page ``` Find here the [source code setting the attribute](https://gitlab.com/gitlab-org/gitlab-ce/blob/cc5095edfce2b4d4083a4fb1cdc7c0a1898b9921/app/views/layouts/application.html.haml#L4). + +### `modal_copy_button` vs `clipboard_button` + +The `clipboard_button` uses the `copy_to_clipboard.js` behaviour, which is +initialized on page load, so if there are vue-based clipboard buttons that +don't exist at page load (such as ones in a `GlModal`), they do not have the +click handlers associated with the clipboard package. + +`modal_copy_button` was added that manages an instance of the +[`clipboard` plugin](https://www.npmjs.com/package/clipboard) specific to +the instance of that component, which means that clipboard events are +bound on mounting and destroyed when the button is, mitigating the above +issue. It also has bindings to a particular container or modal ID +available, to work with the focus trap created by our GlModal. diff --git a/doc/development/fe_guide/style_guide_js.md b/doc/development/fe_guide/style_guide_js.md index 060cd8baf7f..b50159c2b75 100644 --- a/doc/development/fe_guide/style_guide_js.md +++ b/doc/development/fe_guide/style_guide_js.md @@ -95,6 +95,7 @@ See [our current .eslintrc](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/ #### Modules, Imports, and Exports 1. Use ES module syntax to import modules + ```javascript // bad const SomeClass = require('some_class'); @@ -168,6 +169,7 @@ See [our current .eslintrc](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/ Do not use them anymore and feel free to remove them when refactoring legacy code. 1. Avoid adding to the global namespace. + ```javascript // bad window.MyClass = class { /* ... */ }; @@ -176,7 +178,8 @@ See [our current .eslintrc](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/ export default class MyClass { /* ... */ } ``` -1. Side effects are forbidden in any script which contains exports +1. Side effects are forbidden in any script which contains export + ```javascript // bad export default class MyClass { /* ... */ } @@ -449,6 +452,7 @@ Please check this [rules][eslint-plugin-vue-rules] for more documentation. #### Props 1. Props should be declared as an object + ```javascript // bad props: ['foo'] @@ -544,14 +548,24 @@ Please check this [rules][eslint-plugin-vue-rules] for more documentation. <component @click="eventHandler"/> ``` -1. Shorthand `:` is preferable over `v-bind` +2. Shorthand `:` is preferable over `v-bind` ```javascript // bad <component v-bind:class="btn"/> // good - <component :class="btsn"/> + <component :class="btn"/> + ``` + +3. Shorthand `#` is preferable over `v-slot` + + ```javascript + // bad + <template v-slot:header></template> + + // good + <template #header></template> ``` #### Closing tags diff --git a/doc/development/fe_guide/style_guide_scss.md b/doc/development/fe_guide/style_guide_scss.md index 36880dd746d..b25dce65ffe 100644 --- a/doc/development/fe_guide/style_guide_scss.md +++ b/doc/development/fe_guide/style_guide_scss.md @@ -9,20 +9,44 @@ easy to maintain, and performant for the end-user. As part of the effort for [cleaning up our CSS and moving our components into GitLab-UI](https://gitlab.com/groups/gitlab-org/-/epics/950) led by the [GitLab UI WG](https://gitlab.com/gitlab-com/www-gitlab-com/merge_requests/20623) we prefer the use of utility classes over adding new CSS. However, complex CSS can be addressed by adding component classes. -We have a few internal utility classes in [`common.scss`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/assets/stylesheets/framework/common.scss) -and we use [Bootstrap's Utility Classes](https://getbootstrap.com/docs/4.3/utilities/) +#### Where are utility classes defined? + +- [Bootstrap's Utility Classes](https://getbootstrap.com/docs/4.3/utilities/) +- [`common.scss`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/assets/stylesheets/framework/common.scss) (old) +- [`utilities.scss`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/assets/stylesheets/utilities.scss) (new) + +#### Where should I put new utility classes? New utility classes should be added to [`utilities.scss`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/assets/stylesheets/utilities.scss). Existing classes include: -**Background color**: `.bg-variant-shade` e.g. `.bg-warning-400` -**Text color**: `.text-variant-shade` e.g. `.text-success-500` +| Name | Pattern | Example | +|------|---------|---------| +| Background color | `.bg-{variant}-{shade}` | `.bg-warning-400` | +| Text color | `.text-{variant}-{shade}` | `.text-success-500` | +| Font size | `.text-{size}` | `.text-2` | + +- `{variant}` is one of 'primary', 'secondary', 'success', 'warning', 'error' +- `{shade}` is on of the shades listed on [colors](https://design.gitlab.com/foundations/colors/) +- `{size}` is a number from 1-6 from our [Type scale](https://design.gitlab.com/foundations/typography) + +#### When should I create component classes? + +We recommend a "utility-first" approach. + +1. Start with utility classes. +2. If composing utility classes into a component class removes code duplication and encapsulates a clear responsibility, do it. + +This encourages an organic growth of component classes and prevents the creation of one-off unreusable classes. Also, the kind of classes that emerge from "utility-first" tend to be design-centered (e.g. `.button`, `.alert`, `.card`) rather than domain-centered (e.g. `.security-report-widget`, `.commit-header-icon`). + +Examples of component classes that were created using "utility-first" include: -- variant is one of 'primary', 'secondary', 'success', 'warning', 'error' -- shade is on of the shades listed on [colors](https://design.gitlab.com/foundations/colors/) +- [`.circle-icon-container`](https://gitlab.com/gitlab-org/gitlab-ce/blob/579fa8b8ec7eb38d40c96521f517c9dab8c3b97a/app/assets/stylesheets/framework/icons.scss#L85) +- [`.d-flex-center`](https://gitlab.com/gitlab-org/gitlab-ce/blob/900083d89cd6af391d26ab7922b3f64fa2839bef/app/assets/stylesheets/framework/common.scss#L425) -**Font size**: `.text-size` e.g. `.text-2` +Inspiration: -- **size** is number from 1-6 from our [Type scale](https://design.gitlab.com/foundations/typography) +- https://tailwindcss.com/docs/utility-first +- https://tailwindcss.com/docs/extracting-components ### Naming diff --git a/doc/development/fe_guide/vue.md b/doc/development/fe_guide/vue.md index 437ce9abc7d..8c6a73c6824 100644 --- a/doc/development/fe_guide/vue.md +++ b/doc/development/fe_guide/vue.md @@ -234,7 +234,7 @@ One should apply to be a Vue.js expert by opening an MR when the Merge Request's - Deep understanding of Vue and Vuex reactivy - Vue and Vuex code are structured according to both official and our guidelines - Full understanding of testing a Vue and Vuex application -- Vuex code follows the [documented pattern](./vuex.md#actions-pattern-request-and-receive-namespaces) +- Vuex code follows the [documented pattern](vuex.md#actions-pattern-request-and-receive-namespaces) - Knowledge about the existing Vue and Vuex applications and existing reusable components [vue-docs]: http://vuejs.org/guide/index.html diff --git a/doc/development/feature_flags.md b/doc/development/feature_flags.md index 82b09cc0224..13f0c5cc33e 100644 --- a/doc/development/feature_flags.md +++ b/doc/development/feature_flags.md @@ -20,7 +20,7 @@ dynamic (querying the DB etc.). Once defined in `lib/feature.rb`, you will be able to activate a feature for a given feature group via the [`feature_group` param of the features API](../api/features.md#set-or-create-a-feature) -For GitLab.com, team members have access to feature flags through chatops. Only +For GitLab.com, [team members have access to feature flags through Chatops](chatops_on_gitlabcom.md). Only percentage gates are supported at this time. Setting a feature to be used 50% of the time, you should execute `/chatops run feature set my_feature_flag 50`. @@ -28,7 +28,7 @@ the time, you should execute `/chatops run feature set my_feature_flag 50`. This document only covers feature flags used in the development of GitLab itself. Feature flags in deployed user applications can be found at -[Feature Flags](https://docs.gitlab.com/ee/user/project/operations/feature_flags.html) +[Feature Flags](../user/project/operations/feature_flags.md) ## Developing with feature flags @@ -111,7 +111,7 @@ Whenever a feature flag is present, make sure to test _both_ states of the feature flag. See the -[testing guide](testing_guide/best_practices.html#feature-flags-in-tests) +[testing guide](testing_guide/best_practices.md#feature-flags-in-tests) for information and examples on how to stub feature flags in tests. ## Enabling a feature flag (in development) diff --git a/doc/development/geo.md b/doc/development/geo.md index c8e6a86eb52..6e59fab34c7 100644 --- a/doc/development/geo.md +++ b/doc/development/geo.md @@ -10,6 +10,7 @@ the diagram below and are described in more detail within this document. ## Replication layer Geo handles replication for different components: + - [Database](#database-replication): includes the entire application, except cache and jobs. - [Git repositories](#repository-replication): includes both projects and wikis. - [Uploaded blobs](#uploads-replication): includes anything from images attached on issues @@ -90,7 +91,7 @@ projects that need updating. Those projects can be: timestamp that is more recent than the `last_repository_successful_sync_at` timestamp in the `Geo::ProjectRegistry` model. - Manual: The admin can manually flag a repository to resync in the - [Geo admin panel](https://docs.gitlab.com/ee/user/admin_area/geo_nodes.html). + [Geo admin panel](../user/admin_area/geo_nodes.md). When we fail to fetch a repository on the secondary `RETRIES_BEFORE_REDOWNLOAD` times, Geo does a so-called _redownload_. It will do a clean clone @@ -209,20 +210,38 @@ bundle exec rake geo:db:migrate ### Foreign Data Wrapper -The use of [FDW](#fdw) was introduced in GitLab 10.1. +> Introduced in GitLab 10.1. + +Foreign Data Wrapper ([FDW](#fdw)) is used by the [Geo Log Cursor](#geo-log-cursor) and improves +the performance of many synchronization operations. -This is useful for the [Geo Log Cursor](#geo-log-cursor) and improves -the performance of some synchronization operations. +FDW is a PostgreSQL extension ([`postgres_fdw`](https://www.postgresql.org/docs/current/postgres-fdw.html)) that is enabled within +the Geo Tracking Database (on a **secondary** node), which allows it +to connect to the readonly database replica and perform queries and filter +data from both instances. While FDW is available in older versions of PostgreSQL, we needed to raise the minimum required version to 9.6 as this includes many performance improvements to the FDW implementation. +This persistent connection is configured as an FDW server +named `gitlab_secondary`. This configuration exists within the database's user +context only. To access the `gitlab_secondary`, GitLab needs to use the +same database user that had previously been configured. + +The Geo Tracking Database accesses the readonly database replica via FDW as a regular user, +limited by its own restrictions. The credentials are configured as a +`USER MAPPING` associated with the `SERVER` mapped previously +(`gitlab_secondary`). + +FDW configuration and credentials definition are managed automatically by the +Omnibus GitLab `gitlab-ctl reconfigure` command. + #### Refeshing the Foreign Tables -Whenever the database schema changes on the **primary** node, the -**secondary** node will need to refresh its foreign tables by running -the following: +Whenever a new Geo node is configured or the database schema changes on the +**primary** node, you must refresh the foreign tables on the **secondary** node +by running the following: ```sh bundle exec rake geo:db:refresh_foreign_tables @@ -243,6 +262,53 @@ STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod) ORDER BY a.attnum ``` +#### Accessing data from a Foreign Table + +At the SQL level, all you have to do is `SELECT` data from `gitlab_secondary.*`. + +Here's an example of how to access all projects from the Geo Tracking Database's FDW: + +```sql +SELECT * FROM gitlab_secondary.projects; +``` + +As a more real-world example, this is how you filter for unarchived projects +on the Tracking Database: + +```sql +SELECT project_registry.* + FROM project_registry + JOIN gitlab_secondary.projects + ON (project_registry.project_id = gitlab_secondary.projects.id + AND gitlab_secondary.projects.archived IS FALSE) +``` + +At the ActiveRecord level, we have additional Models that represent the +foreign tables. They must be mapped in a slightly different way, and they are read-only. + +Check the existing FDW models in `ee/app/models/geo/fdw` for reference. + +From a developer's perspective, it's no different than creating a model that +represents a Database View. + +With the examples above, you can access the projects with: + +```ruby +Geo::Fdw::Project.all +``` + +and to access the `ProjectRegistry` filtering by unarchived projects: + +```ruby +# We have to use Arel here: +project_registry_table = Geo::ProjectRegistry.arel_table +fdw_project_table = Geo::Fdw::Project.arel_table + +project_registry_table.join(fdw_project_table) + .on(project_registry_table[:project_id].eq(fdw_project_table[:id])) + .where((fdw_project_table[:archived]).eq(true)) # if you append `.to_sql` you can check generated query +``` + ## Finders Geo uses [Finders](https://gitlab.com/gitlab-org/gitlab-ee/tree/master/app/finders), @@ -299,7 +365,7 @@ basically hashes all Git refs together and stores that hash in the The **secondary** node does the same to calculate the hash of its clone, and compares the hash with the value the **primary** node calculated. If there is a mismatch, Geo will mark this as a mismatch -and the administrator can see this in the [Geo admin panel](https://docs.gitlab.com/ee/user/admin_area/geo_nodes.html). +and the administrator can see this in the [Geo admin panel](../user/admin_area/geo_nodes.md). ## Glossary diff --git a/doc/development/go_guide/index.md b/doc/development/go_guide/index.md index cf78b792ffd..4dad8815fcb 100644 --- a/doc/development/go_guide/index.md +++ b/doc/development/go_guide/index.md @@ -40,7 +40,7 @@ of possible security breaches in our code: - SQL injections Remember to run -[SAST](https://docs.gitlab.com/ee/user/application_security/sast/index) +[SAST](../../user/application_security/sast/index.md) **[ULTIMATE]** on your project (or at least the [gosec analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/gosec)), and to follow our [Security @@ -94,9 +94,9 @@ become available, you will be able to share job templates like this Dependencies should be kept to the minimum. The introduction of a new dependency should be argued in the merge request, as per our [Approval Guidelines](../code_review.md#approval-guidelines). Both [License -Management](https://docs.gitlab.com/ee/user/project/merge_requests/license_management.html) +Management](../../user/project/merge_requests/license_management.md) **[ULTIMATE]** and [Dependency -Scanning](https://docs.gitlab.com/ee/user/application_security/dependency_scanning/index) +Scanning](../../user/application_security/dependency_scanning/index.md) **[ULTIMATE]** should be activated on all projects to ensure new dependencies security status and license compatibility. diff --git a/doc/development/i18n/externalization.md b/doc/development/i18n/externalization.md index 38d56322de8..9fb8ea542d9 100644 --- a/doc/development/i18n/externalization.md +++ b/doc/development/i18n/externalization.md @@ -174,7 +174,7 @@ For example use `%{created_at}` in Ruby but `%{createdAt}` in JavaScript. # => When size == 2: 'There are 2 mice.' ``` - Avoid using `%d` or count variables in sigular strings. This allows more natural translation in some languages. + Avoid using `%d` or count variables in singular strings. This allows more natural translation in some languages. - In JavaScript: @@ -195,6 +195,7 @@ For example use `%{created_at}` in Ruby but `%{createdAt}` in JavaScript. Sometimes you need to add some context to the text that you want to translate (if the word occurs in a sentence and/or the word is ambiguous). +Namespaces should be PascalCase. - In Ruby/HAML: @@ -211,7 +212,7 @@ Sometimes you need to add some context to the text that you want to translate ``` Note: The namespace should be removed from the translation. See the [translation -guidelines for more details](./translation.md#namespaced-strings). +guidelines for more details](translation.md#namespaced-strings). ### Dates / times @@ -331,7 +332,7 @@ Errors in `locale/zh_HK/gitlab.po`: Syntax error in msgstr Syntax error in message_line There should be only whitespace until the end of line after the double quote character of a message text. - Parseing result before error: '{:msgid=>["", "You are going to remove %{project_name_with_namespace}.\\n", "Removed project CANNOT be restored!\\n", "Are you ABSOLUTELY sure?"]}' + Parsing result before error: '{:msgid=>["", "You are going to remove %{project_name_with_namespace}.\\n", "Removed project CANNOT be restored!\\n", "Are you ABSOLUTELY sure?"]}' SimplePoParser filtered backtrace: SimplePoParser::ParserError Errors in `locale/zh_TW/gitlab.po`: 1 pipeline diff --git a/doc/development/i18n/merging_translations.md b/doc/development/i18n/merging_translations.md index 85284d8c714..eadf1cd74c5 100644 --- a/doc/development/i18n/merging_translations.md +++ b/doc/development/i18n/merging_translations.md @@ -32,7 +32,7 @@ clicking `Pause sync` on the [Crowdin integration settings page](https://translate.gitlab.com/project/gitlab-ee/settings#integration). When all failures are resolved, the translations need to be double -checked once more as discussed in [confidential issue](https://docs.gitlab.com/ee/user/project/issues/confidential_issues.html) `https://gitlab.com/gitlab-org/gitlab-ce/issues/37850`. +checked once more as discussed in [confidential issue](../../user/project/issues/confidential_issues.md) `https://gitlab.com/gitlab-org/gitlab-ce/issues/37850`. ## Merging translations diff --git a/doc/development/i18n/proofreader.md b/doc/development/i18n/proofreader.md index ac04a21b37a..35c5b155594 100644 --- a/doc/development/i18n/proofreader.md +++ b/doc/development/i18n/proofreader.md @@ -27,7 +27,7 @@ are very appreciative of the work done by translators and proofreaders! - Czech - Proofreaders needed. - Danish - - Proofreaders needed. + - Saederup92 - [GitLab](https://gitlab.com/Saederup92), [Crowdin](https://crowdin.com/profile/Saederup92) - Dutch - Emily Hendle - [GitLab](https://gitlab.com/pundachan), [Crowdin](https://crowdin.com/profile/pandachan) - Esperanto @@ -56,8 +56,8 @@ are very appreciative of the work done by translators and proofreaders! - Italian - Paolo Falomo - [GitLab](https://gitlab.com/paolofalomo), [Crowdin](https://crowdin.com/profile/paolo.falomo) - Japanese - - Yamana Tokiuji - [GitLab](https://gitlab.com/tokiuji), [Crowdin](https://crowdin.com/profile/yamana) - Hiroyuki Sato - [GitLab](https://gitlab.com/hiroponz), [Crowdin](https://crowdin.com/profile/hiroponz) + - Tomo Dote - [Gitlab](https://gitlab.com/fu7mu4), [Crowdin](https://crowdin.com/profile/fu7mu4) - Korean - Chang-Ho Cha - [GitLab](https://gitlab.com/changho-cha), [Crowdin](https://crowdin.com/profile/zzazang) - Ji Hun Oh - [GitLab](https://gitlab.com/Baw-Appie), [Crowdin](https://crowdin.com/profile/BawAppie) @@ -71,6 +71,7 @@ are very appreciative of the work done by translators and proofreaders! - Maksymilian Roman - [GitLab](https://gitlab.com/villaincandle), [Crowdin](https://crowdin.com/profile/villaincandle) - Portuguese - Proofreaders needed. + - Diogo Trindade - [GitLab](https://gitlab.com/luisdiogo2071317), [Crowdin](https://crowdin.com/profile/ldiogotrindade) - Portuguese, Brazilian - Paulo George Gomes Bezerra - [GitLab](https://gitlab.com/paulobezerra), [Crowdin](https://crowdin.com/profile/paulogomes.rep) - André Gama - [GitLab](https://gitlab.com/andregamma), [Crowdin](https://crowdin.com/profile/ToeOficial) @@ -129,7 +130,7 @@ are very appreciative of the work done by translators and proofreaders! your previous translations by [GitLab team members](https://about.gitlab.com/team/) or [Core team members](https://about.gitlab.com/core-team/) who are fluent in the language or current proofreaders. - - When a request is made for the first proofreader for a lanuguage and there are no [GitLab team members](https://about.gitlab.com/team/) + - When a request is made for the first proofreader for a language and there are no [GitLab team members](https://about.gitlab.com/team/) or [Core team members](https://about.gitlab.com/core-team/) who speak the language, we will request links to previous translation work in other communities or projects. diff --git a/doc/development/i18n/translation.md b/doc/development/i18n/translation.md index 99c0fe6db1d..62be3786549 100644 --- a/doc/development/i18n/translation.md +++ b/doc/development/i18n/translation.md @@ -89,9 +89,7 @@ To propose additions to the glossary please ### Inclusive language in French -In French, we should follow the guidelines from [ecriture-inclusive.fr]. For -instance: +In French, the "écriture inclusive" is now over (see on [Legifrance](https://www.legifrance.gouv.fr/affichTexte.do?cidTexte=JORFTEXT000036068906&categorieLien=id)). +So, to include both genders, write “Utilisateurs et utilisatrices” instead of “Utilisateur·rice·s”. +When space is missing, the male gender should be used alone. -- Utilisateur•rice•s - -[ecriture-inclusive.fr]: http://www.ecriture-inclusive.fr/ diff --git a/doc/development/import_export.md b/doc/development/import_export.md index f7f48b03651..fd067b80c16 100644 --- a/doc/development/import_export.md +++ b/doc/development/import_export.md @@ -27,7 +27,7 @@ Read through the current performance problems using the Import/Export below. ### OOM errors -Out of memory (OOM) errors are normally caused by the [Sidekiq Memory Killer](https://docs.gitlab.com/ee/administration/operations/sidekiq_memory_killer.html): +Out of memory (OOM) errors are normally caused by the [Sidekiq Memory Killer](../administration/operations/sidekiq_memory_killer.md): ```bash SIDEKIQ_MEMORY_KILLER_MAX_RSS = 2GB in GitLab.com diff --git a/doc/development/licensed_feature_availability.md b/doc/development/licensed_feature_availability.md index 1657d73e0c9..6f3dd59b2c3 100644 --- a/doc/development/licensed_feature_availability.md +++ b/doc/development/licensed_feature_availability.md @@ -7,8 +7,8 @@ on-premise or GitLab.com plans and features. ## Restricting features scoped by namespaces or projects GitLab.com plans are persisted on user groups and namespaces, therefore, if you're adding a -feature such as [Related issues](https://docs.gitlab.com/ee/user/project/issues/related_issues.html) or -[Service desk](https://docs.gitlab.com/ee/user/project/service_desk.html), +feature such as [Related issues](../user/project/issues/related_issues.md) or +[Service desk](../user/project/service_desk.md), it should be restricted on namespace scope. 1. Add the feature symbol on `EES_FEATURES`, `EEP_FEATURES` or `EEU_FEATURES` constants in @@ -22,8 +22,8 @@ project.feature_available?(:feature_symbol) ## Restricting global features (instance) -However, for features such as [Geo](https://docs.gitlab.com/ee/administration/geo/replication/index.html) and -[Load balancing](https://docs.gitlab.com/ee/administration/database_load_balancing.html), which cannot be restricted +However, for features such as [Geo](../administration/geo/replication/index.md) and +[Load balancing](../administration/database_load_balancing.md), which cannot be restricted to only a subset of projects or namespaces, the check will be made directly in the instance license. diff --git a/doc/development/licensing.md b/doc/development/licensing.md index 0e71cd47481..0db90d2872f 100644 --- a/doc/development/licensing.md +++ b/doc/development/licensing.md @@ -88,9 +88,11 @@ Definitions GitLab means GitLab Inc. and its affiliates and subsidiaries. -## Requesting Approval for Licenses +## Requesting Approval for Licenses or any other Intellectual Property -Libraries that are not listed in the [Acceptable Licenses][Acceptable-Licenses] or [Unacceptable Licenses][Unacceptable-Licenses] list can be submitted to the legal team for review. Please email `legal@gitlab.com` with the details. After a decision has been made, the original requestor is responsible for updating this document. +Libraries that are not already approved and listed on the [Acceptable Licenses][Acceptable-Licenses] list or that may be listed on the [Unacceptable Licenses][Unacceptable-Licenses] list may be submitted to the legal team for review and use on a case-by-case basis. Please email `legal@gitlab.com` with the details of how the software will be used, whether or not it will be modified, and how it will be distributed (if at all). After a decision has been made, the original requestor is responsible for updating this document, if applicable. Not all approvals will be approved for universal use and may continue to remain on the Unacceptable License list. + +All inquiries relating to patents should be directed to the Legal team. ## Notes diff --git a/doc/development/migration_style_guide.md b/doc/development/migration_style_guide.md index 0c326eeb851..9b26f691b55 100644 --- a/doc/development/migration_style_guide.md +++ b/doc/development/migration_style_guide.md @@ -186,7 +186,11 @@ end When adding a foreign-key constraint to either an existing or new column remember to also add a index on the column. -This is _required_ for all foreign-keys. +This is **required** for all foreign-keys, e.g., to support efficient cascading +deleting: when a lot of rows in a table get deleted, the referenced records need +to be deleted too. The database has to look for corresponding records in the +referenced table. Without an index, this will result in a sequential scan on the +table which can take a long time. Here's an example where we add a new column with a foreign key constraint. Note it includes `index: true` to create an index for it. diff --git a/doc/development/new_fe_guide/development/components.md b/doc/development/new_fe_guide/development/components.md index 8ae58d30c35..963ce53423b 100644 --- a/doc/development/new_fe_guide/development/components.md +++ b/doc/development/new_fe_guide/development/components.md @@ -17,5 +17,5 @@ D3 is very popular across many projects outside of GitLab: Within GitLab, D3 has been used for the following notable features -- [Prometheus graphs](https://docs.gitlab.com/ee/user/project/integrations/prometheus.html) +- [Prometheus graphs](../../../user/project/integrations/prometheus.md) - Contribution calendars diff --git a/doc/development/new_fe_guide/tips.md b/doc/development/new_fe_guide/tips.md index 889a5aab2b7..4564f678ec0 100644 --- a/doc/development/new_fe_guide/tips.md +++ b/doc/development/new_fe_guide/tips.md @@ -10,16 +10,16 @@ yarn clean ## Creating feature flags in development -The process for creating a feature flag is the same as [enabling a feature flag in development](https://docs.gitlab.com/ee/development/feature_flags.html#enabling-a-feature-flag-in-development). +The process for creating a feature flag is the same as [enabling a feature flag in development](../feature_flags.md#enabling-a-feature-flag-in-development). Your feature flag can now be: -- [made available to the frontend](https://docs.gitlab.com/ee/development/feature_flags.html#frontend) via the `gon` -- queried in [tests](https://docs.gitlab.com/ee/development/feature_flags.html#specs) +- [made available to the frontend](../feature_flags.md#frontend) via the `gon` +- queried in [tests](../feature_flags.md#specs) - queried in HAML templates and ruby files via the `Feature.enabled?(:my_shiny_new_feature_flag)` method ### More on feature flags -- [Deleting a feature flag](https://docs.gitlab.com/ee/api/features.html#delete-a-feature) -- [Manage feature flags](https://docs.gitlab.com/ee/development/feature_flags.html) -- [Feature flags API](https://docs.gitlab.com/ee/api/features.html) +- [Deleting a feature flag](../../api/features.md#delete-a-feature) +- [Manage feature flags](../feature_flags.md) +- [Feature flags API](../../api/features.md) diff --git a/doc/development/packages.md b/doc/development/packages.md index a3b891d7834..ab0c5f9904d 100644 --- a/doc/development/packages.md +++ b/doc/development/packages.md @@ -1,8 +1,8 @@ # Packages **[PREMIUM]** -This document will guide you through adding another [package management system](https://docs.gitlab.com/ee/administration/packages.html) support to GitLab. +This document will guide you through adding another [package management system](../administration/packages.md) support to GitLab. -See already supported package types in [Packages documentation](https://docs.gitlab.com/ee/administration/packages.html) +See already supported package types in [Packages documentation](../administration/packages.md) Since GitLab packages' UI is pretty generic, it is possible to add new package system support by solely backend changes. This guide is superficial and does @@ -46,7 +46,7 @@ Group-level and instance-level endpoints are good to have but are optional. NOTE: **Note:** To avoid name conflict for instance-level endpoints we use -[the package naming convention](https://docs.gitlab.com/ee/user/project/packages/npm_registry.html#package-naming-convention) +[the package naming convention](../user/project/packages/npm_registry.md#package-naming-convention) ## Configuration diff --git a/doc/development/rake_tasks.md b/doc/development/rake_tasks.md index 27fc3231218..28a12572961 100644 --- a/doc/development/rake_tasks.md +++ b/doc/development/rake_tasks.md @@ -31,7 +31,7 @@ project. #### Seeding issues for Insights charts **[ULTIMATE]** You can seed issues specifically for working with the -[Insights charts](https://docs.gitlab.com/ee/user/group/insights/index.html) with the +[Insights charts](../user/group/insights/index.md) with the `gitlab:seed:insights:issues` task: ```shell @@ -108,11 +108,13 @@ To make sure that indices still fit. You could find great details in: In order to run the test you can use the following commands: -- `rake spec` to run the rspec suite -- `rake karma` to run the karma test suite -- `rake gitlab:test` to run all the tests +- `bin/rake spec` to run the rspec suite +- `bin/rake spec:unit` to run the only the unit tests +- `bin/rake spec:integration` to run the only the integration tests +- `bin/rake spec:system` to run the only the system tests +- `bin/rake karma` to run the karma test suite -Note: `rake spec` takes significant time to pass. +Note: `bin/rake spec` takes significant time to pass. Instead of running full test suite locally you can save a lot of time by running a single test or directory related to your changes. After you submit merge request CI will run full test suite for you. Green CI status in the merge request means @@ -121,6 +123,9 @@ full test suite is passed. Note: You can't run `rspec .` since this will try to run all the `_spec.rb` files it can find, also the ones in `/tmp` +Note: You can pass RSpec command line options to the `spec:unit`, +`spec:integration`, and `spec:system` tasks, e.g. `bin/rake "spec:unit[--tag ~geo --dry-run]"`. + To run a single test file you can use: - `bin/rspec spec/controllers/commit_controller_spec.rb` for a rspec test diff --git a/doc/development/rolling_out_changes_using_feature_flags.md b/doc/development/rolling_out_changes_using_feature_flags.md index 8e4e07c4319..84028b1b342 100644 --- a/doc/development/rolling_out_changes_using_feature_flags.md +++ b/doc/development/rolling_out_changes_using_feature_flags.md @@ -102,7 +102,7 @@ GitLab's feature library (using [Flipper](https://github.com/jnunemaker/flipper), and covered in the [Feature Flags](feature_flags.md) guide) supports rolling out changes to a percentage of users. This in turn can be controlled using [GitLab -chatops](https://docs.gitlab.com/ee/ci/chatops/). +chatops](../ci/chatops/README.md). For an up to date list of feature flag commands please see [the source code](https://gitlab.com/gitlab-com/chatops/blob/master/lib/chatops/commands/feature.rb). @@ -200,10 +200,9 @@ isn't gated by a License or Plan. ### Undefined feature flags default to "on" -An important side-effect of the [implicit feature -flags][#implicit-feature-flags] mentioned above is that unless the feature is -explicitly disabled or limited to a percentage of users, the feature flag check -will default to `true`. +An important side-effect of the [implicit feature flags](#implicit-feature-flags) +mentioned above is that unless the feature is explicitly disabled or limited to a +percentage of users, the feature flag check will default to `true`. As an example, if you were to ship the backend half of a feature behind a flag, you'd want to explicitly disable that flag until the frontend half is also ready diff --git a/doc/development/routing.md b/doc/development/routing.md new file mode 100644 index 00000000000..e9c0ad8d4e8 --- /dev/null +++ b/doc/development/routing.md @@ -0,0 +1,63 @@ +# Routing + +The GitLab backend is written primarily with Rails so it uses [Rails +routing](https://guides.rubyonrails.org/routing.html). Beside Rails best +practices, there are few rules unique to the GitLab application. To +support subgroups, GitLab project and group routes use the wildcard +character to match project and group routes. For example, we might have +a path such as: + + /gitlab-com/customer-success/north-america/west/customerA + +However, paths can be ambiguous. Consider the following example: + + /gitlab-com/edit + +It's ambiguous whether there is a subgroup named `edit` or whether +this is a special endpoint to edit the `gitlab-com` group. + +To eliminate the ambiguity and to make the backend easier to maintain, +we introduced the `/-/` scope. The purpose of it is to separate group or +project paths from the rest of the routes. Also it helps to reduce the +number of [reserved names](../user/reserved_names.md). + +## Global routes + +We have a number of global routes. For example: + + /-/health + /-/metrics + +## Group routes + +Every group route must be under the `/-/` scope. + +Examples: + + gitlab-org/-/edit + gitlab-org/-/activity + gitlab-org/-/security/dashboard + gitlab-org/serverless/-/activity + +To achieve that, use the `scope '-'` method. + +## Project routes + +Every project route must be under the `/-/` scope, except cases where a Git +client or other software requires something different. + +Examples: + + gitlab-org/gitlab-ce/-/activity + gitlab-org/gitlab-ce/-/jobs/123 + gitlab-org/gitlab-ce/-/settings/repository + gitlab-org/serverless/runtimes/-/settings/repository + +Currently, only some project routes are placed under the `/-/` scope. However, +you can help us migrate more of them! To migrate project routes: + +1. Modify existing routes by adding `-` scope. +1. Add redirects for legacy routes by using `Gitlab::Routing.redirect_legacy_paths`. +1. Create a technical debt issue to remove deprecated routes in later releases. + +To get started, see an [example merge request](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/28435). diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md index 63ec9755462..82439c94c5a 100644 --- a/doc/development/testing_guide/best_practices.md +++ b/doc/development/testing_guide/best_practices.md @@ -2,19 +2,29 @@ ## Test Design -Testing at GitLab is a first class citizen, not an afterthought. It's important we consider the design of our tests -as we do the design of our features. +Testing at GitLab is a first class citizen, not an afterthought. It's important we consider the design of our tests +as we do the design of our features. -When implementing a feature, we think about developing the right capabilities the right way, which helps us -narrow our scope to a manageable level. When implementing tests for a feature, we must think about developing -the right tests, but then cover _all_ the important ways the test may fail, which can quickly widen our scope to +When implementing a feature, we think about developing the right capabilities the right way, which helps us +narrow our scope to a manageable level. When implementing tests for a feature, we must think about developing +the right tests, but then cover _all_ the important ways the test may fail, which can quickly widen our scope to a level that is difficult to manage. -Test heuristics can help solve this problem. They concisely address many of the common ways bugs -manifest themselves within our code. When designing our tests, take time to review known test heuristics to inform -our test design. We can find some helpful heuristics documented in the Handbook in the +Test heuristics can help solve this problem. They concisely address many of the common ways bugs +manifest themselves within our code. When designing our tests, take time to review known test heuristics to inform +our test design. We can find some helpful heuristics documented in the Handbook in the [Test Design](https://about.gitlab.com/handbook/engineering/quality/guidelines/test-engineering/test-design/) section. +## Run tests against MySQL + +By default, tests are only run againts PostgreSQL, but you can run them on +demand against MySQL by following one of the following conventions: + +| Convention | Valid example | +|:----------------------|:-----------------------------| +| Include `mysql` in your branch name | `enhance-mysql-support` | +| Include `[run mysql]` in your commit message | `Fix MySQL support<br><br>[run mysql]` | + ## Test speed GitLab has a massive test suite that, without [parallelization], can take hours @@ -184,11 +194,11 @@ instead of 30+ seconds in case of a regular `spec_helper`. ### `let` variables GitLab's RSpec suite has made extensive use of `let`(along with it strict, non-lazy -version `let!`) variables to reduce duplication. However, this sometimes [comes at the cost of clarity][lets-not], +version `let!`) variables to reduce duplication. However, this sometimes [comes at the cost of clarity][lets-not], so we need to set some guidelines for their use going forward: - `let!` variables are preferable to instance variables. `let` variables - are preferable to `let!` variables. Local variables are preferable to + are preferable to `let!` variables. Local variables are preferable to `let` variables. - Use `let` to reduce duplication throughout an entire spec file. - Don't use `let` to define variables used by a single test; define them as @@ -199,8 +209,8 @@ so we need to set some guidelines for their use going forward: - Try to avoid overriding the definition of one `let` variable with another. - Don't define a `let` variable that's only used by the definition of another. Use a helper method instead. -- `let!` variables should be used only in case if strict evaluation with defined - order is required, otherwise `let` will suffice. Remember that `let` is lazy and won't +- `let!` variables should be used only in case if strict evaluation with defined + order is required, otherwise `let` will suffice. Remember that `let` is lazy and won't be evaluated until it is referenced. [lets-not]: https://robots.thoughtbot.com/lets-not diff --git a/doc/development/testing_guide/end_to_end/best_practices.md b/doc/development/testing_guide/end_to_end/best_practices.md new file mode 100644 index 00000000000..89500ef9a90 --- /dev/null +++ b/doc/development/testing_guide/end_to_end/best_practices.md @@ -0,0 +1,38 @@ +# Best practices when writing end-to-end tests + +The majority of the end-to-end tests require some state to be built in the application for the tests to happen. + +A good example is a user being logged in as a pre-condition for testing the feature. + +But if the login feature is already covered with end-to-end tests through the GUI, there is no reason to perform such an expensive task to test the functionality of creating a project, or importing a repo, even if these features depend on a user being logged in. Let's see an example to make things clear. + +Let's say that, on average, the process to perform a successful login through the GUI takes 2 seconds. + +Now, realize that almost all tests need the user to be logged in, and that we need every test to run in isolation, meaning that tests cannot interfere with each other. This would mean that for every test the user needs to log in, and "waste 2 seconds". + +Now, multiply the number of tests per 2 seconds, and as your test suite grows, the time to run it grows with it, and this is not sustainable. + +An alternative to perform a login in a cheaper way would be having an endpoint (available only for testing) where we could pass the user's credentials as encrypted values as query strings, and then we would be redirected to the logged in home page if the credentials are valid. Let's say that, on average, this process takes only 200 miliseconds. + +You see the point right? + +Performing a login through the GUI for every test would cost a lot in terms of tests' execution. + +And there is another reason. + +Let's say that you don't follow the above suggestion, and depend on the GUI for the creation of every application state in order to test a specific feature. In this case we could be talking about the **Issues** feature, that depends on a project to exist, and the user to be logged in. + +What would happen if there was a bug in the project creation page, where the 'Create' button is disabled, not allowing for the creation of a project through the GUI, but the API logic is still working? + +In this case, instead of having only the project creation test failing, we would have many tests that depend on a project to be failing too. + +But, if we were following the best practices, only one test would be failing, and tests for other features that depend on a project to exist would continue to pass, since they could be creating the project behind the scenes interacting directly with the public APIs, ensuring a more reliable metric of test failure rate. + +Finally, interacting with the application only by its GUI generates a higher rate of test flakiness, and we want to avoid that at max. + +**The takeaways here are:** + +- Building state through the GUI is time consuming and it's not sustainable as the test suite grows. +- When depending only on the GUI to create the application's state and tests fail due to front-end issues, we can't rely on the test failures rate, and we generate a higher rate of test flakiness. + +Now that we are aware of all of it, [let's go create some tests](quick_start_guide.md). diff --git a/doc/development/testing_guide/end_to_end/dynamic_element_validation.md b/doc/development/testing_guide/end_to_end/dynamic_element_validation.md new file mode 100644 index 00000000000..f7b3ca8bc89 --- /dev/null +++ b/doc/development/testing_guide/end_to_end/dynamic_element_validation.md @@ -0,0 +1,113 @@ +# Dynamic Element Validation + +We devised a solution to solve common test automation problems such as the dreaded `NoSuchElementException`. + +Other problems that dynamic element validations solve are... + +- When we perform an action with the mouse, we expect something to occur. +- When our test is navigating to (or from) a page, we ensure that we are on the page we expect before +test continuation. + +## How it works + +We interpret user actions on the page to have some sort of effect. These actions are + +- [Navigation](#navigation) +- [Clicks](#clicks) + +### Navigation + +When a page is navigated to, there are elements that will always appear on the page unconditionally. + +Dynamic element validation is instituted when using + +```ruby +Runtime::Browser.visit(:gitlab, Some::Page) +``` + +### Clicks + +When we perform a click within our tests, we expect something to occur. That something could be a component to now +appear on the webpage, or the test to navigate away from the page entirely. + +Dynamic element validation is instituted when using + +```ruby +click_element :my_element, Some::Page +``` + +### Required Elements + +#### Definition + +First it is important to define what a "required element" is. + +Simply put, a required element is a visible HTML element that appears on a UI component without any user input. + +"Visible" can be defined as + +- Not having any CSS preventing its display. E.g.: `display: none` or `width: 0px; height: 0px;` +- Being able to be interacted with by the user + +"UI component" can be defined as + +- Anything the user sees +- A button, a text field +- A layer that sits atop the page + +#### Application + +Requiring elements is very easy. By adding `required: true` as a parameter to an `element`, you've now made it +a requirement that the element appear on the page upon navigation. + +## Examples + +Given ... + +```ruby +class MyPage < Page::Base + view 'app/views/view.html.haml' do + element :my_element, required: true + element :another_element, required: true + element :conditional_element + end + + def open_layer + click_element :my_element, Layer::MyLayer + end +end + +class Layer < Page::Component + view 'app/views/mylayer/layer.html.haml' do + element :message_content, required: true + end +end +``` + +### Navigating + +Given the [source](#examples) ... + +```ruby +Runtime::Browser.visit(:gitlab, Page::MyPage) + +execute_stuff +``` + +will invoke GitLab QA to scan `MyPage` for `my_element` and `another_element` to be on the page before continuing to +`execute_stuff` + +### Clicking + +Given the [source](#examples) ... + +```ruby +def open_layer + click_element :my_element, Layer::MyLayer +end +``` + +will invoke GitLab QA to ensure that `message_content` appears on +the Layer upon clicking `my_element`. + +This will imply that the Layer is indeed rendered before we continue our test. diff --git a/doc/development/testing_guide/end_to_end_tests.md b/doc/development/testing_guide/end_to_end/index.md index fc7aaedca29..afd81ff00b2 100644 --- a/doc/development/testing_guide/end_to_end_tests.md +++ b/doc/development/testing_guide/end_to_end/index.md @@ -65,7 +65,7 @@ Below you can read more about how to use it and how does it work. Currently, we are using _multi-project pipeline_-like approach to run QA pipelines. - + <details> <summary>Show mermaid source</summary> @@ -136,6 +136,12 @@ Once you decided where to put [test environment orchestration scenarios] and the [GitLab QA orchestrator README][gitlab-qa-readme], and [the already existing instance-level scenarios][instance-level scenarios]. +Continued reading: + +- [Quick Start Guide](quick_start_guide.md) +- [Style Guide](style_guide.md) +- [Best Practices](best_practices.md) + ## Where can I ask for help? You can ask question in the `#quality` channel on Slack (GitLab internal) or @@ -149,7 +155,7 @@ you can find an issue you would like to work on in [gitlab-qa-readme]: https://gitlab.com/gitlab-org/gitlab-qa/tree/master/README.md [quality-nightly-pipelines]: https://gitlab.com/gitlab-org/quality/nightly/pipelines [quality-staging-pipelines]: https://gitlab.com/gitlab-org/quality/staging/pipelines -[review-apps]: ./review_apps.md +[review-apps]: ../review_apps.md [gitlab-qa-architecture]: https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/architecture.md [gitlab-qa-issues]: https://gitlab.com/gitlab-org/gitlab-qa/issues?label_name%5B%5D=new+scenario [gitlab-ce-issues]: https://gitlab.com/gitlab-org/gitlab-ce/issues?label_name[]=QA&label_name[]=test diff --git a/doc/development/testing_guide/end_to_end/page_objects.md b/doc/development/testing_guide/end_to_end/page_objects.md new file mode 100644 index 00000000000..73e1fd862c1 --- /dev/null +++ b/doc/development/testing_guide/end_to_end/page_objects.md @@ -0,0 +1,167 @@ +# Page objects in GitLab QA + +In GitLab QA we are using a known pattern, called _Page Objects_. + +This means that we have built an abstraction for all GitLab pages that we use +to drive GitLab QA scenarios. Whenever we do something on a page, like filling +in a form, or clicking a button, we do that only through a page object +associated with this area of GitLab. + +For example, when GitLab QA test harness signs in into GitLab, it needs to fill +in a user login and user password. In order to do that, we have a class, called +`Page::Main::Login` and `sign_in_using_credentials` methods, that is the only +piece of the code, that has knowledge about `user_login` and `user_password` +fields. + +## Why do we need that? + +We need page objects, because we need to reduce duplication and avoid problems +whenever someone changes some selectors in GitLab's source code. + +Imagine that we have a hundred specs in GitLab QA, and we need to sign into +GitLab each time, before we make assertions. Without a page object one would +need to rely on volatile helpers or invoke Capybara methods directly. Imagine +invoking `fill_in :user_login` in every `*_spec.rb` file / test example. + +When someone later changes `t.text_field :login` in the view associated with +this page to `t.text_field :username` it will generate a different field +identifier, what would effectively break all tests. + +Because we are using `Page::Main::Login.act { sign_in_using_credentials }` +everywhere, when we want to sign into GitLab, the page object is the single +source of truth, and we will need to update `fill_in :user_login` +to `fill_in :user_username` only in a one place. + +## What problems did we have in the past? + +We do not run QA tests for every commit, because of performance reasons, and +the time it would take to build packages and test everything. + +That is why when someone changes `t.text_field :login` to +`t.text_field :username` in the _new session_ view we won't know about this +change until our GitLab QA nightly pipeline fails, or until someone triggers +`package-and-qa` action in their merge request. + +Obviously such a change would break all tests. We call this problem a _fragile +tests problem_. + +In order to make GitLab QA more reliable and robust, we had to solve this +problem by introducing coupling between GitLab CE / EE views and GitLab QA. + +## How did we solve fragile tests problem? + +Currently, when you add a new `Page::Base` derived class, you will also need to +define all selectors that your page objects depends on. + +Whenever you push your code to CE / EE repository, `qa:selectors` sanity test +job is going to be run as a part of a CI pipeline. + +This test is going to validate all page objects that we have implemented in +`qa/page` directory. When it fails, you will be notified about missing +or invalid views / selectors definition. + +## How to properly implement a page object? + +We have built a DSL to define coupling between a page object and GitLab views +it is actually implemented by. See an example below. + +```ruby +module Page + module Main + class Login < Page::Base + view 'app/views/devise/passwords/edit.html.haml' do + element :password_field + element :password_confirmation + element :change_password_button + end + + view 'app/views/devise/sessions/_new_base.html.haml' do + element :login_field + element :password_field + element :sign_in_button + end + + # ... + end + end +end +``` + +### Defining Elements + +The `view` DSL method will correspond to the rails View, partial, or vue component that renders the elements. + +The `element` DSL method in turn declares an element for which a corresponding +`qa-element-name-dasherized` CSS class will need to be added to the view file. + +You can also define a value (String or Regexp) to match to the actual view +code but **this is deprecated** in favor of the above method for two reasons: + +- Consistency: there is only one way to define an element +- Separation of concerns: QA uses dedicated CSS classes instead of reusing code + or classes used by other components (e.g. `js-*` classes etc.) + +```ruby +view 'app/views/my/view.html.haml' do + # Implicitly require `.qa-logout-button` CSS class to be present in the view + element :logout_button + + ## This is deprecated and forbidden by the `QA/ElementWithPattern` RuboCop cop. + # Require `f.submit "Sign in"` to be present in `my/view.html.haml + element :my_button, 'f.submit "Sign in"' # rubocop:disable QA/ElementWithPattern + + ## This is deprecated and forbidden by the `QA/ElementWithPattern` RuboCop cop. + # Match every line in `my/view.html.haml` against + # `/link_to .* "My Profile"/` regexp. + element :profile_link, /link_to .* "My Profile"/ # rubocop:disable QA/ElementWithPattern +end +``` + +### Adding Elements to a View + +Given the following elements... + +```ruby +view 'app/views/my/view.html.haml' do + element :login_field + element :password_field + element :sign_in_button +end +``` + +To add these elements to the view, you must change the rails View, partial, or vue component by adding a `qa-element-descriptor` class +for each element defined. + +In our case, `qa-login-field`, `qa-password-field` and `qa-sign-in-button` + +**app/views/my/view.html.haml** + +```haml += f.text_field :login, class: "form-control top qa-login-field", autofocus: "autofocus", autocapitalize: "off", autocorrect: "off", required: true, title: "This field is required." += f.password_field :password, class: "form-control bottom qa-password-field", required: true, title: "This field is required." += f.submit "Sign in", class: "btn btn-success qa-sign-in-button" +``` + +Things to note: + +- The CSS class must be `kebab-cased` (separated with hyphens "`-`") +- If the element appears on the page unconditionally, add `required: true` to the element. See +[Dynamic element validation](dynamic_element_validation.md) + +## Running the test locally + +During development, you can run the `qa:selectors` test by running + +```shell +bin/qa Test::Sanity::Selectors +``` + +from within the `qa` directory. + +## Where to ask for help? + +If you need more information, ask for help on `#quality` channel on Slack +(internal, GitLab Team only). + +If you are not a Team Member, and you still need help to contribute, please +open an issue in GitLab CE issue tracker with the `~QA` label. diff --git a/doc/development/testing_guide/end_to_end/quick_start_guide.md b/doc/development/testing_guide/end_to_end/quick_start_guide.md new file mode 100644 index 00000000000..afe76acf9c9 --- /dev/null +++ b/doc/development/testing_guide/end_to_end/quick_start_guide.md @@ -0,0 +1,585 @@ +# Writing end-to-end tests step-by-step + +In this tutorial, you will find different examples, and the steps involved, in the creation of end-to-end (_e2e_) tests for GitLab CE and GitLab EE, using GitLab QA. + +> When referring to end-to-end tests in this document, this means testing a specific feature end-to-end, such as a user logging in, the creation of a project, the management of labels, breaking down epics into sub-epics and issues, etc. + +## Important information before we start writing tests + +It's important to understand that end-to-end tests of isolated features, such as the ones described in the above note, doesn't mean that everything needs to happen through the GUI. + +If you don't exactly understand what we mean by **not everything needs to happen through the GUI,** please make sure you've read the [best practices](best_practices.md) before moving on. + +## This document covers the following items: + +- [0.](#0-are-end-to-end-tests-needed) Identifying if end-to-end tests are really needed +- [1.](#1-identifying-the-devops-stage) Identifying the [DevOps stage](https://about.gitlab.com/stages-devops-lifecycle/) of the feature that you are going to cover with end-to-end tests +- [2.](#2-test-skeleton) Creating the skeleton of the test file (`*_spec.rb`) +- [3.](#3-test-cases-mvc) The [MVC](https://about.gitlab.com/handbook/values/#minimum-viable-change-mvc) of the test cases' logic +- [4.](#4-extracting-duplicated-code) Extracting duplicated code into methods +- [5.](#5-tests-pre-conditions-using-resources-and-page-objects) Tests' pre-conditions (`before :context` and `before`) using resources and [Page Objects] +- [6.](#6-optimization) Optimizing the test suite +- [7.](#7-resources) Using and implementing resources +- [8.](#8-page-objects) Moving element definitions and methods to [Page Objects] + +### 0. Are end-to-end tests needed? + +At GitLab we respect the [test pyramid](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/testing_guide/testing_levels.md), and so, we recommend you check the code coverage of a specific feature before writing end-to-end tests, for both [CE](https://gitlab-org.gitlab.io/gitlab-ce/coverage-ruby/#_AllFiles) and [EE](https://gitlab-org.gitlab.io/gitlab-ee/coverage-ruby/#_AllFiles) projects. + +Sometimes you may notice that there is already good coverage in other test levels, and we can stay confident that if we break a feature, we will still have quick feedback about it, even without having end-to-end tests. + +If after this analysis you still think that end-to-end tests are needed, keep reading. + +### 1. Identifying the DevOps stage + +The GitLab QA end-to-end tests are organized by the different [stages in the DevOps lifecycle](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa/qa/specs/features/browser_ui), and so, if you are creating tests for issue creation, for instance, you would locate the spec files under the `qa/qa/specs/features/browser_ui/2_plan/` directory since issue creation is part of the Plan stage. + + In another case of a test for listing merged merge requests (MRs), the test should go under the `qa/qa/specs/features/browser_ui/3_create/` directory since merge requests are a feature from the Create stage. + +> There may be sub-directories inside the stages directories, for different features. For example: `.../browser_ui/2_plan/ee_epics/` and `.../browser_ui/2_plan/issues/`. + +Now, let's say we want to create tests for the [scoped labels](https://about.gitlab.com/2019/04/22/gitlab-11-10-released/#scoped-labels) feature, available on GitLab EE Premium (this feature is part of the Plan stage.) + +> Because these tests are for a feature available only on GitLab EE, we need to create them in the [EE repository](https://gitlab.com/gitlab-org/gitlab-ee). + +Since [there is no specific directory for this feature](https://gitlab.com/gitlab-org/gitlab-ee/tree/master/qa/qa/specs/features/browser_ui/2_plan), we should create a sub-directory for it. + +Under `.../browser_ui/2_plan/`, let's create a sub-directory called `ee_scoped_labels/`. + +> Notice that since this feature is only available for GitLab EE we prefix the sub-directory with `ee_`. + +### 2. Test skeleton + +Inside the newly created sub-directory, let's create a file describing the test suite (e.g. `editing_scoped_labels_spec.rb`.) + +#### The `context` and `describe` blocks + +Specs have an outer `context` that indicates the DevOps stage. The next level is the `describe` block, that briefly states the subject of the test suite. See the following example: + +```ruby +module QA + context 'Plan' do + describe 'Editing scoped labels on issues' do + end + end +end +``` + +#### The `it` blocks + +Every test suite is composed of at least one `it` block, and a good way to start writing end-to-end tests is by writing test cases descriptions as `it` blocks. These might help you to think of different test scenarios. Take a look at the following example: + +```ruby +module QA + context 'Plan' do + describe 'Editing scoped labels on issues' do + it 'replaces an existing label if it has the same key' do + end + + it 'keeps both scoped labels when adding a label with a different key' do + end + end + end +end +``` + +### 3. Test cases MVC + +For the [MVC](https://about.gitlab.com/handbook/values/#minimum-viable-change-mvc) of our test cases, let's say that we already have the application in the state needed for the tests, and then let's focus on the logic of the test cases only. + +To evolve the test cases drafted on step 2, let's imagine that the user is already logged into a GitLab EE instance, they already have at least a Premium license in use, there is already a project created, there is already an issue opened in the project, the issue already has a scoped label (e.g. `animal::fox`), there are other scoped labels (for the same scope and for a different scope (e.g. `animal::dolphin` and `plant::orchid`), and finally, the user is already on the issue's page. Let's also suppose that for every test case the application is in a clean state, meaning that one test case won't affect another. + +> Note: there are different approaches to creating an application state for end-to-end tests. Some of them are very time consuming and subject to failures, such as when using the GUI for all the pre-conditions of the tests. On the other hand, other approaches are more efficient, such as using the public APIs. The latter is more efficient since it doesn't depend on the GUI. We won't focus on this part yet, but it's good to keep it in mind. + +Let's now focus on the first test case. + +```ruby +it 'replaces an existing label if it has the same key' do + # This implementation is only for tutorial purposes. We normally encapsulate elements in Page Objects (which we cover on section 8). + page.find('.block.labels .edit-link').click + page.find('.dropdown-menu-labels .dropdown-input-field').send_keys ['animal::dolphin', :enter] + page.find('#content-body').click + page.refresh + + labels_block = page.find('.qa-labels-block') + + expect(labels_block).to have_content('animal::dolphin') + expect(labels_block).not_to have_content('animal::fox') + expect(page).to have_content('added animal::dolphin label and removed animal::fox') +end +``` + +> Notice that the test itself is simple. The most challenging part is the creation of the application state, which will be covered later. + +> The exemplified test case's MVC is not enough for the change to be merged, but it helps to build up the test logic. The reason is that we do not want to use locators directly in the tests, and tests **must** use [Page Objects] before they can be merged. This way we better separate the responsibilities, where the Page Objects encapsulate elements and methods that allow us to interact with pages, while the spec files describe the test cases in more business-related language. + +Below are the steps that the test covers: + +1. The test finds the 'Edit' link for the labels and clicks on it. +2. Then it fills in the 'Assign labels' input field with the value 'animal::dolphin' and press enters. +3. Then it clicks in the content body to apply the label and refreshes the page. +4. Finally, the expectations check that the previous scoped label was removed and that the new one was added. + +Let's now see how the second test case would look. + +```ruby +it 'keeps both scoped labels when adding a label with a different key' do + # This implementation is only for tutorial purposes. We normally encapsulate elements in Page Objects (which we cover on section 8). + page.find('.block.labels .edit-link').click + page.find('.dropdown-menu-labels .dropdown-input-field').send_keys ['plant::orchid', :enter] + page.find('#content-body').click + page.refresh + + labels_block = page.find('.qa-labels-block') + + expect(labels_block).to have_content('animal::fox') + expect(labels_block).to have_content('plant::orchid') + expect(page).to have_content('added animal::fox') + expect(page).to have_content('added plant::orchid') +end +``` + +> Note that elements are always located using CSS selectors, and a good practice is to add test-specific selectors (this is called adding testability to the application and we will talk more about it later.) For example, the `labels_block` element uses the selector `.qa-labels-block`, which was added specifically for testing purposes. + +Below are the steps that the test covers: + +1. The test finds the 'Edit' link for the labels and clicks on it. +2. Then it fills in the 'Assign labels' input field with the value 'plant::orchid' and press enters. +3. Then it clicks in the content body to apply the label and refreshes the page. +4. Finally, the expectations check that both scoped labels are present. + +> Similar to the previous test, this one is also very straightforward, but there is some code duplication. Let's address it. + +### 4. Extracting duplicated code + +If we refactor the tests created on step 3 we could come up with something like this: + +```ruby +before do + ... + + @initial_label = 'animal::fox' + @new_label_same_scope = 'animal::dolphin' + @new_label_different_scope = 'plant::orchid' + + ... +end + +it 'replaces an existing label if it has the same key' do + select_label_and_refresh @new_label_same_scope + + labels_block = page.find('.qa-labels-block') + + expect(labels_block).to have_content(@new_label_same_scope) + expect(labels_block).not_to have_content(@initial_label) + expect(page).to have_content("added #{@new_label_same_scope}") + expect(page).to have_content("and removed #{@initial_label}") +end + +it 'keeps both scoped label when adding a label with a different key' do + select_label_and_refresh @new_label_different_scope + + labels_block = page.find('.qa-labels-block') + + expect(labels_blocks).to have_content(@new_label_different_scope) + expect(labels_blocks).to have_content(@initial_label) + expect(page).to have_content("added #{@new_label_different_scope}") + expect(page).to have_content("added #{@initial_label}") +end + +def select_label_and_refresh(label) + page.find('.block.labels .edit-link').click + page.find('.dropdown-menu-labels .dropdown-input-field').send_keys [label, :enter] + page.find('#content-body').click + page.refresh +end +``` + +First, we remove the duplication of strings by defining the global variables `@initial_label`, `@new_label_same_scope` and `@new_label_different_scope` in the `before` block, and by using them in the expectations. + +Then, by creating a reusable `select_label_and_refresh` method, we remove the code duplication of this action, and later we can move this method to a Page Object class that will be created for easier maintenance purposes. + +> Notice that the reusable method is created at the bottom of the file. The reason for that is that reading the code should be similar to reading a newspaper, where high-level information is at the top, like the title and summary of the news, while low level, or more specific information, is at the bottom (this helps readability). + +### 5. Tests' pre-conditions using resources and Page Objects + +In this section, we will address the previously mentioned subject of creating the application state for the tests, using the `before :context` and `before` blocks, together with resources and Page Objects. + +#### `before :context` + +A pre-condition for the entire test suite is defined in the `before :context` block. + +> For our test suite, due to the need of the tests being completely independent of each other, we won't use the `before :context` block. The `before :context` block would make the tests dependent on each other because the first test changes the label of the issue, and the second one depends on the `'animal::fox'` label being set. + +> **Tip:** In case of a test suite with only one `it` block it's ok to use only the `before` block (see below) with all the test's pre-conditions. + +#### `before` + +As the pre-conditions for our test suite, the things that needs to happen before each test starts are: + +- The user logging in; +- A premium license already being set; +- A project being created with an issue and labels already set; +- The issue page being opened with only one scoped label applied to the it. + +> When running end-to-end tests as part of the GitLab's continuous integration process [a license is already set as an environment variable](https://gitlab.com/gitlab-org/gitlab-ee/blob/1a60d926740db10e3b5724713285780a4f470531/qa/qa/ee/strategy.rb#L20). For running tests locally you can set up such license by following the document [what tests can be run?](https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/what_tests_can_be_run.md#supported-remote-grid-environment-variables), based on the [supported GitLab environment variables](https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/what_tests_can_be_run.md#supported-gitlab-environment-variables). + +#### Implementation + +In the following code we will focus only on the test suite's pre-conditions: + +```ruby +module QA + context 'Plan' do + describe 'Editing scoped labels on issues' do + before do + Runtime::Browser.visit(:gitlab, Page::Main::Login) + Page::Main::Login.perform(&:sign_in_using_credentials) + + @initial_label = 'animal::fox' + @new_label_same_scope = 'animal::dolphin' + @new_label_different_scope = 'plant::orchid' + + issue = Resource::Issue.fabricate_via_api! do |issue| + issue.title = 'Issue to test the scoped labels' + issue.labels = @initial_label + end + + [@new_label_same_scope, @new_label_different_scope].each do |label| + Resource::Label.fabricate_via_api! do |l| + l.project = issue.project.id + l.title = label + end + end + + issue.visit! + end + + it 'replaces an existing label if it has the same key' do + ... + end + + it 'keeps both scoped labels when adding a label with a different key' do + ... + end + + def select_label_and_refresh(label) + ... + end + end + end +end +``` + +In the `before` block we create all the application state needed for the tests to run. We do that by using the `Runtime::Browser.visit` method to go to the login page, by performing a `sign_in_using_credentials` from the `Login` Page Object, by fabricating resources via APIs (`issue`, and `Resource::Label`), and by using the `issue.visit!` to visit the issue page. + +> A project is created in the background by creating the `issue` resource. + +> When creating the [Resources], notice that when calling the `fabricate_via_api` method, we pass some attribute:values, like `title`, and `labels` for the `issue` resource; and `project` and `title` for the `label` resource. + +> What's important to understand here is that by creating the application state mostly using the public APIs we save a lot of time in the test suite setup stage. + +> Soon we will cover the use of the already existing resources' methods and the creation of your own `fabricate_via_api` methods for resources where this is still not available, but first, let's optimize our implementation. + +### 6. Optimization + +As already mentioned in the [best practices](best_practices.md) document, end-to-end tests are very costly in terms of execution time, and it's our responsibility as software engineers to ensure that we optimize them as much as possible. + +> Note that end-to-end tests are slow to run and so they can have several actions and assertions in a single test, which helps us get feedback from the tests sooner. In comparison, unit tests are much faster to run and can exercise every little piece of the application in isolation, and so they usually have only one assertion per test. + +Some improvements that we could make in our test suite to optimize its time to run are: + +1. Having a single test case (an `it` block) that exercises both scenarios to avoid "wasting" time in the tests' pre-conditions, instead of having two different test cases. +2. Making the selection of labels more performant by allowing for the selection of more than one label in the same reusable method. + +Let's look at a suggestion that addresses the above points, one by one: + +```ruby +module QA + context 'Plan' do + describe 'Editing scoped labels on issues' do + before do + ... + end + + it 'correctly applies scoped labels depending on if they are from the same or a different scope' do + select_labels_and_refresh [@new_label_same_scope, @new_label_different_scope] + + labels_block = page.all('.qa-labels-block') + + expect(labels_block).to have_content(@new_label_same_scope) + expect(labels_block).to have_content(@new_label_different_scope) + expect(labels_block).not_to have_content(@initial_label) + expect(page).to have_content("added #{@initial_label}") + expect(page).to have_content("added #{@new_label_same_scope} #{@new_label_different_scope} labels and removed #{@initial_label}") + end + + def select_labels_and_refresh(labels) + find('.block.labels .edit-link').click + labels.each do |label| + find('.dropdown-menu-labels .dropdown-input-field').send_keys [label, :enter] + end + find('#content-body').click + refresh + end + end + end + end +``` + +To address point 1, we changed the test implementation from two `it` blocks into a single one that exercises both scenarios. Now the new test description is: `'correctly applies the scoped labels depending if they are from the same or a different scope'`. It's a long description, but it describes well what the test does. + +> Notice that the implementation of the new and unique `it` block had to change a little bit. Below we describe in details what it does. + +1. It selects two scoped labels simultaneously, one from the same scope of the one already applied in the issue during the setup phase (in the `before` block), and another one from a different scope. +2. It asserts that the correct labels are visible in the `labels_block`, and that the labels were correctly added and removed; +3. Finally, the `select_label_and_refresh` method is changed to `select_labels_and_refresh`, which accepts an array of labels instead of a single label, and it iterates on them for faster label selection (this is what is used in step 1 explained above.) + +### 7. Resources + +**Note:** When writing this document, some code that is now merged to master was not implemented yet, but we left them here for the readers to understand the whole process of end-to-end test creation. + +You can think of [Resources] as anything that can be created on GitLab CE or EE, either through the GUI, the API, or the CLI. + +With that in mind, resources can be a project, an epic, an issue, a label, a commit, etc. + +As you saw in the tests' pre-conditions and the optimization sections, we're already creating some of these resources, and we are doing that by calling the `fabricate_via_api!` method. + +> We could be using the `fabricate!` method instead, which would use the `fabricate_via_api!` method if it exists, and fallback to GUI fabrication otherwise, but we recommend being explicit to make it clear what the test does. Also, we always recommend fabricating resources via API since this makes tests faster and more reliable. + +For our test suite example, the resources that we need to create don't have the necessary code for the `fabricate_via_api!` method to correctly work (e.g., the issue and label resources), so we will have to create them. + +#### Implementation + +In the following we describe the changes needed in each of the resource files mentioned above. + +**Issue resource** + +Now, let's make it possible to create an issue resource through the API. + +First, in the [issue resource](https://gitlab.com/gitlab-org/gitlab-ee/blob/d3584e80b4236acdf393d815d604801573af72cc/qa/qa/resource/issue.rb), let's expose its labels attribute. + +Add the following `attribute :labels` right below the [`attribute :title`](https://gitlab.com/gitlab-org/gitlab-ee/blob/d3584e80b4236acdf393d815d604801573af72cc/qa/qa/resource/issue.rb#L15). + +> This line is needed to allow for labels to be automatically added to an issue when fabricating it via API. + +Next, add the following code right below the [`fabricate!`](https://gitlab.com/gitlab-org/gitlab-ee/blob/d3584e80b4236acdf393d815d604801573af72cc/qa/qa/resource/issue.rb#L27) method. + +```ruby +def api_get_path + "/projects/#{project.id}/issues/#{id}" +end + +def api_post_path + "/projects/#{project.id}/issues" +end + +def api_post_body + { + title: title, + labels: [labels] + } +end +``` + +By defining the `api_get_path` method, we allow the [`ApiFabricator`](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/qa/qa/resource/api_fabricator.rb) module to know which path to use to get a single issue. + +> This `GET` path can be found in the [public API documentation](https://docs.gitlab.com/ee/api/issues.html#single-issue). + +By defining the `api_post_path` method, we allow the [`ApiFabricator`](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/qa/qa/resource/api_fabricator.rb) module to know which path to use to create a new issue in a specific project. + +> This `POST` path can be found in the [public API documentation](https://docs.gitlab.com/ee/api/issues.html#new-issue). + +By defining the `api_post_body` method, we allow the [`ApiFabricator.api_post`](https://gitlab.com/gitlab-org/gitlab-ee/blob/a9177ca1812bac57e2b2fa4560e1d5dd8ffac38b/qa/qa/resource/api_fabricator.rb#L68) method to know which data to send when making the `POST` request. + +> Notice that we pass both `title` and `labels` attributes in the `api_post_body`, where `labels` receives an array of labels, and [`title` is required](https://docs.gitlab.com/ee/api/issues.html#new-issue). + +**Label resource** + +Finally, let's make it possible to create label resources through the API. + +Add the following code right below the [`fabricate!`](https://gitlab.com/gitlab-org/gitlab-ee/blob/a9177ca1812bac57e2b2fa4560e1d5dd8ffac38b/qa/qa/resource/label.rb#L36) method. + +```ruby +def resource_web_url(resource) + super +rescue ResourceURLMissingError + # this particular resource does not expose a web_url property +end + +def api_get_path + raise NotImplementedError, "The Labels API doesn't expose a single-resource endpoint so this method cannot be properly implemented." +end + +def api_post_path + "/projects/#{project}/labels" +end + +def api_post_body + { + name: @title, + color: @color + } +end +``` + +By defining the `resource_web_url(resource)` method, we override the one from the [`ApiFabricator`](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/qa/qa/resource/api_fabricator.rb#L44) module. We do that to avoid failing the test due to this particular resource not exposing a `web_url` property. + +By defining the `api_get_path` method, we **would** allow for the [`ApiFabricator`](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/qa/qa/resource/api_fabricator.rb) module to know which path to use to get a single label, but since there's no path available for that in the publich API, we raise a `NotImplementedError` instead. + +By defining the `api_post_path` method, we allow for the [`ApiFabricator `](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/qa/qa/resource/api_fabricator.rb) module to know which path to use to create a new label in a specific project. + +By defining the `api_post_body` method, we we allow for the [`ApiFabricator.api_post`](https://gitlab.com/gitlab-org/gitlab-ee/blob/a9177ca1812bac57e2b2fa4560e1d5dd8ffac38b/qa/qa/resource/api_fabricator.rb#L68) method to know which data to send when making the `POST` request. + +> Notice that we pass both `name` and `color` attributes in the `api_post_body` since [those are required](https://docs.gitlab.com/ee/api/labels.html#create-a-new-label). + +### 8. Page Objects + +Page Objects are used in end-to-end tests for maintenance reasons, where a page's elements and methods are defined to be reused in any test. + +> Page Objects are auto-loaded in the `qa/qa.rb` file and available in all the test files (`*_spec.rb`). + +Take a look at the [Page Objects] documentation. + +Now, let's go back to our example. + +As you may have noticed, we are defining elements with CSS selectors and the `select_labels_and_refresh` method directly in the test file, and this is an anti-pattern since we need to better separate the responsibilities. + +To address this issue, we will move the implementation to Page Objects, and the test suite will only focus on the business rules that we are testing. + +#### Updates in the test file + +As in a test-driven development approach, let's start changing the test file even before the Page Object implementation is in place. + +Replace the code of the `it` block in the test file by the following: + +```ruby +module QA + context 'Plan' do + describe 'Editing scoped labels on issues' do + before do + ... + end + + it 'correctly applies scoped labels depending on if they are from the same or a different scope' do + Page::Project::Issue::Show.perform do |issue_page| + issue_page.select_labels_and_refresh [@new_label_same_scope, @new_label_different_scope] + + expect(page).to have_content("added #{@initial_label}") + expect(page).to have_content("added #{@new_label_same_scope} #{@new_label_different_scope} labels and removed #{@initial_label}") + expect(issue_page.text_of_labels_block).to have_content(@new_label_same_scope) + expect(issue_page.text_of_labels_block).to have_content(@new_label_different_scope) + expect(issue_page.text_of_labels_block).not_to have_content(@initial_label) + end + end + end + end +end +``` + +Notice that `select_labels_and_refresh` is now a method from the issue Page Object (which is not yet implemented), and that we verify the labels' text by using `text_of_labels_block`, instead of via the `labels_block` element. The `text_of_labels_block` method will also be implemented in the issue Page Object. + +Let's now update the Issue Page Object. + +#### Updates in the Issue Page Object + +> Page Objects are located in the `qa/qa/page/` directory, and its sub-directories. + +The file we will have to change is the [Issue Page Object](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/qa/qa/page/project/issue/show.rb). + +First, add the following code right below the definition of an already implemented view: + +```ruby +view 'app/views/shared/issuable/_sidebar.html.haml' do + element :labels_block + element :edit_link_labels + element :dropdown_menu_labels +end + +view 'app/helpers/dropdowns_helper.rb' do + element :dropdown_input_field +end +``` + +Similarly to what we did before, let's first change the Page Object even without the elements being defined in the view (`_sidebar.html.haml`) and the `dropdowns_helper.rb` files, and later we will update them by adding the appropriate CSS selectors. + +Now, let's implement the methods `select_labels_and_refresh` and `text_of_labels_block`. + +Somewhere between the definition of the views and the private methods, add the following snippet of code: + +```ruby +def select_labels_and_refresh(labels) + click_element(:edit_link_labels) + labels.each do |label| + within_element(:dropdown_menu_labels, text: label) do + send_keys_to_element(:dropdown_input_field, [label, :enter]) + end + end + click_body + labels.each do |label| + has_element?(:labels_block, text: label) + end + refresh +end + +def text_of_labels_block + find_element(:labels_block) +end +``` + +##### Details of `select_labels_and_refresh` + +Notice that we have not only moved the `select_labels_and_refresh` method, but we have also changed its implementation to: +1. Click the `:edit_link_labels` element previously defined, instead of using `find('.block.labels .edit-link').click` +2. Use `within_element(:dropdown_menu_labels, text: label)`, and inside of it, we call `send_keys_to_element(:dropdown_input_field, [label, :enter])`, which is a method that we will implement in the `QA::Page::Base` class to replace `find('.dropdown-menu-labels .dropdown-input-field').send_keys [label, :enter]` +3. Use `click_body` after iterating on each label, instead of using `find('#content-body').click` +4. Iterate on every label again, and then we use `has_element?(:labels_block, text: label)` after clicking the page body (which applies the labels), and before refreshing the page, to avoid test flakiness due to refreshing too fast. + +##### Details of `text_of_labels_block` + +The `text_of_labels_block` method is a simple method that returns the `:labels_block` element (`find_element(:labels_block)`). + +#### Updates in the view (*.html.haml) and `dropdowns_helper.rb` files + +Now let's change the view and the `dropdowns_helper` files to add the selectors that relate to the Page Object. + +In the [app/views/shared/issuable/_sidebar.html.haml](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/app/views/shared/issuable/_sidebar.html.haml) file, on [line 105 ](https://gitlab.com/gitlab-org/gitlab-ee/blob/84043fa72ca7f83ae9cde48ad670e6d5d16501a3/app/views/shared/issuable/_sidebar.html.haml#L105), add an extra class `qa-edit-link-labels`. + +The code should look like this: `= link_to _('Edit'), '#', class: 'js-sidebar-dropdown-toggle edit-link qa-edit-link-labels float-right'`. + +In the same file, on [line 121](https://gitlab.com/gitlab-org/gitlab-ee/blob/84043fa72ca7f83ae9cde48ad670e6d5d16501a3/app/views/shared/issuable/_sidebar.html.haml#L121), add an extra class `.qa-dropdown-menu-labels`. + +The code should look like this: `.dropdown-menu.dropdown-select.dropdown-menu-paging.dropdown-menu-labels.qa-dropdown-menu-labels.dropdown-menu-selectable`. + +In the [`dropdowns_helper.rb`](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/app/helpers/dropdowns_helper.rb) file, on [line 94](https://gitlab.com/gitlab-org/gitlab-ee/blob/99e51a374f2c20bee0989cac802e4b5621f72714/app/helpers/dropdowns_helper.rb#L94), add an extra class `qa-dropdown-input-field`. + +The code should look like this: `filter_output = search_field_tag search_id, nil, class: "dropdown-input-field qa-dropdown-input-field", placeholder: placeholder, autocomplete: 'off'`. + +> Classes starting with `qa-` are used for testing purposes only, and by defining such classes in the elements we add **testability** in the application. + +> When defining a class like `qa-labels-block`, it is transformed into `:labels_block` for usage in the Page Objects. So, `qa-edit-link-labels` is tranformed into `:edit_link_labels`, `qa-dropdown-menu-labels` is transformed into `:dropdown_menu_labels`, and `qa-dropdown-input-field` is transformed into `:dropdown_input_field`. Also, we use a [sanity test](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa/qa/page#how-did-we-solve-fragile-tests-problem) to check that defined elements have their respective `qa-` selectors in the specified views. + +> We did not define the `qa-labels-block` class in the `app/views/shared/issuable/_sidebar.html.haml` file because it was already there to be used. + +#### Updates in the `QA::Page::Base` class + +The last thing that we have to do is to update `QA::Page::Base` class to add the `send_keys_to_element` method on it. + +Add the following snippet of code somewhere where class methods are defined: + +```ruby +def send_keys_to_element(name, keys) + find_element(name).send_keys(keys) +end +``` + +This method receives an element (`name`) and the `keys` that it will send to that element, and the keys are an array that can receive strings, or "special" keys, like `:enter`. + +As you might remember, in the Issue Page Object we call this method like this: `send_keys_to_element(:dropdown_input_field, [label, :enter])`. + +___ + +With that, you should be able to start writing end-to-end tests yourself. *Congratulations!* + +[Page Objects]: page_objects.md +[Resources]: resources.md diff --git a/doc/development/testing_guide/end_to_end/resources.md b/doc/development/testing_guide/end_to_end/resources.md new file mode 100644 index 00000000000..1e32db4f633 --- /dev/null +++ b/doc/development/testing_guide/end_to_end/resources.md @@ -0,0 +1,392 @@ +# Resource class in GitLab QA + +Resources are primarily created using Browser UI steps, but can also +be created via the API or the CLI. + +## How to properly implement a resource class? + +All resource classes should inherit from `Resource::Base`. + +There is only one mandatory method to implement to define a resource class. +This is the `#fabricate!` method, which is used to build the resource via the +browser UI. Note that you should only use [Page objects](page_objects.md) to +interact with a Web page in this method. + +Here is an imaginary example: + +```ruby +module QA + module Resource + class Shirt < Base + attr_accessor :name + + def fabricate! + Page::Dashboard::Index.perform do |dashboard_index| + dashboard_index.go_to_new_shirt + end + + Page::Shirt::New.perform do |shirt_new| + shirt_new.set_name(name) + shirt_new.create_shirt! + end + end + end + end +end +``` + +### Define API implementation + +A resource class may also implement the three following methods to be able to +create the resource via the public GitLab API: + +- `#api_get_path`: The `GET` path to fetch an existing resource. +- `#api_post_path`: The `POST` path to create a new resource. +- `#api_post_body`: The `POST` body (as a Ruby hash) to create a new resource. + +Let's take the `Shirt` resource class, and add these three API methods: + +```ruby +module QA + module Resource + class Shirt < Base + attr_accessor :name + + def fabricate! + # ... same as before + end + + def api_get_path + "/shirt/#{name}" + end + + def api_post_path + "/shirts" + end + + def api_post_body + { + name: name + } + end + end + end +end +``` + +The `Project` resource is a good real example of Browser +UI and API implementations. + +#### Resource attributes + +A resource may need another resource to exist first. For instance, a project +needs a group to be created in. + +To define a resource attribute, you can use the `attribute` method with a +block using the other resource class to fabricate the resource. + +That will allow access to the other resource from your resource object's +methods. You would usually use it in `#fabricate!`, `#api_get_path`, +`#api_post_path`, `#api_post_body`. + +Let's take the `Shirt` resource class, and add a `project` attribute to it: + +```ruby +module QA + module Resource + class Shirt < Base + attr_accessor :name + + attribute :project do + Project.fabricate! do |resource| + resource.name = 'project-to-create-a-shirt' + end + end + + def fabricate! + project.visit! + + Page::Project::Show.perform do |project_show| + project_show.go_to_new_shirt + end + + Page::Shirt::New.perform do |shirt_new| + shirt_new.set_name(name) + shirt_new.create_shirt! + end + end + + def api_get_path + "/project/#{project.path}/shirt/#{name}" + end + + def api_post_path + "/project/#{project.path}/shirts" + end + + def api_post_body + { + name: name + } + end + end + end +end +``` + +**Note that all the attributes are lazily constructed. This means if you want +a specific attribute to be fabricated first, you'll need to call the +attribute method first even if you're not using it.** + +#### Product data attributes + +Once created, you may want to populate a resource with attributes that can be +found in the Web page, or in the API response. +For instance, once you create a project, you may want to store its repository +SSH URL as an attribute. + +Again we could use the `attribute` method with a block, using a page object +to retrieve the data on the page. + +Let's take the `Shirt` resource class, and define a `:brand` attribute: + +```ruby +module QA + module Resource + class Shirt < Base + attr_accessor :name + + attribute :project do + Project.fabricate! do |resource| + resource.name = 'project-to-create-a-shirt' + end + end + + # Attribute populated from the Browser UI (using the block) + attribute :brand do + Page::Shirt::Show.perform do |shirt_show| + shirt_show.fetch_brand_from_page + end + end + + # ... same as before + end + end +end +``` + +**Note again that all the attributes are lazily constructed. This means if +you call `shirt.brand` after moving to the other page, it'll not properly +retrieve the data because we're no longer on the expected page.** + +Consider this: + +```ruby +shirt = + QA::Resource::Shirt.fabricate! do |resource| + resource.name = "GitLab QA" + end + +shirt.project.visit! + +shirt.brand # => FAIL! +``` + +The above example will fail because now we're on the project page, trying to +construct the brand data from the shirt page, however we moved to the project +page already. There are two ways to solve this, one is that we could try to +retrieve the brand before visiting the project again: + +```ruby +shirt = + QA::Resource::Shirt.fabricate! do |resource| + resource.name = "GitLab QA" + end + +shirt.brand # => OK! + +shirt.project.visit! + +shirt.brand # => OK! +``` + +The attribute will be stored in the instance therefore all the following calls +will be fine, using the data previously constructed. If we think that this +might be too brittle, we could eagerly construct the data right before +ending fabrication: + +```ruby +module QA + module Resource + class Shirt < Base + # ... same as before + + def fabricate! + project.visit! + + Page::Project::Show.perform do |project_show| + project_show.go_to_new_shirt + end + + Page::Shirt::New.perform do |shirt_new| + shirt_new.set_name(name) + shirt_new.create_shirt! + end + + populate(:brand) # Eagerly construct the data + end + end + end +end +``` + +The `populate` method will iterate through its arguments and call each +attribute respectively. Here `populate(:brand)` has the same effect as +just `brand`. Using the populate method makes the intention clearer. + +With this, it will make sure we construct the data right after we create the +shirt. The drawback is that this will always construct the data when the +resource is fabricated even if we don't need to use the data. + +Alternatively, we could just make sure we're on the right page before +constructing the brand data: + +```ruby +module QA + module Resource + class Shirt < Base + attr_accessor :name + + attribute :project do + Project.fabricate! do |resource| + resource.name = 'project-to-create-a-shirt' + end + end + + # Attribute populated from the Browser UI (using the block) + attribute :brand do + back_url = current_url + visit! + + Page::Shirt::Show.perform do |shirt_show| + shirt_show.fetch_brand_from_page + end + + visit(back_url) + end + + # ... same as before + end + end +end +``` + +This will make sure it's on the shirt page before constructing brand, and +move back to the previous page to avoid breaking the state. + +#### Define an attribute based on an API response + +Sometimes, you want to define a resource attribute based on the API response +from its `GET` or `POST` request. For instance, if the creation of a shirt via +the API returns + +```ruby +{ + brand: 'a-brand-new-brand', + style: 't-shirt', + materials: [[:cotton, 80], [:polyamide, 20]] +} +``` + +you may want to store `style` as-is in the resource, and fetch the first value +of the first `materials` item in a `main_fabric` attribute. + +Let's take the `Shirt` resource class, and define a `:style` and a +`:main_fabric` attributes: + +```ruby +module QA + module Resource + class Shirt < Base + # ... same as before + + # @style from the instance if present, + # or fetched from the API response if present, + # or a QA::Resource::Base::NoValueError is raised otherwise + attribute :style + + # If @main_fabric is not present, + # and if the API does not contain this field, this block will be + # used to construct the value based on the API response, and + # store the result in @main_fabric + attribute :main_fabric do + api_response.&dig(:materials, 0, 0) + end + + # ... same as before + end + end +end +``` + +**Notes on attributes precedence:** + +- resource instance variables have the highest precedence +- attributes from the API response take precedence over attributes from the + block (usually from Browser UI) +- attributes without a value will raise a `QA::Resource::Base::NoValueError` error + +## Creating resources in your tests + +To create a resource in your tests, you can call the `.fabricate!` method on +the resource class. +Note that if the resource class supports API fabrication, this will use this +fabrication by default. + +Here is an example that will use the API fabrication method under the hood +since it's supported by the `Shirt` resource class: + +```ruby +my_shirt = Resource::Shirt.fabricate! do |shirt| + shirt.name = 'my-shirt' +end + +expect(page).to have_text(my_shirt.name) # => "my-shirt" from the resource's instance variable +expect(page).to have_text(my_shirt.brand) # => "a-brand-new-brand" from the API response +expect(page).to have_text(my_shirt.style) # => "t-shirt" from the API response +expect(page).to have_text(my_shirt.main_fabric) # => "cotton" from the API response via the block +``` + +If you explicitly want to use the Browser UI fabrication method, you can call +the `.fabricate_via_browser_ui!` method instead: + +```ruby +my_shirt = Resource::Shirt.fabricate_via_browser_ui! do |shirt| + shirt.name = 'my-shirt' +end + +expect(page).to have_text(my_shirt.name) # => "my-shirt" from the resource's instance variable +expect(page).to have_text(my_shirt.brand) # => the brand name fetched from the `Page::Shirt::Show` page via the block +expect(page).to have_text(my_shirt.style) # => QA::Resource::Base::NoValueError will be raised because no API response nor a block is provided +expect(page).to have_text(my_shirt.main_fabric) # => QA::Resource::Base::NoValueError will be raised because no API response and the block didn't provide a value (because it's also based on the API response) +``` + +You can also explicitly use the API fabrication method, by calling the +`.fabricate_via_api!` method: + +```ruby +my_shirt = Resource::Shirt.fabricate_via_api! do |shirt| + shirt.name = 'my-shirt' +end +``` + +In this case, the result will be similar to calling +`Resource::Shirt.fabricate!`. + +## Where to ask for help? + +If you need more information, ask for help on `#quality` channel on Slack +(internal, GitLab Team only). + +If you are not a Team Member, and you still need help to contribute, please +open an issue in GitLab CE issue tracker with the `~QA` label. diff --git a/doc/development/testing_guide/end_to_end/style_guide.md b/doc/development/testing_guide/end_to_end/style_guide.md new file mode 100644 index 00000000000..0272e1810f2 --- /dev/null +++ b/doc/development/testing_guide/end_to_end/style_guide.md @@ -0,0 +1,99 @@ +# Style guide for writing end-to-end tests + +This document describes the conventions used at GitLab for writing End-to-end (E2E) tests using the GitLab QA project. + +## `click_` versus `go_to_` + +### When to use `click_`? + +When clicking in a single link to navigate, use `click_`. + +E.g.: + +```ruby +def click_ci_cd_pipelines + within_sidebar do + click_element :link_pipelines + end +end +``` + +From a testing perspective, if we want to check that clicking a link, or a button (a single interaction) is working as intended, we would want the test to read as: + +- Click a certain element +- Verify the action took place + +### When to use `go_to_`? + +When interacting with multiple elements to go to a page, use `go_to_`. + +E.g.: + +```ruby +def go_to_operations_environments + hover_operations do + within_submenu do + click_element(:operations_environments_link) + end + end +end +``` + +`go_to_` fits the definition of interacting with multiple elements very well given it's more of a meta-navigation action that includes multiple interactions. + +Notice that in the above example, before clicking the `:operations_environments_link`, another element is hovered over. + +> We can create these methods as helpers to abstract multi-step navigation. + +### Element naming convention + +When adding new elements to a page, it's important that we have a uniform element naming convention. + +We follow a simple formula roughly based on hungarian notation. + +*Formula*: `element :<descriptor>_<type>` + +- `descriptor`: The natural-language description of what the element is. On the login page, this could be `username`, or `password`. +- `type`: A physical control on the page that can be seen by a user. + - `_button` + - `_link` + - `_tab` + - `_dropdown` + - `_field` + - `_checkbox` + - `_radio` + - `_content` + +*Note: This list is a work in progress. This list will eventually be the end-all enumeration of all available types. + I.e., any element that does not end with something in this list is bad form.* + +#### Examples + +**Good** + +```ruby +view '...' do + element :edit_button + element :notes_tab + element :squash_checkbox + element :username_field + element :issue_title_content +end +``` + +**Bad** + +```ruby +view '...' do + # `_confirmation` should be `_field`. what sort of confirmation? a checkbox confirmation? no real way to disambiguate. + # an appropriate replacement would be `element :password_confirmation_field` + element :password_confirmation + + # `clone_options` is too vague. If it's a dropdown menu, it should be `clone_dropdown`. + # If it's a checkbox, it should be `clone_checkbox` + element :clone_options + + # how is this url being displayed? is it a textbox? a simple span? + element :ssh_clone_url +end +``` diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md index 9bd99e80357..4c9d1684c00 100644 --- a/doc/development/testing_guide/frontend_testing.md +++ b/doc/development/testing_guide/frontend_testing.md @@ -15,10 +15,8 @@ information on general testing practices at GitLab. ## Jest -GitLab has started to migrate tests to the [Jest](https://jestjs.io) -testing framework. You can read a [detailed evaluation](https://gitlab.com/gitlab-org/gitlab-ce/issues/49171) -of Jest compared to our use of Karma and Jasmine. In summary, it will allow us -to improve the performance and consistency of our frontend tests. +We have started to migrate frontend tests to the [Jest](https://jestjs.io) testing framework (see also the corresponding +[epic](https://gitlab.com/groups/gitlab-org/-/epics/895)). Jest tests can be found in `/spec/frontend` and `/ee/spec/frontend` in EE. @@ -26,6 +24,17 @@ It is not yet a requirement to use Jest. You can view the [epic](https://gitlab.com/groups/gitlab-org/-/epics/873) of issues we need to solve before being able to use Jest for all our needs. +### Differences to Karma + +- Jest runs in a Node.js environment, not in a browser. Support for running Jest tests in a browser [is planned](https://gitlab.com/gitlab-org/gitlab-ce/issues/58205). +- Because Jest runs in a Node.js environment, it uses [jsdom](https://github.com/jsdom/jsdom) by default. +- All calls to `setTimeout` and `setInterval` are mocked away. See also [Jest Timer Mocks](https://jestjs.io/docs/en/timer-mocks). +- `rewire` is not required because Jest supports mocking modules. See also [Manual Mocks](https://jestjs.io/docs/en/manual-mocks). +- The following will cause tests to fail in Jest: + - Unmocked requests. + - Unhandled Promise rejections. + - Calls to `console.warn`, including warnings from libraries like Vue. + ### Debugging Jest tests Running `yarn jest-debug` will run Jest in debug mode, allowing you to debug/inspect as described in the [Jest docs](https://jestjs.io/docs/en/troubleshooting#tests-are-failing-and-you-don-t-know-why). @@ -58,9 +67,6 @@ Remember that the performance of each test depends on the environment. GitLab uses the [Karma][karma] test runner with [Jasmine] as its test framework for our JavaScript unit and integration tests. -We generate HTML and JSON fixtures from backend views and controllers -using RSpec (see `spec/javascripts/fixtures/*.rb` for examples). -Fixtures are served during testing by the [jasmine-jquery][jasmine-jquery] plugin. JavaScript tests live in `spec/javascripts/`, matching the folder structure of `app/assets/javascripts/`: `app/assets/javascripts/behaviors/autosize.js` @@ -412,7 +418,7 @@ See this [section][vue-test]. For running the frontend tests, you need the following commands: -- `rake karma:fixtures` (re-)generates fixtures. +- `rake karma:fixtures` (re-)generates [fixtures](#frontend-test-fixtures). - `yarn test` executes the tests. As long as the fixtures don't change, `yarn test` is sufficient (and saves you some time). @@ -460,6 +466,48 @@ yarn karma -f 'spec/javascripts/ide/**/file_spec.js' Information on setting up and running RSpec integration tests with [Capybara] can be found in the [Testing Best Practices](best_practices.md). +## Frontend test fixtures + +Code that is added to HAML templates (in `app/views/`) or makes Ajax requests to the backend has tests that require HTML or JSON from the backend. +Fixtures for these tests are located at: + +- `spec/javascripts/fixtures/`, for running tests in CE. +- `ee/spec/javascripts/fixtures/`, for running tests in EE. + +Fixture files in: + +- The Karma test suite are served by [jasmine-jquery](https://github.com/velesin/jasmine-jquery). +- Jest use `spec/frontend/helpers/fixtures.js`. + +The following are examples of tests that work for both Karma and Jest: + +```javascript +it('makes a request', () => { + const responseBody = getJSONFixture('some/fixture.json'); // loads spec/javascripts/fixtures/some/fixture.json + axiosMock.onGet(endpoint).reply(200, responseBody); + + myButton.click(); + + // ... +}); + +it('uses some HTML element', () => { + loadFixtures('some/page.html'); // loads spec/javascripts/fixtures/some/page.html and adds it to the DOM + + const element = document.getElementById('#my-id'); + + // ... +}); +``` + +HTML and JSON fixtures are generated from backend views and controllers using RSpec (see `spec/javascripts/fixtures/*.rb`). + +For each fixture, the content of the `response` variable is stored in the output file. +This variable gets automagically set if the test is marked as `type: :request` or `type: :controller`. +Fixtures are regenerated using the `bin/rake karma:fixtures` command but you can also generate them individually, +for example `bin/rspec spec/javascripts/fixtures/merge_requests.rb`. +When creating a new fixture, it often makes sense to take a look at the corresponding tests for the endpoint in `(ee/)spec/controllers/` or `(ee/)spec/requests/`. + ## Gotchas ### RSpec errors due to JavaScript @@ -492,7 +540,6 @@ end ``` [jasmine-focus]: https://jasmine.github.io/2.5/focused_specs.html -[jasmine-jquery]: https://github.com/velesin/jasmine-jquery [karma]: http://karma-runner.github.io/ [vue-test]: https://docs.gitlab.com/ce/development/fe_guide/vue.html#testing-vue-components [rspec]: https://github.com/rspec/rspec-rails#feature-specs diff --git a/doc/development/testing_guide/index.md b/doc/development/testing_guide/index.md index ecad9ba48a3..93ee2a6371a 100644 --- a/doc/development/testing_guide/index.md +++ b/doc/development/testing_guide/index.md @@ -71,7 +71,7 @@ Everything you should know about how to test Rake tasks. --- -## [End-to-end tests](end_to_end_tests.md) +## [End-to-end tests](end_to_end/index.md) Everything you should know about how to run end-to-end tests using [GitLab QA][gitlab-qa] testing framework. diff --git a/doc/development/testing_guide/review_apps.md b/doc/development/testing_guide/review_apps.md index f34793c11f4..b5fde92a23d 100644 --- a/doc/development/testing_guide/review_apps.md +++ b/doc/development/testing_guide/review_apps.md @@ -113,7 +113,7 @@ You can also manually start the `review-qa-all`: it runs the full QA suite. On every [pipeline][gitlab-pipeline] in the `qa` stage, the `review-performance` job is automatically started: this job does basic browser performance testing using a -[Sitespeed.io Container](https://docs.gitlab.com/ee/user/project/merge_requests/browser_performance_testing.html). +[Sitespeed.io Container](../../user/project/merge_requests/browser_performance_testing.md). ## How to: @@ -203,7 +203,7 @@ find a way to limit it to only us.** [automated_cleanup.rb]: https://gitlab.com/gitlab-org/gitlab-ee/blob/master/scripts/review_apps/automated_cleanup.rb [Auto-DevOps.gitlab-ci.yml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml [gitlab-ci-yml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab-ci.yml -[gitlab-k8s-integration]: https://docs.gitlab.com/ee/user/project/clusters/index.html +[gitlab-k8s-integration]: ../../user/project/clusters/index.md [password-bug]: https://gitlab.com/gitlab-org/gitlab-ce/issues/53621 --- diff --git a/doc/development/testing_guide/smoke.md b/doc/development/testing_guide/smoke.md index 30d861d7d68..c9d3238fbe9 100644 --- a/doc/development/testing_guide/smoke.md +++ b/doc/development/testing_guide/smoke.md @@ -17,7 +17,7 @@ Currently, our suite consists of this basic functionality coverage: Smoke tests have the `:smoke` RSpec metadata. -See [End-to-end Testing](./end_to_end_tests.md) for more details about +See [End-to-end Testing](end_to_end/index.md) for more details about end-to-end tests. --- diff --git a/doc/development/testing_guide/testing_levels.md b/doc/development/testing_guide/testing_levels.md index 1fa6e38ea5a..e1ce4d3b7d1 100644 --- a/doc/development/testing_guide/testing_levels.md +++ b/doc/development/testing_guide/testing_levels.md @@ -4,12 +4,14 @@ _This diagram demonstrates the relative priority of each test type we use. `e2e` stands for end-to-end._ -As of 2019-04-16, we have the following distribution of tests per level: +As of 2019-05-01, we have the following distribution of tests per level: -- 67 black-box tests at the system level (aka end-to-end or QA tests) in CE, 98 in EE. This represents 0.3% of all the CE tests (0.3% in EE). -- 5,457 white-box tests at the system level (aka system or feature tests) in CE, 6,585 in EE. This represents 24.6% of all the CE tests (20.3% in EE). -- 8,298 integration tests in CE, 10,633 in EE: 0.3% of all the CE tests (0.3% in EE). This represents 37.2% of all the CE tests (32.8% in EE). -- 8,403 unit tests in CE, 15,090 in EE: 0.3% of all the CE tests (0.3% in EE). This represents 37.8% of all the CE tests (46.6% in EE). +| Test level | Community Edition | Enterprise Edition | Community + Enterprise Edition | +| --------- | ---------- | -------------- | ----- | +| Black-box tests at the system level (aka end-to-end or QA tests) | 68 (0.14%) | 31 (0.2%) | 99 (0.17%) | +| White-box tests at the system level (aka system or feature tests) | 5,471 (11.9%) | 969 (7.4%) | 6440 (10.9%) | +| Integration tests | 8,333 (18.2%) | 2,244 (17.2%) | 10,577 (17.9%) | +| Unit tests | 32,031 (69.7%) | 9,778 (75.1%) | 41,809 (71%) | ## Unit tests @@ -176,7 +178,7 @@ Every new feature should come with a [test plan]. | ---------- | -------------- | ----- | | `qa/qa/specs/features/` | [Capybara] + [RSpec] + Custom QA framework | Tests should be placed under their corresponding [Product category] | -> See [end-to-end tests](end_to_end_tests.md) for more information. +> See [end-to-end tests](end_to_end/index.md) for more information. Note that `qa/spec` contains unit tests of the QA framework itself, not to be confused with the application's [unit tests](#unit-tests) or diff --git a/doc/development/understanding_explain_plans.md b/doc/development/understanding_explain_plans.md index 01a0044f096..bfbb7be70e3 100644 --- a/doc/development/understanding_explain_plans.md +++ b/doc/development/understanding_explain_plans.md @@ -80,8 +80,9 @@ Planning time: 2.861 ms Execution time: 3428.596 ms ``` -For more information, refer to the official [EXPLAIN -documentation](https://www.postgresql.org/docs/current/static/sql-explain.html). +For more information, refer to the official +[`EXPLAIN` documentation](https://www.postgresql.org/docs/current/sql-explain.html) +and [using `EXPLAIN` guide](https://www.postgresql.org/docs/current/using-explain.html). ## Nodes @@ -653,9 +654,39 @@ and related tools such as: - <https://explain.depesz.com/> - <http://tatiyants.com/postgres-query-plan-visualization/> -GitLab employees can also use our chatops solution, available in Slack using the -`/chatops` slash command. You can use chatops to get a query plan by running the -following: + +## Producing query plans + +There are a few ways to get the output of a query plan. Of course you +can directly run the `EXPLAIN` query in the `psql` console, or you can +follow one of the other options below. + +### Rails console + +Using the [`activerecord-explain-analyze`](https://github.com/6/activerecord-explain-analyze) +you can directly generate the query plan from the Rails console: + +```ruby +pry(main)> require 'activerecord-explain-analyze' +=> true +pry(main)> Project.where('build_timeout > ?', 3600).explain(analyze: true) + Project Load (1.9ms) SELECT "projects".* FROM "projects" WHERE (build_timeout > 3600) + ↳ (pry):12 +=> EXPLAIN for: SELECT "projects".* FROM "projects" WHERE (build_timeout > 3600) +Seq Scan on public.projects (cost=0.00..2.17 rows=1 width=742) (actual time=0.040..0.041 rows=0 loops=1) + Output: id, name, path, description, created_at, updated_at, creator_id, namespace_id, ... + Filter: (projects.build_timeout > 3600) + Rows Removed by Filter: 14 + Buffers: shared hit=2 +Planning time: 0.411 ms +Execution time: 0.113 ms +``` + +### Chatops + +[GitLab employees can also use our chatops solution, available in Slack using the +`/chatops` slash command](chatops_on_gitlabcom.md). +You can use chatops to get a query plan by running the following: ``` /chatops run explain SELECT COUNT(*) FROM projects WHERE visibility_level IN (0, 20) @@ -674,3 +705,9 @@ For more information about the available options, run: ``` /chatops run explain --help ``` + +## Further reading + +A more extensive guide on understanding query plans can be found in +the [presentation](https://www.dalibo.org/_media/understanding_explain.pdf) +from [Dalibo.org](https://www.dalibo.org/en/). diff --git a/doc/git_hooks/git_hooks.md b/doc/git_hooks/git_hooks.md index 9b8ad1578a0..b251e58410a 100644 --- a/doc/git_hooks/git_hooks.md +++ b/doc/git_hooks/git_hooks.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/push_rules/push_rules.html' +redirect_to: '../push_rules/push_rules.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/push_rules/push_rules.html) +This document was moved to [another location](../push_rules/push_rules.md) diff --git a/doc/gitlab-basics/README.md b/doc/gitlab-basics/README.md index 28ba20bec09..00ac6d6b650 100644 --- a/doc/gitlab-basics/README.md +++ b/doc/gitlab-basics/README.md @@ -15,7 +15,7 @@ This documentation is split into the following groups: The following are guides to basic GitLab functionality: -- [Create and add your SSH Keys](create-your-ssh-keys.md), for enabling Git over SSH. +- [Create and add your SSH public key](create-your-ssh-keys.md), for enabling Git over SSH. - [Create a project](create-project.md), to start using GitLab. - [Create a group](../user/group/index.md#create-a-new-group), to combine and administer projects together. - [Create a branch](create-branch.md), to make changes to files stored in a project's repository. diff --git a/doc/gitlab-basics/create-project.md b/doc/gitlab-basics/create-project.md index 785e2ffb650..a9ae4fb23f9 100644 --- a/doc/gitlab-basics/create-project.md +++ b/doc/gitlab-basics/create-project.md @@ -16,7 +16,7 @@ To create a project in GitLab: - [Import a project](../user/project/import/index.md) from a different repository, if enabled on your GitLab instance. Contact your GitLab admin if this is unavailable. - - Run [CI/CD pipelines for external repositories](https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/index.html). **[PREMIUM]** + - Run [CI/CD pipelines for external repositories](../ci/ci_cd_for_external_repos/index.md). **[PREMIUM]** ## Blank projects diff --git a/doc/gitlab-basics/create-your-ssh-keys.md b/doc/gitlab-basics/create-your-ssh-keys.md index 8fecdc6948e..aac73d4c9c5 100644 --- a/doc/gitlab-basics/create-your-ssh-keys.md +++ b/doc/gitlab-basics/create-your-ssh-keys.md @@ -1,22 +1,22 @@ -# How to create your SSH keys +# Create and add your SSH public key -This topic describes how to create SSH keys. You do this to use Git over SSH instead of Git over HTTP. +This topic describes how to: -## Creating your SSH keys +- Create an SSH key pair to use with GitLab. +- Add the SSH public key to your GitLab account. -1. Go to your [command line](start-using-git.md) and follow the [instructions](../ssh/README.md) to generate your SSH key pair. -1. Log in to GitLab. -1. In the upper-right corner, click your avatar and select **Settings**. -1. On the **User Settings** menu, select **SSH keys**. -1. Paste the **public** key generated in the first step in the **Key** - text field. -1. Optionally, give it a descriptive title so that you can recognize it in the - event you add multiple keys. -1. Finally, click the **Add key** button to add it to GitLab. You will be able to see - its fingerprint, title, and creation date. +You do this to use [Git over SSH instead of Git over HTTP](https://git-scm.com/book/en/v2/Git-on-the-Server-The-Protocols). -  +## Creating your SSH key pair + +1. Go to your [command line](start-using-git.md). +1. Follow the [instructions](../ssh/README.md#generating-a-new-ssh-key-pair) to generate your SSH key pair. + +## Adding your SSH public key to GitLab + +To add the SSH public key to GitLab, +see [Adding an SSH key to your GitLab account](../ssh/README.md#adding-an-ssh-key-to-your-gitlab-account). NOTE: **Note:** Once you add a key, you cannot edit it. If the paste -didn't work, you need to remove the offending key and re-add it. +[didn't work](../ssh/README.md#testing-that-everything-is-set-up-correctly), you need to remove the key and re-add it. diff --git a/doc/gitlab-basics/img/profile_settings_ssh_keys_single_key.png b/doc/gitlab-basics/img/profile_settings_ssh_keys_single_key.png Binary files differdeleted file mode 100644 index 8014f1d5301..00000000000 --- a/doc/gitlab-basics/img/profile_settings_ssh_keys_single_key.png +++ /dev/null diff --git a/doc/gitlab-geo/configuration_source.md b/doc/gitlab-geo/configuration_source.md index f1aab86fadc..b46a2caea4a 100644 --- a/doc/gitlab-geo/configuration_source.md +++ b/doc/gitlab-geo/configuration_source.md @@ -1,5 +1,5 @@ --- -redirect_to: '../administration/geo/replication/configuration_source.md' +redirect_to: '../administration/geo/replication/configuration.md' --- -This document was moved to [another location](../administration/geo/replication/configuration_source.md). +This document was moved to [another location](../administration/geo/replication/configuration.md). diff --git a/doc/gitlab-geo/database_source.md b/doc/gitlab-geo/database_source.md index 3392d0f02c0..b4156dc4ec6 100644 --- a/doc/gitlab-geo/database_source.md +++ b/doc/gitlab-geo/database_source.md @@ -1,5 +1,5 @@ --- -redirect_to: '../administration/geo/replication/database_source.md' +redirect_to: '../administration/geo/replication/database.md' --- -This document was moved to [another location](../administration/geo/replication/database_source.md). +This document was moved to [another location](../administration/geo/replication/database.md). diff --git a/doc/install/README.md b/doc/install/README.md index 53778f7f0d3..9cc21412898 100644 --- a/doc/install/README.md +++ b/doc/install/README.md @@ -1,6 +1,7 @@ --- comments: false description: Read through the GitLab installation methods. +type: index --- # Installation **[CORE ONLY]** @@ -81,7 +82,7 @@ the above methods, provided the cloud provider supports it. - [Install on AWS](aws/index.md): Install Omnibus GitLab on AWS using the community AMIs that GitLab provides. - [Install GitLab on Google Cloud Platform](google_cloud_platform/index.md): Install Omnibus GitLab on a VM in GCP. - [Install GitLab on Azure](azure/index.md): Install Omnibus GitLab from Azure Marketplace. -- [Install GitLab on OpenShift](openshift_and_gitlab/index.md): Install GitLab using the Docker image on OpenShift. +- [Install GitLab on OpenShift](https://docs.gitlab.com/charts/installation/cloud/openshift.html): Install GitLab on OpenShift by using GitLab's Helm charts. - [Install GitLab on DC/OS](https://mesosphere.com/blog/gitlab-dcos/): Install GitLab on Mesosphere DC/OS via the [GitLab-Mesosphere integration](https://about.gitlab.com/2016/09/16/announcing-gitlab-and-mesosphere/). - [Install GitLab on DigitalOcean](https://about.gitlab.com/2016/04/27/getting-started-with-gitlab-and-digitalocean/): Install Omnibus GitLab on DigitalOcean. - _Testing only!_ [DigitalOcean and Docker Machine](digitaloceandocker.md): diff --git a/doc/install/aws/index.md b/doc/install/aws/index.md index 0000e03f1d7..73eaf758923 100644 --- a/doc/install/aws/index.md +++ b/doc/install/aws/index.md @@ -1,3 +1,7 @@ +--- +type: howto +--- + # Installing GitLab HA on Amazon Web Services (AWS) This page offers a walkthrough of a common HA (Highly Available) configuration @@ -383,7 +387,7 @@ after the instance is created. CAUTION: **Caution:** We **do not** recommend using the AWS Elastic File System (EFS), as it can result -in [significantly degraded performance](../../administration/high_availability/nfs.html#avoid-using-awss-elastic-file-system-efs). +in [significantly degraded performance](../../administration/high_availability/nfs.md#avoid-using-awss-elastic-file-system-efs). ### Configure security group @@ -649,12 +653,24 @@ Have a read through these other resources and feel free to [open an issue](https://gitlab.com/gitlab-org/gitlab-ce/issues/new) to request additional material: -- [GitLab High Availability](https://docs.gitlab.com/ee/administration/high_availability/): +- [GitLab High Availability](../../administration/high_availability/README.md): GitLab supports several different types of clustering and high-availability. -- [Geo replication](https://docs.gitlab.com/ee/administration/geo/replication/): +- [Geo replication](../../administration/geo/replication/index.md): Geo is the solution for widely distributed development teams. - [Omnibus GitLab](https://docs.gitlab.com/omnibus/) - Everything you need to know about administering your GitLab instance. -- [Upload a license](https://docs.gitlab.com/ee/user/admin_area/license.html): +- [Upload a license](../../user/admin_area/license.md): Activate all GitLab Enterprise Edition functionality with a license. - [Pricing](https://about.gitlab.com/pricing): Pricing for the different tiers. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/install/azure/index.md b/doc/install/azure/index.md index fa5be1d30f9..b1f79893baf 100644 --- a/doc/install/azure/index.md +++ b/doc/install/azure/index.md @@ -1,14 +1,10 @@ --- -description: 'Learn how to spin up a -pre-configured GitLab VM on Microsoft Azure and have your very own private GitLab instance up and running in around 30 minutes.' +description: 'Learn how to spin up a pre-configured GitLab VM on Microsoft Azure.' +type: howto --- # Install GitLab on Microsoft Azure -> _This article was originally written by Dave Wentzel and [published on the GitLab Blog][Original-Blog-Post]._ -> -> _Ported to the GitLab documentation and updated on 2017-08-24 by [Ian Scorer](https://gitlab.com/iscorer)._ - Azure is Microsoft's business cloud and GitLab is a pre-configured offering on the Azure Marketplace. Hopefully, you aren't surprised to hear that Microsoft and Azure have embraced open source software like Ubuntu, Red Hat Enterprise Linux, and of course - GitLab! This means that you can spin up a @@ -444,3 +440,15 @@ Check out our other [Technical Articles][GitLab-Technical-Articles] or browse th [SSH]: https://en.wikipedia.org/wiki/Secure_Shell [PuTTY]: http://www.putty.org/ [Using-SSH-In-Putty]: https://mediatemple.net/community/products/dv/204404604/using-ssh-in-putty- + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/install/database_mysql.md b/doc/install/database_mysql.md index e89846107b6..cbb3b766b4e 100644 --- a/doc/install/database_mysql.md +++ b/doc/install/database_mysql.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Database MySQL NOTE: **Note:** @@ -301,3 +305,15 @@ Details can be found in the [PostgreSQL][postgres-text-type] and [postgres-text-type]: http://www.postgresql.org/docs/9.2/static/datatype-character.html [ce-38152]: https://gitlab.com/gitlab-org/gitlab-ce/issues/38152 + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/install/digitaloceandocker.md b/doc/install/digitaloceandocker.md index d67695d75b4..63bb941ad47 100644 --- a/doc/install/digitaloceandocker.md +++ b/doc/install/digitaloceandocker.md @@ -1,8 +1,11 @@ +--- +type: howto +--- + # Digital Ocean and Docker Machine test environment -CAUTION: **Caution:** -This guide is for quickly testing different versions of GitLab and not recommended for ease of -future upgrades or keeping the data you create. +This guide is for quickly testing different versions of GitLab and not +recommended for ease of future upgrades or keeping the data you create. ## Initial setup @@ -28,7 +31,7 @@ locally on either macOS or Linux. NOTE: **Note:** The rest of the steps are identical for macOS and Linux. -### Create new docker host +## Create new docker host 1. Login to Digital Ocean. 1. Generate a new API token at <https://cloud.digitalocean.com/settings/api/tokens>. @@ -60,9 +63,9 @@ The rest of the steps are identical for macOS and Linux. Resource: <https://docs.docker.com/machine/drivers/digital-ocean/>. -### Creating GitLab test instance +## Creating GitLab test instance -#### Connect your shell to the new machine +### Connect your shell to the new machine In this example we'll create a GitLab EE 8.10.8 instance. @@ -74,7 +77,7 @@ eval "$(docker-machine env gitlab-test-env-do)" You can add this to your `~/.bash_profile` file to ensure the `docker` client uses the `gitlab-test-env-do` docker host -#### Create new GitLab container +### Create new GitLab container - HTTP port: `8888` - SSH port: `2222` @@ -83,7 +86,7 @@ You can add this to your `~/.bash_profile` file to ensure the `docker` client us - Container name: `gitlab-test-8.10` - GitLab version: **EE** `8.10.8-ee.0` -##### Set up container settings +#### Set up container settings ```sh export SSH_PORT=2222 @@ -92,7 +95,7 @@ export VERSION=8.10.8-ee.0 export NAME=gitlab-test-8.10 ``` -##### Create container +#### Create container ```sh docker run --detach \ @@ -103,9 +106,9 @@ docker run --detach \ gitlab/gitlab-ee:$VERSION ``` -#### Connect to the GitLab container +### Connect to the GitLab container -##### Retrieve the docker host IP +#### Retrieve the docker host IP ```sh docker-machine ip gitlab-test-env-do @@ -114,7 +117,7 @@ docker-machine ip gitlab-test-env-do Browse to: <http://192.168.151.134:8888/>. -##### Execute interactive shell/edit configuration +#### Execute interactive shell/edit configuration ```sh docker exec -it $NAME /bin/bash @@ -126,8 +129,20 @@ root@192:/# vi /etc/gitlab/gitlab.rb root@192:/# gitlab-ctl reconfigure ``` -#### Resources +### Resources - <https://docs.gitlab.com/omnibus/docker/>. - <https://docs.docker.com/machine/get-started/>. - <https://docs.docker.com/machine/reference/ip/>. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/install/docker.md b/doc/install/docker.md index 4568a949b0a..06da65189ba 100644 --- a/doc/install/docker.md +++ b/doc/install/docker.md @@ -1,3 +1,7 @@ +--- +type: index +--- + # Install GitLab with Docker [Docker](https://www.docker.com) and container technology have been revolutionizing the software world for the past few years. They combine the performance and efficiency of native execution with the abstraction, security, and immutability of virtualization. @@ -16,4 +20,4 @@ A [complete usage guide](https://docs.gitlab.com/omnibus/docker/) to these image ## Cloud native images -GitLab is also working towards a [cloud native set of containers](https://gitlab.com/charts/helm.gitlab.io#docker-container-images), with a single image for each component service. We intend for these images to eventually replace the [Omnibus GitLab based images](#omnibus-gitlab-based-images). +GitLab is also working towards a [cloud native set of containers](https://docs.gitlab.com/charts/), with a single image for each component service. We intend for these images to eventually replace the [Omnibus GitLab based images](#omnibus-gitlab-based-images). diff --git a/doc/install/google-protobuf.md b/doc/install/google-protobuf.md index a531b4519b3..434817c48cb 100644 --- a/doc/install/google-protobuf.md +++ b/doc/install/google-protobuf.md @@ -1,26 +1,5 @@ -# Installing a locally compiled google-protobuf gem +--- +redirect_to: 'installation.md#google-protobuf-loaderror-libx86_64-linux-gnulibcso6-version-glibc_214-not-found' +--- -First we must find the exact version of google-protobuf that your -GitLab installation requires. - - cd /home/git/gitlab - - # Only one of the following two commands will print something. It - # will look like: * google-protobuf (3.2.0) - bundle list | grep google-protobuf - bundle check | grep google-protobuf - -Below we use `3.2.0` as an example. Replace it with the version number -you found above. - - cd /home/git/gitlab - sudo -u git -H gem install google-protobuf --version 3.2.0 --platform ruby - -Finally, you can test whether google-protobuf loads correctly. The -following should print 'OK'. - - sudo -u git -H bundle exec ruby -rgoogle/protobuf -e 'puts :OK' - -If the `gem install` command fails you may need to install developer -tools. On Debian: `apt-get install build-essential libgmp-dev`, on -Centos/RedHat `yum groupinstall 'Development Tools'`. +This document was moved to [another location](installation.md#google-protobuf-loaderror-libx86_64-linux-gnulibcso6-version-glibc_214-not-found). diff --git a/doc/install/google_cloud_platform/img/gcp_landing.png b/doc/install/google_cloud_platform/img/gcp_landing.png Binary files differdeleted file mode 100644 index 92a9873728c..00000000000 --- a/doc/install/google_cloud_platform/img/gcp_landing.png +++ /dev/null diff --git a/doc/install/google_cloud_platform/index.md b/doc/install/google_cloud_platform/index.md index 242ad70ae82..77c61acbfd4 100644 --- a/doc/install/google_cloud_platform/index.md +++ b/doc/install/google_cloud_platform/index.md @@ -1,14 +1,13 @@ --- description: 'Learn how to install a GitLab instance on Google Cloud Platform.' +type: howto --- # Installing GitLab on Google Cloud Platform - +This guide will help you install GitLab on a [Google Cloud Platform (GCP)][gcp] instance. -Getting started with GitLab on a [Google Cloud Platform (GCP)][gcp] instance is quick and easy. - -NOTE: **Note:** +NOTE: **Alternative installation method:** Google provides a whitepaper for [deploying production-ready GitLab on Google Kubernetes Engine](https://cloud.google.com/solutions/deploying-production-ready-gitlab-on-gke), including all steps and external resource configuration. These are an alternative to using a GCP VM, and use @@ -26,10 +25,9 @@ Once you have performed those two steps, you can [create a VM](#creating-the-vm) ## Creating the VM -To deploy GitLab on GCP you need to follow five simple steps: - -1. Go to <https://console.cloud.google.com/compute/instances> and login with your Google credentials. +To deploy GitLab on GCP you first need to create a virtual machine: +1. Go to <https://console.cloud.google.com/compute/instances> and log in with your Google credentials. 1. Click on **Create**  @@ -142,3 +140,15 @@ Kerberos, etc. Here are some documents you might be interested in reading: [ssh]: https://cloud.google.com/compute/docs/instances/connecting-to-instance "Connecting to Linux Instances" [omni-smtp]: https://docs.gitlab.com/omnibus/settings/smtp.html#smtp-settings "Omnibus GitLab SMTP settings" [omni-ssl]: https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https "Omnibus GitLab enable HTTPS" + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/install/installation.md b/doc/install/installation.md index 6b24dc8abdc..10436a15a9e 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -1,5 +1,28 @@ +--- +type: howto +--- + # Installation from source +This is the official installation guide to set up a production GitLab server +using the source files. To set up a **development installation** or for many +other installation options, see the [main installation page](index.md). +It was created for and tested on **Debian/Ubuntu** operating systems. +Read [requirements.md](requirements.md) for hardware and operating system requirements. +If you want to install on RHEL/CentOS, we recommend using the +[Omnibus packages](https://about.gitlab.com/downloads/). + +This guide is long because it covers many cases and includes all commands you +need, this is [one of the few installation scripts that actually works out of the box](https://twitter.com/robinvdvleuten/status/424163226532986880). +The following steps have been known to work. **Use caution when you deviate** +from this guide. Make sure you don't violate any assumptions GitLab makes about +its environment. For example, many people run into permission problems because +they changed the location of directories or run services as the wrong user. + +If you find a bug/error in this guide, **submit a merge request** +following the +[contributing guide](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md). + ## Consider the Omnibus package installation Since an installation from source is a lot of work and error prone we strongly recommend the fast and reliable [Omnibus package installation](https://about.gitlab.com/downloads/) (deb/rpm). @@ -12,26 +35,42 @@ After this termination Runit will detect Sidekiq is not running and will start i Since installations from source don't use Runit for process supervision, Sidekiq can't be terminated and its memory usage will grow over time. -## Select Version to Install +## Select version to install Make sure you view [this installation guide](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/install/installation.md) from the branch (version) of GitLab you would like to install (e.g., `11-7-stable`). You can select the branch in the version dropdown in the top left corner of GitLab (below the menu bar). If the highest number stable branch is unclear, check the [GitLab blog](https://about.gitlab.com/blog/) for installation guide links by version. -## Important Notes +## GitLab directory structure -This guide is long because it covers many cases and includes all commands you need, this is [one of the few installation scripts that actually works out of the box](https://twitter.com/robinvdvleuten/status/424163226532986880). +This is the main directory structure you will end up with following the instructions +of this page: -This installation guide was created for and tested on **Debian/Ubuntu** operating systems. Read [requirements.md](requirements.md) for hardware and operating system requirements. If you want to install on RHEL/CentOS, we recommend using the [Omnibus packages](https://about.gitlab.com/downloads/). +``` +|-- home +| |-- git +| |-- .ssh +| |-- gitlab +| |-- gitlab-shell +| |-- repositories +``` -This is the official installation guide to set up a production server. To set up a **development installation** or for many other installation options, see [the installation section of the README](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/README.md#installation). +- `/home/git/.ssh` - Contains OpenSSH settings. Specifically the `authorized_keys` + file managed by gitlab-shell. +- `/home/git/gitlab` - GitLab core software. +- `/home/git/gitlab-shell` - Core add-on component of GitLab. Maintains SSH + cloning and other functionality. +- `/home/git/repositories` - Bare repositories for all projects organized by + namespace. This is where the git repositories which are pushed/pulled are + maintained for all projects. **This area contains critical data for projects. + [Keep a backup](../raketasks/backup_restore.md).** -The following steps have been known to work. **Use caution when you deviate** from this guide. Make sure you don't violate any assumptions GitLab makes about its environment. For example, many people run into permission problems because they changed the location of directories or run services as the wrong user. +NOTE: **Note:** +The default locations for repositories can be configured in `config/gitlab.yml` +of GitLab and `config.yml` of gitlab-shell. -If you find a bug/error in this guide, **submit a merge request** -following the -[contributing guide](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md). +For a more in-depth overview, see the [GitLab architecture doc](../development/architecture.md). ## Overview @@ -324,9 +363,7 @@ Redis 2.8 with: sudo apt-get install redis-server ``` -If you are using Debian 7 or Ubuntu 12.04, follow the special documentation -on [an alternate Redis installation](redis.md). Once done, follow the rest of -the guide here. +Once done, you can configure Redis: ```sh # Configure redis to use sockets @@ -930,8 +967,50 @@ and correctly [configured Nginx](#site-configuration). ### google-protobuf "LoadError: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.14' not found" This can happen on some platforms for some versions of the -google-protobuf gem. The workaround is to [install a source-only -version of this gem](google-protobuf.md). +`google-protobuf` gem. The workaround is to install a source-only +version of this gem. + +First, you must find the exact version of `google-protobuf` that your +GitLab installation requires: + +```sh +cd /home/git/gitlab + +# Only one of the following two commands will print something. It +# will look like: * google-protobuf (3.2.0) +bundle list | grep google-protobuf +bundle check | grep google-protobuf +``` + +Below, `3.2.0` is used as an example. Replace it with the version number +you found above: + +```sh +cd /home/git/gitlab +sudo -u git -H gem install google-protobuf --version 3.2.0 --platform ruby +``` + +Finally, you can test whether `google-protobuf` loads correctly. The +following should print 'OK'. + +```sh +sudo -u git -H bundle exec ruby -rgoogle/protobuf -e 'puts :OK' +``` + +If the `gem install` command fails, you may need to install the developer +tools of your OS. + +On Debian/Ubuntu: + +```sh +sudo apt-get install build-essential libgmp-dev +``` + +On RedHat/CentOS: + +```sh +sudo yum groupinstall 'Development Tools' +``` [RVM]: https://rvm.io/ "RVM Homepage" [rbenv]: https://github.com/sstephenson/rbenv "rbenv on GitHub" diff --git a/doc/install/kubernetes/index.md b/doc/install/kubernetes/index.md index 7312bf2d4f7..43655767002 100644 --- a/doc/install/kubernetes/index.md +++ b/doc/install/kubernetes/index.md @@ -1,41 +1,5 @@ --- -description: 'Read through the different methods to deploy GitLab on Kubernetes.' +redirect_to: https://docs.gitlab.com/charts/ --- -# Installing GitLab on Kubernetes - -NOTE: **Kubernetes experience required:** -Our Helm charts are recommended for those who are familiar with Kubernetes. -If you're not sure if Kubernetes is for you, our -[Omnibus GitLab packages](../README.md#installing-gitlab-using-the-omnibus-gitlab-package-recommended) -are mature, scalable, support [high availability](../../administration/high_availability/README.md) -and are used today on GitLab.com. -It is not necessary to have GitLab installed on Kubernetes in order to use [GitLab Kubernetes integration](https://docs.gitlab.com/ee/user/project/clusters/index.html). - -The easiest method to deploy GitLab on [Kubernetes](https://kubernetes.io/) is -to take advantage of GitLab's Helm charts. [Helm](https://github.com/kubernetes/helm/blob/master/README.md) -is a package management tool for Kubernetes, allowing apps to be easily managed via their -Charts. A [Chart](https://github.com/kubernetes/charts) is a detailed description -of the application including how it should be deployed, upgraded, and configured. - -## GitLab Chart - -This chart contains all the required components to get started, and can scale to -large deployments. It offers a number of benefits, among others: - -- Horizontal scaling of individual components. -- No requirement for shared storage to scale. -- Containers do not need `root` permissions. -- Automatic SSL with Let's Encrypt. -- An unprivileged GitLab Runner. - -Learn more about the [GitLab chart](https://docs.gitlab.com/charts/). - -## GitLab Runner Chart - -If you already have a GitLab instance running, inside or outside of Kubernetes, -and you'd like to leverage the Runner's -[Kubernetes capabilities](https://docs.gitlab.com/runner/executors/kubernetes.html), -it can be deployed with the GitLab Runner chart. - -Learn more about the [GitLab Runner chart](https://docs.gitlab.com/runner/install/kubernetes.html). +This document was moved to [another location](https://docs.gitlab.com/charts/). diff --git a/doc/install/openshift_and_gitlab/index.md b/doc/install/openshift_and_gitlab/index.md index 77bd9e9f7a9..18981c43464 100644 --- a/doc/install/openshift_and_gitlab/index.md +++ b/doc/install/openshift_and_gitlab/index.md @@ -1,8 +1,5 @@ --- -author: Achilleas Pipinellis -author_gitlab: axil -level: intermediary -article_type: tutorial +type: howto date: 2016-06-28 --- diff --git a/doc/install/pivotal/index.md b/doc/install/pivotal/index.md index 896e01ad975..f068572f1e9 100644 --- a/doc/install/pivotal/index.md +++ b/doc/install/pivotal/index.md @@ -1,45 +1,12 @@ # GitLab Pivotal Tile **[PREMIUM ONLY]** -> Introduced in [GitLab Premium][eep] 8.2. - -Easily deploy GitLab as a pre-configured appliance using Ops Manager (BOSH) for -[Pivotal Cloud Foundry][pcf]. - -## Overview - -Enterprise admins want their development toolkit to be more customizable, more -integrated, and more secure. With Pivotal Cloud Foundry, GitLab is installed and -scales easily in a highly available environment. - -The upgrades are pain-free and well tested. All it takes is upload the new tile -and click a button to begin the upgrade process. - -## Use cases - -- You want a highly available deployment with minimal effort. Scale horizontally - as your user base grows. - -## Features - -The GitLab Pivotal Tile is based on [GitLab Premium][eep] and includes nearly all of its features. The features in Premium but _not_ supported on the Tile are: - -* PostgreSQL -* Pages -* Geo -* Registry -* Mattermost -* Subgroups -* Elasticsearch -* Service Desk -* OAuth & Kerberos Authentication - -## Installing GitLab with Pivotal - -The product information and installation documentation is hosted on Pivotal's -website: - -- [Product page](https://network.pivotal.io/products/p-gitlab/) -- [Documentation](https://docs.pivotal.io/partners/gitlab/index.html) - -[eep]: https://about.gitlab.com/pricing/ -[pcf]: https://pivotal.io/platform +CAUTION: **Discontinued:** +As of September 13, 2017, the GitLab Enterprise Plus for Pivotal Cloud Foundry +tile on Pivotal Network has reached its End of Availability (“EoA”) and is no +longer available for download or sale through Pivotal. Current customers with +active subscriptions will continue to receive support from GitLab through their +subscription term. Pivotal and GitLab are collaborating on creating a new +Kubernetes-based tile for the Pivotal Container Service. Please contact GitLab +support with any questions regarding GitLab Enterprise Plus for Pivotal Cloud Foundry. + +Original article: <https://docs.pivotal.io/partners/gitlab/index.html>. diff --git a/doc/install/redis.md b/doc/install/redis.md index 4075e6283d0..cff5c2f2611 100644 --- a/doc/install/redis.md +++ b/doc/install/redis.md @@ -1,60 +1,5 @@ -# Install Redis on old distributions +--- +redirect_to: installation.md#7-redis +--- -GitLab requires at least Redis 2.8. The following guide is for Debian 7 and -Ubuntu 12.04. If you are using Debian 8 or Ubuntu 14.04 and up, follow the -[installation guide](installation.md). - -## Install Redis 2.8 in Debian 7 - -Redis 2.8 is included in the Debian Wheezy [backports] repository. - -1. Edit `/etc/apt/sources.list` and add the following line: - - ``` - deb http://http.debian.net/debian wheezy-backports main - ``` - -1. Update the repositories: - - ``` - sudo apt-get update - ``` - -1. Install `redis-server`: - - ``` - sudo apt-get -t wheezy-backports install redis-server - ``` - -1. Follow the rest of the [installation guide](installation.md). - -## Install Redis 2.8 in Ubuntu 12.04 - -We will [use a PPA](https://launchpad.net/~chris-lea/+archive/ubuntu/redis-server) -to install a recent version of Redis. - -1. Install the PPA repository: - - ``` - sudo add-apt-repository ppa:chris-lea/redis-server - ``` - - Your system will now fetch the PPA's key. This enables your Ubuntu system to - verify that the packages in the PPA have not been interfered with since they - were built. - -1. Update the repositories: - - ``` - sudo apt-get update - ``` - -1. Install `redis-server`: - - ``` - sudo apt-get install redis-server - ``` - -1. Follow the rest of the [installation guide](installation.md). - -[backports]: http://backports.debian.org/Instructions/ "Debian backports website" +This document was moved to [another location](installation.md#7-redis). diff --git a/doc/install/relative_url.md b/doc/install/relative_url.md index 5f129fd3bd1..96b7d0f3648 100644 --- a/doc/install/relative_url.md +++ b/doc/install/relative_url.md @@ -1,18 +1,19 @@ +--- +type: reference +--- + # Install GitLab under a relative URL -NOTE: **Note:** +While it is recommended to install GitLab on its own (sub)domain, sometimes +this is not possible due to a variety of reasons. In that case, GitLab can also +be installed under a relative URL, for example `https://example.com/gitlab`. + This document describes how to run GitLab under a relative URL for installations from source. If you are using an Omnibus package, [the steps are different][omnibus-rel]. Use this guide along with the [installation guide](installation.md) if you are installing GitLab for the first time. ---- - -While it is recommended to install GitLab on its own (sub)domain, sometimes -this is not possible due to a variety of reasons. In that case, GitLab can also -be installed under a relative URL, for example `https://example.com/gitlab`. - There is no limit to how deeply nested the relative URL can be. For example you could serve GitLab under `/foo/bar/gitlab/git` without any issues. @@ -20,8 +21,6 @@ Note that by changing the URL on an existing GitLab installation, all remote URLs will change, so you'll have to manually edit them in any local repository that points to your GitLab instance. ---- - The TL;DR list of configuration files that you need to change in order to serve GitLab under a relative URL is: @@ -126,3 +125,15 @@ To disable the relative URL: [omnibus-rel]: http://docs.gitlab.com/omnibus/settings/configuration.html#configuring-a-relative-url-for-gitlab "How to set up relative URL in Omnibus GitLab" [restart gitlab]: ../administration/restart_gitlab.md#installations-from-source "How to restart GitLab" + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/install/requirements.md b/doc/install/requirements.md index f6a52205a0e..107d48fb90c 100644 --- a/doc/install/requirements.md +++ b/doc/install/requirements.md @@ -1,5 +1,12 @@ +--- +type: reference +--- + # Requirements +This page includes useful information on the supported Operating Systems as well +as the hardware requirements that are needed to install and use GitLab. + ## Operating Systems ### Supported Unix distributions @@ -12,7 +19,7 @@ - Scientific Linux (please use the CentOS packages and instructions) - Oracle Linux (please use the CentOS packages and instructions) -For the installations options please see [the installation page on the GitLab website](https://about.gitlab.com/installation/). +For the installations options, see [the main installation page](README.md). ### Unsupported Unix distributions @@ -105,9 +112,9 @@ features of GitLab work with MySQL/MariaDB: 1. MySQL support for subgroups was [dropped with GitLab 9.3][post]. See [issue #30472][30472] for more information. -1. Geo does [not support MySQL](https://docs.gitlab.com/ee/administration/geo/replication/database.html). This means no supported Disaster Recovery solution if using MySQL. **[PREMIUM ONLY]** +1. Geo does [not support MySQL](../administration/geo/replication/database.md). This means no supported Disaster Recovery solution if using MySQL. **[PREMIUM ONLY]** 1. [Zero downtime migrations](../update/README.md#upgrading-without-downtime) do not work with MySQL. -1. [Database load balancing](https://docs.gitlab.com/ee/administration/database_load_balancing.html) is +1. [Database load balancing](../administration/database_load_balancing.md) is supported only for PostgreSQL. **[PREMIUM ONLY]** 1. GitLab [optimizes the loading of dashboard events](https://gitlab.com/gitlab-org/gitlab-ce/issues/31806) using [PostgreSQL LATERAL JOINs](https://blog.heapanalytics.com/postgresqls-powerful-new-join-type-lateral/). 1. In general, SQL optimized for PostgreSQL may run much slower in MySQL due to @@ -143,14 +150,14 @@ On some systems you may need to install an additional package (e.g. #### Additional requirements for GitLab Geo -If you are using [GitLab Geo](https://docs.gitlab.com/ee/development/geo.html): +If you are using [GitLab Geo](../development/geo.md): - We strongly recommend running Omnibus-managed instances as they are actively developed and tested. We aim to be compatible with most external (not managed by Omnibus) databases (for example, AWS RDS) but we do not guarantee compatibility. - The - [tracking database](https://docs.gitlab.com/ee/development/geo.html#geo-tracking-database) + [tracking database](../development/geo.md#using-the-tracking-database) requires the [postgres_fdw](https://www.postgresql.org/docs/9.6/static/postgres-fdw.html) extension. @@ -201,12 +208,23 @@ you decide to run GitLab Runner and the GitLab Rails application on the same machine. It is also not safe to install everything on a single machine, because of the -[security reasons](https://docs.gitlab.com/runner/security/) -- especially when you plan to use shell executor with GitLab +[security reasons](https://docs.gitlab.com/runner/security/), especially when you plan to use shell executor with GitLab Runner. We recommend using a separate machine for each GitLab Runner, if you plan to use the CI features. +The GitLab Runner server requirements depend on: + +- The type of [executor](https://docs.gitlab.com/runner/executors/) you configured on GitLab Runner. +- Resources required to run build jobs. +- Job concurrency settings. + +Since the nature of the jobs varies for each use case, you will need to experiment by adjusting the job concurrency to get the optimum setting. + +For reference, GitLab.com's [auto-scaling shared runner](../user/gitlab_com/index.md#shared-runners) is configured so that a **single job** will run in a **single instance** with: + +- 1vCPU. +- 3.75GB of RAM. ## Supported web browsers @@ -225,3 +243,15 @@ Each time a new browser version is released, we begin supporting that version an NOTE: **Note:** We do not support running GitLab with JavaScript disabled in the browser and have no plans of supporting that in the future because we have features such as Issue Boards which require JavaScript extensively. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/install/structure.md b/doc/install/structure.md index 8fc6ab4ab2f..87ef11c60fe 100644 --- a/doc/install/structure.md +++ b/doc/install/structure.md @@ -1,19 +1,5 @@ -# GitLab directory structure +--- +redirect_to: installation.md#gitlab-directory-structure +--- -This is the directory structure you will end up with following the instructions in the Installation Guide. - - |-- home - | |-- git - | |-- .ssh - | |-- gitlab - | |-- gitlab-shell - | |-- repositories - -- `/home/git/.ssh` - contains openssh settings. Specifically the `authorized_keys` file managed by gitlab-shell. -- `/home/git/gitlab` - GitLab core software. -- `/home/git/gitlab-shell` - Core add-on component of GitLab. Maintains SSH cloning and other functionality. -- `/home/git/repositories` - bare repositories for all projects organized by namespace. This is where the git repositories which are pushed/pulled are maintained for all projects. **This area is critical data for projects. [Keep a backup](../raketasks/backup_restore.md).** - -*Note: the default locations for repositories can be configured in `config/gitlab.yml` of GitLab and `config.yml` of gitlab-shell.* - -To see a more in-depth overview see the [GitLab architecture doc](../development/architecture.md). +This page was moved to [another location](#installation.md#gitlab-directory-structure). diff --git a/doc/integration/elasticsearch.md b/doc/integration/elasticsearch.md index d1d12dfd064..57a5a42fbed 100644 --- a/doc/integration/elasticsearch.md +++ b/doc/integration/elasticsearch.md @@ -8,8 +8,8 @@ This document describes how to set up Elasticsearch with GitLab. Once enabled, you'll have the benefit of fast search response times and the advantage of two special searches: -- [Advanced Global Search](https://docs.gitlab.com/ee/user/search/advanced_global_search.html) -- [Advanced Syntax Search](https://docs.gitlab.com/ee/user/search/advanced_search_syntax.html) +- [Advanced Global Search](../user/search/advanced_global_search.md) +- [Advanced Syntax Search](../user/search/advanced_search_syntax.md) ## Version Requirements <!-- Please remember to update ee/lib/system_check/app/elasticsearch_check.rb if this changes --> @@ -192,9 +192,6 @@ Performing asynchronous indexing, as this will describe, will generate a lot of Make sure to prepare for this task by either [Horizontally Scaling](../administration/high_availability/README.md#basic-scaling) or creating [extra sidekiq processes](../administration/operations/extra_sidekiq_processes.md) -NOTE: **Note**: -After indexing the repositories asynchronously, you **MUST** index the database to be able to search. - Configure Elasticsearch's host and port in **Admin > Settings > Integrations**. Then create empty indexes using one of the following commands: ```sh @@ -217,78 +214,49 @@ curl --request PUT localhost:9200/gitlab-production/_settings --data '{ } }' ``` -Then enable Elasticsearch indexing and run repository indexing tasks: +Then enable Elasticsearch indexing and run project indexing tasks: ```sh # Omnibus installations -sudo gitlab-rake gitlab:elastic:index_repositories_async +sudo gitlab-rake gitlab:elastic:index_projects # Installations from source -bundle exec rake gitlab:elastic:index_repositories_async RAILS_ENV=production +bundle exec rake gitlab:elastic:index_projects RAILS_ENV=production ``` -This enqueues a number of Sidekiq jobs to index your existing repositories. -You can view the jobs in the admin panel (they are placed in the `elastic_batch_project_indexer`) +This enqueues a Sidekiq job for each project that needs to be indexed. +You can view the jobs in the admin panel (they are placed in the `elastic_indexer` queue), or you can query indexing status using a rake task: ```sh # Omnibus installations -sudo gitlab-rake gitlab:elastic:index_repositories_status +sudo gitlab-rake gitlab:elastic:index_projects_status # Installations from source -bundle exec rake gitlab:elastic:index_repositories_status RAILS_ENV=production +bundle exec rake gitlab:elastic:index_projects_status RAILS_ENV=production Indexing is 65.55% complete (6555/10000 projects) ``` -By default, one job is created for every 300 projects. For large numbers of -projects, you may wish to increase the batch size, by setting the `BATCH` -environment variable. - -You can also run the initial indexing synchronously - this is most useful if -you have a small number of projects or need finer-grained control over indexing -than Sidekiq permits: +If you want to limit the index to a range of projects you can provide the +`ID_FROM` and `ID_TO` parameters: ```sh # Omnibus installations -sudo gitlab-rake gitlab:elastic:index_repositories +sudo gitlab-rake gitlab:elastic:index_projects ID_FROM=1001 ID_TO=2000 # Installations from source -bundle exec rake gitlab:elastic:index_repositories RAILS_ENV=production -``` - -It might take a while depending on how big your Git repositories are. - -If you want to run several tasks in parallel (probably in separate terminal -windows) you can provide the `ID_FROM` and `ID_TO` parameters: - -```sh -# Omnibus installations -sudo gitlab-rake gitlab:elastic:index_repositories ID_FROM=1001 ID_TO=2000 - -# Installations from source -bundle exec rake gitlab:elastic:index_repositories ID_FROM=1001 ID_TO=2000 RAILS_ENV=production +bundle exec rake gitlab:elastic:index_projects ID_FROM=1001 ID_TO=2000 RAILS_ENV=production ``` Where `ID_FROM` and `ID_TO` are project IDs. Both parameters are optional. -As an example, if you have 3,000 repositories and you want to run three separate indexing tasks, you might run: +The above examples will index all projects starting with ID `1001` up to (and including) ID `2000`. -```sh -# Omnibus installations -sudo gitlab-rake gitlab:elastic:index_repositories ID_TO=1000 -sudo gitlab-rake gitlab:elastic:index_repositories ID_FROM=1001 ID_TO=2000 -sudo gitlab-rake gitlab:elastic:index_repositories ID_FROM=2001 - -# Installations from source -bundle exec rake gitlab:elastic:index_repositories RAILS_ENV=production ID_TO=1000 -bundle exec rake gitlab:elastic:index_repositories RAILS_ENV=production ID_FROM=1001 ID_TO=2000 -bundle exec rake gitlab:elastic:index_repositories RAILS_ENV=production ID_FROM=2001 -``` - -Sometimes your repository index process `gitlab:elastic:index_repositories` or -`gitlab:elastic:index_repositories_async` can get interrupted. This may happen -for many reasons, but it's always safe to run the indexing job again - it will -skip those repositories that have already been indexed. +TIP: **Troubleshooting:** +Sometimes the project indexing jobs queued by `gitlab:elastic:index_projects` +can get interrupted. This may happen for many reasons, but it's always safe +to run the indexing task again - it will skip those repositories that have +already been indexed. As the indexer stores the last commit SHA of every indexed repository in the database, you can run the indexer with the special parameter `UPDATE_INDEX` and @@ -297,10 +265,10 @@ that repository is indexed, it can be useful in case if your index is outdated: ```sh # Omnibus installations -sudo gitlab-rake gitlab:elastic:index_repositories UPDATE_INDEX=true ID_TO=1000 +sudo gitlab-rake gitlab:elastic:index_projects UPDATE_INDEX=true ID_TO=1000 # Installations from source -bundle exec rake gitlab:elastic:index_repositories UPDATE_INDEX=true ID_TO=1000 RAILS_ENV=production +bundle exec rake gitlab:elastic:index_projects UPDATE_INDEX=true ID_TO=1000 RAILS_ENV=production ``` You can also use the `gitlab:elastic:clear_index_status` Rake task to force the @@ -320,16 +288,6 @@ bundle exec rake gitlab:elastic:index_wikis RAILS_ENV=production The wiki indexer also supports the `ID_FROM` and `ID_TO` parameters if you want to limit a project set. -Index all database entities (Keep in mind it can take a while, so consider using `screen` or `tmux`): - -```sh -# Omnibus installations -sudo gitlab-rake gitlab:elastic:index_database - -# Installations from source -bundle exec rake gitlab:elastic:index_database RAILS_ENV=production -``` - Enable replication and refreshing again after indexing (only if you previously disabled it): ```bash @@ -340,10 +298,30 @@ curl --request PUT localhost:9200/gitlab-production/_settings --data '{ } }' ``` -A force merge should be called after enabling the refreshing above: +A force merge should be called after enabling the refreshing above. + +For Elasticsearch 6.x, before proceeding with the force merge, the index should be in read-only mode: ```bash -curl --request POST 'http://localhost:9200/_forcemerge?max_num_segments=5' +curl --request PUT localhost:9200/gitlab-production/_settings --data '{ + "settings": { + "index.blocks.write": true + } }' +``` + +Then, initiate the force merge: + +```bash +curl --request POST 'http://localhost:9200/gitlab-production/_forcemerge?max_num_segments=5' +``` + +After this, if your index is in read-only, switch back to read-write: + +```bash +curl --request PUT localhost:9200/gitlab-production/_settings --data '{ + "settings": { + "index.blocks.write": false + } }' ``` Enable Elasticsearch search in **Admin > Settings > Integrations**. That's it. Enjoy it! @@ -356,25 +334,15 @@ There are several rake tasks available to you via the command line: - This is a wrapper task. It does the following: - `sudo gitlab-rake gitlab:elastic:create_empty_index` - `sudo gitlab-rake gitlab:elastic:clear_index_status` + - `sudo gitlab-rake gitlab:elastic:index_projects` - `sudo gitlab-rake gitlab:elastic:index_wikis` - - `sudo gitlab-rake gitlab:elastic:index_database` - - `sudo gitlab-rake gitlab:elastic:index_repositories` -- [sudo gitlab-rake gitlab:elastic:index_repositories_async](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/lib/tasks/gitlab/elastic.rake) - - This iterates over all projects and places them in batches. It then sends these batches to the background via sidekiq jobs to be indexed. -- [sudo gitlab-rake gitlab:elastic:index_repositories_status](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/lib/tasks/gitlab/elastic.rake) + - `sudo gitlab-rake gitlab:elastic:index_snippets` +- [sudo gitlab-rake gitlab:elastic:index_projects](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/lib/tasks/gitlab/elastic.rake) + - This iterates over all projects and queues sidekiq jobs to index them in the background. +- [sudo gitlab-rake gitlab:elastic:index_projects_status](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/lib/tasks/gitlab/elastic.rake) - This determines the overall status of the indexing. It is done by counting the total number of indexed projects, dividing by a count of the total number of projects, then multiplying by 100. -- [sudo gitlab-rake gitlab:elastic:index_repositories](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/lib/tasks/gitlab/elastic.rake) - - This iterates over all projects and places them in batches. It then performs indexing on said batches synchronously. - [sudo gitlab-rake gitlab:elastic:index_wikis](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/lib/tasks/gitlab/elastic.rake) - Iterates over every project, determines if said project contains wiki data, and then indexes the blobs (content) of said wiki data. -- [sudo gitlab-rake gitlab:elastic:index_database](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/lib/tasks/gitlab/elastic.rake) - - This is a [rake multitask](https://www.rubydoc.info/github/ruby/rake/Rake/MultiTask). It does the following: - - `sudo gitlab-rake gitlab:elastic:index_projects` - - `sudo gitlab-rake gitlab:elastic:index_issues` - - `sudo gitlab-rake gitlab:elastic:index_merge_requests` - - `sudo gitlab-rake gitlab:elastic:index_snippets` - - `sudo gitlab-rake gitlab:elastic:index_notes` - - `sudo gitlab-rake gitlab:elastic:index_milestones` - [sudo gitlab-rake gitlab:elastic:create_empty_index](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/lib/tasks/gitlab/elastic.rake) - This generates an empty index on the Elasticsearch side. - [sudo gitlab-rake gitlab:elastic:clear_index_status](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/lib/tasks/gitlab/elastic.rake) @@ -385,18 +353,8 @@ There are several rake tasks available to you via the command line: - Does the same thing as `sudo gitlab-rake gitlab:elastic:create_empty_index` - [sudo gitlab-rake gitlab:elastic:add_feature_visibility_levels_to_project](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/lib/tasks/gitlab/elastic.rake) - Adds visibility information to the indices for projects. -- [sudo gitlab-rake gitlab:elastic:index_projects](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/lib/tasks/gitlab/elastic.rake) - - Performs an Elasticsearch import that indexes projects data. -- [sudo gitlab-rake gitlab:elastic:index_issues](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/lib/tasks/gitlab/elastic.rake) - - Performs an Elasticsearch import that indexes issues data. -- [sudo gitlab-rake gitlab:elastic:index_merge_requests](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/lib/tasks/gitlab/elastic.rake) - - Performs an Elasticsearch import that indexes merge requests data. - [sudo gitlab-rake gitlab:elastic:index_snippets](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/lib/tasks/gitlab/elastic.rake) - Performs an Elasticsearch import that indexes the snippets data. -- [sudo gitlab-rake gitlab:elastic:index_notes](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/lib/tasks/gitlab/elastic.rake) - - Performs an Elasticsearch import that indexes the notes data. -- [sudo gitlab-rake gitlab:elastic:index_milestones](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/lib/tasks/gitlab/elastic.rake) - - Performs an Elasticsearch import that indexes the milestones data. ### Environment Variables @@ -404,40 +362,16 @@ In addition to the rake tasks, there are some environment variables that can be | Environment Variable | Data Type | What it does | | -------------------- |:---------:| ---------------------------------------------------------------------------- | -| `BATCH` | Integer | Modifies the size of the indexing batch (default 300 projects). | | `UPDATE_INDEX` | Boolean | Tells the indexer to overwrite any existing index data (true/false). | | `ID_TO` | Integer | Tells the indexer to only index projects less than or equal to the value. | | `ID_FROM` | Integer | Tells the indexer to only index projects greater than or equal to the value. | -### Batching - -The ability to apply batching makes the indexer run more efficiently. The default -size of a batch is 300 projects, which may or may not be ideal for your setup. -Depending on the resources available to your GitLab instance (sidekiq) and your -Elasticsearch instance (RAM, CPU), you may be able to increase or decrease the -batch size for more efficiency. - -- The larger the batch size is, the less sidekiq jobs and indexing requests get created. -- The larger the batch size is, the more time and RAM it takes to process. -- The smaller the batch size, the more sidekiq jobs, and indexing requests get created. -- The smaller the batch size, the more CPU gets utilized. - -Finding the ideal size can be tricky, and will vary from GitLab instance to GitLab instance. -Generally speaking, if the default is not ideal for you, try reducing it to somewhere in -the 50-150 range (for bigger sized repos) or 450-600 range (for many small-sized repos). - -Example use: - -```sh -sudo gitlab-rake gitlab:elastic:index_repositories_async BATCH=50 -``` - ### Indexing a specific project Because the `ID_TO` and `ID_FROM` environment variables use the `or equal to` comparison, you can index only one project by using both these variables with the same project ID number: ```sh -root@git:~# sudo gitlab-rake gitlab:elastic:index_repositories ID_TO=5 ID_FROM=5 +root@git:~# sudo gitlab-rake gitlab:elastic:index_projects ID_TO=5 ID_FROM=5 Indexing project repositories...I, [2019-03-04T21:27:03.083410 #3384] INFO -- : Indexing GitLab User / test (ID=33)... I, [2019-03-04T21:27:05.215266 #3384] INFO -- : Indexing GitLab User / test (ID=33) is done! ``` @@ -534,7 +468,7 @@ Here are some common pitfalls and how to overcome them: - **The indexing process is taking a very long time** - The more data present in your GitLab instance, the longer the indexing process takes. You might want to try adjusting the BATCH sizes for asynchronous indexing to help speed up the process. + The more data present in your GitLab instance, the longer the indexing process takes. - **No new data is added to the Elasticsearch index when I push code** diff --git a/doc/integration/github.md b/doc/integration/github.md index e145afbdd5e..5b01dd9feb7 100644 --- a/doc/integration/github.md +++ b/doc/integration/github.md @@ -21,10 +21,10 @@ To get the credentials (a pair of Client ID and Client Secret), you must registe - Application name: This can be anything. Consider something like `<Organization>'s GitLab` or `<Your Name>'s GitLab` or something else descriptive. - Homepage URL: The URL of your GitLab installation. For example, `https://gitlab.example.com`. - Application description: Fill this in if you wish. - - Authorization callback URL: `http(s)://${YOUR_DOMAIN}/users/auth/github/callback`. Please make sure the port is included if your GitLab instance is not configured on default port. + - Authorization callback URL: `http(s)://${YOUR_DOMAIN}/users/auth`. Please make sure the port is included if your GitLab instance is not configured on default port.  - NOTE: Be sure to append `/users/auth/github/callback` to the end of the callback URL + NOTE: Be sure to append `/users/auth` to the end of the callback URL to prevent a [OAuth2 convert redirect](http://tetraph.com/covert_redirect/) vulnerability. diff --git a/doc/integration/salesforce.md b/doc/integration/salesforce.md index 8a99641a256..1ef43cfcece 100644 --- a/doc/integration/salesforce.md +++ b/doc/integration/salesforce.md @@ -1,27 +1,28 @@ -# SalesForce OmniAuth Provider +# Salesforce OmniAuth Provider -You can integrate your GitLab instance with [SalesForce](https://www.salesforce.com/) to enable users to login to your GitLab instance with their SalesForce account. +You can integrate your GitLab instance with [Salesforce](https://www.salesforce.com/) to enable users to log in to your GitLab instance with their Salesforce account. -## Create SalesForce Application +## Create a Salesforce Connected App -To enable SalesForce OmniAuth provider, you must use SalesForce's credentials for your GitLab instance. -To get the credentials (a pair of Client ID and Client Secret), you must register an application on SalesForces. +To enable Salesforce OmniAuth provider, you must use Salesforce's credentials for your GitLab instance. +To get the credentials (a pair of Client ID and Client Secret), you must [create a Connected App](https://help.salesforce.com/articleView?id=connected_app_create.htm&type=5) on Salesforce. -1. Sign in to [SalesForce](https://www.salesforce.com/). +1. Sign in to [Salesforce](https://login.salesforce.com/). -1. Navigate to **Platform Tools/Apps/App Manager** and click on **New Connected App**. +1. In Setup, enter `App Manager` in the Quick Find box, click **App Manager**, then click **New Connected App**. 1. Fill in the application details into the following fields: - **Connected App Name** and **API Name**: Set to any value but consider something like `<Organization>'s GitLab`, `<Your Name>'s GitLab`, or something else that is descriptive. + - **Contact Email**: Enter the contact email for Salesforce to use when contacting you or your support team. - **Description**: Description for the application. -  +  1. Select **API (Enable OAuth Settings)** and click on **Enable OAuth Settings**. 1. Fill in the application details into the following fields: - - **Callback URL**: The call callback URL. For example, `https://gitlab.example.com/users/auth/salesforce/callback`. + - **Callback URL**: The callback URL of your GitLab installation. For example, `https://gitlab.example.com/users/auth/salesforce/callback`. - **Selected OAuth Scopes**: Move **Access your basic information (id, profile, email, address, phone)** and **Allow access to your unique identifier (openid)** to the right column. -  +  1. Click **Save**. 1. On your GitLab server, open the configuration file. @@ -63,17 +64,16 @@ To get the credentials (a pair of Client ID and Client Secret), you must registe app_secret: 'SALESFORCE_CLIENT_SECRET' } ``` -1. Change `SALESFORCE_CLIENT_ID` to the Consumer Key from the SalesForce connected application page. -1. Change `SALESFORCE_CLIENT_SECRET` to the Consumer Secret from the SalesForce connected application page. -  +1. Change `SALESFORCE_CLIENT_ID` to the Consumer Key from the Salesforce connected application page. +1. Change `SALESFORCE_CLIENT_SECRET` to the Consumer Secret from the Salesforce connected application page. +  1. Save the configuration file. -1. [Reconfigure GitLab]( ../administration/restart_gitlab.md#omnibus-gitlab-reconfigure ) or [restart GitLab]( ../administration/restart_gitlab.md#installations-from-source ) for the changes to take effect if you - installed GitLab via Omnibus or from source respectively. +1. [Reconfigure GitLab]( ../administration/restart_gitlab.md#omnibus-gitlab-reconfigure ) or [restart GitLab]( ../administration/restart_gitlab.md#installations-from-source ) for the changes to take effect if you installed GitLab via Omnibus or from source respectively. -On the sign in page, there should now be a SalesForce icon below the regular sign in form. -Click the icon to begin the authentication process. SalesForce will ask the user to sign in and authorize the GitLab application. +On the sign in page, there should now be a Salesforce icon below the regular sign in form. +Click the icon to begin the authentication process. Salesforce will ask the user to sign in and authorize the GitLab application. If everything goes well, the user will be returned to GitLab and will be signed in. NOTE: **Note:** -GitLab requires the email address of each new user. Once the user is logged in using SalesForce, GitLab will redirect the user to the profile page where they will have to provide the email and verify the email. +GitLab requires the email address of each new user. Once the user is logged in using Salesforce, GitLab will redirect the user to the profile page where they will have to provide the email and verify the email.
\ No newline at end of file diff --git a/doc/integration/saml.md b/doc/integration/saml.md index 15d9d8c9c74..22e07594d6f 100644 --- a/doc/integration/saml.md +++ b/doc/integration/saml.md @@ -1,6 +1,6 @@ # SAML OmniAuth Provider -> This topic is for SAML on self-managed GitLab instances. For SAML on GitLab.com, see [SAML SSO for GitLab.com Groups](https://docs.gitlab.com/ee/user/group/saml_sso/index.html). +> This topic is for SAML on self-managed GitLab instances. For SAML on GitLab.com, see [SAML SSO for GitLab.com Groups](../user/group/saml_sso/index.md). NOTE: **Note:** You need to [enable OmniAuth](omniauth.md) in order to use this. diff --git a/doc/integration/slash_commands.md b/doc/integration/slash_commands.md index cd755089be8..71ea2e25533 100644 --- a/doc/integration/slash_commands.md +++ b/doc/integration/slash_commands.md @@ -20,8 +20,8 @@ Taking the trigger term as `project-name`, the commands are: | `/project-name deploy <from> to <to>` | Deploy from the `<from>` environment to the `<to>` environment | | `/project-name run <job name> <arguments>` | Execute [ChatOps](../ci/chatops/README.md) job `<job name>` on `master` | -Note that if you are using the [GitLab Slack application](https://docs.gitlab.com/ee/user/project/integrations/gitlab_slack_application.html) for -your GitLab.com projects, you need to [add the `gitlab` keyword at the beginning of the command](https://docs.gitlab.com/ee/user/project/integrations/gitlab_slack_application.html#usage). +Note that if you are using the [GitLab Slack application](../user/project/integrations/gitlab_slack_application.md) for +your GitLab.com projects, you need to [add the `gitlab` keyword at the beginning of the command](../user/project/integrations/gitlab_slack_application.md#usage). ## Issue commands diff --git a/doc/license/README.md b/doc/license/README.md index b9281fd5299..fd110a39b61 100644 --- a/doc/license/README.md +++ b/doc/license/README.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/admin_area/license.html' +redirect_to: '../user/admin_area/license.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/user/admin_area/license.html). +This document was moved to [another location](../user/admin_area/license.md). diff --git a/doc/push_rules/push_rules.md b/doc/push_rules/push_rules.md index 7654023f266..e44eab2556e 100644 --- a/doc/push_rules/push_rules.md +++ b/doc/push_rules/push_rules.md @@ -61,7 +61,7 @@ The following options are available. | --------- | :------------: | ----------- | | Removal of tags with `git push` | **Starter** 7.10 | Forbid users to remove git tags with `git push`. Tags will still be able to be deleted through the web UI. | | Check whether author is a GitLab user | **Starter** 7.10 | Restrict commits by author (email) to existing GitLab users. | -| Check whether committer is the current authenticated user | **Premium** 10.2 | GitLab will reject any commit that was not committed by the current authenticated user | +| Committer restriction | **Premium** 10.2 | GitLab will reject any commit that was not committed by the current authenticated user | | Check whether commit is signed through GPG | **Premium** 10.1 | Reject commit when it is not signed through GPG. Read [signing commits with GPG][signing-commits]. | | Prevent committing secrets to Git | **Starter** 8.12 | GitLab will reject any files that are likely to contain secrets. Read [what files are forbidden](#prevent-pushing-secrets-to-the-repository). | | Restrict by commit message | **Starter** 7.10 | Only commit messages that match this regular expression are allowed to be pushed. Leave empty to allow any commit message. Uses multiline mode, which can be disabled using `(?-m)`. | diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index 56db7b5eb3a..764916ca82d 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -917,9 +917,9 @@ backup beforehand. 1. Clear all the tokens for projects, groups, and the whole instance: -CAUTION: **Caution:** -The last UPDATE operation will stop the runners being able to pick up -new jobs. You must register new runners. + CAUTION: **Caution:** + The last UPDATE operation will stop the runners being able to pick up + new jobs. You must register new runners. ```sql -- Clear project tokens diff --git a/doc/raketasks/import.md b/doc/raketasks/import.md index bb316df5b9a..b59c06a24ea 100644 --- a/doc/raketasks/import.md +++ b/doc/raketasks/import.md @@ -16,7 +16,7 @@ The new folder needs to have git user ownership and read/write/execute access for git user and its group: ``` -sudo -u git mkdir /var/opt/gitlab/git-data/repository-import-<date>/new_group +sudo -u git mkdir -p /var/opt/gitlab/git-data/repository-import-<date>/new_group ``` ### Copy your bare repositories inside this newly created folder: diff --git a/doc/security/img/two_factor_authentication_group_settings.png b/doc/security/img/two_factor_authentication_group_settings.png Binary files differdeleted file mode 100644 index 05d95554fd9..00000000000 --- a/doc/security/img/two_factor_authentication_group_settings.png +++ /dev/null diff --git a/doc/security/img/two_factor_authentication_settings.png b/doc/security/img/two_factor_authentication_settings.png Binary files differdeleted file mode 100644 index 2a2208f98bd..00000000000 --- a/doc/security/img/two_factor_authentication_settings.png +++ /dev/null diff --git a/doc/security/rack_attack.md b/doc/security/rack_attack.md index ad83dc05a93..66081d7e376 100644 --- a/doc/security/rack_attack.md +++ b/doc/security/rack_attack.md @@ -94,7 +94,7 @@ In case you want to remove a blocked IP, follow these steps: 1. Find the IPs that have been blocked in the production log: ```sh - grep "Rack_Attack" /var/log/gitlab/gitlab-rails/production.log + grep "Rack_Attack" /var/log/gitlab/gitlab-rails/auth.log ``` 1. Since the blacklist is stored in Redis, you need to open up `redis-cli`: diff --git a/doc/security/two_factor_authentication.md b/doc/security/two_factor_authentication.md index 4b65b901487..2ece4ed3fc9 100644 --- a/doc/security/two_factor_authentication.md +++ b/doc/security/two_factor_authentication.md @@ -16,39 +16,35 @@ enforce everyone to set up 2FA, you can choose from two different ways: - Enforce on next login. - Suggest on next login, but allow a grace period before enforcing. -In the Admin area under **Settings** (`/admin/application_settings`), look for -the "Sign-in Restrictions" area, where you can configure both. +After the configured grace period has elapsed, users will be able to log in but +won't be able to leave the 2FA configuration area at `/profile/two_factor_auth`. + +To enable 2FA for all users: + +1. Navigate to **Admin area > Settings > General** (`/admin/application_settings`). +1. Expand the **Sign-in restrictions** section, where you can configure both. If you want 2FA enforcement to take effect on next login, change the grace period to `0`. ---- - - +## Enforcing 2FA for all users in a group ---- +If you want to enforce 2FA only for certain groups, you can: -## Enforcing 2FA for all users in a group +1. Enable it in the group's **Settings > General** page. +1. Optionally specify a grace period as above. -If you want to enforce 2FA only for certain groups, you can enable it in the -group settings and specify a grace period as above. To change this setting you -need to be administrator or owner of the group. +To change this setting, you need to be administrator or owner of the group. If there are multiple 2FA requirements (i.e. group + all users, or multiple groups) the shortest grace period will be used. ---- - - - ---- - ## Disabling 2FA for everyone There may be some special situations where you want to disable 2FA for everyone even when forced 2FA is disabled. There is a rake task for that: -``` +```sh # Omnibus installations sudo gitlab-rake gitlab:two_factor:disable_for_all_users @@ -56,5 +52,6 @@ sudo gitlab-rake gitlab:two_factor:disable_for_all_users sudo -u git -H bundle exec rake gitlab:two_factor:disable_for_all_users RAILS_ENV=production ``` -**IMPORTANT: this is a permanent and irreversible action. Users will have to - reactivate 2FA from scratch if they want to use it again.** +CAUTION: **Caution:** +This is a permanent and irreversible action. Users will have to +reactivate 2FA from scratch if they want to use it again. diff --git a/doc/ssh/README.md b/doc/ssh/README.md index 9c4a391e8da..3bfebfc5d9b 100644 --- a/doc/ssh/README.md +++ b/doc/ssh/README.md @@ -55,7 +55,7 @@ As an admin, you can restrict By default, all keys are permitted, which is also the case for [GitLab.com](../user/gitlab_com/index.md#ssh-host-keys-fingerprints). -## ED25519 SSH keys +### ED25519 SSH keys Following [best practices](https://linux-audit.com/using-ed25519-openssh-keys-instead-of-dsa-rsa-ecdsa/), you should always favor [ED25519](https://ed25519.cr.yp.to/) SSH keys, since they @@ -65,7 +65,7 @@ They were introduced in OpenSSH 6.5, so any modern OS should include the option to create them. If for any reason your OS or the GitLab instance you interact with doesn't support this, you can fallback to RSA. -## RSA SSH keys +### RSA SSH keys RSA keys are the most common ones and therefore the most compatible with servers that may have an old OpenSSH version. Use them if the GitLab server @@ -166,12 +166,13 @@ Now, it's time to add the newly created public key to your GitLab account. NOTE: **Note:** If you opted to create an RSA key, the name might differ. -1. Add your public SSH key to your GitLab account by clicking your avatar - in the upper right corner and selecting **Settings**. From there on, - navigate to **SSH Keys** and paste your public key in the "Key" section. - If you created the key with a comment, this will appear under "Title". - If not, give your key an identifiable title like _Work Laptop_ or - _Home Workstation_, and click **Add key**. +1. Add your **public** SSH key to your GitLab account by: + 1. Clicking your avatar in the upper right corner and selecting **Settings**. + 1. Navigating to **SSH Keys** and pasting your **public** key in the **Key** field. If you: + + - Created the key with a comment, this will appear in the **Title** field. + - Created the key without a comment, give your key an identifiable title like _Work Laptop_ or _Home Workstation_. + 1. Click the **Add key** button. NOTE: **Note:** If you manually copied your public SSH key make sure you copied the entire @@ -305,7 +306,7 @@ who needs to know and configure the private key. GitLab administrators set up Global Deploy keys in the Admin area under the section **Deploy Keys**. Ensure keys have a meaningful title as that will be the primary way for project maintainers and owners to identify the correct Global -Deploy key to add. For instance, if the key gives access to a SaaS CI instance, +Deploy key to add. For instance, if the key gives access to a SaaS CI instance, use the name of that service in the key name if that is all it is used for. When creating Global Shared Deploy keys, give some thought to the granularity of keys - they could be of very narrow usage such as just a specific service or diff --git a/doc/subscriptions/index.md b/doc/subscriptions/index.md index b6d16536fee..dfd80f8882e 100644 --- a/doc/subscriptions/index.md +++ b/doc/subscriptions/index.md @@ -38,7 +38,7 @@ Future purchases will use the information in this section. The email listed in t ### Self-managed: Apply your license file -After purchase, the license file is sent to the email address tied to the Customers portal account, which needs to be [uploaded to the GitLab instance](https://docs.gitlab.com/ee/user/admin_area/license.html#uploading-your-license). +After purchase, the license file is sent to the email address tied to the Customers portal account, which needs to be [uploaded to the GitLab instance](../user/admin_area/license.md#uploading-your-license). ### Link your GitLab.com account with your Customers Portal account diff --git a/doc/topics/authentication/index.md b/doc/topics/authentication/index.md index 00b6c1dfdc2..228da2d1f57 100644 --- a/doc/topics/authentication/index.md +++ b/doc/topics/authentication/index.md @@ -17,11 +17,11 @@ This page gathers all the resources for the topic **Authentication** within GitL ## GitLab administrators - [LDAP (Community Edition)](../../administration/auth/ldap.md) -- [LDAP (Enterprise Edition)](https://docs.gitlab.com/ee/administration/auth/ldap-ee.html) **[STARTER]** +- [LDAP (Enterprise Edition)](../../administration/auth/ldap-ee.md) **[STARTER]** - [Enforce Two-factor Authentication (2FA)](../../security/two_factor_authentication.md#enforce-two-factor-authentication-2fa) - **Articles:** - [How to Configure LDAP with GitLab CE](../../administration/auth/how_to_configure_ldap_gitlab_ce/index.md) - - [How to Configure LDAP with GitLab EE](https://docs.gitlab.com/ee/administration/auth/how_to_configure_ldap_gitlab_ee/index.html) **[STARTER]** + - [How to Configure LDAP with GitLab EE](../../administration/auth/how_to_configure_ldap_gitlab_ee/index.md) **[STARTER]** - [Feature Highlight: LDAP Integration](https://about.gitlab.com/2014/07/10/feature-highlight-ldap-sync/) - [Debugging LDAP](https://about.gitlab.com/handbook/support/workflows/support-engineering/ldap/debugging_ldap.html) - **Integrations:** @@ -30,10 +30,10 @@ This page gathers all the resources for the topic **Authentication** within GitL - [Atlassian Crowd OmniAuth Provider](../../administration/auth/crowd.md) - [CAS OmniAuth Provider](../../integration/cas.md) - [SAML OmniAuth Provider](../../integration/saml.md) - - [SAML for GitLab.com Groups](https://docs.gitlab.com/ee/user/group/saml_sso/index.html) **[SILVER ONLY]** - - [SCIM user provisioning for GitLab.com Groups](https://docs.gitlab.com/ee/user/group/saml_sso/scim_setup.html) **[SILVER ONLY]** + - [SAML for GitLab.com Groups](../../user/group/saml_sso/index.md) **[SILVER ONLY]** + - [SCIM user provisioning for GitLab.com Groups](../../user/group/saml_sso/scim_setup.md) **[SILVER ONLY]** - [Okta SSO provider](../../administration/auth/okta.md) - - [Kerberos integration (GitLab EE)](https://docs.gitlab.com/ee/integration/kerberos.html) **[STARTER]** + - [Kerberos integration (GitLab EE)](../../integration/kerberos.md) **[STARTER]** ## API diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index 42c8da1f6f1..b00a8afa386 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -13,7 +13,7 @@ Starting with GitLab 11.3, the Auto DevOps pipeline is enabled by default for al projects. If it has not been explicitly enabled for the project, Auto DevOps will be automatically disabled on the first pipeline failure. Your project will continue to use an alternative [CI/CD configuration file](../../ci/yaml/README.md) if one is found. A GitLab -administrator can [change this setting](../../user/admin_area/settings/continuous_integration.html#auto-devops-core-only) +administrator can [change this setting](../../user/admin_area/settings/continuous_integration.md#auto-devops-core-only) in the admin area. With Auto DevOps, the software development process becomes easier to set up @@ -126,10 +126,6 @@ Auto Deploy, and Auto Monitoring will be silently skipped. ## Auto DevOps base domain -NOTE: **Note** -`AUTO_DEVOPS_DOMAIN` environment variable is deprecated and -[is scheduled to be removed](https://gitlab.com/gitlab-org/gitlab-ce/issues/56959). - The Auto DevOps base domain is required if you want to make use of [Auto Review Apps](#auto-review-apps) and [Auto Deploy](#auto-deploy). It can be defined in any of the following places: @@ -162,6 +158,12 @@ Auto DevOps base domain to `1.2.3.4.nip.io`. Once set up, all requests will hit the load balancer, which in turn will route them to the Kubernetes pods that run your application(s). +NOTE: **Note:** +From GitLab 11.8, `KUBE_INGRESS_BASE_DOMAIN` replaces `AUTO_DEVOPS_DOMAIN`. +Support for `AUTO_DEVOPS_DOMAIN` was [removed in GitLab +12.0](https://gitlab.com/gitlab-org/gitlab-ce/issues/56959). + + ## Using multiple Kubernetes clusters **[PREMIUM]** When using Auto DevOps, you may want to deploy different environments to @@ -179,7 +181,7 @@ Those environments are tied to jobs that use [Auto Deploy](#auto-deploy), so except for the environment scope, they would also need to have a different domain they would be deployed to. This is why you need to define a separate `KUBE_INGRESS_BASE_DOMAIN` variable for all the above -[based on the environment](https://docs.gitlab.com/ee/ci/variables/#limiting-environment-scopes-of-environment-variables-premium). +[based on the environment](../../ci/variables/README.md#limiting-environment-scopes-of-environment-variables-premium). The following table is an example of how the three different clusters would be configured. @@ -209,10 +211,6 @@ and verifying that your app is deployed as a review app in the Kubernetes cluster with the `review/*` environment scope. Similarly, you can check the other environments. -NOTE: **Note:** -From GitLab 11.8, `KUBE_INGRESS_BASE_DOMAIN` replaces `AUTO_DEVOPS_DOMAIN`. -`AUTO_DEVOPS_DOMAIN` [is scheduled to be removed](https://gitlab.com/gitlab-org/gitlab-ce/issues/56959). - ## Enabling/Disabling Auto DevOps When first using Auto Devops, review the [requirements](#requirements) to ensure all necessary components to make @@ -365,7 +363,7 @@ created, and is uploaded as an artifact which you can later download and check out. Any differences between the source and target branches are also -[shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality.html). +[shown in the merge request widget](../../user/project/merge_requests/code_quality.md). ### Auto SAST **[ULTIMATE]** @@ -378,7 +376,7 @@ report is created, it's uploaded as an artifact which you can later download and check out. Any security warnings are also shown in the merge request widget. Read more how -[SAST works](https://docs.gitlab.com/ee/user/application_security/sast/index.html). +[SAST works](../../user/application_security/sast/index.md). NOTE: **Note:** The Auto SAST stage will be skipped on licenses other than Ultimate. @@ -397,7 +395,7 @@ report is created, it's uploaded as an artifact which you can later download and check out. Any security warnings are also shown in the merge request widget. Read more about -[Dependency Scanning](https://docs.gitlab.com/ee/user/application_security/dependency_scanning/index.html). +[Dependency Scanning](../../user/application_security/dependency_scanning/index.md). NOTE: **Note:** The Auto Dependency Scanning stage will be skipped on licenses other than Ultimate. @@ -416,7 +414,7 @@ report is created, it's uploaded as an artifact which you can later download and check out. Any licenses are also shown in the merge request widget. Read more how -[License Management works](https://docs.gitlab.com/ee/user/application_security/license_management/index.html). +[License Management works](../../user/application_security/license_management/index.md). NOTE: **Note:** The Auto License Management stage will be skipped on licenses other than Ultimate. @@ -432,7 +430,7 @@ created, it's uploaded as an artifact which you can later download and check out. Any security warnings are also shown in the merge request widget. Read more how -[Container Scanning works](https://docs.gitlab.com/ee/user/application_security/container_scanning/index.html). +[Container Scanning works](../../user/application_security/container_scanning/index.md). NOTE: **Note:** The Auto Container Scanning stage will be skipped on licenses other than Ultimate. @@ -488,7 +486,7 @@ issues. Once the report is created, it's uploaded as an artifact which you can later download and check out. Any security warnings are also shown in the merge request widget. Read how -[DAST works](https://docs.gitlab.com/ee/user/application_security/dast/index.html). +[DAST works](../../user/application_security/dast/index.md). NOTE: **Note:** The Auto DAST stage will be skipped on licenses other than Ultimate. @@ -506,7 +504,7 @@ Auto Browser Performance Testing utilizes the [Sitespeed.io container](https://h ``` Any performance differences between the source and target branches are also -[shown in the merge request widget](https://docs.gitlab.com/ee//user/project/merge_requests/browser_performance_testing.html). +[shown in the merge request widget](../../user/project/merge_requests/browser_performance_testing.md). ### Auto Deploy @@ -584,16 +582,17 @@ Note that a post-install hook means that if any deploy succeeds, If present, `DB_MIGRATE` will be run as a shell command within an application pod as a helm pre-upgrade hook. -For example, in a Rails application: +For example, in a Rails application in an image built with +[Herokuish](https://github.com/gliderlabs/herokuish): -- `DB_INITIALIZE` can be set to `cd /app && RAILS_ENV=production - bin/setup` -- `DB_MIGRATE` can be set to `cd /app && RAILS_ENV=production bin/update` +- `DB_INITIALIZE` can be set to `RAILS_ENV=production /bin/herokuish procfile exec bin/rails db:setup` +- `DB_MIGRATE` can be set to `RAILS_ENV=production /bin/herokuish procfile exec bin/rails db:migrate` NOTE: **Note:** -The `/app` path is the directory of your project inside the docker image -as [configured by -Herokuish](https://github.com/gliderlabs/herokuish#paths) +Unless you have a `Dockerfile` in your repo, your image is built with +Herokuish. You must prefix commands run in these images with `/bin/herokuish +procfile exec` in order to replicate the the environment your application is +run in. ### Auto Monitoring @@ -675,7 +674,7 @@ repo or by specifying a project variable: ### Custom Helm chart per environment **[PREMIUM]** You can specify the use of a custom Helm chart per environment by scoping the environment variable -to the desired environment. See [Limiting environment scopes of variables](https://docs.gitlab.com/ee/ci/variables/#limiting-environment-scopes-of-variables-premium). +to the desired environment. See [Limiting environment scopes of variables](../../ci/variables/README.md#limiting-environment-scopes-of-environment-variables-premium). ### Customizing `.gitlab-ci.yml` @@ -734,7 +733,6 @@ also be customized, and you can easily use a [custom buildpack](#custom-buildpac | **Variable** | **Description** | | ------------ | --------------- | -| `AUTO_DEVOPS_DOMAIN` | The [Auto DevOps domain](#auto-devops-base-domain). By default, set automatically by the [Auto DevOps setting](#enablingdisabling-auto-devops). This variable is deprecated and [is scheduled to be removed](https://gitlab.com/gitlab-org/gitlab-ce/issues/56959). Use `KUBE_INGRESS_BASE_DOMAIN` instead. | | `AUTO_DEVOPS_CHART` | The Helm Chart used to deploy your apps; defaults to the one [provided by GitLab](https://gitlab.com/gitlab-org/charts/auto-deploy-app). | | `AUTO_DEVOPS_CHART_REPOSITORY` | The Helm Chart repository used to search for charts; defaults to `https://charts.gitlab.io`. | | `AUTO_DEVOPS_CHART_REPOSITORY_NAME` | From Gitlab 11.11, this variable can be used to set the name of the helm repository; defaults to "gitlab" | @@ -742,8 +740,8 @@ also be customized, and you can easily use a [custom buildpack](#custom-buildpac | `AUTO_DEVOPS_CHART_REPOSITORY_PASSWORD` | From Gitlab 11.11, this variable can be used to set a password to connect to the helm repository. Defaults to no credentials. (Also set AUTO_DEVOPS_CHART_REPOSITORY_USERNAME) | | `REPLICAS` | The number of replicas to deploy; defaults to 1. | | `PRODUCTION_REPLICAS` | The number of replicas to deploy in the production environment. This takes precedence over `REPLICAS`; defaults to 1. | -| `CANARY_REPLICAS` | The number of canary replicas to deploy for [Canary Deployments](https://docs.gitlab.com/ee/user/project/canary_deployments.html); defaults to 1 | -| `CANARY_PRODUCTION_REPLICAS` | The number of canary replicas to deploy for [Canary Deployments](https://docs.gitlab.com/ee/user/project/canary_deployments.html) in the production environment. This takes precedence over `CANARY_REPLICAS`; defaults to 1 | +| `CANARY_REPLICAS` | The number of canary replicas to deploy for [Canary Deployments](../../user/project/canary_deployments.md); defaults to 1 | +| `CANARY_PRODUCTION_REPLICAS` | The number of canary replicas to deploy for [Canary Deployments](../../user/project/canary_deployments.md) in the production environment. This takes precedence over `CANARY_REPLICAS`; defaults to 1 | | `ADDITIONAL_HOSTS` | Fully qualified domain names specified as a comma-separated list that are added to the ingress hosts. | | `<ENVIRONMENT>_ADDITIONAL_HOSTS` | For a specific environment, the fully qualified domain names specified as a comma-separated list that are added to the ingress hosts. This takes precedence over `ADDITIONAL_HOSTS`. | | `POSTGRES_ENABLED` | Whether PostgreSQL is enabled; defaults to `"true"`. Set to `false` to disable the automatic deployment of PostgreSQL. | @@ -771,6 +769,7 @@ also be customized, and you can easily use a [custom buildpack](#custom-buildpac | `K8S_SECRET_*` | From GitLab 11.7, any variable prefixed with [`K8S_SECRET_`](#application-secret-variables) will be made available by Auto DevOps as environment variables to the deployed application. | | `KUBE_INGRESS_BASE_DOMAIN` | From GitLab 11.8, this variable can be used to set a domain per cluster. See [cluster domains](../../user/project/clusters/index.md#base-domain) for more information. | | `ROLLOUT_RESOURCE_TYPE` | From GitLab 11.9, this variable allows specification of the resource type being deployed when using a custom helm chart. Default value is `deployment`. | +| `ROLLOUT_STATUS_DISABLED` | From GitLab 12.0, this variable allows to disable rollout status check because it doesn't support all resource types, for example, `cronjob`. | | `HELM_UPGRADE_EXTRA_ARGS` | From GitLab 11.11, this variable allows extra arguments in `helm` commands when deploying the application. Note that using quotes will not prevent word splitting. | TIP: **Tip:** @@ -919,7 +918,7 @@ you when you're ready to manually deploy to production. > [Introduced](https://gitlab.com/gitlab-org/gitlab-ci-yml/merge_requests/171) in GitLab 11.0. -A [canary environment](https://docs.gitlab.com/ee/user/project/canary_deployments.html) can be used +A [canary environment](../../user/project/canary_deployments.md) can be used before any changes are deployed to production. If `CANARY_ENABLED` is defined in your project (e.g., set `CANARY_ENABLED` to diff --git a/doc/topics/autodevops/quick_start_guide.md b/doc/topics/autodevops/quick_start_guide.md index 367e192b85a..cc83d20d65a 100644 --- a/doc/topics/autodevops/quick_start_guide.md +++ b/doc/topics/autodevops/quick_start_guide.md @@ -208,7 +208,7 @@ applications. In the rightmost column for the production environment, you can ma application is running. Right below, there is the -[Deploy Board](https://docs.gitlab.com/ee/user/project/deploy_boards.html). +[Deploy Board](../../user/project/deploy_boards.md). The squares represent pods in your Kubernetes cluster that are associated with the given environment. Hovering above each square you can see the state of a deployment and clicking a square will take you to the pod's logs page. diff --git a/doc/topics/git/how_to_install_git/index.md b/doc/topics/git/how_to_install_git/index.md index d7e1979217e..c7bede2d269 100644 --- a/doc/topics/git/how_to_install_git/index.md +++ b/doc/topics/git/how_to_install_git/index.md @@ -5,14 +5,20 @@ level: beginner article_type: user guide date: 2017-05-15 description: 'This article describes how to install Git on macOS, Ubuntu Linux and Windows.' +type: howto +last_updated: 2019-05-31 --- # Installing Git -To begin contributing to GitLab projects +To begin contributing to GitLab projects, you will need to install the Git client on your computer. + This article will show you how to install Git on macOS, Ubuntu Linux and Windows. +Information on [installing Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) +is also available at the official Git website. + ## Install Git on macOS using the Homebrew package manager Although it is easy to use the version of Git shipped with macOS @@ -21,7 +27,7 @@ we recommend installing it via Homebrew to get access to an extensive selection of dependency managed libraries and applications. If you are sure you don't need access to any additional development libraries -or don't have approximately 15gb of available disk space for Xcode and Homebrew +or don't have approximately 15gb of available disk space for Xcode and Homebrew, use one of the aforementioned methods. ### Installing Xcode @@ -40,11 +46,12 @@ for the official Homebrew installation instructions. With Homebrew installed you are now ready to install Git. Open a Terminal and enter in the following command: -```bash +```sh brew install git ``` Congratulations you should now have Git installed via Homebrew. + Next read our article on [adding an SSH key to GitLab](../../../ssh/README.md). ## Install Git on Ubuntu Linux @@ -55,16 +62,30 @@ it is recommended to use the built in package manager to install Git. Open a Terminal and enter in the following commands to install the latest Git from the official Git maintained package archives: -```bash +```sh sudo apt-add-repository ppa:git-core/ppa sudo apt-get update sudo apt-get install git ``` Congratulations you should now have Git installed via the Ubuntu package manager. + Next read our article on [adding an SSH key to GitLab](../../../ssh/README.md). ## Installing Git on Windows from the Git website Browse to the [Git website](https://git-scm.com/) and download and install Git for Windows. + Next read our article on [adding an SSH key to GitLab](../../../ssh/README.md). + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/topics/git/index.md b/doc/topics/git/index.md index 7707d56764e..841746cc5de 100644 --- a/doc/topics/git/index.md +++ b/doc/topics/git/index.md @@ -1,8 +1,12 @@ -# Git documentation +--- +type: index +--- + +# Git Git is a [free and open source](https://git-scm.com/about/free-and-open-source) distributed version control system designed to handle everything from small to -very large projects with speed and efficiency. +large projects with speed and efficiency. [GitLab](https://about.gitlab.com) is a Git-based fully integrated platform for software development. Besides Git's functionalities, GitLab has a lot of @@ -11,64 +15,71 @@ powerful [features](https://about.gitlab.com/features/) to enhance your We've gathered some resources to help you to get the best from Git with GitLab. +More information is also available on the [Git website](https://git-scm.com). + ## Getting started -- [Git concepts](../../university/training/user_training.md#git-concepts) +The following resources will help you get started with Git: + +- [Git Basics](https://git-scm.com/book/en/v2/Getting-Started-Git-Basics) +- [Git on the Server - GitLab](https://git-scm.com/book/en/v2/Git-on-the-Server-GitLab) - [How to install Git](how_to_install_git/index.md) - [Start using Git on the command line](../../gitlab-basics/start-using-git.md) - [Command Line basic commands](../../gitlab-basics/command-line-commands.md) - [GitLab Git Cheat Sheet (download)](https://about.gitlab.com/images/press/git-cheat-sheet.pdf) -- Commits - - [Revert a commit](../../user/project/merge_requests/revert_changes.md#reverting-a-commit) +- Commits: + - [Revert a commit](../../user/project/merge_requests/revert_changes.md#reverting-a-commit) - [Cherry-picking a commit](../../user/project/merge_requests/cherry_pick_changes.md#cherry-picking-a-commit) - [Squashing commits](../../workflow/gitlab_flow.md#squashing-commits-with-rebase) -**Third-party references:** +### Concepts -- [Getting Started - Git website](https://git-scm.com) -- [Getting Started - Version control](https://git-scm.com/book/en/v2/Getting-Started-About-Version-Control) -- [Getting Started - Git Basics](https://git-scm.com/book/en/v2/Getting-Started-Git-Basics) -- [Getting Started - Installing Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) -- [Git on the Server - GitLab](https://git-scm.com/book/en/v2/Git-on-the-Server-GitLab) +The following are resources about version control concepts: -### Concepts +- [Git concepts](../../university/training/user_training.md#git-concepts) +- [Why Git is Worth the Learning Curve](https://about.gitlab.com/2017/05/17/learning-curve-is-the-biggest-challenge-developers-face-with-git/) +- [The future of SaaS hosted Git repository pricing](https://about.gitlab.com/2016/05/11/git-repository-pricing/) +- [Git website topic about version control](https://git-scm.com/book/en/v2/Getting-Started-About-Version-Control) +- [GitLab University presentation about Version Control](https://docs.google.com/presentation/d/16sX7hUrCZyOFbpvnrAFrg6tVO5_yT98IgdAqOmXwBho/edit?usp=sharing) -- Article (2017-05-17): [Why Git is Worth the Learning Curve](https://about.gitlab.com/2017/05/17/learning-curve-is-the-biggest-challenge-developers-face-with-git/) -- Article (2016-05-11): [The future of SaaS hosted Git repository pricing](https://about.gitlab.com/2016/05/11/git-repository-pricing/) -- GLU Course (Presentation): [About Version Control](https://docs.google.com/presentation/d/16sX7hUrCZyOFbpvnrAFrg6tVO5_yT98IgdAqOmXwBho/edit?usp=sharing) +## Git tips -## Exploring Git +The following resources may help you become more efficient at using Git: - [Git Tips & Tricks](https://about.gitlab.com/2016/12/08/git-tips-and-tricks/) - [Eight Tips to help you work better with Git](https://about.gitlab.com/2015/02/19/8-tips-to-help-you-work-better-with-git/) ## Troubleshooting Git +If you have problems with Git, the following may help: + - [Numerous _undo_ possibilities in Git](numerous_undo_possibilities_in_git/index.md) -- Learn a few [Git troubleshooting](troubleshooting_git.md) techniques to help you out. +- Learn a few [Git troubleshooting](troubleshooting_git.md) techniques ## Branching strategies -- [GitLab Flow](https://about.gitlab.com/2014/09/29/gitlab-flow/) - -**Third-party references:** - - [Git Branching - Branches in a Nutshell](https://git-scm.com/book/en/v2/Git-Branching-Branches-in-a-Nutshell) - [Git Branching - Branching Workflows](https://git-scm.com/book/en/v2/Git-Branching-Branching-Workflows) +- [GitLab Flow](https://about.gitlab.com/2014/09/29/gitlab-flow/) ## Advanced use +The following are advanced topics for those who want to get the most out of Git: + - [Custom Git Hooks](../../administration/custom_hooks.md) - [Git Attributes](../../user/project/git_attributes.md) - Git Submodules: [Using Git submodules with GitLab CI](../../ci/git_submodules.md#using-git-submodules-with-gitlab-ci) ## API -- [Gitignore templates](../../api/templates/gitignores.md) +[Gitignore templates](../../api/templates/gitignores.md) API allow for +Git-related queries from GitLab. ## Git LFS +The following relate to Git Large File Storage: + - [Getting Started with Git LFS](https://about.gitlab.com/2017/01/30/getting-started-with-git-lfs-tutorial/) - [GitLab Git LFS documentation](../../workflow/lfs/manage_large_binaries_with_git_lfs.md) -- [Git-Annex to Git-LFS migration guide](https://docs.gitlab.com/ee/workflow/lfs/migrate_from_git_annex_to_git_lfs.html) -- Article (2015-08-13): [Towards a production quality open source Git LFS server](https://about.gitlab.com/2015/08/13/towards-a-production-quality-open-source-git-lfs-server/) +- [Git-Annex to Git-LFS migration guide](../../workflow/lfs/migrate_from_git_annex_to_git_lfs.md) +- [Towards a production quality open source Git LFS server](https://about.gitlab.com/2015/08/13/towards-a-production-quality-open-source-git-lfs-server/) diff --git a/doc/topics/git/numerous_undo_possibilities_in_git/index.md b/doc/topics/git/numerous_undo_possibilities_in_git/index.md index 8a8021dc36d..84201e11831 100644 --- a/doc/topics/git/numerous_undo_possibilities_in_git/index.md +++ b/doc/topics/git/numerous_undo_possibilities_in_git/index.md @@ -4,25 +4,30 @@ author_gitlab: Letme level: intermediary article_type: tutorial date: 2017-05-15 +type: howto +last_updated: 2019-05-31 --- # Numerous undo possibilities in Git -## Introduction - In this tutorial, we will show you different ways of undoing your work in Git, for which we will assume you have a basic working knowledge of. Check GitLab's -[Git documentation](../index.md#git-documentation) for reference. +[Git documentation](../index.md) for reference. + Also, we will only provide some general info of the commands, which is enough -to get you started for the easy cases/examples, but for anything more advanced please refer to the [Git book](https://git-scm.com/book/en/v2). +to get you started for the easy cases/examples, but for anything more advanced +please refer to the [Git book](https://git-scm.com/book/en/v2). We will explain a few different techniques to undo your changes based on the stage of the change in your current development. Also, keep in mind that [nothing in -Git is really deleted.][git-autoclean-ref] +Git is really deleted][git-autoclean-ref]. + This means that until Git automatically cleans detached commits (which cannot be accessed by branch or tag) it will be possible to view them with `git reflog` command and access them with direct commit-id. Read more about _[redoing the undo](#redoing-the-undo)_ on the section below. +## Introduction + This guide is organized depending on the [stage of development][git-basics] where you want to undo your changes from and if they were shared with other developers or not. Because Git is tracking changes a created or edited file is in the unstaged state @@ -31,35 +36,41 @@ a file into the **staged** state, which is then committed (`git commit`) to your local repository. After that, file can be shared with other developers (`git push`). Here's what we'll cover in this tutorial: - - [Undo local changes](#undo-local-changes) which were not pushed to remote repository +- [Undo local changes](#undo-local-changes) which were not pushed to remote repository: - - Before you commit, in both unstaged and staged state - - After you committed + - Before you commit, in both unstaged and staged state. + - After you committed. - - Undo changes after they are pushed to remote repository +- Undo changes after they are pushed to remote repository: - - [Without history modification](#undo-remote-changes-without-changing-history) (preferred way) - - [With history modification](#undo-remote-changes-with-modifying-history) (requires - coordination with team and force pushes). - - [Usecases when modifying history is generally acceptable](#where-modifying-history-is-generally-acceptable) - - [How to modify history](#how-modifying-history-is-done) - - [How to remove sensitive information from repository](#deleting-sensitive-information-from-commits) + - [Without history modification](#undo-remote-changes-without-changing-history) (preferred way). + - [With history modification](#undo-remote-changes-with-modifying-history) (requires + coordination with team and force pushes). + - [Use cases when modifying history is generally acceptable](#where-modifying-history-is-generally-acceptable). + - [How to modify history](#how-modifying-history-is-done). + - [How to remove sensitive information from repository](#deleting-sensitive-information-from-commits). ### Branching strategy [Git][git-official] is a de-centralized version control system, which means that beside regular versioning of the whole repository, it has possibilities to exchange changes -with other repositories. To avoid chaos with +with other repositories. + +To avoid chaos with [multiple sources of truth][git-distributed], various development workflows have to be followed, and it depends on your internal workflow how certain changes or commits can be undone or changed. + [GitLab Flow][gitlab-flow] provides a good balance between developers clashing with each other while developing the same feature and cooperating seamlessly, but it does not enable joined development of the same feature by multiple developers by default. + When multiple developers develop the same feature on the same branch, clashing with every synchronization is unavoidable, but a proper or chosen Git Workflow will -prevent that anything is lost or out of sync when feature is complete. You can also +prevent that anything is lost or out of sync when feature is complete. + +You can also read through this blog post on [Git Tips & Tricks][gitlab-git-tips-n-tricks] to learn how to easily **do** things in Git. @@ -97,19 +108,19 @@ no changes added to commit (use "git add" and/or "git commit -a") At this point there are 3 options to undo the local changes you have: -- Discard all local changes, but save them for possible re-use [later](#quickly-save-local-changes) +- Discard all local changes, but save them for possible re-use [later](#quickly-save-local-changes): ```shell git stash ``` -- Discarding local changes (permanently) to a file +- Discarding local changes (permanently) to a file: ```shell git checkout -- <file> ``` -- Discard all local changes to all files permanently +- Discard all local changes to all files permanently: ```shell git reset --hard @@ -150,7 +161,7 @@ of the staging tree. You also have an option to discard all changes with Lets start the example by editing a file, with your favorite editor, to change the content and add it to staging -``` +```sh vim <file> git add <file> ``` @@ -164,30 +175,30 @@ Your branch is up-to-date with 'origin/master'. Changes to be committed: (use "git reset HEAD <file>..." to unstage) - new file: <file> + new file: <file> ``` Now you have 4 options to undo your changes: -- Unstage the file to current commit (HEAD) +- Unstage the file to current commit (HEAD): ```shell git reset HEAD <file> ``` -- Unstage everything - retain changes +- Unstage everything - retain changes: ```shell git reset ``` -- Discard all local changes, but save them for [later](#quickly-save-local-changes) +- Discard all local changes, but save them for [later](#quickly-save-local-changes): ```shell git stash ``` -- Discard everything permanently +- Discard everything permanently: ```shell git reset --hard @@ -206,7 +217,9 @@ your code, you'll have less options to troubleshoot your work. Through the development process some of the previously committed changes do not fit anymore in the end solution, or are source of the bugs. Once you find the commit which triggered bug, or once you have a faulty commit, you can simply -revert it with `git revert commit-id`. This command inverts (swaps) the additions and +revert it with `git revert commit-id`. + +This command inverts (swaps) the additions and deletions in that commit, so that it does not modify history. Retaining history can be helpful in future to notice that some changes have been tried unsuccessfully in the past. @@ -225,19 +238,19 @@ through simple bisection process. You can read more about it [in official Git To In our example we will end up with commit `B`, that introduced bug/error. We have 4 options on how to remove it (or part of it) from our repository. -- Undo (swap additions and deletions) changes introduced by commit `B`. +- Undo (swap additions and deletions) changes introduced by commit `B`: ```shell git revert commit-B-id ``` -- Undo changes on a single file or directory from commit `B`, but retain them in the staged state +- Undo changes on a single file or directory from commit `B`, but retain them in the staged state: ```shell git checkout commit-B-id <file> ``` -- Undo changes on a single file or directory from commit `B`, but retain them in the unstaged state +- Undo changes on a single file or directory from commit `B`, but retain them in the unstaged state: ```shell git reset commit-B-id <file> @@ -246,7 +259,9 @@ In our example we will end up with commit `B`, that introduced bug/error. We hav - There is one command we also must not forget: **creating a new branch** from the point where changes are not applicable or where the development has hit a dead end. For example you have done commits `A-B-C-D` on your feature-branch - and then you figure `C` and `D` are wrong. At this point you either reset to `B` + and then you figure `C` and `D` are wrong. + + At this point you either reset to `B` and do commit `F` (which will cause problems with pushing and if forced pushed also with other developers) since branch now looks `A-B-F`, which clashes with what other developers have locally (you will [change history](#with-history-modification)), or you simply checkout commit `B` create @@ -269,13 +284,13 @@ In our example we will end up with commit `B`, that introduced bug/error. We hav There is one command for history modification and that is `git rebase`. Command provides interactive mode (`-i` flag) which enables you to: - - **reword** commit messages (there is also `git commit --amend` for editing - last commit message) - - **edit** the commit content (changes introduced by commit) and message - - **squash** multiple commits into a single one, and have a custom or aggregated - commit message - - **drop** commits - simply delete them - - and few more options +- **reword** commit messages (there is also `git commit --amend` for editing + last commit message). +- **edit** the commit content (changes introduced by commit) and message. +- **squash** multiple commits into a single one, and have a custom or aggregated + commit message. +- **drop** commits - simply delete them. +- and few more options. Let us check few examples. Again there are commits `A-B-C-D` where you want to delete commit `B`. @@ -301,7 +316,7 @@ In case you want to modify something introduced in commit `B`. - Command opens your favorite text editor where you write `edit` in front of commit `B`, but leave default `pick` with all other commits. Save and exit the editor to - perform a rebase + perform a rebase. - Now do your edits and commit changes: @@ -348,7 +363,9 @@ and then on end description of that action. This topic is roughly same as modifying committed local changes without modifying history. **It should be the preferred way of undoing changes on any remote repository or public branch.** Keep in mind that branching is the best solution when you want -to retain the history of faulty development, yet start anew from certain point. Branching +to retain the history of faulty development, yet start anew from certain point. + +Branching enables you to include the existing changes in new development (by merging) and it also provides a clear timeline and development structure. @@ -386,12 +403,14 @@ the cleanup of detached commits (happens automatically). Modified history breaks the development chain of other developers, as changed history does not have matching commits'ids. For that reason it should not be used on any public branch or on branch that *might* be used by other -developers. When contributing to big open source repositories (e.g. [GitLab CE][gitlab-ce]), +developers. When contributing to big open source repositories (for example, [GitLab CE][gitlab-ce]), it is acceptable to *squash* commits into a single one, to present a nicer history of your contribution. + Keep in mind that this also removes the comments attached to certain commits in merge requests, so if you need to retain traceability in GitLab, then modifying history is not acceptable. + A feature-branch of a merge request is a public branch and might be used by other developers, but project process and rules might allow or require you to use `git rebase` (command that changes history) to reduce number of @@ -400,8 +419,8 @@ GitLab). There is a `git merge --squash` command which does exactly that (squashes commits on feature-branch to a single commit on target branch at merge). ->**Note:** -Never modify the commit history of `master` or shared branch +NOTE: **Note:** +Never modify the commit history of `master` or shared branch. ### How modifying history is done @@ -436,7 +455,7 @@ pick <commit3-id> <commit3-commit-message> # Note that empty commits are commented out ``` ->**Note:** +NOTE: **Note:** It is important to notice that comment from the output clearly states that, if you decide to abort, then do not just close your editor (as that will in-fact modify history), but remove all uncommented lines and save. @@ -470,7 +489,7 @@ tools that can use some of Git specifics to enable faster execution of common tasks (which is exactly what removing sensitive information file is about). An alternative is [BFG Repo-cleaner][bfg-repo-cleaner]. Keep in mind that these tools are faster because they do not provide a same fully feature set as `git filter-branch` -does, but focus on specific usecases. +does, but focus on specific use cases. ## Conclusion @@ -480,6 +499,18 @@ depending on the stage of your process. Git also enables rewriting history, but should be avoided as it might cause problems when multiple developers are contributing to the same codebase. +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> + <!-- Identifiers, in alphabetical order --> [bfg-repo-cleaner]: https://rtyley.github.io/bfg-repo-cleaner/ diff --git a/doc/topics/git/troubleshooting_git.md b/doc/topics/git/troubleshooting_git.md index 71651fcf421..417d91bf834 100644 --- a/doc/topics/git/troubleshooting_git.md +++ b/doc/topics/git/troubleshooting_git.md @@ -1,3 +1,7 @@ +--- +type: howto +--- + # Troubleshooting Git Sometimes things don't work the way they should or as you might expect when @@ -9,7 +13,7 @@ with Git. 'Broken pipe' errors can occur when attempting to push to a remote repository. When pushing you will usually see: -``` +```text Write failed: Broken pipe fatal: The remote end hung up unexpectedly ``` @@ -39,14 +43,15 @@ There's another option where you can prevent session timeouts by configuring SSH 'keep alive' either on the client or on the server (if you are a GitLab admin and have access to the server). -NOTE: **Note:** configuring *both* the client and the server is unnecessary. +NOTE: **Note:** +Configuring *both* the client and the server is unnecessary. **To configure SSH on the client side**: -- On UNIX, edit `~/.ssh/config` (create the file if it doesn’t exist) and - add or edit: +- On UNIX, edit `~/.ssh/config` (create the file if it doesn’t exist) and + add or edit: - ``` + ```text Host your-gitlab-instance-url.com ServerAliveInterval 60 ServerAliveCountMax 5 @@ -58,7 +63,7 @@ NOTE: **Note:** configuring *both* the client and the server is unnecessary. **To configure SSH on the server side**, edit `/etc/ssh/sshd_config` and add: -``` +```text ClientAliveInterval 60 ClientAliveCountMax 5 ``` @@ -83,35 +88,35 @@ to >= 2.9 (see [Broken pipe when pushing to Git repository][Broken-Pipe]). Users may experience the following error when attempting to push or pull using Git over SSH: -``` -Please make sure you have the correct access rights -and the repository exists. +```text +Please make sure you have the correct access rights +and the repository exists. ... -ssh_exchange_identification: read: Connection reset by peer -fatal: Could not read from remote repository. +ssh_exchange_identification: read: Connection reset by peer +fatal: Could not read from remote repository. ``` This error usually indicates that SSH daemon's `MaxStartups` value is throttling -SSH connections. This setting specifies the maximum number of unauthenticated +SSH connections. This setting specifies the maximum number of unauthenticated connections to the SSH daemon. This affects users with proper authentication credentials (SSH keys) because every connection is 'unauthenticated' in the -beginning. The default value is `10`. +beginning. The default value is `10`. Increase `MaxStartups` by adding or modifying the value in `/etc/ssh/sshd_config`: -``` +```text MaxStartups 100 ``` -Restart SSHD for the change to take effect. +Restart SSHD for the change to take effect. ## Timeout during git push/pull If pulling/pushing from/to your repository ends up taking more than 50 seconds, -a timeout will be issued with a log of the number of operations performed +a timeout will be issued with a log of the number of operations performed and their respective timings, like the example below: -``` +```text remote: Running checks for branch: master remote: Scanning for LFS objects... (153ms) remote: Calculating new repository size... (cancelled after 729ms) diff --git a/doc/university/README.md b/doc/university/README.md index cf13246067f..61ed72d25fb 100644 --- a/doc/university/README.md +++ b/doc/university/README.md @@ -51,10 +51,10 @@ The curriculum is composed of GitLab videos, screencasts, presentations, project #### 1.5. Migrating from other Source Control -1. [Migrating from BitBucket/Stash](https://docs.gitlab.com/ee/user/project/import/bitbucket.html) -1. [Migrating from GitHub](https://docs.gitlab.com/ee/user/project/import/github.html) -1. [Migrating from SVN](https://docs.gitlab.com/ee/user/project/import/svn.html) -1. [Migrating from Fogbugz](https://docs.gitlab.com/ee/user/project/import/fogbugz.html) +1. [Migrating from BitBucket/Stash](../user/project/import/bitbucket.md) +1. [Migrating from GitHub](../user/project/import/github.md) +1. [Migrating from SVN](../user/project/import/svn.md) +1. [Migrating from Fogbugz](../user/project/import/fogbugz.md) #### 1.6. GitLab Inc. @@ -92,7 +92,7 @@ The curriculum is composed of GitLab videos, screencasts, presentations, project 1. [Using any Static Site Generator with GitLab Pages](https://about.gitlab.com/2016/06/17/ssg-overview-gitlab-pages-part-3-examples-ci/) 1. [Securing GitLab Pages with SSL](https://about.gitlab.com/2016/06/24/secure-gitlab-pages-with-startssl/) -1. [GitLab Pages Documentation](https://docs.gitlab.com/ce/user/project/pages/) +1. [GitLab Pages Documentation](../user/project/pages/index.md) #### 2.2. GitLab Issues @@ -131,7 +131,7 @@ The curriculum is composed of GitLab videos, screencasts, presentations, project 1. [GitLab Flow vs Forking in GitLab - Video](https://www.youtube.com/watch?v=UGotqAUACZA) 1. [GitLab Flow Overview](https://about.gitlab.com/2014/09/29/gitlab-flow/) 1. [Always Start with an Issue](https://about.gitlab.com/2016/03/03/start-with-an-issue/) -1. [GitLab Flow Documentation](https://docs.gitlab.com/ee/workflow/gitlab_flow.html) +1. [GitLab Flow Documentation](../workflow/gitlab_flow.md) #### 2.5. GitLab Comparisons @@ -191,10 +191,10 @@ The curriculum is composed of GitLab videos, screencasts, presentations, project #### 3.9. Integrations 1. [How to Integrate JIRA and Jenkins with GitLab - Video](https://gitlabmeetings.webex.com/gitlabmeetings/ldr.php?RCID=44b548147a67ab4d8a62274047146415) -1. [How to Integrate Jira with GitLab](https://docs.gitlab.com/ce/user/project/integrations/jira.html) -1. [How to Integrate Jenkins with GitLab](https://docs.gitlab.com/ee/integration/jenkins.html) -1. [How to Integrate Bamboo with GitLab](https://docs.gitlab.com/ce/user/project/integrations/bamboo.html) -1. [How to Integrate Slack with GitLab](https://docs.gitlab.com/ce/user/project/integrations/slack.html) +1. [How to Integrate Jira with GitLab](../user/project/integrations/jira.md) +1. [How to Integrate Jenkins with GitLab](../integration/jenkins.md) +1. [How to Integrate Bamboo with GitLab](../user/project/integrations/bamboo.md) +1. [How to Integrate Slack with GitLab](../user/project/integrations/slack.md) 1. [How to Integrate Convox with GitLab](https://about.gitlab.com/2016/06/09/continuous-delivery-with-gitlab-and-convox/) 1. [Getting Started with GitLab and Shippable CI](https://about.gitlab.com/2016/05/05/getting-started-gitlab-and-shippable/) diff --git a/doc/university/glossary/README.md b/doc/university/glossary/README.md index 0af2f8d2f54..f15b0107de5 100644 --- a/doc/university/glossary/README.md +++ b/doc/university/glossary/README.md @@ -62,7 +62,7 @@ Entry level [subscription](https://about.gitlab.com/pricing/) for GitLab EE curr ### Bitbucket -Atlassian's web hosting service for Git and Mercurial Projects. Read about [migrating](https://docs.gitlab.com/ce/workflow/importing/import_projects_from_bitbucket.html) from BitBucket to a GitLab instance. +Atlassian's web hosting service for Git and Mercurial Projects. Read about [migrating](../../user/project/import/bitbucket.md) from BitBucket to a GitLab instance. ### Branch @@ -70,10 +70,10 @@ A branch is a parallel version of a repository. This allows you to work on the r ### Branded Login -Having your own logo on [your GitLab instance login page](https://docs.gitlab.com/ee/customization/branded_login_page.html) instead of the GitLab logo. +Having your own logo on [your GitLab instance login page](../../customization/branded_login_page.md) instead of the GitLab logo. ### Job triggers (Build Triggers) -These protect your code base against breaks, for instance when a team is working on the same project. Learn about [setting up](https://docs.gitlab.com/ce/ci/triggers/README.html) job triggers. +These protect your code base against breaks, for instance when a team is working on the same project. Learn about [setting up](../../ci/triggers/README.md) job triggers. ### CEPH @@ -149,7 +149,7 @@ As in "specify [dependencies](https://gitlab.com/gitlab-org/gitlab-ce/issues/147 ### Deploy Keys -A [SSH key](https://docs.gitlab.com/ce/gitlab-basics/create-your-ssh-keys.html)stored on your server that grants access to a single GitLab repository. This is used by a GitLab runner to clone a project's code so that tests can be run against the checked out code. +A [SSH key](../../gitlab-basics/create-your-ssh-keys.md)stored on your server that grants access to a single GitLab repository. This is used by a GitLab runner to clone a project's code so that tests can be run against the checked out code. ### Developer @@ -169,7 +169,7 @@ A folder used for storing multiple files. ### Docker Container Registry -A [feature](https://docs.gitlab.com/ce/user/project/container_registry.html) of [GitLab projects](https://about.gitlab.com/2016/05/23/gitlab-container-registry/). Containers wrap up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, system libraries – anything you can install on a server. This guarantees that it will always run the same, regardless of the environment it is running in. +A [feature](../../user/project/container_registry.md) of [GitLab projects](https://about.gitlab.com/2016/05/23/gitlab-container-registry/). Containers wrap up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, system libraries – anything you can install on a server. This guarantees that it will always run the same, regardless of the environment it is running in. ### Dynamic Environment (review apps) @@ -191,7 +191,7 @@ First Byte (sometimes referred to as time to first byte or [TTFB](https://en.wik ### Fork -Your [own copy](https://docs.gitlab.com/ce/workflow/forking_workflow.html) of a repository that allows you to make changes to the repository without affecting the original. +Your [own copy](../../workflow/forking_workflow.md) of a repository that allows you to make changes to the repository without affecting the original. ### Funnel, or: TOFU, MOFU, BOFU @@ -221,7 +221,7 @@ A single-tenant solution that provides GitLab CE or EE as a managed service. Git ### GitHub -A web-based Git repository hosting service with an enterprise offering. Its main features are: issue tracking, pull request with code review, abundancy of integrations and wiki. It offers free public repos, private repos and enterprise services are paid. Read about [importing a project](https://docs.gitlab.com/ce/workflow/importing/import_projects_from_github.html) from GitHub to GitLab. +A web-based Git repository hosting service with an enterprise offering. Its main features are: issue tracking, pull request with code review, abundancy of integrations and wiki. It offers free public repos, private repos and enterprise services are paid. Read about [importing a project](../../workflow/importing/import_projects_from_github.md) from GitHub to GitLab. ### GitLab CE @@ -241,7 +241,7 @@ Our free SaaS for public and private repositories. ### GitLab Geo -Allows you to replicate your GitLab instance to other geographical locations as a read-only fully operational version. It [can be used](https://docs.gitlab.com/ee/administration/geo/replication/index.html) for cloning and fetching projects, in addition to reading any data. This will make working with large repositories over large distances much faster. +Allows you to replicate your GitLab instance to other geographical locations as a read-only fully operational version. It [can be used](../../administration/geo/replication/index.md) for cloning and fetching projects, in addition to reading any data. This will make working with large repositories over large distances much faster. ### GitLab High Availability @@ -299,15 +299,15 @@ An [application layer protocol](http://www.irchelp.org/) that facilitates commun ### Issue Tracker -A [tool](https://docs.gitlab.com/ee/integration/external-issue-tracker.html) used to manage, organize, and maintain a list of issues, making it easier for an organization to manage. +A [tool](../../integration/external-issue-tracker.md) used to manage, organize, and maintain a list of issues, making it easier for an organization to manage. ### Jenkins -An Open Source CI tool written using the Java programming language. [Jenkins](https://jenkins.io/) does the same job as GitLab CI, Bamboo, and Travis CI. It is extremely popular. Related [documentation](https://docs.gitlab.com/ee/integration/jenkins.html). +An Open Source CI tool written using the Java programming language. [Jenkins](https://jenkins.io/) does the same job as GitLab CI, Bamboo, and Travis CI. It is extremely popular. Related [documentation](../../integration/jenkins.md). ### Jira -Atlassian's [project management software](https://www.atlassian.com/software/jira), i.e. a complex issue tracker. GitLab [can be configured](https://docs.gitlab.com/ee/project_services/jira.html) to interact with JIRA Core either using an on-premise instance or the SaaS solution that Atlassian offers. +Atlassian's [project management software](https://www.atlassian.com/software/jira), i.e. a complex issue tracker. GitLab [can be configured](../../project_services/jira.md) to interact with JIRA Core either using an on-premise instance or the SaaS solution that Atlassian offers. ### JUnit @@ -323,7 +323,7 @@ An open source container cluster manager originally designed by Google. It's bas ### Labels -An [identifier](https://docs.gitlab.com/ce/user/project/labels.html) to describe a group of one or more specific file revisions. +An [identifier](../../user/project/labels.md) to describe a group of one or more specific file revisions. ### Lightweight Directory Access Protocol (LDAP) @@ -331,7 +331,7 @@ An [identifier](https://docs.gitlab.com/ce/user/project/labels.html) to describe ### LDAP User Authentication -GitLab [integrates](https://docs.gitlab.com/ce/administration/auth/ldap.html) with LDAP to support user authentication. This enables GitLab to sign in people from an LDAP server (i.e., allowing people whose names are on the electronic user directory server to be able to use their LDAP accounts to login.) +GitLab [integrates](../../administration/auth/ldap.md) with LDAP to support user authentication. This enables GitLab to sign in people from an LDAP server (i.e., allowing people whose names are on the electronic user directory server to be able to use their LDAP accounts to login.) ### LDAP Group Sync @@ -381,9 +381,9 @@ Takes changes from one branch, and [applies them](https://git-scm.com/docs/git-m [Arises](https://about.gitlab.com/2016/09/06/resolving-merge-conflicts-from-the-gitlab-ui/) when a merge can't be performed cleanly between two versions of the same file. -#### Merge Request +#### Merge Request (MR) -[Takes changes](https://docs.gitlab.com/ce/gitlab-basics/add-merge-request.html) from one branch, and applies them into another branch. +[Takes changes](../../gitlab-basics/add-merge-request.md) from one branch, and applies them into another branch. ### Meteor @@ -395,11 +395,11 @@ Allow you to [organize issues](../../user/project/milestones/index.md) and merge ### Mirror Repositories -A project that is set up to automatically have its branches, tags, and commits [updated from an upstream repository](https://docs.gitlab.com/ee/workflow/repository_mirroring.html). This is useful when a repository you're interested in is located on a different server, and you want to be able to browse its content and activity using the familiar GitLab interface. +A project that is set up to automatically have its branches, tags, and commits [updated from an upstream repository](../../workflow/repository_mirroring.md). This is useful when a repository you're interested in is located on a different server, and you want to be able to browse its content and activity using the familiar GitLab interface. ### MIT License -A type of software license. It lets people do anything with your code with proper attribution and without warranty. It is the most common license for open source applications written in Ruby on Rails. GitLab CE is issued under this [license](https://docs.gitlab.com/ce/development/licensing.html). This means you can download the code, modify it as you want, and even build a new commercial product using the underlying code and it's not illegal. The only condition is that there is no form of warranty provided by GitLab so whatever happens when you use the code is your own problem. +A type of software license. It lets people do anything with your code with proper attribution and without warranty. It is the most common license for open source applications written in Ruby on Rails. GitLab CE is issued under this [license](../../development/licensing.md). This means you can download the code, modify it as you want, and even build a new commercial product using the underlying code and it's not illegal. The only condition is that there is no form of warranty provided by GitLab so whatever happens when you use the code is your own problem. ### Mondo Rescue @@ -427,7 +427,7 @@ A web [server](https://www.nginx.com/resources/wiki/) (pronounced "engine x"). [ ### OAuth -An open standard for authorization, commonly used as a way for internet users to log into third party websites using their Microsoft, Google, Facebook or Twitter accounts without exposing their password. GitLab [is](https://docs.gitlab.com/ce/integration/oauth_provider.html) an OAuth2 authentication service provider. +An open standard for authorization, commonly used as a way for internet users to log into third party websites using their Microsoft, Google, Facebook or Twitter accounts without exposing their password. GitLab [is](../../integration/oauth_provider.md) an OAuth2 authentication service provider. ### Omnibus Packages @@ -479,11 +479,11 @@ An [object-relational](https://en.wikipedia.org/wiki/PostgreSQL) database. Toute ### Protected Branches -A [feature](https://docs.gitlab.com/ce/user/project/protected_branches.html) that protects branches from unauthorized pushes, force pushing or deletion. +A [feature](../../user/project/protected_branches.md) that protects branches from unauthorized pushes, force pushing or deletion. ### Protected Tags -A [feature](https://docs.gitlab.com/ce/user/project/protected_tags.html) that protects tags from unauthorized creation, update or deletion +A [feature](../../user/project/protected_tags.md) that protects tags from unauthorized creation, update or deletion ### Pull @@ -547,7 +547,7 @@ Actual build machines/containers that [run and execute tests](https://gitlab.com ### Sidekiq -The background job processor GitLab [uses](https://docs.gitlab.com/ce/administration/troubleshooting/sidekiq.html) to asynchronously run tasks. +The background job processor GitLab [uses](../../administration/troubleshooting/sidekiq.md) to asynchronously run tasks. ### Software as a service (SaaS) @@ -567,7 +567,7 @@ The board used to track the status and progress of each of the sprint backlog it ### Shell -Terminal on Mac OSX, GitBash on Windows, or Linux Terminal on Linux. You [use git](https://docs.gitlab.com/ce/gitlab-basics/start-using-git.html) and make changes to GitLab projects in your shell. You [use git](https://docs.gitlab.com/ce/gitlab-basics/start-using-git.html) and make changes to GitLab projects in your shell. +Terminal on Mac OSX, GitBash on Windows, or Linux Terminal on Linux. You [use git](../../gitlab-basics/start-using-git.md) and make changes to GitLab projects in your shell. You [use git](../../gitlab-basics/start-using-git.md) and make changes to GitLab projects in your shell. ### Shell command runner @@ -577,7 +577,7 @@ The tenant purchases their own copy of the software and the software can be cust ### Slack -Real time messaging app for teams that is used internally by GitLab team members. GitLab users can enable [Slack integration](https://docs.gitlab.com/ce/project_services/slack.html) to trigger push, issue, and merge request events among others. +Real time messaging app for teams that is used internally by GitLab team members. GitLab users can enable [Slack integration](../../project_services/slack.md) to trigger push, issue, and merge request events among others. ### Slash commands @@ -595,7 +595,7 @@ Program code as typed by a computer programmer (i.e. it has not yet been compile ### SSH Key -A unique identifier of a computer. It is used to identify computers without the need for a password (e.g., On GitLab I have [added the ssh key](https://docs.gitlab.com/ce/gitlab-basics/create-your-ssh-keys.html) of all my work machines so that the GitLab instance knows that it can accept code pushes and pulls from this trusted machines whose keys are I have added.) +A unique identifier of a computer. It is used to identify computers without the need for a password (e.g., On GitLab I have [added the ssh key](../../gitlab-basics/create-your-ssh-keys.md) of all my work machines so that the GitLab instance knows that it can accept code pushes and pulls from this trusted machines whose keys are I have added.) ### Single Sign On (SSO) @@ -627,11 +627,11 @@ A program that allows you to perform superuser/administrator actions on Unix Ope ### Subversion (SVN) -An open source version control system. Read about [migrating from SVN](https://docs.gitlab.com/ce/workflow/importing/migrating_from_svn.html) to GitLab using SubGit. +An open source version control system. Read about [migrating from SVN](../../workflow/importing/migrating_from_svn.md) to GitLab using SubGit. ### Tag -[Represents](https://docs.gitlab.com/ce/api/tags.html) a version of a particular branch at a moment in time. +[Represents](../../api/tags.md) a version of a particular branch at a moment in time. ### Tenancy @@ -673,7 +673,7 @@ Version control is a system that records changes to a file or set of files over ### Virtual Private Cloud (VPC) -A [VPC](https://docs.gitlab.com/ce/university/glossary/README.html#virtual-private-cloud-vpc) is an on demand configurable pool of shared computing resources allocated within a public cloud environment, providing some isolation between the different users using the resources. GitLab users need to create a new Amazon VPC in order to [set up High Availability](https://docs.gitlab.com/ce/university/high-availability/aws/). +A [VPC](#virtual-private-cloud-vpc) is an on demand configurable pool of shared computing resources allocated within a public cloud environment, providing some isolation between the different users using the resources. GitLab users need to create a new Amazon VPC in order to [set up High Availability](../../install/aws/index.md). ### Virtual private server (VPS) @@ -689,7 +689,7 @@ A [model](http://www.umsl.edu/~hugheyd/is6840/waterfall.html) of building softwa ### Webhooks -A way for an app to [provide](https://docs.gitlab.com/ce/user/project/integrations/webhooks.html) other applications with real-time information (e.g., send a message to a slack channel when a commit is pushed.) Read about setting up [custom git hooks](https://gitlab.com/help/administration/custom_hooks.md) for when webhooks are insufficient. +A way for an app to [provide](../../user/project/integrations/webhooks.md) other applications with real-time information (e.g., send a message to a slack channel when a commit is pushed.) Read about setting up [custom git hooks](../../administration/custom_hooks.md) for when webhooks are insufficient. ### Wiki diff --git a/doc/university/high-availability/aws/README.md b/doc/university/high-availability/aws/README.md index 01d5073ab7e..fa04e988042 100644 --- a/doc/university/high-availability/aws/README.md +++ b/doc/university/high-availability/aws/README.md @@ -390,5 +390,5 @@ some redundancy options but it might also imply Geographic replication. There is a lot of ground yet to cover so have a read through these other resources and feel free to open an issue to request additional material. -- [GitLab High Availability](http://docs.gitlab.com/ce/administration/high_availability/README.html#sts=High%20Availability) -- [GitLab Geo](https://docs.gitlab.com/ee/administration/geo/replication/index.html) +- [GitLab High Availability](../../../administration/high_availability/README.md) +- [GitLab Geo](../../../administration/geo/replication/index.md) diff --git a/doc/university/support/README.md b/doc/university/support/README.md index feb90ae9bad..c8ade54a77c 100644 --- a/doc/university/support/README.md +++ b/doc/university/support/README.md @@ -61,28 +61,27 @@ Sometimes we need to upgrade customers from old versions of GitLab to latest, so - Users - Groups - Projects - - [Backup using our Backup rake task](https://docs.gitlab.com/ce/raketasks/backup_restore.html#create-a-backup-of-the-gitlab-system) + - [Backup using our Backup rake task](../../raketasks/backup_restore.md#creating-a-backup-of-the-gitlab-system) - [Upgrade to 5.0 source using our Upgrade documentation](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/doc/update/4.2-to-5.0.md) - [Upgrade to 5.1 source](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/doc/update/5.0-to-5.1.md) - [Upgrade to 6.0 source](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/doc/update/5.1-to-6.0.md) - [Upgrade to 7.14 source](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/doc/update/6.x-or-7.x-to-7.14.md) - - [Backup using our Backup rake task](https://docs.gitlab.com/ce/raketasks/backup_restore.html#create-a-backup-of-the-gitlab-system) - - [Perform the MySQL to PostgreSQL migration to convert your backup](https://docs.gitlab.com/ce/update/mysql_to_postgresql.html#converting-a-gitlab-backup-file-from-mysql-to-postgres) + - [Perform the MySQL to PostgreSQL migration to convert your backup](../../update/mysql_to_postgresql.md) - [Upgrade to Omnibus 7.14](https://docs.gitlab.com/omnibus/update/README.html#upgrading-from-a-non-omnibus-installation-to-an-omnibus-installation) - - [Restore backup using our Restore rake task](https://docs.gitlab.com/ce/raketasks/backup_restore.html#restore-a-previously-created-backup) + - [Restore backup using our Restore rake task](../../raketasks/backup_restore.md#restore) - [Upgrade to latest EE](https://about.gitlab.com/downloads-ee) - (GitLab inc. only) Acquire and apply a license for the Enterprise Edition product, ask in #support -- Perform a downgrade from [EE to CE](https://docs.gitlab.com/ee/downgrade_ee_to_ce/README.html) +- Perform a downgrade from [EE to CE](../../downgrade_ee_to_ce/README.md) #### Start to learn about some of the integrations that we support Our integrations add great value to GitLab. User questions often relate to integrating GitLab with existing external services and the configuration involved - Learn about our Integrations (specially, not only): - - [LDAP](https://docs.gitlab.com/ee/integration/ldap.html) - - [JIRA](https://docs.gitlab.com/ee/project_services/jira.html) - - [Jenkins](https://docs.gitlab.com/ee/integration/jenkins.html) - - [SAML](https://docs.gitlab.com/ce/integration/saml.html) + - [LDAP](../../integration/ldap.md) + - [JIRA](../../project_services/jira.md) + - [Jenkins](../../integration/jenkins.md) + - [SAML](../../integration/saml.md) #### Goals @@ -94,8 +93,8 @@ Our integrations add great value to GitLab. User questions often relate to integ #### Understand the gathering of diagnostics for GitLab instances - Learn about the GitLab checks that are available - - [Environment Information and maintenance checks](https://docs.gitlab.com/ce/raketasks/maintenance.html) - - [GitLab check](https://docs.gitlab.com/ce/raketasks/check.html) + - [Environment Information and maintenance checks](../../raketasks/maintenance.md) + - [GitLab check](../../raketasks/check.md) - Omnibus commands - [Status](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/maintenance/README.md#get-service-status) - [Starting and stopping services](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/maintenance/README.md#starting-and-stopping) @@ -170,11 +169,11 @@ Some tickets need specific knowledge or a deep understanding of a particular com Move on to understanding some of GitLab's more advanced features. You can make use of GitLab.com to understand the features from an end-user perspective and then use your own instance to understand setup and configuration of the feature from an Administrative perspective -- Set up and try [Git LFS](https://docs.gitlab.com/ee/workflow/lfs/manage_large_binaries_with_git_lfs.html) -- Get to know the [GitLab API](https://docs.gitlab.com/ee/api/README.html), its capabilities and shortcomings -- Learn how to [migrate from SVN to Git](https://docs.gitlab.com/ee/workflow/importing/migrating_from_svn.html) -- Set up [GitLab CI](https://docs.gitlab.com/ee/ci/quick_start/README.html) -- Create your first [GitLab Page](https://docs.gitlab.com/ce/administration/pages/) +- Set up and try [Git LFS](../../workflow/lfs/manage_large_binaries_with_git_lfs.md) +- Get to know the [GitLab API](../../api/README.md), its capabilities and shortcomings +- Learn how to [migrate from SVN to Git](../../user/project/import/svn.md) +- Set up [GitLab CI](../../ci/quick_start/README.md) +- Create your first [GitLab Page](../../administration/pages/index.md) - Get to know the GitLab Codebase by reading through the source code: - Find the differences between the [EE codebase](https://gitlab.com/gitlab-org/gitlab-ce) and the [CE codebase](https://gitlab.com/gitlab-org/gitlab-ce) diff --git a/doc/university/training/topics/unstage.md b/doc/university/training/topics/unstage.md index da36a3218e5..c926f0b4888 100644 --- a/doc/university/training/topics/unstage.md +++ b/doc/university/training/topics/unstage.md @@ -8,13 +8,13 @@ comments: false ## Unstage -- To remove files from stage use reset HEAD. Where HEAD is the last commit of the current branch. +- To remove files from stage use reset HEAD where HEAD is the last commit of the current branch. This will unstage the file but maintain the modifications. ```bash git reset HEAD <file> ``` -- This will unstage the file but maintain the modifications. To revert the file back to the state it was in before the changes we can use: +- To revert the file back to the state it was in before the changes we can use: ```bash git checkout -- <file> diff --git a/doc/update/patch_versions.md b/doc/update/patch_versions.md index c2dff21b028..f2df4277ca8 100644 --- a/doc/update/patch_versions.md +++ b/doc/update/patch_versions.md @@ -108,7 +108,7 @@ sudo -u git -H make ### 8. Install/Update `gitlab-elasticsearch-indexer` (optional) **[STARTER ONLY]** -If you're interested in using GitLab's new [elasticsearch repository indexer](https://docs.gitlab.com/ee/integration/elasticsearch.html#elasticsearch-repository-indexer-beta) (currently in beta) +If you're interested in using GitLab's new [elasticsearch repository indexer](../integration/elasticsearch.md#elasticsearch-repository-indexer-beta) (currently in beta) please follow the instructions on the document linked above and enable the indexer usage in the GitLab admin settings. diff --git a/doc/update/upgrading_from_ce_to_ee.md b/doc/update/upgrading_from_ce_to_ee.md index e74a5c00f7e..428377adb19 100644 --- a/doc/update/upgrading_from_ce_to_ee.md +++ b/doc/update/upgrading_from_ce_to_ee.md @@ -75,7 +75,7 @@ sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS ### 4. Install `gitlab-elasticsearch-indexer` (optional) **[STARTER ONLY]** If you're interested in using GitLab's new [elasticsearch repository -indexer][indexer-beta] (currently in beta) please follow the instructions on the +indexer](../integration/elasticsearch.md) (currently in beta) please follow the instructions on the document linked above and enable the indexer usage in the GitLab admin settings. ### 5. Start application @@ -133,4 +133,3 @@ Additional instructions here. [support@gitlab.com]: mailto:support@gitlab.com [old-ee-upgrade-docs]: https://gitlab.com/gitlab-org/gitlab-ee/tree/11-8-stable-ee/doc/update -[indexer-beta]: https://docs.gitlab.com/ee/integration/elasticsearch.html diff --git a/doc/update/upgrading_from_source.md b/doc/update/upgrading_from_source.md index 82a86f3f343..f82d666c7be 100644 --- a/doc/update/upgrading_from_source.md +++ b/doc/update/upgrading_from_source.md @@ -114,7 +114,47 @@ sudo ln -sf /usr/local/go/bin/{go,godoc,gofmt} /usr/local/bin/ rm go1.10.5.linux-amd64.tar.gz ``` -### 6. Get latest code +### 6. Update git + +NOTE: **Note:** +GitLab 11.11 and higher only supports Git 2.21.x and newer, and +[dropped support for older versions](https://gitlab.com/gitlab-org/gitlab-ce/issues/54255). +Be sure to upgrade your installation if necessary. + +```bash +# Make sure Git is version 2.21.0 or higher +git --version + +# Remove packaged Git +sudo apt-get remove git-core + +# Install dependencies +sudo apt-get install -y libcurl4-openssl-dev libexpat1-dev gettext libz-dev libssl-dev build-essential + +# Download and compile pcre2 from source +curl --silent --show-error --location https://ftp.pcre.org/pub/pcre/pcre2-10.33.tar.gz --output pcre2.tar.gz +tar -xzf pcre2.tar.gz +cd pcre2-10.33 +chmod +x configure +./configure --prefix=/usr --enable-jit +make +make install + +# Download and compile from source +cd /tmp +curl --remote-name --location --progress https://www.kernel.org/pub/software/scm/git/git-2.21.0.tar.gz +echo '85eca51c7404da75e353eba587f87fea9481ba41e162206a6f70ad8118147bee git-2.21.0.tar.gz' | shasum -a256 -c - && tar -xzf git-2.21.0.tar.gz +cd git-2.21.0/ +./configure --with-libpcre +make prefix=/usr/local all + +# Install into /usr/local/bin +sudo make prefix=/usr/local install + +# You should edit config/gitlab.yml, change the git -> bin_path to /usr/local/bin/git +``` + +### 7. Get latest code ```bash cd /home/git/gitlab @@ -142,7 +182,7 @@ cd /home/git/gitlab sudo -u git -H git checkout BRANCH-ee ``` -### 7. Update gitlab-shell +### 8. Update gitlab-shell ```bash cd /home/git/gitlab-shell @@ -152,7 +192,7 @@ sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_SHELL_VERSION) sudo -u git -H bin/compile ``` -### 8. Update gitlab-workhorse +### 9. Update gitlab-workhorse Install and compile gitlab-workhorse. GitLab-Workhorse uses [GNU Make](https://www.gnu.org/software/make/). @@ -167,7 +207,7 @@ sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_WORKHORSE_VERSION) sudo -u git -H make ``` -### 9. Update Gitaly +### 10. Update Gitaly #### Compile Gitaly @@ -178,7 +218,7 @@ sudo -u git -H git checkout v$(</home/git/gitlab/GITALY_SERVER_VERSION) sudo -u git -H make ``` -### 10. Update gitlab-pages +### 11. Update gitlab-pages #### Only needed if you use GitLab Pages @@ -195,7 +235,7 @@ sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_PAGES_VERSION) sudo -u git -H make ``` -### 11. Update MySQL permissions +### 12. Update MySQL permissions If you are using MySQL you need to grant the GitLab user the necessary permissions on the database: @@ -217,7 +257,7 @@ You can make this setting permanent by adding it to your `my.cnf`: log_bin_trust_function_creators=1 ``` -### 12. Update configuration files +### 13. Update configuration files #### New configuration options for `gitlab.yml` @@ -291,7 +331,7 @@ For Ubuntu 16.04.1 LTS: sudo systemctl daemon-reload ``` -### 13. Install libs, migrations, etc. +### 14. Install libs, migrations, etc. ```bash cd /home/git/gitlab @@ -323,14 +363,14 @@ sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production **MySQL installations**: Run through the `MySQL strings limits` and `Tables and data conversion to utf8mb4` [tasks](../install/database_mysql.md). -### 14. Start application +### 15. Start application ```bash sudo service gitlab start sudo service nginx restart ``` -### 15. Check application status +### 16. Check application status Check if GitLab and its environment are configured correctly: diff --git a/doc/user/admin_area/diff_limits.md b/doc/user/admin_area/diff_limits.md index 9205860ef1f..4063c40a751 100644 --- a/doc/user/admin_area/diff_limits.md +++ b/doc/user/admin_area/diff_limits.md @@ -1,21 +1,40 @@ +--- +type: reference +--- + # Diff limits administration +You can set a maximum size for display of diff files (patches). + +## Maximum diff patch size + +Diff files which exceed this value will be presented as 'too large' and won't +be expandable. Instead of an expandable view, a link to the blob view will be +shown. + +Patches greater than 10% of this size will be automatically collapsed, and a +link to expand the diff will be presented. + NOTE: **Note:** Merge requests and branch comparison views will be affected. CAUTION: **Caution:** -These settings are currently under experimental state. They'll -increase the resource consumption of your instance and should -be edited mindfully. +This setting is experimental. An increased maximum will increase resource +consumption of your instance. Keep this in mind when adjusting the maximum. -1. Access **Admin area > Settings > General** -1. Expand **Diff limits** +1. Go to **Admin area > Settings > General**. +1. Expand **Diff limits**. +1. Enter a value for **Maximum diff patch size**, measured in bytes. +1. Click on **Save changes**. -### Maximum diff patch size +<!-- ## Troubleshooting -This is the content size each diff file (patch) is allowed to reach before -it's collapsed, without the possibility of being expanded. A link redirecting -to the blob view will be presented for the patches that surpass this limit. +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. -Patches surpassing 10% of this content size will be automatically collapsed, -but expandable (a link to expand the diff will be presented). +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/admin_area/geo_nodes.md b/doc/user/admin_area/geo_nodes.md index 776ab139c64..d99b87cbc5c 100644 --- a/doc/user/admin_area/geo_nodes.md +++ b/doc/user/admin_area/geo_nodes.md @@ -1,9 +1,13 @@ +--- +type: howto +--- + # Geo nodes admin area **[PREMIUM ONLY]** -For more information about setting up GitLab Geo, read the -[Geo documentation](https://docs.gitlab.com/ee/administration/geo/replication/index.html). +You can configure various settings for GitLab Geo nodes. For more information, see +[Geo documentation](../../administration/geo/replication/index.md). -When you're done, you can navigate to **Admin area > Geo** (`/admin/geo/nodes`). +On the primary node, go to **Admin area > Geo**. On secondary nodes, go to **Admin area > Geo > Nodes**. ## Common settings @@ -25,7 +29,7 @@ changes on the **primary** node! | Setting | Description | |---------------------------|-------------| -| Selective synchronization | Enable Geo [selective sync](https://docs.gitlab.com/ee/administration/geo/replication/configuration.html#selective-synchronization) for this **secondary** node. | +| Selective synchronization | Enable Geo [selective sync](../../administration/geo/replication/configuration.md#selective-synchronization) for this **secondary** node. | | Repository sync capacity | Number of concurrent requests this **secondary** node will make to the **primary** node when backfilling repositories. | | File sync capacity | Number of concurrent requests this **secondary** node will make to the **primary** node when backfilling files. | @@ -68,3 +72,15 @@ a unique `name` is set for each Geo node. The `gitlab.rb` setting The load balancer must use sticky sessions in order to avoid authentication failures and cross site request errors. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. -->
\ No newline at end of file diff --git a/doc/user/admin_area/img/index_runners_search_or_filter.png b/doc/user/admin_area/img/index_runners_search_or_filter.png Binary files differnew file mode 100644 index 00000000000..5176a1a39bf --- /dev/null +++ b/doc/user/admin_area/img/index_runners_search_or_filter.png diff --git a/doc/user/admin_area/index.md b/doc/user/admin_area/index.md index dd4e96c1f4a..527110d53df 100644 --- a/doc/user/admin_area/index.md +++ b/doc/user/admin_area/index.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # GitLab Admin Area **[CORE ONLY]** The Admin Area provides a web UI for administering some features of GitLab self-managed instances. @@ -16,14 +20,14 @@ The Admin Area is made up of the following sections: | Section | Description | |:---------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------| -| Overview | View your GitLab [Dashboard](#admin-dashboard), and administer [projects](#administer-projects), users, groups, jobs, runners, and Gitaly servers. | +| Overview | View your GitLab [Dashboard](#admin-dashboard), and administer [projects](#administer-projects), [users](#administer-users), groups, [jobs](#administer-jobs), [Runners](#administer-runners), and [Gitaly servers](#administer-gitaly-servers). | | Monitoring | View GitLab system information, and information on background jobs, logs, [health checks](monitoring/health_check.md), request profiles, and audit logs. | | Messages | Send and manage [broadcast messages](broadcast_messages.md) for your users. | | System Hooks | Configure [system hooks](../../system_hooks/system_hooks.md) for many events. | | Applications | Create system [OAuth applications](../../integration/oauth_provider.md) for integrations with other services. | | Abuse Reports | Manage [abuse reports](abuse_reports.md) submitted by your users. | | License **[STARTER ONLY]** | Upload, display, and remove [licenses](license.md). | -| Push Rules **[STARTER]** | Configure pre-defined git [push rules](https://docs.gitlab.com/ee/push_rules/push_rules.html) for projects. | +| Push Rules **[STARTER]** | Configure pre-defined git [push rules](../../push_rules/push_rules.md) for projects. | | Geo **[PREMIUM ONLY]** | Configure and maintain [Geo nodes](geo_nodes.md). | | Deploy Keys | Create instance-wide [SSH deploy keys](../../ssh/README.md#deploy-keys). | | Service Templates | Create [service templates](../project/integrations/services_templates.md) for projects. | @@ -61,16 +65,144 @@ Click the **All**, **Private**, **Internal**, or **Public** tab to list only pro criteria. By default, all projects are listed, in reverse order of when they were last updated. For each -project, the name, namespace, description, and size is listed, also options to **Edit** or -**Delete** it. +project, the following information is listed: + +- Name. +- Namespace. +- Description. +- Size, updated every 15 minutes at most. + +Projects can be edited or deleted. -Sort projects by **Name**, **Last created**, **Oldest created**, **Last updated**, **Oldest -updated**, **Owner**, and choose to hide or show archived projects. +The list of projects can be sorted by: + +- Name. +- Last created. +- Oldest created. +- Last updated. +- Oldest updated. +- Owner. + +A user can choose to hide or show archived projects in the list. In the **Filter by name** field, type the project name you want to find, and GitLab will filter them as you type. Select from the **Namespace** dropdown to filter only projects in that namespace. -You can combine the filter options. For example, click the **Public** tab, and enter `score` in -the **Filter by name...** input box to list only public projects with `score` in their name. +You can combine the filter options. For example, to list only public projects with `score` in their name: + +1. Click the **Public** tab. +1. Enter `score` in the **Filter by name...** input box. + +## Administer Users + +You can administer all users in the GitLab instance from the Admin Area's Users page. + +To access the Users page, go to **Admin Area > Overview > Users**. + +Click the **Active**, **Admins**, **2FA Enabled**, or **2FA Disabled**, **External**, or +**Without projects** tab to list only users of that criteria. + +For each user, their username, email address, are listed, also the date their account was +created and the date of last activity. To edit a user, click the **Edit** button in that user's +row. To delete the user, or delete the user and their contributions, click the cog dropdown in +that user's row, and select the desired option. + +To change the sort order: + +1. Click the sort dropdown. +1. Select the desired order. + +By default the sort dropdown shows **Name**. + +To search for users, enter your criteria in the search field. The user search is case +insensitive, and applies partial matching to name and username. To search for an email address, +you must provide the complete email address. + +## Administer Jobs + +You can administer all jobs in the GitLab instance from the Admin Area's Jobs page. + +To access the Jobs page, go to **Admin Area > Overview > Jobs**. + +All jobs are listed, in reverse order of their job ID. + +Click the **All** tab to list all jobs. Click the **Pending**, **Running**, or **Finished** tab to list only jobs of that status. + +For each job, the following details are listed: + +| Field | Description | +|--------- | ----------- | +| Status | Job status, either **passed**, **skipped**, or **failed**. | +| Job | Includes links to the job, branch, and the commit that started the job. | +| Pipeline | Includes a link to the specific pipeline. | +| Project | Name of the project, and organization, to which the job belongs. | +| Runner | Name of the CI runner assigned to execute the job. | +| Stage | Stage that the job is declared in a `.gitlab-ci.yml` file. | +| Name | Name of the job specified in a `.gitlab-ci.yml` file. | +| Timing | Duration of the job, and how long ago the job completed. | +| Coverage | Percentage of tests coverage. | + +## Administer Runners + +You can adminster all Runners in the GitLab instance from the Admin Area's **Runners** page. See +[GitLab Runner](https://docs.gitlab.com/runner/) for more information on Runner itself. + +To access the **Runners** page, go to **Admin Area > Overview > Runners**. + +The **Runners** page features: + +- A description of Runners, and their possible states. +- Instructions on installing a Runner. +- A list of all registered Runners. + +Runners are listed in descending order by the date they were created, by default. You can change +the sort order to *Last Contacted* from the dropdown beside the search field. + +To search Runners' descriptions: + +1. In the **Search or filter results...** field, type the description of the Runner you want to +find. +1. Press Enter. + +You can also filter Runners by status, type, and tag. To filter: + +1. Click in the **Search or filter results...** field. +1. Select **status:**, **type:**, or **tag:** +1. Select or enter your search criteria. + + + +For each Runner, the following attributes are listed: + +| Attribute | Description | +| ------------ | ----------- | +| Type | One or more of the following states: shared, group, specific, locked, or paused | +| Runner token | Token used to identify the Runner, and which the Runner uses to communicate with the GitLab instance | +| Description | Description given to the Runner when it was created | +| Version | GitLab Runner version | +| IP address | IP address of the host on which the Runner is registered | +| Projects | Projects to which the Runner is assigned | +| Jobs | Total of jobs run by the Runner | +| Tags | Tags associated with the Runner | +| Last contact | Timestamp indicating when the GitLab instance last contacted the Runner | + +You can also edit, pause, or remove each Runner. + +## Administer Gitaly servers + +You can list all Gitaly servers in the GitLab instance from the Admin Area's **Gitaly Servers** +page. For more details, see [Gitaly](../../administration/gitaly/index.md). + +To access the **Gitaly Servers** page, go to **Admin Area > Overview > Gitaly Servers**. + +For each Gitaly server, the following details are listed: + +| Field | Description | +| -------------- | ----------- | +| Storage | Repository storage | +| Address | Network address on which the Gitaly server is listening | +| Server version | Gitaly version | +| Git version | Version of Git installed on the Gitaly server | +| Up to date | Indicates if the Gitaly server version is the latest version available. A green dot indicates the server is up to date. | diff --git a/doc/user/admin_area/labels.md b/doc/user/admin_area/labels.md index e383142c33e..eba27548f86 100644 --- a/doc/user/admin_area/labels.md +++ b/doc/user/admin_area/labels.md @@ -1,9 +1,25 @@ +--- +type: reference +--- + # Labels administration **[CORE ONLY]** -## Default Labels +In the Admin Area, you can manage labels for the GitLab instance. For more details, see [Labels](../project/labels.md). -### Define your own default Label Set +## Default Labels -Labels that are created within the Labels view on the Admin Dashboard will be automatically added to each new project. +Labels created in the Admin Area become available to each _new_ project.  + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/admin_area/license.md b/doc/user/admin_area/license.md index 49959a9daef..1e8ce04da92 100644 --- a/doc/user/admin_area/license.md +++ b/doc/user/admin_area/license.md @@ -1,3 +1,7 @@ +--- +type: howto +--- + # Activate all GitLab Enterprise Edition functionality with a license **[STARTER ONLY]** To activate all GitLab Enterprise Edition (EE) functionality, you need to upload @@ -108,3 +112,15 @@ but only the latest license will be used as the active license. [free trial]: https://about.gitlab.com/free-trial/ [pricing]: https://about.gitlab.com/pricing/ + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. -->
\ No newline at end of file diff --git a/doc/user/admin_area/monitoring/health_check.md b/doc/user/admin_area/monitoring/health_check.md index e183898dfb1..ff056490653 100644 --- a/doc/user/admin_area/monitoring/health_check.md +++ b/doc/user/admin_area/monitoring/health_check.md @@ -1,12 +1,14 @@ -# Health Check +--- +type: concepts, howto +--- -> **Notes:** +# Health Check -> - Liveness and readiness probes were [introduced][ce-10416] in GitLab 9.1. -> - The `health_check` endpoint was [introduced][ce-3888] in GitLab 8.8 and was -> be deprecated in GitLab 9.1. -> - [Access token](#access-token-deprecated) has been deprecated in GitLab 9.4 -> in favor of [IP whitelist](#ip-whitelist) +> - Liveness and readiness probes were [introduced][ce-10416] in GitLab 9.1. +> - The `health_check` endpoint was [introduced][ce-3888] in GitLab 8.8 and was +> be deprecated in GitLab 9.1. +> - [Access token](#access-token-deprecated) has been deprecated in GitLab 9.4 +> in favor of [IP whitelist](#ip-whitelist). GitLab provides liveness and readiness probes to indicate service health and reachability to required services. These probes report on the status of the @@ -17,8 +19,7 @@ traffic until the system is ready or restart the container as needed. ## IP whitelist To access monitoring resources, the requesting client IP needs to be included in a whitelist. - -[Read how to add IPs to a whitelist for the monitoring endpoints][admin]. +For details, see [how to add IPs to a whitelist for the monitoring endpoints](../../../administration/monitoring/ip_whitelist.md). ## Using the endpoints @@ -30,7 +31,7 @@ With default whitelist settings, the probes can be accessed from localhost using The first endpoint, `health`, only checks whether the application server is running. It does not verify the database or other services are running. A successful response will return a 200 status code with the following message: -``` +```text GitLab OK ``` @@ -87,9 +88,8 @@ will return a valid successful HTTP status code, and a `success` message. ## Access token (Deprecated) ->**Note:** -Access token has been deprecated in GitLab 9.4 -in favor of [IP whitelist](#ip-whitelist) +> NOTE: **Note:** +> Access token has been deprecated in GitLab 9.4 in favor of [IP whitelist](#ip-whitelist). An access token needs to be provided while accessing the probe endpoints. The current accepted token can be found under the **Admin area ➔ Monitoring ➔ Health check** @@ -99,14 +99,25 @@ accepted token can be found under the **Admin area ➔ Monitoring ➔ Health che The access token can be passed as a URL parameter: -``` +```text https://gitlab.example.com/-/readiness?token=ACCESS_TOKEN ``` +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> + [ce-10416]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10416 [ce-3888]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3888 [pingdom]: https://www.pingdom.com [nagios-health]: https://nagios-plugins.org/doc/man/check_http.html [newrelic-health]: https://docs.newrelic.com/docs/alerts/alert-policies/downtime-alerts/availability-monitoring [kubernetes]: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/ -[admin]: ../../../administration/monitoring/ip_whitelist.md diff --git a/doc/user/admin_area/settings/account_and_limit_settings.md b/doc/user/admin_area/settings/account_and_limit_settings.md index 4a5bfb6b677..001e4b6bf48 100644 --- a/doc/user/admin_area/settings/account_and_limit_settings.md +++ b/doc/user/admin_area/settings/account_and_limit_settings.md @@ -1,25 +1,29 @@ +--- +type: reference +--- + # Account and limit settings ## Repository size limit **[STARTER]** -> [Introduced][ee-740] in [GitLab Enterprise Edition 8.12][ee-8.12]. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/740) in [GitLab Enterprise Edition 8.12](https://about.gitlab.com/2016/09/22/gitlab-8-12-released/#limit-project-size-ee). +> Available in [GitLab Starter](https://about.gitlab.com/pricing/). Repositories within your GitLab instance can grow quickly, especially if you are -using LFS. Their size can grow exponentially and eat up your storage device quite -fast. +using LFS. Their size can grow exponentially, rapidly consuming available storage. -In order to avoid this from happening, you can set a hard limit for your -repositories' size. This limit can be set globally, per group, or per project, -with per project limits taking the highest priority. +To avoid this from happening, you can set a hard limit for your repositories' size. +This limit can be set globally, per group, or per project, with per project limits +taking the highest priority. -There are numerous cases where you'll need to set up a limit for repository size. +There are numerous use cases where you might set up a limit for repository size. For instance, consider the following workflow: -1. Your team develops apps which demand large files to be stored in +1. Your team develops apps which require large files to be stored in the application repository. -1. Although you have enabled [Git LFS](../../../workflow/lfs/manage_large_binaries_with_git_lfs.html#git-lfs) +1. Although you have enabled [Git LFS](../../../workflow/lfs/manage_large_binaries_with_git_lfs.md#git-lfs) to your project, your storage has grown significantly. -1. Before you blow your storage limit up, you set up a limit of 10 GB +1. Before you exceed available storage, you set up a limit of 10 GB per repository. ### How it works @@ -36,18 +40,23 @@ These settings can be found within: - **Admin Area > Settings > General**. - The path `/admin/application_settings`. -The very first push of a new project cannot be checked for size as of now, so -the first push will allow you to upload more than the limit dictates, but every -subsequent push will be denied. LFS objects, however, can be checked on first -push and **will** be rejected if the sum of their sizes exceeds the maximum -allowed repository size. +The first push of a new project, including LFS objects, will be checked for size +and **will** be rejected if the sum of their sizes exceeds the maximum allowed +repository size. + +For details on manually purging files, see [reducing the repository size using Git](../../project/repository/reducing_the_repo_size_using_git.md). + +NOTE: **Note:** +For GitLab.com, the repository size limit is 10 GB. -For more manually purging the files, read the docs on -[reducing the repository size using Git][repo-size]. +<!-- ## Troubleshooting -> **Note:** -> For GitLab.com, the repository size limit is 10 GB. +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. -[ee-740]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/740 -[repo-size]: ../../project/repository/reducing_the_repo_size_using_git.md -[ee-8.12]: https://about.gitlab.com/2016/09/22/gitlab-8-12-released/#limit-project-size-ee +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/admin_area/settings/continuous_integration.md b/doc/user/admin_area/settings/continuous_integration.md index 9dd476656ed..6c4abce83c2 100644 --- a/doc/user/admin_area/settings/continuous_integration.md +++ b/doc/user/admin_area/settings/continuous_integration.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Continuous Integration and Deployment Admin settings **[CORE ONLY]** In this area, you will find settings for Auto DevOps, Runners and job artifacts. @@ -145,3 +149,15 @@ To set the duration for which the jobs will be considered as old and expired: Once that time passes, the jobs will be archived and no longer able to be retried. Make it empty to never expire jobs. It has to be no less than 1 day, for example: <code>15 days</code>, <code>1 month</code>, <code>2 years</code>. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. -->
\ No newline at end of file diff --git a/doc/user/admin_area/settings/email.md b/doc/user/admin_area/settings/email.md index 01a98cf15dc..9555a695b13 100644 --- a/doc/user/admin_area/settings/email.md +++ b/doc/user/admin_area/settings/email.md @@ -1,12 +1,18 @@ +--- +type: reference +--- + # Email +You can customize some of the content in emails sent from your GitLab instance. + ## Custom logo The logo in the header of some emails can be customized, see the [logo customization section](../../../customization/branded_page_and_email_header.md). ## Custom additional text **[PREMIUM ONLY]** ->[Introduced][ee-5031] in [GitLab Premium][eep] 10.7. +> [Introduced][ee-5031] in [GitLab Premium][eep] 10.7. The additional text will appear at the bottom of any email and can be used for legal/auditing/compliance reasons. @@ -24,8 +30,8 @@ legal/auditing/compliance reasons. > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/22560) in GitLab 11.5. -This configuration option sets the email hostname for [private commit emails](../../profile/index.md#private-commit-email), -and it's, by default, set to `users.noreply.YOUR_CONFIGURED_HOSTNAME`. +This configuration option sets the email hostname for [private commit emails](../../profile/index.md#private-commit-email). + By default it is set to `users.noreply.YOUR_CONFIGURED_HOSTNAME`. In order to change this option: @@ -34,5 +40,17 @@ In order to change this option: 1. Hit **Save** for the changes to take effect. NOTE: **Note**: Once the hostname gets configured, every private commit email using the previous hostname, will not get -recognized by GitLab. This can directly conflict with certain [Push rules](https://docs.gitlab.com/ee/push_rules/push_rules.html) such as +recognized by GitLab. This can directly conflict with certain [Push rules](../../../push_rules/push_rules.md) such as `Check whether author is a GitLab user` and `Check whether committer is the current authenticated user`. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. -->
\ No newline at end of file diff --git a/doc/user/admin_area/settings/external_authorization.md b/doc/user/admin_area/settings/external_authorization.md index 06e00e02f3d..11c0867da17 100644 --- a/doc/user/admin_area/settings/external_authorization.md +++ b/doc/user/admin_area/settings/external_authorization.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # External authorization control **[CORE ONLY]** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/4216) in @@ -108,5 +112,17 @@ The label will be shown on all project pages in the upper right corner.  +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> + [omnibus-ssl-docs]: https://docs.gitlab.com/omnibus/settings/ssl.html [omnibus-log-docs]: https://docs.gitlab.com/omnibus/settings/logs.html diff --git a/doc/user/admin_area/settings/instance_template_repository.md b/doc/user/admin_area/settings/instance_template_repository.md index 4010008f694..91286a67c31 100644 --- a/doc/user/admin_area/settings/instance_template_repository.md +++ b/doc/user/admin_area/settings/instance_template_repository.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Instance template repository **[PREMIUM ONLY]** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/5986) in @@ -61,3 +65,15 @@ top of the list. If this feature is disabled or no templates are present, there will be no "Custom" section in the selection dropdown. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/admin_area/settings/sign_up_restrictions.md b/doc/user/admin_area/settings/sign_up_restrictions.md index d3ecfd42903..cebf36c7ec1 100644 --- a/doc/user/admin_area/settings/sign_up_restrictions.md +++ b/doc/user/admin_area/settings/sign_up_restrictions.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Sign-up restrictions You can block email addresses of specific domains, or whitelist only some @@ -37,5 +41,17 @@ semicolon, comma, or a new line.  +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> + [ce-5259]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5259 [ce-598]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/598 diff --git a/doc/user/admin_area/settings/terms.md b/doc/user/admin_area/settings/terms.md index e2290bf0598..a5f8d05f662 100644 --- a/doc/user/admin_area/settings/terms.md +++ b/doc/user/admin_area/settings/terms.md @@ -1,29 +1,35 @@ +--- +type: reference +--- + # Enforce accepting Terms of Service > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/18570) > in [GitLab Core](https://about.gitlab.com/pricing/) 10.8 +An admin can enforce acceptance of a terms of service and privacy policy. When this option is enabled, new and existing users must accept the terms. + ## Configuration -When it is required for all users of the GitLab instance to accept the -Terms of Service, this can be configured by an admin on the settings -page: +To enforce acceptance of a Terms of Service and Privacy Policy: -. +1. Log in to the GitLab instance as an admin user. +1. Go to **Admin Area > Settings > General**. +1. Expand the **Terms of Service and Privacy Policy** section. +1. Check the **Require all users to accept Terms of Service and Privacy Policy when they access +GitLab.** checkbox. +1. Input the text of the **Terms of Service and Privacy Policy**. Markdown formatting can be used in this input box. +1. Click **Save changes**. +1. When you are presented with the **Terms of Service** statement, click **Accept terms**. -The terms itself can be entered using Markdown. For each update to the -terms, a new version is stored. When a user accepts or declines the -terms, GitLab will keep track of which version they accepted or -declined. +. -When an admin enables this feature, they will automattically be -directed to the page to accept the terms themselves. After they -accept, they will be directed back to the settings page. +For each update to the terms, a new version is stored. When a user accepts or declines the terms, +GitLab will record which version they accepted or declined. -## New registrations +## New users -When this feature is enabled, a checkbox will be available in the -sign-up form. +When this feature is enabled, a checkbox is added to the sign-up form.  @@ -49,3 +55,15 @@ If the user was already logged in when the feature was turned on, they will be asked to accept the terms on their next interaction. If a user declines the terms, they will be signed out. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/admin_area/settings/third_party_offers.md b/doc/user/admin_area/settings/third_party_offers.md index 23311801790..d3c9cf7d8ff 100644 --- a/doc/user/admin_area/settings/third_party_offers.md +++ b/doc/user/admin_area/settings/third_party_offers.md @@ -1,9 +1,26 @@ +--- +type: reference +--- + # Third party offers > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/20379) > in [GitLab Core](https://about.gitlab.com/pricing/) 11.1 -Within GitLab, we inform users of available third-party offers they might find valuable in order to enhance the development of their projects. -An example is the Google Cloud Platform free credit for using [Google Kubernetes Engine](https://cloud.google.com/kubernetes-engine/). +Within GitLab, we inform users of available third-party offers they might find valuable in order +to enhance the development of their projects. An example is the Google Cloud Platform free credit +for using [Google Kubernetes Engine](https://cloud.google.com/kubernetes-engine/). + +The display of third-party offers can be toggled in the **Admin Area > Settings** page. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. -The display of third-party offers can be toggled in the Admin area on the Settings page. +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/admin_area/settings/usage_statistics.md b/doc/user/admin_area/settings/usage_statistics.md index 8b5d80efb0d..652d6ad2cdd 100644 --- a/doc/user/admin_area/settings/usage_statistics.md +++ b/doc/user/admin_area/settings/usage_statistics.md @@ -1,10 +1,14 @@ +--- +type: reference +--- + # Usage statistics GitLab Inc. will periodically collect information about your instance in order to perform various actions. All statistics are opt-out, you can enable/disable them from the admin panel -under **Admin area > Settings > Usage statistics**. +under **Admin area > Settings > Metrics and profiling > Usage statistics**. ## Version check **[CORE ONLY]** @@ -83,6 +87,18 @@ of your instance to your users. This can be restricted to admins by selecting "Only admins" in the Instance Statistics visibility section under **Admin area > Settings > Usage statistics**. +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> + [ee-557]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/557 [ee-735]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/735 [ce-23361]: https://gitlab.com/gitlab-org/gitlab-ce/issues/23361 diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md index 5c635b09503..adb6868516e 100644 --- a/doc/user/application_security/container_scanning/index.md +++ b/doc/user/application_security/container_scanning/index.md @@ -119,7 +119,7 @@ container_scanning: variables: DOCKER_DRIVER: overlay2 ## Define two new variables based on GitLab's CI/CD predefined variables - ## https://docs.gitlab.com/ee/ci/variables/#predefined-environment-variables + ## https://docs.gitlab.com/ee/ci/variables/README.html#predefined-environment-variables CI_APPLICATION_REPOSITORY: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG CI_APPLICATION_TAG: $CI_COMMIT_SHA CLAIR_LOCAL_SCAN_VERSION: v2.0.8_fe9b059d930314b54c78f75afe265955faf4fdc1 @@ -162,7 +162,7 @@ container_scanning: variables: DOCKER_DRIVER: overlay2 ## Define two new variables based on GitLab's CI/CD predefined variables - ## https://docs.gitlab.com/ee/ci/variables/#predefined-environment-variables + ## https://docs.gitlab.com/ee/ci/variables/README.html#predefined-environment-variables CI_APPLICATION_REPOSITORY: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG CI_APPLICATION_TAG: $CI_COMMIT_SHA CLAIR_LOCAL_SCAN_VERSION: v2.0.8_fe9b059d930314b54c78f75afe265955faf4fdc1 diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md index f3b7d7fd471..028ff72a160 100644 --- a/doc/user/application_security/dast/index.md +++ b/doc/user/application_security/dast/index.md @@ -39,6 +39,8 @@ However, DAST can be [configured](#full-scan) to also perform a so-called "active scan". That is, attack your application and produce a more extensive security report. It can be very useful combined with [Review Apps](../../../ci/review_apps/index.md). +The [`dast`](https://gitlab.com/gitlab-org/security-products/dast/container_registry) Docker image in GitLab container registry is updated on a weekly basis to have all [`owasp2docker-weekly`](https://hub.docker.com/r/owasp/zap2docker-weekly/) updates in it. + ## Use cases It helps you automatically find security vulnerabilities in your running web @@ -47,10 +49,7 @@ applications while you are developing and testing your applications. ## Requirements To run a DAST job, you need GitLab Runner with the -[`docker`](https://docs.gitlab.com/runner/executors/docker.html#use-docker-in-docker-with-privileged-mode) or -[`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html#running-privileged-containers-for-the-runners) -executor running in privileged mode. If you're using the shared Runners on GitLab.com, -this is enabled by default. +[`docker` executor](https://docs.gitlab.com/runner/executors/docker.html). ## Configuring DAST diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md index 2d0c2be4233..d78cf778110 100644 --- a/doc/user/application_security/dependency_scanning/index.md +++ b/doc/user/application_security/dependency_scanning/index.md @@ -19,8 +19,9 @@ merge request.  -The results are sorted by the priority of the vulnerability: +The results are sorted by the severity of the vulnerability: +1. Critical 1. High 1. Medium 1. Low @@ -223,3 +224,20 @@ vulnerabilities in your groups and projects. Read more about the Once a vulnerability is found, you can interact with it. Read more on how to [interact with the vulnerabilities](../index.md#interacting-with-the-vulnerabilities). + +## Dependency List + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/10075) +in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.0. + +An additional benefit of Dependency Scanning is the ability to get a list of your project's dependencies with their versions. + +This list can be generated only for [supported languages and package managers](#supported-languages-and-package-managers). + +To see the generated dependency list, navigate to the Dependency List page under your project's left sidebar menu **Project > Dependency List**. + +## Contributing to the vulnerability database + +You can search the [gemnasium-db](https://gitlab.com/gitlab-org/security-products/gemnasium-db) project +to find a vulnerability in the Gemnasium database. +You can also [submit new vulnerabilities](https://gitlab.com/gitlab-org/security-products/gemnasium-db/blob/master/CONTRIBUTING.md).
\ No newline at end of file diff --git a/doc/user/application_security/img/dismissed_info.png b/doc/user/application_security/img/dismissed_info.png Binary files differnew file mode 100644 index 00000000000..64d5cf26ed2 --- /dev/null +++ b/doc/user/application_security/img/dismissed_info.png diff --git a/doc/user/application_security/img/interactive_reports.png b/doc/user/application_security/img/interactive_reports.png Binary files differindex 9f9812dc69d..373b39104db 100644 --- a/doc/user/application_security/img/interactive_reports.png +++ b/doc/user/application_security/img/interactive_reports.png diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md index 64e72fab198..679847b76d7 100644 --- a/doc/user/application_security/index.md +++ b/doc/user/application_security/index.md @@ -47,6 +47,16 @@ You can dismiss vulnerabilities by clicking the **Dismiss vulnerability** button This will dismiss the vulnerability and re-render it to reflect its dismissed state. If you wish to undo this dismissal, you can click the **Undo dismiss** button. +#### Adding a dismissal reason + +> Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing) 12.0. + +When dismissing a vulnerability, it's often helpful to provide a reason for doing so. +If you press the comment button next to **Dismiss vulnerability** in the modal, a text box will appear, allowing you to add a comment with your dismissal. +This comment can not currently be edited or removed, but [future versions](https://gitlab.com/gitlab-org/gitlab-ee/issues/11721) will add this functionality. + + + ### Creating an issue for a vulnerability You can create an issue for a vulnerability by selecting the **Create issue** diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index 02c115b7f22..db328262aba 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -63,7 +63,7 @@ The following table shows which languages, package managers and frameworks are s | Javascript | [ESLint security plugin](https://github.com/nodesecurity/eslint-plugin-security) | 11.8 | | Node.js | [NodeJsScan](https://github.com/ajinabraham/NodeJsScan) | 11.1 | | PHP | [phpcs-security-audit](https://github.com/FloeDesignTechnologies/phpcs-security-audit) | 10.8 | -| Python ([pip](https://pip.pypa.io/en/stable/)) | [bandit](https://github.com/openstack/bandit) | 10.3 | +| Python ([pip](https://pip.pypa.io/en/stable/)) | [bandit](https://github.com/PyCQA/bandit) | 10.3 | | Ruby on Rails | [brakeman](https://brakemanscanner.org) | 10.3 | | Scala ([Ant](https://ant.apache.org/), [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) and [SBT](https://www.scala-sbt.org/)) | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 11.0 (SBT) & 11.9 (Ant, Gradle, Maven) | | Typescript | [TSLint config security](https://github.com/webschik/tslint-config-security/) | 11.9 | diff --git a/doc/user/application_security/security_dashboard/img/dashboard.png b/doc/user/application_security/security_dashboard/img/dashboard.png Binary files differindex d52a6dacdbf..a75168b1ce4 100644 --- a/doc/user/application_security/security_dashboard/img/dashboard.png +++ b/doc/user/application_security/security_dashboard/img/dashboard.png diff --git a/doc/user/application_security/security_dashboard/img/project_security_dashboard.png b/doc/user/application_security/security_dashboard/img/project_security_dashboard.png Binary files differindex 3294e59e943..f0dad6c54d0 100644 --- a/doc/user/application_security/security_dashboard/img/project_security_dashboard.png +++ b/doc/user/application_security/security_dashboard/img/project_security_dashboard.png diff --git a/doc/user/application_security/security_dashboard/index.md b/doc/user/application_security/security_dashboard/index.md index ca31e15c65f..19eeb06a259 100644 --- a/doc/user/application_security/security_dashboard/index.md +++ b/doc/user/application_security/security_dashboard/index.md @@ -54,6 +54,7 @@ First, navigate to the Security Dashboard found under your group's Once you're on the dashboard, at the top you should see a series of filters for: - Severity +- Confidence - Report type - Project diff --git a/doc/user/clusters/applications.md b/doc/user/clusters/applications.md new file mode 100644 index 00000000000..b520c4fb579 --- /dev/null +++ b/doc/user/clusters/applications.md @@ -0,0 +1,290 @@ +# GitLab Managed Apps + +GitLab provides **GitLab Managed Apps**, a one-click install for various applications which can +be added directly to your configured cluster. These applications are +needed for [Review Apps](../../ci/review_apps/index.md) and +[deployments](../../ci/environments.md) when using [Auto DevOps](../../topics/autodevops/index.md). +You can install them after you +[create a cluster](../project/clusters/index.md#adding-and-creating-a-new-gke-cluster-via-gitlab). + +## Installing applications + +Applications managed by GitLab will be installed onto the `gitlab-managed-apps` namespace. +This namespace: + +- Is different from the namespace used for project deployments. +- Is created once. +- Has a non-configurable name. + +To see a list of available applications to install: + +1. For a: + - Project-level cluster, navigate to your project's **Operations > Kubernetes**. + - Group-level cluster, navigate to your group's **Kubernetes** page. + +Install Helm first as it's used to install other applications. + +NOTE: **Note:** +As of GitLab 11.6, Helm will be upgraded to the latest version supported +by GitLab before installing any of the applications. + +The following applications can be installed: + +- [Helm](#helm) +- [Ingress](#ingress) +- [Cert-Manager](#cert-manager) +- [Prometheus](#prometheus) +- [GitLab Runner](#gitlab-runner) +- [JupyterHub](#jupyterhub) +- [Knative](#knative) + +With the exception of Knative, the applications will be installed in a dedicated +namespace called `gitlab-managed-apps`. + +NOTE: **Note:** +Some applications are installable only for a project-level cluster. +Support for installing these applications in a group-level cluster is +planned for future releases. +For updates, see [the issue tracking +progress](https://gitlab.com/gitlab-org/gitlab-ce/issues/51989). + +CAUTION: **Caution:** +If you have an existing Kubernetes cluster with Helm already installed, +you should be careful as GitLab cannot detect it. In this case, installing +Helm via the applications will result in the cluster having it twice, which +can lead to confusion during deployments. + +### Helm + +> - Available for project-level clusters since GitLab 10.2. +> - Available for group-level clusters since GitLab 11.6. + +[Helm](https://docs.helm.sh/) is a package manager for Kubernetes and is +required to install all the other applications. It is installed in its +own pod inside the cluster which can run the `helm` CLI in a safe +environment. + +### Cert-Manager + +> - Available for project-level clusters since GitLab 11.6. +> - Available for group-level clusters since GitLab 11.6. + +[Cert-Manager](https://docs.cert-manager.io/en/latest/) is a native +Kubernetes certificate management controller that helps with issuing +certificates. Installing Cert-Manager on your cluster will issue a +certificate by [Let's Encrypt](https://letsencrypt.org/) and ensure that +certificates are valid and up-to-date. + +NOTE: **Note:** +The +[stable/cert-manager](https://github.com/helm/charts/tree/master/stable/cert-manager) +chart is used to install this application with a +[`values.yaml`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/vendor/cert_manager/values.yaml) +file. + +### GitLab Runner + +> - Available for project-level clusters since GitLab 10.6. +> - Available for group-level clusters since GitLab 11.10. + +[GitLab Runner](https://docs.gitlab.com/runner/) is the open source +project that is used to run your jobs and send the results back to +GitLab. It is used in conjunction with [GitLab +CI/CD](../../ci/README.md), the open-source continuous integration +service included with GitLab that coordinates the jobs. When installing +the GitLab Runner via the applications, it will run in **privileged +mode** by default. Make sure you read the [security +implications](../project/clusters/index.md/#security-implications) before doing so. + +NOTE: **Note:** +The +[runner/gitlab-runner](https://gitlab.com/charts/gitlab-runner) +chart is used to install this application with a +[`values.yaml`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/vendor/runner/values.yaml) +file. + +### Ingress + +> - Available for project-level clusters since GitLab 10.2. +> - Available for group-level clusters since GitLab 11.6. + +[Ingress](https://kubernetes.github.io/ingress-nginx/) can provide load +balancing, SSL termination, and name-based virtual hosting. It acts as a +web proxy for your applications and is useful if you want to use [Auto +DevOps] or deploy your own web apps. + +NOTE: **Note:** +The +[stable/nginx-ingress](https://github.com/helm/charts/tree/master/stable/nginx-ingress) +chart is used to install this application with a +[`values.yaml`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/vendor/ingress/values.yaml) +file. + +### JupyterHub + +> Available for project-level clusters since GitLab 11.0. + +[JupyterHub](https://jupyterhub.readthedocs.io/en/stable/) is a +multi-user service for managing notebooks across a team. [Jupyter +Notebooks](https://jupyter-notebook.readthedocs.io/en/latest/) provide a +web-based interactive programming environment used for data analysis, +visualization, and machine learning. + +Authentication will be enabled only for [project +members](../project/members/index.md) with [Developer or +higher](../permissions.md) access to the project. + +We use a [custom Jupyter +image](https://gitlab.com/gitlab-org/jupyterhub-user-image/blob/master/Dockerfile) +that installs additional useful packages on top of the base Jupyter. You +will also see ready-to-use DevOps Runbooks built with Nurtch's [Rubix library](https://github.com/amit1rrr/rubix). + +More information on +creating executable runbooks can be found in [our Runbooks +documentation](../project/clusters/runbooks/index.md#executable-runbooks). Note that +Ingress must be installed and have an IP address assigned before +JupyterHub can be installed. + +NOTE: **Note:** +The +[jupyter/jupyterhub](https://jupyterhub.github.io/helm-chart/) +chart is used to install this application with a +[`values.yaml`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/vendor/jupyter/values.yaml) +file. + +#### Jupyter Git Integration + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/28783) in GitLab 12 for project-level clusters. + +When installing JupyterHub onto your Kubernetes cluster, [JupyterLab's Git extension](https://github.com/jupyterlab/jupyterlab-git) +is automatically provisioned and configured using the authenticated user's: + +- Name +- Email +- Newly created access token + +JupyterLab's Git extension enables full version control of your notebooks as well as issuance of Git commands within Jupyter. +Git commands can be issued via the **Git** tab on the left panel or via Jupyter's command line prompt. + +NOTE: **Note:** +JupyterLab's Git extension stores the user token in the JupyterHub DB in encrypted format +and in the single user Jupyter instance as plain text. This is because [Git requires storing +credentials as plain text](https://git-scm.com/docs/git-credential-store). Potentially, if +a nefarious user finds a way to read from the file system in the single user Jupyter instance +they could retrieve the token. + + + +You can clone repositories from the files tab in Jupyter: + + + +### Knative + +> Available for project-level clusters since GitLab 11.5. + +[Knative](https://cloud.google.com/knative) provides a platform to +create, deploy, and manage serverless workloads from a Kubernetes +cluster. It is used in conjunction with, and includes +[Istio](https://istio.io) to provide an external IP address for all +programs hosted by Knative. + +You will be prompted to enter a wildcard +domain where your applications will be exposed. Configure your DNS +server to use the external IP address for that domain. For any +application created and installed, they will be accessible as +`<program_name>.<kubernetes_namespace>.<domain_name>`. This will require +your kubernetes cluster to have [RBAC +enabled](../project/clusters/index.md#rbac-cluster-resources). + +NOTE: **Note:** +The +[knative/knative](https://storage.googleapis.com/triggermesh-charts) +chart is used to install this application. + +### Prometheus + +> - Available for project-level clusters since GitLab 10.4. +> - Available for group-level clusters since GitLab 11.11. + +[Prometheus](https://prometheus.io/docs/introduction/overview/) is an +open-source monitoring and alerting system useful to supervise your +deployed applications. + +NOTE: **Note:** +The +[stable/prometheus](https://github.com/helm/charts/tree/master/stable/prometheus) +chart is used to install this application with a +[`values.yaml`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/vendor/prometheus/values.yaml) +file. + +## Upgrading applications + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/24789) +in GitLab 11.8. + +The applications below can be upgraded. + +| Application | GitLab version | +| ----------- | -------------- | +| Runner | 11.8+ | + +To upgrade an application: + +1. For a: + - Project-level cluster, navigate to your project's **Operations > Kubernetes**. + - Group-level cluster, navigate to your group's **Kubernetes** page. +1. Select your cluster. +1. If an upgrade is available, the **Upgrade** button is displayed. Click the button to upgrade. + +NOTE: **Note:** +Upgrades will reset values back to the values built into the `runner` +chart plus the values set by +[`values.yaml`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/vendor/runner/values.yaml) + +## Uninstalling applications + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/60665) in +> GitLab 11.11. + +The applications below can be uninstalled. + +| Application | GitLab version | Notes | +| ----------- | -------------- | ----- | +| Prometheus | 11.11+ | All data will be deleted and cannot be restored. | + +To uninstall an application: + +1. For a: + - Project-level cluster, navigate to your project's **Operations > Kubernetes**. + - Group-level cluster, navigate to your group's **Kubernetes** page. +1. Select your cluster. +1. Click the **Uninstall** button for the application. + +Support for uninstalling all applications is planned for progressive rollout. +To follow progress, see [the relevant +epic](https://gitlab.com/groups/gitlab-org/-/epics/1201). + +## Troubleshooting applications + +Applications can fail with the following error: + +```text +Error: remote error: tls: bad certificate +``` + +To avoid installation errors: + +- Before starting the installation of applications, make sure that time is synchronized + between your GitLab server and your Kubernetes cluster. +- Ensure certificates are not out of sync. When installing applications, GitLab expects a new cluster with no previous installation of Helm. + + You can confirm that the certificates match via `kubectl`: + + ```sh + kubectl get configmaps/values-content-configuration-ingress -n gitlab-managed-apps -o \ + "jsonpath={.data['cert\.pem']}" | base64 -d > a.pem + kubectl get secrets/tiller-secret -n gitlab-managed-apps -o "jsonpath={.data['ca\.crt']}" | base64 -d > b.pem + diff a.pem b.pem + ``` + diff --git a/doc/user/clusters/img/jupyter-git-extension.gif b/doc/user/clusters/img/jupyter-git-extension.gif Binary files differnew file mode 100644 index 00000000000..13a88d97425 --- /dev/null +++ b/doc/user/clusters/img/jupyter-git-extension.gif diff --git a/doc/user/clusters/img/jupyter-gitclone.png b/doc/user/clusters/img/jupyter-gitclone.png Binary files differnew file mode 100644 index 00000000000..41d467f806a --- /dev/null +++ b/doc/user/clusters/img/jupyter-gitclone.png diff --git a/doc/user/group/clusters/index.md b/doc/user/group/clusters/index.md index 53c82169e15..3c5e820c1ca 100644 --- a/doc/user/group/clusters/index.md +++ b/doc/user/group/clusters/index.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Group-level Kubernetes clusters > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/34758) in GitLab 11.6. @@ -5,40 +9,17 @@ ## Overview -Similar to [project Kubernetes -clusters](../../project/clusters/index.md), Group-level Kubernetes -clusters allow you to connect a Kubernetes cluster to your group, -enabling you to use the same cluster across multiple projects. +Similar to [project-level](../../project/clusters/index.md) and +[instance-level](../../instance/clusters/index.md) Kubernetes clusters, +group-level Kubernetes clusters allow you to connect a Kubernetes cluster to +your group, enabling you to use the same cluster across multiple projects. ## Installing applications -GitLab provides a one-click install for various applications that can be -added directly to your cluster. - -NOTE: **Note:** -Applications will be installed in a dedicated namespace called -`gitlab-managed-apps`. If you have added an existing Kubernetes cluster -with Tiller already installed, you should be careful as GitLab cannot -detect it. In this event, installing Tiller via the applications will -result in the cluster having it twice. This can lead to confusion during -deployments. - -| Application | GitLab version | Description | Helm Chart | -| ----------- | -------------- | ----------- | ---------- | -| [Helm Tiller](https://docs.helm.sh) | 11.6+ | Helm is a package manager for Kubernetes and is required to install all the other applications. It is installed in its own pod inside the cluster which can run the `helm` CLI in a safe environment. | n/a | -| [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress) | 11.6+ | Ingress can provide load balancing, SSL termination, and name-based virtual hosting. It acts as a web proxy for your applications and is useful if you want to use [Auto DevOps](../../../topics/autodevops/index.md) or deploy your own web apps. | [stable/nginx-ingress](https://github.com/helm/charts/tree/master/stable/nginx-ingress) | -| [Cert-Manager](https://docs.cert-manager.io/en/latest/) | 11.6+ | Cert-Manager is a native Kubernetes certificate management controller that helps with issuing certificates. Installing Cert-Manager on your cluster will issue a certificate by [Let's Encrypt](https://letsencrypt.org/) and ensure that certificates are valid and up-to-date. | [stable/cert-manager](https://github.com/helm/charts/tree/master/stable/cert-manager) | -| [Prometheus](https://prometheus.io/docs/introduction/overview/) | 11.11+ | Prometheus is an open-source monitoring and alerting system useful to supervise your deployed applications. | [stable/prometheus](https://github.com/helm/charts/tree/master/stable/prometheus) | -| [GitLab Runner](https://docs.gitlab.com/runner/) | 11.10+ | GitLab Runner is the open source project that is used to run your jobs and send the results back to GitLab. It is used in conjunction with [GitLab CI/CD](../../../ci/README.md), the open-source continuous integration service included with GitLab that coordinates the jobs. When installing the GitLab Runner via the applications, it will run in **privileged mode** by default. Make sure you read the [security implications](../../project/clusters/index.md#security-implications) before doing so. | [runner/gitlab-runner](https://gitlab.com/charts/gitlab-runner) | - -NOTE: **Note:** -Some [cluster -applications](../../project/clusters/index.md#installing-applications) -are installable only for a project-level cluster. Support for installing these -applications in a group-level cluster is planned for future releases. For updates, see: - -- Support installing [JupyterHub in group-level - clusters](https://gitlab.com/gitlab-org/gitlab-ce/issues/51989) +GitLab can install and manage some applications in your group-level +cluster. For more information on installing, upgrading, uninstalling, +and troubleshooting applications for your group cluster, see +[Gitlab Managed Apps](../../clusters/applications.md). ## RBAC compatibility @@ -71,7 +52,7 @@ Add another cluster similar to the first one and make sure to [set an environment scope](#environment-scopes-premium) that will differentiate the new cluster from the rest. -## Gitlab-managed clusters +## GitLab-managed clusters > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/22011) in GitLab 11.5. > Became [optional](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/26565) in GitLab 11.11. @@ -110,7 +91,7 @@ The domain should have a wildcard DNS configured to the Ingress IP address. When adding more than one Kubernetes cluster to your project, you need to differentiate them with an environment scope. The environment scope associates clusters with [environments](../../../ci/environments.md) similar to how the -[environment-specific variables](https://docs.gitlab.com/ee/ci/variables/#limiting-environment-scopes-of-environment-variables-premium) +[environment-specific variables](../../../ci/variables/README.md#limiting-environment-scopes-of-environment-variables-premium) work. While evaluating which environment matches the environment scope of a @@ -168,3 +149,15 @@ The following features are not currently available for group-level clusters: 1. Terminals (see [related issue](https://gitlab.com/gitlab-org/gitlab-ce/issues/55487)). 1. Pod logs (see [related issue](https://gitlab.com/gitlab-org/gitlab-ce/issues/55488)). 1. Deployment boards (see [related issue](https://gitlab.com/gitlab-org/gitlab-ce/issues/55489)). + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/group/contribution_analytics/index.md b/doc/user/group/contribution_analytics/index.md index bc88eff9ed2..7e6cb24a51e 100644 --- a/doc/user/group/contribution_analytics/index.md +++ b/doc/user/group/contribution_analytics/index.md @@ -1,46 +1,58 @@ -# Contribution Analytics **[STARTER]** +--- +type: reference +--- ->**Note:** -Introduced in [GitLab Starter][ee] 8.3. +# Contribution Analytics **[STARTER]** -Track your team members' activity across your organization. +> Introduced in [GitLab Starter](https://about.gitlab.com/pricing/) 8.3. ## Overview -With Contribution Analytics you can get an overview of the activity of -issues, merge requests, and push events of your organization and its members. +With Contribution Analytics you can get an overview of the following activity in your +group: -The analytics page is located at **Group > Contribution Analytics** -under the URL `/groups/<groupname>/analytics`. +- Issues +- Merge requests +- Push events + +To view the Contribution Analytics, go to your group's **Overview > Contribution Analytics** +page. ## Use cases -- Analyze your team's contributions over a period of time and offer a bonus for the top contributors -- Identify opportunities for improvement with group members who may benefit from additional support +- Analyze your team's contributions over a period of time, and offer a bonus for the top +contributors. +- Identify opportunities for improvement with group members who may benefit from additional +support. ## Using Contribution Analytics -There are three main bar graphs that are deducted from the number of -contributions per group member. These contributions include push events, merge -requests and closed issues. Hovering on each bar you can see the number of -events for a specific member. +There are three main bar graphs that illustrate the number of contributions per group +member for the following: + +- Push events +- Merge requests +- Closed issues + +Hover over each bar to display the number of events for a specific group member.  ## Changing the period time -There are three periods you can choose from, 'Last week', 'Last month' and -'Last three months'. The default is 'Last week'. +You can choose from the following three periods: + +- Last week (default) +- Last month +- Last three months -You can choose which period to display by using the dropdown calendar menu in -the upper right corner. +Select the desired period from the calendar dropdown.  ## Sorting by different factors -Apart from the bar graphs you can also see the contributions per group member -which are depicted in a table that can be sorted by: +Contributions per group member are also presented in tabular format. Click a column header to sort the table by that column: * Member name * Number of pushed events @@ -52,4 +64,14 @@ which are depicted in a table that can be sorted by:  -[ee]: https://about.gitlab.com/pricing/ +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/group/custom_project_templates.md b/doc/user/group/custom_project_templates.md index f67325272a6..aa088d2fcdb 100644 --- a/doc/user/group/custom_project_templates.md +++ b/doc/user/group/custom_project_templates.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Custom group-level project templates **[PREMIUM]** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/6861) in [GitLab Premium](https://about.gitlab.com/pricing) 11.6. @@ -24,3 +28,15 @@ Projects of nested subgroups of a selected template source cannot be used. Repository and database information that are copied over to each new project are identical to the data exported with [GitLab's Project Import/Export](../project/settings/import_export.md). + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/group/epics/index.md b/doc/user/group/epics/index.md index 6035f0c7326..2e4106f55e5 100644 --- a/doc/user/group/epics/index.md +++ b/doc/user/group/epics/index.md @@ -1,3 +1,7 @@ +--- +type: reference, howto +--- + # Epics **[ULTIMATE]** > Introduced in [GitLab Ultimate][ee] 10.2. @@ -215,3 +219,15 @@ Once you wrote your comment, you can either: ## Notifications - [Receive notifications](../../../workflow/notifications.md) for epic events. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/group/index.md b/doc/user/group/index.md index a5e3bfda70e..eb0c7bc998f 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -1,37 +1,49 @@ +--- +type: reference, howto +--- + # Groups -With GitLab Groups you can assemble related projects together -and grant members access to several projects at once. +With GitLab Groups, you can: + +- Assemble related projects together. +- Grant members access to several projects at once. Groups can also be nested in [subgroups](subgroups/index.md). -Find your groups by clicking **Groups** in the top navigation. +Find your groups by clicking **Groups > Your Groups** in the top navigation.  -> The groups dropdown in the top navigation was [introduced][ce-36234] in [GitLab 11.1](https://about.gitlab.com/2018/07/22/gitlab-11-1-released/#groups-dropdown-in-navigation). +> The **Groups** dropdown in the top navigation was [introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/36234) in [GitLab 11.1](https://about.gitlab.com/2018/07/22/gitlab-11-1-released/#groups-dropdown-in-navigation). + +The **Groups** page displays: + +- All groups you are a member of, when **Your groups** is selected. +- A list of public groups, when **Explore public groups** is selected. -The Groups page displays all groups you are a member of, how many projects it holds, -how many members it has, the group visibility, and, if you have enough permissions, -a link to the group settings. By clicking the last button you can leave that group. +Each group on the **Groups** page is listed with: + +- How many subgroups it has. +- How many projects it contains. +- How many members the group has, not including members inherited from parent groups. +- The group's visibility. +- A link to the group's settings, if you have sufficient permissions. +- A link to leave the group, if you are a member. ## Use cases -You can create groups for numerous reasons. To name a few: - -- Organize related projects under the same [namespace](#namespaces), add members to that - group and grant access to all their projects at once -- Create a group, include members of your team, and make it easier to - `@mention` all the team at once in issues and merge requests - - Create a group for your company members, and create [subgroups](subgroups/index.md) - for each individual team. Let's say you create a group called `company-team`, and among others, - you created subgroups in this group for each individual team `backend-team`, - `frontend-team`, and `production-team`: - 1. When you start a new implementation from an issue, you add a comment: +You can create groups for numerous reasons. To name a couple: + +- Grant access to multiple projects and multiple team members in fewer steps by organizing related projects under the same [namespace](#namespaces) and adding members to the top-level group. +- Make it easier to `@mention` all of your team at once in issues and merge requests by creating a group and including the appropriate members. + +For example, you could create a group for your company members, and create a [subgroup](subgroups/index.md) for each individual team. Let's say you create a group called `company-team`, and you create subgroups in this group for the individual teams `backend-team`, `frontend-team`, and `production-team`. + - When you start a new implementation from an issue, you add a comment: _"`@company-team`, let's do it! `@company-team/backend-team` you're good to go!"_ - 1. When your backend team needs help from frontend, they add a comment: + - When your backend team needs help from frontend, they add a comment: _"`@company-team/frontend-team` could you help us here please?"_ - 1. When the frontend team completes their implementation, they comment: + - When the frontend team completes their implementation, they comment: _"`@company-team/backend-team`, it's done! Let's ship it `@company-team/production-team`!"_ ## Namespaces @@ -59,8 +71,8 @@ By doing so: ## Issues and merge requests within a group -Issues and merge requests are part of projects. For a given group, view all the -[issues](../project/issues/index.md#issues-list) and [merge requests](../project/merge_requests/index.md#merge-requests-per-group) across all the projects in that group, +Issues and merge requests are part of projects. For a given group, you can view all of the +[issues](../project/issues/index.md#issues-list) and [merge requests](../project/merge_requests/index.md#merge-requests-per-group) across all projects in that group, together in a single list view. ## Create a new group @@ -68,13 +80,13 @@ together in a single list view. > For a list of words that are not allowed to be used as group names see the > [reserved names](../reserved_names.md). -You can create a group in GitLab from: +To create a new Group, either: -1. The Groups page: from the top menu, click **Groups**, and click the green button **New group**: +- In the top menu, click **Groups** and then **Your Groups**, and click the green button **New group**.  -1. Elsewhere: expand the `plus` sign button on the top navbar and choose **New group**: +- Or, in the top menu, expand the `plus` sign and choose **New group**.  @@ -82,18 +94,18 @@ Add the following information:  -1. The **Group name** will populate the URL automatically. Optionally, you can change it. - This is the name that is displayed in the group views. +1. The **Group name** will automatically populate the URL. Optionally, you can change it. + This is the name that displays in group views. The name can contain only: - - Alphanumeric characters. - - Underscores. - - Dashes and dots. - - Spaces. -1. The **Group URL**, which will be the namespace under which your projects will be hosted. + - Alphanumeric characters + - Underscores + - Dashes and dots + - Spaces +1. The **Group URL** is the namespace under which your projects will be hosted. The URL can contain only: - - Alphanumeric characters. - - Underscores. - - Dashes and dots. It cannot start with dashes or end in dot. + - Alphanumeric characters + - Underscores + - Dashes and dots (it cannot start with dashes or end in a dot) 1. Optionally, you can add a brief description to tell others what this group is about. 1. Optionally, choose an avatar for your group. @@ -101,45 +113,39 @@ Add the following information: ## Add users to a group -Add members to a group by navigating to the group's dashboard, and clicking **Members**: +A benefit of putting multiple projects in one group is that you can +give a user to access to all projects in the group with one action. + +Add members to a group by navigating to the group's dashboard and clicking **Members**.  -Select the [permission level](../permissions.md#permissions) and add the new member. You can also set the expiring -date for that user, from which they will no longer have access to your group. +Select the [permission level](../permissions.md#permissions), and add the new member. You can also set the expiring date for that user; this is the date on which they will no longer have access to your group. -One of the benefits of putting multiple projects in one group is that you can -give a user to access to all projects in the group with one action. +Consider a group with two projects: -Consider we have a group with two projects: - -- On the **Group Members** page we can now add a new user to the group. -- Now because this user is a **Developer** member of the group, he automatically +- On the **Group Members** page, you can now add a new user to the group. +- Now, because this user is a **Developer** member of the group, they automatically gets **Developer** access to **all projects** within that group. -If necessary, you can increase the access level of an individual user for a specific project, -by adding them again as a new member to the project with the new permission levels. +To increase the access level of an existing user for a specific project, +add them again as a new member to the project with the desired permission level. ## Request access to a group -As a group owner you can enable or disable non members to request access to -your group. Go to the group settings and click on **Allow users to request access**. +As a group owner, you can enable or disable the ability for non members to request access to +your group. Go to the group settings, and click **Allow users to request access**. -As a user, you can request to be a member of a group. Go to the group you'd -like to be a member of, and click the **Request Access** button on the right +As a user, you can request to be a member of a group, if that setting is enabled. Go to the group for which you'd like to be a member, and click the **Request Access** button on the right side of your screen.  ---- - Group owners and maintainers will be notified of your request and will be able to approve or decline it on the members page.  ---- - If you change your mind before your request is approved, just click the **Withdraw Access Request** button. @@ -149,29 +155,27 @@ If you change your mind before your request is approved, just click the There are two different ways to add a new project to a group: -- Select a group and then click on the **New project** button. +- Select a group, and then click **New project**. You can then continue [creating your project](../../gitlab-basics/create-project.md).  - You can then continue on [creating a project](../../gitlab-basics/create-project.md). - - While you are creating a project, select a group namespace you've already created from the dropdown menu.  -### Default project creation level +### Default project-creation level > [Introduced][ee-2534] in [GitLab Premium][ee] 10.5. > Brought to [GitLab Starter][ee] in 10.7. > [Moved](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/25975) to [GitLab Core](https://about.gitlab.com/pricing/) in 11.10. -Group owners or administrators can allow users with the +Group owners and administrators can allow users with the Developer role to create projects under groups. -By default, [Developers and Maintainers](../permissions.md#group-members-permissions) can create projects under a group, but this can be changed either within the group settings for a group, or -be set globally by a GitLab administrator in the Admin area -at **Settings > General > Visibility and access controls**. +By default, [Developers and Maintainers](../permissions.md#group-members-permissions) can create projects under a group. You can change this setting for a specific group within the group settings, or +you can set this option globally in the Admin area +at **Settings > General > Visibility and access controls** (you must be a GitLab administrator). Available settings are `No one`, `Maintainers`, or `Developers + Maintainers`. @@ -182,13 +186,13 @@ Learn how to [transfer a project into a group](../project/settings/index.md#tran ## Sharing a project with a group You can [share your projects with a group](../project/members/share_project_with_groups.md) -and give your group members access to the project all at once. +and give all group members access to the project at once. Alternatively, you can [lock the sharing with group feature](#share-with-group-lock). ## Manage group memberships via LDAP -In GitLab Enterprise Edition it is possible to manage GitLab group memberships using LDAP groups. +In GitLab Enterprise Edition, it is possible to manage GitLab group memberships using LDAP groups. See [the GitLab Enterprise Edition documentation](../../integration/ldap.md) for more information. ## Epics **[ULTIMATE]** @@ -211,38 +215,42 @@ Get an overview of the vulnerabilities of all the projects in a group and its su > Introduced in [GitLab Ultimate][ee] 11.9 behind the `insights` feature flag. -Configure the Insights that matter for your groups or projects to explore data -such as triage hygiene, issues created/closed per a given period, average time -for merge requests to be merged and much more. +Configure the Insights that matter for your groups or projects, allowing users to explore data +such as: + +- Triage hygiene +- Issues created/closed per a given period +- Average time for merge requests to be merged +- Much more [Learn more about Insights](insights/index.md). ## Transferring groups -From GitLab 10.5, groups can be transferred in the following ways: +From GitLab 10.5, you can transfer groups in the following ways: -- Top-level groups can be transferred to a group, converting them into subgroups. -- Subgroups can be transferred to a new parent group. -- Subgroups can be transferred out from a parent group, converting them into top-level groups. +- Transfer a subgroup to a new parent group. +- Convert a top-level group into a subgroup by transfering it to the desired group. +- Convert a subgroup into a top-level group by transfering it out of its current group. When transferring groups, note: - Changing a group's parent can have unintended side effects. See [Redirects when changing repository paths](../project/index.md#redirects-when-changing-repository-paths). - You can only transfer groups to groups you manage. -- You will need to update your local repositories to point to the new location. -- If the parent group's visibility is lower than the group's current visibility, visibility levels for subgroups and projects will be changed to match the new parent group's visibility. -- Only explicit group membership is transferred, not inherited membership. If the group's owners have only inherited membership, this would leave the group without an owner. In this case, the user transferring the group becomes the group's owner. +- You must update your local repositories to point to the new location. +- If the parent group's visibility is lower than the group's current visibility, visibility levels for subgroups and projects will change to match the new parent group's visibility. +- Only explicit group membership is transferred, not inherited membership. If the group's owners have only inherited membership, this leaves the group without an owner. In this case, the user transferring the group becomes the group's owner. ## Group settings -Once you have created a group, you can manage its settings by navigating to +After creating a group, you can manage its settings by navigating to the group's dashboard, and clicking **Settings**.  ### General settings -Besides giving you the option to edit any settings you've previously +In addition to editing any settings you previously set when [creating the group](#create-a-new-group), you can also access further configurations for your group. @@ -253,14 +261,14 @@ Changing a group's path can have unintended side effects. Read before proceeding. If you are vacating the path so it can be claimed by another group or user, -you may need to rename the group name as well since both names and paths must +you may need to rename the group, too, since both names and paths must be unique. To change your group path: 1. Navigate to your group's **Settings > General**. -1. Enter a new name under "Group path". -1. Hit **Save group**. +1. Enter a new name under **Group path**. +1. Click **Save group**. CAUTION: **Caution:** It is currently not possible to rename a namespace if it contains a @@ -276,20 +284,17 @@ username, you can create a new group and transfer projects to it. Add a security layer to your group by [enforcing two-factor authentication (2FA)](../../security/two_factor_authentication.md#enforcing-2fa-for-all-users-in-a-group) -to all group members. +for all group members. #### Share with group lock Prevent projects in a group from [sharing -a project with another group](../project/members/share_project_with_groups.md). -This allows for tighter control over project access. +a project with another group](../project/members/share_project_with_groups.md) to enable tighter control over project access. -For example, consider you have two distinct teams (Group A and Group B) -working together in a project. -To inherit the group membership, you share the project between the +For example, let's say you have two distinct teams (Group A and Group B) working together in a project, and to inherit the group membership, you share the project between the two groups A and B. **Share with group lock** prevents any project within -the group from being shared with another group. By doing so, you -guarantee only the right group members have access to those projects. +the group from being shared with another group, +guaranteeing that only the right group members have access to those projects. To enable this feature, navigate to the group settings page. Select **Share with group lock** and **Save the group**. @@ -298,40 +303,37 @@ To enable this feature, navigate to the group settings page. Select #### Member Lock **[STARTER]** -With Member lock, it is possible to lock membership in a project to the -level of members in the group. - -Member lock lets a group owner lock down any new project membership to all the -projects within the group, allowing tighter control over project membership. +Member lock lets a group owner prevent any new project membership to all of the +projects within a group, allowing tighter control over project membership. -For instance, if you want to lock the group for an [Audit Event](https://docs.gitlab.com/ee/administration/audit_events.html), -you enable Member lock to guarantee that membership of a project cannot be modified during that audit. +For example, if you want to lock the group for an [Audit Event](../../administration/audit_events.md), +enable Member lock to guarantee that project membership cannot be modified during that audit. To enable this feature: 1. Navigate to the group's **Settings > General** page. -1. Expand the **Permissions, LFS, 2FA** section and select **Member lock**. -1. Click the **Save changes** button. +1. Expand the **Permissions, LFS, 2FA** section, and select **Member lock**. +1. Click **Save changes**.  This will disable the option for all users who previously had permissions to -operate project memberships so no new users can be added. Furthermore, any +operate project memberships, so no new users can be added. Furthermore, any request to add a new user to a project through API will not be possible. #### Group file templates **[PREMIUM]** Group file templates allow you to share a set of templates for common file types with every project in a group. It is analogous to the -[instance template repository](https://docs.gitlab.com/ee/user/admin_area/settings/instance_template_repository.html) +[instance template repository](../admin_area/settings/instance_template_repository.md) feature, and the selected project should follow the same naming conventions as are documented on that page. -Only projects that are in the group may be chosen as the source of templates. -This includes projects shared with the group, but **excludes** projects in +You can only choose projects in the group as the template source. +This includes projects shared with the group, but it **excludes** projects in subgroups or parent groups of the group being configured. -This feature may be configured for subgroups as well as parent groups. A project +You can configure this feature for both subgroups and parent groups. A project in a subgroup will have access to the templates for that subgroup, as well as any parent groups. @@ -345,32 +347,44 @@ To enable this feature, navigate to the group settings page, expand the #### Group-level project templates **[PREMIUM]** -Define project templates at a group-level by setting a group as a template source. +Define project templates at a group level by setting a group as the template source. [Learn more about group-level project templates](custom_project_templates.md). ### Advanced settings -- **Projects**: view all projects within that group, add members to each project, - access each project's settings, and remove any project from the same screen. -- **Webhooks**: configure [webhooks](../project/integrations/webhooks.md) to your group. -- **Kubernetes cluster integration**: connect your GitLab group with [Kubernetes clusters](clusters/index.md). -- **Audit Events**: view [Audit Events](https://docs.gitlab.com/ee/administration/audit_events.html) +- **Projects**: View all projects within that group, add members to each project, + access each project's settings, and remove any project, all from the same screen. +- **Webhooks**: Configure [webhooks](../project/integrations/webhooks.md) for your group. +- **Kubernetes cluster integration**: Connect your GitLab group with [Kubernetes clusters](clusters/index.md). +- **Audit Events**: View [Audit Events](../../administration/audit_events.md) for the group. **[STARTER ONLY]** -- **Pipelines quota**: keep track of the [pipeline quota](../admin_area/settings/continuous_integration.md) for the group. +- **Pipelines quota**: Keep track of the [pipeline quota](../admin_area/settings/continuous_integration.md) for the group. ## User contribution analysis **[STARTER]** -With [GitLab Contribution Analytics](contribution_analytics/index.md) +With [GitLab Contribution Analytics](contribution_analytics/index.md), you have an overview of the contributions (pushes, merge requests, and issues) performed by your group members. ## Issues analytics **[PREMIUM]** -With [GitLab Issues Analytics](issues_analytics/index.md), in groups, you can see a bar chart of the number of issues created each month. +With [GitLab Issues Analytics](issues_analytics/index.md), you can see a bar chart of the number of issues created each month in your groups. ## Dependency Proxy **[PREMIUM]** Use GitLab as a [dependency proxy](dependency_proxy/index.md) for upstream Docker images. +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> + [ee]: https://about.gitlab.com/pricing/ [ee-2534]: https://gitlab.com/gitlab-org/gitlab-ee/issues/2534 diff --git a/doc/user/group/insights/img/insights_sidebar_link.png b/doc/user/group/insights/img/insights_sidebar_link.png Binary files differnew file mode 100644 index 00000000000..64bbd1fc4d3 --- /dev/null +++ b/doc/user/group/insights/img/insights_sidebar_link.png diff --git a/doc/user/group/insights/index.md b/doc/user/group/insights/index.md index 5283d8da77d..a4ea71074ec 100644 --- a/doc/user/group/insights/index.md +++ b/doc/user/group/insights/index.md @@ -1,11 +1,15 @@ +--- +type: reference, howto +--- + # Insights **[ULTIMATE]** > Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.9 behind the `insights` feature flag. CAUTION: **Beta:** Insights is considered beta, and is not ready for production use. -Follow [gitlab-org&725](https://gitlab.com/groups/gitlab-org/-/epics/725) for -updates. +Follow [gitlab-org/quality/team-tasks#137](https://gitlab.com/gitlab-org/quality/team-tasks/issues/137#general-availability) +for updates. Configure the Insights that matter for your groups to explore data such as triage hygiene, issues created/closed per a given period, average time for merge @@ -13,6 +17,13 @@ requests to be merged and much more.  +## View your group's Insights + +You can access your group's Insights by clicking the **Overview > Insights** +link in the left sidebar: + + + ## Configure your Insights Navigate to your group's **Settings > General**, expand **Insights**, and choose @@ -21,10 +32,10 @@ the project that holds your `.gitlab/insights.yml` configuration file:  If no configuration was set, a [default configuration file]( -https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/fixtures/insights/ee/fixtures/insights/default.yml) +https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/fixtures/insights/default.yml) will be used. -See the [Project's Insights documentation](https://docs.gitlab.com/ee/user/project/insights/index.html) for +See the [Project's Insights documentation](../../project/insights/index.md) for more details about the `.gitlab/insights.yml` configuration file. ## Permissions @@ -37,3 +48,15 @@ access to the project they belong to, or because they are confidential) are filtered out of the Insights charts. You may also consult the [group permissions table](../../permissions.md#group-members-permissions). + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/group/issues_analytics/index.md b/doc/user/group/issues_analytics/index.md index cf53d7423a6..46d5c1e2e09 100644 --- a/doc/user/group/issues_analytics/index.md +++ b/doc/user/group/issues_analytics/index.md @@ -1,16 +1,43 @@ +--- +type: reference +--- + # Issues Analytics **[PREMIUM]** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/7478) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.5. -GitLab by default displays a bar chart of the number of issues created each month, for the -current month, and 12 months prior, for a total of 13 months. +Issues Analytics is a bar graph which illustrates the number of issues created each month. +The default timespan is 13 months, which includes the current month, and the 12 months +prior. -You can change the total number of months displayed by setting a URL parameter. -For example, `https://gitlab.com/groups/gitlab-org/-/issues_analytics?months_back=15` -would show a total of 15 months for the chart in the GitLab.org group. +To access the chart, navigate to a group's sidebar and select **Issues > Analytics**. -The **Search or filter results...** field can be used for filtering the issues by any attribute. For example, labels, assignee, milestone, and author. +Hover over each bar to see the total number of issues. -To access the chart, navigate to a group's sidebar and select **Issues > Analytics**. +To narrow the scope of issues included in the graph, enter your criteria in the +**Search or filter results...** field. Criteria from the following list can be typed in or selected from a menu: + +- Author +- Assignee +- Milestone +- Label +- My reaction +- Weight + +You can change the total number of months displayed by setting a URL parameter. +For example, `https://gitlab.com/groups/gitlab-org/-/issues_analytics?months_back=15` +shows a total of 15 months for the chart in the GitLab.org group.  + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/group/roadmap/index.md b/doc/user/group/roadmap/index.md index 310c4bb88d0..683c715c8d5 100644 --- a/doc/user/group/roadmap/index.md +++ b/doc/user/group/roadmap/index.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Roadmap **[ULTIMATE]** > Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing) 10.5. @@ -19,13 +23,20 @@ Epics in the view can be sorted by: - **Start date** - **Due date** -Each option contains a button that can toggle the order between **ascending** and **descending**. The sort option and order will be persisted to be used wherever epics are browsed including the [epics list view](../epics/index.md). +Each option contains a button that toggles the sort order between **ascending** and **descending**. The sort option and order will be persisted when browsing Epics, +including the [epics list view](../epics/index.md). Roadmaps can also be [visualized inside an epic](../epics/index.md#roadmap-in-epics). ## Timeline duration -Starting with [GitLab Ultimate][ee] 11.0, Roadmap supports three different date ranges; Quarters, Months (Default) and Weeks. +> Introduced in [GitLab Ultimate](https://about.gitlab.com/pricing) 11.0. + +Roadmap supports the following date ranges: + +- Quarters +- Months (Default) +- Weeks ### Quarters @@ -62,3 +73,15 @@ and due date. If an epic doesn't have a due date, the timeline bar fades away towards the future. Similarly, if an epic doesn't have a start date, the timeline bar becomes more visible as it approaches the epic's due date on the timeline. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/group/saml_sso/index.md b/doc/user/group/saml_sso/index.md index 53116606201..fcfd638f185 100644 --- a/doc/user/group/saml_sso/index.md +++ b/doc/user/group/saml_sso/index.md @@ -1,3 +1,7 @@ +--- +type: reference, howto +--- + # SAML SSO for GitLab.com Groups **[SILVER ONLY]** > Introduced in [GitLab.com Silver](https://about.gitlab.com/pricing/) 11.0. @@ -5,17 +9,17 @@ NOTE: **Note:** This topic is for SAML on GitLab.com Silver tier and above. For SAML on self-managed GitLab instances, see [SAML OmniAuth Provider](../../../integration/saml.md). -Currently SAML on GitLab.com can be used to automatically add users to a group, and does not yet sign users into GitLab.com. Users should already have an account on the GitLab instance, or can create one when logging in for the first time. +SAML on GitLab.com allows users to be automatically added to a group, and then allows those users to sign into GitLab.com. Users should already have an account on the GitLab instance, or can create one when logging in for the first time. User synchronization for GitLab.com is partially supported using [SCIM](scim_setup.md). NOTE: **Note:** -SAML SSO for groups is used only as a convenient way to add users and does not sync users between providers without using SCIM. If a group is not using SCIM, group Owners will still need to manage user accounts, such as removing users when necessary. +SAML SSO for GitLab.com groups does not sync users between providers without using SCIM. If a group is not using SCIM, group Owners will still need to manage user accounts (for example, removing users when necessary). ## Configuring your Identity Provider 1. Navigate to the group and click **Settings > SAML SSO**. -1. Configure your SAML server using the **Assertion consumer service URL** and **Issuer**. See [your identity provider's documentation](#providers) for more details. +1. Configure your SAML server using the **Assertion consumer service URL** and **Identifier**. Alternatively GitLab provides [metadata XML configuration](#metadata-configuration). See [your identity provider's documentation](#providers) for more details. 1. Configure the SAML response to include a NameID that uniquely identifies each user. 1. Configure required assertions using the [table below](#assertions). 1. Once the identity provider is set up, move on to [configuring GitLab](#configuring-gitlab). @@ -43,12 +47,20 @@ GitLab.com uses the SAML NameID to identify users. The NameID element: ### Assertions -| Field | Supported keys | Notes | -|-|----------------|-------------| -| Email | `email`, `mail` | (required) | -| Full Name | `name` | | -| First Name | `first_name`, `firstname`, `firstName` | | -| Last Name | `last_name`, `lastname`, `lastName` | | +| Field | Supported keys | +|-------|----------------| +| Email (required)| `email`, `mail` | +| Full Name | `name` | +| First Name | `first_name`, `firstname`, `firstName` | +| Last Name | `last_name`, `lastname`, `lastName` | + +## Metadata configuration + +GitLab provides metadata XML that can be used to configure your Identity Provider. + +1. Navigate to the group and click **Settings > SAML SSO**. +1. Copy the provided **GitLab metadata URL** +1. Follow your Identity Provider's documentation and paste the metadata URL when it is requested. ## Configuring GitLab @@ -75,6 +87,23 @@ Once you've set up your identity provider to work with GitLab, you'll need to co | OneLogin | [Use the OneLogin SAML Test Connector](https://onelogin.service-now.com/support?id=kb_article&sys_id=93f95543db109700d5505eea4b96198f) | | Ping Identity | [Add and configure a new SAML application](https://docs.pingidentity.com/bundle/p1_enterpriseConfigSsoSaml_cas/page/enableAppWithoutURL.html) | +## Linking SAML to your existing GitLab.com account + +To link SAML to your existing GitLab.com account: + +1. Sign in to your GitLab.com account. +1. Locate the SSO URL for the group you are signing in to. A group Admin can find this on the group's **Settings > SAML SSO** page. +1. Visit the SSO URL and click **Authorize**. +1. Enter your credentials on the Identity Provider if prompted. +1. You will be redirected back to GitLab.com and should now have access to the group. In the future, you can use SAML to sign in to GitLab.com. + +## Signing in to GitLab.com with SAML + +1. Locate the SSO URL for the group you are signing in to. A group Admin can find this on a group's **Settings > SAML SSO** page. If configured, it might also be possible to sign in to GitLab starting from your Identity Provider. +1. Visit the SSO URL and click the **Sign in with Single Sign-On** button. +1. Enter your credentials on the Identity Provider if prompted. +1. You will be signed in to GitLab.com and redirected to the group. + ## Unlinking accounts Users can unlink SAML for a group from their profile page. This can be helpful if: @@ -97,3 +126,15 @@ For example, to unlink the `MyOrg` account, the following **Disconnect** button | Assertion consumer service URL | The callback on GitLab where users will be redirected after successfully authenticating with the identity provider. | | Issuer | How GitLab identifies itself to the identity provider. Also known as a "Relying party trust identifier". | | Certificate fingerprint | Used to confirm that communications over SAML are secure by checking that the server is signing communications with the correct certificate. Also known as a certificate thumbprint. | + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/group/saml_sso/scim_setup.md b/doc/user/group/saml_sso/scim_setup.md index ec27ec3e336..c00628bf909 100644 --- a/doc/user/group/saml_sso/scim_setup.md +++ b/doc/user/group/saml_sso/scim_setup.md @@ -2,7 +2,7 @@ > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/9388) in [GitLab.com Silver](https://about.gitlab.com/pricing/) 11.10. -GitLab's [SCIM API](https://docs.gitlab.com/ee/api/scim.html) implements part of [the RFC7644 protocol](https://tools.ietf.org/html/rfc7644). +GitLab's [SCIM API](../../../api/scim.md) implements part of [the RFC7644 protocol](https://tools.ietf.org/html/rfc7644). Currently, the following actions are available: diff --git a/doc/user/group/security_dashboard/index.md b/doc/user/group/security_dashboard/index.md index 43e910b29fe..c59198df081 100644 --- a/doc/user/group/security_dashboard/index.md +++ b/doc/user/group/security_dashboard/index.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/application_security/security_dashboard/index.html' +redirect_to: '../../application_security/security_dashboard/index.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/user/application_security/security_dashboard/index.html). +This document was moved to [another location](../../application_security/security_dashboard/index.md). diff --git a/doc/user/index.md b/doc/user/index.md index 67d5cbf06ec..1fc4e4c43cf 100644 --- a/doc/user/index.md +++ b/doc/user/index.md @@ -49,22 +49,22 @@ GitLab is a Git-based platform that integrates a great number of essential tools With GitLab Enterprise Edition, you can also: -- Provide support with [Service Desk](https://docs.gitlab.com/ee/user/project/service_desk.html). +- Provide support with [Service Desk](project/service_desk.md). - Improve collaboration with - [Merge Request Approvals](https://docs.gitlab.com/ee/user/project/merge_requests/index.html#merge-request-approvals-starter), - [Multiple Assignees for Issues](https://docs.gitlab.com/ee/user/project/issues/multiple_assignees_for_issues.html), + [Merge Request Approvals](project/merge_requests/index.md#merge-request-approvals-starter), + [Multiple Assignees for Issues](project/issues/multiple_assignees_for_issues.md), and [Multiple Issue Boards](project/issue_board.md#multiple-issue-boards-starter). -- Create formal relationships between issues with [Related Issues](https://docs.gitlab.com/ee/user/project/issues/related_issues.html). -- Use [Burndown Charts](https://docs.gitlab.com/ee/user/project/milestones/burndown_charts.html) to track progress during a sprint or while working on a new version of their software. -- Leverage [Elasticsearch](https://docs.gitlab.com/ee/integration/elasticsearch.html) with [Advanced Global Search](search/advanced_global_search.md) and [Advanced Syntax Search](search/advanced_search_syntax.md) for faster, more advanced code search across your entire GitLab instance. -- [Authenticate users with Kerberos](https://docs.gitlab.com/ee/integration/kerberos.html). +- Create formal relationships between issues with [Related Issues](project/issues/related_issues.md). +- Use [Burndown Charts](project/milestones/burndown_charts.md) to track progress during a sprint or while working on a new version of their software. +- Leverage [Elasticsearch](../integration/elasticsearch.md) with [Advanced Global Search](search/advanced_global_search.md) and [Advanced Syntax Search](search/advanced_search_syntax.md) for faster, more advanced code search across your entire GitLab instance. +- [Authenticate users with Kerberos](../integration/kerberos.md). - [Mirror a repository](../workflow/repository_mirroring.md) from elsewhere on your local server. -- [Export issues as CSV](https://docs.gitlab.com/ee/user/project/issues/csv_export.html). -- View your entire CI/CD pipeline involving more than one project with [Multiple-Project Pipelines](https://docs.gitlab.com/ee/ci/multi_project_pipeline_graphs.html). -- [Lock files](https://docs.gitlab.com/ee/user/project/file_lock.html) to prevent conflicts. -- View the current health and status of each CI environment running on Kubernetes with [Deploy Boards](https://docs.gitlab.com/ee/user/project/deploy_boards.html). -- Leverage continuous delivery method with [Canary Deployments](https://docs.gitlab.com/ee/user/project/canary_deployments.html). -- Scan your code for vulnerabilities and [display them in merge requests](https://docs.gitlab.com/ee/user/application_security/sast/index.html). +- [Export issues as CSV](project/issues/csv_export.md). +- View your entire CI/CD pipeline involving more than one project with [Multiple-Project Pipelines](../ci/multi_project_pipeline_graphs.md). +- [Lock files](project/file_lock.md) to prevent conflicts. +- View the current health and status of each CI environment running on Kubernetes with [Deploy Boards](project/deploy_boards.md). +- Leverage continuous delivery method with [Canary Deployments](project/canary_deployments.md). +- Scan your code for vulnerabilities and [display them in merge requests](application_security/sast/index.md). You can also [integrate](project/integrations/project_services.md) GitLab with numerous third-party applications, such as Mattermost, Microsoft Teams, HipChat, Trello, Slack, Bamboo CI, JIRA, and a lot more. diff --git a/doc/user/instance/clusters/index.md b/doc/user/instance/clusters/index.md new file mode 100644 index 00000000000..894f83d3c75 --- /dev/null +++ b/doc/user/instance/clusters/index.md @@ -0,0 +1,23 @@ +# Instance-level Kubernetes clusters + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/39840) in GitLab 11.11. +> Instance-level cluster integration is currently in [Beta](https://about.gitlab.com/handbook/product/#alpha-beta-ga). + +## Overview + +Similar to [project-level](../../project/clusters/index.md) +and [group-level](../../group/clusters/index.md) Kubernetes clusters, +instance-level Kubernetes clusters allow you to connect a Kubernetes cluster to +the GitLab instance, which enables you to use the same cluster across multiple +projects. + +## Cluster precedence + +GitLab will try match to clusters in the following order: + +- Project-level clusters +- Group-level clusters +- Instance level + +To be selected, the cluster must be enabled and +match the [environment selector](../../../ci/environments.md#scoping-environments-with-specs-premium). diff --git a/doc/user/markdown.md b/doc/user/markdown.md index 5dad9621802..6b6e5ab7634 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -1027,7 +1027,7 @@ A link can be constructed relative to the current wiki page using `./<page>`, it would link to `<your_wiki>/documentation/related`: ```markdown - [Link to Related Page](./related) + [Link to Related Page](related) ``` - If this snippet was placed on a page at `<your_wiki>/documentation/related/content`, @@ -1041,7 +1041,7 @@ A link can be constructed relative to the current wiki page using `./<page>`, it would link to `<your_wiki>/documentation/related.md`: ```markdown - [Link to Related Page](./related.md) + [Link to Related Page](related.md) ``` - If this snippet was placed on a page at `<your_wiki>/documentation/related/content`, diff --git a/doc/user/permissions.md b/doc/user/permissions.md index 318053fdabb..a6e2f187090 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -36,91 +36,96 @@ In GitLab 11.0, the Master role was renamed to Maintainer. The following table depicts the various user permission levels in a project. -| Action | Guest | Reporter | Developer |Maintainer| Owner | -|---------------------------------------|---------|------------|-------------|----------|--------| -| Create new issue | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | -| Create confidential issue | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | -| View confidential issues | (✓) [^2] | ✓ | ✓ | ✓ | ✓ | -| Leave comments | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | -| See related issues | ✓ | ✓ | ✓ | ✓ | ✓ | -| See a list of jobs | ✓ [^3] | ✓ | ✓ | ✓ | ✓ | -| See a job log | ✓ [^3] | ✓ | ✓ | ✓ | ✓ | -| Download and browse job artifacts | ✓ [^3] | ✓ | ✓ | ✓ | ✓ | -| View wiki pages | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | -| Create and edit wiki pages | | | ✓ | ✓ | ✓ | -| Delete wiki pages | | | | ✓ | ✓ | -| View license management reports **[ULTIMATE]** | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | -| View Security reports **[ULTIMATE]** | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | -| View project code | [^1] | ✓ | ✓ | ✓ | ✓ | -| Pull project code | [^1] | ✓ | ✓ | ✓ | ✓ | -| Download project | [^1] | ✓ | ✓ | ✓ | ✓ | -| Assign issues | | ✓ | ✓ | ✓ | ✓ | -| Assign merge requests | | | ✓ | ✓ | ✓ | -| Label issues | | ✓ | ✓ | ✓ | ✓ | -| Label merge requests | | | ✓ | ✓ | ✓ | -| Create code snippets | | ✓ | ✓ | ✓ | ✓ | -| Manage issue tracker | | ✓ | ✓ | ✓ | ✓ | -| Manage labels | | ✓ | ✓ | ✓ | ✓ | -| See a commit status | | ✓ | ✓ | ✓ | ✓ | -| See a container registry | | ✓ | ✓ | ✓ | ✓ | -| See environments | | ✓ | ✓ | ✓ | ✓ | -| See a list of merge requests | | ✓ | ✓ | ✓ | ✓ | -| Manage related issues **[STARTER]** | | ✓ | ✓ | ✓ | ✓ | -| Lock issue discussions | | ✓ | ✓ | ✓ | ✓ | -| Create issue from vulnerability **[ULTIMATE]** | | ✓ | ✓ | ✓ | ✓ | -| View Error Tracking list | | ✓ | ✓ | ✓ | ✓ | -| Pull from [Maven repository](https://docs.gitlab.com/ee/user/project/packages/maven_repository.html) or [NPM registry](https://docs.gitlab.com/ee/user/project/packages/npm_registry.html) **[PREMIUM]** | | ✓ | ✓ | ✓ | ✓ | -| Publish to [Maven repository](https://docs.gitlab.com/ee/user/project/packages/maven_repository.html) or [NPM registry](https://docs.gitlab.com/ee/user/project/packages/npm_registry.html) **[PREMIUM]** | | | ✓ | ✓ | ✓ | -| Lock merge request discussions | | | ✓ | ✓ | ✓ | -| Create new environments | | | ✓ | ✓ | ✓ | -| Stop environments | | | ✓ | ✓ | ✓ | -| Manage/Accept merge requests | | | ✓ | ✓ | ✓ | -| Create new merge request | | | ✓ | ✓ | ✓ | -| Create new branches | | | ✓ | ✓ | ✓ | -| Push to non-protected branches | | | ✓ | ✓ | ✓ | -| Force push to non-protected branches | | | ✓ | ✓ | ✓ | -| Remove non-protected branches | | | ✓ | ✓ | ✓ | -| Add tags | | | ✓ | ✓ | ✓ | -| Cancel and retry jobs | | | ✓ | ✓ | ✓ | -| Create or update commit status | | | ✓ | ✓ | ✓ | -| Update a container registry | | | ✓ | ✓ | ✓ | -| Remove a container registry image | | | ✓ | ✓ | ✓ | -| Create/edit/delete project milestones | | | ✓ | ✓ | ✓ | +| Action | Guest | Reporter | Developer |Maintainer| Owner | +|---------------------------------------------------|---------|------------|-------------|----------|--------| +| Download project | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| Leave comments | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| View Insights charts **[ULTIMATE]** | ✓ | ✓ | ✓ | ✓ | ✓ | | View approved/blacklisted licenses **[ULTIMATE]** | ✓ | ✓ | ✓ | ✓ | ✓ | -| Use security dashboard **[ULTIMATE]** | | | ✓ | ✓ | ✓ | -| Dismiss vulnerability **[ULTIMATE]** | | | ✓ | ✓ | ✓ | -| Apply code change suggestions | | | ✓ | ✓ | ✓ | -| Use environment terminals | | | | ✓ | ✓ | -| Run Web IDE's Interactive Web Terminals **[ULTIMATE ONLY]** | | | | ✓ | ✓ | -| Add new team members | | | | ✓ | ✓ | -| Push to protected branches | | | | ✓ | ✓ | -| Enable/disable branch protection | | | | ✓ | ✓ | -| Turn on/off protected branch push for devs| | | | ✓ | ✓ | -| Enable/disable tag protections | | | | ✓ | ✓ | -| Rewrite/remove Git tags | | | | ✓ | ✓ | -| Edit project | | | | ✓ | ✓ | -| Add deploy keys to project | | | | ✓ | ✓ | -| Configure project hooks | | | | ✓ | ✓ | -| Manage Runners | | | | ✓ | ✓ | -| Manage job triggers | | | | ✓ | ✓ | -| Manage variables | | | | ✓ | ✓ | -| Manage GitLab Pages | | | | ✓ | ✓ | -| Manage GitLab Pages domains and certificates | | | | ✓ | ✓ | -| Remove GitLab Pages | | | | ✓ | ✓ | +| View license management reports **[ULTIMATE]** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| View Security reports **[ULTIMATE]** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| View project code | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| Pull project code | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | View GitLab Pages protected by [access control](project/pages/introduction.md#gitlab-pages-access-control-core-only) | ✓ | ✓ | ✓ | ✓ | ✓ | -| Manage clusters | | | | ✓ | ✓ | -| Manage license policy **[ULTIMATE]** | | | | ✓ | ✓ | -| Edit comments (posted by any user) | | | | ✓ | ✓ | -| Manage Error Tracking | | | | ✓ | ✓ | -| Switch visibility level | | | | | ✓ | -| Transfer project to another namespace | | | | | ✓ | -| Remove project | | | | | ✓ | -| Delete issues | | | | | ✓ | -| Force push to protected branches [^4] | | | | | | -| Remove protected branches [^4] | | | | | | -| View project Audit Events | | | | ✓ | ✓ | -| View project statistics | | ✓ | ✓ | ✓ | ✓ | -| View Insights charts **[ULTIMATE]** | ✓ | ✓ | ✓ | ✓ | ✓ | +| View wiki pages | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| See a list of jobs | ✓ (*3*) | ✓ | ✓ | ✓ | ✓ | +| See a job log | ✓ (*3*) | ✓ | ✓ | ✓ | ✓ | +| Download and browse job artifacts | ✓ (*3*) | ✓ | ✓ | ✓ | ✓ | +| Create new issue | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| See related issues | ✓ | ✓ | ✓ | ✓ | ✓ | +| Create confidential issue | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| View confidential issues | (*2*) | ✓ | ✓ | ✓ | ✓ | +| Assign issues | | ✓ | ✓ | ✓ | ✓ | +| Label issues | | ✓ | ✓ | ✓ | ✓ | +| Lock issue discussions | | ✓ | ✓ | ✓ | ✓ | +| Manage issue tracker | | ✓ | ✓ | ✓ | ✓ | +| Manage related issues **[STARTER]** | | ✓ | ✓ | ✓ | ✓ | +| Create issue from vulnerability **[ULTIMATE]** | | ✓ | ✓ | ✓ | ✓ | +| Manage labels | | ✓ | ✓ | ✓ | ✓ | +| Create code snippets | | ✓ | ✓ | ✓ | ✓ | +| See a commit status | | ✓ | ✓ | ✓ | ✓ | +| See a container registry | | ✓ | ✓ | ✓ | ✓ | +| See environments | | ✓ | ✓ | ✓ | ✓ | +| See a list of merge requests | | ✓ | ✓ | ✓ | ✓ | +| View project statistics | | ✓ | ✓ | ✓ | ✓ | +| View Error Tracking list | | ✓ | ✓ | ✓ | ✓ | +| Pull from [Maven repository](project/packages/maven_repository.md) or [NPM registry](project/packages/npm_registry.md) **[PREMIUM]** | | ✓ | ✓ | ✓ | ✓ | +| Publish to [Maven repository](project/packages/maven_repository.md) or [NPM registry](project/packages/npm_registry.md) **[PREMIUM]** | | | ✓ | ✓ | ✓ || +| Create new branches | | | ✓ | ✓ | ✓ | +| Push to non-protected branches | | | ✓ | ✓ | ✓ | +| Force push to non-protected branches | | | ✓ | ✓ | ✓ | +| Remove non-protected branches | | | ✓ | ✓ | ✓ | +| Create new merge request | | | ✓ | ✓ | ✓ | +| Assign merge requests | | | ✓ | ✓ | ✓ | +| Label merge requests | | | ✓ | ✓ | ✓ | +| Lock merge request discussions | | | ✓ | ✓ | ✓ | +| Manage/Accept merge requests | | | ✓ | ✓ | ✓ | +| Create new environments | | | ✓ | ✓ | ✓ | +| Stop environments | | | ✓ | ✓ | ✓ | +| Add tags | | | ✓ | ✓ | ✓ | +| Cancel and retry jobs | | | ✓ | ✓ | ✓ | +| Create or update commit status | | | ✓ | ✓ | ✓ | +| Update a container registry | | | ✓ | ✓ | ✓ | +| Remove a container registry image | | | ✓ | ✓ | ✓ | +| Create/edit/delete project milestones | | | ✓ | ✓ | ✓ | +| Use security dashboard **[ULTIMATE]** | | | ✓ | ✓ | ✓ | +| Dismiss vulnerability **[ULTIMATE]** | | | ✓ | ✓ | ✓ | +| Apply code change suggestions | | | ✓ | ✓ | ✓ | +| Create and edit wiki pages | | | ✓ | ✓ | ✓ | +| Use environment terminals | | | | ✓ | ✓ | +| Run Web IDE's Interactive Web Terminals **[ULTIMATE ONLY]** | | | | ✓ | ✓ | +| Add new team members | | | | ✓ | ✓ | +| Enable/disable branch protection | | | | ✓ | ✓ | +| Push to protected branches | | | | ✓ | ✓ | +| Turn on/off protected branch push for devs | | | | ✓ | ✓ | +| Enable/disable tag protections | | | | ✓ | ✓ | +| Rewrite/remove Git tags | | | | ✓ | ✓ | +| Edit project | | | | ✓ | ✓ | +| Add deploy keys to project | | | | ✓ | ✓ | +| Configure project hooks | | | | ✓ | ✓ | +| Manage Runners | | | | ✓ | ✓ | +| Manage job triggers | | | | ✓ | ✓ | +| Manage variables | | | | ✓ | ✓ | +| Manage GitLab Pages | | | | ✓ | ✓ | +| Manage GitLab Pages domains and certificates | | | | ✓ | ✓ | +| Remove GitLab Pages | | | | ✓ | ✓ | +| Manage clusters | | | | ✓ | ✓ | +| Manage license policy **[ULTIMATE]** | | | | ✓ | ✓ | +| Edit comments (posted by any user) | | | | ✓ | ✓ | +| Manage Error Tracking | | | | ✓ | ✓ | +| Delete wiki pages | | | | ✓ | ✓ | +| View project Audit Events | | | | ✓ | ✓ | +| Switch visibility level | | | | | ✓ | +| Transfer project to another namespace | | | | | ✓ | +| Remove project | | | | | ✓ | +| Delete issues | | | | | ✓ | +| Force push to protected branches [^4] | | | | | | +| Remove protected branches [^4] | | | | | | + +- (*1*): All users are able to perform this action on public and internal projects, but not private projects. +- (*2*): Guest users can only view the confidential issues they created themselves +- (*3*): If **Public pipelines** is enabled in **Project Settings > CI/CD** +- (*4*): Not allowed for Guest, Reporter, Developer, Maintainer, or Owner ## Project features permissions @@ -163,7 +168,7 @@ to learn more. The user that locks a file or directory is the only one that can edit and push their changes back to the repository where the locked objects are located. -Read through the documentation on [permissions for File Locking](https://docs.gitlab.com/ee/user/project/file_lock.html#permissions-on-file-locking) to learn more. +Read through the documentation on [permissions for File Locking](project/file_lock.md#permissions-on-file-locking) to learn more. ### Confidential Issues permissions @@ -191,21 +196,21 @@ Any user can remove themselves from a group, unless they are the last Owner of the group. The following table depicts the various user permission levels in a group. -| Action | Guest | Reporter | Developer | Maintainer | Owner | -|-------------------------|-------|----------|-----------|--------|-------| -| Browse group | ✓ | ✓ | ✓ | ✓ | ✓ | -| Edit group | | | | | ✓ | -| Create subgroup | | | | | ✓ | -| Create project in group | | | ✓ | ✓ | ✓ | -| Manage group members | | | | | ✓ | -| Remove group | | | | | ✓ | -| Manage group labels | | ✓ | ✓ | ✓ | ✓ | -| Create/edit/delete group milestones | | | ✓ | ✓ | ✓ | -| View group epic **[ULTIMATE]** | ✓ | ✓ | ✓ | ✓ | ✓ | -| Create/edit group epic **[ULTIMATE]** | | ✓ | ✓ | ✓ | ✓ | -| Delete group epic **[ULTIMATE]** | | | | | ✓ | -| View group Audit Events | | | | | ✓ | -| View Insights charts **[ULTIMATE]** | ✓ | ✓ | ✓ | ✓ | ✓ | +| Action | Guest | Reporter | Developer | Maintainer | Owner | +|---------------------------------------|-------|----------|-----------|------------|-------| +| Browse group | ✓ | ✓ | ✓ | ✓ | ✓ | +| View Insights charts **[ULTIMATE]** | ✓ | ✓ | ✓ | ✓ | ✓ | +| View group epic **[ULTIMATE]** | ✓ | ✓ | ✓ | ✓ | ✓ | +| Create/edit group epic **[ULTIMATE]** | | ✓ | ✓ | ✓ | ✓ | +| Manage group labels | | ✓ | ✓ | ✓ | ✓ | +| Create project in group | | | ✓ | ✓ | ✓ | +| Create/edit/delete group milestones | | | ✓ | ✓ | ✓ | +| Edit group | | | | | ✓ | +| Create subgroup | | | | | ✓ | +| Manage group members | | | | | ✓ | +| Remove group | | | | | ✓ | +| Delete group epic **[ULTIMATE]** | | | | | ✓ | +| View group Audit Events | | | | | ✓ | ### Subgroup permissions @@ -257,15 +262,15 @@ Please be aware that this regex could lead to a DOS attack, [see](https://en.wik ## Auditor users **[PREMIUM ONLY]** ->[Introduced][ee-998] in [GitLab Premium][eep] 8.17. +>[Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/998) in [GitLab Premium](https://about.gitlab.com/pricing/) 8.17. Auditor users are given read-only access to all projects, groups, and other resources on the GitLab instance. An Auditor user should be able to access all projects and groups of a GitLab instance -with the permissions described on the documentation on [auditor users permissions](https://docs.gitlab.com/ee/administration/auditor_users.html#permissions-and-restrictions-of-an-auditor-user). +with the permissions described on the documentation on [auditor users permissions](../administration/auditor_users.md#permissions-and-restrictions-of-an-auditor-user). -[Read more about Auditor users.](https://docs.gitlab.com/ee/administration/auditor_users.html) +[Read more about Auditor users.](../administration/auditor_users.md) ## Project features @@ -298,7 +303,7 @@ instance and project. In addition, all admins can use the admin interface under |---------------------------------------|-----------------|-------------|----------|--------| | See commits and jobs | ✓ | ✓ | ✓ | ✓ | | Retry or cancel job | | ✓ | ✓ | ✓ | -| Erase job artifacts and trace | | ✓ [^5] | ✓ | ✓ | +| Erase job artifacts and trace | | ✓ (*1*) | ✓ | ✓ | | Remove project | | | ✓ | ✓ | | Create project | | | ✓ | ✓ | | Change project configuration | | | ✓ | ✓ | @@ -307,6 +312,8 @@ instance and project. In addition, all admins can use the admin interface under | See events in the system | | | | ✓ | | Admin interface | | | | ✓ | +- *1*: Only if the job was triggered by the user + ### Job permissions NOTE: **Note:** @@ -314,25 +321,28 @@ In GitLab 11.0, the Master role was renamed to Maintainer. >**Note:** GitLab 8.12 has a completely redesigned job permissions system. -Read all about the [new model and its implications][new-mod]. +Read all about the [new model and its implications](project/new_ci_build_permissions_model.md). This table shows granted privileges for jobs triggered by specific types of users: -| Action | Guest, Reporter | Developer |Maintainer| Admin | -|---------------------------------------------|-----------------|-------------|----------|--------| -| Run CI job | | ✓ | ✓ | ✓ | -| Clone source and LFS from current project | | ✓ | ✓ | ✓ | -| Clone source and LFS from public projects | | ✓ | ✓ | ✓ | -| Clone source and LFS from internal projects | | ✓ [^6] | ✓ [^6] | ✓ | -| Clone source and LFS from private projects | | ✓ [^7] | ✓ [^7] | ✓ [^7] | -| Push source and LFS | | | | | -| Pull container images from current project | | ✓ | ✓ | ✓ | -| Pull container images from public projects | | ✓ | ✓ | ✓ | -| Pull container images from internal projects| | ✓ [^6] | ✓ [^6] | ✓ | -| Pull container images from private projects | | ✓ [^7] | ✓ [^7] | ✓ [^7] | -| Push container images to current project | | ✓ | ✓ | ✓ | -| Push container images to other projects | | | | | +| Action | Guest, Reporter | Developer |Maintainer| Admin | +|---------------------------------------------|-----------------|-------------|----------|---------| +| Run CI job | | ✓ | ✓ | ✓ | +| Clone source and LFS from current project | | ✓ | ✓ | ✓ | +| Clone source and LFS from public projects | | ✓ | ✓ | ✓ | +| Clone source and LFS from internal projects | | ✓ (*1*) | ✓ (*1*) | ✓ | +| Clone source and LFS from private projects | | ✓ (*2*) | ✓ (*2*) | ✓ (*2*) | +| Pull container images from current project | | ✓ | ✓ | ✓ | +| Pull container images from public projects | | ✓ | ✓ | ✓ | +| Pull container images from internal projects| | ✓ (*1*) | ✓ (*1*) | ✓ | +| Pull container images from private projects | | ✓ (*2*) | ✓ (*2*) | ✓ (*2*) | +| Push container images to current project | | ✓ | ✓ | ✓ | +| Push container images to other projects | | | | | +| Push source and LFS | | | | | + +- *1*: Only if the user is not an external one +- *2*: Only if the user is a member of the project ### New CI job permissions model @@ -350,17 +360,4 @@ for details about the pipelines security model. ## LDAP users permissions Since GitLab 8.15, LDAP user permissions can now be manually overridden by an admin user. -Read through the documentation on [LDAP users permissions](https://docs.gitlab.com/ee/administration/auth/how_to_configure_ldap_gitlab_ee/index.html) to learn more. - -[^1]: On public and internal projects, all users are able to perform this action -[^2]: Guest users can only view the confidential issues they created themselves -[^3]: If **Public pipelines** is enabled in **Project Settings > CI/CD** -[^4]: Not allowed for Guest, Reporter, Developer, Maintainer, or Owner -[^5]: Only if the job was triggered by the user -[^6]: Only if user is not external one -[^7]: Only if user is a member of the project - -[ce-18994]: https://gitlab.com/gitlab-org/gitlab-ce/issues/18994 -[new-mod]: project/new_ci_build_permissions_model.md -[ee-998]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/998 -[eep]: https://about.gitlab.com/pricing/ +Read through the documentation on [LDAP users permissions](../administration/auth/how_to_configure_ldap_gitlab_ee/index.html) to learn more. diff --git a/doc/user/profile/personal_access_tokens.md b/doc/user/profile/personal_access_tokens.md index 4085f3b678c..0b224fc7e01 100644 --- a/doc/user/profile/personal_access_tokens.md +++ b/doc/user/profile/personal_access_tokens.md @@ -37,19 +37,18 @@ Personal access tokens can be created with one or more scopes that allow various actions that a given token can perform. The available scopes are depicted in the following table. -| Scope | Description | -| ----- | ----------- | -|`read_user` | Allows access to the read-only endpoints under `/users`. Essentially, any of the `GET` requests in the [Users API][users] are allowed ([introduced][ce-5951] in GitLab 8.15). | -| `api` | Grants complete access to the API and Container Registry (read/write) ([introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5951) in GitLab 8.15). | -| `read_registry` | Allows to read (pull) [container registry] images if a project is private and authorization is required ([introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/11845) in GitLab 9.3). | -| `sudo` | Allows performing API actions as any user in the system (if the authenticated user is an admin) ([introduced][ce-14838] in GitLab 10.2). | -| `read_repository` | Allows read-only access (pull) to the repository through git clone. | -| `write_repository` | Allows read-write access (pull, push) to the repository through git clone. Required for accessing Git repositories over HTTP when 2FA is enabled. | +| Scope | Introduced in | Description | +| ------------------ | ------------- | ----------- | +| `read_user` | [GitLab 8.15](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5951) | Allows access to the read-only endpoints under `/users`. Essentially, any of the `GET` requests in the [Users API][users] are allowed. | +| `api` | [GitLab 8.15](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5951) | Grants complete access to the API and Container Registry (read/write). | +| `read_registry` | [GitLab 9.3](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/11845) | Allows to read (pull) [container registry] images if a project is private and authorization is required. | +| `sudo` | [GitLab 10.2](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14838) | Allows performing API actions as any user in the system (if the authenticated user is an admin). | +| `read_repository` | [GitLab 10.7](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/17894) | Allows read-only access (pull) to the repository through git clone. | +| `write_repository` | [GitLab 11.11](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/26021) | Allows read-write access (pull, push) to the repository through git clone. Required for accessing Git repositories over HTTP when 2FA is enabled. | [2fa]: ../account/two_factor_authentication.md [api]: ../../api/README.md [ce-3749]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3749 -[ce-14838]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14838 [container registry]: ../project/container_registry.md [users]: ../../api/users.md [usage]: ../../api/README.md#personal-access-tokens diff --git a/doc/user/project/ci_cd_for_external_repo.md b/doc/user/project/ci_cd_for_external_repo.md index 51b86a68c7b..a92d3a2c308 100644 --- a/doc/user/project/ci_cd_for_external_repo.md +++ b/doc/user/project/ci_cd_for_external_repo.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/index.html' +redirect_to: '../../ci/ci_cd_for_external_repos/index.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/index.html). +This document was moved to [another location](../../ci/ci_cd_for_external_repos/index.md). diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index afad8f0fa9d..dc21db603d6 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -19,8 +19,10 @@ or provide the credentials to an [existing Kubernetes cluster](#adding-an-existi NOTE: **Note:** From [GitLab 11.6](https://gitlab.com/gitlab-org/gitlab-ce/issues/34758) you -can also associate a Kubernetes cluster to your groups. Learn more about -[group Kubernetes clusters](../../group/clusters/index.md). +can also associate a Kubernetes cluster to your groups and from +[GitLab 11.11](https://gitlab.com/gitlab-org/gitlab-ce/issues/39840), +to the GitLab instance. Learn more about [group-level](../../group/clusters/index.md) +and [instance-level](../../instance/clusters/index.md) Kubernetes clusters. ## Adding and creating a new GKE cluster via GitLab @@ -69,7 +71,7 @@ new Kubernetes cluster to your project: - **Number of nodes** - Enter the number of nodes you wish the cluster to have. - **Machine type** - The [machine type](https://cloud.google.com/compute/docs/machine-types) of the Virtual Machine instance that the cluster will be based on. - - **RBAC-enabled cluster** - Leave this checked if using default GKE creation options, see the [RBAC section](#role-based-access-control-rbac) for more information. + - **RBAC-enabled cluster** - Leave this checked if using default GKE creation options, see the [RBAC section](#rbac-cluster-resources) for more information. - **GitLab-managed cluster** - Leave this checked if you want GitLab to manage namespaces and service accounts for this cluster. See the [Managed clusters section](#gitlab-managed-clusters) for more information. 1. Finally, click the **Create Kubernetes cluster** button. @@ -261,65 +263,66 @@ you can either: ## Access controls -When creating a cluster in GitLab, you will be asked if you would like to create an -[Attribute-based access control (ABAC)](https://kubernetes.io/docs/admin/authorization/abac/) cluster, or -a [Role-based access control (RBAC)](https://kubernetes.io/docs/admin/authorization/rbac/) one. +When creating a cluster in GitLab, you will be asked if you would like to create either: -NOTE: **Note:** -[RBAC](#role-based-access-control-rbac) is recommended and the GitLab default. +- An [Attribute-based access control (ABAC)](https://kubernetes.io/docs/admin/authorization/abac/) cluster. +- A [Role-based access control (RBAC)](https://kubernetes.io/docs/admin/authorization/rbac/) cluster. -Whether [ABAC](#attribute-based-access-control-abac) or [RBAC](#role-based-access-control-rbac) is enabled, -GitLab will create the necessary service accounts and privileges in order to install and run -[GitLab managed applications](#installing-applications): +NOTE: **Note:** +[RBAC](#rbac-cluster-resources) is recommended and the GitLab default. -- If GitLab is creating the cluster, a `gitlab` service account with - `cluster-admin` privileges will be created in the `default` namespace, - which will be used by GitLab to manage the newly created cluster. +GitLab creates the necessary service accounts and privileges to install and run +[GitLab managed applications](#installing-applications). When GitLab creates the cluster: +- A `gitlab` service account with `cluster-admin` privileges is created in the `default` namespace + to manage the newly created cluster. - A project service account with [`edit` privileges](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) - will be created in the project namespace (also created by GitLab), which will - be used in [deployment jobs](#deployment-variables). + is created in the GitLab-created project namespace for [deployment jobs](#deployment-variables). NOTE: **Note:** Restricted service account for deployment was [introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/51716) in GitLab 11.5. -- When you install Helm into your cluster, the `tiller` service account - will be created with `cluster-admin` privileges in the `gitlab-managed-apps` - namespace. This service account will be added to the installed Helm Tiller and will - be used by Helm to install and run [GitLab managed applications](#installing-applications). - Helm will also create additional service accounts and other resources for each - installed application. Consult the documentation of the Helm charts for each application - for details. +When you install Helm into your cluster, the `tiller` service account +is created with `cluster-admin` privileges in the `gitlab-managed-apps` +namespace. This service account will be added to the installed Helm Tiller and will +be used by Helm to install and run [GitLab managed applications](#installing-applications). +Helm will also create additional service accounts and other resources for each +installed application. Consult the documentation of the Helm charts for each application +for details. If you are [adding an existing Kubernetes cluster](#adding-an-existing-kubernetes-cluster), ensure the token of the account has administrator privileges for the cluster. -The following sections summarize which resources will be created on ABAC/RBAC clusters. +The resources created by GitLab differ depending on the type of cluster. + +### ABAC cluster resources -### Attribute-based access control (ABAC) +GitLab creates the following resources for ABAC clusters. -| Name | Kind | Details | Created when | -| --- | --- | --- | --- | -| `gitlab` | `ServiceAccount` | `default` namespace | Creating a new GKE Cluster | -| `gitlab-token` | `Secret` | Token for `gitlab` ServiceAccount | Creating a new GKE Cluster | -| `tiller` | `ServiceAccount` | `gitlab-managed-apps` namespace | Installing Helm Tiller | -| `tiller-admin` | `ClusterRoleBinding` | `cluster-admin` roleRef | Installing Helm Tiller | -| Project namespace | `ServiceAccount` | Uses namespace of Project | Deploying to a cluster | -| Project namespace | `Secret` | Token for project ServiceAccount | Deploying to a cluster | +| Name | Type | Details | Created when | +|:------------------|:---------------------|:----------------------------------|:---------------------------| +| `gitlab` | `ServiceAccount` | `default` namespace | Creating a new GKE Cluster | +| `gitlab-token` | `Secret` | Token for `gitlab` ServiceAccount | Creating a new GKE Cluster | +| `tiller` | `ServiceAccount` | `gitlab-managed-apps` namespace | Installing Helm Tiller | +| `tiller-admin` | `ClusterRoleBinding` | `cluster-admin` roleRef | Installing Helm Tiller | +| Project namespace | `ServiceAccount` | Uses namespace of Project | Deploying to a cluster | +| Project namespace | `Secret` | Token for project ServiceAccount | Deploying to a cluster | -### Role-based access control (RBAC) +### RBAC cluster resources -| Name | Kind | Details | Created when | -| --- | --- | --- | --- | -| `gitlab` | `ServiceAccount` | `default` namespace | Creating a new GKE Cluster | -| `gitlab-admin` | `ClusterRoleBinding` | [`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) roleRef | Creating a new GKE Cluster | -| `gitlab-token` | `Secret` | Token for `gitlab` ServiceAccount | Creating a new GKE Cluster | -| `tiller` | `ServiceAccount` | `gitlab-managed-apps` namespace | Installing Helm Tiller | -| `tiller-admin` | `ClusterRoleBinding` | `cluster-admin` roleRef | Installing Helm Tiller | -| Project namespace | `ServiceAccount` | Uses namespace of Project | Deploying to a cluster | -| Project namespace | `Secret` | Token for project ServiceAccount | Deploying to a cluster | -| Project namespace | `RoleBinding` | [`edit`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) roleRef | Deploying to a cluster | +GitLab creates the following resources for RBAC clusters. + +| Name | Type | Details | Created when | +|:------------------|:---------------------|:-----------------------------------------------------------------------------------------------------------|:---------------------------| +| `gitlab` | `ServiceAccount` | `default` namespace | Creating a new GKE Cluster | +| `gitlab-admin` | `ClusterRoleBinding` | [`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) roleRef | Creating a new GKE Cluster | +| `gitlab-token` | `Secret` | Token for `gitlab` ServiceAccount | Creating a new GKE Cluster | +| `tiller` | `ServiceAccount` | `gitlab-managed-apps` namespace | Installing Helm Tiller | +| `tiller-admin` | `ClusterRoleBinding` | `cluster-admin` roleRef | Installing Helm Tiller | +| Project namespace | `ServiceAccount` | Uses namespace of Project | Deploying to a cluster | +| Project namespace | `Secret` | Token for project ServiceAccount | Deploying to a cluster | +| Project namespace | `RoleBinding` | [`edit`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) roleRef | Deploying to a cluster | NOTE: **Note:** Project-specific resources are only created if your cluster is [managed by GitLab](#gitlab-managed-clusters). @@ -344,111 +347,10 @@ install it manually. ## Installing applications -GitLab provides **GitLab Managed Apps**, a one-click install for various applications which can -be added directly to your configured cluster. These applications are -needed for [Review Apps](../../../ci/review_apps/index.md) and -[deployments](../../../ci/environments.md) when using [Auto DevOps](../../../topics/autodevops/index.md). -You can install them after you -[create a cluster](#adding-and-creating-a-new-gke-cluster-via-gitlab). - -Applications managed by GitLab will be installed onto the `gitlab-managed-apps` namespace. This differrent -from the namespace used for project deployments. It is only created once and its name is not configurable. - -To see a list of available applications to install: - -1. Navigate to your project's **Operations > Kubernetes**. -1. Select your cluster. - -Install Helm first as it's used to install other applications. - -NOTE: **Note:** -As of GitLab 11.6, Helm will be upgraded to the latest version supported -by GitLab before installing any of the applications. - -| Application | GitLab version | Description | Helm Chart | -| ----------- | :------------: | ----------- | --------------- | -| [Helm](https://docs.helm.sh/) | 10.2+ | Helm is a package manager for Kubernetes and is required to install all the other applications. It is installed in its own pod inside the cluster which can run the `helm` CLI in a safe environment. | n/a | -| [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) | 10.2+ | Ingress can provide load balancing, SSL termination, and name-based virtual hosting. It acts as a web proxy for your applications and is useful if you want to use [Auto DevOps] or deploy your own web apps. | [stable/nginx-ingress](https://github.com/helm/charts/tree/master/stable/nginx-ingress) | -| [Cert-Manager](https://docs.cert-manager.io/en/latest/) | 11.6+ | Cert-Manager is a native Kubernetes certificate management controller that helps with issuing certificates. Installing Cert-Manager on your cluster will issue a certificate by [Let's Encrypt](https://letsencrypt.org/) and ensure that certificates are valid and up-to-date. | [stable/cert-manager](https://github.com/helm/charts/tree/master/stable/cert-manager) | -| [Prometheus](https://prometheus.io/docs/introduction/overview/) | 10.4+ | Prometheus is an open-source monitoring and alerting system useful to supervise your deployed applications. | [stable/prometheus](https://github.com/helm/charts/tree/master/stable/prometheus) | -| [GitLab Runner](https://docs.gitlab.com/runner/) | 10.6+ | GitLab Runner is the open source project that is used to run your jobs and send the results back to GitLab. It is used in conjunction with [GitLab CI/CD](../../../ci/README.md), the open-source continuous integration service included with GitLab that coordinates the jobs. When installing the GitLab Runner via the applications, it will run in **privileged mode** by default. Make sure you read the [security implications](#security-implications) before doing so. | [runner/gitlab-runner](https://gitlab.com/charts/gitlab-runner) | -| [JupyterHub](http://jupyter.org/) | 11.0+ | [JupyterHub](https://jupyterhub.readthedocs.io/en/stable/) is a multi-user service for managing notebooks across a team. [Jupyter Notebooks](https://jupyter-notebook.readthedocs.io/en/latest/) provide a web-based interactive programming environment used for data analysis, visualization, and machine learning. We use a [custom Jupyter image](https://gitlab.com/gitlab-org/jupyterhub-user-image/blob/master/Dockerfile) that installs additional useful packages on top of the base Jupyter. Authentication will be enabled only for [project members](../members/index.md) with [Developer or higher](../../permissions.md) access to the project. You will also see ready-to-use DevOps Runbooks built with Nurtch's [Rubix library](https://github.com/amit1rrr/rubix). More information on creating executable runbooks can be found in [our Nurtch documentation](runbooks/index.md#nurtch-executable-runbooks). Note that Ingress must be installed and have an IP address assigned before JupyterHub can be installed. | [jupyter/jupyterhub](https://jupyterhub.github.io/helm-chart/) | -| [Knative](https://cloud.google.com/knative) | 11.5+ | Knative provides a platform to create, deploy, and manage serverless workloads from a Kubernetes cluster. It is used in conjunction with, and includes [Istio](https://istio.io) to provide an external IP address for all programs hosted by Knative. You will be prompted to enter a wildcard domain where your applications will be exposed. Configure your DNS server to use the external IP address for that domain. For any application created and installed, they will be accessible as `<program_name>.<kubernetes_namespace>.<domain_name>`. This will require your kubernetes cluster to have [RBAC enabled](#role-based-access-control-rbac). | [knative/knative](https://storage.googleapis.com/triggermesh-charts) - -With the exception of Knative, the applications will be installed in a dedicated -namespace called `gitlab-managed-apps`. - -CAUTION: **Caution:** -If you have an existing Kubernetes cluster with Helm already installed, -you should be careful as GitLab cannot detect it. In this case, installing -Helm via the applications will result in the cluster having it twice, which -can lead to confusion during deployments. - -### Upgrading applications - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/24789) -in GitLab 11.8. - -Users can perform a one-click upgrade for the GitLab Runner application, -when there is an upgrade available. - -To upgrade the GitLab Runner application: - -1. Navigate to your project's **Operations > Kubernetes**. -1. Select your cluster. -1. Click the **Upgrade** button for the Runnner application. - -The **Upgrade** button will not be shown if there is no upgrade -available. - -NOTE: **Note:** -Upgrades will reset values back to the values built into the `runner` -chart plus the values set by -[`values.yaml`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/vendor/runner/values.yaml) - -### Uninstalling applications - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/60665) in -> GitLab 11.11. - -The applications below can be uninstalled. - -| Application | GitLab version | Notes | -| ----------- | -------------- | ----- | -| Prometheus | 11.11+ | All data will be deleted and cannot be restored. | - -To uninstall an application: - -1. Navigate to your project's **Operations > Kubernetes**. -1. Select your cluster. -1. Click the **Uninstall** button for the application. - -Support for uninstalling all applications will be progressively -introduced (see [related -epic](https://gitlab.com/groups/gitlab-org/-/epics/1201)). - -### Troubleshooting applications - -Applications can fail with the following error: - -```text -Error: remote error: tls: bad certificate -``` - -To avoid installation errors: - -- Before starting the installation of applications, make sure that time is synchronized - between your GitLab server and your Kubernetes cluster. -- Ensure certificates are not out of sync. When installing applications, GitLab expects a new cluster with no previous installation of Helm. - - You can confirm that the certificates match via `kubectl`: - - ```sh - kubectl get configmaps/values-content-configuration-ingress -n gitlab-managed-apps -o \ - "jsonpath={.data['cert\.pem']}" | base64 -d > a.pem - kubectl get secrets/tiller-secret -n gitlab-managed-apps -o "jsonpath={.data['ca\.crt']}" | base64 -d > b.pem - diff a.pem b.pem - ``` +GitLab can install and manage some applications in your project-level +cluster. For more information on installing, upgrading, uninstalling, +and troubleshooting applications for your project cluster, see +[Gitlab Managed Apps](../../clusters/applications.md). ## Getting the external endpoint @@ -548,7 +450,7 @@ differentiate the new cluster with the rest. When adding more than one Kubernetes cluster to your project, you need to differentiate them with an environment scope. The environment scope associates clusters with [environments](../../../ci/environments.md) similar to how the -[environment-specific variables](https://docs.gitlab.com/ee/ci/variables/index.html#limiting-environment-scopes-of-environment-variables-premium) work. +[environment-specific variables](../../../ci/variables/README.md#limiting-environment-scopes-of-environment-variables-premium) work. The default environment scope is `*`, which means all jobs, regardless of their environment, will use that cluster. Each scope can only be used by a single @@ -686,7 +588,7 @@ displaying the status of the pods in the deployment. Developers and other teammates can view the progress and status of a rollout, pod by pod, in the workflow they already use without any need to access Kubernetes. -[Read more about Deploy Boards](https://docs.gitlab.com/ee/user/project/deploy_boards.html) +[Read more about Deploy Boards](../deploy_boards.md) ### Canary Deployments **[PREMIUM]** @@ -694,7 +596,7 @@ Leverage [Kubernetes' Canary deployments](https://kubernetes.io/docs/concepts/cl and visualize your canary deployments right inside the Deploy Board, without the need to leave GitLab. -[Read more about Canary Deployments](https://docs.gitlab.com/ee/user/project/canary_deployments.html) +[Read more about Canary Deployments](../canary_deployments.md) ### Pod logs **[ULTIMATE]** diff --git a/doc/user/project/clusters/kubernetes_pod_logs.md b/doc/user/project/clusters/kubernetes_pod_logs.md index d5b60250860..368031070c1 100644 --- a/doc/user/project/clusters/kubernetes_pod_logs.md +++ b/doc/user/project/clusters/kubernetes_pod_logs.md @@ -7,10 +7,10 @@ By displaying the logs directly in GitLab, developers can avoid having to manage ## Overview -[Kubernetes](https://kubernetes.io) pod logs can be viewed directly within GitLab. Logs can be displayed by clicking on a specific pod from [Deploy Boards](https://docs.gitlab.com/ee/user/project/deploy_boards.html): +[Kubernetes](https://kubernetes.io) pod logs can be viewed directly within GitLab. Logs can be displayed by clicking on a specific pod from [Deploy Boards](../deploy_boards.md): 1. Go to **Operations > Environments** and find the environment which contains the desired pod, like `production`. -1. On the **Environments** page, you should see the status of the environment's pods with [Deploy Boards](https://docs.gitlab.com/ee/user/project/deploy_boards.html). +1. On the **Environments** page, you should see the status of the environment's pods with [Deploy Boards](../deploy_boards.md). 1. When mousing over the list of pods, a tooltip will appear with the exact pod name and status.  1. Click on the desired pod to bring up the logs view, which will contain the last 500 lines for that pod. Support for pods with multiple containers is coming [in a future release](https://gitlab.com/gitlab-org/gitlab-ee/issues/6502). @@ -18,4 +18,4 @@ By displaying the logs directly in GitLab, developers can avoid having to manage ## Requirements -[Enabling Deploy Boards](https://docs.gitlab.com/ee/user/project/deploy_boards.html#enabling-deploy-boards) is required in order to be able to use Pod Logs. +[Enabling Deploy Boards](../deploy_boards.md#enabling-deploy-boards) is required in order to be able to use Pod Logs. diff --git a/doc/user/project/clusters/runbooks/index.md b/doc/user/project/clusters/runbooks/index.md index 54c475a1762..c67b12fb91a 100644 --- a/doc/user/project/clusters/runbooks/index.md +++ b/doc/user/project/clusters/runbooks/index.md @@ -17,13 +17,15 @@ Modern implementations have introduced the concept of an "executable runbooks", where, along with a well-defined process, operators can execute pre-written code blocks or database queries against a given environment. -## Nurtch Executable Runbooks +## Executable Runbooks > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/45912) in GitLab 11.4. The JupyterHub app offered via GitLab’s Kubernetes integration now ships with Nurtch’s Rubix library, providing a simple way to create DevOps -runbooks. A sample runbook is provided, showcasing common operations. +runbooks. A sample runbook is provided, showcasing common operations. While Rubix makes it +simple to create common Kubernetes and AWS workflows, you can also create them manually without +Rubix. **<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Watch this [video](https://www.youtube.com/watch?v=Q_OqHIIUPjE) @@ -34,7 +36,7 @@ for an overview of how this is accomplished in GitLab!** To create an executable runbook, you will need: 1. **Kubernetes** - A Kubernetes cluster is required to deploy the rest of the applications. - The simplest way to get started is to add a cluster using [GitLab's GKE integration](https://docs.gitlab.com/ee/user/project/clusters/#adding-and-creating-a-new-gke-cluster-via-gitlab). + The simplest way to get started is to add a cluster using [GitLab's GKE integration](../index.md#adding-and-creating-a-new-gke-cluster-via-gitlab). 1. **Helm Tiller** - Helm is a package manager for Kubernetes and is required to install all the other applications. It is installed in its own pod inside the cluster which can run the helm CLI in a safe environment. @@ -59,7 +61,7 @@ the components outlined above and the preloaded demo runbook. ### 1. Add a Kubernetes cluster -Follow the steps outlined in [Adding and creating a new GKE cluster via GitLab](https://docs.gitlab.com/ee/user/project/clusters/#adding-and-creating-a-new-gke-cluster-via-gitlab) +Follow the steps outlined in [Adding and creating a new GKE cluster via GitLab](../index.md#adding-and-creating-a-new-gke-cluster-via-gitlab) to add a Kubernetes cluster to your project. ### 2. Install Helm Tiller, Ingress, and JupyterHub @@ -90,7 +92,7 @@ The server will take a couple of seconds to start. ### 4. Configure access In order for the runbook to access your GitLab project, you will need to enter a -[GitLab Access Token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) +[GitLab Access Token](../../../profile/personal_access_tokens.md) as well as your Project ID in the **Setup** section of the demo runbook. Double-click the **DevOps-Runbook-Demo** folder located on the left panel. diff --git a/doc/user/project/code_owners.md b/doc/user/project/code_owners.md index a4937e6cf6b..ae04616943f 100644 --- a/doc/user/project/code_owners.md +++ b/doc/user/project/code_owners.md @@ -43,7 +43,7 @@ Example `CODEOWNERS` file: # app/ @commented-rule -# We can specifiy a default match using wildcards: +# We can specify a default match using wildcards: * @default-codeowner # Rules defined later in the file take precedence over the rules diff --git a/doc/user/project/container_registry.md b/doc/user/project/container_registry.md index 83b268db967..58b7fe33906 100644 --- a/doc/user/project/container_registry.md +++ b/doc/user/project/container_registry.md @@ -53,7 +53,7 @@ If you visit the **Registry** link under your project's menu, you can see the explicit instructions to login to the Container Registry using your GitLab credentials. -For example if the Registry's URL is `registry.example.com`, the you should be +For example if the Registry's URL is `registry.example.com`, then you should be able to login with: ``` @@ -204,7 +204,7 @@ at the communication between the client and the Registry. The REST API between the Docker client and Registry is [described here](https://docs.docker.com/registry/spec/api/). Normally, one would just use Wireshark or tcpdump to capture the traffic and see where things went -wrong. However, since all communication between Docker clients and servers +wrong. However, since all communications between Docker clients and servers are done over HTTPS, it's a bit difficult to decrypt the traffic quickly even if you know the private key. What can we do instead? diff --git a/doc/user/project/deploy_tokens/index.md b/doc/user/project/deploy_tokens/index.md index 7688508c6ac..92a29b68a22 100644 --- a/doc/user/project/deploy_tokens/index.md +++ b/doc/user/project/deploy_tokens/index.md @@ -5,7 +5,7 @@ Deploy tokens allow to download (through `git clone`), or read the container registry images of a project without the need of having a user and a password. Please note, that the expiration of deploy tokens happens on the date you define, -at midnight UTC and that they can be only managed by [maintainers](https://docs.gitlab.com/ee/user/permissions.html). +at midnight UTC and that they can be only managed by [maintainers](../../permissions.md). ## Creating a Deploy Token diff --git a/doc/user/project/description_templates.md b/doc/user/project/description_templates.md index e230444fa67..05ad15476ab 100644 --- a/doc/user/project/description_templates.md +++ b/doc/user/project/description_templates.md @@ -86,7 +86,7 @@ pre-filled with the text you entered in the template(s). We make use of Description Templates for Issues and Merge Requests within the GitLab Community Edition project. Please refer to the [`.gitlab` folder][gitlab-ce-templates] for some examples. > **Tip:** -It is possible to use [quick actions](./quick_actions.md) within description templates to quickly add labels, assignees, and milestones. The quick actions will only be executed if the user submitting the Issue or Merge Request has the permissions perform the relevant actions. +It is possible to use [quick actions](quick_actions.md) within description templates to quickly add labels, assignees, and milestones. The quick actions will only be executed if the user submitting the Issue or Merge Request has the permissions perform the relevant actions. Here is an example for a Bug report template: diff --git a/doc/user/project/file_lock.md b/doc/user/project/file_lock.md index 541023692ea..3386eb9d0d4 100644 --- a/doc/user/project/file_lock.md +++ b/doc/user/project/file_lock.md @@ -1,10 +1,8 @@ # File Locking **[PREMIUM]** -> **Notes:** -> - [Introduced][ee-440] in [GitLab Premium][ee] 8.9. -> - This feature needs to have a license with the "File Lock" option enabled. If -> you are using Premium but you don't see the "Lock" button, -> ask your GitLab administrator. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/440) in [GitLab Premium](https://about.gitlab.com/pricing/) 8.9. +> - This feature needs to have a license with the "File Lock" option enabled. +> - If you are using Premium but you don't see the "Lock" button, ask your GitLab administrator. File Locking helps you avoid merge conflicts and better manage your binary files. Lock any file or directory, make your changes, and then unlock it so another @@ -37,7 +35,7 @@ lies under them is also locked. The user that locks a file or directory **is the only one** that can edit and push their changes back to the repository where the locked objects are located. -Locks can be created by any person who has [push access] to the repository; i.e., +Locks can be created by any person who has [push access](../../user/permissions.md) to the repository; i.e., Developer and higher level, and can be removed solely by their author and any user with Maintainer permissions and above. @@ -101,7 +99,3 @@ To view or manage every existing lock, navigate to the locks and [remove the ones you have permission for](#permissions-on-file-locking).  - -[ee-440]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/440 "File Lock" -[ee]: https://about.gitlab.com/pricing/ -[push access]: ../../user/permissions.md diff --git a/doc/user/project/import/gemnasium.md b/doc/user/project/import/gemnasium.md index dc5b3fcd0bb..7f79ebf6353 100644 --- a/doc/user/project/import/gemnasium.md +++ b/doc/user/project/import/gemnasium.md @@ -9,9 +9,9 @@ Gemnasium has been [acquired by GitLab](https://about.gitlab.com/press/releases/ in January 2018. Since May 15, 2018, the services provided by Gemnasium are no longer available. The team behind Gemnasium has joined GitLab as the new Security Products team and is working on a wider range of tools than just Dependency Scanning: -[SAST](https://docs.gitlab.com/ee/user/application_security/sast/index.html), -[DAST](https://docs.gitlab.com/ee/user/application_security/dast/index.html), -[Container Scanning](https://docs.gitlab.com/ee/user/application_security/container_scanning/index.html) and more. +[SAST](../../application_security/sast/index.md), +[DAST](../../application_security/dast/index.md), +[Container Scanning](../../application_security/container_scanning/index.md) and more. If you want to continue monitoring your dependencies, see the [Migrating to GitLab](#migrating-to-gitlab) section below. @@ -45,7 +45,7 @@ Security features are free for public (open-source) projects hosted on GitLab.co You're almost set! If you're already using [Auto DevOps](../../../topics/autodevops/), you are already covered. Otherwise, you must configure your `.gitlab-ci.yml` according to the -[dependency scanning page](https://docs.gitlab.com/ee/user/application_security/dependency_scanning/index.html). +[dependency scanning page](../../application_security/dependency_scanning/index.md). ### If your project is hosted on GitHub (https://github.com / GitHub Enterprise) @@ -81,7 +81,7 @@ back to both GitLab and GitHub when completed. 1. To set up the dependency scanning job, corresponding to what Gemnasium was doing, you must create a `.gitlab-ci.yml` file, or update it according to - the [dependency scanning docs](https://docs.gitlab.com/ee/user/application_security/dependency_scanning/index.html). + the [dependency scanning docs](../../application_security/dependency_scanning/index.md). The mirroring is pull-only by default, so you may create or update the file on GitHub: diff --git a/doc/user/project/import/github.md b/doc/user/project/import/github.md index 63b90dd76fd..8fba892594b 100644 --- a/doc/user/project/import/github.md +++ b/doc/user/project/import/github.md @@ -119,9 +119,9 @@ Depending your GitLab tier, [project mirroring](../../../workflow/repository_mir your imported project in sync with its GitHub copy. Additionally, you can configure GitLab to send pipeline status updates back GitHub with the -[GitHub Project Integration](https://docs.gitlab.com/ee/user/project/integrations/github.html). **[PREMIUM]** +[GitHub Project Integration](../integrations/github.md). **[PREMIUM]** -If you import your project using [CI/CD for external repo](https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/), then both +If you import your project using [CI/CD for external repo](../../../ci/ci_cd_for_external_repos/index.md), then both of the above are automatically configured. **[PREMIUM]** ## Improving the speed of imports on self-hosted instances diff --git a/doc/user/project/import/gitlab_com.md b/doc/user/project/import/gitlab_com.md index 3b37da67a5b..f48a158e2d3 100644 --- a/doc/user/project/import/gitlab_com.md +++ b/doc/user/project/import/gitlab_com.md @@ -1,8 +1,9 @@ # Project importing from GitLab.com to your private GitLab instance You can import your existing GitLab.com projects to your GitLab instance. But keep in mind that it is possible only if -GitLab support is enabled on your GitLab instance. -You can read more about GitLab support [here](http://docs.gitlab.com/ce/integration/gitlab.html) +GitLab.com integration is enabled on your GitLab instance. +[Read more about GitLab.com integration for self-managed GitLab instances](../../../integration/gitlab.md). + To get to the importer page you need to go to "New project" page. >**Note:** diff --git a/doc/user/project/import/index.md b/doc/user/project/import/index.md index ebbc5ca133b..2b6927bd780 100644 --- a/doc/user/project/import/index.md +++ b/doc/user/project/import/index.md @@ -14,12 +14,13 @@ 1. [From repo by URL](repo_by_url.md) 1. [By uploading a manifest file (AOSP)](manifest.md) 1. [From Gemnasium](gemnasium.md) +1. [From Phabricator](phabricator.md) In addition to the specific migration documentation above, you can import any Git repository via HTTP from the New Project page. Be aware that if the repository is too large the import can timeout. -There is also the option of [connecting your external repository to get CI/CD benefits](https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/index.html). **[PREMIUM]** +There is also the option of [connecting your external repository to get CI/CD benefits](../../../ci/ci_cd_for_external_repos/index.md). **[PREMIUM]** ## Migrating from self-hosted GitLab to GitLab.com diff --git a/doc/user/project/import/phabricator.md b/doc/user/project/import/phabricator.md new file mode 100644 index 00000000000..5c624e3aff6 --- /dev/null +++ b/doc/user/project/import/phabricator.md @@ -0,0 +1,29 @@ +# Import Phabricator tasks into a GitLab project + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/60562) in +GitLab 12.0. + +GitLab allows you to import all tasks from a Phabricator instance into +GitLab issues. The import creates a single project with the +repository disabled. + +Currently, only the following basic fields are imported: + +- Title +- Description +- State (open or closed) +- Created at +- Closed at + +## Enabling this feature + +While this feature is incomplete, a feature flag is required to enable it so that +we can gain early feedback before releasing it for everyone. To enable it: + +1. Run the following command in a Rails console: + + ```ruby + Feature.enable(:phabricator_import) + ``` + +1. Enable Phabricator as an [import source](../../admin_area/settings/visibility_and_access_controls.md#import-sources) in the Admin area. diff --git a/doc/user/project/index.md b/doc/user/project/index.md index 6b3b40bf9f8..a24f525253d 100644 --- a/doc/user/project/index.md +++ b/doc/user/project/index.md @@ -37,7 +37,7 @@ When you create a project in GitLab, you'll have access to a large number of - [Multiple Issue Boards](issue_board.md#multiple-issue-boards-starter): Allow your teams to create their own workflows (Issue Boards) for the same project **[STARTER]** - [Merge Requests](merge_requests/index.md): Apply your branching strategy and get reviewed by your team - - [Merge Request Approvals](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html): Ask for approval before + - [Merge Request Approvals](merge_requests/merge_request_approvals.md): Ask for approval before implementing a change **[STARTER]** - [Fix merge conflicts from the UI](merge_requests/resolve_conflicts.md): Your Git diff tool right from GitLab's UI @@ -74,7 +74,7 @@ When you create a project in GitLab, you'll have access to a large number of timeout (defines the maximum amount of time in minutes that a job is able run), custom path for `.gitlab-ci.yml`, test coverage parsing, pipeline's visibility, and much more - [Kubernetes cluster integration](clusters/index.md): Connecting your GitLab project with a Kubernetes cluster - - [Feature Flags](https://docs.gitlab.com/ee/user/project/operations/feature_flags.html): Feature flags allow you to ship a project in + - [Feature Flags](operations/feature_flags.md): Feature flags allow you to ship a project in different flavors by dynamically toggling certain functionality **[PREMIUM]** - [GitLab Pages](pages/index.md): Build, test, and deploy your static website with GitLab Pages @@ -91,10 +91,10 @@ When you create a project in GitLab, you'll have access to a large number of - [Releases](releases/index.md): a way to track deliverables in your project as snapshot in time of the source, build output, and other metadata or artifacts associated with a released version of your code. -- [Maven packages](https://docs.gitlab.com/ee/user/project/packages/maven_repository.html): your private Maven repository in GitLab. **[PREMIUM]** -- [NPM packages](https://docs.gitlab.com/ee/user/project/packages/npm_registry.html): your private NPM package registry in GitLab. **[PREMIUM]** +- [Maven packages](packages/maven_repository.md): your private Maven repository in GitLab. **[PREMIUM]** +- [NPM packages](packages/npm_registry.md): your private NPM package registry in GitLab. **[PREMIUM]** - [Code owners](code_owners.md): specify code owners for certain files **[STARTER]** -- [License Management](https://docs.gitlab.com/ee/user/application_security/license_management/index.html): approve and blacklist licenses for projects. **[ULTIMATE]** +- [License Management](../application_security/license_management/index.md): approve and blacklist licenses for projects. **[ULTIMATE]** ### Project integrations @@ -135,7 +135,7 @@ Read through the documentation on [project settings](settings/index.md). Instead of importing a repository directly to GitLab, you can connect your repository as a CI/CD project. -Read through the documentation on [CI/CD for external repositories](https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/index.html). +Read through the documentation on [CI/CD for external repositories](../../ci/ci_cd_for_external_repos/index.md). ## Project members diff --git a/doc/user/project/insights/img/insights_sidebar_link.png b/doc/user/project/insights/img/insights_sidebar_link.png Binary files differnew file mode 100644 index 00000000000..aadb5745992 --- /dev/null +++ b/doc/user/project/insights/img/insights_sidebar_link.png diff --git a/doc/user/project/insights/index.md b/doc/user/project/insights/index.md index 720f4c6604e..3344e560870 100644 --- a/doc/user/project/insights/index.md +++ b/doc/user/project/insights/index.md @@ -4,17 +4,24 @@ CAUTION: **Beta:** Insights is considered beta, and is not ready for production use. -Follow [gitlab-org&725](https://gitlab.com/groups/gitlab-org/-/epics/725) for -updates. +Follow [gitlab-org/quality/team-tasks#137](https://gitlab.com/gitlab-org/quality/team-tasks/issues/137#general-availability) +for updates. Configure the Insights that matter for your projects to explore data such as triage hygiene, issues created/closed per a given period, average time for merge requests to be merged and much more. - + NOTE: **Note:** -This feature is [also available at the group level](https://docs.gitlab.com/ee/user/group/insights/index.html). +This feature is [also available at the group level](../../group/insights/index.md). + +## View your project's Insights + +You can access your project's Insights by clicking the **Project > Insights** +link in the left sidebar: + + ## Configure your Insights @@ -26,7 +33,7 @@ for details about the content of this file. NOTE: **Note:** Once the configuration file is created, you can also -[use it for your project's group](https://docs.gitlab.com/ee/user/group/insights/index.html#configure-your-insights). +[use it for your project's group](../../group/insights/index.md#configure-your-insights). NOTE: **Note:** If the project doesn't have any configuration file, it'll try to use @@ -74,8 +81,7 @@ Each chart definition is made up of a hash composed of key-value pairs. For example, here's single chart definition: ```yaml -monthlyBugsCreated: - title: Monthly Bugs Created (bar) +- title: Monthly Bugs Created (bar) type: bar query: issuable_type: issue @@ -173,7 +179,7 @@ Supported values are: Filter by the state of the queried "issuable". -If you omit it, no state filter will be applied. +If you omit it, the `opened` state filter will be applied. Supported values are: @@ -181,6 +187,7 @@ Supported values are: - `closed`: Closed Open issues / merge requests. - `locked`: Issues / merge requests that have their discussion locked. - `merged`: Merged merge requests. +- `all`: Issues / merge requests in all states #### `query.filter_labels` diff --git a/doc/user/project/integrations/img/jira_add_user_to_group.png b/doc/user/project/integrations/img/jira_add_user_to_group.png Binary files differindex 27dac49260c..d8cf541a81e 100644 --- a/doc/user/project/integrations/img/jira_add_user_to_group.png +++ b/doc/user/project/integrations/img/jira_add_user_to_group.png diff --git a/doc/user/project/integrations/img/jira_added_user_to_group.png b/doc/user/project/integrations/img/jira_added_user_to_group.png Binary files differnew file mode 100644 index 00000000000..b3e29a65d6e --- /dev/null +++ b/doc/user/project/integrations/img/jira_added_user_to_group.png diff --git a/doc/user/project/integrations/img/jira_create_new_group.png b/doc/user/project/integrations/img/jira_create_new_group.png Binary files differindex 06c4e84fc61..84be3a94a45 100644 --- a/doc/user/project/integrations/img/jira_create_new_group.png +++ b/doc/user/project/integrations/img/jira_create_new_group.png diff --git a/doc/user/project/integrations/img/jira_create_new_user.png b/doc/user/project/integrations/img/jira_create_new_user.png Binary files differindex e9c03ed770d..8460dc98ef9 100644 --- a/doc/user/project/integrations/img/jira_create_new_user.png +++ b/doc/user/project/integrations/img/jira_create_new_user.png diff --git a/doc/user/project/integrations/img/jira_group_access.png b/doc/user/project/integrations/img/jira_group_access.png Binary files differindex 448cc55504d..58cf114bd55 100644 --- a/doc/user/project/integrations/img/jira_group_access.png +++ b/doc/user/project/integrations/img/jira_group_access.png diff --git a/doc/user/project/integrations/img/jira_user_management_link.png b/doc/user/project/integrations/img/jira_user_management_link.png Binary files differindex 5eb9d031c3e..43ef18da6c8 100644 --- a/doc/user/project/integrations/img/jira_user_management_link.png +++ b/doc/user/project/integrations/img/jira_user_management_link.png diff --git a/doc/user/project/integrations/img/mattermost_configuration.png b/doc/user/project/integrations/img/mattermost_configuration.png Binary files differindex e0b55b23520..75ef0310f2d 100644 --- a/doc/user/project/integrations/img/mattermost_configuration.png +++ b/doc/user/project/integrations/img/mattermost_configuration.png diff --git a/doc/user/project/integrations/img/slack_configuration.png b/doc/user/project/integrations/img/slack_configuration.png Binary files differindex 53b30e0e8cd..a14d2969488 100644 --- a/doc/user/project/integrations/img/slack_configuration.png +++ b/doc/user/project/integrations/img/slack_configuration.png diff --git a/doc/user/project/integrations/jira.md b/doc/user/project/integrations/jira.md index a90167b9767..c652149052e 100644 --- a/doc/user/project/integrations/jira.md +++ b/doc/user/project/integrations/jira.md @@ -47,11 +47,11 @@ project in Jira and then enter the correct values in GitLab. When connecting to **JIRA Server**, which supports basic authentication, a **username and password** are required. Check the link below and proceed to the next step: -- [Setting up an user in JIRA server](jira_server_configuration.md) +- [Setting up a user in JIRA server](jira_server_configuration.md) When connecting to **JIRA Cloud**, which supports authentication via API token, an **email and API token**, are required. Check the link below and proceed to the next step: -- [Setting up an user in JIRA cloud](jira_cloud_configuration.md) +- [Setting up a user in JIRA cloud](jira_cloud_configuration.md) ### Configuring GitLab diff --git a/doc/user/project/integrations/jira_server_configuration.md b/doc/user/project/integrations/jira_server_configuration.md index 20036183187..13d65c4d8e4 100644 --- a/doc/user/project/integrations/jira_server_configuration.md +++ b/doc/user/project/integrations/jira_server_configuration.md @@ -1,18 +1,16 @@ # Creating a username and password for JIRA server We need to create a user in Jira which will have access to all projects that -need to integrate with GitLab. Login to your Jira instance as admin and under -*Administration*, go to *User Management* and create a new user. +need to integrate with GitLab. As an example, we'll create a user named `gitlab` and add it to the `Jira-developers` group. NOTE: **Note** -It is important that the user `gitlab` has 'write' access to projects in Jira. +It is important that the Jira user created for the integration is given 'write' +access to your Jira projects. This is covered in the process below. -We have split this stage in steps so it is easier to follow. - -1. Log in to your Jira instance as an administrator and under **Administration** +1. Log in to your Jira instance as an administrator and under **Jira Administration** go to **User Management** to create a new user.  @@ -27,27 +25,34 @@ We have split this stage in steps so it is easier to follow.  -1. Create a `gitlab-developers` group which will have write access - to projects in Jira. Go to the **Groups** tab and select **Create group**. +1. Create a `gitlab-developers` group. (We will give this group write access to Jira + projects in a later step). Go to the **Groups** tab on the left, and select **Add group**.  - Give it an optional description and click **Create group**. + Give it a name and click **Add group**. -  +1. Add the `gitlab` user to the `gitlab-developers` group by clicking **Edit members**. + The `gitlab-developers` group should be listed in the leftmost box as a selected group. + Under **Add members to selected group(s)**, enter `gitlab`. -1. To give the newly-created group 'write' access, go to - **Application access > View configuration** and add the `gitlab-developers` - group to Jira Core. +  + + Click **Add selected users** and `gitlab` should appear in the **Group member(s)** box. + This membership is saved automatically. + +  + +1. To give the newly-created group 'write' access, you need to create a **Permission Scheme**. + To do this, click the gear icon and select **Issues**. Then click **Permission Schemes**. + Click **Add Permission Scheme** and enter a **Name** and, optionally, a **Description**. + +1. Once your permission scheme is created, you'll be taken back to the permissions scheme list. + Locate your new permissions scheme and click **Permissions**. Next to **Administer Projects**, + click **Edit**. In the resulting dialog box, select **Group** and select `gitlab-developers` + from the dropdown.  -1. Add the `gitlab` user to the `gitlab-developers` group by going to - **Users > GitLab user > Add group** and selecting the `gitlab-developers` - group from the dropdown menu. Notice that the group says _Access_, which is - intended as part of this process. - -  - The Jira configuration is complete. Write down the new Jira username and its password as they will be needed when [configuring GitLab in the next section](jira.md#configuring-gitlab). diff --git a/doc/user/project/integrations/mattermost.md b/doc/user/project/integrations/mattermost.md index 8c5461de42f..d7fd75fd728 100644 --- a/doc/user/project/integrations/mattermost.md +++ b/doc/user/project/integrations/mattermost.md @@ -27,9 +27,11 @@ There, you will see a checkbox with the following events that can be triggered: - Confidential issue - Merge request - Note +- Confidential note - Tag push - Pipeline - Wiki page +- Deployment Below each of these event checkboxes, you have an input field to enter which Mattermost channel you want to send that event message. Enter your preferred channel handle (the hash sign `#` is optional). diff --git a/doc/user/project/integrations/project_services.md b/doc/user/project/integrations/project_services.md index f560de427c5..0bfee3bac99 100644 --- a/doc/user/project/integrations/project_services.md +++ b/doc/user/project/integrations/project_services.md @@ -39,7 +39,7 @@ Click on the service links to see further configuration instructions and details | [HipChat](hipchat.md) | Private group chat and IM | | [Irker (IRC gateway)](irker.md) | Send IRC messages, on update, to a list of recipients through an Irker gateway | | [JIRA](jira.md) | JIRA issue tracker | -| [Jenkins](https://docs.gitlab.com/ee/integration/jenkins.html) **[STARTER]** | An extendable open source continuous integration server | +| [Jenkins](../../../integration/jenkins.md) **[STARTER]** | An extendable open source continuous integration server | | JetBrains TeamCity CI | A continuous integration and build server | | [Mattermost slash commands](mattermost_slash_commands.md) | Mattermost chat and ChatOps slash commands | | [Mattermost Notifications](mattermost.md) | Receive event notifications in Mattermost | diff --git a/doc/user/project/integrations/prometheus.md b/doc/user/project/integrations/prometheus.md index 40113624430..751e8e44e60 100644 --- a/doc/user/project/integrations/prometheus.md +++ b/doc/user/project/integrations/prometheus.md @@ -168,7 +168,7 @@ Alerts can be used to trigger actions, like open an issue automatically. To conf 1. Optionally, select whether to send an email notification to the developers of the project. 1. Click **Save changes**. -Once enabled, an issue will be opened automatically when an alert is triggered. To further customize the issue, you can add labels, mentions, or any other supported [quick action](../quick_actions.md) in the selected issue template. +Once enabled, an issue will be opened automatically when an alert is triggered. The author of the issue will be the GitLab Alert Bot. To further customize the issue, you can add labels, mentions, or any other supported [quick action](../quick_actions.md) in the selected issue template. If the metric exceeds the threshold of the alert for over 5 minutes, an email will be sent to all [Maintainers and Owners](../../permissions.md#project-members-permissions) of the project. diff --git a/doc/user/project/integrations/prometheus_library/kubernetes.md b/doc/user/project/integrations/prometheus_library/kubernetes.md index 436129f1dbc..8b1cf1a251a 100644 --- a/doc/user/project/integrations/prometheus_library/kubernetes.md +++ b/doc/user/project/integrations/prometheus_library/kubernetes.md @@ -27,7 +27,7 @@ integration services must be enabled. Prometheus needs to be deployed into the cluster and configured properly in order to gather Kubernetes metrics. GitLab supports two methods for doing so: -- GitLab [integrates with Kubernetes](../../clusters/index.md), and can [deploy Prometheus into a connected cluster](../prometheus.html#managed-prometheus-on-kubernetes). It is automatically configured to collect Kubernetes metrics. +- GitLab [integrates with Kubernetes](../../clusters/index.md), and can [deploy Prometheus into a connected cluster](../prometheus.md#managed-prometheus-on-kubernetes). It is automatically configured to collect Kubernetes metrics. - To configure your own Prometheus server, you can follow the [Prometheus documentation](https://prometheus.io/docs/introduction/overview/). ## Specifying the Environment @@ -40,7 +40,7 @@ Instead, the [Deployment](https://kubernetes.io/docs/concepts/workloads/controll > Introduced in [GitLab 10.2](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/15201). -GitLab also gathers Kubernetes metrics for [canary deployments](https://docs.gitlab.com/ee/user/project/canary_deployments.html), allowing easy comparison between the current deployed version and the canary. +GitLab also gathers Kubernetes metrics for [canary deployments](../../canary_deployments.md), allowing easy comparison between the current deployed version and the canary. These metrics expect the [Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) or [DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) name to begin with `$CI_ENVIRONMENT_SLUG-canary`, to isolate the canary metrics. diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index 10eb3a4f3b7..a0f500a939f 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -1016,7 +1016,7 @@ X-Gitlab-Event: Pipeline Hook "email": "user@gitlab.com" } }, - "jobs":[ + "builds":[ { "id": 380, "stage": "deploy", diff --git a/doc/user/project/integrations/youtrack.md b/doc/user/project/integrations/youtrack.md index a2a468b6fe4..81c148e41fd 100644 --- a/doc/user/project/integrations/youtrack.md +++ b/doc/user/project/integrations/youtrack.md @@ -31,8 +31,8 @@ To disable the internal issue tracker in a project: ## Referencing YouTrack issues in GitLab Issues in YouTrack can be referenced as `<PROJECT>-<ID>`. `<PROJECT>` -must start with a capital letter and can then be followed by capital or lower case -letters, numbers or underscores. `<ID>` is a number. An example reference is `YT-101` or `Api_32-143`. +must start with a letter and is followed by letters, numbers, or underscores. +`<ID>` is a number. An example reference is `YT-101`, `Api_32-143` or `gl-030`. References to `<PROJECT>-<ID>` in merge requests, commits, or comments are automatically linked to the YouTrack issue URL. For more information, see the [External Issue Tracker](../../../integration/external-issue-tracker.md) documentation. diff --git a/doc/user/project/issues/create_new_issue.md b/doc/user/project/issues/create_new_issue.md index 5e05846b77f..c2916c79876 100644 --- a/doc/user/project/issues/create_new_issue.md +++ b/doc/user/project/issues/create_new_issue.md @@ -67,7 +67,7 @@ or contacts to continue working._ ## New issue via Service Desk **[PREMIUM]** -Enable [Service Desk](https://docs.gitlab.com/ee/user/project/service_desk.html) to your project and offer email support. +Enable [Service Desk](../service_desk.md) to your project and offer email support. By doing so, when your customer sends a new email, a new issue can be created in the appropriate project and followed up from there. diff --git a/doc/user/project/issues/index.md b/doc/user/project/issues/index.md index c82b7f100d2..94865ad46ee 100644 --- a/doc/user/project/issues/index.md +++ b/doc/user/project/issues/index.md @@ -102,13 +102,13 @@ For more information, see the [Issue Boards](../issue_board.md) page. Epics let you manage your portfolio of projects more efficiently and with less effort by tracking groups of issues that share a theme, across projects and milestones. -For more information, see the [Epics](https://docs.gitlab.com/ee/user/group/epics/) page. +For more information, see the [Epics](../../group/epics/index.md) page. ### Related issues **[STARTER]** You can mark two issues as related, so that when viewing each one, the other is always listed in its Related Issues section. This can help display important context, such as past work, dependencies, or duplicates. -For more information, see [Related Issues](https://docs.gitlab.com/ee/user/project/issues/related_issues.html). +For more information, see [Related Issues](related_issues.md). ### Crosslinking issues @@ -129,7 +129,7 @@ For more information, see [Crosslinking issues](crosslinking_issues.md). - [Bulk edit issues](../bulk_editing.md) - From the Issues List, select multiple issues in order to change their status, assignee, milestone, or labels in bulk. - [Import issues](csv_import.md) -- [Export issues](https://docs.gitlab.com/ee/user/project/issues/csv_export.html) **[STARTER]** +- [Export issues](csv_export.md) **[STARTER]** - [Issues API](../../../api/issues.md) - Configure an [external issue tracker](../../../integration/external-issue-tracker.md) such as Jira, Redmine, or Bugzilla. diff --git a/doc/user/project/issues/issue_data_and_actions.md b/doc/user/project/issues/issue_data_and_actions.md index ef9fcaec3e6..fc11c0251e0 100644 --- a/doc/user/project/issues/issue_data_and_actions.md +++ b/doc/user/project/issues/issue_data_and_actions.md @@ -90,7 +90,7 @@ If a label doesn't exist yet, you can click **Edit**, and it opens a dropdown me - Assign a weight. Larger values are used to indicate more effort is required to complete the issue. Only positive values or zero are allowed. -Learn more in the [Issue Weight documentation](https://docs.gitlab.com/ee/workflow/issue_weight.html). +Learn more in the [Issue Weight documentation](../../../workflow/issue_weight.md). #### 9. Participants @@ -103,7 +103,7 @@ Learn more in the [Issue Weight documentation](https://docs.gitlab.com/ee/workfl - Unsubscribe: if you are receiving notifications on that issue but no longer want to receive them, unsubscribe from it. -Read more in the [notifications documentation](https://docs.gitlab.com/ee/workflow/notifications.html#issue--epics--merge-request-events). +Read more in the [notifications documentation](../../../workflow/notifications.md#issue--epics--merge-request-events). #### 11. Reference diff --git a/doc/user/project/issues/multiple_assignees_for_issues.md b/doc/user/project/issues/multiple_assignees_for_issues.md index 8781ebdd5b0..d1db0790d69 100644 --- a/doc/user/project/issues/multiple_assignees_for_issues.md +++ b/doc/user/project/issues/multiple_assignees_for_issues.md @@ -22,7 +22,7 @@ Consider a team formed by frontend developers, backend developers, UX designers, QA testers, and a product manager working together to bring an idea to market. -Multiple Assignees for Issues makes collaboration smother, +Multiple Assignees for Issues makes collaboration smoother, and allows shared responsibilities to be clearly displayed. All assignees are shown across your team's workflows and receive notifications (as they would as single assignees), simplifying communication and ownership. diff --git a/doc/user/project/issues/related_issues.md b/doc/user/project/issues/related_issues.md index db0ab65b442..e679ebf86e6 100644 --- a/doc/user/project/issues/related_issues.md +++ b/doc/user/project/issues/related_issues.md @@ -1,6 +1,6 @@ # Related issues **[STARTER]** -> [Introduced][ee-1797] in [GitLab Starter][ee] 9.4. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/1797) in [GitLab Starter](https://about.gitlab.com/pricing/) 9.4. Related issues are a bi-directional relationship between any two issues and appear in a block below the issue description. Issues can be across groups @@ -35,11 +35,6 @@ will no longer appear in either issue.  -Please access our [permissions] page for more information. +Please access our [permissions](../../permissions.md) page for more information. -Additionally, you are also able to manage related issues through [our API]. - -[ee]: https://about.gitlab.com/pricing/ -[ee-1797]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/1797 -[permissions]: ../../permissions.md -[Our API]: https://docs.gitlab.com/ee/api/issue_links.html +Additionally, you are also able to manage related issues through [our API](../../../api/issue_links.md). diff --git a/doc/user/project/labels.md b/doc/user/project/labels.md index 63ab02541bd..e5f62a3bb8d 100644 --- a/doc/user/project/labels.md +++ b/doc/user/project/labels.md @@ -139,7 +139,7 @@ From the project issue list page and the project merge request list page, you ca From the group issue list page and the group merge request list page, you can [filter](../search/index.md#issues-and-merge-requests) by both group labels (including subgroup ancestors and subgroup descendants) and project labels. -From the group epic list page, you can [filter](../search/index.md#issues-and-merge-requests) by both current group labels as well as decendent group labels. +From the group epic list page, you can [filter](../search/index.md#issues-and-merge-requests) by both current group labels as well as descendant group labels.  diff --git a/doc/user/project/maven_packages.md b/doc/user/project/maven_packages.md index d32d6084b38..48835a2dac7 100644 --- a/doc/user/project/maven_packages.md +++ b/doc/user/project/maven_packages.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/project/packages/maven_repository.html' +redirect_to: 'packages/maven_repository.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/user/project/packages/maven_repository.html). +This document was moved to [another location](packages/maven_repository.md). diff --git a/doc/user/project/merge_requests/code_quality.md b/doc/user/project/merge_requests/code_quality.md index e6811b5df5e..705ff333579 100644 --- a/doc/user/project/merge_requests/code_quality.md +++ b/doc/user/project/merge_requests/code_quality.md @@ -19,7 +19,7 @@ in the merge request widget area: For instance, consider the following workflow: -1. Your backend team member starts a new implementation for making certain feature in your app faster +1. Your backend team member starts a new implementation for making a certain feature in your app faster 1. With Code Quality reports, they analyze how their implementation is impacting the code quality 1. The metrics show that their code degrade the quality in 10 points 1. You ask a co-worker to help them with this modification @@ -63,7 +63,7 @@ Example: NOTE: **Note:** Although the Code Climate spec supports more properties, those are ignored by GitLab. -For more information on how the Code Quality job should look like, check the +For more information on what the Code Quality job should look like, check the example on [analyzing a project's code quality](../../../ci/examples/code_quality.md). GitLab then checks this report, compares the metrics between the source and target diff --git a/doc/user/project/merge_requests/code_quality_diff.md b/doc/user/project/merge_requests/code_quality_diff.md index 890058eec6f..ccc694672a6 100644 --- a/doc/user/project/merge_requests/code_quality_diff.md +++ b/doc/user/project/merge_requests/code_quality_diff.md @@ -1,5 +1,5 @@ --- -redirect_from: 'https://docs.gitlab.com/ee/user/project/merge_requests/code_quality_diff.html' +redirect_from: 'code_quality_diff.md' redirect_to: 'code_quality.md' --- diff --git a/doc/user/project/merge_requests/container_scanning.md b/doc/user/project/merge_requests/container_scanning.md index 4d41e424f4a..a062731ea35 100644 --- a/doc/user/project/merge_requests/container_scanning.md +++ b/doc/user/project/merge_requests/container_scanning.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/application_security/container_scanning/index.html' +redirect_to: '../../application_security/container_scanning/index.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/user/application_security/container_scanning/index.html). +This document was moved to [another location](../../application_security/container_scanning/index.md). diff --git a/doc/user/project/merge_requests/dast.md b/doc/user/project/merge_requests/dast.md index b676c661267..98a2906e560 100644 --- a/doc/user/project/merge_requests/dast.md +++ b/doc/user/project/merge_requests/dast.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/application_security/dast/index.html' +redirect_to: '../../application_security/dast/index.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/user/application_security/dast/index.html). +This document was moved to [another location](../../application_security/dast/index.md). diff --git a/doc/user/project/merge_requests/dependency_scanning.md b/doc/user/project/merge_requests/dependency_scanning.md index 3a8b53b425c..bdc1c355016 100644 --- a/doc/user/project/merge_requests/dependency_scanning.md +++ b/doc/user/project/merge_requests/dependency_scanning.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/application_security/dependency_scanning/index.html' +redirect_to: '../../application_security/dependency_scanning/index.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/user/application_security/dependency_scanning/index.html). +This document was moved to [another location](../../application_security/dependency_scanning/index.md). diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index 2bb2d906453..86e06c2ea55 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -33,15 +33,15 @@ With GitLab merge requests, you can: With **[GitLab Enterprise Edition][ee]**, you can also: -- Prepare a full review and submit it once it's ready with [Merge Request Reviews](https://docs.gitlab.com/ee/user/discussions/index.md#merge-request-reviews-premium) **[PREMIUM]** -- View the deployment process across projects with [Multi-Project Pipelines](https://docs.gitlab.com/ee/ci/multi_project_pipelines.md) **[PREMIUM]** +- Prepare a full review and submit it once it's ready with [Merge Request Reviews](../../discussions/index.md#merge-request-reviews-premium) **[PREMIUM]** +- View the deployment process across projects with [Multi-Project Pipelines](../../../ci/multi_project_pipelines.md) **[PREMIUM]** - Request [approvals](merge_request_approvals.md) from your managers **[STARTER]** - Analyze the impact of your changes with [Code Quality reports](code_quality.md) **[STARTER]** -- Manage the licenses of your dependencies with [License Management](https://docs.gitlab.com/ee/user/application_security/license_management/index.md) **[ULTIMATE]** -- Analyze your source code for vulnerabilities with [Static Application Security Testing](https://docs.gitlab.com/ee/user/application_security/sast/index.md) **[ULTIMATE]** -- Analyze your running web applications for vulnerabilities with [Dynamic Application Security Testing](https://docs.gitlab.com/ee/user/application_security/dast/index.md) **[ULTIMATE]** -- Analyze your dependencies for vulnerabilities with [Dependency Scanning](https://docs.gitlab.com/ee/user/application_security/dependency_scanning/index.md) **[ULTIMATE]** -- Analyze your Docker images for vulnerabilities with [Container Scanning](https://docs.gitlab.com/ee/user/application_security/container_scanning/index.md) **[ULTIMATE]** +- Manage the licenses of your dependencies with [License Management](../../application_security/license_management/index.md) **[ULTIMATE]** +- Analyze your source code for vulnerabilities with [Static Application Security Testing](../../application_security/sast/index.md) **[ULTIMATE]** +- Analyze your running web applications for vulnerabilities with [Dynamic Application Security Testing](../../application_security/dast/index.md) **[ULTIMATE]** +- Analyze your dependencies for vulnerabilities with [Dependency Scanning](../../application_security/dependency_scanning/index.md) **[ULTIMATE]** +- Analyze your Docker images for vulnerabilities with [Container Scanning](../../application_security/container_scanning/index.md) **[ULTIMATE]** - Determine the performance impact of changes with [Browser Performance Testing](#browser-performance-testing-premium) **[PREMIUM]** ## Use cases @@ -174,7 +174,7 @@ Start a review in order to create multiple comments on a diff and publish them o Starting a review allows you to get all your thoughts in order and ensure you haven't missed anything before submitting all your comments. -[Learn more about Merge Request Reviews](https://docs.gitlab.com/ee/user/discussions/index.html#merge-request-reviews-premium) +[Learn more about Merge Request Reviews](../../discussions/index.md#merge-request-reviews-premium) ## Squash and merge @@ -395,7 +395,15 @@ GitLab runs the [Sitespeed.io container][sitespeed-container] and displays the d GitLab can scan and report any vulnerabilities found in your project. -[Read more about security reports.](https://docs.gitlab.com/ee/user/application_security/index.html) +[Read more about security reports.](../../application_security/index.md) + +## JUnit test reports + +Configure your CI jobs to use JUnit test reports, and let GitLab display a report +on the merge request so that it’s easier and faster to identify the failure +without having to check the entire job log. + +[Read more about JUnit test reports](../../../ci/junit_test_reports.md). ## Live preview with Review Apps @@ -503,7 +511,7 @@ seconds and the status will update automatically. Merge Request pipeline statuses can't be retrieved when the following occurs: -1. A Merge Requst is created +1. A Merge Request is created 1. The Merge Request is closed 1. Changes are made in the project 1. The Merge Request is reopened @@ -539,13 +547,13 @@ Add the following alias to your `~/.gitconfig`: Now you can check out a particular merge request from any repository and any remote. For example, to check out the merge request with ID 5 as shown in GitLab -from the `upstream` remote, do: +from the `origin` remote, do: ``` -git mr upstream 5 +git mr origin 5 ``` -This will fetch the merge request into a local `mr-upstream-5` branch and check +This will fetch the merge request into a local `mr-origin-5` branch and check it out. #### Checkout locally by modifying `.git/config` for a given repository @@ -598,6 +606,9 @@ And to check out a particular merge request: git checkout origin/merge-requests/1 ``` +all the above can be done with [git-mr] script. + +[git-mr]: https://gitlab.com/glensc/git-mr [products]: https://about.gitlab.com/products/ "GitLab products page" [protected branches]: ../protected_branches.md [ci]: ../../../ci/README.md diff --git a/doc/user/project/merge_requests/license_management.md b/doc/user/project/merge_requests/license_management.md index 08704425a75..93116ebd7c6 100644 --- a/doc/user/project/merge_requests/license_management.md +++ b/doc/user/project/merge_requests/license_management.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/application_security/license_management/index.html' +redirect_to: '../../application_security/license_management/index.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/user/application_security/license_management/index.html). +This document was moved to [another location](../../application_security/license_management/index.md). diff --git a/doc/user/project/merge_requests/merge_request_approvals.md b/doc/user/project/merge_requests/merge_request_approvals.md index d0291c4cef5..2e9db949890 100644 --- a/doc/user/project/merge_requests/merge_request_approvals.md +++ b/doc/user/project/merge_requests/merge_request_approvals.md @@ -3,7 +3,7 @@ > Introduced in [GitLab Enterprise Edition 7.12](https://about.gitlab.com/2015/06/22/gitlab-7-12-released/#merge-request-approvers-ee-only). NOTE: **Note:** -If you are running a self-managed instance, the new interface shown on +Prior to 12.0, if you are running a self-managed instance, the new interface shown on this page will not be available unless the feature flag `approval_rules` is enabled, which can be done from the Rails console by instance administrators. @@ -105,7 +105,7 @@ any [eligible approver](#eligible-approvers) may approve. The following can approve merge requests: - Users being added as approvers at project or merge request level. -- [Code owners](https://docs.gitlab.com/ee/user/project/code_owners.html) related to the merge request ([introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/7933) in [GitLab Starter](https://about.gitlab.com/pricing/) 11.5). +- [Code owners](../code_owners.md) related to the merge request ([introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/7933) in [GitLab Starter](https://about.gitlab.com/pricing/) 11.5). An individual user can be added as an approver for a project if they are a member of: @@ -168,7 +168,7 @@ or a [failed CI/CD pipeline](merge_when_pipeline_succeeds.md). > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/4418) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.9. It is possible to require at least one approval for each entry in the -[`CODEOWNERS` file](https://docs.gitlab.com/ee/user/project/code_owners.html) that matches a file changed in +[`CODEOWNERS` file](../code_owners.md) that matches a file changed in the merge request. To enable this feature: 1. Navigate to your project's **Settings > General** and expand diff --git a/doc/user/project/merge_requests/sast.md b/doc/user/project/merge_requests/sast.md index 688cc79d0f6..165290eb114 100644 --- a/doc/user/project/merge_requests/sast.md +++ b/doc/user/project/merge_requests/sast.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/application_security/sast/index.html' +redirect_to: '../../application_security/sast/index.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/user/application_security/sast/index.html). +This document was moved to [another location](../../application_security/sast/index.md). diff --git a/doc/user/project/merge_requests/sast_docker.md b/doc/user/project/merge_requests/sast_docker.md index 4d41e424f4a..a062731ea35 100644 --- a/doc/user/project/merge_requests/sast_docker.md +++ b/doc/user/project/merge_requests/sast_docker.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/application_security/container_scanning/index.html' +redirect_to: '../../application_security/container_scanning/index.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/user/application_security/container_scanning/index.html). +This document was moved to [another location](../../application_security/container_scanning/index.md). diff --git a/doc/user/project/milestones/burndown_charts.md b/doc/user/project/milestones/burndown_charts.md index 0ad08da8ff5..7ffeb032d7f 100644 --- a/doc/user/project/milestones/burndown_charts.md +++ b/doc/user/project/milestones/burndown_charts.md @@ -52,7 +52,7 @@ and select a milestone from your current ones, while for group's, access the **G select a group, and go through **Issues > Milestones** on the sidebar. NOTE: **Note:** -You're able to [promote project](https://docs.gitlab.com/ee/user/project/milestones/#promoting-project-milestones-to-group-milestones) to group milestones and still see the **Burndown Chart** for them, respecting license limitations. +You're able to [promote project](index.md#promoting-project-milestones-to-group-milestones) to group milestones and still see the **Burndown Chart** for them, respecting license limitations. The chart indicates the project's progress throughout that milestone (for issues assigned to it). diff --git a/doc/user/project/milestones/index.md b/doc/user/project/milestones/index.md index 0d8ee0a6cd2..6cd866b5c0d 100644 --- a/doc/user/project/milestones/index.md +++ b/doc/user/project/milestones/index.md @@ -138,7 +138,7 @@ For group milestones in [GitLab Premium](https://about.gitlab.com/pricing), a [b The milestone sidebar on the milestone view shows the following: -- Percentage complete, which is calculated as number of closed issues plus number of closed/merged merge requests divided by total number issues and merge requests. +- Percentage complete, which is calculated as number of closed issues divided by total number of issues. - The start date and due date. - The total time spent on all issues that have the milestone assigned. diff --git a/doc/user/project/packages/maven_repository.md b/doc/user/project/packages/maven_repository.md index 94785eb6aec..9b7af738696 100644 --- a/doc/user/project/packages/maven_repository.md +++ b/doc/user/project/packages/maven_repository.md @@ -12,7 +12,7 @@ project can have its own space to store its Maven artifacts. NOTE: **Note:** This option is available only if your GitLab administrator has -[enabled support for the Maven repository](https://docs.gitlab.com/ee/administration/packages.html).**[PREMIUM ONLY]** +[enabled support for the Maven repository](../../../administration/packages.md).**[PREMIUM ONLY]** After the Packages feature is enabled, the Maven Repository will be available for all new projects by default. To enable it for existing projects, or if you want diff --git a/doc/user/project/packages/npm_registry.md b/doc/user/project/packages/npm_registry.md index 9f4c01c9a0a..2e274573434 100644 --- a/doc/user/project/packages/npm_registry.md +++ b/doc/user/project/packages/npm_registry.md @@ -20,7 +20,7 @@ within a subgroup is not supported yet. NOTE: **Note:** This option is available only if your GitLab administrator has -[enabled support for the NPM registry](https://docs.gitlab.com/ee/administration/packages.html).**[PREMIUM ONLY]** +[enabled support for the NPM registry](../../../administration/packages.md).**[PREMIUM ONLY]** After the NPM registry is enabled, it will be available for all new projects by default. To enable it for existing projects, or if you want to disable it: diff --git a/doc/user/project/pages/getting_started_part_four.md b/doc/user/project/pages/getting_started_part_four.md index 87cd4941ae6..8baf41dba78 100644 --- a/doc/user/project/pages/getting_started_part_four.md +++ b/doc/user/project/pages/getting_started_part_four.md @@ -1,10 +1,6 @@ --- -last_updated: 2018-02-16 -author: Marcia Ramos -author_gitlab: marcia -level: intermediate -article_type: user guide -date: 2017-02-22 +last_updated: 2019-06-04 +type: reference, howto --- # Creating and Tweaking GitLab CI/CD for GitLab Pages diff --git a/doc/user/project/pages/getting_started_part_one.md b/doc/user/project/pages/getting_started_part_one.md index 7dbf58b5715..6d538ca2455 100644 --- a/doc/user/project/pages/getting_started_part_one.md +++ b/doc/user/project/pages/getting_started_part_one.md @@ -1,5 +1,6 @@ --- -last_updated: 2018-02-16 +last_updated: 2018-06-04 +type: concepts, reference --- # Static sites and GitLab Pages domains diff --git a/doc/user/project/pages/getting_started_part_three.md b/doc/user/project/pages/getting_started_part_three.md index 9f2bc281f85..d585c19fc5c 100644 --- a/doc/user/project/pages/getting_started_part_three.md +++ b/doc/user/project/pages/getting_started_part_three.md @@ -1,10 +1,6 @@ --- -last_updated: 2018-11-19 -author: Marcia Ramos -author_gitlab: marcia -level: beginner -article_type: user guide -date: 2017-02-22 +last_updated: 2019-06-04 +type: concepts, reference, howto --- # GitLab Pages custom domains and SSL/TLS Certificates @@ -96,7 +92,7 @@ you need to log into your domain's admin control panel and add a DNS `CNAME` record pointing your subdomain to your website URL (`namespace.gitlab.io`) address. -Notice that, despite it's a user or project website, the `CNAME` +Note that, whether it's a user or a project website, the `CNAME` should point to your Pages domain (`namespace.gitlab.io`), without any `/project-name`. diff --git a/doc/user/project/pages/getting_started_part_two.md b/doc/user/project/pages/getting_started_part_two.md index faf51154e3b..3e50cd4887c 100644 --- a/doc/user/project/pages/getting_started_part_two.md +++ b/doc/user/project/pages/getting_started_part_two.md @@ -1,10 +1,6 @@ --- -last_updated: 2019-03-05 -author: Marcia Ramos -author_gitlab: marcia -level: beginner -article_type: user guide -date: 2017-02-22 +last_updated: 2019-06-04 +type: reference, howto --- # Projects for GitLab Pages and URL structure @@ -13,11 +9,11 @@ date: 2017-02-22 To get started with GitLab Pages, you need: -1. A project -1. A configuration file (`.gitlab-ci.yml`) to deploy your site +1. A project, thus a repository to hold your website's codebase. +1. A configuration file (`.gitlab-ci.yml`) to deploy your site. 1. A specific `job` called `pages` in the configuration file - that will make GitLab aware that you are deploying a GitLab Pages website -1. A `public` directory with the content of the website + that will make GitLab aware that you are deploying a GitLab Pages website. +1. A `public` directory with the static content of the website. Optional Features: @@ -88,7 +84,7 @@ website from your project's **Settings > Pages**. You can also take some **optional** further steps: -- _Remove the fork relationship._ The fork relashionship is necessary to contribute back to the project you originally forked from. If you don't have any intentions to do so, you can remove it. To do so, navigate to your project's **Settings**, expand **Advanced settings**, and scroll down to **Remove fork relationship**: +- _Remove the fork relationship._ The fork relationship is necessary to contribute back to the project you originally forked from. If you don't have any intentions to do so, you can remove it. To do so, navigate to your project's **Settings**, expand **Advanced settings**, and scroll down to **Remove fork relationship**:  @@ -140,7 +136,7 @@ where you'll find its default URL. repository to you local computer and moving your site files into it, you can run `git init` in your local website directory, add the remote URL: `git remote add origin git@gitlab.com:namespace/project-name.git`, - then add, commit, and push. + then add, commit, and push to GitLab. ## URLs and Baseurls @@ -173,4 +169,4 @@ baseurl: "" ## Custom Domains GitLab Pages supports custom domains and subdomains, served under HTTP or HTTPS. -Please check the [next part](getting_started_part_three.md) of this series for an overview. +See [GitLab Pages custom domains and SSL/TLS Certificates](getting_started_part_three.md) for more information. diff --git a/doc/user/project/pages/index.md b/doc/user/project/pages/index.md index 91098d51160..04bda212128 100644 --- a/doc/user/project/pages/index.md +++ b/doc/user/project/pages/index.md @@ -1,6 +1,7 @@ --- description: 'Learn how to use GitLab Pages to deploy a static website at no additional cost.' -last_updated: 2019-03-05 +last_updated: 2019-06-04 +type: index, reference --- # GitLab Pages @@ -140,7 +141,7 @@ To learn more about configuration options for GitLab Pages, read the following: | [Static websites and Pages domains](getting_started_part_one.md) | Understand what is a static website, and how GitLab Pages default domains work. | | [Projects and URL structure](getting_started_part_two.md) | Forking projects and creating new ones from scratch, understanding URLs structure and baseurls. | | [GitLab CI/CD for GitLab Pages](getting_started_part_four.md) | Understand how to create your own `.gitlab-ci.yml` for your site. | -| [Exploring GitLab Pages](introduction.md) | Technical aspects, specific configuration options, custom 404 pages, limitations. | +| [Exploring GitLab Pages](introduction.md) | Requirements, technical aspects, specific GitLab CI's configuration options, custom 404 pages, limitations, FAQ. | |---+---| | [Custom domains and SSL/TLS Certificates](getting_started_part_three.md) | How to add custom domains and subdomains to your website, configure DNS records and SSL/TLS certificates. | | [CloudFlare certificates](https://about.gitlab.com/2017/02/07/setting-up-gitlab-pages-with-cloudflare-certificates/) | Secure your Pages site with CloudFlare certificates. | diff --git a/doc/user/project/pages/introduction.md b/doc/user/project/pages/introduction.md index a14a446aead..4fab7f79e0c 100644 --- a/doc/user/project/pages/introduction.md +++ b/doc/user/project/pages/introduction.md @@ -1,3 +1,8 @@ +--- +type: reference +last_updated: 2018-06-04 +--- + # Exploring GitLab Pages This document is a user guide to explore the options and settings @@ -10,7 +15,7 @@ To familiarize yourself with GitLab Pages first: - Learn how to enable GitLab Pages across your GitLab instance on the [administrator documentation](../../../administration/pages/index.md). -## Pages requirements +## GitLab Pages requirements In brief, this is what you need to upload your website in GitLab Pages: @@ -34,6 +39,99 @@ If you are using [GitLab Pages on GitLab.com](#gitlab-pages-on-gitlabcom) to hos Visit the [GitLab Pages group](https://gitlab.com/groups/pages) for a complete list of example projects. Contributions are very welcome. +## Custom error codes Pages + +You can provide your own 403 and 404 error pages by creating the `403.html` and +`404.html` files respectively in the root directory of the `public/` directory +that will be included in the artifacts. Usually this is the root directory of +your project, but that may differ depending on your static generator +configuration. + +If the case of `404.html`, there are different scenarios. For example: + +- If you use project Pages (served under `/projectname/`) and try to access + `/projectname/non/existing_file`, GitLab Pages will try to serve first + `/projectname/404.html`, and then `/404.html`. +- If you use user/group Pages (served under `/`) and try to access + `/non/existing_file` GitLab Pages will try to serve `/404.html`. +- If you use a custom domain and try to access `/non/existing_file`, GitLab + Pages will try to serve only `/404.html`. + +## Redirects in GitLab Pages + +Since you cannot use any custom server configuration files, like `.htaccess` or +any `.conf` file, if you want to redirect a page to another +location, you can use the [HTTP meta refresh tag][metarefresh]. + +Some static site generators provide plugins for that functionality so that you +don't have to create and edit HTML files manually. For example, Jekyll has the +[redirect-from plugin](https://github.com/jekyll/jekyll-redirect-from). + +## GitLab Pages Access Control **[CORE ONLY]** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/33422) in GitLab 11.5. + +NOTE: **Note:** +GitLab Pages access control is not activated on GitLab.com. You can check its +progress on the +[infrastructure issue tracker](https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/5576). + +You can enable Pages access control on your project, so that only +[members of your project](../../permissions.md#project-members-permissions) +(at least Guest) can access your website: + +1. Navigate to your project's **Settings > General > Permissions**. +1. Toggle the **Pages** button to enable the access control. + + NOTE: **Note:** + If you don't see the toggle button, that means that it's not enabled. + Ask your administrator to [enable it](../../../administration/pages/index.md#access-control). + +1. The Pages access control dropdown allows you to set who can view pages hosted + with GitLab Pages, depending on your project's visibility: + + - If your project is private: + - **Only project members**: Only project members will be able to browse the website. + - **Everyone**: Everyone, both logged into and logged out of GitLab, will be able to browse the website, no matter their project membership. + - If your project is internal: + - **Only project members**: Only project members will be able to browse the website. + - **Everyone with access**: Everyone logged into GitLab will be able to browse the website, no matter their project membership. + - **Everyone**: Everyone, both logged into and logged out of GitLab, will be able to browse the website, no matter their project membership. + - If your project is public: + - **Only project members**: Only project members will be able to browse the website. + - **Everyone with access**: Everyone, both logged into and logged out of GitLab, will be able to browse the website, no matter their project membership. + +1. Click **Save changes**. + +--- + +The next time someone tries to access your website and the access control is +enabled, they will be presented with a page to sign into GitLab and verify they +can access the website. + +## Unpublishing your Pages + +If you ever feel the need to purge your Pages content, you can do so by going +to your project's settings through the gear icon in the top right, and then +navigating to **Pages**. Hit the **Remove pages** button and your Pages website +will be deleted. + + + +## Limitations + +When using Pages under the general domain of a GitLab instance (`*.example.io`), +you _cannot_ use HTTPS with sub-subdomains. That means that if your +username/groupname contains a dot, for example `foo.bar`, the domain +`https://foo.bar.example.io` will _not_ work. This is a limitation of the +[HTTP Over TLS protocol][rfc]. HTTP pages will continue to work provided you +don't redirect HTTP to HTTPS. + +[rfc]: https://tools.ietf.org/html/rfc2818#section-3.1 "HTTP Over TLS RFC" + +GitLab Pages [does **not** support group websites for subgroups](../../group/subgroups/index.md#limitations). +You can only create the highest-level group website. + ## Specific configuration options for Pages Learn how to set up GitLab CI/CD for specific use cases. @@ -208,99 +306,6 @@ NOTE: **Note:** When `public/data/index.html` exists, it takes priority over the `public/data.html` file for both the `/data` and `/data/` URL paths. -### Custom error codes pages - -You can provide your own 403 and 404 error pages by creating the `403.html` and -`404.html` files respectively in the root directory of the `public/` directory -that will be included in the artifacts. Usually this is the root directory of -your project, but that may differ depending on your static generator -configuration. - -If the case of `404.html`, there are different scenarios. For example: - -- If you use project Pages (served under `/projectname/`) and try to access - `/projectname/non/existing_file`, GitLab Pages will try to serve first - `/projectname/404.html`, and then `/404.html`. -- If you use user/group Pages (served under `/`) and try to access - `/non/existing_file` GitLab Pages will try to serve `/404.html`. -- If you use a custom domain and try to access `/non/existing_file`, GitLab - Pages will try to serve only `/404.html`. - -### Redirects in GitLab Pages - -Since you cannot use any custom server configuration files, like `.htaccess` or -any `.conf` file, if you want to redirect a page to another -location, you can use the [HTTP meta refresh tag][metarefresh]. - -Some static site generators provide plugins for that functionality so that you -don't have to create and edit HTML files manually. For example, Jekyll has the -[redirect-from plugin](https://github.com/jekyll/jekyll-redirect-from). - -### GitLab Pages Access Control **[CORE ONLY]** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/33422) in GitLab 11.5. - -NOTE: **Note:** -GitLab Pages access control is not activated on GitLab.com. You can check its -progress on the -[infrastructure issue tracker](https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/5576). - -You can enable Pages access control on your project, so that only -[members of your project](../../permissions.md#project-members-permissions) -(at least Guest) can access your website: - -1. Navigate to your project's **Settings > General > Permissions**. -1. Toggle the **Pages** button to enable the access control. - - NOTE: **Note:** - If you don't see the toggle button, that means that it's not enabled. - Ask your administrator to [enable it](../../../administration/pages/index.md#access-control). - -1. The Pages access control dropdown allows you to set who can view pages hosted - with GitLab Pages, depending on your project's visibility: - - - If your project is private: - - **Only project members**: Only project members will be able to browse the website. - - **Everyone**: Everyone, both logged into and logged out of GitLab, will be able to browse the website, no matter their project membership. - - If your project is internal: - - **Only project members**: Only project members will be able to browse the website. - - **Everyone with access**: Everyone logged into GitLab will be able to browse the website, no matter their project membership. - - **Everyone**: Everyone, both logged into and logged out of GitLab, will be able to browse the website, no matter their project membership. - - If your project is public: - - **Only project members**: Only project members will be able to browse the website. - - **Everyone with access**: Everyone, both logged into and logged out of GitLab, will be able to browse the website, no matter their project membership. - -1. Click **Save changes**. - ---- - -The next time someone tries to access your website and the access control is -enabled, they will be presented with a page to sign into GitLab and verify they -can access the website. - -## Unpublishing your Pages - -If you ever feel the need to purge your Pages content, you can do so by going -to your project's settings through the gear icon in the top right, and then -navigating to **Pages**. Hit the **Remove pages** button and your Pages website -will be deleted. - - - -## Limitations - -When using Pages under the general domain of a GitLab instance (`*.example.io`), -you _cannot_ use HTTPS with sub-subdomains. That means that if your -username/groupname contains a dot, for example `foo.bar`, the domain -`https://foo.bar.example.io` will _not_ work. This is a limitation of the -[HTTP Over TLS protocol][rfc]. HTTP pages will continue to work provided you -don't redirect HTTP to HTTPS. - -[rfc]: https://tools.ietf.org/html/rfc2818#section-3.1 "HTTP Over TLS RFC" - -GitLab Pages [does **not** support group websites for subgroups](../../group/subgroups/index.md#limitations). -You can only create the highest-level group website. - ## Frequently Asked Questions ### Can I download my generated pages? diff --git a/doc/user/project/pages/lets_encrypt_for_gitlab_pages.md b/doc/user/project/pages/lets_encrypt_for_gitlab_pages.md index da1b7c59c8e..91a660c0f7a 100644 --- a/doc/user/project/pages/lets_encrypt_for_gitlab_pages.md +++ b/doc/user/project/pages/lets_encrypt_for_gitlab_pages.md @@ -1,5 +1,7 @@ --- description: "How to secure GitLab Pages websites with Let's Encrypt." +type: howto +last_updated: 2019-06-04 --- # Let's Encrypt for GitLab Pages diff --git a/doc/user/project/pipelines/job_artifacts.md b/doc/user/project/pipelines/job_artifacts.md index 629b5e1fde4..002addfc043 100644 --- a/doc/user/project/pipelines/job_artifacts.md +++ b/doc/user/project/pipelines/job_artifacts.md @@ -16,7 +16,7 @@ > [administration/job_artifacts](../../../administration/job_artifacts.md). Artifacts is a list of files and directories which are attached to a job -after it completes successfully. This feature is enabled by default in all +after it finishes. This feature is enabled by default in all GitLab installations. ## Defining artifacts in `.gitlab-ci.yml` @@ -36,12 +36,14 @@ pdf: A job named `pdf` calls the `xelatex` command in order to build a pdf file from the latex source file `mycv.tex`. We then define the `artifacts` paths which in turn are defined with the `paths` keyword. All paths to files and directories -are relative to the repository that was cloned during the build. These uploaded -artifacts will be kept in GitLab for 1 week as defined by the `expire_in` -definition. You have the option to keep the artifacts from expiring via the -[web interface](#browsing-artifacts). If the expiry time is not defined, -it defaults to the [instance wide -setting](../../admin_area/settings/continuous_integration.md#default-artifacts-expiration-core-only). +are relative to the repository that was cloned during the build. + +The artifacts will be uploaded when the job succeeds by default, but can be set to upload +when the job fails, or always, if the [`artifacts:when`](../../../ci/yaml/README.md#artifactswhen) +parameter is used. These uploaded artifacts will be kept in GitLab for 1 week as defined +by the `expire_in` definition. You have the option to keep the artifacts from expiring +via the [web interface](#browsing-artifacts). If the expiry time is not defined, it defaults +to the [instance wide setting](../../admin_area/settings/continuous_integration.md#default-artifacts-expiration-core-only). For more examples on artifacts, follow the [artifacts reference in `.gitlab-ci.yml`](../../../ci/yaml/README.md#artifacts). diff --git a/doc/user/project/pipelines/settings.md b/doc/user/project/pipelines/settings.md index 8b762307ac4..16f48c462eb 100644 --- a/doc/user/project/pipelines/settings.md +++ b/doc/user/project/pipelines/settings.md @@ -20,6 +20,22 @@ There are two options. Using: The default Git strategy can be overridden by the [GIT_STRATEGY variable](../../../ci/yaml/README.md#git-strategy) in `.gitlab-ci.yml`. +## Git shallow clone + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/28919) in GitLab 12.0. + +NOTE: **Note**: As of GitLab 12.0, newly created projects will automaticallyl have a default +`git depth` value of `50`. + +It is possible to limit the number of changes that GitLab CI/CD will fetch when cloning +a repository. Setting a limit to `git depth` can speed up Pipelines execution. Maximum +allowed value is `1000`. + +To disable shallow clone and make GitLab CI/CD fetch all branches and tags each time, +keep the value empty or set to `0`. + +This value can also be [overridden by `GIT_DEPTH`](../../../ci/large_repositories/index.md#shallow-cloning) variable in `.gitlab-ci.yml` file. + ## Timeout Timeout defines the maximum amount of time in minutes that a job is able run. diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md index 15eb862b431..1d640966013 100644 --- a/doc/user/project/quick_actions.md +++ b/doc/user/project/quick_actions.md @@ -35,7 +35,7 @@ discussions, and descriptions: | `/label ~label1 ~label2` | Add label(s). Label names can also start without ~ but mixed syntax is not supported. | ✓ | ✓ | | `/unlabel ~label1 ~label2` | Remove all or specific label(s)| ✓ | ✓ | | `/relabel ~label1 ~label2` | Replace label | ✓ | ✓ | -| <code>/copy_metadata #issue | !merge_request</code> | Copy labels and milestone from other issue or merge request | ✓ | ✓ | +| <code>/copy_metadata #issue | !merge_request</code> | Copy labels and milestone from other issue or merge request in the project | ✓ | ✓ | | <code>/estimate <1w 3d 2h 14m></code> | Set time estimate | ✓ | ✓ | | `/remove_estimate` | Remove time estimate | ✓ | ✓ | | <code>/spend <time(1h 30m | -1h 5m)> <date(YYYY-MM-DD)></code> | Add or subtract spent time; optionally, specify the date that time was spent on | ✓ | ✓ | diff --git a/doc/user/project/repository/index.md b/doc/user/project/repository/index.md index 97ecc4c0d65..6fccfd40987 100644 --- a/doc/user/project/repository/index.md +++ b/doc/user/project/repository/index.md @@ -123,7 +123,7 @@ You can live preview changes submitted to a new branch with [Review Apps](../../../ci/review_apps/index.md). With [GitLab Starter](https://about.gitlab.com/pricing/), you can also request -[approval](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html) from your managers. +[approval](../merge_requests/merge_request_approvals.md) from your managers. To create, delete, and view [branches](branches/index.md) via GitLab's UI: @@ -154,7 +154,7 @@ Via command line, you can commit multiple times before pushing. you will trigger a pipeline per push, not per commit. - **Skip pipelines:** You can add to you commit message the keyword - [`[ci skip]`](../../../ci/yaml/README.html#skipping-jobs) + [`[ci skip]`](../../../ci/yaml/README.md#skipping-jobs) and GitLab CI will skip that pipeline. - **Cross-link issues and merge requests:** [Cross-linking](../issues/crosslinking_issues.md#from-commit-messages) @@ -173,11 +173,15 @@ Via command line, you can commit multiple times before pushing. ## Repository size -On GitLab.com, your [repository size limit is 10GB](../../gitlab_com/index.md#repository-size-limit) -(including LFS). For other instances, the repository size is limited by your -system administrators. +A project's repository size is reported on the project's **Details** page. The reported size is +updated every 15 minutes at most, so may not reflect recent activity. -You can also [reduce a repository size using Git](reducing_the_repo_size_using_git.md). +The repository size for: + +- GitLab.com [is set by GitLab](../../gitlab_com/index.md#repository-size-limit). +- Self-managed instances is set by your GitLab administrators. + +You can [reduce a repository's size using Git](reducing_the_repo_size_using_git.md). ## Contributors @@ -222,7 +226,7 @@ Find it under your project's **Repository > Compare**. ## Locked files **[PREMIUM]** -Use [File Locking](https://docs.gitlab.com/ee/user/project/file_lock.html) to +Use [File Locking](../file_lock.md) to lock your files to prevent any conflicting changes. ## Repository's API diff --git a/doc/user/project/repository/reducing_the_repo_size_using_git.md b/doc/user/project/repository/reducing_the_repo_size_using_git.md index 2339759ecc8..e3d771524ce 100644 --- a/doc/user/project/repository/reducing_the_repo_size_using_git.md +++ b/doc/user/project/repository/reducing_the_repo_size_using_git.md @@ -1,6 +1,6 @@ # Reducing the repository size using Git -A GitLab Enterprise Edition administrator can set a [repository size limit][admin-repo-size] +A GitLab Enterprise Edition administrator can set a [repository size limit](../../admin_area/settings/account_and_limit_settings.md) which will prevent you from exceeding it. When a project has reached its size limit, you will not be able to push to it, @@ -14,7 +14,8 @@ move some blobs to LFS, or remove some old dependency updates from history. Unfortunately, it's not so easy and that workflow won't work. Deleting files in a commit doesn't actually reduce the size of the repo since the earlier commits and blobs are still around. What you need to do is rewrite history with Git's -[`filter-branch` option][gitscm], or a tool like the [BFG Repo-Cleaner][bfg]. +[`filter-branch` option](https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History#The-Nuclear-Option:-filter-branch), +or a tool like the [BFG Repo-Cleaner](https://rtyley.github.io/bfg-repo-cleaner/). Note that even with that method, until `git gc` runs on the GitLab side, the "removed" commits and blobs will still be around. You also need to be able to @@ -137,7 +138,3 @@ remove some of them, but it should not be depended on for security purposes! ``` Your repository should now be below the size limit. - -[admin-repo-size]: https://docs.gitlab.com/ee/user/admin_area/settings/account_and_limit_settings.html#repository-size-limit -[bfg]: https://rtyley.github.io/bfg-repo-cleaner/ -[gitscm]: https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History#The-Nuclear-Option:-filter-branch diff --git a/doc/user/project/security_dashboard.md b/doc/user/project/security_dashboard.md index 43e910b29fe..a3da1ec97d3 100644 --- a/doc/user/project/security_dashboard.md +++ b/doc/user/project/security_dashboard.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/application_security/security_dashboard/index.html' +redirect_to: '../application_security/security_dashboard/index.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/user/application_security/security_dashboard/index.html). +This document was moved to [another location](../application_security/security_dashboard/index.md). diff --git a/doc/user/project/settings/img/import_export_download_export.png b/doc/user/project/settings/img/import_export_download_export.png Binary files differindex 668254073e8..ab81c87bf5f 100644 --- a/doc/user/project/settings/img/import_export_download_export.png +++ b/doc/user/project/settings/img/import_export_download_export.png diff --git a/doc/user/project/settings/img/import_export_export_button.png b/doc/user/project/settings/img/import_export_export_button.png Binary files differindex 7f21bb2335b..9e368739695 100644 --- a/doc/user/project/settings/img/import_export_export_button.png +++ b/doc/user/project/settings/img/import_export_export_button.png diff --git a/doc/user/project/settings/img/import_export_mail_link.png b/doc/user/project/settings/img/import_export_mail_link.png Binary files differindex 48ef42855bc..985c37650d3 100644 --- a/doc/user/project/settings/img/import_export_mail_link.png +++ b/doc/user/project/settings/img/import_export_mail_link.png diff --git a/doc/user/project/settings/img/import_export_new_project.png b/doc/user/project/settings/img/import_export_new_project.png Binary files differindex b335700c5be..fc1f73c5d6e 100644 --- a/doc/user/project/settings/img/import_export_new_project.png +++ b/doc/user/project/settings/img/import_export_new_project.png diff --git a/doc/user/project/settings/img/import_export_select_file.png b/doc/user/project/settings/img/import_export_select_file.png Binary files differindex e1e5e031d81..e3e1a5ef980 100644 --- a/doc/user/project/settings/img/import_export_select_file.png +++ b/doc/user/project/settings/img/import_export_select_file.png diff --git a/doc/user/project/settings/img/settings_edit_button.png b/doc/user/project/settings/img/settings_edit_button.png Binary files differdeleted file mode 100644 index 32bcda03c7e..00000000000 --- a/doc/user/project/settings/img/settings_edit_button.png +++ /dev/null diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index 89008fd15b9..819515d7a4c 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -2,10 +2,11 @@ >**Notes:** > -> - [Introduced][ce-3050] in GitLab 8.9. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/3050) in GitLab 8.9. > - Importing will not be possible if the import instance version differs from > that of the exporter. -> - For GitLab admins, please read through [Project import/export administration](../../../administration/raketasks/project_import_export.md). +> - For GitLab admins, please read through +> [Project import/export administration](../../../administration/raketasks/project_import_export.md). > - For existing installations, the project import option has to be enabled in > application settings (`/admin/application_settings`) under 'Import sources'. > Ask your administrator if you don't see the **GitLab export** button when @@ -14,15 +15,15 @@ > on the GitLab instance in application settings (`/admin/application_settings`) > under 'Visibility and Access Controls'. > - You can find some useful raketasks if you are an administrator in the -> [import_export](../../../administration/raketasks/project_import_export.md) -> raketask. -> - The exports are stored in a temporary [shared directory][tmp] and are deleted -> every 24 hours by a specific worker. +> [import_export](../../../administration/raketasks/project_import_export.md) raketask. +> - The exports are stored in a temporary [shared directory](../../../development/shared_files.md) +> and are deleted every 24 hours by a specific worker. > - Group members will get exported as project members, as long as the user has > maintainer or admin access to the group where the exported project lives. An admin > in the import side is required to map the users, based on email or username. > Otherwise, a supplementary comment is left to mention the original author and > the MRs, notes or issues will be owned by the importer. +> - Project members with owner access will get imported as maintainers. > - Control project Import/Export with the [API](../../../api/project_import_export.md). > - If an imported project contains merge requests originated from forks, > then new branches associated with such merge requests will be created @@ -76,9 +77,9 @@ The following items will NOT be exported: ## Exporting a project and its data -1. Go to the project settings page by clicking on **Edit Project**: +1. Go to your project's homepage. -  +1. Click **Settings** in the sidebar. 1. Scroll down to find the **Export project** button: @@ -97,19 +98,14 @@ The following items will NOT be exported: ## Importing the project -1. The new GitLab project import feature is at the far right of the import - options when creating a New Project. Make sure you are in the right namespace - and you have entered a project name. Click on **GitLab export**: +1. The GitLab project import feature is the first import option when creating a + new project. Click on **GitLab export**:  -1. You can see where the project will be imported to. You can now select file - exported previously: +1. Enter your project name and URL. Then select the file you exported previously:  1. Click on **Import project** to begin importing. Your newly imported project page will appear soon. - -[ce-3050]: https://gitlab.com/gitlab-org/gitlab-ce/issues/3050 -[tmp]: ../../../development/shared_files.md diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md index 99dd018a3ba..e3502a632d9 100644 --- a/doc/user/project/settings/index.md +++ b/doc/user/project/settings/index.md @@ -36,7 +36,7 @@ Set up your project's merge request settings: - Set up the merge request method (merge commit, [fast-forward merge](../merge_requests/fast_forward_merge.html)). - Merge request [description templates](../description_templates.md#description-templates). -- Enable [merge request approvals](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html#merge-request-approvals). **[STARTER]** +- Enable [merge request approvals](../merge_requests/merge_request_approvals.md). **[STARTER]** - Enable [merge only of pipeline succeeds](../merge_requests/merge_when_pipeline_succeeds.md). - Enable [merge only when all discussions are resolved](../../discussions/index.md#only-allow-merge-requests-to-be-merged-if-all-discussions-are-resolved). @@ -44,7 +44,7 @@ Set up your project's merge request settings: ### Service Desk **[PREMIUM]** -Enable [Service Desk](https://docs.gitlab.com/ee/user/project/service_desk.html) for your project to offer customer support. +Enable [Service Desk](../service_desk.md) for your project to offer customer support. ### Export project @@ -100,9 +100,9 @@ Only project Owners and Admin users have the [permissions] to transfer a project You can transfer an existing project into a [group](../../group/index.md) if: -1. you have at least **Maintainer** [permissions] to that group -1. you are an **Owner** of the project. - +1. You have at least **Maintainer** [permissions] to that group. +1. The project is in a subgroup you own. +1. You are at least a **Maintainer** of the project under your personal namespace. Similarly, if you are an owner of a group, you can transfer any of its projects under your own user. diff --git a/doc/user/project/web_ide/index.md b/doc/user/project/web_ide/index.md index 2a2507d98a3..a634a8b2f54 100644 --- a/doc/user/project/web_ide/index.md +++ b/doc/user/project/web_ide/index.md @@ -8,7 +8,7 @@ projects by providing an advanced editor with commit staging. ## Open the Web IDE -The Web IDE can be opened when viewing a file, from the repository file list, +You can open the Web IDE when viewing a file, from the repository file list, and from merge requests.  @@ -45,7 +45,7 @@ Single file editing is based on the [Ace Editor](https://ace.c9.io). ## Stage and commit changes -After making your changes, click the Commit button in the bottom left to +After making your changes, click the **Commit** button in the bottom left to review the list of changed files. Click on each file to review the changes and click the tick icon to stage the file. @@ -67,10 +67,11 @@ shows you a preview of the merge request diff if you commit your changes. > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/19279) in [GitLab Core][ce] 11.0. -The Web IDE can be used to quickly fix failing tests by opening the branch or -merge request in the Web IDE and opening the logs of the failed job. The status -of all jobs for the most recent pipeline and job traces for the current commit -can be accessed by clicking the **Pipelines** button in the top right. +You can use the Web IDE to quickly fix failing tests by opening +the branch or merge request in the Web IDE and opening the logs of the failed +job. You can access the status of all jobs for the most recent pipeline and job +traces for the current commit by clicking the **Pipelines** button in the top +right. The pipeline status is also shown at all times in the status bar in the bottom left. @@ -79,31 +80,31 @@ left. > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/19318) in [GitLab Core][ce] 11.0. -Switching between your authored and assigned merge requests can be done without -leaving the Web IDE. Click the dropdown in the top of the sidebar to open a list -of merge requests. You will need to commit or discard all your changes before -switching to a different merge request. +To switch between your authored and assigned merge requests, click the +dropdown in the top of the sidebar to open a list of merge requests. You will +need to commit or discard all your changes before switching to a different merge +request. ## Switching branches > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/20850) in [GitLab Core][ce] 11.2. -Switching between branches of the current project repository can be done without -leaving the Web IDE. Click the dropdown in the top of the sidebar to open a list -of branches. You will need to commit or discard all your changes before -switching to a different branch. +To switch between branches of the current project repository, click the dropdown +in the top of the sidebar to open a list of branches. +You will need to commit or discard all your changes before switching to a +different branch. ## Client Side Evaluation > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/19764) in [GitLab Core][ce] 11.2. -The Web IDE can be used to preview JavaScript projects right in the browser. +You can use the Web IDE to preview JavaScript projects right in the browser. This feature uses CodeSandbox to compile and bundle the JavaScript used to preview the web application.  -Additionally, for public projects an `Open in CodeSandbox` button is available +Additionally, for public projects an **Open in CodeSandbox** button is available to transfer the contents of the project into a public CodeSandbox project to quickly share your project with others. @@ -115,9 +116,9 @@ GitLab.com  -Once it has been enabled in application settings, projects with a -`package.json` file and a `main` entry point can be previewed inside of the Web -IDE. An example `package.json` is below. +Once you have done that, you can preview projects with a `package.json` file and +a `main` entry point inside the Web IDE. An example `package.json` is shown +below. ```json { diff --git a/doc/user/search/advanced_global_search.md b/doc/user/search/advanced_global_search.md index 38b26f31417..f80f4183802 100644 --- a/doc/user/search/advanced_global_search.md +++ b/doc/user/search/advanced_global_search.md @@ -2,7 +2,7 @@ > - [Introduced][ee-109] in GitLab [Starter][ee] 8.4. > - This is the user documentation. To install and configure Elasticsearch, -> visit the [administrator documentation](https://docs.gitlab.com/ee/integration/elasticsearch.html). +> visit the [administrator documentation](../../integration/elasticsearch.md). NOTE: **Note** Advanced Global Search (powered by Elasticsearch) is not yet available on GitLab.com. We are working on adding it. [Follow this epic for the latest updates](https://gitlab.com/groups/gitlab-org/-/epics/153). diff --git a/doc/user/search/advanced_search_syntax.md b/doc/user/search/advanced_search_syntax.md index 8d4aac5f502..d302cb7a809 100644 --- a/doc/user/search/advanced_search_syntax.md +++ b/doc/user/search/advanced_search_syntax.md @@ -3,7 +3,7 @@ > **Notes:** > - Introduced in [GitLab Enterprise Starter][ee] 9.2 > - This is the user documentation. To install and configure Elasticsearch, -> visit the [administrator documentation](https://docs.gitlab.com/ee/integration/elasticsearch.html). +> visit the [administrator documentation](../../integration/elasticsearch.md). NOTE: **Note** Advanced Global Search (powered by Elasticsearch) is not yet available on GitLab.com. We are working on adding it. [Follow this epic for the latest updates](https://gitlab.com/groups/gitlab-org/-/epics/153). diff --git a/doc/workflow/README.md b/doc/workflow/README.md index 84ab7840140..40e2486ace5 100644 --- a/doc/workflow/README.md +++ b/doc/workflow/README.md @@ -13,15 +13,15 @@ comments: false - [Groups](../user/group/index.md) - Issues - The GitLab Issue Tracker is an advanced and complete tool for tracking the evolution of a new idea or the process of solving a problem. - - [Exporting Issues](https://docs.gitlab.com/ee/user/project/issues/csv_export.html) **[STARTER]** Export issues as a CSV, emailed as an attachment. + - [Exporting Issues](../user/project/issues/csv_export.md) **[STARTER]** Export issues as a CSV, emailed as an attachment. - [Confidential issues](../user/project/issues/confidential_issues.md) - [Due date for issues](../user/project/issues/due_dates.md) - [Issue Board](../user/project/issue_board.md) - [Keyboard shortcuts](shortcuts.md) - [File finder](file_finder.md) -- [File lock](https://docs.gitlab.com/ee/user/project/file_lock.html) **[PREMIUM]** +- [File lock](../user/project/file_lock.md) **[PREMIUM]** - [Labels](../user/project/labels.md) -- [Issue weight](https://docs.gitlab.com/ee/workflow/issue_weight.html) **[STARTER]** +- [Issue weight](issue_weight.md) **[STARTER]** - [Notification emails](notifications.md) - [Projects](../user/project/index.md) - [Project forking workflow](forking_workflow.md) @@ -44,9 +44,9 @@ comments: false - [Merge requests versions](../user/project/merge_requests/versions.md) - ["Work In Progress" merge requests](../user/project/merge_requests/work_in_progress_merge_requests.md) - [Fast-forward merge requests](../user/project/merge_requests/fast_forward_merge.md) - - [Merge request approvals](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html) **[STARTER]** + - [Merge request approvals](../user/project/merge_requests/merge_request_approvals.md) **[STARTER]** - [Repository mirroring](repository_mirroring.md) **[STARTER]** -- [Service Desk](https://docs.gitlab.com/ee/user/project/service_desk.html) **[PREMIUM]** +- [Service Desk](../user/project/service_desk.md) **[PREMIUM]** - [Manage large binaries with Git LFS](lfs/manage_large_binaries_with_git_lfs.md) - [Importing from SVN, GitHub, Bitbucket, etc](importing/README.md) - [Todos](todos.md) diff --git a/doc/workflow/gitlab_flow.md b/doc/workflow/gitlab_flow.md index 1b9fb504b15..7d0abb93262 100644 --- a/doc/workflow/gitlab_flow.md +++ b/doc/workflow/gitlab_flow.md @@ -166,7 +166,7 @@ This branch is the place for any work related to this change. NOTE: **Note:** The name of a branch might be dictated by organizational standards. -For example, in GitLab, any branches in GitLab EE that are equivalent to branches in GitLab CE [must end in `-ee`](https://docs.gitlab.com/ee/development/automatic_ce_ee_merge.html#cherry-picking-from-ce-to-ee). +For example, in GitLab, any branches in GitLab EE that are equivalent to branches in GitLab CE [must end in `-ee`](../development/automatic_ce_ee_merge.md#cherry-picking-from-ce-to-ee). When you are done or want to discuss the code, open a merge request. A merge request is an online place to discuss the change and review the code. diff --git a/doc/workflow/img/notification_global_settings.png b/doc/workflow/img/notification_global_settings.png Binary files differindex 8a5494d16a8..72f7418f1f8 100644 --- a/doc/workflow/img/notification_global_settings.png +++ b/doc/workflow/img/notification_global_settings.png diff --git a/doc/workflow/img/repository_mirroring_pull_settings_upper.png b/doc/workflow/img/repository_mirroring_pull_settings_upper.png Binary files differindex c60354fdca7..8e15b5a0784 100644 --- a/doc/workflow/img/repository_mirroring_pull_settings_upper.png +++ b/doc/workflow/img/repository_mirroring_pull_settings_upper.png diff --git a/doc/workflow/lfs/migrate_from_git_annex_to_git_lfs.md b/doc/workflow/lfs/migrate_from_git_annex_to_git_lfs.md index 1aeab5980a3..71c73e3dffe 100644 --- a/doc/workflow/lfs/migrate_from_git_annex_to_git_lfs.md +++ b/doc/workflow/lfs/migrate_from_git_annex_to_git_lfs.md @@ -42,7 +42,7 @@ Annex to Git LFS. ### TL; DR If you know what you are doing and want to skip the reading, this is what you -need to do (we assume you have [git-annex enabled][annex-gitlab-use] in your +need to do (we assume you have [git-annex enabled](../git_annex.md#using-gitlab-git-annex) in your repository and that you have made backups in case something goes wrong). Fire up a terminal, navigate to your Git repository and: @@ -82,7 +82,7 @@ Make sure the you read about the [`direct` mode][annex-direct] as it contains useful information that may fit in your use case. Note that `annex direct` is deprecated in Git Annex version 6, so you may need to upgrade your repository if the server also has Git Annex 6 installed. Read more in the -[Git Annex troubleshooting tips][annex-tips] section. +[Git Annex troubleshooting tips](../git_annex.md#troubleshooting-tips) section. 1. Backup your repository @@ -166,44 +166,45 @@ GitLab.com), therefore, you don't need to do anything server-side. 1. First, make sure you have `git-lfs` installed locally: - ```bash - git lfs help - ``` + ```bash + git lfs help + ``` - If the terminal doesn't prompt you with a full response on `git-lfs` commands, - [install the Git LFS client][install-lfs] first. + If the terminal doesn't prompt you with a full response on `git-lfs` commands, + [install the Git LFS client][install-lfs] first. 1. Inside the repo, run the following command to initiate LFS: - ```bash - git lfs install - ``` + ```bash + git lfs install + ``` 1. Enable `git-lfs` for the group of files you want to track. You can track specific files, all files containing the same extension, or an entire directory: - ```bash - git lfs track images/01.png # per file - git lfs track **/*.png # per extension - git lfs track images/ # per directory - ``` + ```bash + git lfs track images/01.png # per file + git lfs track **/*.png # per extension + git lfs track images/ # per directory + ``` - Once you do that, run `git status` and you'll see `.gitattributes` added - to your repo. It collects all file patterns that you chose to track via - `git-lfs`. + Once you do that, run `git status` and you'll see `.gitattributes` added + to your repo. It collects all file patterns that you chose to track via + `git-lfs`. 1. Add the files, commit and push them to GitLab: - ```bash - git add . - git commit -m "commit message" - git push - ``` + ```bash + git add . + git commit -m "commit message" + git push + ``` - If your remote is set up with HTTP, you will be asked to enter your login - credentials. If you have [2FA enabled][2fa], make sure to use a - [personal access token][token] instead of your password. + If your remote is set up with HTTP, you will be asked to enter your login + credentials. If you have [2FA enabled](../../user/profile/account/two_factor_authentication.md), make sure to use a + [personal access token](../../user/profile/account/two_factor_authentication.md#personal-access-tokens) + instead of your password. ## Removing the Git Annex branches @@ -238,18 +239,11 @@ git annex uninit - (Blog Post) [Getting Started with Git FLS][post-1] - (Blog Post) [Announcing LFS Support in GitLab][post-2] - (Blog Post) [GitLab Annex Solves the Problem of Versioning Large Binaries with Git][post-3] -- (GitLab Docs) [Git Annex][doc-1] -- (GitLab Docs) [Git LFS][doc-2] +- (GitLab Docs) [Git Annex](../git_annex.md) +- (GitLab Docs) [Git LFS](manage_large_binaries_with_git_lfs.md) -[2fa]: ../../user/profile/account/two_factor_authentication.md -[token]: ../../user/profile/account/two_factor_authentication.html#personal-access-tokens -[annex-tips]: ../git_annex.html#troubleshooting-tips [annex-direct]: https://git-annex.branchable.com/direct_mode/ -[annex-gitlab-use]: ../git_annex.md#using-gitlab-git-annex -[annex-ee]: https://docs.gitlab.com/ee/workflow/git_annex.html [bkp-ext-drive]: https://www.thomas-krenn.com/en/wiki/Git-annex_Repository_on_an_External_Hard_Drive -[doc-1]: https://docs.gitlab.com/ee/workflow/git_annex.html -[doc-2]: https://docs.gitlab.com/ee/workflow/lfs/manage_large_binaries_with_git_lfs.html [Git Annex]: http://git-annex.branchable.com/ [Git LFS]: https://git-lfs.github.com/ [install-lfs]: https://git-lfs.github.com/ diff --git a/doc/workflow/merge_request_approvals.md b/doc/workflow/merge_request_approvals.md index f8a99ec3d57..bfcd8faf236 100644 --- a/doc/workflow/merge_request_approvals.md +++ b/doc/workflow/merge_request_approvals.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html' +redirect_to: '../user/project/merge_requests/merge_request_approvals.md' --- -This document was moved to [user/project/merge_requests/merge_request_approvals](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html). +This document was moved to [another location](../user/project/merge_requests/merge_request_approvals.md). diff --git a/doc/workflow/notifications.md b/doc/workflow/notifications.md index 0eb4f85e0ec..5d560f2e000 100644 --- a/doc/workflow/notifications.md +++ b/doc/workflow/notifications.md @@ -16,12 +16,16 @@ Notification settings are divided into three groups: Each of these settings have levels of notification: +- Global: For groups and projects, notifications as per global settings. - Watch: Receive notifications for any activity. -- On Mention: Receive notifications when `@mentioned` in comments. - Participate: Receive notifications for threads you have participated in. +- On Mention: Receive notifications when `@mentioned` in comments. - Disabled: Turns off notifications. - Custom: Receive notifications for custom selected events. -- Global: For groups and projects, notifications as per global settings. + +> Introduced in GitLab 12.0 + +You can also select an email address to receive notifications for each group you belong to. ### Global Settings diff --git a/doc/workflow/repository_mirroring.md b/doc/workflow/repository_mirroring.md index 2f8f1545b84..9772bd385ba 100644 --- a/doc/workflow/repository_mirroring.md +++ b/doc/workflow/repository_mirroring.md @@ -99,6 +99,7 @@ The repository will push soon. To force a push, click the appropriate button. ## Pulling from a remote repository **[STARTER]** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/51) in GitLab Enterprise Edition 8.2. +> [Added Git LFS support](https://gitlab.com/gitlab-org/gitlab-ee/issues/10871) in [GitLab Starter](https://about.gitlab.com/pricing/) 11.11. You can set up a repository to automatically have its branches, tags, and commits updated from an upstream repository. @@ -282,10 +283,10 @@ project mirroring again by [Forcing an update](#forcing-an-update-core). [GitLab Starter](https://about.gitlab.com/pricing/) 10.3. Pull mirroring uses polling to detect new branches and commits added upstream, often minutes -afterwards. If you notify GitLab by [API](https://docs.gitlab.com/ee/api/projects.html#start-the-pull-mirroring-process-for-a-project), +afterwards. If you notify GitLab by [API](https://docs.gitlab.com/ee/api/projects.html#start-the-pull-mirroring-process-for-a-project-starter), updates will be pulled immediately. -For more information, see [Start the pull mirroring process for a Project](https://docs.gitlab.com/ee/api/projects.html#start-the-pull-mirroring-process-for-a-project). +For more information, see [Start the pull mirroring process for a Project](https://docs.gitlab.com/ee/api/projects.html#start-the-pull-mirroring-process-for-a-project-starter). ## Forcing an update **[CORE]** diff --git a/doc/workflow/shortcuts.md b/doc/workflow/shortcuts.md index 6714d08f2f7..5068b5d4d20 100644 --- a/doc/workflow/shortcuts.md +++ b/doc/workflow/shortcuts.md @@ -1,6 +1,6 @@ # GitLab keyboard shortcuts -You can see GitLab's keyboard shortcuts by using 'shift + ?' +You can see GitLab's keyboard shortcuts by using <kbd>shift</kbd> + <kbd>?</kbd> ## Global Shortcuts diff --git a/doc/workflow/timezone.md b/doc/workflow/timezone.md index 338b3a32265..da51c0f2c93 100644 --- a/doc/workflow/timezone.md +++ b/doc/workflow/timezone.md @@ -1,31 +1,39 @@ # Changing your time zone The global time zone configuration parameter can be changed in `config/gitlab.yml`: + ``` - # time_zone: 'UTC' +# time_zone: 'UTC' ``` -Uncomment and customize if you want to change the default time zone of GitLab application. +Uncomment and customize if you want to change the default time zone of the GitLab application. + + +## Viewing available timezones To see all available time zones, run `bundle exec rake time:zones:all`. -With Omnibus installations, run `gitlab-rake time:zones:all`. +For Omnibus installations, run `gitlab-rake time:zones:all`. + +NOTE: **Note:** +Currently, this rake task does not list timezones in TZInfo format required by GitLab Omnibus during a reconfigure: [#58672](https://gitlab.com/gitlab-org/gitlab-ce/issues/58672). ## Changing time zone in omnibus installations GitLab defaults its time zone to UTC. It has a global timezone configuration parameter in `/etc/gitlab/gitlab.rb`. -To update, add the time zone that best applies to your location. Here are two examples: +To obtain a list of timezones, log in to your GitLab application server and run a command that generates a list of timezones in TZInfo format for the server. For example, install `timedatectl` and run `timedatectl list-timezones`. + +To update, add the timezone that best applies to your location. For example: + ``` gitlab_rails['time_zone'] = 'America/New_York' ``` -or -``` -gitlab_rails['time_zone'] = 'Europe/Brussels' -``` -After you added this field, reconfigure and restart: +After adding the configuration parameter, reconfigure and restart your GitLab instance: + ``` gitlab-ctl reconfigure gitlab-ctl restart ``` + |
