diff options
Diffstat (limited to 'doc')
261 files changed, 4621 insertions, 2287 deletions
diff --git a/doc/administration/auth/google_secure_ldap.md b/doc/administration/auth/google_secure_ldap.md index 65a51fc4aa0..760af0cfd1a 100644 --- a/doc/administration/auth/google_secure_ldap.md +++ b/doc/administration/auth/google_secure_ldap.md @@ -27,7 +27,7 @@ The steps below cover: 'Entire domain (GitLab)' or 'Selected organizational units' for both 'Verify user credentials' and 'Read user information'. Select 'Add LDAP Client' - TIP: **Tip:** If you plan to use GitLab [LDAP Group Sync](https://docs.gitlab.com/ee/administration/auth/ldap-ee.html#group-sync) + TIP: **Tip:** If you plan to use GitLab [LDAP Group Sync](ldap-ee.md#group-sync) , turn on 'Read group information'.  diff --git a/doc/administration/compliance.md b/doc/administration/compliance.md index 9c13ff772b3..246addb6dc9 100644 --- a/doc/administration/compliance.md +++ b/doc/administration/compliance.md @@ -12,7 +12,7 @@ GitLab’s [security features](../security/README.md) may also help you meet rel |**[Email all users of a project, group, or entire server](../user/admin_area/settings/terms.md)**<br>An admin can email groups of users based on project or group membership, or email everyone using the GitLab instance. This is great for scheduled maintenance or upgrades.|Starter+|| |**[Omnibus package supports log forwarding](https://docs.gitlab.com/omnibus/settings/logs.html#udp-log-forwarding)**<br>Forward your logs to a central system.|Starter+|| |**[Lock project membership to group](../user/group/index.md#member-lock-starter)**<br>Group owners can prevent new members from being added to projects within a group.|Starter+|✓| -|**[LDAP group sync](https://docs.gitlab.com/ee/administration/auth/ldap-ee.html#group-sync)**<br>GitLab Enterprise Edition gives admins the ability to automatically sync groups and manage SSH keys, permissions, and authentication, so you can focus on building your product, not configuring your tools.|Starter+|| -|**[LDAP group sync filters](https://docs.gitlab.com/ee/administration/auth/ldap-ee.html#group-sync)**<br>GitLab Enterprise Edition Premium gives more flexibility to synchronize with LDAP based on filters, meaning you can leverage LDAP attributes to map GitLab permissions.|Premium+|| +|**[LDAP group sync](auth/ldap-ee.md#group-sync)**<br>GitLab Enterprise Edition gives admins the ability to automatically sync groups and manage SSH keys, permissions, and authentication, so you can focus on building your product, not configuring your tools.|Starter+|| +|**[LDAP group sync filters](auth/ldap-ee.md#group-sync)**<br>GitLab Enterprise Edition Premium gives more flexibility to synchronize with LDAP based on filters, meaning you can leverage LDAP attributes to map GitLab permissions.|Premium+|| |**[Audit logs](audit_events.md)**<br>To maintain the integrity of your code, GitLab Enterprise Edition Premium gives admins the ability to view any modifications made within the GitLab server in an advanced audit log system, so you can control, analyze and track every change.|Premium+|| |**[Auditor users](auditor_users.md)**<br>Auditor users are users who are given read-only access to all projects, groups, and other resources on the GitLab instance.|Premium+|| diff --git a/doc/administration/custom_hooks.md b/doc/administration/custom_hooks.md index 288cb1bf0bb..113514e1ee8 100644 --- a/doc/administration/custom_hooks.md +++ b/doc/administration/custom_hooks.md @@ -5,11 +5,11 @@ Custom Git hooks must be configured on the filesystem of the GitLab server. Only GitLab server administrators will be able to complete these tasks. Please explore [webhooks] and [CI] as an option if you do not have filesystem access. For a user configurable Git hook interface, see -[Push Rules](https://docs.gitlab.com/ee/push_rules/push_rules.html), +[Push Rules](../push_rules/push_rules.md), available in GitLab Enterprise Edition. NOTE: **Note:** -Custom Git hooks won't be replicated to secondary nodes if you use [GitLab Geo][gitlab-geo] +Custom Git hooks won't be replicated to secondary nodes if you use [GitLab Geo](geo/replication/index.md) Git natively supports hooks that are executed on different actions. Examples of server-side git hooks include pre-receive, post-receive, and update. @@ -123,6 +123,5 @@ exit 1 [CI]: ../ci/README.md [hooks]: https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks#Server-Side-Hooks [webhooks]: ../user/project/integrations/webhooks.md -[gitlab-geo]: https://docs.gitlab.com/ee/administration/geo/replication/index.html [5073]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5073 [93]: https://gitlab.com/gitlab-org/gitlab-shell/merge_requests/93 diff --git a/doc/administration/geo/disaster_recovery/background_verification.md b/doc/administration/geo/disaster_recovery/background_verification.md index c7299b6e196..d4c8c2d3624 100644 --- a/doc/administration/geo/disaster_recovery/background_verification.md +++ b/doc/administration/geo/disaster_recovery/background_verification.md @@ -29,12 +29,7 @@ the node more time before scheduling a planned failover. Run the following commands in a Rails console on the **primary** node: ```sh -# Omnibus GitLab gitlab-rails console - -# Installation from source -cd /home/git/gitlab -sudo -u git -H bin/rails console RAILS_ENV=production ``` To check if automatic background verification is enabled: @@ -102,12 +97,7 @@ disable if you need. Run the following commands in a Rails console on the **primary** node: ```sh -# Omnibus GitLab gitlab-rails console - -# Installation from source -cd /home/git/gitlab -sudo -u git -H bin/rails console RAILS_ENV=production ``` To disable automatic background re-verification: @@ -131,31 +121,15 @@ to be resynced without the backoff period: For repositories: -- Omnibus Installation - - ```sh - sudo gitlab-rake geo:verification:repository:reset - ``` - -- Source Installation - - ```sh - sudo -u git -H bundle exec rake geo:verification:repository:reset RAILS_ENV=production - ``` +```sh +sudo gitlab-rake geo:verification:repository:reset +``` For wikis: -- Omnibus Installation - - ```sh - sudo gitlab-rake geo:verification:wiki:reset - ``` - -- Source Installation - - ```sh - sudo -u git -H bundle exec rake geo:verification:wiki:reset RAILS_ENV=production - ``` +```sh +sudo gitlab-rake geo:verification:wiki:reset +``` ## Reconcile differences with checksum mismatches @@ -169,7 +143,7 @@ If the **primary** and **secondary** nodes have a checksum verification mismatch 1. On the project admin page get the **Gitaly storage name**, and **Gitaly relative path**:  -1. Navigate to the project's repository directory on both **primary** and **secondary** nodes. For an installation from source, the path is usually `/home/git/repositories`. For Omnibus installs, the path is usually `/var/opt/gitlab/git-data/repositories`. Note that if `git_data_dirs` is customized, check the directory layout on your server to be sure. +1. Navigate to the project's repository directory on both **primary** and **secondary** nodes (the path is usually `/var/opt/gitlab/git-data/repositories`). Note that if `git_data_dirs` is customized, check the directory layout on your server to be sure. ```sh cd /var/opt/gitlab/git-data/repositories diff --git a/doc/administration/geo/disaster_recovery/planned_failover.md b/doc/administration/geo/disaster_recovery/planned_failover.md index 88ab12d910a..b8071b5993f 100644 --- a/doc/administration/geo/disaster_recovery/planned_failover.md +++ b/doc/administration/geo/disaster_recovery/planned_failover.md @@ -113,7 +113,7 @@ If any objects are failing to replicate, this should be investigated before scheduling the maintenance window. Following a planned failover, anything that failed to replicate will be **lost**. -You can use the [Geo status API](https://docs.gitlab.com/ee/api/geo_nodes.html#retrieve-project-sync-or-verification-failures-that-occurred-on-the-current-node) to review failed objects and +You can use the [Geo status API](../../../api/geo_nodes.md#retrieve-project-sync-or-verification-failures-that-occurred-on-the-current-node) to review failed objects and the reasons for failure. A common cause of replication failures is the data being missing on the diff --git a/doc/administration/geo/replication/configuration.md b/doc/administration/geo/replication/configuration.md index 57735b21cda..3d4f69d3abe 100644 --- a/doc/administration/geo/replication/configuration.md +++ b/doc/administration/geo/replication/configuration.md @@ -1,9 +1,4 @@ -# Geo configuration (GitLab Omnibus) **[PREMIUM ONLY]** - -NOTE: **Note:** -This is the documentation for the Omnibus GitLab packages. For installations -from source, follow the [**Geo nodes configuration for installations -from source**][configuration-source] guide. +# Geo configuration **[PREMIUM ONLY]** ## Configuring a new **secondary** node @@ -303,7 +298,6 @@ See the [updating the Geo nodes document](updating_the_geo_nodes.md). See the [troubleshooting document](troubleshooting.md). -[configuration-source]: configuration_source.md [setup-geo-omnibus]: index.md#using-omnibus-gitlab [Hashed Storage]: ../../repository_storage_types.md [Disaster Recovery]: ../disaster_recovery/index.md diff --git a/doc/administration/geo/replication/configuration_source.md b/doc/administration/geo/replication/configuration_source.md deleted file mode 100644 index 10dd9405b4b..00000000000 --- a/doc/administration/geo/replication/configuration_source.md +++ /dev/null @@ -1,172 +0,0 @@ -# Geo configuration (source) **[PREMIUM ONLY]** - -NOTE: **Note:** -This documentation applies to GitLab source installations. In GitLab 11.5, this documentation was deprecated and will be removed in a future release. -Please consider [migrating to GitLab Omnibus install](https://docs.gitlab.com/omnibus/update/convert_to_omnibus.html). For installations -using the Omnibus GitLab packages, follow the -[**Omnibus Geo nodes configuration**][configuration] guide. - -## Configuring a new **secondary** node - -NOTE: **Note:** -This is the final step in setting up a **secondary** node. Stages of the setup -process must be completed in the documented order. Before attempting the steps -in this stage, [complete all prior stages](index.md#using-gitlab-installed-from-source-deprecated). - -The basic steps of configuring a **secondary** node are to: - -- Replicate required configurations between the **primary** and **secondary** nodes. -- Configure a tracking database on each **secondary** node. -- Start GitLab on the **secondary** node. - -You are encouraged to first read through all the steps before executing them -in your testing/production environment. - -NOTE: **Note:** -**Do not** set up any custom authentication on **secondary** nodes, this will be handled by the **primary** node. - -NOTE: **Note:** -**Do not** add anything in the **secondary** node's admin area (**Admin Area > Geo**). This is handled solely by the **primary** node. - -### Step 1. Manually replicate secret GitLab values - -GitLab stores a number of secret values in the `/home/git/gitlab/config/secrets.yml` -file which *must* match between the **primary** and **secondary** nodes. Until there is -a means of automatically replicating these between nodes (see [gitlab-org/gitlab-ee#3789]), they must -be manually replicated to **secondary** nodes. - -1. SSH into the **primary** node, and execute the command below: - - ```sh - sudo cat /home/git/gitlab/config/secrets.yml - ``` - - This will display the secrets that need to be replicated, in YAML format. - -1. SSH into the **secondary** node and login as the `git` user: - - ```sh - sudo -i -u git - ``` - -1. Make a backup of any existing secrets: - - ```sh - mv /home/git/gitlab/config/secrets.yml /home/git/gitlab/config/secrets.yml.`date +%F` - ``` - -1. Copy `/home/git/gitlab/config/secrets.yml` from the **primary** node to the **secondary** node, or - copy-and-paste the file contents between nodes: - - ```sh - sudo editor /home/git/gitlab/config/secrets.yml - - # paste the output of the `cat` command you ran on the primary - # save and exit - ``` - -1. Ensure the file permissions are correct: - - ```sh - chown git:git /home/git/gitlab/config/secrets.yml - chmod 0600 /home/git/gitlab/config/secrets.yml - ``` - -1. Restart GitLab - - ```sh - service gitlab restart - ``` - -Once restarted, the **secondary** node will automatically start replicating missing data -from the **primary** node in a process known as backfill. Meanwhile, the **primary** node -will start to notify the **secondary** node of any changes, so that the **secondary** node can -act on those notifications immediately. - -Make sure the **secondary** node is running and accessible. You can login to -the **secondary** node with the same credentials as used for the **primary** node. - -### Step 2. Manually replicate the **primary** node's SSH host keys - -Read [Manually replicate the **primary** node's SSH host keys](configuration.md#step-2-manually-replicate-the-primary-nodes-ssh-host-keys) - -### Step 3. Add the **secondary** GitLab node - -1. Navigate to the **primary** node's **Admin Area > Geo** - (`/admin/geo/nodes`) in your browser. -1. Add the **secondary** node by providing its full URL. **Do NOT** check the - **This is a primary node** checkbox. -1. Optionally, choose which namespaces should be replicated by the - **secondary** node. Leave blank to replicate all. Read more in - [selective synchronization](#selective-synchronization). -1. Click the **Add node** button. -1. SSH into your GitLab **secondary** server and restart the services: - - ```sh - service gitlab restart - ``` - - Check if there are any common issue with your Geo setup by running: - - ```sh - bundle exec rake gitlab:geo:check - ``` - -1. SSH into your GitLab **primary** server and login as root to verify the - **secondary** node is reachable or there are any common issue with your Geo setup: - - ```sh - bundle exec rake gitlab:geo:check - ``` - -Once reconfigured, the **secondary** node will automatically start -replicating missing data from the **primary** node in a process known as backfill. -Meanwhile, the **primary** node will start to notify the **secondary** node of any changes, so -that the **secondary** node can act on those notifications immediately. - -Make sure the **secondary** node is running and accessible. -You can log in to the **secondary** node with the same credentials as used for the **primary** node. - -### Step 4. Enabling Hashed Storage - -Read [Enabling Hashed Storage](configuration.md#step-4-enabling-hashed-storage). - -### Step 5. (Optional) Configuring the secondary to trust the primary - -You can safely skip this step if your **primary** node uses a CA-issued HTTPS certificate. - -If your **primary** node is using a self-signed certificate for *HTTPS* support, you will -need to add that certificate to the **secondary** node's trust store. Retrieve the -certificate from the **primary** node and follow your distribution's instructions for -adding it to the **secondary** node's trust store. In Debian/Ubuntu, you would follow these steps: - -```sh -sudo -i -cp <primary_node_certification_file> /usr/local/share/ca-certificates -update-ca-certificates -``` - -### Step 6. Enable Git access over HTTP/HTTPS - -Geo synchronizes repositories over HTTP/HTTPS, and therefore requires this clone -method to be enabled. Navigate to **Admin Area > Settings** -(`/admin/application_settings`) on the **primary** node, and set -`Enabled Git access protocols` to `Both SSH and HTTP(S)` or `Only HTTP(S)`. - -### Step 7. Verify proper functioning of the secondary node - -Read [Verify proper functioning of the secondary node][configuration-verify-node]. - -## Selective synchronization - -Read [Selective synchronization][configuration-selective-replication]. - -## Troubleshooting - -Read the [troubleshooting document][troubleshooting]. - -[gitlab-org/gitlab-ee#3789]: https://gitlab.com/gitlab-org/gitlab-ee/issues/3789 -[configuration]: configuration.md -[configuration-selective-replication]: configuration.md#selective-synchronization -[configuration-verify-node]: configuration.md#step-7-verify-proper-functioning-of-the-secondary-node -[troubleshooting]: troubleshooting.md diff --git a/doc/administration/geo/replication/database.md b/doc/administration/geo/replication/database.md index e57583a3bf9..c0cdea216cb 100644 --- a/doc/administration/geo/replication/database.md +++ b/doc/administration/geo/replication/database.md @@ -1,9 +1,7 @@ -# Geo database replication (GitLab Omnibus) **[PREMIUM ONLY]** +# Geo database replication **[PREMIUM ONLY]** NOTE: **Note:** -This is the documentation for the Omnibus GitLab packages. For installations -from source, follow the -[Geo database replication (source)](database_source.md) guide. +The following steps are for Omnibus installs only. Using Geo with source-based installs was **deprecated** in GitLab 11.5. NOTE: **Note:** If your GitLab installation uses external (not managed by Omnibus) PostgreSQL @@ -102,10 +100,15 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o else. If you are using an external database not managed by Omnibus GitLab, you need - to create the replicator user and define a password to it manually. - For information on how to create a replication user, refer to the - [appropriate step](database_source.md#step-1-configure-the-primary-server) - in [Geo database replication (source)](database_source.md). + to create the replicator user and define a password to it manually: + + ```sql + --- Create a new user 'replicator' + CREATE USER gitlab_replicator; + + --- Set/change a password and grants replication privilege + ALTER USER gitlab_replicator WITH REPLICATION ENCRYPTED PASSWORD '<replication_password>'; + ``` 1. Configure PostgreSQL to listen on network interfaces: @@ -340,7 +343,7 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o ## ## Secondary address - ## - replace '<secondary_node_ip>' with the public or VPC address of your Geo secondary node + ## - replace '<secondary_node_ip>' with the public or VPC address of your Geo secondary node ## postgresql['listen_address'] = '<secondary_node_ip>' postgresql['md5_auth_cidr_addresses'] = ['<secondary_node_ip>/32'] @@ -383,8 +386,7 @@ the database on the **primary** node, replicates the database, and creates the needed files for streaming replication. The directories used are the defaults that are set up in Omnibus. If you have -changed any defaults or are using a source installation, configure it as you -see fit replacing the directories and paths. +changed any defaults, configure it as you see fit replacing the directories and paths. CAUTION: **Warning:** Make sure to run this on the **secondary** server as it removes all PostgreSQL's @@ -443,7 +445,7 @@ The replication process is now complete. PostgreSQL connections. We recommend using PGBouncer if you use GitLab in a high-availability configuration with a cluster of nodes supporting a Geo **primary** node and another cluster of nodes supporting a Geo **secondary** node. For more -information, see the [Omnibus HA](https://docs.gitlab.com/ee/administration/high_availability/database.html#configure-using-omnibus-for-high-availability) +information, see the [Omnibus HA](../../high_availability/database.md#configure-using-omnibus) documentation. For a Geo **secondary** node to work properly with PGBouncer in front of the database, diff --git a/doc/administration/geo/replication/database_source.md b/doc/administration/geo/replication/database_source.md deleted file mode 100644 index 67cf8b6535f..00000000000 --- a/doc/administration/geo/replication/database_source.md +++ /dev/null @@ -1,439 +0,0 @@ -# Geo database replication (source) **[PREMIUM ONLY]** - -NOTE: **Note:** -This documentation applies to GitLab source installations. In GitLab 11.5, this documentation was deprecated and will be removed in a future release. -Please consider [migrating to GitLab Omnibus install](https://docs.gitlab.com/omnibus/update/convert_to_omnibus.html). For installations -using the Omnibus GitLab packages, follow the -[**database replication for Omnibus GitLab**][database] guide. - -NOTE: **Note:** -The stages of the setup process must be completed in the documented order. -Before attempting the steps in this stage, [complete all prior stages](index.md#using-gitlab-installed-from-source-deprecated). - -This document describes the minimal steps you have to take in order to -replicate your **primary** GitLab database to a **secondary** node's database. You may -have to change some values according to your database setup, how big it is, etc. - -You are encouraged to first read through all the steps before executing them -in your testing/production environment. - -## PostgreSQL replication - -The GitLab **primary** node where the write operations happen will connect to -**primary** database server, and the **secondary** ones which are read-only will -connect to **secondary** database servers (which are read-only too). - -NOTE: **Note:** -In many databases' documentation, you will see "**primary**" being referenced as "master" -and "**secondary**" as either "slave" or "standby" server (read-only). - -We recommend using [PostgreSQL replication slots][replication-slots-article] -to ensure the **primary** node retains all the data necessary for the secondaries to -recover. See below for more details. - -The following guide assumes that: - -- You are using PostgreSQL 9.6 or later which includes the - [`pg_basebackup` tool][pgback] and improved [Foreign Data Wrapper][FDW] support. -- You have a **primary** node already set up (the GitLab server you are - replicating from), running PostgreSQL 9.6 or later, and - you have a new **secondary** server set up with the same versions of the OS, - PostgreSQL, and GitLab on all nodes. -- The IP of the **primary** server for our examples is `198.51.100.1`, whereas the - **secondary** node's IP is `198.51.100.2`. Note that the **primary** and **secondary** servers - **must** be able to communicate over these addresses. These IP addresses can either - be public or private. - -CAUTION: **Warning:** -Geo works with streaming replication. Logical replication is not supported at this time. -There is an [issue where support is being discussed](https://gitlab.com/gitlab-org/gitlab-ee/issues/7420). - -### Step 1. Configure the **primary** server - -1. SSH into your GitLab **primary** server and login as root: - - ```sh - sudo -i - ``` - -1. Add this node as the Geo **primary** by running: - - ```sh - bundle exec rake geo:set_primary_node - ``` - -1. Create a [replication user] named `gitlab_replicator`: - - ```sql - --- Create a new user 'replicator' - CREATE USER gitlab_replicator; - - --- Set/change a password and grants replication privilege - ALTER USER gitlab_replicator WITH REPLICATION ENCRYPTED PASSWORD '<replication_password>'; - ``` - -1. Make sure your the `gitlab` database user has a password defined: - - ```sh - sudo \ - -u postgres psql \ - -d template1 \ - -c "ALTER USER gitlab WITH ENCRYPTED PASSWORD '<database_password>';" - ``` - -1. Edit the content of `database.yml` in `production:` and add the password like the example below: - - ```yaml - # - # PRODUCTION - # - production: - adapter: postgresql - encoding: unicode - database: gitlabhq_production - pool: 10 - username: gitlab - password: <database_password> - host: /var/opt/gitlab/geo-postgresql - ``` - -1. Set up TLS support for the PostgreSQL **primary** server: - - CAUTION: **Warning**: - Only skip this step if you **know** that PostgreSQL traffic - between the **primary** and **secondary** nodes will be secured through some other - means, e.g., a known-safe physical network path or a site-to-site VPN that - you have configured. - - If you are replicating your database across the open Internet, it is - **essential** that the connection is TLS-secured. Correctly configured, this - provides protection against both passive eavesdroppers and active - "man-in-the-middle" attackers. - - To generate a self-signed certificate and key, run this command: - - ```sh - openssl req \ - -nodes \ - -batch \ - -x509 \ - -newkey rsa:4096 \ - -keyout server.key \ - -out server.crt \ - -days 3650 - ``` - - This will create two files - `server.key` and `server.crt` - that you can - use for authentication. - - Copy them to the correct location for your PostgreSQL installation: - - ```sh - # Copying a self-signed certificate and key - install -o postgres -g postgres -m 0400 -T server.crt ~postgres/9.x/main/data/server.crt - install -o postgres -g postgres -m 0400 -T server.key ~postgres/9.x/main/data/server.key - ``` - - Add this configuration to `postgresql.conf`, removing any existing - configuration for `ssl_cert_file` or `ssl_key_file`: - - ``` - ssl = on - ssl_cert_file='server.crt' - ssl_key_file='server.key' - ``` - -1. Edit `postgresql.conf` to configure the **primary** server for streaming replication - (for Debian/Ubuntu that would be `/etc/postgresql/9.x/main/postgresql.conf`): - - ``` - listen_address = '<primary_node_ip>' - wal_level = hot_standby - max_wal_senders = 5 - min_wal_size = 80MB - max_wal_size = 1GB - max_replicaton_slots = 1 # Number of Geo secondary nodes - wal_keep_segments = 10 - hot_standby = on - ``` - - NOTE: **Note**: - Be sure to set `max_replication_slots` to the number of Geo **secondary** - nodes that you may potentially have (at least 1). - - For security reasons, PostgreSQL by default only listens on the local - interface (e.g. 127.0.0.1). However, Geo needs to communicate - between the **primary** and **secondary** nodes over a common network, such as a - corporate LAN or the public Internet. For this reason, we need to - configure PostgreSQL to listen on more interfaces. - - The `listen_address` option opens PostgreSQL up to external connections - with the interface corresponding to the given IP. See [the PostgreSQL - documentation][pg-docs-runtime-conn] for more details. - - You may also want to edit the `wal_keep_segments` and `max_wal_senders` to - match your database replication requirements. Consult the - [PostgreSQL - Replication documentation][pg-docs-runtime-replication] for more information. - -1. Set the access control on the **primary** node to allow TCP connections using the - server's public IP and set the connection from the **secondary** node to require a - password. Edit `pg_hba.conf` (for Debian/Ubuntu that would be - `/etc/postgresql/9.x/main/pg_hba.conf`): - - ```sh - host all all <primary_node_ip>/32 md5 - host replication gitlab_replicator <secondary_node_ip>/32 md5 - ``` - - If you want to add another secondary, add one more row like the replication - one and change the IP address: - - ```sh - host all all <primary_node_ip>/32 md5 - host replication gitlab_replicator <secondary_node_ip>/32 md5 - host replication gitlab_replicator <another_secondary_node_ip>/32 md5 - ``` - -1. Restart PostgreSQL for the changes to take effect. - -1. Choose a database-friendly name to use for your secondary to use as the - replication slot name. For example, if your domain is - `secondary.geo.example.com`, you may use `secondary_example` as the slot - name. - -1. Create the replication slot on the **primary** node: - - ```sh - $ sudo -u postgres psql -c "SELECT * FROM pg_create_physical_replication_slot('secondary_example');" - slot_name | xlog_position - ------------------+--------------- - secondary_example | - (1 row) - ``` - -1. Now that the PostgreSQL server is set up to accept remote connections, run - `netstat -plnt` to make sure that PostgreSQL is listening to the server's - public IP. - -### Step 2. Configure the secondary server - -Follow the first steps in ["configure the secondary server"][database-replication] and note that since you are installing from source, the username and -group listed as `gitlab-psql` in those steps should be replaced by `postgres` -instead. After completing the "Test that the `gitlab-psql` user can connect to -the **primary** node's database" step, continue here: - -1. Edit `postgresql.conf` to configure the secondary for streaming replication - (for Debian/Ubuntu that would be `/etc/postgresql/9.*/main/postgresql.conf`): - - ```sh - wal_level = hot_standby - max_wal_senders = 5 - checkpoint_segments = 10 - wal_keep_segments = 10 - hot_standby = on - ``` - -1. Restart PostgreSQL for the changes to take effect. - -#### Enable tracking database on the secondary server - -Geo secondary nodes use a tracking database to keep track of replication status -and recover automatically from some replication issues. Follow the steps below to create -the tracking database. - -1. On the secondary node, run the following command to create `database_geo.yml` with the - information of your secondary PostgreSQL instance: - - ```sh - sudo cp /home/git/gitlab/config/database_geo.yml.postgresql /home/git/gitlab/config/database_geo.yml - ``` - -1. Edit the content of `database_geo.yml` in `production:` as in the example below: - - ```yaml - # - # PRODUCTION - # - production: - adapter: postgresql - encoding: unicode - database: gitlabhq_geo_production - pool: 10 - username: gitlab_geo - # password: - host: /var/opt/gitlab/geo-postgresql - ``` - -1. Create the database `gitlabhq_geo_production` on the PostgreSQL instance of the **secondary** node. - -1. Set up the Geo tracking database: - - ```sh - bundle exec rake geo:db:migrate - ``` - -1. Configure the [PostgreSQL FDW][FDW] connection and credentials: - - Save the script below in a file, ex. `/tmp/geo_fdw.sh` and modify the connection - params to match your environment. Execute it to set up the FDW connection. - - ```sh - #!/bin/bash - - # Secondary Database connection params: - DB_HOST="/var/opt/gitlab/postgresql" # change to the public IP or VPC private IP if its an external server - DB_NAME="gitlabhq_production" - DB_USER="gitlab" - DB_PORT="5432" - - # Tracking Database connection params: - GEO_DB_HOST="/var/opt/gitlab/geo-postgresql" # change to the public IP or VPC private IP if its an external server - GEO_DB_NAME="gitlabhq_geo_production" - GEO_DB_USER="gitlab_geo" - GEO_DB_PORT="5432" - - query_exec () { - gitlab-psql -h $GEO_DB_HOST -d $GEO_DB_NAME -p $GEO_DB_PORT -c "${1}" - } - - query_exec "CREATE EXTENSION postgres_fdw;" - query_exec "CREATE SERVER gitlab_secondary FOREIGN DATA WRAPPER postgres_fdw OPTIONS (host '${DB_HOST}', dbname '${DB_NAME}', port '${DB_PORT}');" - query_exec "CREATE USER MAPPING FOR ${GEO_DB_USER} SERVER gitlab_secondary OPTIONS (user '${DB_USER}');" - query_exec "CREATE SCHEMA gitlab_secondary;" - query_exec "GRANT USAGE ON FOREIGN SERVER gitlab_secondary TO ${GEO_DB_USER};" - ``` - - And edit the content of `database_geo.yml` and to add `fdw: true` to - the `production:` block. - -### Step 3. Initiate the replication process - -Below we provide a script that connects the database on the **secondary** node to -the database on the **primary** node, replicates the database, and creates the -needed files for streaming replication. - -The directories used are the defaults for Debian/Ubuntu. If you have changed -any defaults, configure it as you see fit replacing the directories and paths. - -CAUTION: **Warning:** -Make sure to run this on the **secondary** server as it removes all PostgreSQL's -data before running `pg_basebackup`. - -1. SSH into your GitLab **secondary** server and login as root: - - ```sh - sudo -i - ``` - -1. Save the snippet below in a file, let's say `/tmp/replica.sh`. Modify the - embedded paths if necessary: - - ``` - #!/bin/bash - - PORT="5432" - USER="gitlab_replicator" - echo --------------------------------------------------------------- - echo WARNING: Make sure this script is run from the secondary server - echo --------------------------------------------------------------- - echo - echo Enter the IP or FQDN of the primary PostgreSQL server - read HOST - echo Enter the password for $USER@$HOST - read -s PASSWORD - echo Enter the required sslmode - read SSLMODE - - echo Stopping PostgreSQL and all GitLab services - sudo service gitlab stop - sudo service postgresql stop - - echo Backing up postgresql.conf - sudo -u postgres mv /var/opt/gitlab/postgresql/data/postgresql.conf /var/opt/gitlab/postgresql/ - - echo Cleaning up old cluster directory - sudo -u postgres rm -rf /var/opt/gitlab/postgresql/data - - echo Starting base backup as the replicator user - echo Enter the password for $USER@$HOST - sudo -u postgres /opt/gitlab/embedded/bin/pg_basebackup -h $HOST -D /var/opt/gitlab/postgresql/data -U gitlab_replicator -v -x -P - - echo Writing recovery.conf file - sudo -u postgres bash -c "cat > /var/opt/gitlab/postgresql/data/recovery.conf <<- _EOF1_ - standby_mode = 'on' - primary_conninfo = 'host=$HOST port=$PORT user=$USER password=$PASSWORD sslmode=$SSLMODE' - _EOF1_ - " - - echo Restoring postgresql.conf - sudo -u postgres mv /var/opt/gitlab/postgresql/postgresql.conf /var/opt/gitlab/postgresql/data/ - - echo Starting PostgreSQL - sudo service postgresql start - ``` - -1. Run it with: - - ```sh - bash /tmp/replica.sh - ``` - - When prompted, enter the IP/FQDN of the **primary** node, and the password you set up - for the `gitlab_replicator` user in the first step. - - You should use `verify-ca` for the `sslmode`. You can use `disable` if you - are happy to skip PostgreSQL TLS authentication altogether (e.g., you know - the network path is secure, or you are using a site-to-site VPN). This is - **not** safe over the public Internet! - - You can read more details about each `sslmode` in the - [PostgreSQL documentation][pg-docs-ssl]; - the instructions above are carefully written to ensure protection against - both passive eavesdroppers and active "man-in-the-middle" attackers. - -The replication process is now over. - -## PGBouncer support (optional) - -1. First, enter the PostgreSQL console as an admin user. - -1. Then create the read-only user: - - ```sql - -- NOTE: Use the password defined earlier - CREATE USER gitlab_geo_fdw WITH password '<your_password_here>'; - GRANT CONNECT ON DATABASE gitlabhq_production to gitlab_geo_fdw; - GRANT USAGE ON SCHEMA public TO gitlab_geo_fdw; - GRANT SELECT ON ALL TABLES IN SCHEMA public TO gitlab_geo_fdw; - GRANT SELECT ON ALL SEQUENCES IN SCHEMA public TO gitlab_geo_fdw; - - -- Tables created by "gitlab" should be made read-only for "gitlab_geo_fdw" - -- automatically. - ALTER DEFAULT PRIVILEGES FOR USER gitlab IN SCHEMA public GRANT SELECT ON TABLES TO gitlab_geo_fdw; - ALTER DEFAULT PRIVILEGES FOR USER gitlab IN SCHEMA public GRANT SELECT ON SEQUENCES TO gitlab_geo_fdw; - ``` - -1. Enter the PostgreSQL console on the **secondary** tracking database and change the user mapping to this new user: - - ``` - ALTER USER MAPPING FOR gitlab_geo SERVER gitlab_secondary OPTIONS (SET user 'gitlab_geo_fdw') - ``` - -## MySQL replication - -MySQL replication is not supported for Geo. - -## Troubleshooting - -Read the [troubleshooting document](troubleshooting.md). - -[replication-slots-article]: https://medium.com/@tk512/replication-slots-in-postgresql-b4b03d277c75 -[pgback]: http://www.postgresql.org/docs/9.6/static/app-pgbasebackup.html -[replication user]:https://wiki.postgresql.org/wiki/Streaming_Replication -[FDW]: https://www.postgresql.org/docs/9.6/static/postgres-fdw.html -[database]: database.md -[add-geo-node]: configuration.md#step-3-add-the-secondary-gitlab-node -[database-replication]: database.md#step-2-configure-the-secondary-server -[pg-docs-ssl]: https://www.postgresql.org/docs/9.6/static/libpq-ssl.html#LIBPQ-SSL-PROTECTION -[pg-docs-runtime-conn]: https://www.postgresql.org/docs/9.6/static/runtime-config-connection.html -[pg-docs-runtime-replication]: https://www.postgresql.org/docs/9.6/static/runtime-config-replication.html diff --git a/doc/administration/geo/replication/external_database.md b/doc/administration/geo/replication/external_database.md index dae5ed911b0..177ca68613e 100644 --- a/doc/administration/geo/replication/external_database.md +++ b/doc/administration/geo/replication/external_database.md @@ -129,7 +129,7 @@ To configure the connection to the external read-replica database and enable Log database to keep track of replication status and automatically recover from potential replication issues. Omnibus automatically configures a tracking database when `roles ['geo_secondary_role']` is set. For high availability, -refer to [Geo High Availability](https://docs.gitlab.com/ee/administration/high_availability). +refer to [Geo High Availability](../../high_availability/README.md). If you want to run this database external to Omnibus, please follow the instructions below. The tracking database requires an [FDW](https://www.postgresql.org/docs/9.6/static/postgres-fdw.html) diff --git a/doc/administration/geo/replication/high_availability.md b/doc/administration/geo/replication/high_availability.md index 715a83a9ff3..921a3ef1c7a 100644 --- a/doc/administration/geo/replication/high_availability.md +++ b/doc/administration/geo/replication/high_availability.md @@ -6,7 +6,7 @@ described, it is possible to adapt these instructions to your needs. ## Architecture overview - + _[diagram source - gitlab employees only][diagram-source]_ @@ -68,7 +68,7 @@ NOTE: **Note:** PostgreSQL and Redis should have already been disabled on the application servers, and connections from the application servers to those services on the backend servers configured, during normal GitLab HA set up. See high availability configuration documentation for -[PostgreSQL](https://docs.gitlab.com/ee/administration/high_availability/database.html#configuring-the-application-nodes) +[PostgreSQL](../../high_availability/database.md#configuring-the-application-nodes) and [Redis](../../high_availability/redis.md#example-configuration-for-the-gitlab-application). The **primary** database will require modification later, as part of diff --git a/doc/administration/geo/replication/index.md b/doc/administration/geo/replication/index.md index 6abea2cf271..54377f7ae4e 100644 --- a/doc/administration/geo/replication/index.md +++ b/doc/administration/geo/replication/index.md @@ -179,37 +179,20 @@ The steps below should be followed in the order they appear. **Make sure the Git If you installed GitLab using the Omnibus packages (highly recommended): 1. [Install GitLab Enterprise Edition](https://about.gitlab.com/installation/) on the server that will serve as the **secondary** node. Do not create an account or log in to the new **secondary** node. -1. [Upload the GitLab License](https://docs.gitlab.com/ee/user/admin_area/license.html) on the **primary** node to unlock Geo. The license must be for [GitLab Premium](https://about.gitlab.com/pricing/) or higher. +1. [Upload the GitLab License](../../../user/admin_area/license.md) on the **primary** node to unlock Geo. The license must be for [GitLab Premium](https://about.gitlab.com/pricing/) or higher. 1. [Set up the database replication](database.md) (`primary (read-write) <-> secondary (read-only)` topology). 1. [Configure fast lookup of authorized SSH keys in the database](../../operations/fast_ssh_key_lookup.md). This step is required and needs to be done on **both** the **primary** and **secondary** nodes. 1. [Configure GitLab](configuration.md) to set the **primary** and **secondary** nodes. 1. Optional: [Configure a secondary LDAP server](../../auth/ldap.md) for the **secondary** node. See [notes on LDAP](#ldap). 1. [Follow the "Using a Geo Server" guide](using_a_geo_server.md). -### Using GitLab installed from source (Deprecated) - -NOTE: **Note:** -In GitLab 11.5, support for using Geo in GitLab source installations was deprecated and will be removed in a future release. Please consider [migrating to GitLab Omnibus install](https://docs.gitlab.com/omnibus/update/convert_to_omnibus.html). - -If you installed GitLab from source: - -1. [Install GitLab Enterprise Edition](../../../install/installation.md) on the server that will serve as the **secondary** node. Do not create an account or log in to the new **secondary** node. -1. [Upload the GitLab License](https://docs.gitlab.com/ee/user/admin_area/license.html) on the **primary** node to unlock Geo. The license must be for [GitLab Premium](https://about.gitlab.com/pricing/) or higher. -1. [Set up the database replication](database_source.md) (`primary (read-write) <-> secondary (read-only)` topology). -1. [Configure fast lookup of authorized SSH keys in the database](../../operations/fast_ssh_key_lookup.md). Do this step for **both** **primary** and **secondary** nodes. -1. [Configure GitLab](configuration_source.md) to set the **primary** and **secondary** nodes. -1. [Follow the "Using a Geo Server" guide](using_a_geo_server.md). - ## Post-installation documentation After installing GitLab on the **secondary** nodes and performing the initial configuration, see the following documentation for post-installation information. ### Configuring Geo -For information on configuring Geo, see: - -- [Geo configuration (GitLab Omnibus)](configuration.md). -- [Geo configuration (source)](configuration_source.md). Configuring Geo in GitLab source installations was **deprecated** in GitLab 11.5. +For information on configuring Geo, see [Geo configuration](configuration.md). ### Updating Geo @@ -278,7 +261,7 @@ Take special note that these examples of GitLab features are both: Examples include: -- [Elasticsearch integration](https://docs.gitlab.com/ee/integration/elasticsearch.html). +- [Elasticsearch integration](../../../integration/elasticsearch.md). - [Container Registry](../../container_registry.md). [Object Storage](object_storage.md) can mitigate this. - [GitLab Pages](../../pages/index.md). - [Mattermost integration](https://docs.gitlab.com/omnibus/gitlab-mattermost/). diff --git a/doc/administration/geo/replication/security_review.md b/doc/administration/geo/replication/security_review.md index 46d3e68ab63..cd54e2dc8c4 100644 --- a/doc/administration/geo/replication/security_review.md +++ b/doc/administration/geo/replication/security_review.md @@ -120,9 +120,7 @@ questions from [owasp.org](https://www.owasp.org). ### What details regarding required OS components and lock‐down needs have been defined? -- The recommended installation method (Omnibus) packages most components itself. - A from-source installation method exists. Both are documented at - <https://docs.gitlab.com/ee/administration/geo/replication/index.html> +- The supported installation method (Omnibus) packages most components itself. - There are significant dependencies on the system-installed OpenSSH daemon (Geo requires users to set up custom authentication methods) and the omnibus or system-provided PostgreSQL daemon (it must be configured to listen on TCP, diff --git a/doc/administration/geo/replication/troubleshooting.md b/doc/administration/geo/replication/troubleshooting.md index 9c95720487d..c5bdd36ba70 100644 --- a/doc/administration/geo/replication/troubleshooting.md +++ b/doc/administration/geo/replication/troubleshooting.md @@ -23,6 +23,8 @@ to help identify if something is wrong:  +For information on how to resolve common errors reported from the UI, see [common errors](#common-errors). + If the UI is not working, or you are unable to log in, you can run the Geo health check manually to get this information as well as a few more details. This rake task can be run on an app node in the **primary** or **secondary** @@ -40,7 +42,8 @@ Checking Geo ... GitLab Geo is available ... yes GitLab Geo is enabled ... yes GitLab Geo secondary database is correctly configured ... yes -Using database streaming replication? ... yes +Database replication enabled? ... yes +Database replication working? ... yes GitLab Geo tracking database is configured to use Foreign Data Wrapper? ... yes GitLab Geo tracking database Foreign Data Wrapper schema is up-to-date? ... yes GitLab Geo HTTP(S) connectivity ... @@ -68,22 +71,22 @@ Example output: ``` http://secondary.example.com/ ----------------------------------------------------- - GitLab Version: 11.8.1-ee + GitLab Version: 11.10.4-ee Geo Role: Secondary Health Status: Healthy - Repositories: 190/190 (100%) - Verified Repositories: 190/190 (100%) - Wikis: 190/190 (100%) - Verified Wikis: 190/190 (100%) - LFS Objects: 35/35 (100%) - Attachments: 528/528 (100%) - CI job artifacts: 477/477 (100%) - Repositories Checked: 0/190 (0%) + Repositories: 289/289 (100%) + Verified Repositories: 289/289 (100%) + Wikis: 289/289 (100%) + Verified Wikis: 289/289 (100%) + LFS Objects: 8/8 (100%) + Attachments: 5/5 (100%) + CI job artifacts: 0/0 (0%) + Repositories Checked: 0/289 (0%) Sync Settings: Full Database replication lag: 0 seconds - Last event ID seen from primary: 2158 (about 2 minute ago) - Last event ID processed by cursor: 2158 (about 2 minute ago) - Last status report was: 4 minutes ago + Last event ID seen from primary: 10215 (about 2 minutes ago) + Last event ID processed by cursor: 10215 (about 2 minutes ago) + Last status report was: 2 minutes ago ``` ## Is Postgres replication working? @@ -307,7 +310,7 @@ same host on different ports. That is, 5432 and 5431 respectively. ### Checking configuration NOTE: **Note:** -The following steps are for Omnibus installs only. Using Geo with source-based installs [is deprecated](index.md#using-gitlab-installed-from-source-deprecated). +The following steps are for Omnibus installs only. Using Geo with source-based installs was **deprecated** in GitLab 11.5. To check the configuration: @@ -455,3 +458,57 @@ reload of the FDW schema. To manually reload the FDW schema: [database-start-replication]: database.md#step-3-initiate-the-replication-process [database-pg-replication]: database.md#postgresql-replication + +## Common errors + +This section documents common errors reported in the admin UI and how to fix them. + +### Geo database configuration file is missing + +GitLab cannot find or doesn't have permission to access the `database_geo.yml` configuration file. + +In an Omnibus GitLab installation, the file should be in `/var/opt/gitlab/gitlab-rails/etc`. +If it doesn't exist or inadvertent changes have been made to it, run `sudo gitlab-ctl reconfigure` to restore it to its correct state. + + +If this path is mounted on a remote volume, please check your volume configuration and that it has correct permissions. + +### Geo node has a database that is writable which is an indication it is not configured for replication with the primary node. + +This error refers to a problem with the database replica on a **secondary** node, +which Geo expects to have access to. It usually means, either: + +- An unsupported replication method was used (for example, logical replication). +- The instructions to setup a [Geo database replication](database.md) were not followed correctly. + +A common source of confusion with **secondary** nodes is that it requires two separate +PostgreSQL instances: + +- A read-only replica of the **primary** node. +- A regular, writable instance that holds replication metadata. That is, the Geo tracking database. + +### Geo node does not appear to be replicating the database from the primary node. + +The most common problems that prevent the database from replicating correctly are: + +- **Secondary** nodes cannot reach the **primary** node. Check credentials, firewall rules, etc. +- SSL certificate problems. Make sure you copied `/etc/gitlab/gitlab-secrets.json` from the **primary** node. +- Database storage disk is full. +- Database replication slot is misconfigured. +- Database is not using a replication slot or another alternative and cannot catch-up because WAL files were purged. + +Make sure you follow the [Geo database replication](database.md) instructions for supported configuration. + +### Geo database version (...) does not match latest migration (...) + +If you are using GitLab Omnibus installation, something might have failed during upgrade. You can: + +- Run `sudo gitlab-ctl reconfigure`. +- Manually trigger the database migration by running: `sudo gitlab-rake geo:db:migrate` as root on the **secondary** node. + +### Geo database is not configured to use Foreign Data Wrapper + +This error means the Geo Tracking Database doesn't have the FDW server and credentials +configured. + +See [How do I fix a "Foreign Data Wrapper (FDW) is not configured" error?](#how-do-i-fix-a-foreign-data-wrapper-fdw-is-not-configured-error). diff --git a/doc/administration/geo/replication/tuning.md b/doc/administration/geo/replication/tuning.md index b9921b2e69f..1943f2230df 100644 --- a/doc/administration/geo/replication/tuning.md +++ b/doc/administration/geo/replication/tuning.md @@ -13,5 +13,5 @@ However, this may not lead to more downloads in parallel unless the number of available Sidekiq threads is also increased. For example, if repository sync capacity is increased from 25 to 50, you may also want to increase the number of Sidekiq threads from 25 to 50. See the -[Sidekiq concurrency documentation](https://docs.gitlab.com/ee/administration/operations/extra_sidekiq_processes.html#number-of-threads) +[Sidekiq concurrency documentation](../../operations/extra_sidekiq_processes.md#number-of-threads) for more details. diff --git a/doc/administration/geo/replication/updating_the_geo_nodes.md b/doc/administration/geo/replication/updating_the_geo_nodes.md index 66728366e24..933a75c47d8 100644 --- a/doc/administration/geo/replication/updating_the_geo_nodes.md +++ b/doc/administration/geo/replication/updating_the_geo_nodes.md @@ -337,6 +337,53 @@ is prepended with the relevant node for better clarity: 1. **[secondary]** Create the `replica.sh` script as described in the [database configuration document][database-source-replication]. + 1. 1. **[secondary]** Save the snippet below in a file, let's say `/tmp/replica.sh`. Modify the + embedded paths if necessary: + + ``` + #!/bin/bash + + PORT="5432" + USER="gitlab_replicator" + echo --------------------------------------------------------------- + echo WARNING: Make sure this script is run from the secondary server + echo --------------------------------------------------------------- + echo + echo Enter the IP or FQDN of the primary PostgreSQL server + read HOST + echo Enter the password for $USER@$HOST + read -s PASSWORD + echo Enter the required sslmode + read SSLMODE + + echo Stopping PostgreSQL and all GitLab services + sudo service gitlab stop + sudo service postgresql stop + + echo Backing up postgresql.conf + sudo -u postgres mv /var/opt/gitlab/postgresql/data/postgresql.conf /var/opt/gitlab/postgresql/ + + echo Cleaning up old cluster directory + sudo -u postgres rm -rf /var/opt/gitlab/postgresql/data + + echo Starting base backup as the replicator user + echo Enter the password for $USER@$HOST + sudo -u postgres /opt/gitlab/embedded/bin/pg_basebackup -h $HOST -D /var/opt/gitlab/postgresql/data -U gitlab_replicator -v -x -P + + echo Writing recovery.conf file + sudo -u postgres bash -c "cat > /var/opt/gitlab/postgresql/data/recovery.conf <<- _EOF1_ + standby_mode = 'on' + primary_conninfo = 'host=$HOST port=$PORT user=$USER password=$PASSWORD sslmode=$SSLMODE' + _EOF1_ + " + + echo Restoring postgresql.conf + sudo -u postgres mv /var/opt/gitlab/postgresql/postgresql.conf /var/opt/gitlab/postgresql/data/ + + echo Starting PostgreSQL + sudo service postgresql start + ``` + 1. **[secondary]** Run the recovery script using the credentials from the previous step: @@ -396,8 +443,6 @@ and it is required since 10.0. [update]: ../../../update/README.md [database]: database.md -[database-replication]: database.md#step-3-initiate-the-replication-process -[database-source-replication]: database_source.md#step-3-initiate-the-replication-process [Hashed Storage]: ../../repository_storage_types.md [hashed-migration]: ../../raketasks/storage.md [ssh-fast-lookup]: ../../operations/fast_ssh_key_lookup.md diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md index f1cedb85455..dcf8d8715ca 100644 --- a/doc/administration/gitaly/index.md +++ b/doc/administration/gitaly/index.md @@ -53,6 +53,10 @@ But since 11.8 the indexer uses Gitaly for data access as well. NFS can still be leveraged for redudancy on block level of the Git data. But only has to be mounted on the Gitaly server. +NOTE: **Note:** While Gitaly can be used as a replacement for NFS, we do not recommend +using EFS as it may impact GitLab's performance. Please review the [relevant documentation](../high_availability/nfs.md#avoid-using-awss-elastic-file-system-efs) +for more details. + ### Network architecture - gitlab-rails shards repositories into "repository storages" @@ -73,18 +77,29 @@ be mounted on the Gitaly server. - Gitaly servers must not be exposed to the public internet Gitaly network traffic is unencrypted by default, but supports -[TLS](#tls-support). Authentication is done through a static token. For -security in depth, its recommended to use a firewall to restrict access -to your Gitaly server. +[TLS](#tls-support). Authentication is done through a static token. + +NOTE: **Note:** Gitaly network traffic is unencrypted so we recommend a firewall to +restrict access to your Gitaly server. Below we describe how to configure a Gitaly server at address `gitaly.internal:8075` with secret token `abc123secret`. We assume your GitLab installation has two repository storages, `default` and `storage1`. +### Installation + +First install Gitaly using either Omnibus or from source. + +Omnibus: [Download/install](https://about.gitlab.com/installation) the Omnibus GitLab +package you want using **steps 1 and 2** from the GitLab downloads page but +**_do not_** provide the `EXTERNAL_URL=` value. + +Source: [Install Gitaly](../../install/installation.md#install-gitaly) + ### Client side token configuration -Start by configuring a token on the client side. +Configure a token on the client side. Omnibus installations: @@ -110,7 +125,7 @@ changes to be picked up. Next, on the Gitaly server, we need to configure storage paths, enable the network listener and configure the token. -Note: if you want to reduce the risk of downtime when you enable +NOTE: **Note:** if you want to reduce the risk of downtime when you enable authentication you can temporarily disable enforcement, see [the documentation on configuring Gitaly authentication](https://gitlab.com/gitlab-org/gitaly/blob/master/doc/configuration/README.md#authentication) @@ -122,12 +137,17 @@ the Gitaly server. The easiest way to accomplish this is to copy `/etc/gitlab/gi from an existing GitLab server to the Gitaly server. Without this shared secret, Git operations in GitLab will result in an API error. -> **NOTE:** In most or all cases the storage paths below end in `/repositories` which is +NOTE: **Note:** In most or all cases the storage paths below end in `/repositories` which is different than `path` in `git_data_dirs` of Omnibus installations. Check the directory layout on your Gitaly server to be sure. Omnibus installations: +<!-- +updates to following example must also be made at +https://gitlab.com/charts/gitlab/blob/master/doc/advanced/external-gitaly/external-omnibus-gitaly.md#configure-omnibus-gitlab +--> + ```ruby # /etc/gitlab/gitlab.rb @@ -147,6 +167,7 @@ gitlab_rails['auto_migrate'] = false # Configure the gitlab-shell API callback URL. Without this, `git push` will # fail. This can be your 'front door' GitLab URL or an internal load # balancer. +# Don't forget to copy `/etc/gitlab/gitlab-secrets.json` from web server to Gitaly server. gitlab_rails['internal_api_url'] = 'https://gitlab.example.com' # Make Gitaly accept connections on all network interfaces. You must use diff --git a/doc/administration/high_availability/README.md b/doc/administration/high_availability/README.md index 24db1c28778..e5701525077 100644 --- a/doc/administration/high_availability/README.md +++ b/doc/administration/high_availability/README.md @@ -64,8 +64,8 @@ larger one. - 1 PostgreSQL node - 1 Redis node -- 2 or more GitLab application nodes (Unicorn, Workhorse, Sidekiq) - 1 NFS/Gitaly storage server +- 2 or more GitLab application nodes (Unicorn, Workhorse, Sidekiq) #### Installation Instructions @@ -88,9 +88,9 @@ in size, indicating that there is contention or not enough resources. - 1 PostgreSQL node - 1 Redis node -- 2 or more GitLab application nodes (Unicorn, Workhorse) -- 2 or more Sidekiq nodes - 2 or more NFS/Gitaly storage servers +- 2 or more Sidekiq nodes +- 2 or more GitLab application nodes (Unicorn, Workhorse) ## High Availability Architecture Examples @@ -135,7 +135,7 @@ the contention. - 2 or more GitLab application nodes (Unicorn, Workhorse, Sidekiq, PGBouncer) - 1 NFS/Gitaly server - + ### Hybrid @@ -153,7 +153,7 @@ contention due to certain workloads. - 1 or more NFS/Gitaly servers - 1 Monitoring node (Prometheus, Grafana) - + #### Reference Architecture @@ -194,7 +194,7 @@ with the added complexity of many more nodes to configure, manage and monitor. - 2 or more Web nodes (All other web requests) - 2 or more NFS/Gitaly servers - + The following pages outline the steps necessary to configure each component separately: diff --git a/doc/administration/high_availability/database.md b/doc/administration/high_availability/database.md index 1648b6b848a..a446ed9637c 100644 --- a/doc/administration/high_availability/database.md +++ b/doc/administration/high_availability/database.md @@ -134,7 +134,7 @@ otherwise the networks will become a single point of failure. #### Architecture - + Database nodes run two services with PostgreSQL: diff --git a/doc/administration/high_availability/gitaly.md b/doc/administration/high_availability/gitaly.md index d44744f2af8..40f85f28cb8 100644 --- a/doc/administration/high_availability/gitaly.md +++ b/doc/administration/high_availability/gitaly.md @@ -12,77 +12,8 @@ environments and [High Availability Architecture](./README.md#high-availability- ## Running Gitaly on its own server -Starting with GitLab 11.4, Gitaly is a replacement for NFS except -when the [Elastic Search indexer](https://gitlab.com/gitlab-org/gitlab-elasticsearch-indexer) -is used. - -NOTE: **Note:** While Gitaly can be used as a replacement for NFS, we do not recommend using EFS as it may impact GitLab's performance. Please review the [relevant documentation](nfs.md#avoid-using-awss-elastic-file-system-efs) for more details. - -NOTE: **Note:** Gitaly network traffic is unencrypted so we recommend a firewall to -restrict access to your Gitaly server. - -The steps below are the minimum necessary to configure a Gitaly server with -Omnibus: - -1. SSH into the Gitaly server. -1. [Download/install](https://about.gitlab.com/installation) the Omnibus GitLab - package you want using **steps 1 and 2** from the GitLab downloads page. - - Do not complete any other steps on the download page. - -1. Edit `/etc/gitlab/gitlab.rb` and add the contents: - - Gitaly must trigger some callbacks to GitLab via GitLab Shell. As a result, - the GitLab Shell secret must be the same between the other GitLab servers and - the Gitaly server. The easiest way to accomplish this is to copy `/etc/gitlab/gitlab-secrets.json` - from an existing GitLab server to the Gitaly server. Without this shared secret, - Git operations in GitLab will result in an API error. - - > **NOTE:** In most or all cases the storage paths below end in `repositories` which is - different than `path` in `git_data_dirs` of Omnibus installations. Check the - directory layout on your Gitaly server to be sure. - - ```ruby - # Enable Gitaly - gitaly['enable'] = true - - ## Disable all other services - sidekiq['enable'] = false - gitlab_workhorse['enable'] = false - unicorn['enable'] = false - postgresql['enable'] = false - nginx['enable'] = false - prometheus['enable'] = false - alertmanager['enable'] = false - pgbouncer_exporter['enable'] = false - redis_exporter['enable'] = false - gitlab_monitor['enable'] = false - - # Prevent database connections during 'gitlab-ctl reconfigure' - gitlab_rails['rake_cache_clear'] = false - gitlab_rails['auto_migrate'] = false - - # Configure the gitlab-shell API callback URL. Without this, `git push` will - # fail. This can be your 'front door' GitLab URL or an internal load - # balancer. - gitlab_rails['internal_api_url'] = 'https://gitlab.example.com' - - # Make Gitaly accept connections on all network interfaces. You must use - # firewalls to restrict access to this address/port. - gitaly['listen_addr'] = "0.0.0.0:8075" - gitaly['auth_token'] = 'abc123secret' - - gitaly['storage'] = [ - { 'name' => 'default', 'path' => '/mnt/gitlab/default/repositories' }, - { 'name' => 'storage1', 'path' => '/mnt/gitlab/storage1/repositories' }, - ] - - # To use tls for gitaly you need to add - gitaly['tls_listen_addr'] = "0.0.0.0:9999" - gitaly['certificate_path'] = "path/to/cert.pem" - gitaly['key_path'] = "path/to/key.pem" - ``` - -Again, reconfigure (Omnibus) or restart (source). +See [Running Gitaly on its own server](../gitaly/index.md#running-gitaly-on-its-own-server) +in Gitaly documentation. Continue configuration of other components by going back to: diff --git a/doc/administration/high_availability/img/fully-distributed.png b/doc/administration/high_availability/img/fully-distributed.png Binary files differnew file mode 100644 index 00000000000..ad23207134e --- /dev/null +++ b/doc/administration/high_availability/img/fully-distributed.png diff --git a/doc/administration/high_availability/img/geo-ha-diagram.png b/doc/administration/high_availability/img/geo-ha-diagram.png Binary files differnew file mode 100644 index 00000000000..da5d612827c --- /dev/null +++ b/doc/administration/high_availability/img/geo-ha-diagram.png diff --git a/doc/administration/high_availability/img/horizontal.png b/doc/administration/high_availability/img/horizontal.png Binary files differnew file mode 100644 index 00000000000..c3bd489d96f --- /dev/null +++ b/doc/administration/high_availability/img/horizontal.png diff --git a/doc/administration/high_availability/img/hybrid.png b/doc/administration/high_availability/img/hybrid.png Binary files differnew file mode 100644 index 00000000000..7d4a56bf0ea --- /dev/null +++ b/doc/administration/high_availability/img/hybrid.png diff --git a/doc/administration/high_availability/pg_ha_architecture.png b/doc/administration/high_availability/img/pg_ha_architecture.png Binary files differindex ef870f652ae..ef870f652ae 100644 --- a/doc/administration/high_availability/pg_ha_architecture.png +++ b/doc/administration/high_availability/img/pg_ha_architecture.png diff --git a/doc/administration/high_availability/nfs.md b/doc/administration/high_availability/nfs.md index caadec3ac4e..f6bbc8935b4 100644 --- a/doc/administration/high_availability/nfs.md +++ b/doc/administration/high_availability/nfs.md @@ -39,13 +39,8 @@ options: ### Improving NFS performance with GitLab -NOTE: **Note:** -This is only available starting in certain versions of GitLab: - * 11.5.11 - * 11.6.11 - * 11.7.12 - * 11.8.8 - * 11.9.0 and up (e.g. 11.10, 11.11, etc.) +NOTE: **Note:** This is only available starting in certain versions of GitLab: 11.5.11, +11.6.11, 11.7.12, 11.8.8, 11.9.0 and up (e.g. 11.10, 11.11, etc.) If you are using NFS to share Git data, we recommend that you enable a number of feature flags that will allow GitLab application processes to @@ -107,6 +102,11 @@ stored on a local volume. For more details on another person's experience with EFS, see [Amazon's Elastic File System: Burst Credits](https://rawkode.com/2017/04/16/amazons-elastic-file-system-burst-credits/) +## Avoid using CephFS and GlusterFS + +GitLab strongly recommends against using CephFS and GlusterFS. +These distributed file systems are not well-suited for GitLab's input/output access patterns because git uses many small files and access times and file locking times to propagate will make git activity very slow. + ## Avoid using PostgreSQL with NFS GitLab strongly recommends against running your PostgreSQL database @@ -122,7 +122,7 @@ Additionally, this configuration is specifically warned against in the >to the NFS server can cause data corruption problems. For supported database architecture, please see our documentation on -[Configuring a Database for GitLab HA](https://docs.gitlab.com/ee/administration/high_availability/database.html). +[Configuring a Database for GitLab HA](database.md). ## NFS Client mount options diff --git a/doc/administration/high_availability/redis.md b/doc/administration/high_availability/redis.md index 46ad3ecd9bb..1f37224a184 100644 --- a/doc/administration/high_availability/redis.md +++ b/doc/administration/high_availability/redis.md @@ -863,7 +863,7 @@ You can check if everything is correct by connecting to each server using `redis-cli` application, and sending the `info replication` command as below. ``` -/opt/gitlab/embedded/bin/redis-cli -a <redis-password> info replication +/opt/gitlab/embedded/bin/redis-cli -h <redis-host-or-ip> -a '<redis-password>' info replication ``` When connected to a `master` redis, you will see the number of connected diff --git a/doc/administration/incoming_email.md b/doc/administration/incoming_email.md index 9ac310af248..8271c579f5b 100644 --- a/doc/administration/incoming_email.md +++ b/doc/administration/incoming_email.md @@ -10,7 +10,7 @@ GitLab has several features based on receiving incoming emails: - [New merge request by email](../user/project/merge_requests/index.md#create-new-merge-requests-by-email): allow GitLab users to create a new merge request by sending an email to a user-specific email address. -- [Service Desk](https://docs.gitlab.com/ee/user/project/service_desk.html): provide e-mail support to +- [Service Desk](../user/project/service_desk.md): provide e-mail support to your customers through GitLab. **[PREMIUM]** ## Requirements diff --git a/doc/administration/index.md b/doc/administration/index.md index 797a7242bd0..95a0e84deb6 100644 --- a/doc/administration/index.md +++ b/doc/administration/index.md @@ -20,7 +20,7 @@ GitLab Community Edition installations only have access to Core features. GitLab.com is administered by GitLab, Inc., therefore, only GitLab team members have access to its admin configurations. If you're a GitLab.com user, please check the -[user documentation](../user/index.html). +[user documentation](../user/index.md). NOTE: **Note:** Non-administrator users don’t have access to GitLab administration tools and settings. @@ -37,9 +37,9 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Omnibus support for log forwarding](https://docs.gitlab.com/omnibus/settings/logs.html#udp-log-shipping-gitlab-enterprise-edition-only) **[STARTER ONLY]** - [High Availability](high_availability/README.md): Configure multiple servers for scaling or high availability. - [High Availability on AWS](../university/high-availability/aws/README.md): Set up GitLab HA on Amazon AWS. -- [Geo](https://docs.gitlab.com/ee/administration/geo/replication/index.html): Replicate your GitLab instance to other geographic locations as a read-only fully operational version. **[PREMIUM ONLY]** -- [Disaster Recovery](https://docs.gitlab.com/ee/administration/geo/disaster_recovery/index.html): Quickly fail-over to a different site with minimal effort in a disaster situation. **[PREMIUM ONLY]** -- [Pivotal Tile](https://docs.gitlab.com/ee/install/pivotal/index.html): Deploy GitLab as a pre-configured appliance using Ops Manager (BOSH) for Pivotal Cloud Foundry. **[PREMIUM ONLY]** +- [Geo](geo/replication/index.md): Replicate your GitLab instance to other geographic locations as a read-only fully operational version. **[PREMIUM ONLY]** +- [Disaster Recovery](geo/disaster_recovery/index.md): Quickly fail-over to a different site with minimal effort in a disaster situation. **[PREMIUM ONLY]** +- [Pivotal Tile](../install/pivotal/index.md): Deploy GitLab as a pre-configured appliance using Ops Manager (BOSH) for Pivotal Cloud Foundry. **[PREMIUM ONLY]** - [Add License](../user/admin_area/license.md): Upload a license at install time to unlock features that are in paid tiers of GitLab. **[STARTER ONLY]** ### Configuring GitLab @@ -61,9 +61,9 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Diff limits](../user/admin_area/diff_limits.md): Configure the diff rendering size limits of branch comparison pages. - [Merge request diffs storage](merge_request_diffs.md): Configure merge requests diffs external storage. - [Broadcast Messages](../user/admin_area/broadcast_messages.md): Send messages to GitLab users through the UI. -- [Elasticsearch](https://docs.gitlab.com/ee/integration/elasticsearch.html): Enable Elasticsearch to empower GitLab's Advanced Global Search. Useful when you deal with a huge amount of data. **[STARTER ONLY]** +- [Elasticsearch](../integration/elasticsearch.md): Enable Elasticsearch to empower GitLab's Advanced Global Search. Useful when you deal with a huge amount of data. **[STARTER ONLY]** - [External Classification Policy Authorization](../user/admin_area/settings/external_authorization.md) **[PREMIUM ONLY]** -- [Upload a license](https://docs.gitlab.com/ee/user/admin_area/license.html): Upload a license to unlock features that are in paid tiers of GitLab. **[STARTER ONLY]** +- [Upload a license](../user/admin_area/license.md): Upload a license to unlock features that are in paid tiers of GitLab. **[STARTER ONLY]** - [Admin Area](../user/admin_area/index.md): for self-managed instance-wide configuration and maintenance. #### Customizing GitLab's appearance @@ -73,7 +73,7 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Branded login page](../customization/branded_login_page.md): Customize the login page with your own logo, title, and description. - [Welcome message](../customization/welcome_message.md): Add a custom welcome message to the sign-in page. - ["New Project" page](../customization/new_project_page.md): Customize the text to be displayed on the page that opens whenever your users create a new project. -- [Additional custom email text](https://docs.gitlab.com/ee/user/admin_area/settings/email.html#custom-additional-text-premium-only): Add additional custom text to emails sent from GitLab. **[PREMIUM ONLY]** +- [Additional custom email text](../user/admin_area/settings/email.md#custom-additional-text-premium-only): Add additional custom text to emails sent from GitLab. **[PREMIUM ONLY]** ### Maintaining GitLab @@ -108,34 +108,29 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Sign-up restrictions](../user/admin_area/settings/sign_up_restrictions.md): block email addresses of specific domains, or whitelist only specific domains. - [Access restrictions](../user/admin_area/settings/visibility_and_access_controls.md#enabled-git-access-protocols): Define which Git access protocols can be used to talk to GitLab (SSH, HTTP, HTTPS). - [Authentication and Authorization](auth/README.md): Configure external authentication with LDAP, SAML, CAS and additional providers. - - [Sync LDAP](https://docs.gitlab.com/ee/administration/auth/ldap-ee.html) **[STARTER ONLY]** - - [Kerberos authentication](https://docs.gitlab.com/ee/integration/kerberos.html) **[STARTER ONLY]** + - [Sync LDAP](auth/ldap-ee.md) **[STARTER ONLY]** + - [Kerberos authentication](../integration/kerberos.md) **[STARTER ONLY]** - See also other [authentication](../topics/authentication/index.md#gitlab-administrators) topics (for example, enforcing 2FA). -- [Email users](https://docs.gitlab.com/ee/tools/email.html): Email GitLab users from within GitLab. **[STARTER ONLY]** +- [Email users](../tools/email.md): Email GitLab users from within GitLab. **[STARTER ONLY]** - [User Cohorts](../user/admin_area/user_cohorts.md): Display the monthly cohorts of new users and their activities over time. - [Audit logs and events](audit_events.md): View the changes made within the GitLab server for: - Groups and projects. **[STARTER]** - Instances. **[PREMIUM ONLY]** - [Auditor users](auditor_users.md): Users with read-only access to all projects, groups, and other resources on the GitLab instance. **[PREMIUM ONLY]** - [Incoming email](incoming_email.md): Configure incoming emails to allow - users to [reply by email], create [issues by email] and - [merge requests by email], and to enable [Service Desk]. + users to [reply by email](reply_by_email.md), create [issues by email](../user/project/issues/create_new_issue.md#new-issue-via-email) and + [merge requests by email](../user/project/merge_requests/index.md#create-new-merge-requests-by-email), and to enable [Service Desk](../user/project/service_desk.md). - [Postfix for incoming email](reply_by_email_postfix_setup.md): Set up a basic Postfix mail server with IMAP authentication on Ubuntu for incoming emails. - [Abuse reports](../user/admin_area/abuse_reports.md): View and resolve abuse reports from your users. -[reply by email]: reply_by_email.md -[issues by email]: ../user/project/issues/create_new_issue.md#new-issue-via-email -[merge requests by email]: ../user/project/merge_requests/index.md#create-new-merge-requests-by-email -[Service Desk]: https://docs.gitlab.com/ee/user/project/service_desk.html - ## Project settings - [Container Registry](container_registry.md): Configure Container Registry with GitLab. - [Issue closing pattern](issue_closing_pattern.md): Customize how to close an issue from commit messages. - [Gitaly](gitaly/index.md): Configuring Gitaly, GitLab's Git repository storage service. -- [Default labels](../user/admin_area/labels.html): Create labels that will be automatically added to every new project. +- [Default labels](../user/admin_area/labels.md): Create labels that will be automatically added to every new project. - [Restrict the use of public or internal projects](../public_access/public_access.md#restricting-the-use-of-public-or-internal-projects): Restrict the use of visibility levels for users when they create a project or a snippet. - [Custom project templates](../user/admin_area/custom_project_templates.md): Configure a set of projects to be used as custom templates when creating a new project. **[PREMIUM ONLY]** - [Packages](packages.md): Enable GitLab to act as a Maven repository or NPM registry. **[PREMIUM ONLY]** @@ -145,7 +140,7 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Repository checks](repository_checks.md): Periodic Git repository checks. - [Repository storage paths](repository_storage_paths.md): Manage the paths used to store repositories. - [Repository storage rake tasks](raketasks/storage.md): A collection of rake tasks to list and migrate existing projects and attachments associated with it from Legacy storage to Hashed storage. -- [Limit repository size](https://docs.gitlab.com/ee/user/admin_area/settings/account_and_limit_settings.html): Set a hard limit for your repositories' size. **[STARTER ONLY]** +- [Limit repository size](../user/admin_area/settings/account_and_limit_settings.md): Set a hard limit for your repositories' size. **[STARTER ONLY]** ## Continuous Integration settings @@ -154,7 +149,7 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Job artifacts](job_artifacts.md): Enable, disable, and configure job artifacts (a set of files and directories which are outputted by a job when it completes successfully). - [Job traces](job_traces.md): Information about the job traces (logs). - [Register Shared and specific Runners](../ci/runners/README.md#registering-a-shared-runner): Learn how to register and configure Shared and specific Runners to your own instance. -- [Shared Runners pipelines quota](https://docs.gitlab.com/ee/user/admin_area/settings/continuous_integration.html#shared-runners-pipeline-minutes-quota-starter-only): Limit the usage of pipeline minutes for Shared Runners. **[STARTER ONLY]** +- [Shared Runners pipelines quota](../user/admin_area/settings/continuous_integration.md#shared-runners-pipeline-minutes-quota-starter-only): Limit the usage of pipeline minutes for Shared Runners. **[STARTER ONLY]** - [Enable/disable Auto DevOps](../topics/autodevops/index.md#enablingdisabling-auto-devops): Enable or disable Auto DevOps for your instance. ## Git configuration options diff --git a/doc/administration/integration/plantuml.md b/doc/administration/integration/plantuml.md index b7b820abb40..82e0c14ffc2 100644 --- a/doc/administration/integration/plantuml.md +++ b/doc/administration/integration/plantuml.md @@ -27,7 +27,7 @@ own PlantUML server is easy in Debian/Ubuntu distributions using Tomcat. First you need to create a `plantuml.war` file from the source code: ``` -sudo apt-get install graphviz openjdk-7-jdk git-core maven +sudo apt-get install graphviz openjdk-8-jdk git-core maven git clone https://github.com/plantuml/plantuml-server.git cd plantuml-server mvn package diff --git a/doc/administration/integration/terminal.md b/doc/administration/integration/terminal.md index 2596e3fe68b..c34858cd0db 100644 --- a/doc/administration/integration/terminal.md +++ b/doc/administration/integration/terminal.md @@ -43,6 +43,11 @@ detail below. ## Enabling and disabling terminal support +NOTE: **Note:** AWS Elastic Load Balancers (ELBs) do not support web sockets. +AWS Application Load Balancers (ALBs) must be used if you want web terminals +to work. See [AWS Elastic Load Balancing Product Comparison](https://aws.amazon.com/elasticloadbalancing/features/#compare) +for more information. + As web terminals use WebSockets, every HTTP/HTTPS reverse proxy in front of Workhorse needs to be configured to pass the `Connection` and `Upgrade` headers through to the next one in the chain. If you installed GitLab using Omnibus, or diff --git a/doc/administration/job_artifacts.md b/doc/administration/job_artifacts.md index e7792106f81..ef370573a98 100644 --- a/doc/administration/job_artifacts.md +++ b/doc/administration/job_artifacts.md @@ -100,6 +100,9 @@ artifacts, you can use an object storage like AWS S3 instead. This configuration relies on valid AWS credentials to be configured already. Use an object storage option like AWS S3 to store job artifacts. +DANGER: **Danger:** +If you're enabling S3 in [GitLab HA](high_availability/README.md), you will need to have an [NFS mount set up for CI traces and artifacts](high_availability/nfs.md#a-single-nfs-mount) or enable [live tracing](job_traces.md#new-live-trace-architecture). If these settings are not set, you will risk job traces disappearing or not being saved. + ### Object Storage Settings For source installations the following settings are nested under `artifacts:` and then `object_store:`. On omnibus installs they are prefixed by `artifacts_object_store_`. diff --git a/doc/administration/logs.md b/doc/administration/logs.md index 3d40cda491a..ac41f9177dd 100644 --- a/doc/administration/logs.md +++ b/doc/administration/logs.md @@ -280,6 +280,28 @@ installations from source. Currently it logs the progress of project imports from the Bitbucket Server importer. Future importers may use this file. +## `auth.log` + +Introduced in GitLab 12.0. This file lives in `/var/log/gitlab/gitlab-rails/auth.log` for +Omnibus GitLab packages or in `/home/git/gitlab/log/auth.log` for +installations from source. + +It logs information whenever [Rack Attack] registers an abusive request. + +## `graphql_json.log` + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/59587) in GitLab 12.0. + +This file lives in `/var/log/gitlab/gitlab-rails/graphql_json.log` for +Omnibus GitLab packages or in `/home/git/gitlab/log/graphql_json.log` for +installations from source. + +GraphQL queries are recorded in that file. For example: + +```json +{"query_string":"query IntrospectionQuery{__schema {queryType { name },mutationType { name }}}...(etc)","variables":{"a":1,"b":2},"complexity":181,"depth":1,"duration":7} +``` + ## Reconfigure Logs Reconfigure log files live in `/var/log/gitlab/reconfigure` for Omnibus GitLab @@ -298,3 +320,4 @@ Omnibus GitLab packages or in `/home/git/gitlab/log/sidekiq_exporter.log` for installations from source. [repocheck]: repository_checks.md +[Rack Attack]: ../security/rack_attack.md diff --git a/doc/administration/monitoring/performance/grafana_configuration.md b/doc/administration/monitoring/performance/grafana_configuration.md index ab43ec2cc4f..187fb2f73a1 100644 --- a/doc/administration/monitoring/performance/grafana_configuration.md +++ b/doc/administration/monitoring/performance/grafana_configuration.md @@ -3,7 +3,7 @@ [Grafana](http://grafana.org/) is a tool that allows you to visualize time series metrics through graphs and dashboards. It supports several backend data stores, including InfluxDB. GitLab writes performance data to InfluxDB -and Grafana will allow you to query InfluxDB to display useful graphs. +and Grafana will allow you to query to display useful graphs. For the easiest installation and configuration, install Grafana on the same server as InfluxDB. For larger installations, you may want to split out these @@ -11,11 +11,13 @@ services. ## Installation -Grafana supplies package repositories (Yum/Apt) for easy installation. +[GitLab Omnibus can help you install Grafana (recommended)](https://docs.gitlab.com/omnibus/settings/grafana.html) +or Grafana supplies package repositories (Yum/Apt) for easy installation. See [Grafana installation documentation](http://docs.grafana.org/installation/) for detailed steps. -> **Note**: Before starting Grafana for the first time, set the admin user +NOTE: **Note:** +Before starting Grafana for the first time, set the admin user and password in `/etc/grafana/grafana.ini`. Otherwise, the default password will be `admin`. diff --git a/doc/administration/monitoring/prometheus/gitlab_metrics.md b/doc/administration/monitoring/prometheus/gitlab_metrics.md index c243dd9edbb..3dcd1593099 100644 --- a/doc/administration/monitoring/prometheus/gitlab_metrics.md +++ b/doc/administration/monitoring/prometheus/gitlab_metrics.md @@ -43,10 +43,11 @@ The following metrics are available: | redis_ping_latency_seconds | Gauge | 9.4 | Round trip time of the redis ping | | user_session_logins_total | Counter | 9.4 | Counter of how many users have logged in | | upload_file_does_not_exist | Counter | 10.7 in EE, 11.5 in CE | Number of times an upload record could not find its file | -| failed_login_captcha_total | Gauge | 11.0 | Counter of failed CAPTCHA attempts during login | -| successful_login_captcha_total | Gauge | 11.0 | Counter of successful CAPTCHA attempts during login | -| unicorn_active_connections | Gauge | 11.0 | The number of active Unicorn connections (workers) | -| unicorn_queued_connections | Gauge | 11.0 | The number of queued Unicorn connections | +| failed_login_captcha_total | Gauge | 11.0 | Counter of failed CAPTCHA attempts during login | +| successful_login_captcha_total | Gauge | 11.0 | Counter of successful CAPTCHA attempts during login | +| unicorn_active_connections | Gauge | 11.0 | The number of active Unicorn connections (workers) | +| unicorn_queued_connections | Gauge | 11.0 | The number of queued Unicorn connections | +| unicorn_workers | Gauge | 12.0 | The number of Unicorn workers | ## Sidekiq Metrics available for Geo **[PREMIUM]** @@ -86,8 +87,8 @@ the `monitoring.sidekiq_exporter` configuration option in `gitlab.yml`. | geo_wikis_checksum_mismatch_count | Gauge | 10.7 | Number of wikis that checksum mismatch on secondary | url | geo_repositories_checked_count | Gauge | 11.1 | Number of repositories that have been checked via `git fsck` | url | geo_repositories_checked_failed_count | Gauge | 11.1 | Number of repositories that have a failure from `git fsck` | url -| geo_repositories_retrying_verification_count | Gauge | 11.2 | Number of repositories verification failures that Geo is actively trying to correct on secondary | url -| geo_wikis_retrying_verification_count | Gauge | 11.2 | Number of wikis verification failures that Geo is actively trying to correct on secondary | url +| geo_repositories_retrying_verification_count | Gauge | 11.2 | Number of repositories verification failures that Geo is actively trying to correct on secondary | url +| geo_wikis_retrying_verification_count | Gauge | 11.2 | Number of wikis verification failures that Geo is actively trying to correct on secondary | url ### Ruby metrics @@ -100,9 +101,31 @@ Some basic Ruby runtime metrics are available: | ruby_file_descriptors | Gauge | 11.1 | File descriptors per process | | ruby_memory_bytes | Gauge | 11.1 | Memory usage by process | | ruby_sampler_duration_seconds_total | Counter | 11.1 | Time spent collecting stats | +| ruby_process_cpu_seconds_total | Gauge | 12.0 | Total amount of CPU time per process | +| ruby_process_max_fds | Gauge | 12.0 | Maximum number of open file descriptors per process | +| ruby_process_resident_memory_bytes | Gauge | 12.0 | Memory usage by process, measured in bytes | +| ruby_process_start_time_seconds | Gauge | 12.0 | The elapsed time between system boot and the process started, measured in seconds | [GC.stat]: https://ruby-doc.org/core-2.3.0/GC.html#method-c-stat +## Puma Metrics **[EXPERIMENTAL]** + +When Puma is used instead of Unicorn, following metrics are available: + +| Metric | Type | Since | Description | +|:-------------------------------------------- |:------- |:----- |:----------- | +| puma_workers | Gauge | 12.0 | Total number of workers | +| puma_running_workers | Gauge | 12.0 | Number of booted workers | +| puma_stale_workers | Gauge | 12.0 | Number of old workers | +| puma_phase | Gauge | 12.0 | Phase number (increased during phased restarts) | +| puma_running | Gauge | 12.0 | Number of running threads | +| puma_queued_connections | Gauge | 12.0 | Number of connections in that worker's "todo" set waiting for a worker thread | +| puma_active_connections | Gauge | 12.0 | Number of threads processing a request | +| puma_pool_capacity | Gauge | 12.0 | Number of requests the worker is capable of taking right now | +| puma_max_threads | Gauge | 12.0 | Maximum number of worker threads | +| puma_idle_threads | Gauge | 12.0 | Number of spawned threads which are not processing a request | +| rack_state_total | Gauge | 12.0 | Number of requests in a given rack state | + ## Metrics shared directory GitLab's Prometheus client requires a directory to store metrics data shared between multi-process services. diff --git a/doc/administration/operations/fast_ssh_key_lookup.md b/doc/administration/operations/fast_ssh_key_lookup.md index 69f110805b7..3631ea0822f 100644 --- a/doc/administration/operations/fast_ssh_key_lookup.md +++ b/doc/administration/operations/fast_ssh_key_lookup.md @@ -34,10 +34,10 @@ feature for CentOS 6, follow [the instructions on how to build and install a cus By default, GitLab manages an `authorized_keys` file, which contains all the public SSH keys for users allowed to access GitLab. However, to maintain a -single source of truth, [Geo](https://docs.gitlab.com/ee/administration/geo/replication/index.html) needs to be configured to perform SSH fingerprint +single source of truth, [Geo](../geo/replication/index.md) needs to be configured to perform SSH fingerprint lookups via database lookup. -As part of [setting up Geo](https://docs.gitlab.com/ee/administration/geo/replication/index.html#setup-instructions), +As part of [setting up Geo](../geo/replication/index.md#setup-instructions), you will be required to follow the steps outlined below for both the primary and secondary nodes, but note that the `Write to "authorized keys" file` checkbox only needs to be unchecked on the primary node since it will be reflected diff --git a/doc/administration/packages.md b/doc/administration/packages.md index 5b9a13e3859..0d5f784b71e 100644 --- a/doc/administration/packages.md +++ b/doc/administration/packages.md @@ -8,12 +8,12 @@ The Packages feature allows GitLab to act as a repository for the following: | Software repository | Description | Available in GitLab version | | ------------------- | ----------- | --------------------------- | -| [Maven Repository](https://docs.gitlab.com/ee/user/project/packages/maven_repository.html) | The GitLab Maven Repository enables every project in GitLab to have its own space to store [Maven](https://maven.apache.org/) packages. | 11.3+ | -| [NPM Registry](https://docs.gitlab.com/ee/user/project/packages/npm_registry.html) | The GitLab NPM Registry enables every project in GitLab to have its own space to store [NPM](https://www.npmjs.com/) packages. | 11.7+ | +| [Maven Repository](../user/project/packages/maven_repository.md) | The GitLab Maven Repository enables every project in GitLab to have its own space to store [Maven](https://maven.apache.org/) packages. | 11.3+ | +| [NPM Registry](../user/project/packages/npm_registry.md) | The GitLab NPM Registry enables every project in GitLab to have its own space to store [NPM](https://www.npmjs.com/) packages. | 11.7+ | Don't you see your package management system supported yet? Please consider contributing -to GitLab. This [development documentation](https://docs.gitlab.com/ee/development/packages.html) will guide you through the process. +to GitLab. This [development documentation](../development/packages.md) will guide you through the process. ## Enabling the Packages feature diff --git a/doc/administration/pages/index.md b/doc/administration/pages/index.md index 1373bd56fe3..3a7ca517d56 100644 --- a/doc/administration/pages/index.md +++ b/doc/administration/pages/index.md @@ -291,6 +291,20 @@ Pages access control is disabled by default. To enable it: 1. [Reconfigure GitLab][reconfigure]. 1. Users can now configure it in their [projects' settings](../../user/project/pages/introduction.md#gitlab-pages-access-control-core-only). +### Running behind a proxy + +Like the rest of GitLab, Pages can be used in those environments where external +internet connectivity is gated by a proxy. In order to use a proxy for GitLab +pages: + +1. Configure in `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_pages['http_proxy'] = 'http://example:8080' + ``` + +1. [Reconfigure Gitlab][reconfigure] for the changes to take effect. + ## Activate verbose logging for daemon Verbose logging was [introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/2533) in @@ -376,6 +390,7 @@ Follow the steps below to configure GitLab Pages in a separate server. gitaly['enable'] = false alertmanager['enable'] = false node_exporter['enable'] = false + gitlab_rails['auto_migrate'] = false ``` 1. Run `sudo gitlab-ctl reconfigure`. 1. On `app1` apply the following changes to `/etc/gitlab/gitlab.rb`: diff --git a/doc/administration/raketasks/maintenance.md b/doc/administration/raketasks/maintenance.md index b295b7d5dc4..0b4c1ae15d6 100644 --- a/doc/administration/raketasks/maintenance.md +++ b/doc/administration/raketasks/maintenance.md @@ -205,25 +205,6 @@ cd /home/git/gitlab sudo -u git -H bundle exec rake gitlab:track_deployment RAILS_ENV=production ``` -## Create or repair repository hooks symlink - -If the GitLab shell hooks directory location changes or another circumstance -leads to the hooks symlink becoming missing or invalid, run this Rake task -to create or repair the symlinks. - -**Omnibus Installation** - -``` -sudo gitlab-rake gitlab:shell:create_hooks -``` - -**Source Installation** - -``` -cd /home/git/gitlab -sudo -u git -H bundle exec rake gitlab:shell:create_hooks RAILS_ENV=production -``` - ## Check TCP connectivity to a remote site Sometimes you need to know if your GitLab installation can connect to a TCP diff --git a/doc/administration/raketasks/project_import_export.md b/doc/administration/raketasks/project_import_export.md index f43bba0a7a7..6ca23aabdec 100644 --- a/doc/administration/raketasks/project_import_export.md +++ b/doc/administration/raketasks/project_import_export.md @@ -32,5 +32,4 @@ bundle exec rake gitlab:import_export:data RAILS_ENV=production ``` [ce-3050]: https://gitlab.com/gitlab-org/gitlab-ce/issues/3050 -[feature-flags]: https://docs.gitlab.com/ee/api/features.html [tmp]: ../../development/shared_files.md diff --git a/doc/analytics/README.md b/doc/analytics/README.md index 6b63edb5174..bfb15f6c4f3 100644 --- a/doc/analytics/README.md +++ b/doc/analytics/README.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/group/index.html#user-contribution-analysis-starter' +redirect_to: '../user/group/index.md#user-contribution-analysis-starter' --- -This document was moved to [another location](https://docs.gitlab.com/ee/user/group/index.html#user-contribution-analysis-starter) +This document was moved to [another location](../user/group/index.md#user-contribution-analysis-starter) diff --git a/doc/analytics/contribution_analytics.md b/doc/analytics/contribution_analytics.md index 38d71263bc1..e36f55071a4 100644 --- a/doc/analytics/contribution_analytics.md +++ b/doc/analytics/contribution_analytics.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/group/contribution_analytics/index.html' +redirect_to: '../user/group/contribution_analytics/index.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/user/group/contribution_analytics/index.html). +This document was moved to [another location](../user/group/contribution_analytics/index.md). diff --git a/doc/api/boards.md b/doc/api/boards.md index 28c73db6b98..a96206f5df3 100644 --- a/doc/api/boards.md +++ b/doc/api/boards.md @@ -141,6 +141,173 @@ Example response: } ``` +## Create a board **[STARTER]** + +Creates a board. + +``` +POST /projects/:id/boards +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `name` | string | yes | The name of the new board | + +```bash +curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/5/boards?name=newboard +``` + +Example response: + +```json + { + "id": 1, + "project": { + "id": 5, + "name": "Diaspora Project Site", + "name_with_namespace": "Diaspora / Diaspora Project Site", + "path": "diaspora-project-site", + "path_with_namespace": "diaspora/diaspora-project-site", + "http_url_to_repo": "http://example.com/diaspora/diaspora-project-site.git", + "web_url": "http://example.com/diaspora/diaspora-project-site" + }, + "name": "newboard", + "milestone": { + "id": 12 + "title": "10.0" + }, + "lists" : [ + { + "id" : 1, + "label" : { + "name" : "Testing", + "color" : "#F0AD4E", + "description" : null + }, + "position" : 1 + }, + { + "id" : 2, + "label" : { + "name" : "Ready", + "color" : "#FF0000", + "description" : null + }, + "position" : 2 + }, + { + "id" : 3, + "label" : { + "name" : "Production", + "color" : "#FF5F00", + "description" : null + }, + "position" : 3 + } + ] + } +``` + +## Update a board **[STARTER]** + +> [Introduced][ee-5954] in [GitLab Starter](https://about.gitlab.com/pricing/) 11.1. + +Updates a board. + +``` +PUT /projects/:id/boards/:board_id +``` + +| Attribute | Type | Required | Description | +| ------------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `board_id` | integer | yes | The ID of a board | +| `name` | string | no | The new name of the board | +| `assignee_id` | integer | no | The assignee the board should be scoped to | +| `milestone_id` | integer | no | The milestone the board should be scoped to | +| `labels` | string | no | Comma-separated list of label names which the board should be scoped to | +| `weight` | integer | no | The weight range from 0 to 9, to which the board should be scoped to | + + +```bash +curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/5/boards/1?name=new_name&milestone_id=43&assignee_id=1&labels=Doing&weight=4 +``` + +Example response: + +```json + { + "id": 1, + "project": { + "id": 5, + "name": "Diaspora Project Site", + "name_with_namespace": "Diaspora / Diaspora Project Site", + "path": "diaspora-project-site", + "path_with_namespace": "diaspora/diaspora-project-site", + "created_at": "2018-07-03T05:48:49.982Z", + "default_branch": null, + "tag_list": [], + "ssh_url_to_repo": "ssh://user@example.com/diaspora/diaspora-project-site.git", + "http_url_to_repo": "http://example.com/diaspora/diaspora-project-site.git", + "web_url": "http://example.com/diaspora/diaspora-project-site", + "readme_url": null, + "avatar_url": null, + "star_count": 0, + "forks_count": 0, + "last_activity_at": "2018-07-03T05:48:49.982Z" + }, + "lists": [], + "name": "new_name", + "group": null, + "milestone": { + "id": 43, + "iid": 1, + "project_id": 15, + "title": "Milestone 1", + "description": "Milestone 1 desc", + "state": "active", + "created_at": "2018-07-03T06:36:42.618Z", + "updated_at": "2018-07-03T06:36:42.618Z", + "due_date": null, + "start_date": null, + "web_url": "http://example.com/root/board1/milestones/1" + }, + "assignee": { + "id": 1, + "name": "Administrator", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", + "web_url": "http://example.com/root" + }, + "labels": [{ + "id": 10, + "name": "Doing", + "color": "#5CB85C", + "description": null + }], + "weight": 4 + } +``` + +## Delete a board **[STARTER]** + +Deletes a board. + +``` +DELETE /projects/:id/boards/:board_id +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | +| `board_id` | integer | yes | The ID of a board | + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/5/boards/1 +``` + ## List board lists Get a list of the board's lists. @@ -237,7 +404,15 @@ POST /projects/:id/boards/:board_id/lists | --------- | ---- | -------- | ----------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user | | `board_id` | integer | yes | The ID of a board | -| `label_id` | integer | yes | The ID of a label | +| `label_id` | integer | no | The ID of a label | +| `assignee_id` **[PREMIUM]** | integer | no | The ID of a user | +| `milestone_id` **[PREMIUM]** | integer | no | The ID of a milestone | + +NOTE: **Note**: +Label, assignee and milestone arguments are mutually exclusive, +that is, only one of them are accepted in a request. +Check the [Issue Board docs](../user/project/issue_board.md#summary-of-features-per-tier) +for more information regarding the required license for each list type. ```bash curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/5/boards/1/lists?label_id=5 @@ -307,3 +482,5 @@ DELETE /projects/:id/boards/:board_id/lists/:list_id ```bash curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/5/boards/1/lists/1 ``` + +[ee-5954]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/5954 diff --git a/doc/api/commits.md b/doc/api/commits.md index 92f53c7b5e6..25015fad9e3 100644 --- a/doc/api/commits.md +++ b/doc/api/commits.md @@ -75,6 +75,7 @@ POST /projects/:id/repository/commits | `branch` | string | yes | Name of the branch to commit into. To create a new branch, also provide `start_branch`. | | `commit_message` | string | yes | Commit message | | `start_branch` | string | no | Name of the branch to start the new commit from | +| `start_project` | integer/string | no | The project ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) to start the commit from. Defaults to the value of `id`. | | `actions[]` | array | yes | An array of action hashes to commit as a batch. See the next table for what attributes it can take. | | `author_email` | string | no | Specify the commit author's email address | | `author_name` | string | no | Specify the commit author's name | diff --git a/doc/api/discussions.md b/doc/api/discussions.md index 07a6201b10b..9defef4fd53 100644 --- a/doc/api/discussions.md +++ b/doc/api/discussions.md @@ -1,6 +1,12 @@ # Discussions API -Discussions are set of related notes on snippets, issues, merge requests or commits. +Discussions are a set of related notes on: + +- Snippets +- Issues +- Epics **[ULTIMATE]** +- Merge requests +- Commits This includes system notes, which are notes about changes to the object (for example, when a milestone changes, there will be a corresponding system note). Label notes are not part of this API, but recorded as separate events in [resource label events](resource_label_events.md). @@ -424,6 +430,214 @@ Parameters: curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/5/snippets/11/discussions/636 ``` +## Epics **[ULTIMATE]** + +### List group epic discussions + +Gets a list of all discussions for a single epic. + +``` +GET /groups/:id/epics/:epic_id/discussions +``` + +| Attribute | Type | Required | Description | +| ------------------- | ---------------- | ---------- | ------------ | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `epic_id` | integer | yes | The ID of an epic | + +```json +[ + { + "id": "6a9c1750b37d513a43987b574953fceb50b03ce7", + "individual_note": false, + "notes": [ + { + "id": 1126, + "type": "DiscussionNote", + "body": "discussion text", + "attachment": null, + "author": { + "id": 1, + "name": "root", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/00afb8fb6ab07c3ee3e9c1f38777e2f4?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "created_at": "2018-03-03T21:54:39.668Z", + "updated_at": "2018-03-03T21:54:39.668Z", + "system": false, + "noteable_id": 3, + "noteable_type": "Epic", + "noteable_id": null, + "resolvable": false + }, + { + "id": 1129, + "type": "DiscussionNote", + "body": "reply to the discussion", + "attachment": null, + "author": { + "id": 1, + "name": "root", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/00afb8fb6ab07c3ee3e9c1f38777e2f4?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "created_at": "2018-03-04T13:38:02.127Z", + "updated_at": "2018-03-04T13:38:02.127Z", + "system": false, + "noteable_id": 3, + "noteable_type": "Epic", + "noteable_id": null, + "resolvable": false + } + ] + }, + { + "id": "87805b7c09016a7058e91bdbe7b29d1f284a39e6", + "individual_note": true, + "notes": [ + { + "id": 1128, + "type": null, + "body": "a single comment", + "attachment": null, + "author": { + "id": 1, + "name": "root", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/00afb8fb6ab07c3ee3e9c1f38777e2f4?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "created_at": "2018-03-04T09:17:22.520Z", + "updated_at": "2018-03-04T09:17:22.520Z", + "system": false, + "noteable_id": 3, + "noteable_type": "Epic", + "noteable_id": null, + "resolvable": false + } + ] + } +] +``` + +```bash +curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/5/epics/11/discussions +``` + +### Get single epic discussion + +Returns a single discussion for a specific group epic + +``` +GET /groups/:id/epics/:epic_id/discussions/:discussion_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `epic_id` | integer | yes | The ID of an epic | +| `discussion_id` | integer | yes | The ID of a discussion | + +```bash +curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/5/epics/11/discussions/6a9c1750b37d513a43987b574953fceb50b03ce7 +``` + +### Create new epic discussion + +Creates a new discussion to a single group epic. This is similar to creating +a note but but another comments (replies) can be added to it later. + +``` +POST /groups/:id/epics/:epic_id/discussions +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `epic_id` | integer | yes | The ID of an epic | +| `body` | string | yes | The content of a discussion | +| `created_at` | string | no | Date time string, ISO 8601 formatted, e.g. 2016-03-11T03:45:40Z (requires admin or project/group owner rights) | + +```bash +curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/5/epics/11/discussions?body=comment +``` + +### Add note to existing epic discussion + +Adds a new note to the discussion. This can also +[create a discussion from a single comment](../user/discussions/#start-a-discussion-by-replying-to-a-standard-comment). + +``` +POST /groups/:id/epics/:epic_id/discussions/:discussion_id/notes +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `epic_id` | integer | yes | The ID of an epic | +| `discussion_id` | integer | yes | The ID of a discussion | +| `note_id` | integer | yes | The ID of a discussion note | +| `body` | string | yes | The content of a discussion | +| `created_at` | string | no | Date time string, ISO 8601 formatted, e.g. 2016-03-11T03:45:40Z (requires admin or project/group owner rights) | + +```bash +curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/5/epics/11/discussions/6a9c1750b37d513a43987b574953fceb50b03ce7/notes?body=comment +``` + +### Modify existing epic discussion note + +Modify existing discussion note of an epic. + +``` +PUT /groups/:id/epics/:epic_id/discussions/:discussion_id/notes/:note_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `epic_id` | integer | yes | The ID of an epic | +| `discussion_id` | integer | yes | The ID of a discussion | +| `note_id` | integer | yes | The ID of a discussion note | +| `body` | string | yes | The content of a discussion | + +```bash +curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/5/epics/11/discussions/6a9c1750b37d513a43987b574953fceb50b03ce7/notes/1108?body=comment +``` + +### Delete an epic discussion note + +Deletes an existing discussion note of an epic. + +``` +DELETE /groups/:id/epics/:epic_id/discussions/:discussion_id/notes/:note_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `epic_id` | integer | yes | The ID of an epic | +| `discussion_id` | integer | yes | The ID of a discussion | +| `note_id` | integer | yes | The ID of a discussion note | + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/5/epics/11/discussions/636 +``` + ## Merge requests ### List project merge request discussions diff --git a/doc/api/epic_issues.md b/doc/api/epic_issues.md index 438a3361dcc..ec59ea7068e 100644 --- a/doc/api/epic_issues.md +++ b/doc/api/epic_issues.md @@ -7,6 +7,7 @@ If a user is not a member of a group and the group is private, a `GET` request o Epics are available only in Ultimate. If epics feature is not available a `403` status code will be returned. ## List issues for an epic + Gets all issues that are assigned to an epic and the authenticated user has access to. ``` diff --git a/doc/api/epic_links.md b/doc/api/epic_links.md index 619ae6ea2dc..9ad90a6d0f1 100644 --- a/doc/api/epic_links.md +++ b/doc/api/epic_links.md @@ -12,6 +12,7 @@ If a user is not a member of a group and the group is private, a `GET` request o Epics are available only in the [Ultimate/Gold tier](https://about.gitlab.com/pricing/). If the epics feature is not available, a `403` status code will be returned. ## List epics related to a given epic + Gets all child epics of an epic. ``` @@ -125,7 +126,7 @@ POST /groups/:id/epics/:epic_iid/epics | --------------- | -------------- | -------- | ------------------------------------------------------------------------------------------------------------------ | | `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | | `epic_iid` | integer | yes | The internal ID of the (future parent) epic. | -| `title` | integer | yes | The global ID of the child epic. Internal ID can't be used because they can conflict with epics from other groups. | +| `title` | string | yes | The title of a newly created epic. | ```bash curl --header POST "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/1/epics/5/epics?title=Newpic @@ -141,6 +142,7 @@ Example response: "group_id": 49, "parent_id": 23, "has_children": false, + "has_issues": false, "reference": "&2", "url": "http://localhost/groups/group16/-/epics/2", "relation_url": "http://localhost/groups/group16/-/epics/1/links/24" diff --git a/doc/api/geo_nodes.md b/doc/api/geo_nodes.md index a1cb524499f..ea31abdd87e 100644 --- a/doc/api/geo_nodes.md +++ b/doc/api/geo_nodes.md @@ -192,12 +192,10 @@ Example response: "job_artifacts_failed_count": nil, "job_artifacts_synced_missing_on_primary_count": 0, "job_artifacts_synced_in_percentage": "0.00%", - "repositories_count": 41, "projects_count": 41, "repositories_failed_count": nil, "repositories_synced_count": nil, "repositories_synced_in_percentage": "0.00%", - "wikis_count": 41, "wikis_failed_count": nil, "wikis_synced_count": nil, "wikis_synced_in_percentage": "0.00%", @@ -257,12 +255,10 @@ Example response: "job_artifacts_failed_count": 1, "job_artifacts_synced_missing_on_primary_count": 0, "job_artifacts_synced_in_percentage": "50.00%", - "repositories_count": 41, "projects_count": 41, "repositories_failed_count": 1, "repositories_synced_count": 40, "repositories_synced_in_percentage": "97.56%", - "wikis_count": 41, "wikis_failed_count": 0, "wikis_synced_count": 41, "wikis_synced_in_percentage": "100.00%", @@ -300,7 +296,8 @@ Example response: ] ``` -Note: fields `wikis_count` and `repositories_count` are deprecated and will be deleted soon. Please use `projects_count` instead. +NOTE: **Note:** +In GitLab 12.0, deprecated fields `wikis_count` and `repositories_count` were removed. Use `projects_count` instead. ## Retrieve status about a specific Geo node @@ -337,12 +334,10 @@ Example response: "job_artifacts_failed_count": 1, "job_artifacts_synced_missing_on_primary_count": 0, "job_artifacts_synced_in_percentage": "50.00%", - "repositories_count": 41, "projects_count": 41, "repositories_failed_count": 1, "repositories_synced_count": 40, "repositories_synced_in_percentage": "97.56%", - "wikis_count": 41, "wikis_failed_count": 0, "wikis_synced_count": 41, "wikis_synced_in_percentage": "100.00%", @@ -362,7 +357,8 @@ Example response: Note: The `health_status` parameter can only be in an "Healthy" or "Unhealthy" state, while the `health` parameter can be empty, "Healthy", or contain the actual error message. -Note: Fields `wikis_count` and `repositories_count` are deprecated and will be deleted soon. Please use `projects_count` instead. +NOTE: **Note:** +In GitLab 12.0, deprecated fields `wikis_count` and `repositories_count` were removed. Use `projects_count` instead. ## Retrieve project sync or verification failures that occurred on the current node diff --git a/doc/api/graphql/index.md b/doc/api/graphql/index.md index 10e1ef0e533..88e657a5d2f 100644 --- a/doc/api/graphql/index.md +++ b/doc/api/graphql/index.md @@ -47,6 +47,15 @@ A first iteration of a GraphQL API includes the following queries 1. `project` : Within a project it is also possible to fetch a `mergeRequest` by IID. 1. `group` : Only basic group information is currently supported. +1. `namespace` : Within a namespace it is also possible to fetch `projects`. + +### Multiplex queries + +GitLab supports batching queries into a single request using +[apollo-link-batch-http](https://www.apollographql.com/docs/link/links/batch-http). More +info about multiplexed queries is also available for +[graphql-ruby](https://graphql-ruby.org/queries/multiplex.html) the +library GitLab uses on the backend. ## GraphiQL diff --git a/doc/api/groups.md b/doc/api/groups.md index 907b443d355..20789a1d4a4 100644 --- a/doc/api/groups.md +++ b/doc/api/groups.md @@ -68,6 +68,7 @@ GET /groups?statistics=true "statistics": { "storage_size" : 212, "repository_size" : 33, + "wiki_size" : 100, "lfs_objects_size" : 123, "job_artifacts_size" : 57 diff --git a/doc/api/members.md b/doc/api/members.md index 0593d2c20ea..8784d577f99 100644 --- a/doc/api/members.md +++ b/doc/api/members.md @@ -62,7 +62,9 @@ Example response: ## List all members of a group or project including inherited members Gets a list of group or project members viewable by the authenticated user, including inherited members through ancestor groups. -Returns multiple times the same user (with different member attributes) when the user is a member of the project/group and of one or more ancestor group. +When a user is a member of the project/group and of one or more ancestor groups the user is returned only once with the project access_level (if exists) +or the access_level for the user in the first group which he belongs to in the project groups ancestors chain. +**Note:** We plan to [change](https://gitlab.com/gitlab-org/gitlab-ce/issues/62284) this behavior to return highest access_level instead. ``` GET /groups/:id/members/all diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index 7992af15448..9529a9ec1f5 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -1159,33 +1159,29 @@ Parameters: } ``` -## Merge to default merge ref path +## Returns the up to date merge-ref HEAD commit Merge the changes between the merge request source and target branches into `refs/merge-requests/:iid/merge` -ref, of the target project repository. This ref will have the state the target branch would have if +ref, of the target project repository, if possible. This ref will have the state the target branch would have if a regular merge action was taken. -This is not a regular merge action given it doesn't change the merge request state in any manner. +This is not a regular merge action given it doesn't change the merge request target branch state in any manner. -This ref (`refs/merge-requests/:iid/merge`) is **always** overwritten when submitting -requests to this API, so none of its state is kept or used in the process. +This ref (`refs/merge-requests/:iid/merge`) isn't necessarily overwritten when submitting +requests to this API, though it'll make sure the ref has the latest possible state. -If the merge request has conflicts, is empty or already merged, -you'll get a `400` and a descriptive error message. If you don't have permissions to do so, -you'll get a `403`. +If the merge request has conflicts, is empty or already merged, you'll get a `400` and a descriptive error message. -It returns the HEAD commit of `refs/merge-requests/:iid/merge` in the response body in -case of `200`. +It returns the HEAD commit of `refs/merge-requests/:iid/merge` in the response body in case of `200`. ``` -PUT /projects/:id/merge_requests/:merge_request_iid/merge_to_ref +GET /projects/:id/merge_requests/:merge_request_iid/merge_ref ``` Parameters: - `id` (required) - The ID or [URL-encoded path of the project](README.md#namespaced-path-encoding) owned by the authenticated user - `merge_request_iid` (required) - Internal ID of MR -- `merge_commit_message` (optional) - Custom merge commit message ```json { diff --git a/doc/api/notes.md b/doc/api/notes.md index dfde80c6441..c09129c22d4 100644 --- a/doc/api/notes.md +++ b/doc/api/notes.md @@ -1,6 +1,11 @@ # Notes API -Notes are comments on snippets, issues or merge requests. +Notes are comments on: + +- Snippets +- Issues +- Merge requests +- Epics **[ULTIMATE]** This includes system notes, which are notes about changes to the object (for example, when a milestone changes, there will be a corresponding system note). Label notes are not part of this API, but recorded as separate events in [resource label events](resource_label_events.md). @@ -390,3 +395,126 @@ Parameters: ```bash curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/5/merge_requests/7/notes/1602 ``` + +## Epics **[ULTIMATE]** + +### List all epic notes + +Gets a list of all notes for a single epic. Epic notes are comments users can post to an epic. + +``` +GET /groups/:id/epics/:epic_id/notes +GET /groups/:id/epics/:epic_id/notes?sort=asc&order_by=updated_at +``` + +| Attribute | Type | Required | Description | +| ------------------- | ---------------- | ---------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `epic_id` | integer | yes | The ID of a group epic | +| `sort` | string | no | Return epic notes sorted in `asc` or `desc` order. Default is `desc` | +| `order_by` | string | no | Return epic notes ordered by `created_at` or `updated_at` fields. Default is `created_at` | + +```bash +curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/5/epics/11/notes +``` + +### Get single epic note + +Returns a single note for a given epic. + +``` +GET /groups/:id/epics/:epic_id/notes/:note_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `epic_id` | integer | yes | The ID of an epic | +| `note_id` | integer | yes | The ID of a note | + +```json +{ + "id": 52, + "title": "Epic", + "file_name": "epic.rb", + "author": { + "id": 1, + "username": "pipin", + "email": "admin@example.com", + "name": "Pip", + "state": "active", + "created_at": "2013-09-30T13:46:01Z" + }, + "expires_at": null, + "updated_at": "2013-10-02T07:34:20Z", + "created_at": "2013-10-02T07:34:20Z" +} +``` + +```bash +curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/5/epics/11/notes/1 +``` + +### Create new epic note + +Creates a new note for a single epic. Epic notes are comments users can post to an epic. +If you create a note where the body only contains an Award Emoji, you'll receive this object back. + +``` +POST /groups/:id/epics/:epic_id/notes +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `epic_id` | integer | yes | The ID of an epic | +| `body` | string | yes | The content of a note | + +```bash +curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/5/snippet/11/notes?body=note +``` + +### Modify existing epic note + +Modify existing note of an epic. + +``` +PUT /groups/:id/epics/:epic_id/notes/:note_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `epic_id` | integer | yes | The ID of an epic | +| `note_id` | integer | yes | The ID of a note | +| `body` | string | yes | The content of a note | + +```bash +curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/5/snippet/11/notes?body=note +``` + +### Delete an epic note + +Deletes an existing note of an epic. + +``` +DELETE /groups/:id/epics/:epic_id/notes/:note_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `epic_id` | integer | yes | The ID of an epic | +| `note_id` | integer | yes | The ID of a note | + +```bash +curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/5/epics/52/notes/1659 +``` diff --git a/doc/api/project_level_variables.md b/doc/api/project_level_variables.md index 4a6f5624394..3b00f6f140e 100644 --- a/doc/api/project_level_variables.md +++ b/doc/api/project_level_variables.md @@ -52,7 +52,9 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a { "key": "TEST_VARIABLE_1", "variable_type": "env_var", - "value": "TEST_1" + "value": "TEST_1", + "protected": false, + "masked": true } ``` @@ -71,6 +73,7 @@ POST /projects/:id/variables | `value` | string | yes | The `value` of a variable | | `variable_type` | string | no | The type of a variable. Available types are: `env_var` (default) and `file` | | `protected` | boolean | no | Whether the variable is protected | +| `masked` | boolean | no | Whether the variable is masked | ``` curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/variables" --form "key=NEW_VARIABLE" --form "value=new value" @@ -81,7 +84,8 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitla "key": "NEW_VARIABLE", "value": "new value", "variable_type": "env_var", - "protected": false + "protected": false, + "masked": false } ``` @@ -100,6 +104,7 @@ PUT /projects/:id/variables/:key | `value` | string | yes | The `value` of a variable | | `variable_type` | string | no | The type of a variable. Available types are: `env_var` (default) and `file` | | `protected` | boolean | no | Whether the variable is protected | +| `masked` | boolean | no | Whether the variable is masked | ``` curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/variables/NEW_VARIABLE" --form "value=updated value" @@ -110,7 +115,8 @@ curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab "key": "NEW_VARIABLE", "value": "updated value", "variable_type": "env_var", - "protected": true + "protected": true, + "masked": false } ``` diff --git a/doc/api/projects.md b/doc/api/projects.md index 951961e45ff..75669d85803 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -153,6 +153,7 @@ When the user is authenticated and `simple` is not set this returns something li "commit_count": 37, "storage_size": 1038090, "repository_size": 1038090, + "wiki_size" : 0, "lfs_objects_size": 0, "job_artifacts_size": 0 }, @@ -234,6 +235,7 @@ When the user is authenticated and `simple` is not set this returns something li "commit_count": 12, "storage_size": 2066080, "repository_size": 2066080, + "wiki_size" : 0, "lfs_objects_size": 0, "job_artifacts_size": 0 }, @@ -342,6 +344,7 @@ GET /users/:user_id/projects "commit_count": 37, "storage_size": 1038090, "repository_size": 1038090, + "wiki_size" : 0, "lfs_objects_size": 0, "job_artifacts_size": 0 }, @@ -423,6 +426,7 @@ GET /users/:user_id/projects "commit_count": 12, "storage_size": 2066080, "repository_size": 2066080, + "wiki_size" : 0, "lfs_objects_size": 0, "job_artifacts_size": 0 }, @@ -548,6 +552,7 @@ GET /projects/:id "commit_count": 37, "storage_size": 1038090, "repository_size": 1038090, + "wiki_size" : 0, "lfs_objects_size": 0, "job_artifacts_size": 0 }, diff --git a/doc/api/repository_storage_health.md b/doc/api/repository_storage_health.md deleted file mode 100644 index edf4b04acea..00000000000 --- a/doc/api/repository_storage_health.md +++ /dev/null @@ -1,5 +0,0 @@ -# Circuitbreaker API - -NOTE: **Deprecated:** -Support of the circuit breaker is removed, as Gitaly can be configured to -to work without NFS and [communicate solely over HTTP](../administration/gitaly/index.md). diff --git a/doc/api/resource_label_events.md b/doc/api/resource_label_events.md index e1f9ffa9472..f0a7ac4e41d 100644 --- a/doc/api/resource_label_events.md +++ b/doc/api/resource_label_events.md @@ -88,6 +88,92 @@ Parameters: curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/5/issues/11/resource_label_events/1 ``` +## Epics **[ULTIMATE]** + +### List group epic label events + +Gets a list of all label events for a single epic. + +``` +GET /groups/:id/epics/:epic_id/resource_label_events +``` + +| Attribute | Type | Required | Description | +| ------------------- | ---------------- | ---------- | ------------ | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `epic_id` | integer | yes | The ID of an epic | + +```json +[ + { + "id": 106, + "user": { + "id": 1, + "name": "Administrator", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", + "web_url": "http://gitlab.example.com/root" + }, + "created_at": "2018-08-19T11:43:01.746Z", + "resource_type": "Epic", + "resource_id": 33, + "label": { + "id": 73, + "name": "a1", + "color": "#34495E", + "description": "" + }, + "action": "add" + }, + { + "id": 107, + "user": { + "id": 1, + "name": "Administrator", + "username": "root", + "state": "active", + "avatar_url": "https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", + "web_url": "http://gitlab.example.com/root" + }, + "created_at": "2018-08-19T11:43:01.746Z", + "resource_type": "Epic", + "resource_id": 33, + "label": { + "id": 37, + "name": "glabel2", + "color": "#A8D695", + "description": "" + }, + "action": "add" + } +] +``` + +```bash +curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/1/epics/11/resource_label_events +``` + +### Get single epic label event + +Returns a single label event for a specific group epic + +``` +GET /groups/:id/epics/:epic_id/resource_label_events/:resource_label_event_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| --------------- | -------------- | -------- | ----------- | +| `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) | +| `epic_id` | integer | yes | The ID of an epic | +| `resource_label_event_id` | integer | yes | The ID of a label event | + +```bash +curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/1/epics/11/resource_label_events/107 +``` + ## Merge requests ### List project merge request label events diff --git a/doc/api/scim.md b/doc/api/scim.md index 4595c6f2ed3..3870ea788e7 100644 --- a/doc/api/scim.md +++ b/doc/api/scim.md @@ -1,4 +1,4 @@ -# SCIM API +# SCIM API **[SILVER ONLY]** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/9388) in [GitLab Silver](https://about.gitlab.com/pricing/) 11.10. diff --git a/doc/api/services.md b/doc/api/services.md index 742abccb69e..898cfad7254 100644 --- a/doc/api/services.md +++ b/doc/api/services.md @@ -754,6 +754,7 @@ Parameters: | `recipients` | string | yes | Comma-separated list of recipient email addresses | | `add_pusher` | boolean | no | Add pusher to recipients list | | `notify_only_broken_pipelines` | boolean | no | Notify only broken pipelines | +| `notify_only_default_branch` | boolean | no | Send notifications only for the default branch ([introduced in GitLab 12.0](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/28271)) | ### Delete Pipeline-Emails service @@ -1100,6 +1101,75 @@ Get JetBrains TeamCity CI service settings for a project. GET /projects/:id/services/teamcity ``` +## Jenkins CI **[STARTER]** + +A continuous integration and build server + +### Create/Edit Jenkins CI service + +Set Jenkins CI service for a project. + +``` +PUT /projects/:id/services/jenkins +``` + +Parameters: + +- `jenkins_url` (**required**) - Jenkins URL like http://jenkins.example.com +- `project_name` (**required**) - The URL-friendly project name. Example: my_project_name +- `username` (optional) - A user with access to the Jenkins server, if applicable +- `password` (optional) - The password of the user + +### Delete Jenkins CI service + +Delete Jenkins CI service for a project. + +``` +DELETE /projects/:id/services/jenkins +``` + +### Get Jenkins CI service settings + +Get Jenkins CI service settings for a project. + +``` +GET /projects/:id/services/jenkins +``` + +## Jenkins CI (Deprecated) Service + +A continuous integration and build server + +### Create/Edit Jenkins CI (Deprecated) service + +Set Jenkins CI (Deprecated) service for a project. + +``` +PUT /projects/:id/services/jenkins-deprecated +``` + +Parameters: + +- `project_url` (**required**) - Jenkins project URL like http://jenkins.example.com/job/my-project/ +- `multiproject_enabled` (optional) - Multi-project mode is configured in Jenkins GitLab Hook plugin +- `pass_unstable` (optional) - Unstable builds will be treated as passing + +### Delete Jenkins CI (Deprecated) service + +Delete Jenkins CI (Deprecated) service for a project. + +``` +DELETE /projects/:id/services/jenkins-deprecated +``` + +### Get Jenkins CI (Deprecated) service settings + +Get Jenkins CI (Deprecated) service settings for a project. + +``` +GET /projects/:id/services/jenkins-deprecated +``` + [jira-doc]: ../user/project/integrations/jira.md [old-jira-api]: https://gitlab.com/gitlab-org/gitlab-ce/blob/8-13-stable/doc/api/services.md#jira diff --git a/doc/api/snippets.md b/doc/api/snippets.md index f90447e124e..1ce0b1e7a62 100644 --- a/doc/api/snippets.md +++ b/doc/api/snippets.md @@ -165,15 +165,15 @@ Parameters: |:--------------|:-------|:---------|:---------------------------------------------------| | `title` | string | yes | Title of a snippet. | | `file_name` | string | yes | Name of a snippet file. | -| `content` | string | yes | Content of a snippet. | +| `code` | string | yes | Content of a snippet. | | `description` | string | no | Description of a snippet. | -| `visibility` | string | no | Snippet's [visibility](#snippet-visibility-level). | +| `visibility` | string | yes | Snippet's [visibility](#snippet-visibility-level). | Example request: ```sh curl --request POST \ - --data '{"title": "This is a snippet", "content": "Hello world", "description": "Hello World snippet", "file_name": "test.txt", "visibility": "internal" }' \ + --data '{"title": "This is a snippet", "code": "Hello world", "description": "Hello World snippet", "file_name": "test.txt", "visibility": "internal" }' \ --header 'Content-Type: application/json' \ --header "PRIVATE-TOKEN: valid_api_token" \ https://gitlab.example.com/api/v4/snippets @@ -222,14 +222,14 @@ Parameters: | `title` | string | no | Title of a snippet. | | `file_name` | string | no | Name of a snippet file. | | `description` | string | no | Description of a snippet. | -| `content` | string | no | Content of a snippet. | +| `code` | string | no | Content of a snippet. | | `visibility` | string | no | Snippet's [visibility](#snippet-visibility-level). | Example request: ```sh curl --request PUT \ - --data '{"title": "foo", "content": "bar"}' \ + --data '{"title": "foo", "code": "bar"}' \ --header 'Content-Type: application/json' \ --header "PRIVATE-TOKEN: valid_api_token" \ https://gitlab.example.com/api/v4/snippets/1 diff --git a/doc/api/users.md b/doc/api/users.md index d3e67d3d510..47028c679b8 100644 --- a/doc/api/users.md +++ b/doc/api/users.md @@ -140,8 +140,7 @@ GET /users "can_create_project": true, "two_factor_enabled": true, "external": false, - "private_profile": false, - "highest_role":10 + "private_profile": false } ] ``` @@ -257,7 +256,8 @@ Parameters: "can_create_project": true, "two_factor_enabled": true, "external": false, - "private_profile": false + "private_profile": false, + "highest_role":10 } ``` diff --git a/doc/api/vulnerabilities.md b/doc/api/vulnerabilities.md index 87f77613ad3..390d0966244 100644 --- a/doc/api/vulnerabilities.md +++ b/doc/api/vulnerabilities.md @@ -1,4 +1,4 @@ -# Vulnerabilities API +# Vulnerabilities API **[ULTIMATE]** Every API call to vulnerabilities must be authenticated. diff --git a/doc/articles/how_to_configure_ldap_gitlab_ee/index.md b/doc/articles/how_to_configure_ldap_gitlab_ee/index.md index 4ce96fcb230..3e6f3130437 100644 --- a/doc/articles/how_to_configure_ldap_gitlab_ee/index.md +++ b/doc/articles/how_to_configure_ldap_gitlab_ee/index.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/administration/auth/how_to_configure_ldap_gitlab_ee/index.html' +redirect_to: '../../administration/auth/how_to_configure_ldap_gitlab_ee/index.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/administration/auth/how_to_configure_ldap_gitlab_ee/index.html). +This document was moved to [another location](../../administration/auth/how_to_configure_ldap_gitlab_ee/index.md). diff --git a/doc/ci/caching/index.md b/doc/ci/caching/index.md index 3f72fe3e9eb..d703e106e76 100644 --- a/doc/ci/caching/index.md +++ b/doc/ci/caching/index.md @@ -100,7 +100,7 @@ From the perspective of the Runner, in order for cache to work effectively, one of the following must be true: - Use a single Runner for all your jobs. -- Use multiple Runners (in autoscale mode or not) that use. +- Use multiple Runners (in autoscale mode or not) that use [distributed caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching), where the cache is stored in S3 buckets (like shared Runners on GitLab.com). - Use multiple Runners (not in autoscale mode) of the same architecture that diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 5222cc45bc4..dd112dadc40 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -142,9 +142,12 @@ In order to do that, follow the steps: # The 'docker' hostname is the alias of the service container as described at # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services # - # Note that if you're using Kubernetes executor, the variable should be set to - # tcp://localhost:2375 because of how Kubernetes executor connects services + # Note that if you're using the Kubernetes executor, the variable should be set to + # tcp://localhost:2375/ because of how the Kubernetes executor connects services # to the job container + # DOCKER_HOST: tcp://localhost:2375/ + # + # For non-Kubernetes executors, we use tcp://docker:2375/ DOCKER_HOST: tcp://docker:2375/ # When using dind, it's wise to use the overlayfs driver for # improved performance. diff --git a/doc/ci/enable_or_disable_ci.md b/doc/ci/enable_or_disable_ci.md index 7aa7de97c43..9934d543991 100644 --- a/doc/ci/enable_or_disable_ci.md +++ b/doc/ci/enable_or_disable_ci.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # How to enable or disable GitLab CI/CD To effectively use GitLab CI/CD, you need a valid [`.gitlab-ci.yml`](yaml/README.md) diff --git a/doc/ci/git_submodules.md b/doc/ci/git_submodules.md index 37078230b34..551044dd76f 100644 --- a/doc/ci/git_submodules.md +++ b/doc/ci/git_submodules.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Using Git submodules with GitLab CI > **Notes:** diff --git a/doc/ci/interactive_web_terminal/index.md b/doc/ci/interactive_web_terminal/index.md index 7109b2ec583..1387d4df500 100644 --- a/doc/ci/interactive_web_terminal/index.md +++ b/doc/ci/interactive_web_terminal/index.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Interactive Web Terminals > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/50144) in GitLab 11.3. diff --git a/doc/ci/junit_test_reports.md b/doc/ci/junit_test_reports.md index 799217c9a08..fa78f53f563 100644 --- a/doc/ci/junit_test_reports.md +++ b/doc/ci/junit_test_reports.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # JUnit test reports > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/45318) in GitLab 11.2. diff --git a/doc/ci/large_repositories/index.md b/doc/ci/large_repositories/index.md index 244ccbb92b0..29d649ad717 100644 --- a/doc/ci/large_repositories/index.md +++ b/doc/ci/large_repositories/index.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Optimizing GitLab for large repositories Large repositories consisting of more than 50k files in a worktree diff --git a/doc/ci/merge_request_pipelines/index.md b/doc/ci/merge_request_pipelines/index.md index b3ff55daea2..fe2fc790505 100644 --- a/doc/ci/merge_request_pipelines/index.md +++ b/doc/ci/merge_request_pipelines/index.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Pipelines for merge requests NOTE: **Note**: diff --git a/doc/ci/metrics_reports.md b/doc/ci/metrics_reports.md index 83a7094faaa..b7824402d45 100644 --- a/doc/ci/metrics_reports.md +++ b/doc/ci/metrics_reports.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Metrics Reports **[PREMIUM]** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/9788) in [GitLab Premium](https://about.gitlab.com/pricing) 11.10. diff --git a/doc/ci/multi_project_pipelines.md b/doc/ci/multi_project_pipelines.md index 556059c01b6..bcd92243d97 100644 --- a/doc/ci/multi_project_pipelines.md +++ b/doc/ci/multi_project_pipelines.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Multi-project pipelines **[PREMIUM]** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/2121) in @@ -134,6 +138,34 @@ staging: The `ENVIRONMENT` variable will be passed to every job defined in a downstream pipeline. It will be available as an environment variable when GitLab Runner picks a job. +In the following configuration, the `MY_VARIABLE` variable will be passed +downstream, because jobs inherit variables declared in top-level `variables`: + +```yaml +variables: + MY_VARIABLE: my-value + +my-pipeline: + variables: + ENVIRONMENT: something + trigger: my/project +``` + +You might want to pass some information about the upstream pipeline using, for +example, predefined variables. In order to do that, you can use interpolation +to pass any variable. For example: + +```yaml +my-pipeline: + variables: + UPSTREAM_BRANCH: $CI_COMMIT_REF_NAME + trigger: my/project +``` + +In this scenario, the `UPSTREAM_BRANCH` variable with a value related to the +upstream pipeline will be passed to a `downstream` job, and will be available +within the context of all downstream builds. + ### Limitations Because bridge jobs are a little different to regular jobs, it is not diff --git a/doc/ci/pipelines.md b/doc/ci/pipelines.md index 4dbe1a85588..8b0634ed268 100644 --- a/doc/ci/pipelines.md +++ b/doc/ci/pipelines.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Creating and using CI/CD pipelines > Introduced in GitLab 8.8. diff --git a/doc/ci/quick_start/README.md b/doc/ci/quick_start/README.md index 015f1c0dc0f..11bcfd5dc2c 100644 --- a/doc/ci/quick_start/README.md +++ b/doc/ci/quick_start/README.md @@ -1,9 +1,18 @@ +--- +type: reference +--- + # Getting started with GitLab CI/CD ->**Note:** Starting from version 8.0, GitLab [Continuous Integration][ci] (CI) +NOTE: **Note:** +Starting from version 8.0, GitLab [Continuous Integration][ci] (CI) is fully integrated into GitLab itself and is [enabled] by default on all projects. +NOTE: **Note:** +Please keep in mind that only project Maintainers and Admin users have +the permissions to access a project's settings. + GitLab offers a [continuous integration][ci] service. If you [add a `.gitlab-ci.yml` file][yaml] to the root directory of your repository, and configure your GitLab project to use a [Runner], then each commit or @@ -35,11 +44,12 @@ project's **Pipelines** page. --- -This guide assumes that you: +This guide assumes that you have: -- have a working GitLab instance of version 8.0+r or are using - [GitLab.com](https://gitlab.com) -- have a project in GitLab that you would like to use CI for +- A working GitLab instance of version 8.0+r or are using + [GitLab.com](https://gitlab.com). +- A project in GitLab that you would like to use CI for. +- Maintainer or owner access to the project Let's break it down to pieces and work on solving the GitLab CI puzzle. @@ -73,6 +83,8 @@ You need to create a file named `.gitlab-ci.yml` in the root directory of your repository. Below is an example for a Ruby on Rails project. ```yaml +image: "ruby:2.5" + before_script: - apt-get update -qq && apt-get install -y -qq sqlite3 libsqlite3-dev nodejs - ruby -v diff --git a/doc/ci/review_apps/index.md b/doc/ci/review_apps/index.md index 1a71c5fd258..7b039fe6654 100644 --- a/doc/ci/review_apps/index.md +++ b/doc/ci/review_apps/index.md @@ -1,9 +1,13 @@ +--- +type: reference +--- + # Review Apps > - [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/21971) in GitLab 8.12. Further additions were made in GitLab 8.13 and 8.14. > - Inspired by [Heroku's Review Apps](https://devcenter.heroku.com/articles/github-integration-review-apps), which itself was inspired by [Fourchette](https://github.com/rainforestapp/fourchette). -Review Apps are a collaboration tool that takes the hard work out of providing an environment to showcase product changes. +Review Apps is a collaboration tool that takes the hard work out of providing an environment to showcase product changes. ## Introduction @@ -18,7 +22,7 @@ Review Apps: In the above example: -- A Review App is built every time a commit is pushed to`topic branch`. +- A Review App is built every time a commit is pushed to `topic branch`. - The reviewer fails two reviews before passing the third review. - Once the review as passed, `topic branch` is merged into `master` where it's deploy to staging. - After been approved in staging, the changes that were merged into `master` are deployed in to production. diff --git a/doc/ci/runners/README.md b/doc/ci/runners/README.md index ce55b231666..b089229ab58 100644 --- a/doc/ci/runners/README.md +++ b/doc/ci/runners/README.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Configuring GitLab Runners In GitLab CI, Runners run the code defined in [`.gitlab-ci.yml`](../yaml/README.md). diff --git a/doc/ci/services/README.md b/doc/ci/services/README.md index 2eda5d23976..7fe12eb53e7 100644 --- a/doc/ci/services/README.md +++ b/doc/ci/services/README.md @@ -1,13 +1,18 @@ --- comments: false +type: index --- -# GitLab CI Services +# GitLab CI services examples -GitLab CI uses the `services` keyword to define what docker containers should -be linked with your base image. Below is a list of examples you may use. +The [`services`](../docker/using_docker_images.md#what-is-a-service) +keyword defines a Docker image that runs during a `job` linked to the +Docker image that the image keyword defines. This allows you to access +the service image during build time. + +The service image can run any application, but the most common use +case is to run a database container, for example: - [Using MySQL](mysql.md) - [Using PostgreSQL](postgres.md) - [Using Redis](redis.md) -- [Using Other Services](../docker/using_docker_images.md#what-is-a-service) diff --git a/doc/ci/services/mysql.md b/doc/ci/services/mysql.md index 5fa378fc4c2..697452cee83 100644 --- a/doc/ci/services/mysql.md +++ b/doc/ci/services/mysql.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Using MySQL As many applications depend on MySQL as their database, you will eventually diff --git a/doc/ci/services/postgres.md b/doc/ci/services/postgres.md index 2e6d7ae94d2..211eea26eb0 100644 --- a/doc/ci/services/postgres.md +++ b/doc/ci/services/postgres.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Using PostgreSQL As many applications depend on PostgreSQL as their database, you will diff --git a/doc/ci/services/redis.md b/doc/ci/services/redis.md index 36f71427ae7..8b227154b06 100644 --- a/doc/ci/services/redis.md +++ b/doc/ci/services/redis.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Using Redis As many applications depend on Redis as their key-value store, you will diff --git a/doc/ci/ssh_keys/README.md b/doc/ci/ssh_keys/README.md index 9ed1ec5aa5c..69591ed605c 100644 --- a/doc/ci/ssh_keys/README.md +++ b/doc/ci/ssh_keys/README.md @@ -1,5 +1,6 @@ --- last_updated: 2017-12-13 +type: tutorial --- # Using SSH keys with GitLab CI/CD diff --git a/doc/ci/triggers/README.md b/doc/ci/triggers/README.md index ad80b5d8818..04c541fefe7 100644 --- a/doc/ci/triggers/README.md +++ b/doc/ci/triggers/README.md @@ -1,3 +1,7 @@ +--- +type: tutorial +--- + # Triggering pipelines through the API > **Notes**: diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index 67e1d316f02..2157a6dc097 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -91,10 +91,9 @@ This means that the value of the variable will be hidden in job logs, though it must match certain requirements to do so: - The value must be in a single line. -- The value must not have escape characters. -- The value must not use variables. -- The value must not have any whitespace. +- The value must contain only letters, numbers, or underscores. - The value must be at least 8 characters long. +- The value must not use variables. If the value does not meet the requirements above, then the CI variable will fail to save. In order to save, either alter the value to meet the masking requirements @@ -612,8 +611,8 @@ $'\''git'\'' "checkout" "-f" "-q" "dd648b2e48ce6518303b0bb580b2ee32fadaf045" Running on runner-8a2f473d-project-1796893-concurrent-0 via runner-8a2f473d-machine-1480971377-317a7d0f-digital-ocean-4gb... ++ export CI=true ++ CI=true -++ export CI_API_V4_API_URL=https://example.com:3000/api/v4 -++ CI_API_V4_API_URL=https://example.com:3000/api/v4 +++ export CI_API_V4_URL=https://example.com:3000/api/v4 +++ CI_API_V4_URL=https://example.com:3000/api/v4 ++ export CI_DEBUG_TRACE=false ++ CI_DEBUG_TRACE=false ++ export CI_COMMIT_SHA=dd648b2e48ce6518303b0bb580b2ee32fadaf045 @@ -652,8 +651,8 @@ Running on runner-8a2f473d-project-1796893-concurrent-0 via runner-8a2f473d-mach ++ GITLAB_CI=true ++ export CI=true ++ CI=true -++ export CI_API_V4_API_URL=https://example.com:3000/api/v4 -++ CI_API_V4_API_URL=https://example.com:3000/api/v4 +++ export CI_API_V4_URL=https://example.com:3000/api/v4 +++ CI_API_V4_URL=https://example.com:3000/api/v4 ++ export GITLAB_CI=true ++ GITLAB_CI=true ++ export CI_JOB_ID=7046507 diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 31ff56e06f8..18c85618b1b 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -386,17 +386,12 @@ job: - branches@gitlab-org/gitlab-ce except: - master@gitlab-org/gitlab-ce - - release/.*@gitlab-org/gitlab-ce + - /^release/.*$/@gitlab-org/gitlab-ce ``` The above example will run `job` for all branches on `gitlab-org/gitlab-ce`, except `master` and those with names prefixed with `release/`. -NOTE: **Note:** -Because `@` is used to denote the beginning of a ref's repository path, -matching a ref name containing the `@` character in a regular expression -requires the use of the hex character code match `\x40`. - If a job does not have an `only` rule, `only: ['branches', 'tags']` is set by default. If it doesn't have an `except` rule, it is empty. @@ -415,6 +410,28 @@ job: only: ['branches', 'tags'] ``` +#### Regular expressions + +Because `@` is used to denote the beginning of a ref's repository path, +matching a ref name containing the `@` character in a regular expression +requires the use of the hex character code match `\x40`. + +Only the tag or branch name can be matched by a regular expression. +The repository path, if given, is always matched literally. + +If a regular expression shall be used to match the tag or branch name, +the entire ref name part of the pattern has to be a regular expression, +and must be surrounded by `/`. +(With regular expression flags appended after the closing `/`.) +So `issue-/.*/` won't work to match all tag names or branch names +that begin with `issue-`. + +TIP: **Tip** +Use anchors `^` and `$` to avoid the regular expression +matching only a substring of the tag name or branch name. +For example, `/^issue-.*$/` is equivalent to `/^issue-/`, +while just `/issue/` would also match a branch called `severe-issues`. + ### Supported `only`/`except` regexp syntax CAUTION: **Warning:** @@ -1986,7 +2003,7 @@ production: - deploy environment: name: production - url: https://$CI_PROJECT_PATH_SLUG.$AUTO_DEVOPS_DOMAIN + url: https://$CI_PROJECT_PATH_SLUG.$KUBE_INGRESS_BASE_DOMAIN only: - master ``` diff --git a/doc/customization/issue_and_merge_request_template.md b/doc/customization/issue_and_merge_request_template.md index 01c31728c21..adaa120a37e 100644 --- a/doc/customization/issue_and_merge_request_template.md +++ b/doc/customization/issue_and_merge_request_template.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/project/description_templates.html#setting-a-default-template-for-issues-and-merge-requests--starter' +redirect_to: '../user/project/description_templates.md#setting-a-default-template-for-issues-and-merge-requests--starter' --- -This document was moved to [description_templates](https://docs.gitlab.com/ee/user/project/description_templates.html#setting-a-default-template-for-issues-and-merge-requests--starter). +This document was moved to [description_templates](../user/project/description_templates.md#setting-a-default-template-for-issues-and-merge-requests--starter). diff --git a/doc/development/README.md b/doc/development/README.md index 2ff38d68a47..624665a42d1 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -59,6 +59,7 @@ description: 'Learn how to contribute to GitLab.' - [DeclarativePolicy framework](policies.md) - [How Git object deduplication works in GitLab](git_object_deduplication.md) - [Geo development](geo.md) +- [Routing](routing.md) ## Performance guides diff --git a/doc/development/architecture.md b/doc/development/architecture.md index 9a012f4299b..a0e4020da09 100644 --- a/doc/development/architecture.md +++ b/doc/development/architecture.md @@ -59,7 +59,7 @@ graph TB PgBouncerExporter[PgBouncer Exporter] --> PgBouncer Prometheus -- TCP 9187 --> PostgreSQLExporter Prometheus -- TCP 9100 --> NodeExporter[Node Exporter] - Prometheus -- TCP 9168 --> GitLabMonito[GitLab Monitor] + Prometheus -- TCP 9168 --> GitLabMonitor[GitLab Monitor] Prometheus -- TCP 9127 --> PgBouncerExporter GitLabMonitor --> PostgreSQL GitLabMonitor --> GitLabShell @@ -106,43 +106,43 @@ Component statuses are linked to configuration documentation for each component. ### Component list -| Component | Description | [Omnibus GitLab](https://docs.gitlab.com/omnibus/README.html) | [GitLab chart](https://docs.gitlab.com/charts/) | [Minikube Minimal](https://docs.gitlab.com/charts/development/minikube/#deploying-gitlab-with-minimal-settings) | [GitLab.com](https://gitlab.com) | CE/EE | -| --------- | ----------- |:--------------------:|:------------------:|:-----:|:--------:|:--------:| -| [NGINX](#nginx) | Routes requests to appropriate components, terminates SSL | [✅][nginx-omnibus] | [✅][nginx-charts] | [⚙][nginx-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | CE & EE | -| [Unicorn (GitLab Rails)](#unicorn) | Handles requests for the web interface and API | [✅][unicorn-omnibus] | [✅][unicorn-charts] | [✅][unicorn-charts] | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#unicorn) | CE & EE | -| [Sidekiq](#sidekiq) | Background jobs processor | [✅][sidekiq-omnibus] | [✅][sidekiq-charts] | [✅](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/index.html) | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#sidekiq) | CE & EE | -| [Gitaly](#gitaly) | Git RPC service for handling all git calls made by GitLab | [✅][gitaly-omnibus] | [✅][gitaly-charts] | [✅][gitaly-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | CE & EE | -| [GitLab Workhorse](#gitlab-workhorse) | Smart reverse proxy, handles large HTTP requests | [✅][workhorse-omnibus] | [✅][workhorse-charts] | [✅][workhorse-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | CE & EE | -| [GitLab Shell](#gitlab-shell) | Handles `git` over SSH sessions | [✅][shell-omnibus] | [✅][shell-charts] | [✅][shell-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | CE & EE | -| [GitLab Pages](#gitlab-pages) | Hosts static websites | [⚙][pages-omnibus] | [❌][pages-charts] | [❌][pages-charts] | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#gitlab-pages) | CE & EE | -| [Registry](#registry) | Container registry, allows pushing and pulling of images | [⚙][registry-omnibus] | [✅][registry-charts] | [✅][registry-charts] | [✅](https://docs.gitlab.com/ee/user/project/container_registry.html#build-and-push-images) | CE & EE | -| [Redis](#redis) | Caching service | [✅][redis-omnibus] | [✅][redis-omnibus] | [✅][redis-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | CE & EE | -| [PostgreSQL](#postgresql) | Database | [✅][postgres-omnibus] | [✅][postgres-charts] | [✅][postgres-charts] | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#postgresql) | CE & EE | -| [PgBouncer](#pgbouncer) | Database connection pooling, failover | [⚙][pgbouncer-omnibus] | [❌][pgbouncer-charts] | [❌][pgbouncer-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#database-architecture) | EE Only | -| [Consul](#consul) | Database node discovery, failover | [⚙][consul-omnibus] | [❌][consul-charts] | [❌][consul-charts] | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#consul) | EE Only | -| [GitLab self-monitoring: Prometheus](#prometheus) | Time-series database, metrics collection, and query service | [✅][prometheus-omnibus] | [✅][prometheus-charts] | [⚙][prometheus-charts] | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#prometheus) | CE & EE | -| [GitLab self-monitoring: Alertmanager](#alertmanager) | Deduplicates, groups, and routes alerts from Prometheus | [✅][alertmanager-omnibus] | [✅][alertmanager-charts] | [⚙][alertmanager-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | CE & EE | -| [GitLab self-monitoring: Grafana](#grafana) | Metrics dashboard | [⚙][grafana-omnibus] | [⤓][grafana-charts] | [⤓][grafana-charts] | [✅](https://dashboards.gitlab.com/d/RZmbBr7mk/gitlab-triage?refresh=30s) | CE & EE | -| [GitLab self-monitoring: Sentry](#sentry) | Track errors generated by the GitLab instance | [⤓][sentry-omnibus] | [❌][sentry-charts] | [❌][sentry-charts] | [✅](https://about.gitlab.com/handbook/support/workflows/services/gitlab_com/500_errors.html#searching-sentry) | CE & EE | -| [GitLab self-monitoring: Jaeger](#jaeger) | View traces generated by the GitLab instance | [❌][jaeger-omnibus] | [❌][jaeger-charts] | [❌][jaeger-charts] | [❌](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/4104) | CE & EE | -| [Redis Exporter](#redis-exporter) | Prometheus endpoint with Redis metrics | [✅][redis-exporter-omnibus] | [✅][redis-exporter-charts] | [✅][redis-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | CE & EE | -| [Postgres Exporter](#postgres-exporter) | Prometheus endpoint with PostgreSQL metrics | [✅][postgres-exporter-omnibus] | [✅][postgres-exporter-charts] | [✅][postgres-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | CE & EE | -| [PgBouncer Exporter](#pgbouncer-exporter) | Prometheus endpoint with PgBouncer metrics | [⚙][pgbouncer-exporter-omnibus] | [❌][pgbouncer-exporter-charts] | [❌][pgbouncer-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | CE & EE | -| [GitLab Monitor](#gitlab-monitor) | Generates a variety of GitLab metrics | [✅][gitlab-monitor-omnibus] | [❌][gitab-monitor-charts] | [❌][gitab-monitor-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | CE & EE | -| [Node Exporter](#node-exporter) | Prometheus endpoint with system metrics | [✅][node-exporter-omnibus] | [❌][node-exporter-charts] | [❌][node-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | CE & EE | -| [Mattermost](#mattermost) | Open-source Slack alternative | [⚙][mattermost-omnibus] | [⤓][mattermost-charts] | [⤓][mattermost-charts] | [⤓](https://docs.gitlab.com/ee/user/project/integrations/mattermost_slash_commands.html#manual-configuration), [⤓](https://docs.gitlab.com/ee/user/project/integrations/mattermost.html) | CE & EE | -| [Minio](#minio) | Object storage service | [⤓][minio-omnibus] | [✅][minio-charts] | [✅][minio-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#storage-architecture) | CE & EE | -| [Runner](#gitlab-runner) | Executes GitLab CI jobs | [⤓][runner-omnibus] | [✅][runner-charts] | [⚙][runner-charts] | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#shared-runners) | CE & EE | -| [Database Migrations](#database-migrations) | Database migrations | [✅][database-migrations-omnibus] | [✅]() | [✅][database-migrations-charts] | [✅][database-migrations-charts] | CE & EE | -| [Certificate Management](#certificate-management) | TLS Settings, Let's Encrypt | [✅][certificate-management-omnibus] | [✅][certificate-management-charts] | [⚙][certificate-management-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#secrets-management) | CE & EE | -| [GitLab Geo Node](#gitlab-geo) | Geographically distributed GitLab nodes | [⚙][geo-omnibus] | [❌][geo-charts] | [❌][geo-charts] | ✅ | EE Only | -| [LDAP Authentication](#ldap-authentication) | Authenticate users against centralized LDAP directory | [⤓][ldap-omnibus] | [⤓][ldap-charts] | [⤓][ldap-charts] | [❌](https://about.gitlab.com/pricing/#gitlab-com) | CE & EE | -| [Outbound email (SMTP)](#outbound-email) | Send email messages to users | [⤓][outbound-email-omnibus] | [⤓][outbound-email-charts] | [⤓][outbound-email-charts] | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#mail-configuration) | CE & EE | -| [Inbound email (SMTP)](#inbound-email) | Receive messages to update issues | [⤓][inbound-email-omnibus] | [⤓][inbound-email-charts] | [⤓][inbound-email-charts] | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#mail-configuration) | CE & EE | -| [ElasticSearch](#elasticsearch) | Improved search within GitLab | [⤓][elasticsearch-omnibus] | [⤓][elasticsearch-charts] | [⤓][elasticsearch-charts] | [❌](https://gitlab.com/groups/gitlab-org/-/epics/153) | EE Only | -| [Sentry integration](#sentry) | Error tracking for deployed apps | [⤓][sentry-integration] | [⤓][sentry-integration] | [⤓][sentry-integration] | [⤓][sentry-integration] | CE & EE | -| [Jaeger integration](#jaeger) | Distributed tracing for deployed apps | [⤓][jaeger-integration] | [⤓][jaeger-integration] | [⤓][jaeger-integration] | [⤓][jaeger-integration] | EE Only | -| [Kubernetes cluster apps](#kubernetes-cluster-apps) | Deploy [Helm](https://docs.helm.sh/), [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/), [Cert-Manager](https://docs.cert-manager.io/en/latest/), [Prometheus](https://prometheus.io/docs/introduction/overview/), a [Runner](https://docs.gitlab.com/runner/), [JupyterHub](http://jupyter.org/), [Knative](https://cloud.google.com/knative) to a cluster | [⤓][managed-k8s-apps] | [⤓][managed-k8s-apps] | [⤓][managed-k8s-apps] | [⤓][managed-k8s-apps] | CE & EE | +| Component | Description | [Omnibus GitLab](https://docs.gitlab.com/omnibus/) | [GitLab chart](https://docs.gitlab.com/charts/) | [Minikube Minimal](https://docs.gitlab.com/charts/development/minikube/#deploying-gitlab-with-minimal-settings) | [GitLab.com](https://gitlab.com) | [Source](../install/installation.md) | [GDK](https://gitlab.com/gitlab-org/gitlab-development-kit) | CE/EE | +| --------- | ----------- |:--------------------:|:------------------:|:-----:|:--------:|:--------:|:-------:|:-------:| +| [NGINX](#nginx) | Routes requests to appropriate components, terminates SSL | [✅][nginx-omnibus] | [✅][nginx-charts] | [⚙][nginx-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | [⤓][nginx-source] | ❌ | CE & EE | +| [Unicorn (GitLab Rails)](#unicorn) | Handles requests for the web interface and API | [✅][unicorn-omnibus] | [✅][unicorn-charts] | [✅][unicorn-charts] | [✅](../user/gitlab_com/index.md#unicorn) | [⚙][unicorn-source] | [✅][gitlab-yml] | CE & EE | +| [Sidekiq](#sidekiq) | Background jobs processor | [✅][sidekiq-omnibus] | [✅][sidekiq-charts] | [✅](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/index.html) | [✅](../user/gitlab_com/index.md#sidekiq) | [✅][gitlab-yml] | [✅][gitlab-yml] | CE & EE | +| [Gitaly](#gitaly) | Git RPC service for handling all git calls made by GitLab | [✅][gitaly-omnibus] | [✅][gitaly-charts] | [✅][gitaly-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | [⚙][gitaly-source] | ✅ | CE & EE | +| [GitLab Workhorse](#gitlab-workhorse) | Smart reverse proxy, handles large HTTP requests | [✅][workhorse-omnibus] | [✅][workhorse-charts] | [✅][workhorse-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | [⚙][workhorse-source] | ✅ | CE & EE | +| [GitLab Shell](#gitlab-shell) | Handles `git` over SSH sessions | [✅][shell-omnibus] | [✅][shell-charts] | [✅][shell-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | [⚙][shell-source] | [✅][gitlab-yml] | CE & EE | +| [GitLab Pages](#gitlab-pages) | Hosts static websites | [⚙][pages-omnibus] | [❌][pages-charts] | [❌][pages-charts] | [✅](../user/gitlab_com/index.md#gitlab-pages) | [⚙][pages-source] | [⚙][pages-gdk] | CE & EE | +| [Registry](#registry) | Container registry, allows pushing and pulling of images | [⚙][registry-omnibus] | [✅][registry-charts] | [✅][registry-charts] | [✅](../user/project/container_registry.md#build-and-push-images) | [⤓][registry-source] | [⚙][registry-gdk] | CE & EE | +| [Redis](#redis) | Caching service | [✅][redis-omnibus] | [✅][redis-omnibus] | [✅][redis-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | [⤓][redis-source] | ✅ | CE & EE | +| [PostgreSQL](#postgresql) | Database | [✅][postgres-omnibus] | [✅][postgres-charts] | [✅][postgres-charts] | [✅](../user/gitlab_com/index.md#postgresql) | [⤓][postgres-source] | ✅ | CE & EE | +| [PgBouncer](#pgbouncer) | Database connection pooling, failover | [⚙][pgbouncer-omnibus] | [❌][pgbouncer-charts] | [❌][pgbouncer-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#database-architecture) | ❌ | ❌ | EE Only | +| [Consul](#consul) | Database node discovery, failover | [⚙][consul-omnibus] | [❌][consul-charts] | [❌][consul-charts] | [✅](../user/gitlab_com/index.md#consul) | ❌ | ❌ | EE Only | +| [GitLab self-monitoring: Prometheus](#prometheus) | Time-series database, metrics collection, and query service | [✅][prometheus-omnibus] | [✅][prometheus-charts] | [⚙][prometheus-charts] | [✅](../user/gitlab_com/index.md#prometheus) | ❌ | ❌ | CE & EE | +| [GitLab self-monitoring: Alertmanager](#alertmanager) | Deduplicates, groups, and routes alerts from Prometheus | [✅][alertmanager-omnibus] | [✅][alertmanager-charts] | [⚙][alertmanager-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | +| [GitLab self-monitoring: Grafana](#grafana) | Metrics dashboard | [⚙][grafana-omnibus] | [⤓][grafana-charts] | [⤓][grafana-charts] | [✅](https://dashboards.gitlab.com/d/RZmbBr7mk/gitlab-triage?refresh=30s) | ❌ | ❌ | CE & EE | +| [GitLab self-monitoring: Sentry](#sentry) | Track errors generated by the GitLab instance | [⤓][sentry-omnibus] | [❌][sentry-charts] | [❌][sentry-charts] | [✅](https://about.gitlab.com/handbook/support/workflows/services/gitlab_com/500_errors.html#searching-sentry) | [⤓][gitlab-yml] | [⤓][gitlab-yml] | CE & EE | +| [GitLab self-monitoring: Jaeger](#jaeger) | View traces generated by the GitLab instance | [❌][jaeger-omnibus] | [❌][jaeger-charts] | [❌][jaeger-charts] | [❌](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/4104) | [⤓][jaeger-source] | [⚙][jaeger-gdk] | CE & EE | +| [Redis Exporter](#redis-exporter) | Prometheus endpoint with Redis metrics | [✅][redis-exporter-omnibus] | [✅][redis-exporter-charts] | [✅][redis-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | +| [Postgres Exporter](#postgres-exporter) | Prometheus endpoint with PostgreSQL metrics | [✅][postgres-exporter-omnibus] | [✅][postgres-exporter-charts] | [✅][postgres-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | +| [PgBouncer Exporter](#pgbouncer-exporter) | Prometheus endpoint with PgBouncer metrics | [⚙][pgbouncer-exporter-omnibus] | [❌][pgbouncer-exporter-charts] | [❌][pgbouncer-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | +| [GitLab Monitor](#gitlab-monitor) | Generates a variety of GitLab metrics | [✅][gitlab-monitor-omnibus] | [❌][gitab-monitor-charts] | [❌][gitab-monitor-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | +| [Node Exporter](#node-exporter) | Prometheus endpoint with system metrics | [✅][node-exporter-omnibus] | [❌][node-exporter-charts] | [❌][node-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | +| [Mattermost](#mattermost) | Open-source Slack alternative | [⚙][mattermost-omnibus] | [⤓][mattermost-charts] | [⤓][mattermost-charts] | [⤓](../user/project/integrations/mattermost_slash_commands.md#manual-configuration), [⤓](../user/project/integrations/mattermost.html) | ❌ | ❌ | CE & EE | +| [MinIO](#minio) | Object storage service | [⤓][minio-omnibus] | [✅][minio-charts] | [✅][minio-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#storage-architecture) | ❌ | [⚙][minio-gdk] | CE & EE | +| [Runner](#gitlab-runner) | Executes GitLab CI jobs | [⤓][runner-omnibus] | [✅][runner-charts] | [⚙][runner-charts] | [✅](../user/gitlab_com/index.md#shared-runners) | [⚙][runner-source] | [⚙][runner-gdk] | CE & EE | +| [Database Migrations](#database-migrations) | Database migrations | [✅][database-migrations-omnibus] | [✅][database-migrations-charts] | [✅][database-migrations-charts] | ✅ | [⚙][database-migrations-source] | ✅ | CE & EE | +| [Certificate Management](#certificate-management) | TLS Settings, Let's Encrypt | [✅][certificate-management-omnibus] | [✅][certificate-management-charts] | [⚙][certificate-management-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#secrets-management) | [⚙][certificate-management-source] | [⚙][certificate-management-gdk] | CE & EE | +| [GitLab Geo Node](#gitlab-geo) | Geographically distributed GitLab nodes | [⚙][geo-omnibus] | [❌][geo-charts] | [❌][geo-charts] | ✅ | ❌ | [⚙][geo-gdk] | EE Only | +| [LDAP Authentication](#ldap-authentication) | Authenticate users against centralized LDAP directory | [⤓][ldap-omnibus] | [⤓][ldap-charts] | [⤓][ldap-charts] | [❌](https://about.gitlab.com/pricing/#gitlab-com) | [⤓][gitlab-yml] | [⤓][ldap-gdk] | CE & EE | +| [Outbound email (SMTP)](#outbound-email) | Send email messages to users | [⤓][outbound-email-omnibus] | [⤓][outbound-email-charts] | [⤓][outbound-email-charts] | [✅](../user/gitlab_com/index.md#mail-configuration) | [⤓][gitlab-yml] | [⤓][gitlab-yml] | CE & EE | +| [Inbound email (SMTP)](#inbound-email) | Receive messages to update issues | [⤓][inbound-email-omnibus] | [⤓][inbound-email-charts] | [⤓][inbound-email-charts] | [✅](../user/gitlab_com/index.md#mail-configuration) | [⤓][gitlab-yml] | [⤓][gitlab-yml] | CE & EE | +| [ElasticSearch](#elasticsearch) | Improved search within GitLab | [⤓][elasticsearch-omnibus] | [⤓][elasticsearch-charts] | [⤓][elasticsearch-charts] | [❌](https://gitlab.com/groups/gitlab-org/-/epics/153) | [⤓][elasticsearch-source] | [⤓][elasticsearch-gdk] | EE Only | +| [Sentry integration](#sentry) | Error tracking for deployed apps | [⤓][sentry-integration] | [⤓][sentry-integration] | [⤓][sentry-integration] | [⤓][sentry-integration] | [⤓][sentry-integration] | [⤓][sentry-integration] | CE & EE | +| [Jaeger integration](#jaeger) | Distributed tracing for deployed apps | [⤓][jaeger-integration] | [⤓][jaeger-integration] | [⤓][jaeger-integration] | [⤓][jaeger-integration] | [⤓][jaeger-integration] | [⤓][jaeger-integration] | EE Only | +| [GitLab Managed Apps](#gitlab-managed-apps) | Deploy [Helm](https://docs.helm.sh/), [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/), [Cert-Manager](https://docs.cert-manager.io/en/latest/), [Prometheus](https://prometheus.io/docs/introduction/overview/), a [Runner](https://docs.gitlab.com/runner/), [JupyterHub](http://jupyter.org/), [Knative](https://cloud.google.com/knative) to a cluster | [⤓][managed-k8s-apps] | [⤓][managed-k8s-apps] | [⤓][managed-k8s-apps] | [⤓][managed-k8s-apps] | [⤓][managed-k8s-apps] | [⤓][managed-k8s-apps] | CE & EE | ### Component details @@ -164,13 +164,14 @@ GitLab can be considered to have two layers from a process perspective: - [Project page](https://github.com/prometheus/alertmanager/blob/master/README.md) - Configuration: [Omnibus][alertmanager-omnibus], [Charts][alertmanager-charts] - Layer: Monitoring +- Process: `alertmanager` [Alert manager](https://prometheus.io/docs/alerting/alertmanager/) is a tool provided by Prometheus that _"handles alerts sent by client applications such as the Prometheus server. It takes care of deduplicating, grouping, and routing them to the correct receiver integration such as email, PagerDuty, or OpsGenie. It also takes care of silencing and inhibition of alerts."_ You can read more in [issue gitlab-ce#45740](https://gitlab.com/gitlab-org/gitlab-ce/issues/45740) about what we will be alerting on. #### Certificate management - Project page: [Omnibus](https://github.com/certbot/certbot/blob/master/README.rst), [Charts](https://github.com/jetstack/cert-manager/blob/master/README.md) -- Configuration: [Omnibus][certificate-management-omnibus], [Charts][certificate-management-charts] +- Configuration: [Omnibus][certificate-management-omnibus], [Charts][certificate-management-charts], [Source][certificate-management-source], [GDK][certificate-management-gdk] - Layer: Core Service (Processor) #### Consul @@ -183,13 +184,13 @@ Consul is a tool for service discovery and configuration. Consul is distributed, #### Database migrations -- Configuration: [Omnibus][registry-omnibus], [Charts][registry-charts] +- Configuration: [Omnibus][registry-omnibus], [Charts][registry-charts], [Source][database-migrations-source] - Layer: Core Service (Data) #### Elasticsearch - [Project page](https://github.com/elastic/elasticsearch/blob/master/README.textile) -- Configuration: [Omnibus][elasticsearch-omnibus], [Charts][elasticsearch-charts] +- Configuration: [Omnibus][elasticsearch-omnibus], [Charts][elasticsearch-charts], [Source][elasticsearch-source], [GDK][elasticsearch-gdk] - Layer: Core Service (Data) Elasticsearch is a distributed RESTful search engine built for the cloud. @@ -197,14 +198,15 @@ Elasticsearch is a distributed RESTful search engine built for the cloud. #### Gitaly - [Project page](https://gitlab.com/gitlab-org/gitaly/blob/master/README.md) -- Configuration: [Omnibus][gitaly-omnibus], [Charts][gitaly-charts] +- Configuration: [Omnibus][gitaly-omnibus], [Charts][gitaly-charts], [Source][gitaly-source] - Layer: Core Service (Data) +- Process: `gitaly` Gitaly is a service designed by GitLab to remove our need for NFS for Git storage in distributed deployments of GitLab (think GitLab.com or High Availability Deployments). As of 11.3.0, this service handles all Git level access in GitLab. You can read more about the project [in the project's readme](https://gitlab.com/gitlab-org/gitaly). #### Gitlab Geo -- Configuration: [Omnibus][geo-omnibus], [Charts][geo-charts] +- Configuration: [Omnibus][geo-omnibus], [Charts][geo-charts], [GDK][geo-gdk] - Layer: Core Service (Processor) #### Gitlab Monitor @@ -212,12 +214,13 @@ Gitaly is a service designed by GitLab to remove our need for NFS for Git storag - [Project page](https://gitlab.com/gitlab-org/gitlab-monitor) - Configuration: [Omnibus][gitlab-monitor-omnibus], [Charts][gitlab-monitor-charts] - Layer: Monitoring +- Process: `gitlab-monitor` GitLab Monitor is a process designed in house that allows us to export metrics about GitLab application internals to Prometheus. You can read more [in the project's readme](https://gitlab.com/gitlab-org/gitlab-monitor). #### Gitlab Pages -- Configuration: [Omnibus][pages-omnibus], [Charts][pages-charts] +- Configuration: [Omnibus][pages-omnibus], [Charts][pages-charts], [Source][pages-source], [GDK][pages-gdk] - Layer: Core Service (Processor) GitLab Pages is a feature that allows you to publish static websites directly from a repository in GitLab. @@ -227,7 +230,7 @@ You can use it either for personal or business websites, such as portfolios, doc #### Gitlab Runner - [Project page](https://gitlab.com/gitlab-org/gitlab-runner/blob/master/README.md) -- Configuration: [Omnibus][runner-omnibus], [Charts][runner-charts] +- Configuration: [Omnibus][runner-omnibus], [Charts][runner-charts], [Source][runner-source], [GDK][runner-gdk] - Layer: Core Service (Processor) GitLab Runner runs tests and sends the results to GitLab. @@ -237,7 +240,7 @@ GitLab CI is the open-source continuous integration service included with GitLab #### Gitlab Shell - [Project page](https://gitlab.com/gitlab-org/gitlab-shell/blob/master/README.md) -- Configuration: [Omnibus][shell-omnibus], [Charts][shell-charts] +- Configuration: [Omnibus][shell-omnibus], [Charts][shell-charts], [Source][shell-source], [GDK][gitlab-yml] - Layer: Core Service (Processor) [GitLab Shell](https://gitlab.com/gitlab-org/gitlab-shell) is a program designed at GitLab to handle ssh-based `git` sessions, and modifies the list of authorized keys. GitLab Shell is not a Unix shell nor a replacement for Bash or Zsh. @@ -245,8 +248,9 @@ GitLab CI is the open-source continuous integration service included with GitLab #### Gitlab Workhorse - [Project page](https://gitlab.com/gitlab-org/gitlab-workhorse/blob/master/README.md) -- Configuration: [Omnibus][gitlab-workhorse-omnibus], [Charts][gitlab-workhorse-charts] +- Configuration: [Omnibus][gitlab-workhorse-omnibus], [Charts][gitlab-workhorse-charts], [Source][workhorse-source] - Layer: Core Service (Processor) +- Process: `gitlab-workhorse` [GitLab Workhorse](https://gitlab.com/gitlab-org/gitlab-workhorse) is a program designed at GitLab to help alleviate pressure from Unicorn. You can read more about the [historical reasons for developing](https://about.gitlab.com/2016/04/12/a-brief-history-of-gitlab-workhorse/). It's designed to act as a smart reverse proxy to help speed up GitLab as a whole. @@ -261,7 +265,7 @@ Grafana is an open source, feature rich metrics dashboard and graph editor for G #### Jaeger - [Project page](https://github.com/jaegertracing/jaeger/blob/master/README.md) -- Configuration: [Omnibus][jaeger-omnibus], [Charts][jaeger-charts] +- Configuration: [Omnibus][jaeger-omnibus], [Charts][jaeger-charts], [Source][jaeger-source], [GDK][jaeger-gdk] - Layer: Monitoring Jaeger, inspired by Dapper and OpenZipkin, is a distributed tracing system. It can be used for monitoring microservices-based distributed systems. @@ -271,6 +275,7 @@ Jaeger, inspired by Dapper and OpenZipkin, is a distributed tracing system. It c - [Project page](https://github.com/logrotate/logrotate/blob/master/README.md) - Configuration: [Omnibus](https://docs.gitlab.com/omnibus/settings/logs.html#logrotate) - Layer: Core Service +- Process: `logrotate` GitLab is comprised of a large number of services that all log. We started bundling our own logrotate as of 7.4 to make sure we were logging responsibly. This is just a packaged version of the common open source offering. @@ -285,7 +290,7 @@ Mattermost is an open source, private cloud, Slack-alternative from https://matt #### MinIO - [Project page](https://github.com/minio/minio/blob/master/README.md) -- Configuration: [Omnibus][minio-omnibus], [Charts][minio-charts] +- Configuration: [Omnibus][minio-omnibus], [Charts][minio-charts], [GDK][minio-gdk] - Layer: Core Service (Data) MinIO is an object storage server released under Apache License v2.0. It is compatible with Amazon S3 cloud storage service. It is best suited for storing unstructured data such as photos, videos, log files, backups and container / VM images. Size of an object can range from a few KBs to a maximum of 5TB. @@ -293,8 +298,9 @@ MinIO is an object storage server released under Apache License v2.0. It is comp #### NGINX - Project page: [Omnibus](https://github.com/nginx/nginx), [Charts](https://github.com/kubernetes/ingress-nginx/blob/master/README.md) -- Configuration: [Omnibus][nginx-omnibus], [Charts][nginx-charts] +- Configuration: [Omnibus][nginx-omnibus], [Charts][nginx-charts], [Source][nginx-source] - Layer: Core Service (Processor) +- Process: `nginx` Nginx as an ingress port for all HTTP requests and routes them to the approriate sub-systems within GitLab. We are bundling an unmodified version of the popular open source webserver. @@ -303,6 +309,7 @@ Nginx as an ingress port for all HTTP requests and routes them to the approriate - [Project page](https://github.com/prometheus/node_exporter/blob/master/README.md) - Configuration: [Omnibus][node-exporter-omnibus], [Charts][node-exporter-charts] - Layer: Monitoring +- Process: `node-exporter` [Node Exporter](https://github.com/prometheus/node_exporter) is a Prometheus tool that gives us metrics on the underlying machine (think CPU/Disk/Load). It's just a packaged version of the common open source offering from the Prometheus project. @@ -325,8 +332,9 @@ Prometheus exporter for PgBouncer. Exports metrics at 9127/metrics. #### Postgresql - [Project page](https://github.com/postgres/postgres/blob/master/README) -- Configuration: [Omnibus][postgres-omnibus], [Charts][postgres-charts] +- Configuration: [Omnibus][postgres-omnibus], [Charts][postgres-charts], [Source][postgres-source] - Layer: Core Service (Data) +- Process: `postgresql` GitLab packages the popular Database to provide storage for Application meta data and user information. @@ -335,6 +343,7 @@ GitLab packages the popular Database to provide storage for Application meta dat - [Project page](https://github.com/wrouesnel/postgres_exporter/blob/master/README.md) - Configuration: [Omnibus][postgres-exporter-omnibus], [Charts][postgres-exporter-charts] - Layer: Monitoring +- Process: `postgres-exporter` [Postgres-exporter](https://github.com/wrouesnel/postgres_exporter) is the community provided Prometheus exporter that will deliver data about Postgres to Prometheus for use in Grafana Dashboards. @@ -343,14 +352,16 @@ GitLab packages the popular Database to provide storage for Application meta dat - [Project page](https://github.com/prometheus/prometheus/blob/master/README.md) - Configuration: [Omnibus][prometheus-omnibus], [Charts][prometheus-charts] - Layer: Monitoring +- Process: `prometheus` Prometheus is a time-series tool that helps GitLab administrators expose metrics about the individual processes used to provide GitLab the service. #### Redis - [Project page](https://github.com/antirez/redis/blob/unstable/README.md) -- Configuration: [Omnibus][redis-omnibus], [Charts][redis-charts] +- Configuration: [Omnibus][redis-omnibus], [Charts][redis-charts], [Source][redis-source] - Layer: Core Service (Data) +- Process: `redis` Redis is packaged to provide a place to store: @@ -363,13 +374,14 @@ Redis is packaged to provide a place to store: - [Project page](https://github.com/oliver006/redis_exporter/blob/master/README.md) - Configuration: [Omnibus][redis-exporter-omnibus], [Charts][redis-exporter-charts] - Layer: Monitoring +- Process: `redis-exporter` [Redis Exporter](https://github.com/oliver006/redis_exporter) is designed to give specific metrics about the Redis process to Prometheus so that we can graph these metrics in Grafana. #### Registry - [Project page](https://github.com/docker/distribution/blob/master/README.md) -- Configuration: [Omnibus][registry-omnibus], [Charts][registry-charts] +- Configuration: [Omnibus][registry-omnibus], [Charts][registry-charts], [Source][registry-source], [GDK][registry-gdk] - Layer: Core Service (Processor) The registry is what users use to store their own Docker images. The bundled @@ -385,7 +397,7 @@ An external registry can also be configured to use GitLab as an auth endpoint. #### Sentry - [Project page](https://github.com/getsentry/sentry/blob/master/README.rst) -- Configuration: [Omnibus][sentry-omnibus], [Charts][sentry-charts] +- Configuration: [Omnibus][sentry-omnibus], [Charts][sentry-charts], [Source][gitlab-yml], [GDK][gitlab-yml] - Layer: Monitoring Sentry fundamentally is a service that helps you monitor and fix crashes in realtime. The server is in Python, but it contains a full API for sending events from any language, in any application. @@ -393,40 +405,42 @@ Sentry fundamentally is a service that helps you monitor and fix crashes in real #### Sidekiq - [Project page](https://github.com/mperham/sidekiq/blob/master/README.md) -- Configuration: [Omnibus][sidekiq-omnibus], [Charts][sidekiq-charts] +- Configuration: [Omnibus][sidekiq-omnibus], [Charts][sidekiq-charts], [Source][gitlab-yml], [GDK][gitlab-yml] - Layer: Core Service (Processor) +- Process: `sidekiq` Sidekiq is a Ruby background job processor that pulls jobs from the redis queue and processes them. Background jobs allow GitLab to provide a faster request/response cycle by moving work into the background. #### Unicorn - [Project page](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/README.md) -- Configuration: [Omnibus][unicorn-omnibus], [Charts][unicorn-charts] +- Configuration: [Omnibus][unicorn-omnibus], [Charts][unicorn-charts], [Source][unicorn-source], [GDK][gitlab-yml] - Layer: Core Service (Processor) +- Process: `unicorn` [Unicorn](https://bogomips.org/unicorn/) is a Ruby application server that is used to run the core Rails Application that provides the user facing features in GitLab. Often process output you will see this as `bundle` or `config.ru` depending on the GitLab version. #### LDAP Authentication -- Configuration: [Omnibus][ldap-omnibus], [Charts][ldap-charts] +- Configuration: [Omnibus][ldap-omnibus], [Charts][ldap-charts], [Source][gitlab-yml], [GDK][ldap-gdk] - Layer: Core Service (Processor) #### Outbound Email -- Configuration: [Omnibus][outbound-email-omnibus], [Charts][outbound-email-charts] +- Configuration: [Omnibus][outbound-email-omnibus], [Charts][outbound-email-charts], [Source][gitlab-yml], [GDK][gitlab-yml] - Layer: Core Service (Processor) #### Inbound Email -- Configuration: [Omnibus][inbound-email-omnibus], [Charts][inbound-email-charts] +- Configuration: [Omnibus][inbound-email-omnibus], [Charts][inbound-email-charts], [Source][gitlab-yml], [GDK][gitlab-yml] - Layer: Core Service (Processor) -#### Kubernetes Cluster Apps +#### GitLab Managed Apps -- Configuration: [Omnibus][managed-k8s-apps], [Charts][managed-k8s-apps] +- Configuration: [Omnibus][managed-k8s-apps], [Charts][managed-k8s-apps], [Source][managed-k8s-apps], [GDK][managed-k8s-apps] - Layer: Core Service (Processor) -GitLab provides [GitLab Managed Apps](https://docs.gitlab.com/ee/user/project/clusters/#installing-applications), a one-click install for various applications which can be added directly to your configured cluster. These applications are needed for Review Apps and deployments when using Auto DevOps. You can install them after you create a cluster. +GitLab provides [GitLab Managed Apps](../user/project/clusters/index.md#installing-applications), a one-click install for various applications which can be added directly to your configured cluster. These applications are needed for Review Apps and deployments when using Auto DevOps. You can install them after you create a cluster. ## GitLab by Request Type @@ -595,68 +609,92 @@ We've also detailed [our architecture of GitLab.com](https://about.gitlab.com/ha [alertmanager-omnibus]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template [alertmanager-charts]: https://github.com/helm/charts/tree/master/stable/prometheus -[nginx-omnibus]: https://docs.gitlab.com/omnibus/settings/nginx.html -[nginx-charts]: https://docs.gitlab.com/charts/charts/nginx/index.html +[nginx-omnibus]: https://docs.gitlab.com/omnibus/settings/ +[nginx-charts]: https://docs.gitlab.com/charts/charts/nginx/ +[nginx-source]: ../install/installation.md#9-nginx [unicorn-omnibus]: https://docs.gitlab.com/omnibus/settings/unicorn.html -[unicorn-charts]: https://docs.gitlab.com/charts/charts/gitlab/unicorn/index.html +[unicorn-charts]: https://docs.gitlab.com/charts/charts/gitlab/unicorn/ +[unicorn-source]: ../install/installation.md#configure-it +[gitlab-yml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/gitlab.yml.example [sidekiq-omnibus]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template -[sidekiq-charts]: https://docs.gitlab.com/charts/charts/gitlab/sidekiq/index.html -[gitaly-omnibus]: https://docs.gitlab.com/ee/administration/gitaly/ -[gitaly-charts]: https://docs.gitlab.com/charts/charts/gitlab/gitaly/index.html +[sidekiq-charts]: https://docs.gitlab.com/charts/charts/gitlab/sidekiq/ +[gitaly-omnibus]: ../administration/gitaly/index.md +[gitaly-charts]: https://docs.gitlab.com/charts/charts/gitlab/gitaly/ +[gitaly-source]: ../install/installation.md#install-gitaly [workhorse-omnibus]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template -[workhorse-charts]: https://docs.gitlab.com/charts/charts/gitlab/unicorn/index.html +[workhorse-charts]: https://docs.gitlab.com/charts/charts/gitlab/unicorn/ +[workhorse-source]: ../install/installation.md#install-gitlab-workhorse [shell-omnibus]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template -[shell-charts]: https://docs.gitlab.com/charts/charts/gitlab/gitlab-shell/index.html -[pages-omnibus]: https://docs.gitlab.com/ee/administration/pages/ +[shell-charts]: https://docs.gitlab.com/charts/charts/gitlab/gitlab-shell/ +[shell-source]: ../install/installation.md#install-gitlab-shell +[pages-omnibus]: ../administration/pages/index.md [pages-charts]: https://gitlab.com/charts/gitlab/issues/37 -[registry-omnibus]: https://docs.gitlab.com/ee/administration/container_registry.html#container-registry-domain-configuration -[registry-charts]: https://docs.gitlab.com/charts/charts/registry/index.html +[pages-source]: ../install/installation.md#install-gitlab-pages +[pages-gdk]: https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/pages.md +[registry-omnibus]: ../administration/container_registry.md#container-registry-domain-configuration +[registry-charts]: https://docs.gitlab.com/charts/charts/registry/ +[registry-source]: ../administration/container_registry.md#enable-the-container-registry +[registry-gdk]: https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/registry.md [redis-omnibus]: https://docs.gitlab.com/omnibus/settings/redis.html -[redis-charts]: https://docs.gitlab.com/charts/charts/redis/index.html +[redis-charts]: https://docs.gitlab.com/charts/charts/redis/ +[redis-source]: ../install/installation.md#7-redis [postgres-omnibus]: https://docs.gitlab.com/omnibus/settings/database.html [postgres-charts]: https://github.com/helm/charts/tree/master/stable/postgresql -[pgbouncer-omnibus]: https://docs.gitlab.com/ee/administration/high_availability/pgbouncer.html +[postgres-source]: ../install/installation.md#6-database +[pgbouncer-omnibus]: ../administration/high_availability/pgbouncer.md [pgbouncer-charts]: https://docs.gitlab.com/charts/installation/deployment.html#postgresql -[consul-omnibus]: https://docs.gitlab.com/ee/administration/high_availability/consul.html +[consul-omnibus]: ../administration/high_availability/consul.md [consul-charts]: https://docs.gitlab.com/charts/installation/deployment.html#postgresql -[prometheus-omnibus]: https://docs.gitlab.com/ee/administration/monitoring/prometheus/ +[prometheus-omnibus]: ../administration/monitoring/prometheus/index.md [prometheus-charts]: https://github.com/helm/charts/tree/master/stable/prometheus -[grafana-omnibus]: https://docs.gitlab.com/ee/administration/monitoring/performance/grafana_configuration.html +[grafana-omnibus]: ../administration/monitoring/performance/grafana_configuration.md [grafana-charts]: https://github.com/helm/charts/tree/master/stable/grafana [sentry-omnibus]: https://docs.gitlab.com/omnibus/settings/configuration.html#error-reporting-and-logging-with-sentry [sentry-charts]: https://gitlab.com/charts/gitlab/issues/1319 [jaeger-omnibus]: https://gitlab.com/gitlab-org/omnibus-gitlab/issues/4104 [jaeger-charts]: https://gitlab.com/charts/gitlab/issues/1320 -[redis-exporter-omnibus]: https://docs.gitlab.com/ee/administration/monitoring/prometheus/redis_exporter.html -[redis-exporter-charts]: https://docs.gitlab.com/charts/charts/redis/index.html -[postgres-exporter-omnibus]: https://docs.gitlab.com/ee/administration/monitoring/prometheus/postgres_exporter.html +[jaeger-source]: ../development/distributed_tracing.md#enabling-distributed-tracing +[jaeger-gdk]: ../development/distributed_tracing.html#using-jaeger-in-the-gitlab-development-kit +[redis-exporter-omnibus]: ../administration/monitoring/prometheus/redis_exporter.md +[redis-exporter-charts]: https://docs.gitlab.com/charts/charts/redis/ +[postgres-exporter-omnibus]: ../administration/monitoring/prometheus/postgres_exporter.md [postgres-exporter-charts]: https://github.com/helm/charts/tree/master/stable/postgresql -[pgbouncer-exporter-omnibus]: https://docs.gitlab.com/ee/administration/monitoring/prometheus/pgbouncer_exporter.html +[pgbouncer-exporter-omnibus]: ../administration/monitoring/prometheus/pgbouncer_exporter.md [pgbouncer-exporter-charts]: https://docs.gitlab.com/charts/installation/deployment.html#postgresql -[gitlab-monitor-omnibus]: https://docs.gitlab.com/ee/administration/monitoring/prometheus/gitlab_monitor_exporter.html +[gitlab-monitor-omnibus]: ../administration/monitoring/prometheus/gitlab_monitor_exporter.md [gitab-monitor-charts]: https://gitlab.com/charts/gitlab/issues/319 -[node-exporter-omnibus]: https://docs.gitlab.com/ee/administration/monitoring/prometheus/node_exporter.html +[node-exporter-omnibus]: ../administration/monitoring/prometheus/node_exporter.md [node-exporter-charts]: https://gitlab.com/charts/gitlab/issues/1332 [mattermost-omnibus]: https://docs.gitlab.com/omnibus/gitlab-mattermost/ [mattermost-charts]: https://docs.mattermost.com/install/install-mmte-helm-gitlab-helm.html [minio-omnibus]: https://min.io/download -[minio-charts]: https://docs.gitlab.com/charts/charts/minio/index.html +[minio-charts]: https://docs.gitlab.com/charts/charts/minio/ +[minio-gdk]: https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/object_storage.md [runner-omnibus]: https://docs.gitlab.com/runner/ [runner-charts]: https://docs.gitlab.com/runner/install/kubernetes.html +[runner-source]: https://docs.gitlab.com/runner/ +[runner-gdk]: https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/runner.md [database-migrations-omnibus]: https://docs.gitlab.com/omnibus/settings/database.html#disabling-automatic-database-migration -[database-migrations-charts]: https://docs.gitlab.com/charts/charts/gitlab/migrations/index.html +[database-migrations-charts]: https://docs.gitlab.com/charts/charts/gitlab/migrations/ +[database-migrations-source]: ../update/upgrading_from_source.md#13-install-libs-migrations-etc [certificate-management-omnibus]: https://docs.gitlab.com/omnibus/settings/ssl.html [certificate-management-charts]: https://docs.gitlab.com/charts/installation/tls.html -[geo-omnibus]: https://docs.gitlab.com/ee/administration/geo/replication/index.html#setup-instructions +[certificate-management-source]: ../install/installation.md#using-https +[certificate-management-gdk]: https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/https.md +[geo-omnibus]: ../administration/geo/replication/index.md#setup-instructions [geo-charts]: https://gitlab.com/charts/gitlab/issues/8 -[ldap-omnibus]: https://docs.gitlab.com/ee/administration/auth/ldap.html +[geo-gdk]: https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/geo.md +[ldap-omnibus]: ../administration/auth/ldap.md [ldap-charts]: https://docs.gitlab.com/charts/charts/globals.html#ldap +[ldap-gdk]: https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/ldap.md [outbound-email-omnibus]: https://docs.gitlab.com/omnibus/settings/smtp.html [outbound-email-charts]: https://docs.gitlab.com/charts/installation/command-line-options.html#outgoing-email-configuration -[inbound-email-omnibus]: https://docs.gitlab.com/ee/administration/incoming_email.html +[inbound-email-omnibus]: ../administration/incoming_email.md [inbound-email-charts]: https://docs.gitlab.com/charts/installation/command-line-options.html#incoming-email-configuration -[elasticsearch-omnibus]: https://docs.gitlab.com/ee/integration/elasticsearch.html -[elasticsearch-charts]: https://docs.gitlab.com/ee/integration/elasticsearch.html -[sentry-integration]: https://docs.gitlab.com/ee/user/project/operations/error_tracking.html -[jaeger-integration]: https://docs.gitlab.com/ee/user/project/operations/tracing.html -[managed-k8s-apps]: https://docs.gitlab.com/ee/user/project/clusters/#installing-applications +[elasticsearch-omnibus]: ../integration/elasticsearch.md +[elasticsearch-charts]: ../integration/elasticsearch.md +[elasticsearch-source]: ../integration/elasticsearch.md +[elasticsearch-gdk]: https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/elasticsearch.md +[sentry-integration]: ../user/project/operations/error_tracking.md +[jaeger-integration]: ../user/project/operations/tracing.md +[managed-k8s-apps]: ../user/project/clusters/index.md#installing-applications diff --git a/doc/development/code_review.md b/doc/development/code_review.md index c4e5995714d..29e2aa1a581 100644 --- a/doc/development/code_review.md +++ b/doc/development/code_review.md @@ -27,7 +27,7 @@ Depending on the areas your merge request touches, it must be **approved** by on or more [maintainers](https://about.gitlab.com/handbook/engineering/workflow/code-review/#maintainer): For approvals, we use the approval functionality found in the merge request -widget. Reviewers can add their approval by [approving additionally](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html#adding-or-removing-an-approval). +widget. Reviewers can add their approval by [approving additionally](../user/project/merge_requests/merge_request_approvals.md#adding-or-removing-an-approval). Getting your merge request **merged** also requires a maintainer. If it requires more than one approval, the last maintainer to review and approve it will also merge it. @@ -43,7 +43,7 @@ It picks reviewers and maintainers from the list at the [engineering projects](https://about.gitlab.com/handbook/engineering/projects/) page, with these behaviours: -1. It will not pick people whose [GitLab status](../user/profile/#current-status) +1. It will not pick people whose [GitLab status](../user/profile/index.md#current-status) contains the string 'OOO'. 2. [Trainee maintainers](https://about.gitlab.com/handbook/engineering/workflow/code-review/#trainee-maintainer) are three times as likely to be picked as other reviewers. @@ -152,7 +152,7 @@ required approvers. Maintainers must check before merging if the merge request is introducing new vulnerabilities, by inspecting the list in the Merge Request [Security -Widget](https://docs.gitlab.com/ee/user/project/merge_requests/#security-reports-ultimate). +Widget](../user/project/merge_requests/index.md#security-reports-ultimate). When in doubt, a [Security Engineer][team] can be involved. The list of detected vulnerabilities must be either empty or containing: @@ -286,7 +286,7 @@ experience, refactors the existing code). Then: author has already set this option or if the merge request clearly contains a messy commit history that is intended to be squashed. -[squash-and-merge]: https://docs.gitlab.com/ee/user/project/merge_requests/squash_and_merge.html#squash-and-merge +[squash-and-merge]: ../user/project/merge_requests/squash_and_merge.md#squash-and-merge ### The right balance @@ -319,7 +319,7 @@ reviewee. GitLab is used in a lot of places. Many users use our [Omnibus packages](https://about.gitlab.com/installation/), but some use the [Docker images](https://docs.gitlab.com/omnibus/docker/), some are -[installed from source](https://docs.gitlab.com/ce/install/installation.html), +[installed from source](../install/installation.md), and there are other installation methods available. GitLab.com itself is a large Enterprise Edition instance. This has some implications: diff --git a/doc/development/contributing/index.md b/doc/development/contributing/index.md index 8b1d014e101..59cf5014da4 100644 --- a/doc/development/contributing/index.md +++ b/doc/development/contributing/index.md @@ -87,7 +87,7 @@ Sometimes style guides will be followed but the code will lack structural integr GitLab will do its best to review community contributions as quickly as possible. Specially appointed developers review community contributions daily. You may take a look at the [team page](https://about.gitlab.com/team/) for the merge request coach who specializes in the type of code you have written and mention them in the merge request. For example, if you have written some JavaScript in your code then you should mention the frontend merge request coach. If your code has multiple disciplines you may mention multiple merge request coaches. -GitLab receives a lot of community contributions, so if your code has not been reviewed within 4 days of its initial submission feel free to re-mention the appropriate merge request coach. +GitLab receives a lot of community contributions, so if your code has not been reviewed within two days (excluding weekend and public holidays) of its initial submission feel free to re-mention the appropriate merge request coach. When submitting code to GitLab, you may feel that your contribution requires the aid of an external library. If your code includes an external library please provide a link to the library, as well as reasons for including it. diff --git a/doc/development/contributing/issue_workflow.md b/doc/development/contributing/issue_workflow.md index 0e1ab8663ed..e3a1dc711fd 100644 --- a/doc/development/contributing/issue_workflow.md +++ b/doc/development/contributing/issue_workflow.md @@ -7,7 +7,7 @@ scheduling into milestones. Labelling is a task for everyone. Most issues will have labels for at least one of the following: - Type: ~feature, ~bug, ~customer, etc. -- Subject: ~wiki, ~"container registry", ~ldap, ~api, ~frontend, etc. +- Subject: ~wiki, ~"Container Registry", ~ldap, ~api, ~frontend, etc. - Team: ~Plan, ~Manage, ~Quality, etc. - Stage: ~"devops:plan", ~"devops:create", etc. - Release Scoping: ~Deliverable, ~Stretch, ~"Next Patch Release" @@ -44,7 +44,7 @@ Subject labels are labels that define what area or feature of GitLab this issue hits. They are not always necessary, but very convenient. Examples of subject labels are ~wiki, ~ldap, ~api, -~issues, ~"merge requests", ~labels, and ~"container registry". +~issues, ~"merge requests", ~labels, and ~"Container Registry". If you are an expert in a particular area, it makes it easier to find issues to work on. You can also subscribe to those labels to receive an email each time an @@ -92,20 +92,21 @@ Stage labels specify which [DevOps stage][devops-stages] the issue belongs to. The current stage labels are: -- ~"devops:manage" -- ~"devops:plan" -- ~"devops:create" -- ~"devops:verify" -- ~"devops:package" -- ~"devops:release" -- ~"devops:configure" -- ~"devops:monitor" -- ~"devops:secure" -- ~"devops:defend" -- ~"devops:enablement" - -These labels should be mutually exclusive. If an issue belongs to multiple -stages, the most relevant should be used. +- ~"devops::manage" +- ~"devops::plan" +- ~"devops::create" +- ~"devops::verify" +- ~"devops::package" +- ~"devops::release" +- ~"devops::configure" +- ~"devops::monitor" +- ~"devops::secure" +- ~"devops::defend" +- ~"devops::growth" +- ~"devops::enablement" + +These labels are [scoped labels](../../user/project/labels.md#scoped-labels-premium) +and thus are mutually exclusive. They differ from the [Team labels](#team-labels) because teams may work on issues outside their stage. @@ -121,6 +122,25 @@ The Stage labels are used to generate the [direction pages][direction-pages] aut [devops-stages]: https://about.gitlab.com/direction/#devops-stages [direction-pages]: https://about.gitlab.com/direction/ +## Group labels + +Group labels specify which [groups][structure-groups] the issue belongs to. + +Examples include: + +- ~"group::control" +- ~"group::editor" + +These labels are [scoped labels](../../user/project/labels.md#scoped-labels-premium) +and thus are mutually exclusive. + +Groups are nested beneath a particular stage, so only one stage label and one group label +can be applied to a single issue. You can find the groups listed in the +[Product Categories pages][product-categories]. + +[structure-groups]: https://about.gitlab.com/company/team/structure/#groups +[product-categories]: https://about.gitlab.com/handbook/product/categories/ + ## Release Scoping labels Release Scoping labels help us clearly communicate expectations of the work for the diff --git a/doc/development/distributed_tracing.md b/doc/development/distributed_tracing.md index 038e3de10d7..bfce7488a8d 100644 --- a/doc/development/distributed_tracing.md +++ b/doc/development/distributed_tracing.md @@ -34,7 +34,7 @@ GITLAB_TRACING=opentracing://<driver>?<param_name>=<param_value>&<param_name_2>= In this example, we have the following hypothetical values: - `driver`: the driver. [GitLab supports - `jaeger`](https://docs.gitlab.com/ee/user/project/operations/tracing.html). In future, other + `jaeger`](../user/project/operations/tracing.md). In future, other tracing implementations may also be supported. - `param_name`, `param_value`: these are driver specific configuration values. Configuration parameters for Jaeger are documented [further on in this diff --git a/doc/development/documentation/feature-change-workflow.md b/doc/development/documentation/feature-change-workflow.md index 1f68b6a6a70..ca29353ecbe 100644 --- a/doc/development/documentation/feature-change-workflow.md +++ b/doc/development/documentation/feature-change-workflow.md @@ -86,7 +86,7 @@ Everyone is encouraged to draft the requirements in the issue, but a product man do the following: - When the issue is assigned a release milestone, review and update the Documentation details. -- By the kickoff, finalizie the Documentation details. +- By the kickoff, finalize the Documentation details. ### Developer and maintainer roles @@ -117,7 +117,7 @@ Follow the process below unless otherwise agreed with the product manager and te #### Reviews and merging -All reviewers can help ensure accuracy, clarity, completeness, and adherence to the plans in the issue, as well as the [Documentation Guidelines](https://docs.gitlab.com/ee/development/documentation/) and [Style Guide](https://docs.gitlab.com/ee/development/documentation/styleguide.html). +All reviewers can help ensure accuracy, clarity, completeness, and adherence to the plans in the issue, as well as the [Documentation Guidelines](index.md) and [Style Guide](styleguide.md). - **Prior to merging**, documentation changes committed by the developer must be reviewed by: @@ -136,7 +136,7 @@ All reviewers can help ensure accuracy, clarity, completeness, and adherence to 1. **The maintainer** who is assigned to merge the MR, to verify clarity, completeness, and quality, to the best of their ability. - Upon merging, if a technical writer review has not been performed and there is not yet a linked issue for a follow-up review, the maintainer should [create an issue using the Doc Review template](https://gitlab.com/gitlab-org/gitlab-ce/issues/new?issuable_template=Doc%20Review), link it from the MR, and - mention the original MR author in the new issue. Alternatively, the mainitainer can ask the MR author to create and link this issue before the MR is merged. + mention the original MR author in the new issue. Alternatively, the maintainer can ask the MR author to create and link this issue before the MR is merged. - After merging, documentation changes are reviewed by: @@ -157,14 +157,14 @@ All reviewers can help ensure accuracy, clarity, completeness, and adherence to #### Collaboration By default, the developer will work on documentation changes independently, but -the developer, PM, or technicial writer can propose a broader collaboration for +the developer, PM, or technical writer can propose a broader collaboration for any given issue. Additionally, technical writers are available for questions at any time. #### Review -- Techncial writers provide non-blocking reviews of all documentation changes, +- Technical writers provide non-blocking reviews of all documentation changes, before or after the change is merged. However, if the docs are ready in the MR while there's time before the freeze, the technical writer's review can commence early, on request. - The technical writer will confirm that the doc is clear, grammatically correct, @@ -173,7 +173,7 @@ Additionally, technical writers are available for questions at any time. the developer and code reviewer should have already made a good-faith effort to ensure: - Clarity. - Adherence to the plans and goals in the issue. - - Location (make sure the docs are in the correct directorkes and has the correct name). + - Location (make sure the docs are in the correct directories and has the correct name). - Syntax, typos, and broken links. - Improvements to the content. - Accordance with the [Documentation Style Guide](styleguide.md), and [Structure and Template](structure.md) doc. diff --git a/doc/development/documentation/index.md b/doc/development/documentation/index.md index 4bf8401c0e8..6dfd3b2a690 100644 --- a/doc/development/documentation/index.md +++ b/doc/development/documentation/index.md @@ -466,7 +466,7 @@ If you want to know the in-depth details, here's what's really happening: The following GitLab features are used among others: - [Manual actions](../../ci/yaml/README.md#whenmanual) -- [Multi project pipelines](https://docs.gitlab.com/ee/ci/multi_project_pipeline_graphs.html) +- [Multi project pipelines](../../ci/multi_project_pipeline_graphs.md) - [Review Apps](../../ci/review_apps/index.md) - [Artifacts](../../ci/yaml/README.md#artifacts) - [Specific Runner](../../ci/runners/README.md#locking-a-specific-runner-from-being-enabled-for-other-projects) diff --git a/doc/development/documentation/structure.md b/doc/development/documentation/structure.md index 95b5fcd99a1..fe676efa94d 100644 --- a/doc/development/documentation/structure.md +++ b/doc/development/documentation/structure.md @@ -77,8 +77,8 @@ for use (e.g. variations on the main use case), but if that's not applicable, th Examples of use cases on feature pages: - CE and EE: [Issues](../../user/project/issues/index.md#use-cases) - CE and EE: [Merge Requests](../../user/project/merge_requests/index.md) -- EE-only: [Geo](https://docs.gitlab.com/ee/administration/geo/replication/index.html) -- EE-only: [Jenkins integration](https://docs.gitlab.com/ee/integration/jenkins.html) +- EE-only: [Geo](../../administration/geo/replication/index.md) +- EE-only: [Jenkins integration](../../integration/jenkins.md) ## Requirements @@ -121,8 +121,8 @@ but commented out to help encourage others to add to it in the future. --> Notes: -- (1): Apply the [tier badges](https://docs.gitlab.com/ee/development/documentation/styleguide.html#product-badges) accordingly -- (2): Apply the correct format for the [GitLab version introducing the feature](https://docs.gitlab.com/ee/development/documentation/styleguide.html#gitlab-versions-and-tiers) +- (1): Apply the [tier badges](styleguide.md#product-badges) accordingly +- (2): Apply the correct format for the [GitLab version introducing the feature](styleguide.md#gitlab-versions-and-tiers) ``` ## Help and feedback section diff --git a/doc/development/ee_features.md b/doc/development/ee_features.md index bc472bb5b0a..d0db1a61935 100644 --- a/doc/development/ee_features.md +++ b/doc/development/ee_features.md @@ -459,15 +459,6 @@ Resolving an EE template path that is relative to the CE view path will not work = render_if_exists 'projects/button' # Will render `ee/app/views/projects/_button` ``` -You should not explicitly set render options like `partial` or provide a `locals` hash. -The first argument should be a path string and the second can be a hash replacing `locals`. - -```ruby -render partial: 'projects/button', locals: { project: project } -# becomes -render_if_exists 'projects/button', project: project -``` - #### Using `render_ce` For `render` and `render_if_exists`, they search for the EE partial first, @@ -557,40 +548,56 @@ due to `prepend`, but Grape is complex internally and we couldn't easily do that, so we'll follow regular object-oriented practices that we define the interface first here. -For example, suppose we have a few more optional params for EE, given this CE -API code: +For example, suppose we have a few more optional params for EE. We can move the +params out of the `Grape::API` class to a helper module, so we can `prepend` it +before it would be used in the class. ```ruby module API - class MergeRequests < Grape::API - # EE::API::MergeRequests would override the following helpers - helpers do - params :optional_params_ee do + class Projects < Grape::API + helpers Helpers::ProjectsHelpers + end +end +``` + +Given this CE API `params`: + +```ruby +module API + module Helpers + module ProjectsHelpers + extend ActiveSupport::Concern + extend Grape::API::Helpers + + params :optional_project_params_ce do + # CE specific params go here... end - end - params :optional_params do - # CE specific params go here... + params :optional_project_params_ee do + end - use :optional_params_ee + params :optional_project_params do + use :optional_project_params_ce + use :optional_project_params_ee + end end end end -API::MergeRequests.prepend(EE::API::MergeRequests) +API::Helpers::ProjectsHelpers.prepend(EE::API::Helpers::ProjectsHelpers) ``` -And then we could override it in EE module: +We could override it in EE module: ```ruby module EE module API - module MergeRequests - extend ActiveSupport::Concern + module Helpers + module ProjectsHelpers + extend ActiveSupport::Concern - prepended do - helpers do - params :optional_params_ee do + prepended do + params :optional_project_params_ee do # EE specific params go here... end end @@ -600,9 +607,6 @@ module EE end ``` -This way, the only difference between CE and EE for that API file would be -`prepend EE::API::MergeRequests`. - #### EE helpers To make it easy for an EE module to override the CE helpers, we need to define @@ -933,7 +937,7 @@ export default { ``` #### For JS code that is EE only, like props, computed properties, methods, etc, we will keep the current approach - - Since we [can't async load a mixin](https://github.com/vuejs/vue-loader/issues/418#issuecomment-254032223) we will use the [`ee_else_ce`](https://docs.gitlab.com/ee/development/ee_features.html#javascript-code-in-assetsjavascripts) alias we already have for webpack. + - Since we [can't async load a mixin](https://github.com/vuejs/vue-loader/issues/418#issuecomment-254032223) we will use the [`ee_else_ce`](../development/ee_features.md#javascript-code-in-assetsjavascripts) alias we already have for webpack. - This means all the EE specific props, computed properties, methods, etc that are EE only should be in a mixin in the `ee/` folder and we need to create a CE counterpart of the mixin ##### Example: @@ -960,7 +964,7 @@ import mixin from 'ee_else_ce/path/mixin'; ### Non Vue Files For regular JS files, the approach is similar. -1. We will keep using the [`ee_else_ce`](https://docs.gitlab.com/ee/development/ee_features.html#javascript-code-in-assetsjavascripts) helper, this means that EE only code should be inside the `ee/` folder. +1. We will keep using the [`ee_else_ce`](../development/ee_features.md#javascript-code-in-assetsjavascripts) helper, this means that EE only code should be inside the `ee/` folder. 1. An EE file should be created with the EE only code, and it should extend the CE counterpart. 1. For code inside functions that can't be extended, the code should be moved into a new file and we should use `ee_else_ce` helper: diff --git a/doc/development/elasticsearch.md b/doc/development/elasticsearch.md index 0c9e7908713..8b0f4f02d19 100644 --- a/doc/development/elasticsearch.md +++ b/doc/development/elasticsearch.md @@ -2,7 +2,7 @@ This area is to maintain a compendium of useful information when working with elasticsearch. -Information on how to enable ElasticSearch and perform the initial indexing is kept in https://docs.gitlab.com/ee/integration/elasticsearch.html#enabling-elasticsearch +Information on how to enable ElasticSearch and perform the initial indexing is kept in ../integration/elasticsearch.md#enabling-elasticsearch ## Initial installation on OS X @@ -16,7 +16,7 @@ and use `docker stop elastic56` and `docker start elastic56` to stop/start it. ### Installing on the host -We currently only support Elasticsearch [5.6 to 6.x](https://docs.gitlab.com/ee/integration/elasticsearch.html#requirements) +We currently only support Elasticsearch [5.6 to 6.x](../integration/elasticsearch.md#version-requirements) Version 5.6 is available on homebrew and is the recommended version to use in order to test compatibility. @@ -43,7 +43,7 @@ this adds `gitlab-elasticsearch-indexer` to `$GOPATH/bin`, please make sure that - `gitlab:elastic:test:index_size`: Tells you how much space the current index is using, as well as how many documents are in the index. - `gitlab:elastic:test:index_size_change`: Outputs index size, reindexes, and outputs index size again. Useful when testing improvements to indexing size. -Additionally, if you need large repos or multiple forks for testing, please consider [following these instructions](https://docs.gitlab.com/ee/development/rake_tasks.html#extra-project-seed-options) +Additionally, if you need large repos or multiple forks for testing, please consider [following these instructions](rake_tasks.md#extra-project-seed-options) ## How does it work? diff --git a/doc/development/fe_guide/style_guide_js.md b/doc/development/fe_guide/style_guide_js.md index 060cd8baf7f..b50159c2b75 100644 --- a/doc/development/fe_guide/style_guide_js.md +++ b/doc/development/fe_guide/style_guide_js.md @@ -95,6 +95,7 @@ See [our current .eslintrc](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/ #### Modules, Imports, and Exports 1. Use ES module syntax to import modules + ```javascript // bad const SomeClass = require('some_class'); @@ -168,6 +169,7 @@ See [our current .eslintrc](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/ Do not use them anymore and feel free to remove them when refactoring legacy code. 1. Avoid adding to the global namespace. + ```javascript // bad window.MyClass = class { /* ... */ }; @@ -176,7 +178,8 @@ See [our current .eslintrc](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/ export default class MyClass { /* ... */ } ``` -1. Side effects are forbidden in any script which contains exports +1. Side effects are forbidden in any script which contains export + ```javascript // bad export default class MyClass { /* ... */ } @@ -449,6 +452,7 @@ Please check this [rules][eslint-plugin-vue-rules] for more documentation. #### Props 1. Props should be declared as an object + ```javascript // bad props: ['foo'] @@ -544,14 +548,24 @@ Please check this [rules][eslint-plugin-vue-rules] for more documentation. <component @click="eventHandler"/> ``` -1. Shorthand `:` is preferable over `v-bind` +2. Shorthand `:` is preferable over `v-bind` ```javascript // bad <component v-bind:class="btn"/> // good - <component :class="btsn"/> + <component :class="btn"/> + ``` + +3. Shorthand `#` is preferable over `v-slot` + + ```javascript + // bad + <template v-slot:header></template> + + // good + <template #header></template> ``` #### Closing tags diff --git a/doc/development/fe_guide/style_guide_scss.md b/doc/development/fe_guide/style_guide_scss.md index 36880dd746d..b25dce65ffe 100644 --- a/doc/development/fe_guide/style_guide_scss.md +++ b/doc/development/fe_guide/style_guide_scss.md @@ -9,20 +9,44 @@ easy to maintain, and performant for the end-user. As part of the effort for [cleaning up our CSS and moving our components into GitLab-UI](https://gitlab.com/groups/gitlab-org/-/epics/950) led by the [GitLab UI WG](https://gitlab.com/gitlab-com/www-gitlab-com/merge_requests/20623) we prefer the use of utility classes over adding new CSS. However, complex CSS can be addressed by adding component classes. -We have a few internal utility classes in [`common.scss`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/assets/stylesheets/framework/common.scss) -and we use [Bootstrap's Utility Classes](https://getbootstrap.com/docs/4.3/utilities/) +#### Where are utility classes defined? + +- [Bootstrap's Utility Classes](https://getbootstrap.com/docs/4.3/utilities/) +- [`common.scss`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/assets/stylesheets/framework/common.scss) (old) +- [`utilities.scss`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/assets/stylesheets/utilities.scss) (new) + +#### Where should I put new utility classes? New utility classes should be added to [`utilities.scss`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/assets/stylesheets/utilities.scss). Existing classes include: -**Background color**: `.bg-variant-shade` e.g. `.bg-warning-400` -**Text color**: `.text-variant-shade` e.g. `.text-success-500` +| Name | Pattern | Example | +|------|---------|---------| +| Background color | `.bg-{variant}-{shade}` | `.bg-warning-400` | +| Text color | `.text-{variant}-{shade}` | `.text-success-500` | +| Font size | `.text-{size}` | `.text-2` | + +- `{variant}` is one of 'primary', 'secondary', 'success', 'warning', 'error' +- `{shade}` is on of the shades listed on [colors](https://design.gitlab.com/foundations/colors/) +- `{size}` is a number from 1-6 from our [Type scale](https://design.gitlab.com/foundations/typography) + +#### When should I create component classes? + +We recommend a "utility-first" approach. + +1. Start with utility classes. +2. If composing utility classes into a component class removes code duplication and encapsulates a clear responsibility, do it. + +This encourages an organic growth of component classes and prevents the creation of one-off unreusable classes. Also, the kind of classes that emerge from "utility-first" tend to be design-centered (e.g. `.button`, `.alert`, `.card`) rather than domain-centered (e.g. `.security-report-widget`, `.commit-header-icon`). + +Examples of component classes that were created using "utility-first" include: -- variant is one of 'primary', 'secondary', 'success', 'warning', 'error' -- shade is on of the shades listed on [colors](https://design.gitlab.com/foundations/colors/) +- [`.circle-icon-container`](https://gitlab.com/gitlab-org/gitlab-ce/blob/579fa8b8ec7eb38d40c96521f517c9dab8c3b97a/app/assets/stylesheets/framework/icons.scss#L85) +- [`.d-flex-center`](https://gitlab.com/gitlab-org/gitlab-ce/blob/900083d89cd6af391d26ab7922b3f64fa2839bef/app/assets/stylesheets/framework/common.scss#L425) -**Font size**: `.text-size` e.g. `.text-2` +Inspiration: -- **size** is number from 1-6 from our [Type scale](https://design.gitlab.com/foundations/typography) +- https://tailwindcss.com/docs/utility-first +- https://tailwindcss.com/docs/extracting-components ### Naming diff --git a/doc/development/feature_flags.md b/doc/development/feature_flags.md index 82b09cc0224..c871015aaf6 100644 --- a/doc/development/feature_flags.md +++ b/doc/development/feature_flags.md @@ -28,7 +28,7 @@ the time, you should execute `/chatops run feature set my_feature_flag 50`. This document only covers feature flags used in the development of GitLab itself. Feature flags in deployed user applications can be found at -[Feature Flags](https://docs.gitlab.com/ee/user/project/operations/feature_flags.html) +[Feature Flags](../user/project/operations/feature_flags.md) ## Developing with feature flags @@ -111,7 +111,7 @@ Whenever a feature flag is present, make sure to test _both_ states of the feature flag. See the -[testing guide](testing_guide/best_practices.html#feature-flags-in-tests) +[testing guide](testing_guide/best_practices.md#feature-flags-in-tests) for information and examples on how to stub feature flags in tests. ## Enabling a feature flag (in development) diff --git a/doc/development/geo.md b/doc/development/geo.md index c8e6a86eb52..6e59fab34c7 100644 --- a/doc/development/geo.md +++ b/doc/development/geo.md @@ -10,6 +10,7 @@ the diagram below and are described in more detail within this document. ## Replication layer Geo handles replication for different components: + - [Database](#database-replication): includes the entire application, except cache and jobs. - [Git repositories](#repository-replication): includes both projects and wikis. - [Uploaded blobs](#uploads-replication): includes anything from images attached on issues @@ -90,7 +91,7 @@ projects that need updating. Those projects can be: timestamp that is more recent than the `last_repository_successful_sync_at` timestamp in the `Geo::ProjectRegistry` model. - Manual: The admin can manually flag a repository to resync in the - [Geo admin panel](https://docs.gitlab.com/ee/user/admin_area/geo_nodes.html). + [Geo admin panel](../user/admin_area/geo_nodes.md). When we fail to fetch a repository on the secondary `RETRIES_BEFORE_REDOWNLOAD` times, Geo does a so-called _redownload_. It will do a clean clone @@ -209,20 +210,38 @@ bundle exec rake geo:db:migrate ### Foreign Data Wrapper -The use of [FDW](#fdw) was introduced in GitLab 10.1. +> Introduced in GitLab 10.1. + +Foreign Data Wrapper ([FDW](#fdw)) is used by the [Geo Log Cursor](#geo-log-cursor) and improves +the performance of many synchronization operations. -This is useful for the [Geo Log Cursor](#geo-log-cursor) and improves -the performance of some synchronization operations. +FDW is a PostgreSQL extension ([`postgres_fdw`](https://www.postgresql.org/docs/current/postgres-fdw.html)) that is enabled within +the Geo Tracking Database (on a **secondary** node), which allows it +to connect to the readonly database replica and perform queries and filter +data from both instances. While FDW is available in older versions of PostgreSQL, we needed to raise the minimum required version to 9.6 as this includes many performance improvements to the FDW implementation. +This persistent connection is configured as an FDW server +named `gitlab_secondary`. This configuration exists within the database's user +context only. To access the `gitlab_secondary`, GitLab needs to use the +same database user that had previously been configured. + +The Geo Tracking Database accesses the readonly database replica via FDW as a regular user, +limited by its own restrictions. The credentials are configured as a +`USER MAPPING` associated with the `SERVER` mapped previously +(`gitlab_secondary`). + +FDW configuration and credentials definition are managed automatically by the +Omnibus GitLab `gitlab-ctl reconfigure` command. + #### Refeshing the Foreign Tables -Whenever the database schema changes on the **primary** node, the -**secondary** node will need to refresh its foreign tables by running -the following: +Whenever a new Geo node is configured or the database schema changes on the +**primary** node, you must refresh the foreign tables on the **secondary** node +by running the following: ```sh bundle exec rake geo:db:refresh_foreign_tables @@ -243,6 +262,53 @@ STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod) ORDER BY a.attnum ``` +#### Accessing data from a Foreign Table + +At the SQL level, all you have to do is `SELECT` data from `gitlab_secondary.*`. + +Here's an example of how to access all projects from the Geo Tracking Database's FDW: + +```sql +SELECT * FROM gitlab_secondary.projects; +``` + +As a more real-world example, this is how you filter for unarchived projects +on the Tracking Database: + +```sql +SELECT project_registry.* + FROM project_registry + JOIN gitlab_secondary.projects + ON (project_registry.project_id = gitlab_secondary.projects.id + AND gitlab_secondary.projects.archived IS FALSE) +``` + +At the ActiveRecord level, we have additional Models that represent the +foreign tables. They must be mapped in a slightly different way, and they are read-only. + +Check the existing FDW models in `ee/app/models/geo/fdw` for reference. + +From a developer's perspective, it's no different than creating a model that +represents a Database View. + +With the examples above, you can access the projects with: + +```ruby +Geo::Fdw::Project.all +``` + +and to access the `ProjectRegistry` filtering by unarchived projects: + +```ruby +# We have to use Arel here: +project_registry_table = Geo::ProjectRegistry.arel_table +fdw_project_table = Geo::Fdw::Project.arel_table + +project_registry_table.join(fdw_project_table) + .on(project_registry_table[:project_id].eq(fdw_project_table[:id])) + .where((fdw_project_table[:archived]).eq(true)) # if you append `.to_sql` you can check generated query +``` + ## Finders Geo uses [Finders](https://gitlab.com/gitlab-org/gitlab-ee/tree/master/app/finders), @@ -299,7 +365,7 @@ basically hashes all Git refs together and stores that hash in the The **secondary** node does the same to calculate the hash of its clone, and compares the hash with the value the **primary** node calculated. If there is a mismatch, Geo will mark this as a mismatch -and the administrator can see this in the [Geo admin panel](https://docs.gitlab.com/ee/user/admin_area/geo_nodes.html). +and the administrator can see this in the [Geo admin panel](../user/admin_area/geo_nodes.md). ## Glossary diff --git a/doc/development/go_guide/index.md b/doc/development/go_guide/index.md index cf78b792ffd..4dad8815fcb 100644 --- a/doc/development/go_guide/index.md +++ b/doc/development/go_guide/index.md @@ -40,7 +40,7 @@ of possible security breaches in our code: - SQL injections Remember to run -[SAST](https://docs.gitlab.com/ee/user/application_security/sast/index) +[SAST](../../user/application_security/sast/index.md) **[ULTIMATE]** on your project (or at least the [gosec analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/gosec)), and to follow our [Security @@ -94,9 +94,9 @@ become available, you will be able to share job templates like this Dependencies should be kept to the minimum. The introduction of a new dependency should be argued in the merge request, as per our [Approval Guidelines](../code_review.md#approval-guidelines). Both [License -Management](https://docs.gitlab.com/ee/user/project/merge_requests/license_management.html) +Management](../../user/project/merge_requests/license_management.md) **[ULTIMATE]** and [Dependency -Scanning](https://docs.gitlab.com/ee/user/application_security/dependency_scanning/index) +Scanning](../../user/application_security/dependency_scanning/index.md) **[ULTIMATE]** should be activated on all projects to ensure new dependencies security status and license compatibility. diff --git a/doc/development/i18n/externalization.md b/doc/development/i18n/externalization.md index 38d56322de8..38425674567 100644 --- a/doc/development/i18n/externalization.md +++ b/doc/development/i18n/externalization.md @@ -195,6 +195,7 @@ For example use `%{created_at}` in Ruby but `%{createdAt}` in JavaScript. Sometimes you need to add some context to the text that you want to translate (if the word occurs in a sentence and/or the word is ambiguous). +Namespaces should be PascalCase. - In Ruby/HAML: diff --git a/doc/development/i18n/merging_translations.md b/doc/development/i18n/merging_translations.md index 85284d8c714..eadf1cd74c5 100644 --- a/doc/development/i18n/merging_translations.md +++ b/doc/development/i18n/merging_translations.md @@ -32,7 +32,7 @@ clicking `Pause sync` on the [Crowdin integration settings page](https://translate.gitlab.com/project/gitlab-ee/settings#integration). When all failures are resolved, the translations need to be double -checked once more as discussed in [confidential issue](https://docs.gitlab.com/ee/user/project/issues/confidential_issues.html) `https://gitlab.com/gitlab-org/gitlab-ce/issues/37850`. +checked once more as discussed in [confidential issue](../../user/project/issues/confidential_issues.md) `https://gitlab.com/gitlab-org/gitlab-ce/issues/37850`. ## Merging translations diff --git a/doc/development/i18n/proofreader.md b/doc/development/i18n/proofreader.md index eb492c9818b..fb5cfb6c157 100644 --- a/doc/development/i18n/proofreader.md +++ b/doc/development/i18n/proofreader.md @@ -27,7 +27,7 @@ are very appreciative of the work done by translators and proofreaders! - Czech - Proofreaders needed. - Danish - - Proofreaders needed. + - Saederup92 - [GitLab](https://gitlab.com/Saederup92), [Crowdin](https://crowdin.com/profile/Saederup92) - Dutch - Emily Hendle - [GitLab](https://gitlab.com/pundachan), [Crowdin](https://crowdin.com/profile/pandachan) - Esperanto diff --git a/doc/development/import_export.md b/doc/development/import_export.md index f7f48b03651..fd067b80c16 100644 --- a/doc/development/import_export.md +++ b/doc/development/import_export.md @@ -27,7 +27,7 @@ Read through the current performance problems using the Import/Export below. ### OOM errors -Out of memory (OOM) errors are normally caused by the [Sidekiq Memory Killer](https://docs.gitlab.com/ee/administration/operations/sidekiq_memory_killer.html): +Out of memory (OOM) errors are normally caused by the [Sidekiq Memory Killer](../administration/operations/sidekiq_memory_killer.md): ```bash SIDEKIQ_MEMORY_KILLER_MAX_RSS = 2GB in GitLab.com diff --git a/doc/development/licensed_feature_availability.md b/doc/development/licensed_feature_availability.md index 1657d73e0c9..6f3dd59b2c3 100644 --- a/doc/development/licensed_feature_availability.md +++ b/doc/development/licensed_feature_availability.md @@ -7,8 +7,8 @@ on-premise or GitLab.com plans and features. ## Restricting features scoped by namespaces or projects GitLab.com plans are persisted on user groups and namespaces, therefore, if you're adding a -feature such as [Related issues](https://docs.gitlab.com/ee/user/project/issues/related_issues.html) or -[Service desk](https://docs.gitlab.com/ee/user/project/service_desk.html), +feature such as [Related issues](../user/project/issues/related_issues.md) or +[Service desk](../user/project/service_desk.md), it should be restricted on namespace scope. 1. Add the feature symbol on `EES_FEATURES`, `EEP_FEATURES` or `EEU_FEATURES` constants in @@ -22,8 +22,8 @@ project.feature_available?(:feature_symbol) ## Restricting global features (instance) -However, for features such as [Geo](https://docs.gitlab.com/ee/administration/geo/replication/index.html) and -[Load balancing](https://docs.gitlab.com/ee/administration/database_load_balancing.html), which cannot be restricted +However, for features such as [Geo](../administration/geo/replication/index.md) and +[Load balancing](../administration/database_load_balancing.md), which cannot be restricted to only a subset of projects or namespaces, the check will be made directly in the instance license. diff --git a/doc/development/migration_style_guide.md b/doc/development/migration_style_guide.md index 0c326eeb851..9b26f691b55 100644 --- a/doc/development/migration_style_guide.md +++ b/doc/development/migration_style_guide.md @@ -186,7 +186,11 @@ end When adding a foreign-key constraint to either an existing or new column remember to also add a index on the column. -This is _required_ for all foreign-keys. +This is **required** for all foreign-keys, e.g., to support efficient cascading +deleting: when a lot of rows in a table get deleted, the referenced records need +to be deleted too. The database has to look for corresponding records in the +referenced table. Without an index, this will result in a sequential scan on the +table which can take a long time. Here's an example where we add a new column with a foreign key constraint. Note it includes `index: true` to create an index for it. diff --git a/doc/development/new_fe_guide/development/components.md b/doc/development/new_fe_guide/development/components.md index 8ae58d30c35..963ce53423b 100644 --- a/doc/development/new_fe_guide/development/components.md +++ b/doc/development/new_fe_guide/development/components.md @@ -17,5 +17,5 @@ D3 is very popular across many projects outside of GitLab: Within GitLab, D3 has been used for the following notable features -- [Prometheus graphs](https://docs.gitlab.com/ee/user/project/integrations/prometheus.html) +- [Prometheus graphs](../../../user/project/integrations/prometheus.md) - Contribution calendars diff --git a/doc/development/new_fe_guide/tips.md b/doc/development/new_fe_guide/tips.md index 889a5aab2b7..4564f678ec0 100644 --- a/doc/development/new_fe_guide/tips.md +++ b/doc/development/new_fe_guide/tips.md @@ -10,16 +10,16 @@ yarn clean ## Creating feature flags in development -The process for creating a feature flag is the same as [enabling a feature flag in development](https://docs.gitlab.com/ee/development/feature_flags.html#enabling-a-feature-flag-in-development). +The process for creating a feature flag is the same as [enabling a feature flag in development](../feature_flags.md#enabling-a-feature-flag-in-development). Your feature flag can now be: -- [made available to the frontend](https://docs.gitlab.com/ee/development/feature_flags.html#frontend) via the `gon` -- queried in [tests](https://docs.gitlab.com/ee/development/feature_flags.html#specs) +- [made available to the frontend](../feature_flags.md#frontend) via the `gon` +- queried in [tests](../feature_flags.md#specs) - queried in HAML templates and ruby files via the `Feature.enabled?(:my_shiny_new_feature_flag)` method ### More on feature flags -- [Deleting a feature flag](https://docs.gitlab.com/ee/api/features.html#delete-a-feature) -- [Manage feature flags](https://docs.gitlab.com/ee/development/feature_flags.html) -- [Feature flags API](https://docs.gitlab.com/ee/api/features.html) +- [Deleting a feature flag](../../api/features.md#delete-a-feature) +- [Manage feature flags](../feature_flags.md) +- [Feature flags API](../../api/features.md) diff --git a/doc/development/packages.md b/doc/development/packages.md index a3b891d7834..ab0c5f9904d 100644 --- a/doc/development/packages.md +++ b/doc/development/packages.md @@ -1,8 +1,8 @@ # Packages **[PREMIUM]** -This document will guide you through adding another [package management system](https://docs.gitlab.com/ee/administration/packages.html) support to GitLab. +This document will guide you through adding another [package management system](../administration/packages.md) support to GitLab. -See already supported package types in [Packages documentation](https://docs.gitlab.com/ee/administration/packages.html) +See already supported package types in [Packages documentation](../administration/packages.md) Since GitLab packages' UI is pretty generic, it is possible to add new package system support by solely backend changes. This guide is superficial and does @@ -46,7 +46,7 @@ Group-level and instance-level endpoints are good to have but are optional. NOTE: **Note:** To avoid name conflict for instance-level endpoints we use -[the package naming convention](https://docs.gitlab.com/ee/user/project/packages/npm_registry.html#package-naming-convention) +[the package naming convention](../user/project/packages/npm_registry.md#package-naming-convention) ## Configuration diff --git a/doc/development/rake_tasks.md b/doc/development/rake_tasks.md index 27fc3231218..28a12572961 100644 --- a/doc/development/rake_tasks.md +++ b/doc/development/rake_tasks.md @@ -31,7 +31,7 @@ project. #### Seeding issues for Insights charts **[ULTIMATE]** You can seed issues specifically for working with the -[Insights charts](https://docs.gitlab.com/ee/user/group/insights/index.html) with the +[Insights charts](../user/group/insights/index.md) with the `gitlab:seed:insights:issues` task: ```shell @@ -108,11 +108,13 @@ To make sure that indices still fit. You could find great details in: In order to run the test you can use the following commands: -- `rake spec` to run the rspec suite -- `rake karma` to run the karma test suite -- `rake gitlab:test` to run all the tests +- `bin/rake spec` to run the rspec suite +- `bin/rake spec:unit` to run the only the unit tests +- `bin/rake spec:integration` to run the only the integration tests +- `bin/rake spec:system` to run the only the system tests +- `bin/rake karma` to run the karma test suite -Note: `rake spec` takes significant time to pass. +Note: `bin/rake spec` takes significant time to pass. Instead of running full test suite locally you can save a lot of time by running a single test or directory related to your changes. After you submit merge request CI will run full test suite for you. Green CI status in the merge request means @@ -121,6 +123,9 @@ full test suite is passed. Note: You can't run `rspec .` since this will try to run all the `_spec.rb` files it can find, also the ones in `/tmp` +Note: You can pass RSpec command line options to the `spec:unit`, +`spec:integration`, and `spec:system` tasks, e.g. `bin/rake "spec:unit[--tag ~geo --dry-run]"`. + To run a single test file you can use: - `bin/rspec spec/controllers/commit_controller_spec.rb` for a rspec test diff --git a/doc/development/rolling_out_changes_using_feature_flags.md b/doc/development/rolling_out_changes_using_feature_flags.md index 8e4e07c4319..84028b1b342 100644 --- a/doc/development/rolling_out_changes_using_feature_flags.md +++ b/doc/development/rolling_out_changes_using_feature_flags.md @@ -102,7 +102,7 @@ GitLab's feature library (using [Flipper](https://github.com/jnunemaker/flipper), and covered in the [Feature Flags](feature_flags.md) guide) supports rolling out changes to a percentage of users. This in turn can be controlled using [GitLab -chatops](https://docs.gitlab.com/ee/ci/chatops/). +chatops](../ci/chatops/README.md). For an up to date list of feature flag commands please see [the source code](https://gitlab.com/gitlab-com/chatops/blob/master/lib/chatops/commands/feature.rb). @@ -200,10 +200,9 @@ isn't gated by a License or Plan. ### Undefined feature flags default to "on" -An important side-effect of the [implicit feature -flags][#implicit-feature-flags] mentioned above is that unless the feature is -explicitly disabled or limited to a percentage of users, the feature flag check -will default to `true`. +An important side-effect of the [implicit feature flags](#implicit-feature-flags) +mentioned above is that unless the feature is explicitly disabled or limited to a +percentage of users, the feature flag check will default to `true`. As an example, if you were to ship the backend half of a feature behind a flag, you'd want to explicitly disable that flag until the frontend half is also ready diff --git a/doc/development/routing.md b/doc/development/routing.md new file mode 100644 index 00000000000..e9c0ad8d4e8 --- /dev/null +++ b/doc/development/routing.md @@ -0,0 +1,63 @@ +# Routing + +The GitLab backend is written primarily with Rails so it uses [Rails +routing](https://guides.rubyonrails.org/routing.html). Beside Rails best +practices, there are few rules unique to the GitLab application. To +support subgroups, GitLab project and group routes use the wildcard +character to match project and group routes. For example, we might have +a path such as: + + /gitlab-com/customer-success/north-america/west/customerA + +However, paths can be ambiguous. Consider the following example: + + /gitlab-com/edit + +It's ambiguous whether there is a subgroup named `edit` or whether +this is a special endpoint to edit the `gitlab-com` group. + +To eliminate the ambiguity and to make the backend easier to maintain, +we introduced the `/-/` scope. The purpose of it is to separate group or +project paths from the rest of the routes. Also it helps to reduce the +number of [reserved names](../user/reserved_names.md). + +## Global routes + +We have a number of global routes. For example: + + /-/health + /-/metrics + +## Group routes + +Every group route must be under the `/-/` scope. + +Examples: + + gitlab-org/-/edit + gitlab-org/-/activity + gitlab-org/-/security/dashboard + gitlab-org/serverless/-/activity + +To achieve that, use the `scope '-'` method. + +## Project routes + +Every project route must be under the `/-/` scope, except cases where a Git +client or other software requires something different. + +Examples: + + gitlab-org/gitlab-ce/-/activity + gitlab-org/gitlab-ce/-/jobs/123 + gitlab-org/gitlab-ce/-/settings/repository + gitlab-org/serverless/runtimes/-/settings/repository + +Currently, only some project routes are placed under the `/-/` scope. However, +you can help us migrate more of them! To migrate project routes: + +1. Modify existing routes by adding `-` scope. +1. Add redirects for legacy routes by using `Gitlab::Routing.redirect_legacy_paths`. +1. Create a technical debt issue to remove deprecated routes in later releases. + +To get started, see an [example merge request](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/28435). diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md index 63ec9755462..82439c94c5a 100644 --- a/doc/development/testing_guide/best_practices.md +++ b/doc/development/testing_guide/best_practices.md @@ -2,19 +2,29 @@ ## Test Design -Testing at GitLab is a first class citizen, not an afterthought. It's important we consider the design of our tests -as we do the design of our features. +Testing at GitLab is a first class citizen, not an afterthought. It's important we consider the design of our tests +as we do the design of our features. -When implementing a feature, we think about developing the right capabilities the right way, which helps us -narrow our scope to a manageable level. When implementing tests for a feature, we must think about developing -the right tests, but then cover _all_ the important ways the test may fail, which can quickly widen our scope to +When implementing a feature, we think about developing the right capabilities the right way, which helps us +narrow our scope to a manageable level. When implementing tests for a feature, we must think about developing +the right tests, but then cover _all_ the important ways the test may fail, which can quickly widen our scope to a level that is difficult to manage. -Test heuristics can help solve this problem. They concisely address many of the common ways bugs -manifest themselves within our code. When designing our tests, take time to review known test heuristics to inform -our test design. We can find some helpful heuristics documented in the Handbook in the +Test heuristics can help solve this problem. They concisely address many of the common ways bugs +manifest themselves within our code. When designing our tests, take time to review known test heuristics to inform +our test design. We can find some helpful heuristics documented in the Handbook in the [Test Design](https://about.gitlab.com/handbook/engineering/quality/guidelines/test-engineering/test-design/) section. +## Run tests against MySQL + +By default, tests are only run againts PostgreSQL, but you can run them on +demand against MySQL by following one of the following conventions: + +| Convention | Valid example | +|:----------------------|:-----------------------------| +| Include `mysql` in your branch name | `enhance-mysql-support` | +| Include `[run mysql]` in your commit message | `Fix MySQL support<br><br>[run mysql]` | + ## Test speed GitLab has a massive test suite that, without [parallelization], can take hours @@ -184,11 +194,11 @@ instead of 30+ seconds in case of a regular `spec_helper`. ### `let` variables GitLab's RSpec suite has made extensive use of `let`(along with it strict, non-lazy -version `let!`) variables to reduce duplication. However, this sometimes [comes at the cost of clarity][lets-not], +version `let!`) variables to reduce duplication. However, this sometimes [comes at the cost of clarity][lets-not], so we need to set some guidelines for their use going forward: - `let!` variables are preferable to instance variables. `let` variables - are preferable to `let!` variables. Local variables are preferable to + are preferable to `let!` variables. Local variables are preferable to `let` variables. - Use `let` to reduce duplication throughout an entire spec file. - Don't use `let` to define variables used by a single test; define them as @@ -199,8 +209,8 @@ so we need to set some guidelines for their use going forward: - Try to avoid overriding the definition of one `let` variable with another. - Don't define a `let` variable that's only used by the definition of another. Use a helper method instead. -- `let!` variables should be used only in case if strict evaluation with defined - order is required, otherwise `let` will suffice. Remember that `let` is lazy and won't +- `let!` variables should be used only in case if strict evaluation with defined + order is required, otherwise `let` will suffice. Remember that `let` is lazy and won't be evaluated until it is referenced. [lets-not]: https://robots.thoughtbot.com/lets-not diff --git a/doc/development/testing_guide/end_to_end/best_practices.md b/doc/development/testing_guide/end_to_end/best_practices.md new file mode 100644 index 00000000000..89500ef9a90 --- /dev/null +++ b/doc/development/testing_guide/end_to_end/best_practices.md @@ -0,0 +1,38 @@ +# Best practices when writing end-to-end tests + +The majority of the end-to-end tests require some state to be built in the application for the tests to happen. + +A good example is a user being logged in as a pre-condition for testing the feature. + +But if the login feature is already covered with end-to-end tests through the GUI, there is no reason to perform such an expensive task to test the functionality of creating a project, or importing a repo, even if these features depend on a user being logged in. Let's see an example to make things clear. + +Let's say that, on average, the process to perform a successful login through the GUI takes 2 seconds. + +Now, realize that almost all tests need the user to be logged in, and that we need every test to run in isolation, meaning that tests cannot interfere with each other. This would mean that for every test the user needs to log in, and "waste 2 seconds". + +Now, multiply the number of tests per 2 seconds, and as your test suite grows, the time to run it grows with it, and this is not sustainable. + +An alternative to perform a login in a cheaper way would be having an endpoint (available only for testing) where we could pass the user's credentials as encrypted values as query strings, and then we would be redirected to the logged in home page if the credentials are valid. Let's say that, on average, this process takes only 200 miliseconds. + +You see the point right? + +Performing a login through the GUI for every test would cost a lot in terms of tests' execution. + +And there is another reason. + +Let's say that you don't follow the above suggestion, and depend on the GUI for the creation of every application state in order to test a specific feature. In this case we could be talking about the **Issues** feature, that depends on a project to exist, and the user to be logged in. + +What would happen if there was a bug in the project creation page, where the 'Create' button is disabled, not allowing for the creation of a project through the GUI, but the API logic is still working? + +In this case, instead of having only the project creation test failing, we would have many tests that depend on a project to be failing too. + +But, if we were following the best practices, only one test would be failing, and tests for other features that depend on a project to exist would continue to pass, since they could be creating the project behind the scenes interacting directly with the public APIs, ensuring a more reliable metric of test failure rate. + +Finally, interacting with the application only by its GUI generates a higher rate of test flakiness, and we want to avoid that at max. + +**The takeaways here are:** + +- Building state through the GUI is time consuming and it's not sustainable as the test suite grows. +- When depending only on the GUI to create the application's state and tests fail due to front-end issues, we can't rely on the test failures rate, and we generate a higher rate of test flakiness. + +Now that we are aware of all of it, [let's go create some tests](quick_start_guide.md). diff --git a/doc/development/testing_guide/end_to_end_tests.md b/doc/development/testing_guide/end_to_end/index.md index fc7aaedca29..afd81ff00b2 100644 --- a/doc/development/testing_guide/end_to_end_tests.md +++ b/doc/development/testing_guide/end_to_end/index.md @@ -65,7 +65,7 @@ Below you can read more about how to use it and how does it work. Currently, we are using _multi-project pipeline_-like approach to run QA pipelines. - + <details> <summary>Show mermaid source</summary> @@ -136,6 +136,12 @@ Once you decided where to put [test environment orchestration scenarios] and the [GitLab QA orchestrator README][gitlab-qa-readme], and [the already existing instance-level scenarios][instance-level scenarios]. +Continued reading: + +- [Quick Start Guide](quick_start_guide.md) +- [Style Guide](style_guide.md) +- [Best Practices](best_practices.md) + ## Where can I ask for help? You can ask question in the `#quality` channel on Slack (GitLab internal) or @@ -149,7 +155,7 @@ you can find an issue you would like to work on in [gitlab-qa-readme]: https://gitlab.com/gitlab-org/gitlab-qa/tree/master/README.md [quality-nightly-pipelines]: https://gitlab.com/gitlab-org/quality/nightly/pipelines [quality-staging-pipelines]: https://gitlab.com/gitlab-org/quality/staging/pipelines -[review-apps]: ./review_apps.md +[review-apps]: ../review_apps.md [gitlab-qa-architecture]: https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/architecture.md [gitlab-qa-issues]: https://gitlab.com/gitlab-org/gitlab-qa/issues?label_name%5B%5D=new+scenario [gitlab-ce-issues]: https://gitlab.com/gitlab-org/gitlab-ce/issues?label_name[]=QA&label_name[]=test diff --git a/doc/development/testing_guide/end_to_end/page_objects.md b/doc/development/testing_guide/end_to_end/page_objects.md new file mode 100644 index 00000000000..d0de33892c4 --- /dev/null +++ b/doc/development/testing_guide/end_to_end/page_objects.md @@ -0,0 +1,134 @@ +# Page objects in GitLab QA + +In GitLab QA we are using a known pattern, called _Page Objects_. + +This means that we have built an abstraction for all GitLab pages that we use +to drive GitLab QA scenarios. Whenever we do something on a page, like filling +in a form, or clicking a button, we do that only through a page object +associated with this area of GitLab. + +For example, when GitLab QA test harness signs in into GitLab, it needs to fill +in a user login and user password. In order to do that, we have a class, called +`Page::Main::Login` and `sign_in_using_credentials` methods, that is the only +piece of the code, that has knowledge about `user_login` and `user_password` +fields. + +## Why do we need that? + +We need page objects, because we need to reduce duplication and avoid problems +whenever someone changes some selectors in GitLab's source code. + +Imagine that we have a hundred specs in GitLab QA, and we need to sign into +GitLab each time, before we make assertions. Without a page object one would +need to rely on volatile helpers or invoke Capybara methods directly. Imagine +invoking `fill_in :user_login` in every `*_spec.rb` file / test example. + +When someone later changes `t.text_field :login` in the view associated with +this page to `t.text_field :username` it will generate a different field +identifier, what would effectively break all tests. + +Because we are using `Page::Main::Login.act { sign_in_using_credentials }` +everywhere, when we want to sign into GitLab, the page object is the single +source of truth, and we will need to update `fill_in :user_login` +to `fill_in :user_username` only in a one place. + +## What problems did we have in the past? + +We do not run QA tests for every commit, because of performance reasons, and +the time it would take to build packages and test everything. + +That is why when someone changes `t.text_field :login` to +`t.text_field :username` in the _new session_ view we won't know about this +change until our GitLab QA nightly pipeline fails, or until someone triggers +`package-and-qa` action in their merge request. + +Obviously such a change would break all tests. We call this problem a _fragile +tests problem_. + +In order to make GitLab QA more reliable and robust, we had to solve this +problem by introducing coupling between GitLab CE / EE views and GitLab QA. + +## How did we solve fragile tests problem? + +Currently, when you add a new `Page::Base` derived class, you will also need to +define all selectors that your page objects depends on. + +Whenever you push your code to CE / EE repository, `qa:selectors` sanity test +job is going to be run as a part of a CI pipeline. + +This test is going to validate all page objects that we have implemented in +`qa/page` directory. When it fails, you will be notified about missing +or invalid views / selectors definition. + +## How to properly implement a page object? + +We have built a DSL to define coupling between a page object and GitLab views +it is actually implemented by. See an example below. + +```ruby +module Page + module Main + class Login < Page::Base + view 'app/views/devise/passwords/edit.html.haml' do + element :password_field + element :password_confirmation + element :change_password_button + end + + view 'app/views/devise/sessions/_new_base.html.haml' do + element :login_field + element :password_field + element :sign_in_button + end + + # ... + end +end +``` + +The `view` DSL method declares the filename of the view where an +`element` is implemented. + +The `element` DSL method in turn declares an element for which a corresponding +`qa-element-name-dasherized` CSS class need to be added to the view file. + +You can also define a value (String or Regexp) to match to the actual view +code but **this is deprecated** in favor of the above method for two reasons: + +- Consistency: there is only one way to define an element +- Separation of concerns: QA uses dedicated CSS classes instead of reusing code + or classes used by other components (e.g. `js-*` classes etc.) + +```ruby +view 'app/views/my/view.html.haml' do + # Implicitly require `.qa-logout-button` CSS class to be present in the view + element :logout_button + + ## This is deprecated and forbidden by the `QA/ElementWithPattern` RuboCop cop. + # Require `f.submit "Sign in"` to be present in `my/view.html.haml + element :my_button, 'f.submit "Sign in"' # rubocop:disable QA/ElementWithPattern + + ## This is deprecated and forbidden by the `QA/ElementWithPattern` RuboCop cop. + # Match every line in `my/view.html.haml` against + # `/link_to .* "My Profile"/` regexp. + element :profile_link, /link_to .* "My Profile"/ # rubocop:disable QA/ElementWithPattern +end +``` + +## Running the test locally + +During development, you can run the `qa:selectors` test by running + +```shell +bin/qa Test::Sanity::Selectors +``` + +from within the `qa` directory. + +## Where to ask for help? + +If you need more information, ask for help on `#quality` channel on Slack +(internal, GitLab Team only). + +If you are not a Team Member, and you still need help to contribute, please +open an issue in GitLab CE issue tracker with the `~QA` label. diff --git a/doc/development/testing_guide/end_to_end/quick_start_guide.md b/doc/development/testing_guide/end_to_end/quick_start_guide.md new file mode 100644 index 00000000000..afe76acf9c9 --- /dev/null +++ b/doc/development/testing_guide/end_to_end/quick_start_guide.md @@ -0,0 +1,585 @@ +# Writing end-to-end tests step-by-step + +In this tutorial, you will find different examples, and the steps involved, in the creation of end-to-end (_e2e_) tests for GitLab CE and GitLab EE, using GitLab QA. + +> When referring to end-to-end tests in this document, this means testing a specific feature end-to-end, such as a user logging in, the creation of a project, the management of labels, breaking down epics into sub-epics and issues, etc. + +## Important information before we start writing tests + +It's important to understand that end-to-end tests of isolated features, such as the ones described in the above note, doesn't mean that everything needs to happen through the GUI. + +If you don't exactly understand what we mean by **not everything needs to happen through the GUI,** please make sure you've read the [best practices](best_practices.md) before moving on. + +## This document covers the following items: + +- [0.](#0-are-end-to-end-tests-needed) Identifying if end-to-end tests are really needed +- [1.](#1-identifying-the-devops-stage) Identifying the [DevOps stage](https://about.gitlab.com/stages-devops-lifecycle/) of the feature that you are going to cover with end-to-end tests +- [2.](#2-test-skeleton) Creating the skeleton of the test file (`*_spec.rb`) +- [3.](#3-test-cases-mvc) The [MVC](https://about.gitlab.com/handbook/values/#minimum-viable-change-mvc) of the test cases' logic +- [4.](#4-extracting-duplicated-code) Extracting duplicated code into methods +- [5.](#5-tests-pre-conditions-using-resources-and-page-objects) Tests' pre-conditions (`before :context` and `before`) using resources and [Page Objects] +- [6.](#6-optimization) Optimizing the test suite +- [7.](#7-resources) Using and implementing resources +- [8.](#8-page-objects) Moving element definitions and methods to [Page Objects] + +### 0. Are end-to-end tests needed? + +At GitLab we respect the [test pyramid](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/testing_guide/testing_levels.md), and so, we recommend you check the code coverage of a specific feature before writing end-to-end tests, for both [CE](https://gitlab-org.gitlab.io/gitlab-ce/coverage-ruby/#_AllFiles) and [EE](https://gitlab-org.gitlab.io/gitlab-ee/coverage-ruby/#_AllFiles) projects. + +Sometimes you may notice that there is already good coverage in other test levels, and we can stay confident that if we break a feature, we will still have quick feedback about it, even without having end-to-end tests. + +If after this analysis you still think that end-to-end tests are needed, keep reading. + +### 1. Identifying the DevOps stage + +The GitLab QA end-to-end tests are organized by the different [stages in the DevOps lifecycle](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa/qa/specs/features/browser_ui), and so, if you are creating tests for issue creation, for instance, you would locate the spec files under the `qa/qa/specs/features/browser_ui/2_plan/` directory since issue creation is part of the Plan stage. + + In another case of a test for listing merged merge requests (MRs), the test should go under the `qa/qa/specs/features/browser_ui/3_create/` directory since merge requests are a feature from the Create stage. + +> There may be sub-directories inside the stages directories, for different features. For example: `.../browser_ui/2_plan/ee_epics/` and `.../browser_ui/2_plan/issues/`. + +Now, let's say we want to create tests for the [scoped labels](https://about.gitlab.com/2019/04/22/gitlab-11-10-released/#scoped-labels) feature, available on GitLab EE Premium (this feature is part of the Plan stage.) + +> Because these tests are for a feature available only on GitLab EE, we need to create them in the [EE repository](https://gitlab.com/gitlab-org/gitlab-ee). + +Since [there is no specific directory for this feature](https://gitlab.com/gitlab-org/gitlab-ee/tree/master/qa/qa/specs/features/browser_ui/2_plan), we should create a sub-directory for it. + +Under `.../browser_ui/2_plan/`, let's create a sub-directory called `ee_scoped_labels/`. + +> Notice that since this feature is only available for GitLab EE we prefix the sub-directory with `ee_`. + +### 2. Test skeleton + +Inside the newly created sub-directory, let's create a file describing the test suite (e.g. `editing_scoped_labels_spec.rb`.) + +#### The `context` and `describe` blocks + +Specs have an outer `context` that indicates the DevOps stage. The next level is the `describe` block, that briefly states the subject of the test suite. See the following example: + +```ruby +module QA + context 'Plan' do + describe 'Editing scoped labels on issues' do + end + end +end +``` + +#### The `it` blocks + +Every test suite is composed of at least one `it` block, and a good way to start writing end-to-end tests is by writing test cases descriptions as `it` blocks. These might help you to think of different test scenarios. Take a look at the following example: + +```ruby +module QA + context 'Plan' do + describe 'Editing scoped labels on issues' do + it 'replaces an existing label if it has the same key' do + end + + it 'keeps both scoped labels when adding a label with a different key' do + end + end + end +end +``` + +### 3. Test cases MVC + +For the [MVC](https://about.gitlab.com/handbook/values/#minimum-viable-change-mvc) of our test cases, let's say that we already have the application in the state needed for the tests, and then let's focus on the logic of the test cases only. + +To evolve the test cases drafted on step 2, let's imagine that the user is already logged into a GitLab EE instance, they already have at least a Premium license in use, there is already a project created, there is already an issue opened in the project, the issue already has a scoped label (e.g. `animal::fox`), there are other scoped labels (for the same scope and for a different scope (e.g. `animal::dolphin` and `plant::orchid`), and finally, the user is already on the issue's page. Let's also suppose that for every test case the application is in a clean state, meaning that one test case won't affect another. + +> Note: there are different approaches to creating an application state for end-to-end tests. Some of them are very time consuming and subject to failures, such as when using the GUI for all the pre-conditions of the tests. On the other hand, other approaches are more efficient, such as using the public APIs. The latter is more efficient since it doesn't depend on the GUI. We won't focus on this part yet, but it's good to keep it in mind. + +Let's now focus on the first test case. + +```ruby +it 'replaces an existing label if it has the same key' do + # This implementation is only for tutorial purposes. We normally encapsulate elements in Page Objects (which we cover on section 8). + page.find('.block.labels .edit-link').click + page.find('.dropdown-menu-labels .dropdown-input-field').send_keys ['animal::dolphin', :enter] + page.find('#content-body').click + page.refresh + + labels_block = page.find('.qa-labels-block') + + expect(labels_block).to have_content('animal::dolphin') + expect(labels_block).not_to have_content('animal::fox') + expect(page).to have_content('added animal::dolphin label and removed animal::fox') +end +``` + +> Notice that the test itself is simple. The most challenging part is the creation of the application state, which will be covered later. + +> The exemplified test case's MVC is not enough for the change to be merged, but it helps to build up the test logic. The reason is that we do not want to use locators directly in the tests, and tests **must** use [Page Objects] before they can be merged. This way we better separate the responsibilities, where the Page Objects encapsulate elements and methods that allow us to interact with pages, while the spec files describe the test cases in more business-related language. + +Below are the steps that the test covers: + +1. The test finds the 'Edit' link for the labels and clicks on it. +2. Then it fills in the 'Assign labels' input field with the value 'animal::dolphin' and press enters. +3. Then it clicks in the content body to apply the label and refreshes the page. +4. Finally, the expectations check that the previous scoped label was removed and that the new one was added. + +Let's now see how the second test case would look. + +```ruby +it 'keeps both scoped labels when adding a label with a different key' do + # This implementation is only for tutorial purposes. We normally encapsulate elements in Page Objects (which we cover on section 8). + page.find('.block.labels .edit-link').click + page.find('.dropdown-menu-labels .dropdown-input-field').send_keys ['plant::orchid', :enter] + page.find('#content-body').click + page.refresh + + labels_block = page.find('.qa-labels-block') + + expect(labels_block).to have_content('animal::fox') + expect(labels_block).to have_content('plant::orchid') + expect(page).to have_content('added animal::fox') + expect(page).to have_content('added plant::orchid') +end +``` + +> Note that elements are always located using CSS selectors, and a good practice is to add test-specific selectors (this is called adding testability to the application and we will talk more about it later.) For example, the `labels_block` element uses the selector `.qa-labels-block`, which was added specifically for testing purposes. + +Below are the steps that the test covers: + +1. The test finds the 'Edit' link for the labels and clicks on it. +2. Then it fills in the 'Assign labels' input field with the value 'plant::orchid' and press enters. +3. Then it clicks in the content body to apply the label and refreshes the page. +4. Finally, the expectations check that both scoped labels are present. + +> Similar to the previous test, this one is also very straightforward, but there is some code duplication. Let's address it. + +### 4. Extracting duplicated code + +If we refactor the tests created on step 3 we could come up with something like this: + +```ruby +before do + ... + + @initial_label = 'animal::fox' + @new_label_same_scope = 'animal::dolphin' + @new_label_different_scope = 'plant::orchid' + + ... +end + +it 'replaces an existing label if it has the same key' do + select_label_and_refresh @new_label_same_scope + + labels_block = page.find('.qa-labels-block') + + expect(labels_block).to have_content(@new_label_same_scope) + expect(labels_block).not_to have_content(@initial_label) + expect(page).to have_content("added #{@new_label_same_scope}") + expect(page).to have_content("and removed #{@initial_label}") +end + +it 'keeps both scoped label when adding a label with a different key' do + select_label_and_refresh @new_label_different_scope + + labels_block = page.find('.qa-labels-block') + + expect(labels_blocks).to have_content(@new_label_different_scope) + expect(labels_blocks).to have_content(@initial_label) + expect(page).to have_content("added #{@new_label_different_scope}") + expect(page).to have_content("added #{@initial_label}") +end + +def select_label_and_refresh(label) + page.find('.block.labels .edit-link').click + page.find('.dropdown-menu-labels .dropdown-input-field').send_keys [label, :enter] + page.find('#content-body').click + page.refresh +end +``` + +First, we remove the duplication of strings by defining the global variables `@initial_label`, `@new_label_same_scope` and `@new_label_different_scope` in the `before` block, and by using them in the expectations. + +Then, by creating a reusable `select_label_and_refresh` method, we remove the code duplication of this action, and later we can move this method to a Page Object class that will be created for easier maintenance purposes. + +> Notice that the reusable method is created at the bottom of the file. The reason for that is that reading the code should be similar to reading a newspaper, where high-level information is at the top, like the title and summary of the news, while low level, or more specific information, is at the bottom (this helps readability). + +### 5. Tests' pre-conditions using resources and Page Objects + +In this section, we will address the previously mentioned subject of creating the application state for the tests, using the `before :context` and `before` blocks, together with resources and Page Objects. + +#### `before :context` + +A pre-condition for the entire test suite is defined in the `before :context` block. + +> For our test suite, due to the need of the tests being completely independent of each other, we won't use the `before :context` block. The `before :context` block would make the tests dependent on each other because the first test changes the label of the issue, and the second one depends on the `'animal::fox'` label being set. + +> **Tip:** In case of a test suite with only one `it` block it's ok to use only the `before` block (see below) with all the test's pre-conditions. + +#### `before` + +As the pre-conditions for our test suite, the things that needs to happen before each test starts are: + +- The user logging in; +- A premium license already being set; +- A project being created with an issue and labels already set; +- The issue page being opened with only one scoped label applied to the it. + +> When running end-to-end tests as part of the GitLab's continuous integration process [a license is already set as an environment variable](https://gitlab.com/gitlab-org/gitlab-ee/blob/1a60d926740db10e3b5724713285780a4f470531/qa/qa/ee/strategy.rb#L20). For running tests locally you can set up such license by following the document [what tests can be run?](https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/what_tests_can_be_run.md#supported-remote-grid-environment-variables), based on the [supported GitLab environment variables](https://gitlab.com/gitlab-org/gitlab-qa/blob/master/docs/what_tests_can_be_run.md#supported-gitlab-environment-variables). + +#### Implementation + +In the following code we will focus only on the test suite's pre-conditions: + +```ruby +module QA + context 'Plan' do + describe 'Editing scoped labels on issues' do + before do + Runtime::Browser.visit(:gitlab, Page::Main::Login) + Page::Main::Login.perform(&:sign_in_using_credentials) + + @initial_label = 'animal::fox' + @new_label_same_scope = 'animal::dolphin' + @new_label_different_scope = 'plant::orchid' + + issue = Resource::Issue.fabricate_via_api! do |issue| + issue.title = 'Issue to test the scoped labels' + issue.labels = @initial_label + end + + [@new_label_same_scope, @new_label_different_scope].each do |label| + Resource::Label.fabricate_via_api! do |l| + l.project = issue.project.id + l.title = label + end + end + + issue.visit! + end + + it 'replaces an existing label if it has the same key' do + ... + end + + it 'keeps both scoped labels when adding a label with a different key' do + ... + end + + def select_label_and_refresh(label) + ... + end + end + end +end +``` + +In the `before` block we create all the application state needed for the tests to run. We do that by using the `Runtime::Browser.visit` method to go to the login page, by performing a `sign_in_using_credentials` from the `Login` Page Object, by fabricating resources via APIs (`issue`, and `Resource::Label`), and by using the `issue.visit!` to visit the issue page. + +> A project is created in the background by creating the `issue` resource. + +> When creating the [Resources], notice that when calling the `fabricate_via_api` method, we pass some attribute:values, like `title`, and `labels` for the `issue` resource; and `project` and `title` for the `label` resource. + +> What's important to understand here is that by creating the application state mostly using the public APIs we save a lot of time in the test suite setup stage. + +> Soon we will cover the use of the already existing resources' methods and the creation of your own `fabricate_via_api` methods for resources where this is still not available, but first, let's optimize our implementation. + +### 6. Optimization + +As already mentioned in the [best practices](best_practices.md) document, end-to-end tests are very costly in terms of execution time, and it's our responsibility as software engineers to ensure that we optimize them as much as possible. + +> Note that end-to-end tests are slow to run and so they can have several actions and assertions in a single test, which helps us get feedback from the tests sooner. In comparison, unit tests are much faster to run and can exercise every little piece of the application in isolation, and so they usually have only one assertion per test. + +Some improvements that we could make in our test suite to optimize its time to run are: + +1. Having a single test case (an `it` block) that exercises both scenarios to avoid "wasting" time in the tests' pre-conditions, instead of having two different test cases. +2. Making the selection of labels more performant by allowing for the selection of more than one label in the same reusable method. + +Let's look at a suggestion that addresses the above points, one by one: + +```ruby +module QA + context 'Plan' do + describe 'Editing scoped labels on issues' do + before do + ... + end + + it 'correctly applies scoped labels depending on if they are from the same or a different scope' do + select_labels_and_refresh [@new_label_same_scope, @new_label_different_scope] + + labels_block = page.all('.qa-labels-block') + + expect(labels_block).to have_content(@new_label_same_scope) + expect(labels_block).to have_content(@new_label_different_scope) + expect(labels_block).not_to have_content(@initial_label) + expect(page).to have_content("added #{@initial_label}") + expect(page).to have_content("added #{@new_label_same_scope} #{@new_label_different_scope} labels and removed #{@initial_label}") + end + + def select_labels_and_refresh(labels) + find('.block.labels .edit-link').click + labels.each do |label| + find('.dropdown-menu-labels .dropdown-input-field').send_keys [label, :enter] + end + find('#content-body').click + refresh + end + end + end + end +``` + +To address point 1, we changed the test implementation from two `it` blocks into a single one that exercises both scenarios. Now the new test description is: `'correctly applies the scoped labels depending if they are from the same or a different scope'`. It's a long description, but it describes well what the test does. + +> Notice that the implementation of the new and unique `it` block had to change a little bit. Below we describe in details what it does. + +1. It selects two scoped labels simultaneously, one from the same scope of the one already applied in the issue during the setup phase (in the `before` block), and another one from a different scope. +2. It asserts that the correct labels are visible in the `labels_block`, and that the labels were correctly added and removed; +3. Finally, the `select_label_and_refresh` method is changed to `select_labels_and_refresh`, which accepts an array of labels instead of a single label, and it iterates on them for faster label selection (this is what is used in step 1 explained above.) + +### 7. Resources + +**Note:** When writing this document, some code that is now merged to master was not implemented yet, but we left them here for the readers to understand the whole process of end-to-end test creation. + +You can think of [Resources] as anything that can be created on GitLab CE or EE, either through the GUI, the API, or the CLI. + +With that in mind, resources can be a project, an epic, an issue, a label, a commit, etc. + +As you saw in the tests' pre-conditions and the optimization sections, we're already creating some of these resources, and we are doing that by calling the `fabricate_via_api!` method. + +> We could be using the `fabricate!` method instead, which would use the `fabricate_via_api!` method if it exists, and fallback to GUI fabrication otherwise, but we recommend being explicit to make it clear what the test does. Also, we always recommend fabricating resources via API since this makes tests faster and more reliable. + +For our test suite example, the resources that we need to create don't have the necessary code for the `fabricate_via_api!` method to correctly work (e.g., the issue and label resources), so we will have to create them. + +#### Implementation + +In the following we describe the changes needed in each of the resource files mentioned above. + +**Issue resource** + +Now, let's make it possible to create an issue resource through the API. + +First, in the [issue resource](https://gitlab.com/gitlab-org/gitlab-ee/blob/d3584e80b4236acdf393d815d604801573af72cc/qa/qa/resource/issue.rb), let's expose its labels attribute. + +Add the following `attribute :labels` right below the [`attribute :title`](https://gitlab.com/gitlab-org/gitlab-ee/blob/d3584e80b4236acdf393d815d604801573af72cc/qa/qa/resource/issue.rb#L15). + +> This line is needed to allow for labels to be automatically added to an issue when fabricating it via API. + +Next, add the following code right below the [`fabricate!`](https://gitlab.com/gitlab-org/gitlab-ee/blob/d3584e80b4236acdf393d815d604801573af72cc/qa/qa/resource/issue.rb#L27) method. + +```ruby +def api_get_path + "/projects/#{project.id}/issues/#{id}" +end + +def api_post_path + "/projects/#{project.id}/issues" +end + +def api_post_body + { + title: title, + labels: [labels] + } +end +``` + +By defining the `api_get_path` method, we allow the [`ApiFabricator`](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/qa/qa/resource/api_fabricator.rb) module to know which path to use to get a single issue. + +> This `GET` path can be found in the [public API documentation](https://docs.gitlab.com/ee/api/issues.html#single-issue). + +By defining the `api_post_path` method, we allow the [`ApiFabricator`](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/qa/qa/resource/api_fabricator.rb) module to know which path to use to create a new issue in a specific project. + +> This `POST` path can be found in the [public API documentation](https://docs.gitlab.com/ee/api/issues.html#new-issue). + +By defining the `api_post_body` method, we allow the [`ApiFabricator.api_post`](https://gitlab.com/gitlab-org/gitlab-ee/blob/a9177ca1812bac57e2b2fa4560e1d5dd8ffac38b/qa/qa/resource/api_fabricator.rb#L68) method to know which data to send when making the `POST` request. + +> Notice that we pass both `title` and `labels` attributes in the `api_post_body`, where `labels` receives an array of labels, and [`title` is required](https://docs.gitlab.com/ee/api/issues.html#new-issue). + +**Label resource** + +Finally, let's make it possible to create label resources through the API. + +Add the following code right below the [`fabricate!`](https://gitlab.com/gitlab-org/gitlab-ee/blob/a9177ca1812bac57e2b2fa4560e1d5dd8ffac38b/qa/qa/resource/label.rb#L36) method. + +```ruby +def resource_web_url(resource) + super +rescue ResourceURLMissingError + # this particular resource does not expose a web_url property +end + +def api_get_path + raise NotImplementedError, "The Labels API doesn't expose a single-resource endpoint so this method cannot be properly implemented." +end + +def api_post_path + "/projects/#{project}/labels" +end + +def api_post_body + { + name: @title, + color: @color + } +end +``` + +By defining the `resource_web_url(resource)` method, we override the one from the [`ApiFabricator`](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/qa/qa/resource/api_fabricator.rb#L44) module. We do that to avoid failing the test due to this particular resource not exposing a `web_url` property. + +By defining the `api_get_path` method, we **would** allow for the [`ApiFabricator`](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/qa/qa/resource/api_fabricator.rb) module to know which path to use to get a single label, but since there's no path available for that in the publich API, we raise a `NotImplementedError` instead. + +By defining the `api_post_path` method, we allow for the [`ApiFabricator `](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/qa/qa/resource/api_fabricator.rb) module to know which path to use to create a new label in a specific project. + +By defining the `api_post_body` method, we we allow for the [`ApiFabricator.api_post`](https://gitlab.com/gitlab-org/gitlab-ee/blob/a9177ca1812bac57e2b2fa4560e1d5dd8ffac38b/qa/qa/resource/api_fabricator.rb#L68) method to know which data to send when making the `POST` request. + +> Notice that we pass both `name` and `color` attributes in the `api_post_body` since [those are required](https://docs.gitlab.com/ee/api/labels.html#create-a-new-label). + +### 8. Page Objects + +Page Objects are used in end-to-end tests for maintenance reasons, where a page's elements and methods are defined to be reused in any test. + +> Page Objects are auto-loaded in the `qa/qa.rb` file and available in all the test files (`*_spec.rb`). + +Take a look at the [Page Objects] documentation. + +Now, let's go back to our example. + +As you may have noticed, we are defining elements with CSS selectors and the `select_labels_and_refresh` method directly in the test file, and this is an anti-pattern since we need to better separate the responsibilities. + +To address this issue, we will move the implementation to Page Objects, and the test suite will only focus on the business rules that we are testing. + +#### Updates in the test file + +As in a test-driven development approach, let's start changing the test file even before the Page Object implementation is in place. + +Replace the code of the `it` block in the test file by the following: + +```ruby +module QA + context 'Plan' do + describe 'Editing scoped labels on issues' do + before do + ... + end + + it 'correctly applies scoped labels depending on if they are from the same or a different scope' do + Page::Project::Issue::Show.perform do |issue_page| + issue_page.select_labels_and_refresh [@new_label_same_scope, @new_label_different_scope] + + expect(page).to have_content("added #{@initial_label}") + expect(page).to have_content("added #{@new_label_same_scope} #{@new_label_different_scope} labels and removed #{@initial_label}") + expect(issue_page.text_of_labels_block).to have_content(@new_label_same_scope) + expect(issue_page.text_of_labels_block).to have_content(@new_label_different_scope) + expect(issue_page.text_of_labels_block).not_to have_content(@initial_label) + end + end + end + end +end +``` + +Notice that `select_labels_and_refresh` is now a method from the issue Page Object (which is not yet implemented), and that we verify the labels' text by using `text_of_labels_block`, instead of via the `labels_block` element. The `text_of_labels_block` method will also be implemented in the issue Page Object. + +Let's now update the Issue Page Object. + +#### Updates in the Issue Page Object + +> Page Objects are located in the `qa/qa/page/` directory, and its sub-directories. + +The file we will have to change is the [Issue Page Object](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/qa/qa/page/project/issue/show.rb). + +First, add the following code right below the definition of an already implemented view: + +```ruby +view 'app/views/shared/issuable/_sidebar.html.haml' do + element :labels_block + element :edit_link_labels + element :dropdown_menu_labels +end + +view 'app/helpers/dropdowns_helper.rb' do + element :dropdown_input_field +end +``` + +Similarly to what we did before, let's first change the Page Object even without the elements being defined in the view (`_sidebar.html.haml`) and the `dropdowns_helper.rb` files, and later we will update them by adding the appropriate CSS selectors. + +Now, let's implement the methods `select_labels_and_refresh` and `text_of_labels_block`. + +Somewhere between the definition of the views and the private methods, add the following snippet of code: + +```ruby +def select_labels_and_refresh(labels) + click_element(:edit_link_labels) + labels.each do |label| + within_element(:dropdown_menu_labels, text: label) do + send_keys_to_element(:dropdown_input_field, [label, :enter]) + end + end + click_body + labels.each do |label| + has_element?(:labels_block, text: label) + end + refresh +end + +def text_of_labels_block + find_element(:labels_block) +end +``` + +##### Details of `select_labels_and_refresh` + +Notice that we have not only moved the `select_labels_and_refresh` method, but we have also changed its implementation to: +1. Click the `:edit_link_labels` element previously defined, instead of using `find('.block.labels .edit-link').click` +2. Use `within_element(:dropdown_menu_labels, text: label)`, and inside of it, we call `send_keys_to_element(:dropdown_input_field, [label, :enter])`, which is a method that we will implement in the `QA::Page::Base` class to replace `find('.dropdown-menu-labels .dropdown-input-field').send_keys [label, :enter]` +3. Use `click_body` after iterating on each label, instead of using `find('#content-body').click` +4. Iterate on every label again, and then we use `has_element?(:labels_block, text: label)` after clicking the page body (which applies the labels), and before refreshing the page, to avoid test flakiness due to refreshing too fast. + +##### Details of `text_of_labels_block` + +The `text_of_labels_block` method is a simple method that returns the `:labels_block` element (`find_element(:labels_block)`). + +#### Updates in the view (*.html.haml) and `dropdowns_helper.rb` files + +Now let's change the view and the `dropdowns_helper` files to add the selectors that relate to the Page Object. + +In the [app/views/shared/issuable/_sidebar.html.haml](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/app/views/shared/issuable/_sidebar.html.haml) file, on [line 105 ](https://gitlab.com/gitlab-org/gitlab-ee/blob/84043fa72ca7f83ae9cde48ad670e6d5d16501a3/app/views/shared/issuable/_sidebar.html.haml#L105), add an extra class `qa-edit-link-labels`. + +The code should look like this: `= link_to _('Edit'), '#', class: 'js-sidebar-dropdown-toggle edit-link qa-edit-link-labels float-right'`. + +In the same file, on [line 121](https://gitlab.com/gitlab-org/gitlab-ee/blob/84043fa72ca7f83ae9cde48ad670e6d5d16501a3/app/views/shared/issuable/_sidebar.html.haml#L121), add an extra class `.qa-dropdown-menu-labels`. + +The code should look like this: `.dropdown-menu.dropdown-select.dropdown-menu-paging.dropdown-menu-labels.qa-dropdown-menu-labels.dropdown-menu-selectable`. + +In the [`dropdowns_helper.rb`](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/app/helpers/dropdowns_helper.rb) file, on [line 94](https://gitlab.com/gitlab-org/gitlab-ee/blob/99e51a374f2c20bee0989cac802e4b5621f72714/app/helpers/dropdowns_helper.rb#L94), add an extra class `qa-dropdown-input-field`. + +The code should look like this: `filter_output = search_field_tag search_id, nil, class: "dropdown-input-field qa-dropdown-input-field", placeholder: placeholder, autocomplete: 'off'`. + +> Classes starting with `qa-` are used for testing purposes only, and by defining such classes in the elements we add **testability** in the application. + +> When defining a class like `qa-labels-block`, it is transformed into `:labels_block` for usage in the Page Objects. So, `qa-edit-link-labels` is tranformed into `:edit_link_labels`, `qa-dropdown-menu-labels` is transformed into `:dropdown_menu_labels`, and `qa-dropdown-input-field` is transformed into `:dropdown_input_field`. Also, we use a [sanity test](https://gitlab.com/gitlab-org/gitlab-ce/tree/master/qa/qa/page#how-did-we-solve-fragile-tests-problem) to check that defined elements have their respective `qa-` selectors in the specified views. + +> We did not define the `qa-labels-block` class in the `app/views/shared/issuable/_sidebar.html.haml` file because it was already there to be used. + +#### Updates in the `QA::Page::Base` class + +The last thing that we have to do is to update `QA::Page::Base` class to add the `send_keys_to_element` method on it. + +Add the following snippet of code somewhere where class methods are defined: + +```ruby +def send_keys_to_element(name, keys) + find_element(name).send_keys(keys) +end +``` + +This method receives an element (`name`) and the `keys` that it will send to that element, and the keys are an array that can receive strings, or "special" keys, like `:enter`. + +As you might remember, in the Issue Page Object we call this method like this: `send_keys_to_element(:dropdown_input_field, [label, :enter])`. + +___ + +With that, you should be able to start writing end-to-end tests yourself. *Congratulations!* + +[Page Objects]: page_objects.md +[Resources]: resources.md diff --git a/doc/development/testing_guide/end_to_end/resources.md b/doc/development/testing_guide/end_to_end/resources.md new file mode 100644 index 00000000000..1e32db4f633 --- /dev/null +++ b/doc/development/testing_guide/end_to_end/resources.md @@ -0,0 +1,392 @@ +# Resource class in GitLab QA + +Resources are primarily created using Browser UI steps, but can also +be created via the API or the CLI. + +## How to properly implement a resource class? + +All resource classes should inherit from `Resource::Base`. + +There is only one mandatory method to implement to define a resource class. +This is the `#fabricate!` method, which is used to build the resource via the +browser UI. Note that you should only use [Page objects](page_objects.md) to +interact with a Web page in this method. + +Here is an imaginary example: + +```ruby +module QA + module Resource + class Shirt < Base + attr_accessor :name + + def fabricate! + Page::Dashboard::Index.perform do |dashboard_index| + dashboard_index.go_to_new_shirt + end + + Page::Shirt::New.perform do |shirt_new| + shirt_new.set_name(name) + shirt_new.create_shirt! + end + end + end + end +end +``` + +### Define API implementation + +A resource class may also implement the three following methods to be able to +create the resource via the public GitLab API: + +- `#api_get_path`: The `GET` path to fetch an existing resource. +- `#api_post_path`: The `POST` path to create a new resource. +- `#api_post_body`: The `POST` body (as a Ruby hash) to create a new resource. + +Let's take the `Shirt` resource class, and add these three API methods: + +```ruby +module QA + module Resource + class Shirt < Base + attr_accessor :name + + def fabricate! + # ... same as before + end + + def api_get_path + "/shirt/#{name}" + end + + def api_post_path + "/shirts" + end + + def api_post_body + { + name: name + } + end + end + end +end +``` + +The `Project` resource is a good real example of Browser +UI and API implementations. + +#### Resource attributes + +A resource may need another resource to exist first. For instance, a project +needs a group to be created in. + +To define a resource attribute, you can use the `attribute` method with a +block using the other resource class to fabricate the resource. + +That will allow access to the other resource from your resource object's +methods. You would usually use it in `#fabricate!`, `#api_get_path`, +`#api_post_path`, `#api_post_body`. + +Let's take the `Shirt` resource class, and add a `project` attribute to it: + +```ruby +module QA + module Resource + class Shirt < Base + attr_accessor :name + + attribute :project do + Project.fabricate! do |resource| + resource.name = 'project-to-create-a-shirt' + end + end + + def fabricate! + project.visit! + + Page::Project::Show.perform do |project_show| + project_show.go_to_new_shirt + end + + Page::Shirt::New.perform do |shirt_new| + shirt_new.set_name(name) + shirt_new.create_shirt! + end + end + + def api_get_path + "/project/#{project.path}/shirt/#{name}" + end + + def api_post_path + "/project/#{project.path}/shirts" + end + + def api_post_body + { + name: name + } + end + end + end +end +``` + +**Note that all the attributes are lazily constructed. This means if you want +a specific attribute to be fabricated first, you'll need to call the +attribute method first even if you're not using it.** + +#### Product data attributes + +Once created, you may want to populate a resource with attributes that can be +found in the Web page, or in the API response. +For instance, once you create a project, you may want to store its repository +SSH URL as an attribute. + +Again we could use the `attribute` method with a block, using a page object +to retrieve the data on the page. + +Let's take the `Shirt` resource class, and define a `:brand` attribute: + +```ruby +module QA + module Resource + class Shirt < Base + attr_accessor :name + + attribute :project do + Project.fabricate! do |resource| + resource.name = 'project-to-create-a-shirt' + end + end + + # Attribute populated from the Browser UI (using the block) + attribute :brand do + Page::Shirt::Show.perform do |shirt_show| + shirt_show.fetch_brand_from_page + end + end + + # ... same as before + end + end +end +``` + +**Note again that all the attributes are lazily constructed. This means if +you call `shirt.brand` after moving to the other page, it'll not properly +retrieve the data because we're no longer on the expected page.** + +Consider this: + +```ruby +shirt = + QA::Resource::Shirt.fabricate! do |resource| + resource.name = "GitLab QA" + end + +shirt.project.visit! + +shirt.brand # => FAIL! +``` + +The above example will fail because now we're on the project page, trying to +construct the brand data from the shirt page, however we moved to the project +page already. There are two ways to solve this, one is that we could try to +retrieve the brand before visiting the project again: + +```ruby +shirt = + QA::Resource::Shirt.fabricate! do |resource| + resource.name = "GitLab QA" + end + +shirt.brand # => OK! + +shirt.project.visit! + +shirt.brand # => OK! +``` + +The attribute will be stored in the instance therefore all the following calls +will be fine, using the data previously constructed. If we think that this +might be too brittle, we could eagerly construct the data right before +ending fabrication: + +```ruby +module QA + module Resource + class Shirt < Base + # ... same as before + + def fabricate! + project.visit! + + Page::Project::Show.perform do |project_show| + project_show.go_to_new_shirt + end + + Page::Shirt::New.perform do |shirt_new| + shirt_new.set_name(name) + shirt_new.create_shirt! + end + + populate(:brand) # Eagerly construct the data + end + end + end +end +``` + +The `populate` method will iterate through its arguments and call each +attribute respectively. Here `populate(:brand)` has the same effect as +just `brand`. Using the populate method makes the intention clearer. + +With this, it will make sure we construct the data right after we create the +shirt. The drawback is that this will always construct the data when the +resource is fabricated even if we don't need to use the data. + +Alternatively, we could just make sure we're on the right page before +constructing the brand data: + +```ruby +module QA + module Resource + class Shirt < Base + attr_accessor :name + + attribute :project do + Project.fabricate! do |resource| + resource.name = 'project-to-create-a-shirt' + end + end + + # Attribute populated from the Browser UI (using the block) + attribute :brand do + back_url = current_url + visit! + + Page::Shirt::Show.perform do |shirt_show| + shirt_show.fetch_brand_from_page + end + + visit(back_url) + end + + # ... same as before + end + end +end +``` + +This will make sure it's on the shirt page before constructing brand, and +move back to the previous page to avoid breaking the state. + +#### Define an attribute based on an API response + +Sometimes, you want to define a resource attribute based on the API response +from its `GET` or `POST` request. For instance, if the creation of a shirt via +the API returns + +```ruby +{ + brand: 'a-brand-new-brand', + style: 't-shirt', + materials: [[:cotton, 80], [:polyamide, 20]] +} +``` + +you may want to store `style` as-is in the resource, and fetch the first value +of the first `materials` item in a `main_fabric` attribute. + +Let's take the `Shirt` resource class, and define a `:style` and a +`:main_fabric` attributes: + +```ruby +module QA + module Resource + class Shirt < Base + # ... same as before + + # @style from the instance if present, + # or fetched from the API response if present, + # or a QA::Resource::Base::NoValueError is raised otherwise + attribute :style + + # If @main_fabric is not present, + # and if the API does not contain this field, this block will be + # used to construct the value based on the API response, and + # store the result in @main_fabric + attribute :main_fabric do + api_response.&dig(:materials, 0, 0) + end + + # ... same as before + end + end +end +``` + +**Notes on attributes precedence:** + +- resource instance variables have the highest precedence +- attributes from the API response take precedence over attributes from the + block (usually from Browser UI) +- attributes without a value will raise a `QA::Resource::Base::NoValueError` error + +## Creating resources in your tests + +To create a resource in your tests, you can call the `.fabricate!` method on +the resource class. +Note that if the resource class supports API fabrication, this will use this +fabrication by default. + +Here is an example that will use the API fabrication method under the hood +since it's supported by the `Shirt` resource class: + +```ruby +my_shirt = Resource::Shirt.fabricate! do |shirt| + shirt.name = 'my-shirt' +end + +expect(page).to have_text(my_shirt.name) # => "my-shirt" from the resource's instance variable +expect(page).to have_text(my_shirt.brand) # => "a-brand-new-brand" from the API response +expect(page).to have_text(my_shirt.style) # => "t-shirt" from the API response +expect(page).to have_text(my_shirt.main_fabric) # => "cotton" from the API response via the block +``` + +If you explicitly want to use the Browser UI fabrication method, you can call +the `.fabricate_via_browser_ui!` method instead: + +```ruby +my_shirt = Resource::Shirt.fabricate_via_browser_ui! do |shirt| + shirt.name = 'my-shirt' +end + +expect(page).to have_text(my_shirt.name) # => "my-shirt" from the resource's instance variable +expect(page).to have_text(my_shirt.brand) # => the brand name fetched from the `Page::Shirt::Show` page via the block +expect(page).to have_text(my_shirt.style) # => QA::Resource::Base::NoValueError will be raised because no API response nor a block is provided +expect(page).to have_text(my_shirt.main_fabric) # => QA::Resource::Base::NoValueError will be raised because no API response and the block didn't provide a value (because it's also based on the API response) +``` + +You can also explicitly use the API fabrication method, by calling the +`.fabricate_via_api!` method: + +```ruby +my_shirt = Resource::Shirt.fabricate_via_api! do |shirt| + shirt.name = 'my-shirt' +end +``` + +In this case, the result will be similar to calling +`Resource::Shirt.fabricate!`. + +## Where to ask for help? + +If you need more information, ask for help on `#quality` channel on Slack +(internal, GitLab Team only). + +If you are not a Team Member, and you still need help to contribute, please +open an issue in GitLab CE issue tracker with the `~QA` label. diff --git a/doc/development/testing_guide/end_to_end/style_guide.md b/doc/development/testing_guide/end_to_end/style_guide.md new file mode 100644 index 00000000000..0272e1810f2 --- /dev/null +++ b/doc/development/testing_guide/end_to_end/style_guide.md @@ -0,0 +1,99 @@ +# Style guide for writing end-to-end tests + +This document describes the conventions used at GitLab for writing End-to-end (E2E) tests using the GitLab QA project. + +## `click_` versus `go_to_` + +### When to use `click_`? + +When clicking in a single link to navigate, use `click_`. + +E.g.: + +```ruby +def click_ci_cd_pipelines + within_sidebar do + click_element :link_pipelines + end +end +``` + +From a testing perspective, if we want to check that clicking a link, or a button (a single interaction) is working as intended, we would want the test to read as: + +- Click a certain element +- Verify the action took place + +### When to use `go_to_`? + +When interacting with multiple elements to go to a page, use `go_to_`. + +E.g.: + +```ruby +def go_to_operations_environments + hover_operations do + within_submenu do + click_element(:operations_environments_link) + end + end +end +``` + +`go_to_` fits the definition of interacting with multiple elements very well given it's more of a meta-navigation action that includes multiple interactions. + +Notice that in the above example, before clicking the `:operations_environments_link`, another element is hovered over. + +> We can create these methods as helpers to abstract multi-step navigation. + +### Element naming convention + +When adding new elements to a page, it's important that we have a uniform element naming convention. + +We follow a simple formula roughly based on hungarian notation. + +*Formula*: `element :<descriptor>_<type>` + +- `descriptor`: The natural-language description of what the element is. On the login page, this could be `username`, or `password`. +- `type`: A physical control on the page that can be seen by a user. + - `_button` + - `_link` + - `_tab` + - `_dropdown` + - `_field` + - `_checkbox` + - `_radio` + - `_content` + +*Note: This list is a work in progress. This list will eventually be the end-all enumeration of all available types. + I.e., any element that does not end with something in this list is bad form.* + +#### Examples + +**Good** + +```ruby +view '...' do + element :edit_button + element :notes_tab + element :squash_checkbox + element :username_field + element :issue_title_content +end +``` + +**Bad** + +```ruby +view '...' do + # `_confirmation` should be `_field`. what sort of confirmation? a checkbox confirmation? no real way to disambiguate. + # an appropriate replacement would be `element :password_confirmation_field` + element :password_confirmation + + # `clone_options` is too vague. If it's a dropdown menu, it should be `clone_dropdown`. + # If it's a checkbox, it should be `clone_checkbox` + element :clone_options + + # how is this url being displayed? is it a textbox? a simple span? + element :ssh_clone_url +end +``` diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md index 58d6f08954d..4c9d1684c00 100644 --- a/doc/development/testing_guide/frontend_testing.md +++ b/doc/development/testing_guide/frontend_testing.md @@ -67,9 +67,6 @@ Remember that the performance of each test depends on the environment. GitLab uses the [Karma][karma] test runner with [Jasmine] as its test framework for our JavaScript unit and integration tests. -We generate HTML and JSON fixtures from backend views and controllers -using RSpec (see `spec/javascripts/fixtures/*.rb` for examples). -Fixtures are served during testing by the [jasmine-jquery][jasmine-jquery] plugin. JavaScript tests live in `spec/javascripts/`, matching the folder structure of `app/assets/javascripts/`: `app/assets/javascripts/behaviors/autosize.js` @@ -421,7 +418,7 @@ See this [section][vue-test]. For running the frontend tests, you need the following commands: -- `rake karma:fixtures` (re-)generates fixtures. +- `rake karma:fixtures` (re-)generates [fixtures](#frontend-test-fixtures). - `yarn test` executes the tests. As long as the fixtures don't change, `yarn test` is sufficient (and saves you some time). @@ -469,6 +466,48 @@ yarn karma -f 'spec/javascripts/ide/**/file_spec.js' Information on setting up and running RSpec integration tests with [Capybara] can be found in the [Testing Best Practices](best_practices.md). +## Frontend test fixtures + +Code that is added to HAML templates (in `app/views/`) or makes Ajax requests to the backend has tests that require HTML or JSON from the backend. +Fixtures for these tests are located at: + +- `spec/javascripts/fixtures/`, for running tests in CE. +- `ee/spec/javascripts/fixtures/`, for running tests in EE. + +Fixture files in: + +- The Karma test suite are served by [jasmine-jquery](https://github.com/velesin/jasmine-jquery). +- Jest use `spec/frontend/helpers/fixtures.js`. + +The following are examples of tests that work for both Karma and Jest: + +```javascript +it('makes a request', () => { + const responseBody = getJSONFixture('some/fixture.json'); // loads spec/javascripts/fixtures/some/fixture.json + axiosMock.onGet(endpoint).reply(200, responseBody); + + myButton.click(); + + // ... +}); + +it('uses some HTML element', () => { + loadFixtures('some/page.html'); // loads spec/javascripts/fixtures/some/page.html and adds it to the DOM + + const element = document.getElementById('#my-id'); + + // ... +}); +``` + +HTML and JSON fixtures are generated from backend views and controllers using RSpec (see `spec/javascripts/fixtures/*.rb`). + +For each fixture, the content of the `response` variable is stored in the output file. +This variable gets automagically set if the test is marked as `type: :request` or `type: :controller`. +Fixtures are regenerated using the `bin/rake karma:fixtures` command but you can also generate them individually, +for example `bin/rspec spec/javascripts/fixtures/merge_requests.rb`. +When creating a new fixture, it often makes sense to take a look at the corresponding tests for the endpoint in `(ee/)spec/controllers/` or `(ee/)spec/requests/`. + ## Gotchas ### RSpec errors due to JavaScript @@ -501,7 +540,6 @@ end ``` [jasmine-focus]: https://jasmine.github.io/2.5/focused_specs.html -[jasmine-jquery]: https://github.com/velesin/jasmine-jquery [karma]: http://karma-runner.github.io/ [vue-test]: https://docs.gitlab.com/ce/development/fe_guide/vue.html#testing-vue-components [rspec]: https://github.com/rspec/rspec-rails#feature-specs diff --git a/doc/development/testing_guide/index.md b/doc/development/testing_guide/index.md index ecad9ba48a3..93ee2a6371a 100644 --- a/doc/development/testing_guide/index.md +++ b/doc/development/testing_guide/index.md @@ -71,7 +71,7 @@ Everything you should know about how to test Rake tasks. --- -## [End-to-end tests](end_to_end_tests.md) +## [End-to-end tests](end_to_end/index.md) Everything you should know about how to run end-to-end tests using [GitLab QA][gitlab-qa] testing framework. diff --git a/doc/development/testing_guide/review_apps.md b/doc/development/testing_guide/review_apps.md index f34793c11f4..b5fde92a23d 100644 --- a/doc/development/testing_guide/review_apps.md +++ b/doc/development/testing_guide/review_apps.md @@ -113,7 +113,7 @@ You can also manually start the `review-qa-all`: it runs the full QA suite. On every [pipeline][gitlab-pipeline] in the `qa` stage, the `review-performance` job is automatically started: this job does basic browser performance testing using a -[Sitespeed.io Container](https://docs.gitlab.com/ee/user/project/merge_requests/browser_performance_testing.html). +[Sitespeed.io Container](../../user/project/merge_requests/browser_performance_testing.md). ## How to: @@ -203,7 +203,7 @@ find a way to limit it to only us.** [automated_cleanup.rb]: https://gitlab.com/gitlab-org/gitlab-ee/blob/master/scripts/review_apps/automated_cleanup.rb [Auto-DevOps.gitlab-ci.yml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml [gitlab-ci-yml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab-ci.yml -[gitlab-k8s-integration]: https://docs.gitlab.com/ee/user/project/clusters/index.html +[gitlab-k8s-integration]: ../../user/project/clusters/index.md [password-bug]: https://gitlab.com/gitlab-org/gitlab-ce/issues/53621 --- diff --git a/doc/development/testing_guide/smoke.md b/doc/development/testing_guide/smoke.md index 30d861d7d68..c9d3238fbe9 100644 --- a/doc/development/testing_guide/smoke.md +++ b/doc/development/testing_guide/smoke.md @@ -17,7 +17,7 @@ Currently, our suite consists of this basic functionality coverage: Smoke tests have the `:smoke` RSpec metadata. -See [End-to-end Testing](./end_to_end_tests.md) for more details about +See [End-to-end Testing](end_to_end/index.md) for more details about end-to-end tests. --- diff --git a/doc/development/testing_guide/testing_levels.md b/doc/development/testing_guide/testing_levels.md index 1fa6e38ea5a..e1ce4d3b7d1 100644 --- a/doc/development/testing_guide/testing_levels.md +++ b/doc/development/testing_guide/testing_levels.md @@ -4,12 +4,14 @@ _This diagram demonstrates the relative priority of each test type we use. `e2e` stands for end-to-end._ -As of 2019-04-16, we have the following distribution of tests per level: +As of 2019-05-01, we have the following distribution of tests per level: -- 67 black-box tests at the system level (aka end-to-end or QA tests) in CE, 98 in EE. This represents 0.3% of all the CE tests (0.3% in EE). -- 5,457 white-box tests at the system level (aka system or feature tests) in CE, 6,585 in EE. This represents 24.6% of all the CE tests (20.3% in EE). -- 8,298 integration tests in CE, 10,633 in EE: 0.3% of all the CE tests (0.3% in EE). This represents 37.2% of all the CE tests (32.8% in EE). -- 8,403 unit tests in CE, 15,090 in EE: 0.3% of all the CE tests (0.3% in EE). This represents 37.8% of all the CE tests (46.6% in EE). +| Test level | Community Edition | Enterprise Edition | Community + Enterprise Edition | +| --------- | ---------- | -------------- | ----- | +| Black-box tests at the system level (aka end-to-end or QA tests) | 68 (0.14%) | 31 (0.2%) | 99 (0.17%) | +| White-box tests at the system level (aka system or feature tests) | 5,471 (11.9%) | 969 (7.4%) | 6440 (10.9%) | +| Integration tests | 8,333 (18.2%) | 2,244 (17.2%) | 10,577 (17.9%) | +| Unit tests | 32,031 (69.7%) | 9,778 (75.1%) | 41,809 (71%) | ## Unit tests @@ -176,7 +178,7 @@ Every new feature should come with a [test plan]. | ---------- | -------------- | ----- | | `qa/qa/specs/features/` | [Capybara] + [RSpec] + Custom QA framework | Tests should be placed under their corresponding [Product category] | -> See [end-to-end tests](end_to_end_tests.md) for more information. +> See [end-to-end tests](end_to_end/index.md) for more information. Note that `qa/spec` contains unit tests of the QA framework itself, not to be confused with the application's [unit tests](#unit-tests) or diff --git a/doc/development/understanding_explain_plans.md b/doc/development/understanding_explain_plans.md index 01a0044f096..2ef8b3148e4 100644 --- a/doc/development/understanding_explain_plans.md +++ b/doc/development/understanding_explain_plans.md @@ -80,8 +80,9 @@ Planning time: 2.861 ms Execution time: 3428.596 ms ``` -For more information, refer to the official [EXPLAIN -documentation](https://www.postgresql.org/docs/current/static/sql-explain.html). +For more information, refer to the official +[`EXPLAIN` documentation](https://www.postgresql.org/docs/current/sql-explain.html) +and [using `EXPLAIN` guide](https://www.postgresql.org/docs/current/using-explain.html). ## Nodes @@ -653,6 +654,35 @@ and related tools such as: - <https://explain.depesz.com/> - <http://tatiyants.com/postgres-query-plan-visualization/> +## Producing query plans + +There are a few ways to get the output of a query plan. Of course you +can directly run the `EXPLAIN` query in the `psql` console, or you can +follow one of the other options below. + +### Rails console + +Using the [`activerecord-explain-analyze`](https://github.com/6/activerecord-explain-analyze) +you can directly generate the query plan from the Rails console: + +```ruby +pry(main)> require 'activerecord-explain-analyze' +=> true +pry(main)> Project.where('build_timeout > ?', 3600).explain(analyze: true) + Project Load (1.9ms) SELECT "projects".* FROM "projects" WHERE (build_timeout > 3600) + ↳ (pry):12 +=> EXPLAIN for: SELECT "projects".* FROM "projects" WHERE (build_timeout > 3600) +Seq Scan on public.projects (cost=0.00..2.17 rows=1 width=742) (actual time=0.040..0.041 rows=0 loops=1) + Output: id, name, path, description, created_at, updated_at, creator_id, namespace_id, ... + Filter: (projects.build_timeout > 3600) + Rows Removed by Filter: 14 + Buffers: shared hit=2 +Planning time: 0.411 ms +Execution time: 0.113 ms +``` + +### Chatops + GitLab employees can also use our chatops solution, available in Slack using the `/chatops` slash command. You can use chatops to get a query plan by running the following: @@ -674,3 +704,9 @@ For more information about the available options, run: ``` /chatops run explain --help ``` + +## Further reading + +A more extensive guide on understanding query plans can be found in +the [presentation](https://www.dalibo.org/_media/understanding_explain.pdf) +from [Dalibo.org](https://www.dalibo.org/en/). diff --git a/doc/git_hooks/git_hooks.md b/doc/git_hooks/git_hooks.md index 9b8ad1578a0..b251e58410a 100644 --- a/doc/git_hooks/git_hooks.md +++ b/doc/git_hooks/git_hooks.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/push_rules/push_rules.html' +redirect_to: '../push_rules/push_rules.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/push_rules/push_rules.html) +This document was moved to [another location](../push_rules/push_rules.md) diff --git a/doc/gitlab-basics/README.md b/doc/gitlab-basics/README.md index 28ba20bec09..00ac6d6b650 100644 --- a/doc/gitlab-basics/README.md +++ b/doc/gitlab-basics/README.md @@ -15,7 +15,7 @@ This documentation is split into the following groups: The following are guides to basic GitLab functionality: -- [Create and add your SSH Keys](create-your-ssh-keys.md), for enabling Git over SSH. +- [Create and add your SSH public key](create-your-ssh-keys.md), for enabling Git over SSH. - [Create a project](create-project.md), to start using GitLab. - [Create a group](../user/group/index.md#create-a-new-group), to combine and administer projects together. - [Create a branch](create-branch.md), to make changes to files stored in a project's repository. diff --git a/doc/gitlab-basics/create-project.md b/doc/gitlab-basics/create-project.md index 785e2ffb650..a9ae4fb23f9 100644 --- a/doc/gitlab-basics/create-project.md +++ b/doc/gitlab-basics/create-project.md @@ -16,7 +16,7 @@ To create a project in GitLab: - [Import a project](../user/project/import/index.md) from a different repository, if enabled on your GitLab instance. Contact your GitLab admin if this is unavailable. - - Run [CI/CD pipelines for external repositories](https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/index.html). **[PREMIUM]** + - Run [CI/CD pipelines for external repositories](../ci/ci_cd_for_external_repos/index.md). **[PREMIUM]** ## Blank projects diff --git a/doc/gitlab-basics/create-your-ssh-keys.md b/doc/gitlab-basics/create-your-ssh-keys.md index 8fecdc6948e..aac73d4c9c5 100644 --- a/doc/gitlab-basics/create-your-ssh-keys.md +++ b/doc/gitlab-basics/create-your-ssh-keys.md @@ -1,22 +1,22 @@ -# How to create your SSH keys +# Create and add your SSH public key -This topic describes how to create SSH keys. You do this to use Git over SSH instead of Git over HTTP. +This topic describes how to: -## Creating your SSH keys +- Create an SSH key pair to use with GitLab. +- Add the SSH public key to your GitLab account. -1. Go to your [command line](start-using-git.md) and follow the [instructions](../ssh/README.md) to generate your SSH key pair. -1. Log in to GitLab. -1. In the upper-right corner, click your avatar and select **Settings**. -1. On the **User Settings** menu, select **SSH keys**. -1. Paste the **public** key generated in the first step in the **Key** - text field. -1. Optionally, give it a descriptive title so that you can recognize it in the - event you add multiple keys. -1. Finally, click the **Add key** button to add it to GitLab. You will be able to see - its fingerprint, title, and creation date. +You do this to use [Git over SSH instead of Git over HTTP](https://git-scm.com/book/en/v2/Git-on-the-Server-The-Protocols). -  +## Creating your SSH key pair + +1. Go to your [command line](start-using-git.md). +1. Follow the [instructions](../ssh/README.md#generating-a-new-ssh-key-pair) to generate your SSH key pair. + +## Adding your SSH public key to GitLab + +To add the SSH public key to GitLab, +see [Adding an SSH key to your GitLab account](../ssh/README.md#adding-an-ssh-key-to-your-gitlab-account). NOTE: **Note:** Once you add a key, you cannot edit it. If the paste -didn't work, you need to remove the offending key and re-add it. +[didn't work](../ssh/README.md#testing-that-everything-is-set-up-correctly), you need to remove the key and re-add it. diff --git a/doc/gitlab-basics/img/profile_settings_ssh_keys_single_key.png b/doc/gitlab-basics/img/profile_settings_ssh_keys_single_key.png Binary files differdeleted file mode 100644 index 8014f1d5301..00000000000 --- a/doc/gitlab-basics/img/profile_settings_ssh_keys_single_key.png +++ /dev/null diff --git a/doc/gitlab-geo/configuration_source.md b/doc/gitlab-geo/configuration_source.md index f1aab86fadc..b46a2caea4a 100644 --- a/doc/gitlab-geo/configuration_source.md +++ b/doc/gitlab-geo/configuration_source.md @@ -1,5 +1,5 @@ --- -redirect_to: '../administration/geo/replication/configuration_source.md' +redirect_to: '../administration/geo/replication/configuration.md' --- -This document was moved to [another location](../administration/geo/replication/configuration_source.md). +This document was moved to [another location](../administration/geo/replication/configuration.md). diff --git a/doc/gitlab-geo/database_source.md b/doc/gitlab-geo/database_source.md index 3392d0f02c0..b4156dc4ec6 100644 --- a/doc/gitlab-geo/database_source.md +++ b/doc/gitlab-geo/database_source.md @@ -1,5 +1,5 @@ --- -redirect_to: '../administration/geo/replication/database_source.md' +redirect_to: '../administration/geo/replication/database.md' --- -This document was moved to [another location](../administration/geo/replication/database_source.md). +This document was moved to [another location](../administration/geo/replication/database.md). diff --git a/doc/install/README.md b/doc/install/README.md index 53778f7f0d3..9cc21412898 100644 --- a/doc/install/README.md +++ b/doc/install/README.md @@ -1,6 +1,7 @@ --- comments: false description: Read through the GitLab installation methods. +type: index --- # Installation **[CORE ONLY]** @@ -81,7 +82,7 @@ the above methods, provided the cloud provider supports it. - [Install on AWS](aws/index.md): Install Omnibus GitLab on AWS using the community AMIs that GitLab provides. - [Install GitLab on Google Cloud Platform](google_cloud_platform/index.md): Install Omnibus GitLab on a VM in GCP. - [Install GitLab on Azure](azure/index.md): Install Omnibus GitLab from Azure Marketplace. -- [Install GitLab on OpenShift](openshift_and_gitlab/index.md): Install GitLab using the Docker image on OpenShift. +- [Install GitLab on OpenShift](https://docs.gitlab.com/charts/installation/cloud/openshift.html): Install GitLab on OpenShift by using GitLab's Helm charts. - [Install GitLab on DC/OS](https://mesosphere.com/blog/gitlab-dcos/): Install GitLab on Mesosphere DC/OS via the [GitLab-Mesosphere integration](https://about.gitlab.com/2016/09/16/announcing-gitlab-and-mesosphere/). - [Install GitLab on DigitalOcean](https://about.gitlab.com/2016/04/27/getting-started-with-gitlab-and-digitalocean/): Install Omnibus GitLab on DigitalOcean. - _Testing only!_ [DigitalOcean and Docker Machine](digitaloceandocker.md): diff --git a/doc/install/aws/index.md b/doc/install/aws/index.md index 0000e03f1d7..73eaf758923 100644 --- a/doc/install/aws/index.md +++ b/doc/install/aws/index.md @@ -1,3 +1,7 @@ +--- +type: howto +--- + # Installing GitLab HA on Amazon Web Services (AWS) This page offers a walkthrough of a common HA (Highly Available) configuration @@ -383,7 +387,7 @@ after the instance is created. CAUTION: **Caution:** We **do not** recommend using the AWS Elastic File System (EFS), as it can result -in [significantly degraded performance](../../administration/high_availability/nfs.html#avoid-using-awss-elastic-file-system-efs). +in [significantly degraded performance](../../administration/high_availability/nfs.md#avoid-using-awss-elastic-file-system-efs). ### Configure security group @@ -649,12 +653,24 @@ Have a read through these other resources and feel free to [open an issue](https://gitlab.com/gitlab-org/gitlab-ce/issues/new) to request additional material: -- [GitLab High Availability](https://docs.gitlab.com/ee/administration/high_availability/): +- [GitLab High Availability](../../administration/high_availability/README.md): GitLab supports several different types of clustering and high-availability. -- [Geo replication](https://docs.gitlab.com/ee/administration/geo/replication/): +- [Geo replication](../../administration/geo/replication/index.md): Geo is the solution for widely distributed development teams. - [Omnibus GitLab](https://docs.gitlab.com/omnibus/) - Everything you need to know about administering your GitLab instance. -- [Upload a license](https://docs.gitlab.com/ee/user/admin_area/license.html): +- [Upload a license](../../user/admin_area/license.md): Activate all GitLab Enterprise Edition functionality with a license. - [Pricing](https://about.gitlab.com/pricing): Pricing for the different tiers. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/install/azure/index.md b/doc/install/azure/index.md index fa5be1d30f9..b1f79893baf 100644 --- a/doc/install/azure/index.md +++ b/doc/install/azure/index.md @@ -1,14 +1,10 @@ --- -description: 'Learn how to spin up a -pre-configured GitLab VM on Microsoft Azure and have your very own private GitLab instance up and running in around 30 minutes.' +description: 'Learn how to spin up a pre-configured GitLab VM on Microsoft Azure.' +type: howto --- # Install GitLab on Microsoft Azure -> _This article was originally written by Dave Wentzel and [published on the GitLab Blog][Original-Blog-Post]._ -> -> _Ported to the GitLab documentation and updated on 2017-08-24 by [Ian Scorer](https://gitlab.com/iscorer)._ - Azure is Microsoft's business cloud and GitLab is a pre-configured offering on the Azure Marketplace. Hopefully, you aren't surprised to hear that Microsoft and Azure have embraced open source software like Ubuntu, Red Hat Enterprise Linux, and of course - GitLab! This means that you can spin up a @@ -444,3 +440,15 @@ Check out our other [Technical Articles][GitLab-Technical-Articles] or browse th [SSH]: https://en.wikipedia.org/wiki/Secure_Shell [PuTTY]: http://www.putty.org/ [Using-SSH-In-Putty]: https://mediatemple.net/community/products/dv/204404604/using-ssh-in-putty- + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/install/database_mysql.md b/doc/install/database_mysql.md index e89846107b6..cbb3b766b4e 100644 --- a/doc/install/database_mysql.md +++ b/doc/install/database_mysql.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Database MySQL NOTE: **Note:** @@ -301,3 +305,15 @@ Details can be found in the [PostgreSQL][postgres-text-type] and [postgres-text-type]: http://www.postgresql.org/docs/9.2/static/datatype-character.html [ce-38152]: https://gitlab.com/gitlab-org/gitlab-ce/issues/38152 + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/install/digitaloceandocker.md b/doc/install/digitaloceandocker.md index d67695d75b4..63bb941ad47 100644 --- a/doc/install/digitaloceandocker.md +++ b/doc/install/digitaloceandocker.md @@ -1,8 +1,11 @@ +--- +type: howto +--- + # Digital Ocean and Docker Machine test environment -CAUTION: **Caution:** -This guide is for quickly testing different versions of GitLab and not recommended for ease of -future upgrades or keeping the data you create. +This guide is for quickly testing different versions of GitLab and not +recommended for ease of future upgrades or keeping the data you create. ## Initial setup @@ -28,7 +31,7 @@ locally on either macOS or Linux. NOTE: **Note:** The rest of the steps are identical for macOS and Linux. -### Create new docker host +## Create new docker host 1. Login to Digital Ocean. 1. Generate a new API token at <https://cloud.digitalocean.com/settings/api/tokens>. @@ -60,9 +63,9 @@ The rest of the steps are identical for macOS and Linux. Resource: <https://docs.docker.com/machine/drivers/digital-ocean/>. -### Creating GitLab test instance +## Creating GitLab test instance -#### Connect your shell to the new machine +### Connect your shell to the new machine In this example we'll create a GitLab EE 8.10.8 instance. @@ -74,7 +77,7 @@ eval "$(docker-machine env gitlab-test-env-do)" You can add this to your `~/.bash_profile` file to ensure the `docker` client uses the `gitlab-test-env-do` docker host -#### Create new GitLab container +### Create new GitLab container - HTTP port: `8888` - SSH port: `2222` @@ -83,7 +86,7 @@ You can add this to your `~/.bash_profile` file to ensure the `docker` client us - Container name: `gitlab-test-8.10` - GitLab version: **EE** `8.10.8-ee.0` -##### Set up container settings +#### Set up container settings ```sh export SSH_PORT=2222 @@ -92,7 +95,7 @@ export VERSION=8.10.8-ee.0 export NAME=gitlab-test-8.10 ``` -##### Create container +#### Create container ```sh docker run --detach \ @@ -103,9 +106,9 @@ docker run --detach \ gitlab/gitlab-ee:$VERSION ``` -#### Connect to the GitLab container +### Connect to the GitLab container -##### Retrieve the docker host IP +#### Retrieve the docker host IP ```sh docker-machine ip gitlab-test-env-do @@ -114,7 +117,7 @@ docker-machine ip gitlab-test-env-do Browse to: <http://192.168.151.134:8888/>. -##### Execute interactive shell/edit configuration +#### Execute interactive shell/edit configuration ```sh docker exec -it $NAME /bin/bash @@ -126,8 +129,20 @@ root@192:/# vi /etc/gitlab/gitlab.rb root@192:/# gitlab-ctl reconfigure ``` -#### Resources +### Resources - <https://docs.gitlab.com/omnibus/docker/>. - <https://docs.docker.com/machine/get-started/>. - <https://docs.docker.com/machine/reference/ip/>. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/install/docker.md b/doc/install/docker.md index 4568a949b0a..06da65189ba 100644 --- a/doc/install/docker.md +++ b/doc/install/docker.md @@ -1,3 +1,7 @@ +--- +type: index +--- + # Install GitLab with Docker [Docker](https://www.docker.com) and container technology have been revolutionizing the software world for the past few years. They combine the performance and efficiency of native execution with the abstraction, security, and immutability of virtualization. @@ -16,4 +20,4 @@ A [complete usage guide](https://docs.gitlab.com/omnibus/docker/) to these image ## Cloud native images -GitLab is also working towards a [cloud native set of containers](https://gitlab.com/charts/helm.gitlab.io#docker-container-images), with a single image for each component service. We intend for these images to eventually replace the [Omnibus GitLab based images](#omnibus-gitlab-based-images). +GitLab is also working towards a [cloud native set of containers](https://docs.gitlab.com/charts/), with a single image for each component service. We intend for these images to eventually replace the [Omnibus GitLab based images](#omnibus-gitlab-based-images). diff --git a/doc/install/google-protobuf.md b/doc/install/google-protobuf.md index a531b4519b3..434817c48cb 100644 --- a/doc/install/google-protobuf.md +++ b/doc/install/google-protobuf.md @@ -1,26 +1,5 @@ -# Installing a locally compiled google-protobuf gem +--- +redirect_to: 'installation.md#google-protobuf-loaderror-libx86_64-linux-gnulibcso6-version-glibc_214-not-found' +--- -First we must find the exact version of google-protobuf that your -GitLab installation requires. - - cd /home/git/gitlab - - # Only one of the following two commands will print something. It - # will look like: * google-protobuf (3.2.0) - bundle list | grep google-protobuf - bundle check | grep google-protobuf - -Below we use `3.2.0` as an example. Replace it with the version number -you found above. - - cd /home/git/gitlab - sudo -u git -H gem install google-protobuf --version 3.2.0 --platform ruby - -Finally, you can test whether google-protobuf loads correctly. The -following should print 'OK'. - - sudo -u git -H bundle exec ruby -rgoogle/protobuf -e 'puts :OK' - -If the `gem install` command fails you may need to install developer -tools. On Debian: `apt-get install build-essential libgmp-dev`, on -Centos/RedHat `yum groupinstall 'Development Tools'`. +This document was moved to [another location](installation.md#google-protobuf-loaderror-libx86_64-linux-gnulibcso6-version-glibc_214-not-found). diff --git a/doc/install/google_cloud_platform/img/gcp_landing.png b/doc/install/google_cloud_platform/img/gcp_landing.png Binary files differdeleted file mode 100644 index 92a9873728c..00000000000 --- a/doc/install/google_cloud_platform/img/gcp_landing.png +++ /dev/null diff --git a/doc/install/google_cloud_platform/index.md b/doc/install/google_cloud_platform/index.md index 242ad70ae82..77c61acbfd4 100644 --- a/doc/install/google_cloud_platform/index.md +++ b/doc/install/google_cloud_platform/index.md @@ -1,14 +1,13 @@ --- description: 'Learn how to install a GitLab instance on Google Cloud Platform.' +type: howto --- # Installing GitLab on Google Cloud Platform - +This guide will help you install GitLab on a [Google Cloud Platform (GCP)][gcp] instance. -Getting started with GitLab on a [Google Cloud Platform (GCP)][gcp] instance is quick and easy. - -NOTE: **Note:** +NOTE: **Alternative installation method:** Google provides a whitepaper for [deploying production-ready GitLab on Google Kubernetes Engine](https://cloud.google.com/solutions/deploying-production-ready-gitlab-on-gke), including all steps and external resource configuration. These are an alternative to using a GCP VM, and use @@ -26,10 +25,9 @@ Once you have performed those two steps, you can [create a VM](#creating-the-vm) ## Creating the VM -To deploy GitLab on GCP you need to follow five simple steps: - -1. Go to <https://console.cloud.google.com/compute/instances> and login with your Google credentials. +To deploy GitLab on GCP you first need to create a virtual machine: +1. Go to <https://console.cloud.google.com/compute/instances> and log in with your Google credentials. 1. Click on **Create**  @@ -142,3 +140,15 @@ Kerberos, etc. Here are some documents you might be interested in reading: [ssh]: https://cloud.google.com/compute/docs/instances/connecting-to-instance "Connecting to Linux Instances" [omni-smtp]: https://docs.gitlab.com/omnibus/settings/smtp.html#smtp-settings "Omnibus GitLab SMTP settings" [omni-ssl]: https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https "Omnibus GitLab enable HTTPS" + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/install/installation.md b/doc/install/installation.md index 6b24dc8abdc..10436a15a9e 100644 --- a/doc/install/installation.md +++ b/doc/install/installation.md @@ -1,5 +1,28 @@ +--- +type: howto +--- + # Installation from source +This is the official installation guide to set up a production GitLab server +using the source files. To set up a **development installation** or for many +other installation options, see the [main installation page](index.md). +It was created for and tested on **Debian/Ubuntu** operating systems. +Read [requirements.md](requirements.md) for hardware and operating system requirements. +If you want to install on RHEL/CentOS, we recommend using the +[Omnibus packages](https://about.gitlab.com/downloads/). + +This guide is long because it covers many cases and includes all commands you +need, this is [one of the few installation scripts that actually works out of the box](https://twitter.com/robinvdvleuten/status/424163226532986880). +The following steps have been known to work. **Use caution when you deviate** +from this guide. Make sure you don't violate any assumptions GitLab makes about +its environment. For example, many people run into permission problems because +they changed the location of directories or run services as the wrong user. + +If you find a bug/error in this guide, **submit a merge request** +following the +[contributing guide](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md). + ## Consider the Omnibus package installation Since an installation from source is a lot of work and error prone we strongly recommend the fast and reliable [Omnibus package installation](https://about.gitlab.com/downloads/) (deb/rpm). @@ -12,26 +35,42 @@ After this termination Runit will detect Sidekiq is not running and will start i Since installations from source don't use Runit for process supervision, Sidekiq can't be terminated and its memory usage will grow over time. -## Select Version to Install +## Select version to install Make sure you view [this installation guide](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/install/installation.md) from the branch (version) of GitLab you would like to install (e.g., `11-7-stable`). You can select the branch in the version dropdown in the top left corner of GitLab (below the menu bar). If the highest number stable branch is unclear, check the [GitLab blog](https://about.gitlab.com/blog/) for installation guide links by version. -## Important Notes +## GitLab directory structure -This guide is long because it covers many cases and includes all commands you need, this is [one of the few installation scripts that actually works out of the box](https://twitter.com/robinvdvleuten/status/424163226532986880). +This is the main directory structure you will end up with following the instructions +of this page: -This installation guide was created for and tested on **Debian/Ubuntu** operating systems. Read [requirements.md](requirements.md) for hardware and operating system requirements. If you want to install on RHEL/CentOS, we recommend using the [Omnibus packages](https://about.gitlab.com/downloads/). +``` +|-- home +| |-- git +| |-- .ssh +| |-- gitlab +| |-- gitlab-shell +| |-- repositories +``` -This is the official installation guide to set up a production server. To set up a **development installation** or for many other installation options, see [the installation section of the README](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/README.md#installation). +- `/home/git/.ssh` - Contains OpenSSH settings. Specifically the `authorized_keys` + file managed by gitlab-shell. +- `/home/git/gitlab` - GitLab core software. +- `/home/git/gitlab-shell` - Core add-on component of GitLab. Maintains SSH + cloning and other functionality. +- `/home/git/repositories` - Bare repositories for all projects organized by + namespace. This is where the git repositories which are pushed/pulled are + maintained for all projects. **This area contains critical data for projects. + [Keep a backup](../raketasks/backup_restore.md).** -The following steps have been known to work. **Use caution when you deviate** from this guide. Make sure you don't violate any assumptions GitLab makes about its environment. For example, many people run into permission problems because they changed the location of directories or run services as the wrong user. +NOTE: **Note:** +The default locations for repositories can be configured in `config/gitlab.yml` +of GitLab and `config.yml` of gitlab-shell. -If you find a bug/error in this guide, **submit a merge request** -following the -[contributing guide](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md). +For a more in-depth overview, see the [GitLab architecture doc](../development/architecture.md). ## Overview @@ -324,9 +363,7 @@ Redis 2.8 with: sudo apt-get install redis-server ``` -If you are using Debian 7 or Ubuntu 12.04, follow the special documentation -on [an alternate Redis installation](redis.md). Once done, follow the rest of -the guide here. +Once done, you can configure Redis: ```sh # Configure redis to use sockets @@ -930,8 +967,50 @@ and correctly [configured Nginx](#site-configuration). ### google-protobuf "LoadError: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.14' not found" This can happen on some platforms for some versions of the -google-protobuf gem. The workaround is to [install a source-only -version of this gem](google-protobuf.md). +`google-protobuf` gem. The workaround is to install a source-only +version of this gem. + +First, you must find the exact version of `google-protobuf` that your +GitLab installation requires: + +```sh +cd /home/git/gitlab + +# Only one of the following two commands will print something. It +# will look like: * google-protobuf (3.2.0) +bundle list | grep google-protobuf +bundle check | grep google-protobuf +``` + +Below, `3.2.0` is used as an example. Replace it with the version number +you found above: + +```sh +cd /home/git/gitlab +sudo -u git -H gem install google-protobuf --version 3.2.0 --platform ruby +``` + +Finally, you can test whether `google-protobuf` loads correctly. The +following should print 'OK'. + +```sh +sudo -u git -H bundle exec ruby -rgoogle/protobuf -e 'puts :OK' +``` + +If the `gem install` command fails, you may need to install the developer +tools of your OS. + +On Debian/Ubuntu: + +```sh +sudo apt-get install build-essential libgmp-dev +``` + +On RedHat/CentOS: + +```sh +sudo yum groupinstall 'Development Tools' +``` [RVM]: https://rvm.io/ "RVM Homepage" [rbenv]: https://github.com/sstephenson/rbenv "rbenv on GitHub" diff --git a/doc/install/kubernetes/index.md b/doc/install/kubernetes/index.md index 7312bf2d4f7..43655767002 100644 --- a/doc/install/kubernetes/index.md +++ b/doc/install/kubernetes/index.md @@ -1,41 +1,5 @@ --- -description: 'Read through the different methods to deploy GitLab on Kubernetes.' +redirect_to: https://docs.gitlab.com/charts/ --- -# Installing GitLab on Kubernetes - -NOTE: **Kubernetes experience required:** -Our Helm charts are recommended for those who are familiar with Kubernetes. -If you're not sure if Kubernetes is for you, our -[Omnibus GitLab packages](../README.md#installing-gitlab-using-the-omnibus-gitlab-package-recommended) -are mature, scalable, support [high availability](../../administration/high_availability/README.md) -and are used today on GitLab.com. -It is not necessary to have GitLab installed on Kubernetes in order to use [GitLab Kubernetes integration](https://docs.gitlab.com/ee/user/project/clusters/index.html). - -The easiest method to deploy GitLab on [Kubernetes](https://kubernetes.io/) is -to take advantage of GitLab's Helm charts. [Helm](https://github.com/kubernetes/helm/blob/master/README.md) -is a package management tool for Kubernetes, allowing apps to be easily managed via their -Charts. A [Chart](https://github.com/kubernetes/charts) is a detailed description -of the application including how it should be deployed, upgraded, and configured. - -## GitLab Chart - -This chart contains all the required components to get started, and can scale to -large deployments. It offers a number of benefits, among others: - -- Horizontal scaling of individual components. -- No requirement for shared storage to scale. -- Containers do not need `root` permissions. -- Automatic SSL with Let's Encrypt. -- An unprivileged GitLab Runner. - -Learn more about the [GitLab chart](https://docs.gitlab.com/charts/). - -## GitLab Runner Chart - -If you already have a GitLab instance running, inside or outside of Kubernetes, -and you'd like to leverage the Runner's -[Kubernetes capabilities](https://docs.gitlab.com/runner/executors/kubernetes.html), -it can be deployed with the GitLab Runner chart. - -Learn more about the [GitLab Runner chart](https://docs.gitlab.com/runner/install/kubernetes.html). +This document was moved to [another location](https://docs.gitlab.com/charts/). diff --git a/doc/install/openshift_and_gitlab/index.md b/doc/install/openshift_and_gitlab/index.md index 77bd9e9f7a9..18981c43464 100644 --- a/doc/install/openshift_and_gitlab/index.md +++ b/doc/install/openshift_and_gitlab/index.md @@ -1,8 +1,5 @@ --- -author: Achilleas Pipinellis -author_gitlab: axil -level: intermediary -article_type: tutorial +type: howto date: 2016-06-28 --- diff --git a/doc/install/pivotal/index.md b/doc/install/pivotal/index.md index 896e01ad975..f068572f1e9 100644 --- a/doc/install/pivotal/index.md +++ b/doc/install/pivotal/index.md @@ -1,45 +1,12 @@ # GitLab Pivotal Tile **[PREMIUM ONLY]** -> Introduced in [GitLab Premium][eep] 8.2. - -Easily deploy GitLab as a pre-configured appliance using Ops Manager (BOSH) for -[Pivotal Cloud Foundry][pcf]. - -## Overview - -Enterprise admins want their development toolkit to be more customizable, more -integrated, and more secure. With Pivotal Cloud Foundry, GitLab is installed and -scales easily in a highly available environment. - -The upgrades are pain-free and well tested. All it takes is upload the new tile -and click a button to begin the upgrade process. - -## Use cases - -- You want a highly available deployment with minimal effort. Scale horizontally - as your user base grows. - -## Features - -The GitLab Pivotal Tile is based on [GitLab Premium][eep] and includes nearly all of its features. The features in Premium but _not_ supported on the Tile are: - -* PostgreSQL -* Pages -* Geo -* Registry -* Mattermost -* Subgroups -* Elasticsearch -* Service Desk -* OAuth & Kerberos Authentication - -## Installing GitLab with Pivotal - -The product information and installation documentation is hosted on Pivotal's -website: - -- [Product page](https://network.pivotal.io/products/p-gitlab/) -- [Documentation](https://docs.pivotal.io/partners/gitlab/index.html) - -[eep]: https://about.gitlab.com/pricing/ -[pcf]: https://pivotal.io/platform +CAUTION: **Discontinued:** +As of September 13, 2017, the GitLab Enterprise Plus for Pivotal Cloud Foundry +tile on Pivotal Network has reached its End of Availability (“EoA”) and is no +longer available for download or sale through Pivotal. Current customers with +active subscriptions will continue to receive support from GitLab through their +subscription term. Pivotal and GitLab are collaborating on creating a new +Kubernetes-based tile for the Pivotal Container Service. Please contact GitLab +support with any questions regarding GitLab Enterprise Plus for Pivotal Cloud Foundry. + +Original article: <https://docs.pivotal.io/partners/gitlab/index.html>. diff --git a/doc/install/redis.md b/doc/install/redis.md index 4075e6283d0..cff5c2f2611 100644 --- a/doc/install/redis.md +++ b/doc/install/redis.md @@ -1,60 +1,5 @@ -# Install Redis on old distributions +--- +redirect_to: installation.md#7-redis +--- -GitLab requires at least Redis 2.8. The following guide is for Debian 7 and -Ubuntu 12.04. If you are using Debian 8 or Ubuntu 14.04 and up, follow the -[installation guide](installation.md). - -## Install Redis 2.8 in Debian 7 - -Redis 2.8 is included in the Debian Wheezy [backports] repository. - -1. Edit `/etc/apt/sources.list` and add the following line: - - ``` - deb http://http.debian.net/debian wheezy-backports main - ``` - -1. Update the repositories: - - ``` - sudo apt-get update - ``` - -1. Install `redis-server`: - - ``` - sudo apt-get -t wheezy-backports install redis-server - ``` - -1. Follow the rest of the [installation guide](installation.md). - -## Install Redis 2.8 in Ubuntu 12.04 - -We will [use a PPA](https://launchpad.net/~chris-lea/+archive/ubuntu/redis-server) -to install a recent version of Redis. - -1. Install the PPA repository: - - ``` - sudo add-apt-repository ppa:chris-lea/redis-server - ``` - - Your system will now fetch the PPA's key. This enables your Ubuntu system to - verify that the packages in the PPA have not been interfered with since they - were built. - -1. Update the repositories: - - ``` - sudo apt-get update - ``` - -1. Install `redis-server`: - - ``` - sudo apt-get install redis-server - ``` - -1. Follow the rest of the [installation guide](installation.md). - -[backports]: http://backports.debian.org/Instructions/ "Debian backports website" +This document was moved to [another location](installation.md#7-redis). diff --git a/doc/install/relative_url.md b/doc/install/relative_url.md index 5f129fd3bd1..96b7d0f3648 100644 --- a/doc/install/relative_url.md +++ b/doc/install/relative_url.md @@ -1,18 +1,19 @@ +--- +type: reference +--- + # Install GitLab under a relative URL -NOTE: **Note:** +While it is recommended to install GitLab on its own (sub)domain, sometimes +this is not possible due to a variety of reasons. In that case, GitLab can also +be installed under a relative URL, for example `https://example.com/gitlab`. + This document describes how to run GitLab under a relative URL for installations from source. If you are using an Omnibus package, [the steps are different][omnibus-rel]. Use this guide along with the [installation guide](installation.md) if you are installing GitLab for the first time. ---- - -While it is recommended to install GitLab on its own (sub)domain, sometimes -this is not possible due to a variety of reasons. In that case, GitLab can also -be installed under a relative URL, for example `https://example.com/gitlab`. - There is no limit to how deeply nested the relative URL can be. For example you could serve GitLab under `/foo/bar/gitlab/git` without any issues. @@ -20,8 +21,6 @@ Note that by changing the URL on an existing GitLab installation, all remote URLs will change, so you'll have to manually edit them in any local repository that points to your GitLab instance. ---- - The TL;DR list of configuration files that you need to change in order to serve GitLab under a relative URL is: @@ -126,3 +125,15 @@ To disable the relative URL: [omnibus-rel]: http://docs.gitlab.com/omnibus/settings/configuration.html#configuring-a-relative-url-for-gitlab "How to set up relative URL in Omnibus GitLab" [restart gitlab]: ../administration/restart_gitlab.md#installations-from-source "How to restart GitLab" + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/install/requirements.md b/doc/install/requirements.md index f6a52205a0e..107d48fb90c 100644 --- a/doc/install/requirements.md +++ b/doc/install/requirements.md @@ -1,5 +1,12 @@ +--- +type: reference +--- + # Requirements +This page includes useful information on the supported Operating Systems as well +as the hardware requirements that are needed to install and use GitLab. + ## Operating Systems ### Supported Unix distributions @@ -12,7 +19,7 @@ - Scientific Linux (please use the CentOS packages and instructions) - Oracle Linux (please use the CentOS packages and instructions) -For the installations options please see [the installation page on the GitLab website](https://about.gitlab.com/installation/). +For the installations options, see [the main installation page](README.md). ### Unsupported Unix distributions @@ -105,9 +112,9 @@ features of GitLab work with MySQL/MariaDB: 1. MySQL support for subgroups was [dropped with GitLab 9.3][post]. See [issue #30472][30472] for more information. -1. Geo does [not support MySQL](https://docs.gitlab.com/ee/administration/geo/replication/database.html). This means no supported Disaster Recovery solution if using MySQL. **[PREMIUM ONLY]** +1. Geo does [not support MySQL](../administration/geo/replication/database.md). This means no supported Disaster Recovery solution if using MySQL. **[PREMIUM ONLY]** 1. [Zero downtime migrations](../update/README.md#upgrading-without-downtime) do not work with MySQL. -1. [Database load balancing](https://docs.gitlab.com/ee/administration/database_load_balancing.html) is +1. [Database load balancing](../administration/database_load_balancing.md) is supported only for PostgreSQL. **[PREMIUM ONLY]** 1. GitLab [optimizes the loading of dashboard events](https://gitlab.com/gitlab-org/gitlab-ce/issues/31806) using [PostgreSQL LATERAL JOINs](https://blog.heapanalytics.com/postgresqls-powerful-new-join-type-lateral/). 1. In general, SQL optimized for PostgreSQL may run much slower in MySQL due to @@ -143,14 +150,14 @@ On some systems you may need to install an additional package (e.g. #### Additional requirements for GitLab Geo -If you are using [GitLab Geo](https://docs.gitlab.com/ee/development/geo.html): +If you are using [GitLab Geo](../development/geo.md): - We strongly recommend running Omnibus-managed instances as they are actively developed and tested. We aim to be compatible with most external (not managed by Omnibus) databases (for example, AWS RDS) but we do not guarantee compatibility. - The - [tracking database](https://docs.gitlab.com/ee/development/geo.html#geo-tracking-database) + [tracking database](../development/geo.md#using-the-tracking-database) requires the [postgres_fdw](https://www.postgresql.org/docs/9.6/static/postgres-fdw.html) extension. @@ -201,12 +208,23 @@ you decide to run GitLab Runner and the GitLab Rails application on the same machine. It is also not safe to install everything on a single machine, because of the -[security reasons](https://docs.gitlab.com/runner/security/) -- especially when you plan to use shell executor with GitLab +[security reasons](https://docs.gitlab.com/runner/security/), especially when you plan to use shell executor with GitLab Runner. We recommend using a separate machine for each GitLab Runner, if you plan to use the CI features. +The GitLab Runner server requirements depend on: + +- The type of [executor](https://docs.gitlab.com/runner/executors/) you configured on GitLab Runner. +- Resources required to run build jobs. +- Job concurrency settings. + +Since the nature of the jobs varies for each use case, you will need to experiment by adjusting the job concurrency to get the optimum setting. + +For reference, GitLab.com's [auto-scaling shared runner](../user/gitlab_com/index.md#shared-runners) is configured so that a **single job** will run in a **single instance** with: + +- 1vCPU. +- 3.75GB of RAM. ## Supported web browsers @@ -225,3 +243,15 @@ Each time a new browser version is released, we begin supporting that version an NOTE: **Note:** We do not support running GitLab with JavaScript disabled in the browser and have no plans of supporting that in the future because we have features such as Issue Boards which require JavaScript extensively. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/install/structure.md b/doc/install/structure.md index 8fc6ab4ab2f..87ef11c60fe 100644 --- a/doc/install/structure.md +++ b/doc/install/structure.md @@ -1,19 +1,5 @@ -# GitLab directory structure +--- +redirect_to: installation.md#gitlab-directory-structure +--- -This is the directory structure you will end up with following the instructions in the Installation Guide. - - |-- home - | |-- git - | |-- .ssh - | |-- gitlab - | |-- gitlab-shell - | |-- repositories - -- `/home/git/.ssh` - contains openssh settings. Specifically the `authorized_keys` file managed by gitlab-shell. -- `/home/git/gitlab` - GitLab core software. -- `/home/git/gitlab-shell` - Core add-on component of GitLab. Maintains SSH cloning and other functionality. -- `/home/git/repositories` - bare repositories for all projects organized by namespace. This is where the git repositories which are pushed/pulled are maintained for all projects. **This area is critical data for projects. [Keep a backup](../raketasks/backup_restore.md).** - -*Note: the default locations for repositories can be configured in `config/gitlab.yml` of GitLab and `config.yml` of gitlab-shell.* - -To see a more in-depth overview see the [GitLab architecture doc](../development/architecture.md). +This page was moved to [another location](#installation.md#gitlab-directory-structure). diff --git a/doc/integration/elasticsearch.md b/doc/integration/elasticsearch.md index 15176ede733..0a037b3876b 100644 --- a/doc/integration/elasticsearch.md +++ b/doc/integration/elasticsearch.md @@ -8,8 +8,8 @@ This document describes how to set up Elasticsearch with GitLab. Once enabled, you'll have the benefit of fast search response times and the advantage of two special searches: -- [Advanced Global Search](https://docs.gitlab.com/ee/user/search/advanced_global_search.html) -- [Advanced Syntax Search](https://docs.gitlab.com/ee/user/search/advanced_search_syntax.html) +- [Advanced Global Search](../user/search/advanced_global_search.md) +- [Advanced Syntax Search](../user/search/advanced_search_syntax.md) ## Version Requirements <!-- Please remember to update ee/lib/system_check/app/elasticsearch_check.rb if this changes --> diff --git a/doc/integration/salesforce.md b/doc/integration/salesforce.md index 8a99641a256..1ef43cfcece 100644 --- a/doc/integration/salesforce.md +++ b/doc/integration/salesforce.md @@ -1,27 +1,28 @@ -# SalesForce OmniAuth Provider +# Salesforce OmniAuth Provider -You can integrate your GitLab instance with [SalesForce](https://www.salesforce.com/) to enable users to login to your GitLab instance with their SalesForce account. +You can integrate your GitLab instance with [Salesforce](https://www.salesforce.com/) to enable users to log in to your GitLab instance with their Salesforce account. -## Create SalesForce Application +## Create a Salesforce Connected App -To enable SalesForce OmniAuth provider, you must use SalesForce's credentials for your GitLab instance. -To get the credentials (a pair of Client ID and Client Secret), you must register an application on SalesForces. +To enable Salesforce OmniAuth provider, you must use Salesforce's credentials for your GitLab instance. +To get the credentials (a pair of Client ID and Client Secret), you must [create a Connected App](https://help.salesforce.com/articleView?id=connected_app_create.htm&type=5) on Salesforce. -1. Sign in to [SalesForce](https://www.salesforce.com/). +1. Sign in to [Salesforce](https://login.salesforce.com/). -1. Navigate to **Platform Tools/Apps/App Manager** and click on **New Connected App**. +1. In Setup, enter `App Manager` in the Quick Find box, click **App Manager**, then click **New Connected App**. 1. Fill in the application details into the following fields: - **Connected App Name** and **API Name**: Set to any value but consider something like `<Organization>'s GitLab`, `<Your Name>'s GitLab`, or something else that is descriptive. + - **Contact Email**: Enter the contact email for Salesforce to use when contacting you or your support team. - **Description**: Description for the application. -  +  1. Select **API (Enable OAuth Settings)** and click on **Enable OAuth Settings**. 1. Fill in the application details into the following fields: - - **Callback URL**: The call callback URL. For example, `https://gitlab.example.com/users/auth/salesforce/callback`. + - **Callback URL**: The callback URL of your GitLab installation. For example, `https://gitlab.example.com/users/auth/salesforce/callback`. - **Selected OAuth Scopes**: Move **Access your basic information (id, profile, email, address, phone)** and **Allow access to your unique identifier (openid)** to the right column. -  +  1. Click **Save**. 1. On your GitLab server, open the configuration file. @@ -63,17 +64,16 @@ To get the credentials (a pair of Client ID and Client Secret), you must registe app_secret: 'SALESFORCE_CLIENT_SECRET' } ``` -1. Change `SALESFORCE_CLIENT_ID` to the Consumer Key from the SalesForce connected application page. -1. Change `SALESFORCE_CLIENT_SECRET` to the Consumer Secret from the SalesForce connected application page. -  +1. Change `SALESFORCE_CLIENT_ID` to the Consumer Key from the Salesforce connected application page. +1. Change `SALESFORCE_CLIENT_SECRET` to the Consumer Secret from the Salesforce connected application page. +  1. Save the configuration file. -1. [Reconfigure GitLab]( ../administration/restart_gitlab.md#omnibus-gitlab-reconfigure ) or [restart GitLab]( ../administration/restart_gitlab.md#installations-from-source ) for the changes to take effect if you - installed GitLab via Omnibus or from source respectively. +1. [Reconfigure GitLab]( ../administration/restart_gitlab.md#omnibus-gitlab-reconfigure ) or [restart GitLab]( ../administration/restart_gitlab.md#installations-from-source ) for the changes to take effect if you installed GitLab via Omnibus or from source respectively. -On the sign in page, there should now be a SalesForce icon below the regular sign in form. -Click the icon to begin the authentication process. SalesForce will ask the user to sign in and authorize the GitLab application. +On the sign in page, there should now be a Salesforce icon below the regular sign in form. +Click the icon to begin the authentication process. Salesforce will ask the user to sign in and authorize the GitLab application. If everything goes well, the user will be returned to GitLab and will be signed in. NOTE: **Note:** -GitLab requires the email address of each new user. Once the user is logged in using SalesForce, GitLab will redirect the user to the profile page where they will have to provide the email and verify the email. +GitLab requires the email address of each new user. Once the user is logged in using Salesforce, GitLab will redirect the user to the profile page where they will have to provide the email and verify the email.
\ No newline at end of file diff --git a/doc/integration/saml.md b/doc/integration/saml.md index 15d9d8c9c74..22e07594d6f 100644 --- a/doc/integration/saml.md +++ b/doc/integration/saml.md @@ -1,6 +1,6 @@ # SAML OmniAuth Provider -> This topic is for SAML on self-managed GitLab instances. For SAML on GitLab.com, see [SAML SSO for GitLab.com Groups](https://docs.gitlab.com/ee/user/group/saml_sso/index.html). +> This topic is for SAML on self-managed GitLab instances. For SAML on GitLab.com, see [SAML SSO for GitLab.com Groups](../user/group/saml_sso/index.md). NOTE: **Note:** You need to [enable OmniAuth](omniauth.md) in order to use this. diff --git a/doc/integration/slash_commands.md b/doc/integration/slash_commands.md index cd755089be8..71ea2e25533 100644 --- a/doc/integration/slash_commands.md +++ b/doc/integration/slash_commands.md @@ -20,8 +20,8 @@ Taking the trigger term as `project-name`, the commands are: | `/project-name deploy <from> to <to>` | Deploy from the `<from>` environment to the `<to>` environment | | `/project-name run <job name> <arguments>` | Execute [ChatOps](../ci/chatops/README.md) job `<job name>` on `master` | -Note that if you are using the [GitLab Slack application](https://docs.gitlab.com/ee/user/project/integrations/gitlab_slack_application.html) for -your GitLab.com projects, you need to [add the `gitlab` keyword at the beginning of the command](https://docs.gitlab.com/ee/user/project/integrations/gitlab_slack_application.html#usage). +Note that if you are using the [GitLab Slack application](../user/project/integrations/gitlab_slack_application.md) for +your GitLab.com projects, you need to [add the `gitlab` keyword at the beginning of the command](../user/project/integrations/gitlab_slack_application.md#usage). ## Issue commands diff --git a/doc/license/README.md b/doc/license/README.md index b9281fd5299..fd110a39b61 100644 --- a/doc/license/README.md +++ b/doc/license/README.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/admin_area/license.html' +redirect_to: '../user/admin_area/license.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/user/admin_area/license.html). +This document was moved to [another location](../user/admin_area/license.md). diff --git a/doc/push_rules/push_rules.md b/doc/push_rules/push_rules.md index 7654023f266..e44eab2556e 100644 --- a/doc/push_rules/push_rules.md +++ b/doc/push_rules/push_rules.md @@ -61,7 +61,7 @@ The following options are available. | --------- | :------------: | ----------- | | Removal of tags with `git push` | **Starter** 7.10 | Forbid users to remove git tags with `git push`. Tags will still be able to be deleted through the web UI. | | Check whether author is a GitLab user | **Starter** 7.10 | Restrict commits by author (email) to existing GitLab users. | -| Check whether committer is the current authenticated user | **Premium** 10.2 | GitLab will reject any commit that was not committed by the current authenticated user | +| Committer restriction | **Premium** 10.2 | GitLab will reject any commit that was not committed by the current authenticated user | | Check whether commit is signed through GPG | **Premium** 10.1 | Reject commit when it is not signed through GPG. Read [signing commits with GPG][signing-commits]. | | Prevent committing secrets to Git | **Starter** 8.12 | GitLab will reject any files that are likely to contain secrets. Read [what files are forbidden](#prevent-pushing-secrets-to-the-repository). | | Restrict by commit message | **Starter** 7.10 | Only commit messages that match this regular expression are allowed to be pushed. Leave empty to allow any commit message. Uses multiline mode, which can be disabled using `(?-m)`. | diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index 56db7b5eb3a..764916ca82d 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -917,9 +917,9 @@ backup beforehand. 1. Clear all the tokens for projects, groups, and the whole instance: -CAUTION: **Caution:** -The last UPDATE operation will stop the runners being able to pick up -new jobs. You must register new runners. + CAUTION: **Caution:** + The last UPDATE operation will stop the runners being able to pick up + new jobs. You must register new runners. ```sql -- Clear project tokens diff --git a/doc/raketasks/import.md b/doc/raketasks/import.md index bb316df5b9a..b59c06a24ea 100644 --- a/doc/raketasks/import.md +++ b/doc/raketasks/import.md @@ -16,7 +16,7 @@ The new folder needs to have git user ownership and read/write/execute access for git user and its group: ``` -sudo -u git mkdir /var/opt/gitlab/git-data/repository-import-<date>/new_group +sudo -u git mkdir -p /var/opt/gitlab/git-data/repository-import-<date>/new_group ``` ### Copy your bare repositories inside this newly created folder: diff --git a/doc/security/rack_attack.md b/doc/security/rack_attack.md index ad83dc05a93..66081d7e376 100644 --- a/doc/security/rack_attack.md +++ b/doc/security/rack_attack.md @@ -94,7 +94,7 @@ In case you want to remove a blocked IP, follow these steps: 1. Find the IPs that have been blocked in the production log: ```sh - grep "Rack_Attack" /var/log/gitlab/gitlab-rails/production.log + grep "Rack_Attack" /var/log/gitlab/gitlab-rails/auth.log ``` 1. Since the blacklist is stored in Redis, you need to open up `redis-cli`: diff --git a/doc/ssh/README.md b/doc/ssh/README.md index 9c4a391e8da..3bfebfc5d9b 100644 --- a/doc/ssh/README.md +++ b/doc/ssh/README.md @@ -55,7 +55,7 @@ As an admin, you can restrict By default, all keys are permitted, which is also the case for [GitLab.com](../user/gitlab_com/index.md#ssh-host-keys-fingerprints). -## ED25519 SSH keys +### ED25519 SSH keys Following [best practices](https://linux-audit.com/using-ed25519-openssh-keys-instead-of-dsa-rsa-ecdsa/), you should always favor [ED25519](https://ed25519.cr.yp.to/) SSH keys, since they @@ -65,7 +65,7 @@ They were introduced in OpenSSH 6.5, so any modern OS should include the option to create them. If for any reason your OS or the GitLab instance you interact with doesn't support this, you can fallback to RSA. -## RSA SSH keys +### RSA SSH keys RSA keys are the most common ones and therefore the most compatible with servers that may have an old OpenSSH version. Use them if the GitLab server @@ -166,12 +166,13 @@ Now, it's time to add the newly created public key to your GitLab account. NOTE: **Note:** If you opted to create an RSA key, the name might differ. -1. Add your public SSH key to your GitLab account by clicking your avatar - in the upper right corner and selecting **Settings**. From there on, - navigate to **SSH Keys** and paste your public key in the "Key" section. - If you created the key with a comment, this will appear under "Title". - If not, give your key an identifiable title like _Work Laptop_ or - _Home Workstation_, and click **Add key**. +1. Add your **public** SSH key to your GitLab account by: + 1. Clicking your avatar in the upper right corner and selecting **Settings**. + 1. Navigating to **SSH Keys** and pasting your **public** key in the **Key** field. If you: + + - Created the key with a comment, this will appear in the **Title** field. + - Created the key without a comment, give your key an identifiable title like _Work Laptop_ or _Home Workstation_. + 1. Click the **Add key** button. NOTE: **Note:** If you manually copied your public SSH key make sure you copied the entire @@ -305,7 +306,7 @@ who needs to know and configure the private key. GitLab administrators set up Global Deploy keys in the Admin area under the section **Deploy Keys**. Ensure keys have a meaningful title as that will be the primary way for project maintainers and owners to identify the correct Global -Deploy key to add. For instance, if the key gives access to a SaaS CI instance, +Deploy key to add. For instance, if the key gives access to a SaaS CI instance, use the name of that service in the key name if that is all it is used for. When creating Global Shared Deploy keys, give some thought to the granularity of keys - they could be of very narrow usage such as just a specific service or diff --git a/doc/subscriptions/index.md b/doc/subscriptions/index.md index b6d16536fee..dfd80f8882e 100644 --- a/doc/subscriptions/index.md +++ b/doc/subscriptions/index.md @@ -38,7 +38,7 @@ Future purchases will use the information in this section. The email listed in t ### Self-managed: Apply your license file -After purchase, the license file is sent to the email address tied to the Customers portal account, which needs to be [uploaded to the GitLab instance](https://docs.gitlab.com/ee/user/admin_area/license.html#uploading-your-license). +After purchase, the license file is sent to the email address tied to the Customers portal account, which needs to be [uploaded to the GitLab instance](../user/admin_area/license.md#uploading-your-license). ### Link your GitLab.com account with your Customers Portal account diff --git a/doc/topics/authentication/index.md b/doc/topics/authentication/index.md index 00b6c1dfdc2..228da2d1f57 100644 --- a/doc/topics/authentication/index.md +++ b/doc/topics/authentication/index.md @@ -17,11 +17,11 @@ This page gathers all the resources for the topic **Authentication** within GitL ## GitLab administrators - [LDAP (Community Edition)](../../administration/auth/ldap.md) -- [LDAP (Enterprise Edition)](https://docs.gitlab.com/ee/administration/auth/ldap-ee.html) **[STARTER]** +- [LDAP (Enterprise Edition)](../../administration/auth/ldap-ee.md) **[STARTER]** - [Enforce Two-factor Authentication (2FA)](../../security/two_factor_authentication.md#enforce-two-factor-authentication-2fa) - **Articles:** - [How to Configure LDAP with GitLab CE](../../administration/auth/how_to_configure_ldap_gitlab_ce/index.md) - - [How to Configure LDAP with GitLab EE](https://docs.gitlab.com/ee/administration/auth/how_to_configure_ldap_gitlab_ee/index.html) **[STARTER]** + - [How to Configure LDAP with GitLab EE](../../administration/auth/how_to_configure_ldap_gitlab_ee/index.md) **[STARTER]** - [Feature Highlight: LDAP Integration](https://about.gitlab.com/2014/07/10/feature-highlight-ldap-sync/) - [Debugging LDAP](https://about.gitlab.com/handbook/support/workflows/support-engineering/ldap/debugging_ldap.html) - **Integrations:** @@ -30,10 +30,10 @@ This page gathers all the resources for the topic **Authentication** within GitL - [Atlassian Crowd OmniAuth Provider](../../administration/auth/crowd.md) - [CAS OmniAuth Provider](../../integration/cas.md) - [SAML OmniAuth Provider](../../integration/saml.md) - - [SAML for GitLab.com Groups](https://docs.gitlab.com/ee/user/group/saml_sso/index.html) **[SILVER ONLY]** - - [SCIM user provisioning for GitLab.com Groups](https://docs.gitlab.com/ee/user/group/saml_sso/scim_setup.html) **[SILVER ONLY]** + - [SAML for GitLab.com Groups](../../user/group/saml_sso/index.md) **[SILVER ONLY]** + - [SCIM user provisioning for GitLab.com Groups](../../user/group/saml_sso/scim_setup.md) **[SILVER ONLY]** - [Okta SSO provider](../../administration/auth/okta.md) - - [Kerberos integration (GitLab EE)](https://docs.gitlab.com/ee/integration/kerberos.html) **[STARTER]** + - [Kerberos integration (GitLab EE)](../../integration/kerberos.md) **[STARTER]** ## API diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index 5a8744d71f9..b00a8afa386 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -13,7 +13,7 @@ Starting with GitLab 11.3, the Auto DevOps pipeline is enabled by default for al projects. If it has not been explicitly enabled for the project, Auto DevOps will be automatically disabled on the first pipeline failure. Your project will continue to use an alternative [CI/CD configuration file](../../ci/yaml/README.md) if one is found. A GitLab -administrator can [change this setting](../../user/admin_area/settings/continuous_integration.html#auto-devops-core-only) +administrator can [change this setting](../../user/admin_area/settings/continuous_integration.md#auto-devops-core-only) in the admin area. With Auto DevOps, the software development process becomes easier to set up @@ -126,10 +126,6 @@ Auto Deploy, and Auto Monitoring will be silently skipped. ## Auto DevOps base domain -NOTE: **Note** -`AUTO_DEVOPS_DOMAIN` environment variable is deprecated and -[is scheduled to be removed](https://gitlab.com/gitlab-org/gitlab-ce/issues/56959). - The Auto DevOps base domain is required if you want to make use of [Auto Review Apps](#auto-review-apps) and [Auto Deploy](#auto-deploy). It can be defined in any of the following places: @@ -162,6 +158,12 @@ Auto DevOps base domain to `1.2.3.4.nip.io`. Once set up, all requests will hit the load balancer, which in turn will route them to the Kubernetes pods that run your application(s). +NOTE: **Note:** +From GitLab 11.8, `KUBE_INGRESS_BASE_DOMAIN` replaces `AUTO_DEVOPS_DOMAIN`. +Support for `AUTO_DEVOPS_DOMAIN` was [removed in GitLab +12.0](https://gitlab.com/gitlab-org/gitlab-ce/issues/56959). + + ## Using multiple Kubernetes clusters **[PREMIUM]** When using Auto DevOps, you may want to deploy different environments to @@ -179,7 +181,7 @@ Those environments are tied to jobs that use [Auto Deploy](#auto-deploy), so except for the environment scope, they would also need to have a different domain they would be deployed to. This is why you need to define a separate `KUBE_INGRESS_BASE_DOMAIN` variable for all the above -[based on the environment](https://docs.gitlab.com/ee/ci/variables/#limiting-environment-scopes-of-environment-variables-premium). +[based on the environment](../../ci/variables/README.md#limiting-environment-scopes-of-environment-variables-premium). The following table is an example of how the three different clusters would be configured. @@ -209,10 +211,6 @@ and verifying that your app is deployed as a review app in the Kubernetes cluster with the `review/*` environment scope. Similarly, you can check the other environments. -NOTE: **Note:** -From GitLab 11.8, `KUBE_INGRESS_BASE_DOMAIN` replaces `AUTO_DEVOPS_DOMAIN`. -`AUTO_DEVOPS_DOMAIN` [is scheduled to be removed](https://gitlab.com/gitlab-org/gitlab-ce/issues/56959). - ## Enabling/Disabling Auto DevOps When first using Auto Devops, review the [requirements](#requirements) to ensure all necessary components to make @@ -365,7 +363,7 @@ created, and is uploaded as an artifact which you can later download and check out. Any differences between the source and target branches are also -[shown in the merge request widget](https://docs.gitlab.com/ee/user/project/merge_requests/code_quality.html). +[shown in the merge request widget](../../user/project/merge_requests/code_quality.md). ### Auto SAST **[ULTIMATE]** @@ -378,7 +376,7 @@ report is created, it's uploaded as an artifact which you can later download and check out. Any security warnings are also shown in the merge request widget. Read more how -[SAST works](https://docs.gitlab.com/ee/user/application_security/sast/index.html). +[SAST works](../../user/application_security/sast/index.md). NOTE: **Note:** The Auto SAST stage will be skipped on licenses other than Ultimate. @@ -397,7 +395,7 @@ report is created, it's uploaded as an artifact which you can later download and check out. Any security warnings are also shown in the merge request widget. Read more about -[Dependency Scanning](https://docs.gitlab.com/ee/user/application_security/dependency_scanning/index.html). +[Dependency Scanning](../../user/application_security/dependency_scanning/index.md). NOTE: **Note:** The Auto Dependency Scanning stage will be skipped on licenses other than Ultimate. @@ -416,7 +414,7 @@ report is created, it's uploaded as an artifact which you can later download and check out. Any licenses are also shown in the merge request widget. Read more how -[License Management works](https://docs.gitlab.com/ee/user/application_security/license_management/index.html). +[License Management works](../../user/application_security/license_management/index.md). NOTE: **Note:** The Auto License Management stage will be skipped on licenses other than Ultimate. @@ -432,7 +430,7 @@ created, it's uploaded as an artifact which you can later download and check out. Any security warnings are also shown in the merge request widget. Read more how -[Container Scanning works](https://docs.gitlab.com/ee/user/application_security/container_scanning/index.html). +[Container Scanning works](../../user/application_security/container_scanning/index.md). NOTE: **Note:** The Auto Container Scanning stage will be skipped on licenses other than Ultimate. @@ -488,7 +486,7 @@ issues. Once the report is created, it's uploaded as an artifact which you can later download and check out. Any security warnings are also shown in the merge request widget. Read how -[DAST works](https://docs.gitlab.com/ee/user/application_security/dast/index.html). +[DAST works](../../user/application_security/dast/index.md). NOTE: **Note:** The Auto DAST stage will be skipped on licenses other than Ultimate. @@ -506,7 +504,7 @@ Auto Browser Performance Testing utilizes the [Sitespeed.io container](https://h ``` Any performance differences between the source and target branches are also -[shown in the merge request widget](https://docs.gitlab.com/ee//user/project/merge_requests/browser_performance_testing.html). +[shown in the merge request widget](../../user/project/merge_requests/browser_performance_testing.md). ### Auto Deploy @@ -584,16 +582,17 @@ Note that a post-install hook means that if any deploy succeeds, If present, `DB_MIGRATE` will be run as a shell command within an application pod as a helm pre-upgrade hook. -For example, in a Rails application: +For example, in a Rails application in an image built with +[Herokuish](https://github.com/gliderlabs/herokuish): -- `DB_INITIALIZE` can be set to `cd /app && RAILS_ENV=production - bin/setup` -- `DB_MIGRATE` can be set to `cd /app && RAILS_ENV=production bin/update` +- `DB_INITIALIZE` can be set to `RAILS_ENV=production /bin/herokuish procfile exec bin/rails db:setup` +- `DB_MIGRATE` can be set to `RAILS_ENV=production /bin/herokuish procfile exec bin/rails db:migrate` NOTE: **Note:** -The `/app` path is the directory of your project inside the docker image -as [configured by -Herokuish](https://github.com/gliderlabs/herokuish#paths) +Unless you have a `Dockerfile` in your repo, your image is built with +Herokuish. You must prefix commands run in these images with `/bin/herokuish +procfile exec` in order to replicate the the environment your application is +run in. ### Auto Monitoring @@ -675,7 +674,7 @@ repo or by specifying a project variable: ### Custom Helm chart per environment **[PREMIUM]** You can specify the use of a custom Helm chart per environment by scoping the environment variable -to the desired environment. See [Limiting environment scopes of variables](https://docs.gitlab.com/ee/ci/variables/#limiting-environment-scopes-of-variables-premium). +to the desired environment. See [Limiting environment scopes of variables](../../ci/variables/README.md#limiting-environment-scopes-of-environment-variables-premium). ### Customizing `.gitlab-ci.yml` @@ -734,7 +733,6 @@ also be customized, and you can easily use a [custom buildpack](#custom-buildpac | **Variable** | **Description** | | ------------ | --------------- | -| `AUTO_DEVOPS_DOMAIN` | The [Auto DevOps domain](#auto-devops-base-domain). By default, set automatically by the [Auto DevOps setting](#enablingdisabling-auto-devops). This variable is deprecated and [is scheduled to be removed](https://gitlab.com/gitlab-org/gitlab-ce/issues/56959). Use `KUBE_INGRESS_BASE_DOMAIN` instead. | | `AUTO_DEVOPS_CHART` | The Helm Chart used to deploy your apps; defaults to the one [provided by GitLab](https://gitlab.com/gitlab-org/charts/auto-deploy-app). | | `AUTO_DEVOPS_CHART_REPOSITORY` | The Helm Chart repository used to search for charts; defaults to `https://charts.gitlab.io`. | | `AUTO_DEVOPS_CHART_REPOSITORY_NAME` | From Gitlab 11.11, this variable can be used to set the name of the helm repository; defaults to "gitlab" | @@ -742,8 +740,8 @@ also be customized, and you can easily use a [custom buildpack](#custom-buildpac | `AUTO_DEVOPS_CHART_REPOSITORY_PASSWORD` | From Gitlab 11.11, this variable can be used to set a password to connect to the helm repository. Defaults to no credentials. (Also set AUTO_DEVOPS_CHART_REPOSITORY_USERNAME) | | `REPLICAS` | The number of replicas to deploy; defaults to 1. | | `PRODUCTION_REPLICAS` | The number of replicas to deploy in the production environment. This takes precedence over `REPLICAS`; defaults to 1. | -| `CANARY_REPLICAS` | The number of canary replicas to deploy for [Canary Deployments](https://docs.gitlab.com/ee/user/project/canary_deployments.html); defaults to 1 | -| `CANARY_PRODUCTION_REPLICAS` | The number of canary replicas to deploy for [Canary Deployments](https://docs.gitlab.com/ee/user/project/canary_deployments.html) in the production environment. This takes precedence over `CANARY_REPLICAS`; defaults to 1 | +| `CANARY_REPLICAS` | The number of canary replicas to deploy for [Canary Deployments](../../user/project/canary_deployments.md); defaults to 1 | +| `CANARY_PRODUCTION_REPLICAS` | The number of canary replicas to deploy for [Canary Deployments](../../user/project/canary_deployments.md) in the production environment. This takes precedence over `CANARY_REPLICAS`; defaults to 1 | | `ADDITIONAL_HOSTS` | Fully qualified domain names specified as a comma-separated list that are added to the ingress hosts. | | `<ENVIRONMENT>_ADDITIONAL_HOSTS` | For a specific environment, the fully qualified domain names specified as a comma-separated list that are added to the ingress hosts. This takes precedence over `ADDITIONAL_HOSTS`. | | `POSTGRES_ENABLED` | Whether PostgreSQL is enabled; defaults to `"true"`. Set to `false` to disable the automatic deployment of PostgreSQL. | @@ -920,7 +918,7 @@ you when you're ready to manually deploy to production. > [Introduced](https://gitlab.com/gitlab-org/gitlab-ci-yml/merge_requests/171) in GitLab 11.0. -A [canary environment](https://docs.gitlab.com/ee/user/project/canary_deployments.html) can be used +A [canary environment](../../user/project/canary_deployments.md) can be used before any changes are deployed to production. If `CANARY_ENABLED` is defined in your project (e.g., set `CANARY_ENABLED` to diff --git a/doc/topics/autodevops/quick_start_guide.md b/doc/topics/autodevops/quick_start_guide.md index 367e192b85a..cc83d20d65a 100644 --- a/doc/topics/autodevops/quick_start_guide.md +++ b/doc/topics/autodevops/quick_start_guide.md @@ -208,7 +208,7 @@ applications. In the rightmost column for the production environment, you can ma application is running. Right below, there is the -[Deploy Board](https://docs.gitlab.com/ee/user/project/deploy_boards.html). +[Deploy Board](../../user/project/deploy_boards.md). The squares represent pods in your Kubernetes cluster that are associated with the given environment. Hovering above each square you can see the state of a deployment and clicking a square will take you to the pod's logs page. diff --git a/doc/topics/git/index.md b/doc/topics/git/index.md index 7707d56764e..db6d1a62f59 100644 --- a/doc/topics/git/index.md +++ b/doc/topics/git/index.md @@ -70,5 +70,5 @@ We've gathered some resources to help you to get the best from Git with GitLab. - [Getting Started with Git LFS](https://about.gitlab.com/2017/01/30/getting-started-with-git-lfs-tutorial/) - [GitLab Git LFS documentation](../../workflow/lfs/manage_large_binaries_with_git_lfs.md) -- [Git-Annex to Git-LFS migration guide](https://docs.gitlab.com/ee/workflow/lfs/migrate_from_git_annex_to_git_lfs.html) +- [Git-Annex to Git-LFS migration guide](../../workflow/lfs/migrate_from_git_annex_to_git_lfs.md) - Article (2015-08-13): [Towards a production quality open source Git LFS server](https://about.gitlab.com/2015/08/13/towards-a-production-quality-open-source-git-lfs-server/) diff --git a/doc/university/README.md b/doc/university/README.md index cf13246067f..61ed72d25fb 100644 --- a/doc/university/README.md +++ b/doc/university/README.md @@ -51,10 +51,10 @@ The curriculum is composed of GitLab videos, screencasts, presentations, project #### 1.5. Migrating from other Source Control -1. [Migrating from BitBucket/Stash](https://docs.gitlab.com/ee/user/project/import/bitbucket.html) -1. [Migrating from GitHub](https://docs.gitlab.com/ee/user/project/import/github.html) -1. [Migrating from SVN](https://docs.gitlab.com/ee/user/project/import/svn.html) -1. [Migrating from Fogbugz](https://docs.gitlab.com/ee/user/project/import/fogbugz.html) +1. [Migrating from BitBucket/Stash](../user/project/import/bitbucket.md) +1. [Migrating from GitHub](../user/project/import/github.md) +1. [Migrating from SVN](../user/project/import/svn.md) +1. [Migrating from Fogbugz](../user/project/import/fogbugz.md) #### 1.6. GitLab Inc. @@ -92,7 +92,7 @@ The curriculum is composed of GitLab videos, screencasts, presentations, project 1. [Using any Static Site Generator with GitLab Pages](https://about.gitlab.com/2016/06/17/ssg-overview-gitlab-pages-part-3-examples-ci/) 1. [Securing GitLab Pages with SSL](https://about.gitlab.com/2016/06/24/secure-gitlab-pages-with-startssl/) -1. [GitLab Pages Documentation](https://docs.gitlab.com/ce/user/project/pages/) +1. [GitLab Pages Documentation](../user/project/pages/index.md) #### 2.2. GitLab Issues @@ -131,7 +131,7 @@ The curriculum is composed of GitLab videos, screencasts, presentations, project 1. [GitLab Flow vs Forking in GitLab - Video](https://www.youtube.com/watch?v=UGotqAUACZA) 1. [GitLab Flow Overview](https://about.gitlab.com/2014/09/29/gitlab-flow/) 1. [Always Start with an Issue](https://about.gitlab.com/2016/03/03/start-with-an-issue/) -1. [GitLab Flow Documentation](https://docs.gitlab.com/ee/workflow/gitlab_flow.html) +1. [GitLab Flow Documentation](../workflow/gitlab_flow.md) #### 2.5. GitLab Comparisons @@ -191,10 +191,10 @@ The curriculum is composed of GitLab videos, screencasts, presentations, project #### 3.9. Integrations 1. [How to Integrate JIRA and Jenkins with GitLab - Video](https://gitlabmeetings.webex.com/gitlabmeetings/ldr.php?RCID=44b548147a67ab4d8a62274047146415) -1. [How to Integrate Jira with GitLab](https://docs.gitlab.com/ce/user/project/integrations/jira.html) -1. [How to Integrate Jenkins with GitLab](https://docs.gitlab.com/ee/integration/jenkins.html) -1. [How to Integrate Bamboo with GitLab](https://docs.gitlab.com/ce/user/project/integrations/bamboo.html) -1. [How to Integrate Slack with GitLab](https://docs.gitlab.com/ce/user/project/integrations/slack.html) +1. [How to Integrate Jira with GitLab](../user/project/integrations/jira.md) +1. [How to Integrate Jenkins with GitLab](../integration/jenkins.md) +1. [How to Integrate Bamboo with GitLab](../user/project/integrations/bamboo.md) +1. [How to Integrate Slack with GitLab](../user/project/integrations/slack.md) 1. [How to Integrate Convox with GitLab](https://about.gitlab.com/2016/06/09/continuous-delivery-with-gitlab-and-convox/) 1. [Getting Started with GitLab and Shippable CI](https://about.gitlab.com/2016/05/05/getting-started-gitlab-and-shippable/) diff --git a/doc/university/glossary/README.md b/doc/university/glossary/README.md index 0af2f8d2f54..f15b0107de5 100644 --- a/doc/university/glossary/README.md +++ b/doc/university/glossary/README.md @@ -62,7 +62,7 @@ Entry level [subscription](https://about.gitlab.com/pricing/) for GitLab EE curr ### Bitbucket -Atlassian's web hosting service for Git and Mercurial Projects. Read about [migrating](https://docs.gitlab.com/ce/workflow/importing/import_projects_from_bitbucket.html) from BitBucket to a GitLab instance. +Atlassian's web hosting service for Git and Mercurial Projects. Read about [migrating](../../user/project/import/bitbucket.md) from BitBucket to a GitLab instance. ### Branch @@ -70,10 +70,10 @@ A branch is a parallel version of a repository. This allows you to work on the r ### Branded Login -Having your own logo on [your GitLab instance login page](https://docs.gitlab.com/ee/customization/branded_login_page.html) instead of the GitLab logo. +Having your own logo on [your GitLab instance login page](../../customization/branded_login_page.md) instead of the GitLab logo. ### Job triggers (Build Triggers) -These protect your code base against breaks, for instance when a team is working on the same project. Learn about [setting up](https://docs.gitlab.com/ce/ci/triggers/README.html) job triggers. +These protect your code base against breaks, for instance when a team is working on the same project. Learn about [setting up](../../ci/triggers/README.md) job triggers. ### CEPH @@ -149,7 +149,7 @@ As in "specify [dependencies](https://gitlab.com/gitlab-org/gitlab-ce/issues/147 ### Deploy Keys -A [SSH key](https://docs.gitlab.com/ce/gitlab-basics/create-your-ssh-keys.html)stored on your server that grants access to a single GitLab repository. This is used by a GitLab runner to clone a project's code so that tests can be run against the checked out code. +A [SSH key](../../gitlab-basics/create-your-ssh-keys.md)stored on your server that grants access to a single GitLab repository. This is used by a GitLab runner to clone a project's code so that tests can be run against the checked out code. ### Developer @@ -169,7 +169,7 @@ A folder used for storing multiple files. ### Docker Container Registry -A [feature](https://docs.gitlab.com/ce/user/project/container_registry.html) of [GitLab projects](https://about.gitlab.com/2016/05/23/gitlab-container-registry/). Containers wrap up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, system libraries – anything you can install on a server. This guarantees that it will always run the same, regardless of the environment it is running in. +A [feature](../../user/project/container_registry.md) of [GitLab projects](https://about.gitlab.com/2016/05/23/gitlab-container-registry/). Containers wrap up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, system libraries – anything you can install on a server. This guarantees that it will always run the same, regardless of the environment it is running in. ### Dynamic Environment (review apps) @@ -191,7 +191,7 @@ First Byte (sometimes referred to as time to first byte or [TTFB](https://en.wik ### Fork -Your [own copy](https://docs.gitlab.com/ce/workflow/forking_workflow.html) of a repository that allows you to make changes to the repository without affecting the original. +Your [own copy](../../workflow/forking_workflow.md) of a repository that allows you to make changes to the repository without affecting the original. ### Funnel, or: TOFU, MOFU, BOFU @@ -221,7 +221,7 @@ A single-tenant solution that provides GitLab CE or EE as a managed service. Git ### GitHub -A web-based Git repository hosting service with an enterprise offering. Its main features are: issue tracking, pull request with code review, abundancy of integrations and wiki. It offers free public repos, private repos and enterprise services are paid. Read about [importing a project](https://docs.gitlab.com/ce/workflow/importing/import_projects_from_github.html) from GitHub to GitLab. +A web-based Git repository hosting service with an enterprise offering. Its main features are: issue tracking, pull request with code review, abundancy of integrations and wiki. It offers free public repos, private repos and enterprise services are paid. Read about [importing a project](../../workflow/importing/import_projects_from_github.md) from GitHub to GitLab. ### GitLab CE @@ -241,7 +241,7 @@ Our free SaaS for public and private repositories. ### GitLab Geo -Allows you to replicate your GitLab instance to other geographical locations as a read-only fully operational version. It [can be used](https://docs.gitlab.com/ee/administration/geo/replication/index.html) for cloning and fetching projects, in addition to reading any data. This will make working with large repositories over large distances much faster. +Allows you to replicate your GitLab instance to other geographical locations as a read-only fully operational version. It [can be used](../../administration/geo/replication/index.md) for cloning and fetching projects, in addition to reading any data. This will make working with large repositories over large distances much faster. ### GitLab High Availability @@ -299,15 +299,15 @@ An [application layer protocol](http://www.irchelp.org/) that facilitates commun ### Issue Tracker -A [tool](https://docs.gitlab.com/ee/integration/external-issue-tracker.html) used to manage, organize, and maintain a list of issues, making it easier for an organization to manage. +A [tool](../../integration/external-issue-tracker.md) used to manage, organize, and maintain a list of issues, making it easier for an organization to manage. ### Jenkins -An Open Source CI tool written using the Java programming language. [Jenkins](https://jenkins.io/) does the same job as GitLab CI, Bamboo, and Travis CI. It is extremely popular. Related [documentation](https://docs.gitlab.com/ee/integration/jenkins.html). +An Open Source CI tool written using the Java programming language. [Jenkins](https://jenkins.io/) does the same job as GitLab CI, Bamboo, and Travis CI. It is extremely popular. Related [documentation](../../integration/jenkins.md). ### Jira -Atlassian's [project management software](https://www.atlassian.com/software/jira), i.e. a complex issue tracker. GitLab [can be configured](https://docs.gitlab.com/ee/project_services/jira.html) to interact with JIRA Core either using an on-premise instance or the SaaS solution that Atlassian offers. +Atlassian's [project management software](https://www.atlassian.com/software/jira), i.e. a complex issue tracker. GitLab [can be configured](../../project_services/jira.md) to interact with JIRA Core either using an on-premise instance or the SaaS solution that Atlassian offers. ### JUnit @@ -323,7 +323,7 @@ An open source container cluster manager originally designed by Google. It's bas ### Labels -An [identifier](https://docs.gitlab.com/ce/user/project/labels.html) to describe a group of one or more specific file revisions. +An [identifier](../../user/project/labels.md) to describe a group of one or more specific file revisions. ### Lightweight Directory Access Protocol (LDAP) @@ -331,7 +331,7 @@ An [identifier](https://docs.gitlab.com/ce/user/project/labels.html) to describe ### LDAP User Authentication -GitLab [integrates](https://docs.gitlab.com/ce/administration/auth/ldap.html) with LDAP to support user authentication. This enables GitLab to sign in people from an LDAP server (i.e., allowing people whose names are on the electronic user directory server to be able to use their LDAP accounts to login.) +GitLab [integrates](../../administration/auth/ldap.md) with LDAP to support user authentication. This enables GitLab to sign in people from an LDAP server (i.e., allowing people whose names are on the electronic user directory server to be able to use their LDAP accounts to login.) ### LDAP Group Sync @@ -381,9 +381,9 @@ Takes changes from one branch, and [applies them](https://git-scm.com/docs/git-m [Arises](https://about.gitlab.com/2016/09/06/resolving-merge-conflicts-from-the-gitlab-ui/) when a merge can't be performed cleanly between two versions of the same file. -#### Merge Request +#### Merge Request (MR) -[Takes changes](https://docs.gitlab.com/ce/gitlab-basics/add-merge-request.html) from one branch, and applies them into another branch. +[Takes changes](../../gitlab-basics/add-merge-request.md) from one branch, and applies them into another branch. ### Meteor @@ -395,11 +395,11 @@ Allow you to [organize issues](../../user/project/milestones/index.md) and merge ### Mirror Repositories -A project that is set up to automatically have its branches, tags, and commits [updated from an upstream repository](https://docs.gitlab.com/ee/workflow/repository_mirroring.html). This is useful when a repository you're interested in is located on a different server, and you want to be able to browse its content and activity using the familiar GitLab interface. +A project that is set up to automatically have its branches, tags, and commits [updated from an upstream repository](../../workflow/repository_mirroring.md). This is useful when a repository you're interested in is located on a different server, and you want to be able to browse its content and activity using the familiar GitLab interface. ### MIT License -A type of software license. It lets people do anything with your code with proper attribution and without warranty. It is the most common license for open source applications written in Ruby on Rails. GitLab CE is issued under this [license](https://docs.gitlab.com/ce/development/licensing.html). This means you can download the code, modify it as you want, and even build a new commercial product using the underlying code and it's not illegal. The only condition is that there is no form of warranty provided by GitLab so whatever happens when you use the code is your own problem. +A type of software license. It lets people do anything with your code with proper attribution and without warranty. It is the most common license for open source applications written in Ruby on Rails. GitLab CE is issued under this [license](../../development/licensing.md). This means you can download the code, modify it as you want, and even build a new commercial product using the underlying code and it's not illegal. The only condition is that there is no form of warranty provided by GitLab so whatever happens when you use the code is your own problem. ### Mondo Rescue @@ -427,7 +427,7 @@ A web [server](https://www.nginx.com/resources/wiki/) (pronounced "engine x"). [ ### OAuth -An open standard for authorization, commonly used as a way for internet users to log into third party websites using their Microsoft, Google, Facebook or Twitter accounts without exposing their password. GitLab [is](https://docs.gitlab.com/ce/integration/oauth_provider.html) an OAuth2 authentication service provider. +An open standard for authorization, commonly used as a way for internet users to log into third party websites using their Microsoft, Google, Facebook or Twitter accounts without exposing their password. GitLab [is](../../integration/oauth_provider.md) an OAuth2 authentication service provider. ### Omnibus Packages @@ -479,11 +479,11 @@ An [object-relational](https://en.wikipedia.org/wiki/PostgreSQL) database. Toute ### Protected Branches -A [feature](https://docs.gitlab.com/ce/user/project/protected_branches.html) that protects branches from unauthorized pushes, force pushing or deletion. +A [feature](../../user/project/protected_branches.md) that protects branches from unauthorized pushes, force pushing or deletion. ### Protected Tags -A [feature](https://docs.gitlab.com/ce/user/project/protected_tags.html) that protects tags from unauthorized creation, update or deletion +A [feature](../../user/project/protected_tags.md) that protects tags from unauthorized creation, update or deletion ### Pull @@ -547,7 +547,7 @@ Actual build machines/containers that [run and execute tests](https://gitlab.com ### Sidekiq -The background job processor GitLab [uses](https://docs.gitlab.com/ce/administration/troubleshooting/sidekiq.html) to asynchronously run tasks. +The background job processor GitLab [uses](../../administration/troubleshooting/sidekiq.md) to asynchronously run tasks. ### Software as a service (SaaS) @@ -567,7 +567,7 @@ The board used to track the status and progress of each of the sprint backlog it ### Shell -Terminal on Mac OSX, GitBash on Windows, or Linux Terminal on Linux. You [use git](https://docs.gitlab.com/ce/gitlab-basics/start-using-git.html) and make changes to GitLab projects in your shell. You [use git](https://docs.gitlab.com/ce/gitlab-basics/start-using-git.html) and make changes to GitLab projects in your shell. +Terminal on Mac OSX, GitBash on Windows, or Linux Terminal on Linux. You [use git](../../gitlab-basics/start-using-git.md) and make changes to GitLab projects in your shell. You [use git](../../gitlab-basics/start-using-git.md) and make changes to GitLab projects in your shell. ### Shell command runner @@ -577,7 +577,7 @@ The tenant purchases their own copy of the software and the software can be cust ### Slack -Real time messaging app for teams that is used internally by GitLab team members. GitLab users can enable [Slack integration](https://docs.gitlab.com/ce/project_services/slack.html) to trigger push, issue, and merge request events among others. +Real time messaging app for teams that is used internally by GitLab team members. GitLab users can enable [Slack integration](../../project_services/slack.md) to trigger push, issue, and merge request events among others. ### Slash commands @@ -595,7 +595,7 @@ Program code as typed by a computer programmer (i.e. it has not yet been compile ### SSH Key -A unique identifier of a computer. It is used to identify computers without the need for a password (e.g., On GitLab I have [added the ssh key](https://docs.gitlab.com/ce/gitlab-basics/create-your-ssh-keys.html) of all my work machines so that the GitLab instance knows that it can accept code pushes and pulls from this trusted machines whose keys are I have added.) +A unique identifier of a computer. It is used to identify computers without the need for a password (e.g., On GitLab I have [added the ssh key](../../gitlab-basics/create-your-ssh-keys.md) of all my work machines so that the GitLab instance knows that it can accept code pushes and pulls from this trusted machines whose keys are I have added.) ### Single Sign On (SSO) @@ -627,11 +627,11 @@ A program that allows you to perform superuser/administrator actions on Unix Ope ### Subversion (SVN) -An open source version control system. Read about [migrating from SVN](https://docs.gitlab.com/ce/workflow/importing/migrating_from_svn.html) to GitLab using SubGit. +An open source version control system. Read about [migrating from SVN](../../workflow/importing/migrating_from_svn.md) to GitLab using SubGit. ### Tag -[Represents](https://docs.gitlab.com/ce/api/tags.html) a version of a particular branch at a moment in time. +[Represents](../../api/tags.md) a version of a particular branch at a moment in time. ### Tenancy @@ -673,7 +673,7 @@ Version control is a system that records changes to a file or set of files over ### Virtual Private Cloud (VPC) -A [VPC](https://docs.gitlab.com/ce/university/glossary/README.html#virtual-private-cloud-vpc) is an on demand configurable pool of shared computing resources allocated within a public cloud environment, providing some isolation between the different users using the resources. GitLab users need to create a new Amazon VPC in order to [set up High Availability](https://docs.gitlab.com/ce/university/high-availability/aws/). +A [VPC](#virtual-private-cloud-vpc) is an on demand configurable pool of shared computing resources allocated within a public cloud environment, providing some isolation between the different users using the resources. GitLab users need to create a new Amazon VPC in order to [set up High Availability](../../install/aws/index.md). ### Virtual private server (VPS) @@ -689,7 +689,7 @@ A [model](http://www.umsl.edu/~hugheyd/is6840/waterfall.html) of building softwa ### Webhooks -A way for an app to [provide](https://docs.gitlab.com/ce/user/project/integrations/webhooks.html) other applications with real-time information (e.g., send a message to a slack channel when a commit is pushed.) Read about setting up [custom git hooks](https://gitlab.com/help/administration/custom_hooks.md) for when webhooks are insufficient. +A way for an app to [provide](../../user/project/integrations/webhooks.md) other applications with real-time information (e.g., send a message to a slack channel when a commit is pushed.) Read about setting up [custom git hooks](../../administration/custom_hooks.md) for when webhooks are insufficient. ### Wiki diff --git a/doc/university/high-availability/aws/README.md b/doc/university/high-availability/aws/README.md index 01d5073ab7e..fa04e988042 100644 --- a/doc/university/high-availability/aws/README.md +++ b/doc/university/high-availability/aws/README.md @@ -390,5 +390,5 @@ some redundancy options but it might also imply Geographic replication. There is a lot of ground yet to cover so have a read through these other resources and feel free to open an issue to request additional material. -- [GitLab High Availability](http://docs.gitlab.com/ce/administration/high_availability/README.html#sts=High%20Availability) -- [GitLab Geo](https://docs.gitlab.com/ee/administration/geo/replication/index.html) +- [GitLab High Availability](../../../administration/high_availability/README.md) +- [GitLab Geo](../../../administration/geo/replication/index.md) diff --git a/doc/university/support/README.md b/doc/university/support/README.md index feb90ae9bad..c8ade54a77c 100644 --- a/doc/university/support/README.md +++ b/doc/university/support/README.md @@ -61,28 +61,27 @@ Sometimes we need to upgrade customers from old versions of GitLab to latest, so - Users - Groups - Projects - - [Backup using our Backup rake task](https://docs.gitlab.com/ce/raketasks/backup_restore.html#create-a-backup-of-the-gitlab-system) + - [Backup using our Backup rake task](../../raketasks/backup_restore.md#creating-a-backup-of-the-gitlab-system) - [Upgrade to 5.0 source using our Upgrade documentation](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/doc/update/4.2-to-5.0.md) - [Upgrade to 5.1 source](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/doc/update/5.0-to-5.1.md) - [Upgrade to 6.0 source](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/doc/update/5.1-to-6.0.md) - [Upgrade to 7.14 source](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/doc/update/6.x-or-7.x-to-7.14.md) - - [Backup using our Backup rake task](https://docs.gitlab.com/ce/raketasks/backup_restore.html#create-a-backup-of-the-gitlab-system) - - [Perform the MySQL to PostgreSQL migration to convert your backup](https://docs.gitlab.com/ce/update/mysql_to_postgresql.html#converting-a-gitlab-backup-file-from-mysql-to-postgres) + - [Perform the MySQL to PostgreSQL migration to convert your backup](../../update/mysql_to_postgresql.md) - [Upgrade to Omnibus 7.14](https://docs.gitlab.com/omnibus/update/README.html#upgrading-from-a-non-omnibus-installation-to-an-omnibus-installation) - - [Restore backup using our Restore rake task](https://docs.gitlab.com/ce/raketasks/backup_restore.html#restore-a-previously-created-backup) + - [Restore backup using our Restore rake task](../../raketasks/backup_restore.md#restore) - [Upgrade to latest EE](https://about.gitlab.com/downloads-ee) - (GitLab inc. only) Acquire and apply a license for the Enterprise Edition product, ask in #support -- Perform a downgrade from [EE to CE](https://docs.gitlab.com/ee/downgrade_ee_to_ce/README.html) +- Perform a downgrade from [EE to CE](../../downgrade_ee_to_ce/README.md) #### Start to learn about some of the integrations that we support Our integrations add great value to GitLab. User questions often relate to integrating GitLab with existing external services and the configuration involved - Learn about our Integrations (specially, not only): - - [LDAP](https://docs.gitlab.com/ee/integration/ldap.html) - - [JIRA](https://docs.gitlab.com/ee/project_services/jira.html) - - [Jenkins](https://docs.gitlab.com/ee/integration/jenkins.html) - - [SAML](https://docs.gitlab.com/ce/integration/saml.html) + - [LDAP](../../integration/ldap.md) + - [JIRA](../../project_services/jira.md) + - [Jenkins](../../integration/jenkins.md) + - [SAML](../../integration/saml.md) #### Goals @@ -94,8 +93,8 @@ Our integrations add great value to GitLab. User questions often relate to integ #### Understand the gathering of diagnostics for GitLab instances - Learn about the GitLab checks that are available - - [Environment Information and maintenance checks](https://docs.gitlab.com/ce/raketasks/maintenance.html) - - [GitLab check](https://docs.gitlab.com/ce/raketasks/check.html) + - [Environment Information and maintenance checks](../../raketasks/maintenance.md) + - [GitLab check](../../raketasks/check.md) - Omnibus commands - [Status](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/maintenance/README.md#get-service-status) - [Starting and stopping services](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/maintenance/README.md#starting-and-stopping) @@ -170,11 +169,11 @@ Some tickets need specific knowledge or a deep understanding of a particular com Move on to understanding some of GitLab's more advanced features. You can make use of GitLab.com to understand the features from an end-user perspective and then use your own instance to understand setup and configuration of the feature from an Administrative perspective -- Set up and try [Git LFS](https://docs.gitlab.com/ee/workflow/lfs/manage_large_binaries_with_git_lfs.html) -- Get to know the [GitLab API](https://docs.gitlab.com/ee/api/README.html), its capabilities and shortcomings -- Learn how to [migrate from SVN to Git](https://docs.gitlab.com/ee/workflow/importing/migrating_from_svn.html) -- Set up [GitLab CI](https://docs.gitlab.com/ee/ci/quick_start/README.html) -- Create your first [GitLab Page](https://docs.gitlab.com/ce/administration/pages/) +- Set up and try [Git LFS](../../workflow/lfs/manage_large_binaries_with_git_lfs.md) +- Get to know the [GitLab API](../../api/README.md), its capabilities and shortcomings +- Learn how to [migrate from SVN to Git](../../user/project/import/svn.md) +- Set up [GitLab CI](../../ci/quick_start/README.md) +- Create your first [GitLab Page](../../administration/pages/index.md) - Get to know the GitLab Codebase by reading through the source code: - Find the differences between the [EE codebase](https://gitlab.com/gitlab-org/gitlab-ce) and the [CE codebase](https://gitlab.com/gitlab-org/gitlab-ce) diff --git a/doc/university/training/topics/unstage.md b/doc/university/training/topics/unstage.md index da36a3218e5..c926f0b4888 100644 --- a/doc/university/training/topics/unstage.md +++ b/doc/university/training/topics/unstage.md @@ -8,13 +8,13 @@ comments: false ## Unstage -- To remove files from stage use reset HEAD. Where HEAD is the last commit of the current branch. +- To remove files from stage use reset HEAD where HEAD is the last commit of the current branch. This will unstage the file but maintain the modifications. ```bash git reset HEAD <file> ``` -- This will unstage the file but maintain the modifications. To revert the file back to the state it was in before the changes we can use: +- To revert the file back to the state it was in before the changes we can use: ```bash git checkout -- <file> diff --git a/doc/update/patch_versions.md b/doc/update/patch_versions.md index c2dff21b028..f2df4277ca8 100644 --- a/doc/update/patch_versions.md +++ b/doc/update/patch_versions.md @@ -108,7 +108,7 @@ sudo -u git -H make ### 8. Install/Update `gitlab-elasticsearch-indexer` (optional) **[STARTER ONLY]** -If you're interested in using GitLab's new [elasticsearch repository indexer](https://docs.gitlab.com/ee/integration/elasticsearch.html#elasticsearch-repository-indexer-beta) (currently in beta) +If you're interested in using GitLab's new [elasticsearch repository indexer](../integration/elasticsearch.md#elasticsearch-repository-indexer-beta) (currently in beta) please follow the instructions on the document linked above and enable the indexer usage in the GitLab admin settings. diff --git a/doc/update/upgrading_from_ce_to_ee.md b/doc/update/upgrading_from_ce_to_ee.md index e74a5c00f7e..428377adb19 100644 --- a/doc/update/upgrading_from_ce_to_ee.md +++ b/doc/update/upgrading_from_ce_to_ee.md @@ -75,7 +75,7 @@ sudo -u git -H bundle exec rake assets:clean assets:precompile cache:clear RAILS ### 4. Install `gitlab-elasticsearch-indexer` (optional) **[STARTER ONLY]** If you're interested in using GitLab's new [elasticsearch repository -indexer][indexer-beta] (currently in beta) please follow the instructions on the +indexer](../integration/elasticsearch.md) (currently in beta) please follow the instructions on the document linked above and enable the indexer usage in the GitLab admin settings. ### 5. Start application @@ -133,4 +133,3 @@ Additional instructions here. [support@gitlab.com]: mailto:support@gitlab.com [old-ee-upgrade-docs]: https://gitlab.com/gitlab-org/gitlab-ee/tree/11-8-stable-ee/doc/update -[indexer-beta]: https://docs.gitlab.com/ee/integration/elasticsearch.html diff --git a/doc/user/admin_area/diff_limits.md b/doc/user/admin_area/diff_limits.md index 9205860ef1f..4063c40a751 100644 --- a/doc/user/admin_area/diff_limits.md +++ b/doc/user/admin_area/diff_limits.md @@ -1,21 +1,40 @@ +--- +type: reference +--- + # Diff limits administration +You can set a maximum size for display of diff files (patches). + +## Maximum diff patch size + +Diff files which exceed this value will be presented as 'too large' and won't +be expandable. Instead of an expandable view, a link to the blob view will be +shown. + +Patches greater than 10% of this size will be automatically collapsed, and a +link to expand the diff will be presented. + NOTE: **Note:** Merge requests and branch comparison views will be affected. CAUTION: **Caution:** -These settings are currently under experimental state. They'll -increase the resource consumption of your instance and should -be edited mindfully. +This setting is experimental. An increased maximum will increase resource +consumption of your instance. Keep this in mind when adjusting the maximum. -1. Access **Admin area > Settings > General** -1. Expand **Diff limits** +1. Go to **Admin area > Settings > General**. +1. Expand **Diff limits**. +1. Enter a value for **Maximum diff patch size**, measured in bytes. +1. Click on **Save changes**. -### Maximum diff patch size +<!-- ## Troubleshooting -This is the content size each diff file (patch) is allowed to reach before -it's collapsed, without the possibility of being expanded. A link redirecting -to the blob view will be presented for the patches that surpass this limit. +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. -Patches surpassing 10% of this content size will be automatically collapsed, -but expandable (a link to expand the diff will be presented). +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/admin_area/geo_nodes.md b/doc/user/admin_area/geo_nodes.md index 776ab139c64..d99b87cbc5c 100644 --- a/doc/user/admin_area/geo_nodes.md +++ b/doc/user/admin_area/geo_nodes.md @@ -1,9 +1,13 @@ +--- +type: howto +--- + # Geo nodes admin area **[PREMIUM ONLY]** -For more information about setting up GitLab Geo, read the -[Geo documentation](https://docs.gitlab.com/ee/administration/geo/replication/index.html). +You can configure various settings for GitLab Geo nodes. For more information, see +[Geo documentation](../../administration/geo/replication/index.md). -When you're done, you can navigate to **Admin area > Geo** (`/admin/geo/nodes`). +On the primary node, go to **Admin area > Geo**. On secondary nodes, go to **Admin area > Geo > Nodes**. ## Common settings @@ -25,7 +29,7 @@ changes on the **primary** node! | Setting | Description | |---------------------------|-------------| -| Selective synchronization | Enable Geo [selective sync](https://docs.gitlab.com/ee/administration/geo/replication/configuration.html#selective-synchronization) for this **secondary** node. | +| Selective synchronization | Enable Geo [selective sync](../../administration/geo/replication/configuration.md#selective-synchronization) for this **secondary** node. | | Repository sync capacity | Number of concurrent requests this **secondary** node will make to the **primary** node when backfilling repositories. | | File sync capacity | Number of concurrent requests this **secondary** node will make to the **primary** node when backfilling files. | @@ -68,3 +72,15 @@ a unique `name` is set for each Geo node. The `gitlab.rb` setting The load balancer must use sticky sessions in order to avoid authentication failures and cross site request errors. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. -->
\ No newline at end of file diff --git a/doc/user/admin_area/img/index_runners_search_or_filter.png b/doc/user/admin_area/img/index_runners_search_or_filter.png Binary files differnew file mode 100755 index 00000000000..5176a1a39bf --- /dev/null +++ b/doc/user/admin_area/img/index_runners_search_or_filter.png diff --git a/doc/user/admin_area/index.md b/doc/user/admin_area/index.md index d2995d48833..527110d53df 100644 --- a/doc/user/admin_area/index.md +++ b/doc/user/admin_area/index.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # GitLab Admin Area **[CORE ONLY]** The Admin Area provides a web UI for administering some features of GitLab self-managed instances. @@ -16,14 +20,14 @@ The Admin Area is made up of the following sections: | Section | Description | |:---------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------| -| Overview | View your GitLab [Dashboard](#admin-dashboard), and administer [projects](#administer-projects), users, groups, jobs, runners, and Gitaly servers. | +| Overview | View your GitLab [Dashboard](#admin-dashboard), and administer [projects](#administer-projects), [users](#administer-users), groups, [jobs](#administer-jobs), [Runners](#administer-runners), and [Gitaly servers](#administer-gitaly-servers). | | Monitoring | View GitLab system information, and information on background jobs, logs, [health checks](monitoring/health_check.md), request profiles, and audit logs. | | Messages | Send and manage [broadcast messages](broadcast_messages.md) for your users. | | System Hooks | Configure [system hooks](../../system_hooks/system_hooks.md) for many events. | | Applications | Create system [OAuth applications](../../integration/oauth_provider.md) for integrations with other services. | | Abuse Reports | Manage [abuse reports](abuse_reports.md) submitted by your users. | | License **[STARTER ONLY]** | Upload, display, and remove [licenses](license.md). | -| Push Rules **[STARTER]** | Configure pre-defined git [push rules](https://docs.gitlab.com/ee/push_rules/push_rules.html) for projects. | +| Push Rules **[STARTER]** | Configure pre-defined git [push rules](../../push_rules/push_rules.md) for projects. | | Geo **[PREMIUM ONLY]** | Configure and maintain [Geo nodes](geo_nodes.md). | | Deploy Keys | Create instance-wide [SSH deploy keys](../../ssh/README.md#deploy-keys). | | Service Templates | Create [service templates](../project/integrations/services_templates.md) for projects. | @@ -90,3 +94,115 @@ You can combine the filter options. For example, to list only public projects wi 1. Click the **Public** tab. 1. Enter `score` in the **Filter by name...** input box. + +## Administer Users + +You can administer all users in the GitLab instance from the Admin Area's Users page. + +To access the Users page, go to **Admin Area > Overview > Users**. + +Click the **Active**, **Admins**, **2FA Enabled**, or **2FA Disabled**, **External**, or +**Without projects** tab to list only users of that criteria. + +For each user, their username, email address, are listed, also the date their account was +created and the date of last activity. To edit a user, click the **Edit** button in that user's +row. To delete the user, or delete the user and their contributions, click the cog dropdown in +that user's row, and select the desired option. + +To change the sort order: + +1. Click the sort dropdown. +1. Select the desired order. + +By default the sort dropdown shows **Name**. + +To search for users, enter your criteria in the search field. The user search is case +insensitive, and applies partial matching to name and username. To search for an email address, +you must provide the complete email address. + +## Administer Jobs + +You can administer all jobs in the GitLab instance from the Admin Area's Jobs page. + +To access the Jobs page, go to **Admin Area > Overview > Jobs**. + +All jobs are listed, in reverse order of their job ID. + +Click the **All** tab to list all jobs. Click the **Pending**, **Running**, or **Finished** tab to list only jobs of that status. + +For each job, the following details are listed: + +| Field | Description | +|--------- | ----------- | +| Status | Job status, either **passed**, **skipped**, or **failed**. | +| Job | Includes links to the job, branch, and the commit that started the job. | +| Pipeline | Includes a link to the specific pipeline. | +| Project | Name of the project, and organization, to which the job belongs. | +| Runner | Name of the CI runner assigned to execute the job. | +| Stage | Stage that the job is declared in a `.gitlab-ci.yml` file. | +| Name | Name of the job specified in a `.gitlab-ci.yml` file. | +| Timing | Duration of the job, and how long ago the job completed. | +| Coverage | Percentage of tests coverage. | + +## Administer Runners + +You can adminster all Runners in the GitLab instance from the Admin Area's **Runners** page. See +[GitLab Runner](https://docs.gitlab.com/runner/) for more information on Runner itself. + +To access the **Runners** page, go to **Admin Area > Overview > Runners**. + +The **Runners** page features: + +- A description of Runners, and their possible states. +- Instructions on installing a Runner. +- A list of all registered Runners. + +Runners are listed in descending order by the date they were created, by default. You can change +the sort order to *Last Contacted* from the dropdown beside the search field. + +To search Runners' descriptions: + +1. In the **Search or filter results...** field, type the description of the Runner you want to +find. +1. Press Enter. + +You can also filter Runners by status, type, and tag. To filter: + +1. Click in the **Search or filter results...** field. +1. Select **status:**, **type:**, or **tag:** +1. Select or enter your search criteria. + + + +For each Runner, the following attributes are listed: + +| Attribute | Description | +| ------------ | ----------- | +| Type | One or more of the following states: shared, group, specific, locked, or paused | +| Runner token | Token used to identify the Runner, and which the Runner uses to communicate with the GitLab instance | +| Description | Description given to the Runner when it was created | +| Version | GitLab Runner version | +| IP address | IP address of the host on which the Runner is registered | +| Projects | Projects to which the Runner is assigned | +| Jobs | Total of jobs run by the Runner | +| Tags | Tags associated with the Runner | +| Last contact | Timestamp indicating when the GitLab instance last contacted the Runner | + +You can also edit, pause, or remove each Runner. + +## Administer Gitaly servers + +You can list all Gitaly servers in the GitLab instance from the Admin Area's **Gitaly Servers** +page. For more details, see [Gitaly](../../administration/gitaly/index.md). + +To access the **Gitaly Servers** page, go to **Admin Area > Overview > Gitaly Servers**. + +For each Gitaly server, the following details are listed: + +| Field | Description | +| -------------- | ----------- | +| Storage | Repository storage | +| Address | Network address on which the Gitaly server is listening | +| Server version | Gitaly version | +| Git version | Version of Git installed on the Gitaly server | +| Up to date | Indicates if the Gitaly server version is the latest version available. A green dot indicates the server is up to date. | diff --git a/doc/user/admin_area/labels.md b/doc/user/admin_area/labels.md index e383142c33e..eba27548f86 100644 --- a/doc/user/admin_area/labels.md +++ b/doc/user/admin_area/labels.md @@ -1,9 +1,25 @@ +--- +type: reference +--- + # Labels administration **[CORE ONLY]** -## Default Labels +In the Admin Area, you can manage labels for the GitLab instance. For more details, see [Labels](../project/labels.md). -### Define your own default Label Set +## Default Labels -Labels that are created within the Labels view on the Admin Dashboard will be automatically added to each new project. +Labels created in the Admin Area become available to each _new_ project.  + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/admin_area/license.md b/doc/user/admin_area/license.md index 49959a9daef..1e8ce04da92 100644 --- a/doc/user/admin_area/license.md +++ b/doc/user/admin_area/license.md @@ -1,3 +1,7 @@ +--- +type: howto +--- + # Activate all GitLab Enterprise Edition functionality with a license **[STARTER ONLY]** To activate all GitLab Enterprise Edition (EE) functionality, you need to upload @@ -108,3 +112,15 @@ but only the latest license will be used as the active license. [free trial]: https://about.gitlab.com/free-trial/ [pricing]: https://about.gitlab.com/pricing/ + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. -->
\ No newline at end of file diff --git a/doc/user/admin_area/monitoring/health_check.md b/doc/user/admin_area/monitoring/health_check.md index e183898dfb1..ff056490653 100644 --- a/doc/user/admin_area/monitoring/health_check.md +++ b/doc/user/admin_area/monitoring/health_check.md @@ -1,12 +1,14 @@ -# Health Check +--- +type: concepts, howto +--- -> **Notes:** +# Health Check -> - Liveness and readiness probes were [introduced][ce-10416] in GitLab 9.1. -> - The `health_check` endpoint was [introduced][ce-3888] in GitLab 8.8 and was -> be deprecated in GitLab 9.1. -> - [Access token](#access-token-deprecated) has been deprecated in GitLab 9.4 -> in favor of [IP whitelist](#ip-whitelist) +> - Liveness and readiness probes were [introduced][ce-10416] in GitLab 9.1. +> - The `health_check` endpoint was [introduced][ce-3888] in GitLab 8.8 and was +> be deprecated in GitLab 9.1. +> - [Access token](#access-token-deprecated) has been deprecated in GitLab 9.4 +> in favor of [IP whitelist](#ip-whitelist). GitLab provides liveness and readiness probes to indicate service health and reachability to required services. These probes report on the status of the @@ -17,8 +19,7 @@ traffic until the system is ready or restart the container as needed. ## IP whitelist To access monitoring resources, the requesting client IP needs to be included in a whitelist. - -[Read how to add IPs to a whitelist for the monitoring endpoints][admin]. +For details, see [how to add IPs to a whitelist for the monitoring endpoints](../../../administration/monitoring/ip_whitelist.md). ## Using the endpoints @@ -30,7 +31,7 @@ With default whitelist settings, the probes can be accessed from localhost using The first endpoint, `health`, only checks whether the application server is running. It does not verify the database or other services are running. A successful response will return a 200 status code with the following message: -``` +```text GitLab OK ``` @@ -87,9 +88,8 @@ will return a valid successful HTTP status code, and a `success` message. ## Access token (Deprecated) ->**Note:** -Access token has been deprecated in GitLab 9.4 -in favor of [IP whitelist](#ip-whitelist) +> NOTE: **Note:** +> Access token has been deprecated in GitLab 9.4 in favor of [IP whitelist](#ip-whitelist). An access token needs to be provided while accessing the probe endpoints. The current accepted token can be found under the **Admin area ➔ Monitoring ➔ Health check** @@ -99,14 +99,25 @@ accepted token can be found under the **Admin area ➔ Monitoring ➔ Health che The access token can be passed as a URL parameter: -``` +```text https://gitlab.example.com/-/readiness?token=ACCESS_TOKEN ``` +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> + [ce-10416]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10416 [ce-3888]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3888 [pingdom]: https://www.pingdom.com [nagios-health]: https://nagios-plugins.org/doc/man/check_http.html [newrelic-health]: https://docs.newrelic.com/docs/alerts/alert-policies/downtime-alerts/availability-monitoring [kubernetes]: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/ -[admin]: ../../../administration/monitoring/ip_whitelist.md diff --git a/doc/user/admin_area/settings/account_and_limit_settings.md b/doc/user/admin_area/settings/account_and_limit_settings.md index 4a5bfb6b677..1d355824760 100644 --- a/doc/user/admin_area/settings/account_and_limit_settings.md +++ b/doc/user/admin_area/settings/account_and_limit_settings.md @@ -1,25 +1,29 @@ +--- +type: reference +--- + # Account and limit settings ## Repository size limit **[STARTER]** -> [Introduced][ee-740] in [GitLab Enterprise Edition 8.12][ee-8.12]. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/740) in [GitLab Enterprise Edition 8.12](https://about.gitlab.com/2016/09/22/gitlab-8-12-released/#limit-project-size-ee). +> Available in [GitLab Starter](https://about.gitlab.com/pricing/). Repositories within your GitLab instance can grow quickly, especially if you are -using LFS. Their size can grow exponentially and eat up your storage device quite -fast. +using LFS. Their size can grow exponentially, rapidly consuming available storage. -In order to avoid this from happening, you can set a hard limit for your -repositories' size. This limit can be set globally, per group, or per project, -with per project limits taking the highest priority. +To avoid this from happening, you can set a hard limit for your repositories' size. +This limit can be set globally, per group, or per project, with per project limits +taking the highest priority. -There are numerous cases where you'll need to set up a limit for repository size. +There are numerous use cases where you might set up a limit for repository size. For instance, consider the following workflow: -1. Your team develops apps which demand large files to be stored in +1. Your team develops apps which require large files to be stored in the application repository. -1. Although you have enabled [Git LFS](../../../workflow/lfs/manage_large_binaries_with_git_lfs.html#git-lfs) +1. Although you have enabled [Git LFS](../../../workflow/lfs/manage_large_binaries_with_git_lfs.md#git-lfs) to your project, your storage has grown significantly. -1. Before you blow your storage limit up, you set up a limit of 10 GB +1. Before you exceed available storage, you set up a limit of 10 GB per repository. ### How it works @@ -42,12 +46,19 @@ subsequent push will be denied. LFS objects, however, can be checked on first push and **will** be rejected if the sum of their sizes exceeds the maximum allowed repository size. -For more manually purging the files, read the docs on -[reducing the repository size using Git][repo-size]. +For details on manually purging files, see [reducing the repository size using Git](../../project/repository/reducing_the_repo_size_using_git.md). + +NOTE: **Note:** +For GitLab.com, the repository size limit is 10 GB. + +<!-- ## Troubleshooting -> **Note:** -> For GitLab.com, the repository size limit is 10 GB. +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. -[ee-740]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/740 -[repo-size]: ../../project/repository/reducing_the_repo_size_using_git.md -[ee-8.12]: https://about.gitlab.com/2016/09/22/gitlab-8-12-released/#limit-project-size-ee +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/admin_area/settings/continuous_integration.md b/doc/user/admin_area/settings/continuous_integration.md index 9dd476656ed..6c4abce83c2 100644 --- a/doc/user/admin_area/settings/continuous_integration.md +++ b/doc/user/admin_area/settings/continuous_integration.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Continuous Integration and Deployment Admin settings **[CORE ONLY]** In this area, you will find settings for Auto DevOps, Runners and job artifacts. @@ -145,3 +149,15 @@ To set the duration for which the jobs will be considered as old and expired: Once that time passes, the jobs will be archived and no longer able to be retried. Make it empty to never expire jobs. It has to be no less than 1 day, for example: <code>15 days</code>, <code>1 month</code>, <code>2 years</code>. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. -->
\ No newline at end of file diff --git a/doc/user/admin_area/settings/email.md b/doc/user/admin_area/settings/email.md index 01a98cf15dc..9555a695b13 100644 --- a/doc/user/admin_area/settings/email.md +++ b/doc/user/admin_area/settings/email.md @@ -1,12 +1,18 @@ +--- +type: reference +--- + # Email +You can customize some of the content in emails sent from your GitLab instance. + ## Custom logo The logo in the header of some emails can be customized, see the [logo customization section](../../../customization/branded_page_and_email_header.md). ## Custom additional text **[PREMIUM ONLY]** ->[Introduced][ee-5031] in [GitLab Premium][eep] 10.7. +> [Introduced][ee-5031] in [GitLab Premium][eep] 10.7. The additional text will appear at the bottom of any email and can be used for legal/auditing/compliance reasons. @@ -24,8 +30,8 @@ legal/auditing/compliance reasons. > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/22560) in GitLab 11.5. -This configuration option sets the email hostname for [private commit emails](../../profile/index.md#private-commit-email), -and it's, by default, set to `users.noreply.YOUR_CONFIGURED_HOSTNAME`. +This configuration option sets the email hostname for [private commit emails](../../profile/index.md#private-commit-email). + By default it is set to `users.noreply.YOUR_CONFIGURED_HOSTNAME`. In order to change this option: @@ -34,5 +40,17 @@ In order to change this option: 1. Hit **Save** for the changes to take effect. NOTE: **Note**: Once the hostname gets configured, every private commit email using the previous hostname, will not get -recognized by GitLab. This can directly conflict with certain [Push rules](https://docs.gitlab.com/ee/push_rules/push_rules.html) such as +recognized by GitLab. This can directly conflict with certain [Push rules](../../../push_rules/push_rules.md) such as `Check whether author is a GitLab user` and `Check whether committer is the current authenticated user`. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. -->
\ No newline at end of file diff --git a/doc/user/admin_area/settings/external_authorization.md b/doc/user/admin_area/settings/external_authorization.md index 06e00e02f3d..11c0867da17 100644 --- a/doc/user/admin_area/settings/external_authorization.md +++ b/doc/user/admin_area/settings/external_authorization.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # External authorization control **[CORE ONLY]** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/4216) in @@ -108,5 +112,17 @@ The label will be shown on all project pages in the upper right corner.  +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> + [omnibus-ssl-docs]: https://docs.gitlab.com/omnibus/settings/ssl.html [omnibus-log-docs]: https://docs.gitlab.com/omnibus/settings/logs.html diff --git a/doc/user/admin_area/settings/instance_template_repository.md b/doc/user/admin_area/settings/instance_template_repository.md index 4010008f694..91286a67c31 100644 --- a/doc/user/admin_area/settings/instance_template_repository.md +++ b/doc/user/admin_area/settings/instance_template_repository.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Instance template repository **[PREMIUM ONLY]** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/5986) in @@ -61,3 +65,15 @@ top of the list. If this feature is disabled or no templates are present, there will be no "Custom" section in the selection dropdown. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/admin_area/settings/sign_up_restrictions.md b/doc/user/admin_area/settings/sign_up_restrictions.md index d3ecfd42903..cebf36c7ec1 100644 --- a/doc/user/admin_area/settings/sign_up_restrictions.md +++ b/doc/user/admin_area/settings/sign_up_restrictions.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Sign-up restrictions You can block email addresses of specific domains, or whitelist only some @@ -37,5 +41,17 @@ semicolon, comma, or a new line.  +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> + [ce-5259]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5259 [ce-598]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/598 diff --git a/doc/user/admin_area/settings/third_party_offers.md b/doc/user/admin_area/settings/third_party_offers.md index 23311801790..d3c9cf7d8ff 100644 --- a/doc/user/admin_area/settings/third_party_offers.md +++ b/doc/user/admin_area/settings/third_party_offers.md @@ -1,9 +1,26 @@ +--- +type: reference +--- + # Third party offers > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/20379) > in [GitLab Core](https://about.gitlab.com/pricing/) 11.1 -Within GitLab, we inform users of available third-party offers they might find valuable in order to enhance the development of their projects. -An example is the Google Cloud Platform free credit for using [Google Kubernetes Engine](https://cloud.google.com/kubernetes-engine/). +Within GitLab, we inform users of available third-party offers they might find valuable in order +to enhance the development of their projects. An example is the Google Cloud Platform free credit +for using [Google Kubernetes Engine](https://cloud.google.com/kubernetes-engine/). + +The display of third-party offers can be toggled in the **Admin Area > Settings** page. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. -The display of third-party offers can be toggled in the Admin area on the Settings page. +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/admin_area/settings/usage_statistics.md b/doc/user/admin_area/settings/usage_statistics.md index 8b5d80efb0d..01d1eb1cd0e 100644 --- a/doc/user/admin_area/settings/usage_statistics.md +++ b/doc/user/admin_area/settings/usage_statistics.md @@ -4,7 +4,7 @@ GitLab Inc. will periodically collect information about your instance in order to perform various actions. All statistics are opt-out, you can enable/disable them from the admin panel -under **Admin area > Settings > Usage statistics**. +under **Admin area > Settings > Metrics and profiling > Usage statistics**. ## Version check **[CORE ONLY]** diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md index 5c635b09503..adb6868516e 100644 --- a/doc/user/application_security/container_scanning/index.md +++ b/doc/user/application_security/container_scanning/index.md @@ -119,7 +119,7 @@ container_scanning: variables: DOCKER_DRIVER: overlay2 ## Define two new variables based on GitLab's CI/CD predefined variables - ## https://docs.gitlab.com/ee/ci/variables/#predefined-environment-variables + ## https://docs.gitlab.com/ee/ci/variables/README.html#predefined-environment-variables CI_APPLICATION_REPOSITORY: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG CI_APPLICATION_TAG: $CI_COMMIT_SHA CLAIR_LOCAL_SCAN_VERSION: v2.0.8_fe9b059d930314b54c78f75afe265955faf4fdc1 @@ -162,7 +162,7 @@ container_scanning: variables: DOCKER_DRIVER: overlay2 ## Define two new variables based on GitLab's CI/CD predefined variables - ## https://docs.gitlab.com/ee/ci/variables/#predefined-environment-variables + ## https://docs.gitlab.com/ee/ci/variables/README.html#predefined-environment-variables CI_APPLICATION_REPOSITORY: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG CI_APPLICATION_TAG: $CI_COMMIT_SHA CLAIR_LOCAL_SCAN_VERSION: v2.0.8_fe9b059d930314b54c78f75afe265955faf4fdc1 diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index 02c115b7f22..db328262aba 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -63,7 +63,7 @@ The following table shows which languages, package managers and frameworks are s | Javascript | [ESLint security plugin](https://github.com/nodesecurity/eslint-plugin-security) | 11.8 | | Node.js | [NodeJsScan](https://github.com/ajinabraham/NodeJsScan) | 11.1 | | PHP | [phpcs-security-audit](https://github.com/FloeDesignTechnologies/phpcs-security-audit) | 10.8 | -| Python ([pip](https://pip.pypa.io/en/stable/)) | [bandit](https://github.com/openstack/bandit) | 10.3 | +| Python ([pip](https://pip.pypa.io/en/stable/)) | [bandit](https://github.com/PyCQA/bandit) | 10.3 | | Ruby on Rails | [brakeman](https://brakemanscanner.org) | 10.3 | | Scala ([Ant](https://ant.apache.org/), [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) and [SBT](https://www.scala-sbt.org/)) | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 11.0 (SBT) & 11.9 (Ant, Gradle, Maven) | | Typescript | [TSLint config security](https://github.com/webschik/tslint-config-security/) | 11.9 | diff --git a/doc/user/application_security/security_dashboard/index.md b/doc/user/application_security/security_dashboard/index.md index ca31e15c65f..19eeb06a259 100644 --- a/doc/user/application_security/security_dashboard/index.md +++ b/doc/user/application_security/security_dashboard/index.md @@ -54,6 +54,7 @@ First, navigate to the Security Dashboard found under your group's Once you're on the dashboard, at the top you should see a series of filters for: - Severity +- Confidence - Report type - Project diff --git a/doc/user/clusters/applications.md b/doc/user/clusters/applications.md new file mode 100644 index 00000000000..97abe99fe62 --- /dev/null +++ b/doc/user/clusters/applications.md @@ -0,0 +1,263 @@ +# GitLab Managed Apps + +GitLab provides **GitLab Managed Apps**, a one-click install for various applications which can +be added directly to your configured cluster. These applications are +needed for [Review Apps](../../ci/review_apps/index.md) and +[deployments](../../ci/environments.md) when using [Auto DevOps](../../topics/autodevops/index.md). +You can install them after you +[create a cluster](../project/clusters/index.md#adding-and-creating-a-new-gke-cluster-via-gitlab). + +## Installing applications + +Applications managed by GitLab will be installed onto the `gitlab-managed-apps` namespace. +This namespace: + +- Is different from the namespace used for project deployments. +- Is created once. +- Has a non-configurable name. + +To see a list of available applications to install: + +1. For a: + - Project-level cluster, navigate to your project's **Operations > Kubernetes**. + - Group-level cluster, navigate to your group's **Kubernetes** page. + +Install Helm first as it's used to install other applications. + +NOTE: **Note:** +As of GitLab 11.6, Helm will be upgraded to the latest version supported +by GitLab before installing any of the applications. + +The following applications can be installed: + +- [Helm](#helm) +- [Ingress](#ingress) +- [Cert-Manager](#cert-manager) +- [Prometheus](#prometheus) +- [GitLab Runner](#gitlab-runner) +- [JupyterHub](#jupyterhub) +- [Knative](#knative) + +With the exception of Knative, the applications will be installed in a dedicated +namespace called `gitlab-managed-apps`. + +NOTE: **Note:** +Some applications are installable only for a project-level cluster. +Support for installing these applications in a group-level cluster is +planned for future releases. +For updates, see [the issue tracking +progress](https://gitlab.com/gitlab-org/gitlab-ce/issues/51989). + +CAUTION: **Caution:** +If you have an existing Kubernetes cluster with Helm already installed, +you should be careful as GitLab cannot detect it. In this case, installing +Helm via the applications will result in the cluster having it twice, which +can lead to confusion during deployments. + +### Helm + +> - Available for project-level clusters since GitLab 10.2. +> - Available for group-level clusters since GitLab 11.6. + +[Helm](https://docs.helm.sh/) is a package manager for Kubernetes and is +required to install all the other applications. It is installed in its +own pod inside the cluster which can run the `helm` CLI in a safe +environment. + +### Cert-Manager + +> - Available for project-level clusters since GitLab 11.6. +> - Available for group-level clusters since GitLab 11.6. + +[Cert-Manager](https://docs.cert-manager.io/en/latest/) is a native +Kubernetes certificate management controller that helps with issuing +certificates. Installing Cert-Manager on your cluster will issue a +certificate by [Let's Encrypt](https://letsencrypt.org/) and ensure that +certificates are valid and up-to-date. + +NOTE: **Note:** +The +[stable/cert-manager](https://github.com/helm/charts/tree/master/stable/cert-manager) +chart is used to install this application with a +[`values.yaml`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/vendor/cert_manager/values.yaml) +file. + +### GitLab Runner + +> - Available for project-level clusters since GitLab 10.6. +> - Available for group-level clusters since GitLab 11.10. + +[GitLab Runner](https://docs.gitlab.com/runner/) is the open source +project that is used to run your jobs and send the results back to +GitLab. It is used in conjunction with [GitLab +CI/CD](../../ci/README.md), the open-source continuous integration +service included with GitLab that coordinates the jobs. When installing +the GitLab Runner via the applications, it will run in **privileged +mode** by default. Make sure you read the [security +implications](../project/clusters/index.md/#security-implications) before doing so. + +NOTE: **Note:** +The +[runner/gitlab-runner](https://gitlab.com/charts/gitlab-runner) +chart is used to install this application with a +[`values.yaml`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/vendor/runner/values.yaml) +file. + +### Ingress + +> - Available for project-level clusters since GitLab 10.2. +> - Available for group-level clusters since GitLab 11.6. + +[Ingress](https://kubernetes.github.io/ingress-nginx/) can provide load +balancing, SSL termination, and name-based virtual hosting. It acts as a +web proxy for your applications and is useful if you want to use [Auto +DevOps] or deploy your own web apps. + +NOTE: **Note:** +The +[stable/nginx-ingress](https://github.com/helm/charts/tree/master/stable/nginx-ingress) +chart is used to install this application with a +[`values.yaml`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/vendor/ingress/values.yaml) +file. + +### JupyterHub + +> Available for project-level clusters since GitLab 11.0. + +[JupyterHub](https://jupyterhub.readthedocs.io/en/stable/) is a +multi-user service for managing notebooks across a team. [Jupyter +Notebooks](https://jupyter-notebook.readthedocs.io/en/latest/) provide a +web-based interactive programming environment used for data analysis, +visualization, and machine learning. + +Authentication will be enabled only for [project +members](../project/members/index.md) with [Developer or +higher](../permissions.md) access to the project. + +We use a [custom Jupyter +image](https://gitlab.com/gitlab-org/jupyterhub-user-image/blob/master/Dockerfile) +that installs additional useful packages on top of the base Jupyter. You +will also see ready-to-use DevOps Runbooks built with Nurtch's [Rubix library](https://github.com/amit1rrr/rubix). + +More information on +creating executable runbooks can be found in [our Nurtch +documentation](../project/clusters/runbooks/index.md#nurtch-executable-runbooks). Note that +Ingress must be installed and have an IP address assigned before +JupyterHub can be installed. + +NOTE: **Note:** +The +[jupyter/jupyterhub](https://jupyterhub.github.io/helm-chart/) +chart is used to install this application with a +[`values.yaml`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/vendor/jupyter/values.yaml) +file. + +### Knative + +> Available for project-level clusters since GitLab 11.5. + +[Knative](https://cloud.google.com/knative) provides a platform to +create, deploy, and manage serverless workloads from a Kubernetes +cluster. It is used in conjunction with, and includes +[Istio](https://istio.io) to provide an external IP address for all +programs hosted by Knative. + +You will be prompted to enter a wildcard +domain where your applications will be exposed. Configure your DNS +server to use the external IP address for that domain. For any +application created and installed, they will be accessible as +`<program_name>.<kubernetes_namespace>.<domain_name>`. This will require +your kubernetes cluster to have [RBAC +enabled](../project/clusters/index.md#rbac-cluster-resources). + +NOTE: **Note:** +The +[knative/knative](https://storage.googleapis.com/triggermesh-charts) +chart is used to install this application. + +### Prometheus + +> - Available for project-level clusters since GitLab 10.4. +> - Available for group-level clusters since GitLab 11.11. + +[Prometheus](https://prometheus.io/docs/introduction/overview/) is an +open-source monitoring and alerting system useful to supervise your +deployed applications. + +NOTE: **Note:** +The +[stable/prometheus](https://github.com/helm/charts/tree/master/stable/prometheus) +chart is used to install this application with a +[`values.yaml`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/vendor/prometheus/values.yaml) +file. + +## Upgrading applications + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/24789) +in GitLab 11.8. + +The applications below can be upgraded. + +| Application | GitLab version | +| ----------- | -------------- | +| Runner | 11.8+ | + +To upgrade an application: + +1. For a: + - Project-level cluster, navigate to your project's **Operations > Kubernetes**. + - Group-level cluster, navigate to your group's **Kubernetes** page. +1. Select your cluster. +1. If an upgrade is available, the **Upgrade** button is displayed. Click the button to upgrade. + +NOTE: **Note:** +Upgrades will reset values back to the values built into the `runner` +chart plus the values set by +[`values.yaml`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/vendor/runner/values.yaml) + +## Uninstalling applications + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/60665) in +> GitLab 11.11. + +The applications below can be uninstalled. + +| Application | GitLab version | Notes | +| ----------- | -------------- | ----- | +| Prometheus | 11.11+ | All data will be deleted and cannot be restored. | + +To uninstall an application: + +1. For a: + - Project-level cluster, navigate to your project's **Operations > Kubernetes**. + - Group-level cluster, navigate to your group's **Kubernetes** page. +1. Select your cluster. +1. Click the **Uninstall** button for the application. + +Support for uninstalling all applications is planned for progressive rollout. +To follow progress, see [the relevant +epic](https://gitlab.com/groups/gitlab-org/-/epics/1201). + +## Troubleshooting applications + +Applications can fail with the following error: + +```text +Error: remote error: tls: bad certificate +``` + +To avoid installation errors: + +- Before starting the installation of applications, make sure that time is synchronized + between your GitLab server and your Kubernetes cluster. +- Ensure certificates are not out of sync. When installing applications, GitLab expects a new cluster with no previous installation of Helm. + + You can confirm that the certificates match via `kubectl`: + + ```sh + kubectl get configmaps/values-content-configuration-ingress -n gitlab-managed-apps -o \ + "jsonpath={.data['cert\.pem']}" | base64 -d > a.pem + kubectl get secrets/tiller-secret -n gitlab-managed-apps -o "jsonpath={.data['ca\.crt']}" | base64 -d > b.pem + diff a.pem b.pem + ``` + diff --git a/doc/user/group/clusters/index.md b/doc/user/group/clusters/index.md index ff6aa4f5930..9c7b83252b0 100644 --- a/doc/user/group/clusters/index.md +++ b/doc/user/group/clusters/index.md @@ -12,33 +12,10 @@ your group, enabling you to use the same cluster across multiple projects. ## Installing applications -GitLab provides a one-click install for various applications that can be -added directly to your cluster. - -NOTE: **Note:** -Applications will be installed in a dedicated namespace called -`gitlab-managed-apps`. If you have added an existing Kubernetes cluster -with Tiller already installed, you should be careful as GitLab cannot -detect it. In this event, installing Tiller via the applications will -result in the cluster having it twice. This can lead to confusion during -deployments. - -| Application | GitLab version | Description | Helm Chart | -| ----------- | -------------- | ----------- | ---------- | -| [Helm Tiller](https://docs.helm.sh) | 11.6+ | Helm is a package manager for Kubernetes and is required to install all the other applications. It is installed in its own pod inside the cluster which can run the `helm` CLI in a safe environment. | n/a | -| [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress) | 11.6+ | Ingress can provide load balancing, SSL termination, and name-based virtual hosting. It acts as a web proxy for your applications and is useful if you want to use [Auto DevOps](../../../topics/autodevops/index.md) or deploy your own web apps. | [stable/nginx-ingress](https://github.com/helm/charts/tree/master/stable/nginx-ingress) | -| [Cert-Manager](https://docs.cert-manager.io/en/latest/) | 11.6+ | Cert-Manager is a native Kubernetes certificate management controller that helps with issuing certificates. Installing Cert-Manager on your cluster will issue a certificate by [Let's Encrypt](https://letsencrypt.org/) and ensure that certificates are valid and up-to-date. | [stable/cert-manager](https://github.com/helm/charts/tree/master/stable/cert-manager) | -| [Prometheus](https://prometheus.io/docs/introduction/overview/) | 11.11+ | Prometheus is an open-source monitoring and alerting system useful to supervise your deployed applications. | [stable/prometheus](https://github.com/helm/charts/tree/master/stable/prometheus) | -| [GitLab Runner](https://docs.gitlab.com/runner/) | 11.10+ | GitLab Runner is the open source project that is used to run your jobs and send the results back to GitLab. It is used in conjunction with [GitLab CI/CD](../../../ci/README.md), the open-source continuous integration service included with GitLab that coordinates the jobs. When installing the GitLab Runner via the applications, it will run in **privileged mode** by default. Make sure you read the [security implications](../../project/clusters/index.md#security-implications) before doing so. | [runner/gitlab-runner](https://gitlab.com/charts/gitlab-runner) | - -NOTE: **Note:** -Some [cluster -applications](../../project/clusters/index.md#installing-applications) -are installable only for a project-level cluster. Support for installing these -applications in a group-level cluster is planned for future releases. For updates, see: - -- Support installing [JupyterHub in group-level - clusters](https://gitlab.com/gitlab-org/gitlab-ce/issues/51989) +GitLab can install and manage some applications in your group-level +cluster. For more information on installing, upgrading, uninstalling, +and troubleshooting applications for your group cluster, see +[Gitlab Managed Apps](../../clusters/applications.md). ## RBAC compatibility @@ -110,7 +87,7 @@ The domain should have a wildcard DNS configured to the Ingress IP address. When adding more than one Kubernetes cluster to your project, you need to differentiate them with an environment scope. The environment scope associates clusters with [environments](../../../ci/environments.md) similar to how the -[environment-specific variables](https://docs.gitlab.com/ee/ci/variables/#limiting-environment-scopes-of-environment-variables-premium) +[environment-specific variables](../../../ci/variables/README.md#limiting-environment-scopes-of-environment-variables-premium) work. While evaluating which environment matches the environment scope of a diff --git a/doc/user/group/index.md b/doc/user/group/index.md index a5e3bfda70e..853b00f1f67 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -5,33 +5,35 @@ and grant members access to several projects at once. Groups can also be nested in [subgroups](subgroups/index.md). -Find your groups by clicking **Groups** in the top navigation. +Find your groups by clicking **Groups > Your Groups** in the top navigation.  > The groups dropdown in the top navigation was [introduced][ce-36234] in [GitLab 11.1](https://about.gitlab.com/2018/07/22/gitlab-11-1-released/#groups-dropdown-in-navigation). -The Groups page displays all groups you are a member of, how many projects it holds, -how many members it has, the group visibility, and, if you have enough permissions, -a link to the group settings. By clicking the last button you can leave that group. +The Groups page displays: + +- All groups you are a member of. +- How many projects each group contains. +- How many members a group has. +- The group visibility. +- A link to the group settings if you have sufficient permissions. + +By clicking the last button, you can leave that group. ## Use cases -You can create groups for numerous reasons. To name a few: - -- Organize related projects under the same [namespace](#namespaces), add members to that - group and grant access to all their projects at once -- Create a group, include members of your team, and make it easier to - `@mention` all the team at once in issues and merge requests - - Create a group for your company members, and create [subgroups](subgroups/index.md) - for each individual team. Let's say you create a group called `company-team`, and among others, - you created subgroups in this group for each individual team `backend-team`, - `frontend-team`, and `production-team`: - 1. When you start a new implementation from an issue, you add a comment: +You can create groups for numerous reasons. To name a couple: + +- Grant access to multiple projects and multiple team members in fewer steps by organizing related projects under the same [namespace](#namespaces) and adding members to the top-level group. +- Make it easier to `@mention` all of your team at once in issues and merge requests by creating a group and including the appropriate members. + +For example, you could create a group for your company members, and create a [subgroup](subgroups/index.md) for each individual team. Let's say you create a group called `company-team`, and you create subgroups in this group for the individual teams `backend-team`, `frontend-team`, and `production-team`. + - When you start a new implementation from an issue, you add a comment: _"`@company-team`, let's do it! `@company-team/backend-team` you're good to go!"_ - 1. When your backend team needs help from frontend, they add a comment: + - When your backend team needs help from frontend, they add a comment: _"`@company-team/frontend-team` could you help us here please?"_ - 1. When the frontend team completes their implementation, they comment: + - When the frontend team completes their implementation, they comment: _"`@company-team/backend-team`, it's done! Let's ship it `@company-team/production-team`!"_ ## Namespaces @@ -59,8 +61,8 @@ By doing so: ## Issues and merge requests within a group -Issues and merge requests are part of projects. For a given group, view all the -[issues](../project/issues/index.md#issues-list) and [merge requests](../project/merge_requests/index.md#merge-requests-per-group) across all the projects in that group, +Issues and merge requests are part of projects. For a given group, you can view all of the +[issues](../project/issues/index.md#issues-list) and [merge requests](../project/merge_requests/index.md#merge-requests-per-group) across all projects in that group, together in a single list view. ## Create a new group @@ -68,13 +70,13 @@ together in a single list view. > For a list of words that are not allowed to be used as group names see the > [reserved names](../reserved_names.md). -You can create a group in GitLab from: +To create a new Group, either: -1. The Groups page: from the top menu, click **Groups**, and click the green button **New group**: +- In the top menu, click **Groups** and then **Your Groups**, and click the green button **New group**.  -1. Elsewhere: expand the `plus` sign button on the top navbar and choose **New group**: +- Or, in the top menu, expand the `plus` sign and choose **New group**.  @@ -82,18 +84,18 @@ Add the following information:  -1. The **Group name** will populate the URL automatically. Optionally, you can change it. - This is the name that is displayed in the group views. +1. The **Group name** will automatically populate the URL. Optionally, you can change it. + This is the name that displays in group views. The name can contain only: - - Alphanumeric characters. - - Underscores. - - Dashes and dots. - - Spaces. -1. The **Group URL**, which will be the namespace under which your projects will be hosted. + - Alphanumeric characters + - Underscores + - Dashes and dots + - Spaces +1. The **Group URL** is the namespace under which your projects will be hosted. The URL can contain only: - - Alphanumeric characters. - - Underscores. - - Dashes and dots. It cannot start with dashes or end in dot. + - Alphanumeric characters + - Underscores + - Dashes and dots (it cannot start with dashes or end in a dot) 1. Optionally, you can add a brief description to tell others what this group is about. 1. Optionally, choose an avatar for your group. @@ -101,45 +103,39 @@ Add the following information: ## Add users to a group -Add members to a group by navigating to the group's dashboard, and clicking **Members**: +A benefit of putting multiple projects in one group is that you can +give a user to access to all projects in the group with one action. + +Add members to a group by navigating to the group's dashboard and clicking **Members**.  -Select the [permission level](../permissions.md#permissions) and add the new member. You can also set the expiring -date for that user, from which they will no longer have access to your group. +Select the [permission level](../permissions.md#permissions), and add the new member. You can also set the expiring date for that user; this is the date on which they will no longer have access to your group. -One of the benefits of putting multiple projects in one group is that you can -give a user to access to all projects in the group with one action. +Consider a group with two projects: -Consider we have a group with two projects: - -- On the **Group Members** page we can now add a new user to the group. -- Now because this user is a **Developer** member of the group, he automatically +- On the **Group Members** page, you can now add a new user to the group. +- Now, because this user is a **Developer** member of the group, they automatically gets **Developer** access to **all projects** within that group. -If necessary, you can increase the access level of an individual user for a specific project, -by adding them again as a new member to the project with the new permission levels. +To increase the access level of an existing user for a specific project, +add them again as a new member to the project with the desired permission level. ## Request access to a group -As a group owner you can enable or disable non members to request access to -your group. Go to the group settings and click on **Allow users to request access**. +As a group owner, you can enable or disable the ability for non members to request access to +your group. Go to the group settings, and click **Allow users to request access**. -As a user, you can request to be a member of a group. Go to the group you'd -like to be a member of, and click the **Request Access** button on the right +As a user, you can request to be a member of a group, if that setting is enabled. Go to the group for which you'd like to be a member, and click the **Request Access** button on the right side of your screen.  ---- - Group owners and maintainers will be notified of your request and will be able to approve or decline it on the members page.  ---- - If you change your mind before your request is approved, just click the **Withdraw Access Request** button. @@ -149,29 +145,27 @@ If you change your mind before your request is approved, just click the There are two different ways to add a new project to a group: -- Select a group and then click on the **New project** button. +- Select a group, and then click **New project**. You can then continue [creating your project](../../gitlab-basics/create-project.md).  - You can then continue on [creating a project](../../gitlab-basics/create-project.md). - - While you are creating a project, select a group namespace you've already created from the dropdown menu.  -### Default project creation level +### Default project-creation level > [Introduced][ee-2534] in [GitLab Premium][ee] 10.5. > Brought to [GitLab Starter][ee] in 10.7. > [Moved](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/25975) to [GitLab Core](https://about.gitlab.com/pricing/) in 11.10. -Group owners or administrators can allow users with the +Group owners and administrators can allow users with the Developer role to create projects under groups. -By default, [Developers and Maintainers](../permissions.md#group-members-permissions) can create projects under a group, but this can be changed either within the group settings for a group, or -be set globally by a GitLab administrator in the Admin area -at **Settings > General > Visibility and access controls**. +By default, [Developers and Maintainers](../permissions.md#group-members-permissions) can create projects under a group. You can change this setting for a specific group within the group settings, or +you can set this option globally in the Admin area +at **Settings > General > Visibility and access controls** (you must be a GitLab administrator). Available settings are `No one`, `Maintainers`, or `Developers + Maintainers`. @@ -182,13 +176,13 @@ Learn how to [transfer a project into a group](../project/settings/index.md#tran ## Sharing a project with a group You can [share your projects with a group](../project/members/share_project_with_groups.md) -and give your group members access to the project all at once. +and give all group members access to the project at once. Alternatively, you can [lock the sharing with group feature](#share-with-group-lock). ## Manage group memberships via LDAP -In GitLab Enterprise Edition it is possible to manage GitLab group memberships using LDAP groups. +In GitLab Enterprise Edition, it is possible to manage GitLab group memberships using LDAP groups. See [the GitLab Enterprise Edition documentation](../../integration/ldap.md) for more information. ## Epics **[ULTIMATE]** @@ -211,38 +205,42 @@ Get an overview of the vulnerabilities of all the projects in a group and its su > Introduced in [GitLab Ultimate][ee] 11.9 behind the `insights` feature flag. -Configure the Insights that matter for your groups or projects to explore data -such as triage hygiene, issues created/closed per a given period, average time -for merge requests to be merged and much more. +Configure the Insights that matter for your groups or projects, allowing users to explore data +such as: + +- Triage hygiene +- Issues created/closed per a given period +- Average time for merge requests to be merged +- Much more [Learn more about Insights](insights/index.md). ## Transferring groups -From GitLab 10.5, groups can be transferred in the following ways: +From GitLab 10.5, you can transfer groups in the following ways: -- Top-level groups can be transferred to a group, converting them into subgroups. -- Subgroups can be transferred to a new parent group. -- Subgroups can be transferred out from a parent group, converting them into top-level groups. +- Transfer a subgroup to a new parent group. +- Convert a top-level group into a subgroup by transfering it to the desired group. +- Convert a subgroup into a top-level group by transfering it out of its current group. When transferring groups, note: - Changing a group's parent can have unintended side effects. See [Redirects when changing repository paths](../project/index.md#redirects-when-changing-repository-paths). - You can only transfer groups to groups you manage. -- You will need to update your local repositories to point to the new location. -- If the parent group's visibility is lower than the group's current visibility, visibility levels for subgroups and projects will be changed to match the new parent group's visibility. -- Only explicit group membership is transferred, not inherited membership. If the group's owners have only inherited membership, this would leave the group without an owner. In this case, the user transferring the group becomes the group's owner. +- You must update your local repositories to point to the new location. +- If the parent group's visibility is lower than the group's current visibility, visibility levels for subgroups and projects will change to match the new parent group's visibility. +- Only explicit group membership is transferred, not inherited membership. If the group's owners have only inherited membership, this leaves the group without an owner. In this case, the user transferring the group becomes the group's owner. ## Group settings -Once you have created a group, you can manage its settings by navigating to +After creating a group, you can manage its settings by navigating to the group's dashboard, and clicking **Settings**.  ### General settings -Besides giving you the option to edit any settings you've previously +In addition to editing any settings you previously set when [creating the group](#create-a-new-group), you can also access further configurations for your group. @@ -253,14 +251,14 @@ Changing a group's path can have unintended side effects. Read before proceeding. If you are vacating the path so it can be claimed by another group or user, -you may need to rename the group name as well since both names and paths must +you may need to rename the group, too, since both names and paths must be unique. To change your group path: 1. Navigate to your group's **Settings > General**. -1. Enter a new name under "Group path". -1. Hit **Save group**. +1. Enter a new name under **Group path**. +1. Click **Save group**. CAUTION: **Caution:** It is currently not possible to rename a namespace if it contains a @@ -276,20 +274,17 @@ username, you can create a new group and transfer projects to it. Add a security layer to your group by [enforcing two-factor authentication (2FA)](../../security/two_factor_authentication.md#enforcing-2fa-for-all-users-in-a-group) -to all group members. +for all group members. #### Share with group lock Prevent projects in a group from [sharing -a project with another group](../project/members/share_project_with_groups.md). -This allows for tighter control over project access. +a project with another group](../project/members/share_project_with_groups.md) to enable tighter control over project access. -For example, consider you have two distinct teams (Group A and Group B) -working together in a project. -To inherit the group membership, you share the project between the +For example, let's say you have two distinct teams (Group A and Group B) working together in a project, and to inherit the group membership, you share the project between the two groups A and B. **Share with group lock** prevents any project within -the group from being shared with another group. By doing so, you -guarantee only the right group members have access to those projects. +the group from being shared with another group, +guaranteeing that only the right group members have access to those projects. To enable this feature, navigate to the group settings page. Select **Share with group lock** and **Save the group**. @@ -298,40 +293,37 @@ To enable this feature, navigate to the group settings page. Select #### Member Lock **[STARTER]** -With Member lock, it is possible to lock membership in a project to the -level of members in the group. - -Member lock lets a group owner lock down any new project membership to all the -projects within the group, allowing tighter control over project membership. +Member lock lets a group owner prevent any new project membership to all of the +projects within a group, allowing tighter control over project membership. -For instance, if you want to lock the group for an [Audit Event](https://docs.gitlab.com/ee/administration/audit_events.html), -you enable Member lock to guarantee that membership of a project cannot be modified during that audit. +For example, if you want to lock the group for an [Audit Event](../../administration/audit_events.md), +enable Member lock to guarantee that project membership cannot be modified during that audit. To enable this feature: 1. Navigate to the group's **Settings > General** page. -1. Expand the **Permissions, LFS, 2FA** section and select **Member lock**. -1. Click the **Save changes** button. +1. Expand the **Permissions, LFS, 2FA** section, and select **Member lock**. +1. Click **Save changes**.  This will disable the option for all users who previously had permissions to -operate project memberships so no new users can be added. Furthermore, any +operate project memberships, so no new users can be added. Furthermore, any request to add a new user to a project through API will not be possible. #### Group file templates **[PREMIUM]** Group file templates allow you to share a set of templates for common file types with every project in a group. It is analogous to the -[instance template repository](https://docs.gitlab.com/ee/user/admin_area/settings/instance_template_repository.html) +[instance template repository](../admin_area/settings/instance_template_repository.md) feature, and the selected project should follow the same naming conventions as are documented on that page. -Only projects that are in the group may be chosen as the source of templates. -This includes projects shared with the group, but **excludes** projects in +You can only choose projects in the group as the template source. +This includes projects shared with the group, but it **excludes** projects in subgroups or parent groups of the group being configured. -This feature may be configured for subgroups as well as parent groups. A project +You can configure this feature for both subgroups and parent groups. A project in a subgroup will have access to the templates for that subgroup, as well as any parent groups. @@ -345,28 +337,28 @@ To enable this feature, navigate to the group settings page, expand the #### Group-level project templates **[PREMIUM]** -Define project templates at a group-level by setting a group as a template source. +Define project templates at a group level by setting a group as the template source. [Learn more about group-level project templates](custom_project_templates.md). ### Advanced settings -- **Projects**: view all projects within that group, add members to each project, - access each project's settings, and remove any project from the same screen. -- **Webhooks**: configure [webhooks](../project/integrations/webhooks.md) to your group. -- **Kubernetes cluster integration**: connect your GitLab group with [Kubernetes clusters](clusters/index.md). -- **Audit Events**: view [Audit Events](https://docs.gitlab.com/ee/administration/audit_events.html) +- **Projects**: View all projects within that group, add members to each project, + access each project's settings, and remove any project, all from the same screen. +- **Webhooks**: Configure [webhooks](../project/integrations/webhooks.md) for your group. +- **Kubernetes cluster integration**: Connect your GitLab group with [Kubernetes clusters](clusters/index.md). +- **Audit Events**: View [Audit Events](../../administration/audit_events.md) for the group. **[STARTER ONLY]** -- **Pipelines quota**: keep track of the [pipeline quota](../admin_area/settings/continuous_integration.md) for the group. +- **Pipelines quota**: Keep track of the [pipeline quota](../admin_area/settings/continuous_integration.md) for the group. ## User contribution analysis **[STARTER]** -With [GitLab Contribution Analytics](contribution_analytics/index.md) +With [GitLab Contribution Analytics](contribution_analytics/index.md), you have an overview of the contributions (pushes, merge requests, and issues) performed by your group members. ## Issues analytics **[PREMIUM]** -With [GitLab Issues Analytics](issues_analytics/index.md), in groups, you can see a bar chart of the number of issues created each month. +With [GitLab Issues Analytics](issues_analytics/index.md), you can see a bar chart of the number of issues created each month in your groups. ## Dependency Proxy **[PREMIUM]** diff --git a/doc/user/group/insights/index.md b/doc/user/group/insights/index.md index 20f76c54ae7..427b474ca39 100644 --- a/doc/user/group/insights/index.md +++ b/doc/user/group/insights/index.md @@ -31,7 +31,7 @@ If no configuration was set, a [default configuration file]( https://gitlab.com/gitlab-org/gitlab-ee/blob/master/ee/fixtures/insights/ee/fixtures/insights/default.yml) will be used. -See the [Project's Insights documentation](https://docs.gitlab.com/ee/user/project/insights/index.html) for +See the [Project's Insights documentation](../../project/insights/index.md) for more details about the `.gitlab/insights.yml` configuration file. ## Permissions diff --git a/doc/user/group/saml_sso/index.md b/doc/user/group/saml_sso/index.md index 53116606201..62a3ef52c34 100644 --- a/doc/user/group/saml_sso/index.md +++ b/doc/user/group/saml_sso/index.md @@ -5,17 +5,17 @@ NOTE: **Note:** This topic is for SAML on GitLab.com Silver tier and above. For SAML on self-managed GitLab instances, see [SAML OmniAuth Provider](../../../integration/saml.md). -Currently SAML on GitLab.com can be used to automatically add users to a group, and does not yet sign users into GitLab.com. Users should already have an account on the GitLab instance, or can create one when logging in for the first time. +SAML on GitLab.com allows users to be automatically added to a group, and then allows those users to sign into GitLab.com. Users should already have an account on the GitLab instance, or can create one when logging in for the first time. User synchronization for GitLab.com is partially supported using [SCIM](scim_setup.md). NOTE: **Note:** -SAML SSO for groups is used only as a convenient way to add users and does not sync users between providers without using SCIM. If a group is not using SCIM, group Owners will still need to manage user accounts, such as removing users when necessary. +SAML SSO for GitLab.com groups does not sync users between providers without using SCIM. If a group is not using SCIM, group Owners will still need to manage user accounts (for example, removing users when necessary). ## Configuring your Identity Provider 1. Navigate to the group and click **Settings > SAML SSO**. -1. Configure your SAML server using the **Assertion consumer service URL** and **Issuer**. See [your identity provider's documentation](#providers) for more details. +1. Configure your SAML server using the **Assertion consumer service URL** and **Issuer**. Alternatively GitLab provides [metadata XML configuration](#metadata-configuration). See [your identity provider's documentation](#providers) for more details. 1. Configure the SAML response to include a NameID that uniquely identifies each user. 1. Configure required assertions using the [table below](#assertions). 1. Once the identity provider is set up, move on to [configuring GitLab](#configuring-gitlab). @@ -50,6 +50,14 @@ GitLab.com uses the SAML NameID to identify users. The NameID element: | First Name | `first_name`, `firstname`, `firstName` | | | Last Name | `last_name`, `lastname`, `lastName` | | +## Metadata configuration + +GitLab provides metadata XML that can be used to configure your Identity Provider. + +1. Navigate to the group and click **Settings > SAML SSO**. +1. Copy the provided **GitLab metadata URL** +1. Follow your Identity Provider's documentation and paste the metadata URL when it is requested. + ## Configuring GitLab Once you've set up your identity provider to work with GitLab, you'll need to configure GitLab to use it for authentication: @@ -75,6 +83,23 @@ Once you've set up your identity provider to work with GitLab, you'll need to co | OneLogin | [Use the OneLogin SAML Test Connector](https://onelogin.service-now.com/support?id=kb_article&sys_id=93f95543db109700d5505eea4b96198f) | | Ping Identity | [Add and configure a new SAML application](https://docs.pingidentity.com/bundle/p1_enterpriseConfigSsoSaml_cas/page/enableAppWithoutURL.html) | +## Linking SAML to your existing GitLab.com account + +To link SAML to your existing GitLab.com account: + +1. Sign in to your GitLab.com account. +1. Locate the SSO URL for the group you are signing in to. A group Admin can find this on the group's **Settings > SAML SSO** page. +1. Visit the SSO URL and click **Authorize**. +1. Enter your credentials on the Identity Provider if prompted. +1. You will be redirected back to GitLab.com and should now have access to the group. In the future, you can use SAML to sign in to GitLab.com. + +## Signing in to GitLab.com with SAML + +1. Locate the SSO URL for the group you are signing in to. A group Admin can find this on a group's **Settings > SAML SSO** page. If configured, it might also be possible to sign in to GitLab starting from your Identity Provider. +1. Visit the SSO URL and click the **Sign in with Single Sign-On** button. +1. Enter your credentials on the Identity Provider if prompted. +1. You will be signed in to GitLab.com and redirected to the group. + ## Unlinking accounts Users can unlink SAML for a group from their profile page. This can be helpful if: diff --git a/doc/user/group/saml_sso/scim_setup.md b/doc/user/group/saml_sso/scim_setup.md index ec27ec3e336..c00628bf909 100644 --- a/doc/user/group/saml_sso/scim_setup.md +++ b/doc/user/group/saml_sso/scim_setup.md @@ -2,7 +2,7 @@ > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/9388) in [GitLab.com Silver](https://about.gitlab.com/pricing/) 11.10. -GitLab's [SCIM API](https://docs.gitlab.com/ee/api/scim.html) implements part of [the RFC7644 protocol](https://tools.ietf.org/html/rfc7644). +GitLab's [SCIM API](../../../api/scim.md) implements part of [the RFC7644 protocol](https://tools.ietf.org/html/rfc7644). Currently, the following actions are available: diff --git a/doc/user/group/security_dashboard/index.md b/doc/user/group/security_dashboard/index.md index 43e910b29fe..c59198df081 100644 --- a/doc/user/group/security_dashboard/index.md +++ b/doc/user/group/security_dashboard/index.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/application_security/security_dashboard/index.html' +redirect_to: '../../application_security/security_dashboard/index.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/user/application_security/security_dashboard/index.html). +This document was moved to [another location](../../application_security/security_dashboard/index.md). diff --git a/doc/user/index.md b/doc/user/index.md index 67d5cbf06ec..1fc4e4c43cf 100644 --- a/doc/user/index.md +++ b/doc/user/index.md @@ -49,22 +49,22 @@ GitLab is a Git-based platform that integrates a great number of essential tools With GitLab Enterprise Edition, you can also: -- Provide support with [Service Desk](https://docs.gitlab.com/ee/user/project/service_desk.html). +- Provide support with [Service Desk](project/service_desk.md). - Improve collaboration with - [Merge Request Approvals](https://docs.gitlab.com/ee/user/project/merge_requests/index.html#merge-request-approvals-starter), - [Multiple Assignees for Issues](https://docs.gitlab.com/ee/user/project/issues/multiple_assignees_for_issues.html), + [Merge Request Approvals](project/merge_requests/index.md#merge-request-approvals-starter), + [Multiple Assignees for Issues](project/issues/multiple_assignees_for_issues.md), and [Multiple Issue Boards](project/issue_board.md#multiple-issue-boards-starter). -- Create formal relationships between issues with [Related Issues](https://docs.gitlab.com/ee/user/project/issues/related_issues.html). -- Use [Burndown Charts](https://docs.gitlab.com/ee/user/project/milestones/burndown_charts.html) to track progress during a sprint or while working on a new version of their software. -- Leverage [Elasticsearch](https://docs.gitlab.com/ee/integration/elasticsearch.html) with [Advanced Global Search](search/advanced_global_search.md) and [Advanced Syntax Search](search/advanced_search_syntax.md) for faster, more advanced code search across your entire GitLab instance. -- [Authenticate users with Kerberos](https://docs.gitlab.com/ee/integration/kerberos.html). +- Create formal relationships between issues with [Related Issues](project/issues/related_issues.md). +- Use [Burndown Charts](project/milestones/burndown_charts.md) to track progress during a sprint or while working on a new version of their software. +- Leverage [Elasticsearch](../integration/elasticsearch.md) with [Advanced Global Search](search/advanced_global_search.md) and [Advanced Syntax Search](search/advanced_search_syntax.md) for faster, more advanced code search across your entire GitLab instance. +- [Authenticate users with Kerberos](../integration/kerberos.md). - [Mirror a repository](../workflow/repository_mirroring.md) from elsewhere on your local server. -- [Export issues as CSV](https://docs.gitlab.com/ee/user/project/issues/csv_export.html). -- View your entire CI/CD pipeline involving more than one project with [Multiple-Project Pipelines](https://docs.gitlab.com/ee/ci/multi_project_pipeline_graphs.html). -- [Lock files](https://docs.gitlab.com/ee/user/project/file_lock.html) to prevent conflicts. -- View the current health and status of each CI environment running on Kubernetes with [Deploy Boards](https://docs.gitlab.com/ee/user/project/deploy_boards.html). -- Leverage continuous delivery method with [Canary Deployments](https://docs.gitlab.com/ee/user/project/canary_deployments.html). -- Scan your code for vulnerabilities and [display them in merge requests](https://docs.gitlab.com/ee/user/application_security/sast/index.html). +- [Export issues as CSV](project/issues/csv_export.md). +- View your entire CI/CD pipeline involving more than one project with [Multiple-Project Pipelines](../ci/multi_project_pipeline_graphs.md). +- [Lock files](project/file_lock.md) to prevent conflicts. +- View the current health and status of each CI environment running on Kubernetes with [Deploy Boards](project/deploy_boards.md). +- Leverage continuous delivery method with [Canary Deployments](project/canary_deployments.md). +- Scan your code for vulnerabilities and [display them in merge requests](application_security/sast/index.md). You can also [integrate](project/integrations/project_services.md) GitLab with numerous third-party applications, such as Mattermost, Microsoft Teams, HipChat, Trello, Slack, Bamboo CI, JIRA, and a lot more. diff --git a/doc/user/permissions.md b/doc/user/permissions.md index 318053fdabb..a6e2f187090 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -36,91 +36,96 @@ In GitLab 11.0, the Master role was renamed to Maintainer. The following table depicts the various user permission levels in a project. -| Action | Guest | Reporter | Developer |Maintainer| Owner | -|---------------------------------------|---------|------------|-------------|----------|--------| -| Create new issue | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | -| Create confidential issue | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | -| View confidential issues | (✓) [^2] | ✓ | ✓ | ✓ | ✓ | -| Leave comments | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | -| See related issues | ✓ | ✓ | ✓ | ✓ | ✓ | -| See a list of jobs | ✓ [^3] | ✓ | ✓ | ✓ | ✓ | -| See a job log | ✓ [^3] | ✓ | ✓ | ✓ | ✓ | -| Download and browse job artifacts | ✓ [^3] | ✓ | ✓ | ✓ | ✓ | -| View wiki pages | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | -| Create and edit wiki pages | | | ✓ | ✓ | ✓ | -| Delete wiki pages | | | | ✓ | ✓ | -| View license management reports **[ULTIMATE]** | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | -| View Security reports **[ULTIMATE]** | ✓ [^1] | ✓ | ✓ | ✓ | ✓ | -| View project code | [^1] | ✓ | ✓ | ✓ | ✓ | -| Pull project code | [^1] | ✓ | ✓ | ✓ | ✓ | -| Download project | [^1] | ✓ | ✓ | ✓ | ✓ | -| Assign issues | | ✓ | ✓ | ✓ | ✓ | -| Assign merge requests | | | ✓ | ✓ | ✓ | -| Label issues | | ✓ | ✓ | ✓ | ✓ | -| Label merge requests | | | ✓ | ✓ | ✓ | -| Create code snippets | | ✓ | ✓ | ✓ | ✓ | -| Manage issue tracker | | ✓ | ✓ | ✓ | ✓ | -| Manage labels | | ✓ | ✓ | ✓ | ✓ | -| See a commit status | | ✓ | ✓ | ✓ | ✓ | -| See a container registry | | ✓ | ✓ | ✓ | ✓ | -| See environments | | ✓ | ✓ | ✓ | ✓ | -| See a list of merge requests | | ✓ | ✓ | ✓ | ✓ | -| Manage related issues **[STARTER]** | | ✓ | ✓ | ✓ | ✓ | -| Lock issue discussions | | ✓ | ✓ | ✓ | ✓ | -| Create issue from vulnerability **[ULTIMATE]** | | ✓ | ✓ | ✓ | ✓ | -| View Error Tracking list | | ✓ | ✓ | ✓ | ✓ | -| Pull from [Maven repository](https://docs.gitlab.com/ee/user/project/packages/maven_repository.html) or [NPM registry](https://docs.gitlab.com/ee/user/project/packages/npm_registry.html) **[PREMIUM]** | | ✓ | ✓ | ✓ | ✓ | -| Publish to [Maven repository](https://docs.gitlab.com/ee/user/project/packages/maven_repository.html) or [NPM registry](https://docs.gitlab.com/ee/user/project/packages/npm_registry.html) **[PREMIUM]** | | | ✓ | ✓ | ✓ | -| Lock merge request discussions | | | ✓ | ✓ | ✓ | -| Create new environments | | | ✓ | ✓ | ✓ | -| Stop environments | | | ✓ | ✓ | ✓ | -| Manage/Accept merge requests | | | ✓ | ✓ | ✓ | -| Create new merge request | | | ✓ | ✓ | ✓ | -| Create new branches | | | ✓ | ✓ | ✓ | -| Push to non-protected branches | | | ✓ | ✓ | ✓ | -| Force push to non-protected branches | | | ✓ | ✓ | ✓ | -| Remove non-protected branches | | | ✓ | ✓ | ✓ | -| Add tags | | | ✓ | ✓ | ✓ | -| Cancel and retry jobs | | | ✓ | ✓ | ✓ | -| Create or update commit status | | | ✓ | ✓ | ✓ | -| Update a container registry | | | ✓ | ✓ | ✓ | -| Remove a container registry image | | | ✓ | ✓ | ✓ | -| Create/edit/delete project milestones | | | ✓ | ✓ | ✓ | +| Action | Guest | Reporter | Developer |Maintainer| Owner | +|---------------------------------------------------|---------|------------|-------------|----------|--------| +| Download project | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| Leave comments | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| View Insights charts **[ULTIMATE]** | ✓ | ✓ | ✓ | ✓ | ✓ | | View approved/blacklisted licenses **[ULTIMATE]** | ✓ | ✓ | ✓ | ✓ | ✓ | -| Use security dashboard **[ULTIMATE]** | | | ✓ | ✓ | ✓ | -| Dismiss vulnerability **[ULTIMATE]** | | | ✓ | ✓ | ✓ | -| Apply code change suggestions | | | ✓ | ✓ | ✓ | -| Use environment terminals | | | | ✓ | ✓ | -| Run Web IDE's Interactive Web Terminals **[ULTIMATE ONLY]** | | | | ✓ | ✓ | -| Add new team members | | | | ✓ | ✓ | -| Push to protected branches | | | | ✓ | ✓ | -| Enable/disable branch protection | | | | ✓ | ✓ | -| Turn on/off protected branch push for devs| | | | ✓ | ✓ | -| Enable/disable tag protections | | | | ✓ | ✓ | -| Rewrite/remove Git tags | | | | ✓ | ✓ | -| Edit project | | | | ✓ | ✓ | -| Add deploy keys to project | | | | ✓ | ✓ | -| Configure project hooks | | | | ✓ | ✓ | -| Manage Runners | | | | ✓ | ✓ | -| Manage job triggers | | | | ✓ | ✓ | -| Manage variables | | | | ✓ | ✓ | -| Manage GitLab Pages | | | | ✓ | ✓ | -| Manage GitLab Pages domains and certificates | | | | ✓ | ✓ | -| Remove GitLab Pages | | | | ✓ | ✓ | +| View license management reports **[ULTIMATE]** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| View Security reports **[ULTIMATE]** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| View project code | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| Pull project code | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | | View GitLab Pages protected by [access control](project/pages/introduction.md#gitlab-pages-access-control-core-only) | ✓ | ✓ | ✓ | ✓ | ✓ | -| Manage clusters | | | | ✓ | ✓ | -| Manage license policy **[ULTIMATE]** | | | | ✓ | ✓ | -| Edit comments (posted by any user) | | | | ✓ | ✓ | -| Manage Error Tracking | | | | ✓ | ✓ | -| Switch visibility level | | | | | ✓ | -| Transfer project to another namespace | | | | | ✓ | -| Remove project | | | | | ✓ | -| Delete issues | | | | | ✓ | -| Force push to protected branches [^4] | | | | | | -| Remove protected branches [^4] | | | | | | -| View project Audit Events | | | | ✓ | ✓ | -| View project statistics | | ✓ | ✓ | ✓ | ✓ | -| View Insights charts **[ULTIMATE]** | ✓ | ✓ | ✓ | ✓ | ✓ | +| View wiki pages | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| See a list of jobs | ✓ (*3*) | ✓ | ✓ | ✓ | ✓ | +| See a job log | ✓ (*3*) | ✓ | ✓ | ✓ | ✓ | +| Download and browse job artifacts | ✓ (*3*) | ✓ | ✓ | ✓ | ✓ | +| Create new issue | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| See related issues | ✓ | ✓ | ✓ | ✓ | ✓ | +| Create confidential issue | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| View confidential issues | (*2*) | ✓ | ✓ | ✓ | ✓ | +| Assign issues | | ✓ | ✓ | ✓ | ✓ | +| Label issues | | ✓ | ✓ | ✓ | ✓ | +| Lock issue discussions | | ✓ | ✓ | ✓ | ✓ | +| Manage issue tracker | | ✓ | ✓ | ✓ | ✓ | +| Manage related issues **[STARTER]** | | ✓ | ✓ | ✓ | ✓ | +| Create issue from vulnerability **[ULTIMATE]** | | ✓ | ✓ | ✓ | ✓ | +| Manage labels | | ✓ | ✓ | ✓ | ✓ | +| Create code snippets | | ✓ | ✓ | ✓ | ✓ | +| See a commit status | | ✓ | ✓ | ✓ | ✓ | +| See a container registry | | ✓ | ✓ | ✓ | ✓ | +| See environments | | ✓ | ✓ | ✓ | ✓ | +| See a list of merge requests | | ✓ | ✓ | ✓ | ✓ | +| View project statistics | | ✓ | ✓ | ✓ | ✓ | +| View Error Tracking list | | ✓ | ✓ | ✓ | ✓ | +| Pull from [Maven repository](project/packages/maven_repository.md) or [NPM registry](project/packages/npm_registry.md) **[PREMIUM]** | | ✓ | ✓ | ✓ | ✓ | +| Publish to [Maven repository](project/packages/maven_repository.md) or [NPM registry](project/packages/npm_registry.md) **[PREMIUM]** | | | ✓ | ✓ | ✓ || +| Create new branches | | | ✓ | ✓ | ✓ | +| Push to non-protected branches | | | ✓ | ✓ | ✓ | +| Force push to non-protected branches | | | ✓ | ✓ | ✓ | +| Remove non-protected branches | | | ✓ | ✓ | ✓ | +| Create new merge request | | | ✓ | ✓ | ✓ | +| Assign merge requests | | | ✓ | ✓ | ✓ | +| Label merge requests | | | ✓ | ✓ | ✓ | +| Lock merge request discussions | | | ✓ | ✓ | ✓ | +| Manage/Accept merge requests | | | ✓ | ✓ | ✓ | +| Create new environments | | | ✓ | ✓ | ✓ | +| Stop environments | | | ✓ | ✓ | ✓ | +| Add tags | | | ✓ | ✓ | ✓ | +| Cancel and retry jobs | | | ✓ | ✓ | ✓ | +| Create or update commit status | | | ✓ | ✓ | ✓ | +| Update a container registry | | | ✓ | ✓ | ✓ | +| Remove a container registry image | | | ✓ | ✓ | ✓ | +| Create/edit/delete project milestones | | | ✓ | ✓ | ✓ | +| Use security dashboard **[ULTIMATE]** | | | ✓ | ✓ | ✓ | +| Dismiss vulnerability **[ULTIMATE]** | | | ✓ | ✓ | ✓ | +| Apply code change suggestions | | | ✓ | ✓ | ✓ | +| Create and edit wiki pages | | | ✓ | ✓ | ✓ | +| Use environment terminals | | | | ✓ | ✓ | +| Run Web IDE's Interactive Web Terminals **[ULTIMATE ONLY]** | | | | ✓ | ✓ | +| Add new team members | | | | ✓ | ✓ | +| Enable/disable branch protection | | | | ✓ | ✓ | +| Push to protected branches | | | | ✓ | ✓ | +| Turn on/off protected branch push for devs | | | | ✓ | ✓ | +| Enable/disable tag protections | | | | ✓ | ✓ | +| Rewrite/remove Git tags | | | | ✓ | ✓ | +| Edit project | | | | ✓ | ✓ | +| Add deploy keys to project | | | | ✓ | ✓ | +| Configure project hooks | | | | ✓ | ✓ | +| Manage Runners | | | | ✓ | ✓ | +| Manage job triggers | | | | ✓ | ✓ | +| Manage variables | | | | ✓ | ✓ | +| Manage GitLab Pages | | | | ✓ | ✓ | +| Manage GitLab Pages domains and certificates | | | | ✓ | ✓ | +| Remove GitLab Pages | | | | ✓ | ✓ | +| Manage clusters | | | | ✓ | ✓ | +| Manage license policy **[ULTIMATE]** | | | | ✓ | ✓ | +| Edit comments (posted by any user) | | | | ✓ | ✓ | +| Manage Error Tracking | | | | ✓ | ✓ | +| Delete wiki pages | | | | ✓ | ✓ | +| View project Audit Events | | | | ✓ | ✓ | +| Switch visibility level | | | | | ✓ | +| Transfer project to another namespace | | | | | ✓ | +| Remove project | | | | | ✓ | +| Delete issues | | | | | ✓ | +| Force push to protected branches [^4] | | | | | | +| Remove protected branches [^4] | | | | | | + +- (*1*): All users are able to perform this action on public and internal projects, but not private projects. +- (*2*): Guest users can only view the confidential issues they created themselves +- (*3*): If **Public pipelines** is enabled in **Project Settings > CI/CD** +- (*4*): Not allowed for Guest, Reporter, Developer, Maintainer, or Owner ## Project features permissions @@ -163,7 +168,7 @@ to learn more. The user that locks a file or directory is the only one that can edit and push their changes back to the repository where the locked objects are located. -Read through the documentation on [permissions for File Locking](https://docs.gitlab.com/ee/user/project/file_lock.html#permissions-on-file-locking) to learn more. +Read through the documentation on [permissions for File Locking](project/file_lock.md#permissions-on-file-locking) to learn more. ### Confidential Issues permissions @@ -191,21 +196,21 @@ Any user can remove themselves from a group, unless they are the last Owner of the group. The following table depicts the various user permission levels in a group. -| Action | Guest | Reporter | Developer | Maintainer | Owner | -|-------------------------|-------|----------|-----------|--------|-------| -| Browse group | ✓ | ✓ | ✓ | ✓ | ✓ | -| Edit group | | | | | ✓ | -| Create subgroup | | | | | ✓ | -| Create project in group | | | ✓ | ✓ | ✓ | -| Manage group members | | | | | ✓ | -| Remove group | | | | | ✓ | -| Manage group labels | | ✓ | ✓ | ✓ | ✓ | -| Create/edit/delete group milestones | | | ✓ | ✓ | ✓ | -| View group epic **[ULTIMATE]** | ✓ | ✓ | ✓ | ✓ | ✓ | -| Create/edit group epic **[ULTIMATE]** | | ✓ | ✓ | ✓ | ✓ | -| Delete group epic **[ULTIMATE]** | | | | | ✓ | -| View group Audit Events | | | | | ✓ | -| View Insights charts **[ULTIMATE]** | ✓ | ✓ | ✓ | ✓ | ✓ | +| Action | Guest | Reporter | Developer | Maintainer | Owner | +|---------------------------------------|-------|----------|-----------|------------|-------| +| Browse group | ✓ | ✓ | ✓ | ✓ | ✓ | +| View Insights charts **[ULTIMATE]** | ✓ | ✓ | ✓ | ✓ | ✓ | +| View group epic **[ULTIMATE]** | ✓ | ✓ | ✓ | ✓ | ✓ | +| Create/edit group epic **[ULTIMATE]** | | ✓ | ✓ | ✓ | ✓ | +| Manage group labels | | ✓ | ✓ | ✓ | ✓ | +| Create project in group | | | ✓ | ✓ | ✓ | +| Create/edit/delete group milestones | | | ✓ | ✓ | ✓ | +| Edit group | | | | | ✓ | +| Create subgroup | | | | | ✓ | +| Manage group members | | | | | ✓ | +| Remove group | | | | | ✓ | +| Delete group epic **[ULTIMATE]** | | | | | ✓ | +| View group Audit Events | | | | | ✓ | ### Subgroup permissions @@ -257,15 +262,15 @@ Please be aware that this regex could lead to a DOS attack, [see](https://en.wik ## Auditor users **[PREMIUM ONLY]** ->[Introduced][ee-998] in [GitLab Premium][eep] 8.17. +>[Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/998) in [GitLab Premium](https://about.gitlab.com/pricing/) 8.17. Auditor users are given read-only access to all projects, groups, and other resources on the GitLab instance. An Auditor user should be able to access all projects and groups of a GitLab instance -with the permissions described on the documentation on [auditor users permissions](https://docs.gitlab.com/ee/administration/auditor_users.html#permissions-and-restrictions-of-an-auditor-user). +with the permissions described on the documentation on [auditor users permissions](../administration/auditor_users.md#permissions-and-restrictions-of-an-auditor-user). -[Read more about Auditor users.](https://docs.gitlab.com/ee/administration/auditor_users.html) +[Read more about Auditor users.](../administration/auditor_users.md) ## Project features @@ -298,7 +303,7 @@ instance and project. In addition, all admins can use the admin interface under |---------------------------------------|-----------------|-------------|----------|--------| | See commits and jobs | ✓ | ✓ | ✓ | ✓ | | Retry or cancel job | | ✓ | ✓ | ✓ | -| Erase job artifacts and trace | | ✓ [^5] | ✓ | ✓ | +| Erase job artifacts and trace | | ✓ (*1*) | ✓ | ✓ | | Remove project | | | ✓ | ✓ | | Create project | | | ✓ | ✓ | | Change project configuration | | | ✓ | ✓ | @@ -307,6 +312,8 @@ instance and project. In addition, all admins can use the admin interface under | See events in the system | | | | ✓ | | Admin interface | | | | ✓ | +- *1*: Only if the job was triggered by the user + ### Job permissions NOTE: **Note:** @@ -314,25 +321,28 @@ In GitLab 11.0, the Master role was renamed to Maintainer. >**Note:** GitLab 8.12 has a completely redesigned job permissions system. -Read all about the [new model and its implications][new-mod]. +Read all about the [new model and its implications](project/new_ci_build_permissions_model.md). This table shows granted privileges for jobs triggered by specific types of users: -| Action | Guest, Reporter | Developer |Maintainer| Admin | -|---------------------------------------------|-----------------|-------------|----------|--------| -| Run CI job | | ✓ | ✓ | ✓ | -| Clone source and LFS from current project | | ✓ | ✓ | ✓ | -| Clone source and LFS from public projects | | ✓ | ✓ | ✓ | -| Clone source and LFS from internal projects | | ✓ [^6] | ✓ [^6] | ✓ | -| Clone source and LFS from private projects | | ✓ [^7] | ✓ [^7] | ✓ [^7] | -| Push source and LFS | | | | | -| Pull container images from current project | | ✓ | ✓ | ✓ | -| Pull container images from public projects | | ✓ | ✓ | ✓ | -| Pull container images from internal projects| | ✓ [^6] | ✓ [^6] | ✓ | -| Pull container images from private projects | | ✓ [^7] | ✓ [^7] | ✓ [^7] | -| Push container images to current project | | ✓ | ✓ | ✓ | -| Push container images to other projects | | | | | +| Action | Guest, Reporter | Developer |Maintainer| Admin | +|---------------------------------------------|-----------------|-------------|----------|---------| +| Run CI job | | ✓ | ✓ | ✓ | +| Clone source and LFS from current project | | ✓ | ✓ | ✓ | +| Clone source and LFS from public projects | | ✓ | ✓ | ✓ | +| Clone source and LFS from internal projects | | ✓ (*1*) | ✓ (*1*) | ✓ | +| Clone source and LFS from private projects | | ✓ (*2*) | ✓ (*2*) | ✓ (*2*) | +| Pull container images from current project | | ✓ | ✓ | ✓ | +| Pull container images from public projects | | ✓ | ✓ | ✓ | +| Pull container images from internal projects| | ✓ (*1*) | ✓ (*1*) | ✓ | +| Pull container images from private projects | | ✓ (*2*) | ✓ (*2*) | ✓ (*2*) | +| Push container images to current project | | ✓ | ✓ | ✓ | +| Push container images to other projects | | | | | +| Push source and LFS | | | | | + +- *1*: Only if the user is not an external one +- *2*: Only if the user is a member of the project ### New CI job permissions model @@ -350,17 +360,4 @@ for details about the pipelines security model. ## LDAP users permissions Since GitLab 8.15, LDAP user permissions can now be manually overridden by an admin user. -Read through the documentation on [LDAP users permissions](https://docs.gitlab.com/ee/administration/auth/how_to_configure_ldap_gitlab_ee/index.html) to learn more. - -[^1]: On public and internal projects, all users are able to perform this action -[^2]: Guest users can only view the confidential issues they created themselves -[^3]: If **Public pipelines** is enabled in **Project Settings > CI/CD** -[^4]: Not allowed for Guest, Reporter, Developer, Maintainer, or Owner -[^5]: Only if the job was triggered by the user -[^6]: Only if user is not external one -[^7]: Only if user is a member of the project - -[ce-18994]: https://gitlab.com/gitlab-org/gitlab-ce/issues/18994 -[new-mod]: project/new_ci_build_permissions_model.md -[ee-998]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/998 -[eep]: https://about.gitlab.com/pricing/ +Read through the documentation on [LDAP users permissions](../administration/auth/how_to_configure_ldap_gitlab_ee/index.html) to learn more. diff --git a/doc/user/profile/personal_access_tokens.md b/doc/user/profile/personal_access_tokens.md index 4085f3b678c..0b224fc7e01 100644 --- a/doc/user/profile/personal_access_tokens.md +++ b/doc/user/profile/personal_access_tokens.md @@ -37,19 +37,18 @@ Personal access tokens can be created with one or more scopes that allow various actions that a given token can perform. The available scopes are depicted in the following table. -| Scope | Description | -| ----- | ----------- | -|`read_user` | Allows access to the read-only endpoints under `/users`. Essentially, any of the `GET` requests in the [Users API][users] are allowed ([introduced][ce-5951] in GitLab 8.15). | -| `api` | Grants complete access to the API and Container Registry (read/write) ([introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5951) in GitLab 8.15). | -| `read_registry` | Allows to read (pull) [container registry] images if a project is private and authorization is required ([introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/11845) in GitLab 9.3). | -| `sudo` | Allows performing API actions as any user in the system (if the authenticated user is an admin) ([introduced][ce-14838] in GitLab 10.2). | -| `read_repository` | Allows read-only access (pull) to the repository through git clone. | -| `write_repository` | Allows read-write access (pull, push) to the repository through git clone. Required for accessing Git repositories over HTTP when 2FA is enabled. | +| Scope | Introduced in | Description | +| ------------------ | ------------- | ----------- | +| `read_user` | [GitLab 8.15](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5951) | Allows access to the read-only endpoints under `/users`. Essentially, any of the `GET` requests in the [Users API][users] are allowed. | +| `api` | [GitLab 8.15](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5951) | Grants complete access to the API and Container Registry (read/write). | +| `read_registry` | [GitLab 9.3](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/11845) | Allows to read (pull) [container registry] images if a project is private and authorization is required. | +| `sudo` | [GitLab 10.2](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14838) | Allows performing API actions as any user in the system (if the authenticated user is an admin). | +| `read_repository` | [GitLab 10.7](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/17894) | Allows read-only access (pull) to the repository through git clone. | +| `write_repository` | [GitLab 11.11](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/26021) | Allows read-write access (pull, push) to the repository through git clone. Required for accessing Git repositories over HTTP when 2FA is enabled. | [2fa]: ../account/two_factor_authentication.md [api]: ../../api/README.md [ce-3749]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3749 -[ce-14838]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14838 [container registry]: ../project/container_registry.md [users]: ../../api/users.md [usage]: ../../api/README.md#personal-access-tokens diff --git a/doc/user/project/ci_cd_for_external_repo.md b/doc/user/project/ci_cd_for_external_repo.md index 51b86a68c7b..a92d3a2c308 100644 --- a/doc/user/project/ci_cd_for_external_repo.md +++ b/doc/user/project/ci_cd_for_external_repo.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/index.html' +redirect_to: '../../ci/ci_cd_for_external_repos/index.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/index.html). +This document was moved to [another location](../../ci/ci_cd_for_external_repos/index.md). diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index 3bc3beb2055..dc21db603d6 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -71,7 +71,7 @@ new Kubernetes cluster to your project: - **Number of nodes** - Enter the number of nodes you wish the cluster to have. - **Machine type** - The [machine type](https://cloud.google.com/compute/docs/machine-types) of the Virtual Machine instance that the cluster will be based on. - - **RBAC-enabled cluster** - Leave this checked if using default GKE creation options, see the [RBAC section](#role-based-access-control-rbac) for more information. + - **RBAC-enabled cluster** - Leave this checked if using default GKE creation options, see the [RBAC section](#rbac-cluster-resources) for more information. - **GitLab-managed cluster** - Leave this checked if you want GitLab to manage namespaces and service accounts for this cluster. See the [Managed clusters section](#gitlab-managed-clusters) for more information. 1. Finally, click the **Create Kubernetes cluster** button. @@ -263,65 +263,66 @@ you can either: ## Access controls -When creating a cluster in GitLab, you will be asked if you would like to create an -[Attribute-based access control (ABAC)](https://kubernetes.io/docs/admin/authorization/abac/) cluster, or -a [Role-based access control (RBAC)](https://kubernetes.io/docs/admin/authorization/rbac/) one. +When creating a cluster in GitLab, you will be asked if you would like to create either: -NOTE: **Note:** -[RBAC](#role-based-access-control-rbac) is recommended and the GitLab default. +- An [Attribute-based access control (ABAC)](https://kubernetes.io/docs/admin/authorization/abac/) cluster. +- A [Role-based access control (RBAC)](https://kubernetes.io/docs/admin/authorization/rbac/) cluster. -Whether [ABAC](#attribute-based-access-control-abac) or [RBAC](#role-based-access-control-rbac) is enabled, -GitLab will create the necessary service accounts and privileges in order to install and run -[GitLab managed applications](#installing-applications): +NOTE: **Note:** +[RBAC](#rbac-cluster-resources) is recommended and the GitLab default. -- If GitLab is creating the cluster, a `gitlab` service account with - `cluster-admin` privileges will be created in the `default` namespace, - which will be used by GitLab to manage the newly created cluster. +GitLab creates the necessary service accounts and privileges to install and run +[GitLab managed applications](#installing-applications). When GitLab creates the cluster: +- A `gitlab` service account with `cluster-admin` privileges is created in the `default` namespace + to manage the newly created cluster. - A project service account with [`edit` privileges](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) - will be created in the project namespace (also created by GitLab), which will - be used in [deployment jobs](#deployment-variables). + is created in the GitLab-created project namespace for [deployment jobs](#deployment-variables). NOTE: **Note:** Restricted service account for deployment was [introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/51716) in GitLab 11.5. -- When you install Helm into your cluster, the `tiller` service account - will be created with `cluster-admin` privileges in the `gitlab-managed-apps` - namespace. This service account will be added to the installed Helm Tiller and will - be used by Helm to install and run [GitLab managed applications](#installing-applications). - Helm will also create additional service accounts and other resources for each - installed application. Consult the documentation of the Helm charts for each application - for details. +When you install Helm into your cluster, the `tiller` service account +is created with `cluster-admin` privileges in the `gitlab-managed-apps` +namespace. This service account will be added to the installed Helm Tiller and will +be used by Helm to install and run [GitLab managed applications](#installing-applications). +Helm will also create additional service accounts and other resources for each +installed application. Consult the documentation of the Helm charts for each application +for details. If you are [adding an existing Kubernetes cluster](#adding-an-existing-kubernetes-cluster), ensure the token of the account has administrator privileges for the cluster. -The following sections summarize which resources will be created on ABAC/RBAC clusters. +The resources created by GitLab differ depending on the type of cluster. + +### ABAC cluster resources -### Attribute-based access control (ABAC) +GitLab creates the following resources for ABAC clusters. -| Name | Kind | Details | Created when | -| --- | --- | --- | --- | -| `gitlab` | `ServiceAccount` | `default` namespace | Creating a new GKE Cluster | -| `gitlab-token` | `Secret` | Token for `gitlab` ServiceAccount | Creating a new GKE Cluster | -| `tiller` | `ServiceAccount` | `gitlab-managed-apps` namespace | Installing Helm Tiller | -| `tiller-admin` | `ClusterRoleBinding` | `cluster-admin` roleRef | Installing Helm Tiller | -| Project namespace | `ServiceAccount` | Uses namespace of Project | Deploying to a cluster | -| Project namespace | `Secret` | Token for project ServiceAccount | Deploying to a cluster | +| Name | Type | Details | Created when | +|:------------------|:---------------------|:----------------------------------|:---------------------------| +| `gitlab` | `ServiceAccount` | `default` namespace | Creating a new GKE Cluster | +| `gitlab-token` | `Secret` | Token for `gitlab` ServiceAccount | Creating a new GKE Cluster | +| `tiller` | `ServiceAccount` | `gitlab-managed-apps` namespace | Installing Helm Tiller | +| `tiller-admin` | `ClusterRoleBinding` | `cluster-admin` roleRef | Installing Helm Tiller | +| Project namespace | `ServiceAccount` | Uses namespace of Project | Deploying to a cluster | +| Project namespace | `Secret` | Token for project ServiceAccount | Deploying to a cluster | -### Role-based access control (RBAC) +### RBAC cluster resources -| Name | Kind | Details | Created when | -| --- | --- | --- | --- | -| `gitlab` | `ServiceAccount` | `default` namespace | Creating a new GKE Cluster | -| `gitlab-admin` | `ClusterRoleBinding` | [`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) roleRef | Creating a new GKE Cluster | -| `gitlab-token` | `Secret` | Token for `gitlab` ServiceAccount | Creating a new GKE Cluster | -| `tiller` | `ServiceAccount` | `gitlab-managed-apps` namespace | Installing Helm Tiller | -| `tiller-admin` | `ClusterRoleBinding` | `cluster-admin` roleRef | Installing Helm Tiller | -| Project namespace | `ServiceAccount` | Uses namespace of Project | Deploying to a cluster | -| Project namespace | `Secret` | Token for project ServiceAccount | Deploying to a cluster | -| Project namespace | `RoleBinding` | [`edit`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) roleRef | Deploying to a cluster | +GitLab creates the following resources for RBAC clusters. + +| Name | Type | Details | Created when | +|:------------------|:---------------------|:-----------------------------------------------------------------------------------------------------------|:---------------------------| +| `gitlab` | `ServiceAccount` | `default` namespace | Creating a new GKE Cluster | +| `gitlab-admin` | `ClusterRoleBinding` | [`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) roleRef | Creating a new GKE Cluster | +| `gitlab-token` | `Secret` | Token for `gitlab` ServiceAccount | Creating a new GKE Cluster | +| `tiller` | `ServiceAccount` | `gitlab-managed-apps` namespace | Installing Helm Tiller | +| `tiller-admin` | `ClusterRoleBinding` | `cluster-admin` roleRef | Installing Helm Tiller | +| Project namespace | `ServiceAccount` | Uses namespace of Project | Deploying to a cluster | +| Project namespace | `Secret` | Token for project ServiceAccount | Deploying to a cluster | +| Project namespace | `RoleBinding` | [`edit`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) roleRef | Deploying to a cluster | NOTE: **Note:** Project-specific resources are only created if your cluster is [managed by GitLab](#gitlab-managed-clusters). @@ -346,111 +347,10 @@ install it manually. ## Installing applications -GitLab provides **GitLab Managed Apps**, a one-click install for various applications which can -be added directly to your configured cluster. These applications are -needed for [Review Apps](../../../ci/review_apps/index.md) and -[deployments](../../../ci/environments.md) when using [Auto DevOps](../../../topics/autodevops/index.md). -You can install them after you -[create a cluster](#adding-and-creating-a-new-gke-cluster-via-gitlab). - -Applications managed by GitLab will be installed onto the `gitlab-managed-apps` namespace. This differrent -from the namespace used for project deployments. It is only created once and its name is not configurable. - -To see a list of available applications to install: - -1. Navigate to your project's **Operations > Kubernetes**. -1. Select your cluster. - -Install Helm first as it's used to install other applications. - -NOTE: **Note:** -As of GitLab 11.6, Helm will be upgraded to the latest version supported -by GitLab before installing any of the applications. - -| Application | GitLab version | Description | Helm Chart | -| ----------- | :------------: | ----------- | --------------- | -| [Helm](https://docs.helm.sh/) | 10.2+ | Helm is a package manager for Kubernetes and is required to install all the other applications. It is installed in its own pod inside the cluster which can run the `helm` CLI in a safe environment. | n/a | -| [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) | 10.2+ | Ingress can provide load balancing, SSL termination, and name-based virtual hosting. It acts as a web proxy for your applications and is useful if you want to use [Auto DevOps] or deploy your own web apps. | [stable/nginx-ingress](https://github.com/helm/charts/tree/master/stable/nginx-ingress) | -| [Cert-Manager](https://docs.cert-manager.io/en/latest/) | 11.6+ | Cert-Manager is a native Kubernetes certificate management controller that helps with issuing certificates. Installing Cert-Manager on your cluster will issue a certificate by [Let's Encrypt](https://letsencrypt.org/) and ensure that certificates are valid and up-to-date. | [stable/cert-manager](https://github.com/helm/charts/tree/master/stable/cert-manager) | -| [Prometheus](https://prometheus.io/docs/introduction/overview/) | 10.4+ | Prometheus is an open-source monitoring and alerting system useful to supervise your deployed applications. | [stable/prometheus](https://github.com/helm/charts/tree/master/stable/prometheus) | -| [GitLab Runner](https://docs.gitlab.com/runner/) | 10.6+ | GitLab Runner is the open source project that is used to run your jobs and send the results back to GitLab. It is used in conjunction with [GitLab CI/CD](../../../ci/README.md), the open-source continuous integration service included with GitLab that coordinates the jobs. When installing the GitLab Runner via the applications, it will run in **privileged mode** by default. Make sure you read the [security implications](#security-implications) before doing so. | [runner/gitlab-runner](https://gitlab.com/charts/gitlab-runner) | -| [JupyterHub](http://jupyter.org/) | 11.0+ | [JupyterHub](https://jupyterhub.readthedocs.io/en/stable/) is a multi-user service for managing notebooks across a team. [Jupyter Notebooks](https://jupyter-notebook.readthedocs.io/en/latest/) provide a web-based interactive programming environment used for data analysis, visualization, and machine learning. We use a [custom Jupyter image](https://gitlab.com/gitlab-org/jupyterhub-user-image/blob/master/Dockerfile) that installs additional useful packages on top of the base Jupyter. Authentication will be enabled only for [project members](../members/index.md) with [Developer or higher](../../permissions.md) access to the project. You will also see ready-to-use DevOps Runbooks built with Nurtch's [Rubix library](https://github.com/amit1rrr/rubix). More information on creating executable runbooks can be found in [our Nurtch documentation](runbooks/index.md#nurtch-executable-runbooks). Note that Ingress must be installed and have an IP address assigned before JupyterHub can be installed. | [jupyter/jupyterhub](https://jupyterhub.github.io/helm-chart/) | -| [Knative](https://cloud.google.com/knative) | 11.5+ | Knative provides a platform to create, deploy, and manage serverless workloads from a Kubernetes cluster. It is used in conjunction with, and includes [Istio](https://istio.io) to provide an external IP address for all programs hosted by Knative. You will be prompted to enter a wildcard domain where your applications will be exposed. Configure your DNS server to use the external IP address for that domain. For any application created and installed, they will be accessible as `<program_name>.<kubernetes_namespace>.<domain_name>`. This will require your kubernetes cluster to have [RBAC enabled](#role-based-access-control-rbac). | [knative/knative](https://storage.googleapis.com/triggermesh-charts) - -With the exception of Knative, the applications will be installed in a dedicated -namespace called `gitlab-managed-apps`. - -CAUTION: **Caution:** -If you have an existing Kubernetes cluster with Helm already installed, -you should be careful as GitLab cannot detect it. In this case, installing -Helm via the applications will result in the cluster having it twice, which -can lead to confusion during deployments. - -### Upgrading applications - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/24789) -in GitLab 11.8. - -Users can perform a one-click upgrade for the GitLab Runner application, -when there is an upgrade available. - -To upgrade the GitLab Runner application: - -1. Navigate to your project's **Operations > Kubernetes**. -1. Select your cluster. -1. Click the **Upgrade** button for the Runnner application. - -The **Upgrade** button will not be shown if there is no upgrade -available. - -NOTE: **Note:** -Upgrades will reset values back to the values built into the `runner` -chart plus the values set by -[`values.yaml`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/vendor/runner/values.yaml) - -### Uninstalling applications - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/60665) in -> GitLab 11.11. - -The applications below can be uninstalled. - -| Application | GitLab version | Notes | -| ----------- | -------------- | ----- | -| Prometheus | 11.11+ | All data will be deleted and cannot be restored. | - -To uninstall an application: - -1. Navigate to your project's **Operations > Kubernetes**. -1. Select your cluster. -1. Click the **Uninstall** button for the application. - -Support for uninstalling all applications is planned for progressive rollout. -To follow progress, see [the relevant -epic](https://gitlab.com/groups/gitlab-org/-/epics/1201). - -### Troubleshooting applications - -Applications can fail with the following error: - -```text -Error: remote error: tls: bad certificate -``` - -To avoid installation errors: - -- Before starting the installation of applications, make sure that time is synchronized - between your GitLab server and your Kubernetes cluster. -- Ensure certificates are not out of sync. When installing applications, GitLab expects a new cluster with no previous installation of Helm. - - You can confirm that the certificates match via `kubectl`: - - ```sh - kubectl get configmaps/values-content-configuration-ingress -n gitlab-managed-apps -o \ - "jsonpath={.data['cert\.pem']}" | base64 -d > a.pem - kubectl get secrets/tiller-secret -n gitlab-managed-apps -o "jsonpath={.data['ca\.crt']}" | base64 -d > b.pem - diff a.pem b.pem - ``` +GitLab can install and manage some applications in your project-level +cluster. For more information on installing, upgrading, uninstalling, +and troubleshooting applications for your project cluster, see +[Gitlab Managed Apps](../../clusters/applications.md). ## Getting the external endpoint @@ -550,7 +450,7 @@ differentiate the new cluster with the rest. When adding more than one Kubernetes cluster to your project, you need to differentiate them with an environment scope. The environment scope associates clusters with [environments](../../../ci/environments.md) similar to how the -[environment-specific variables](https://docs.gitlab.com/ee/ci/variables/index.html#limiting-environment-scopes-of-environment-variables-premium) work. +[environment-specific variables](../../../ci/variables/README.md#limiting-environment-scopes-of-environment-variables-premium) work. The default environment scope is `*`, which means all jobs, regardless of their environment, will use that cluster. Each scope can only be used by a single @@ -688,7 +588,7 @@ displaying the status of the pods in the deployment. Developers and other teammates can view the progress and status of a rollout, pod by pod, in the workflow they already use without any need to access Kubernetes. -[Read more about Deploy Boards](https://docs.gitlab.com/ee/user/project/deploy_boards.html) +[Read more about Deploy Boards](../deploy_boards.md) ### Canary Deployments **[PREMIUM]** @@ -696,7 +596,7 @@ Leverage [Kubernetes' Canary deployments](https://kubernetes.io/docs/concepts/cl and visualize your canary deployments right inside the Deploy Board, without the need to leave GitLab. -[Read more about Canary Deployments](https://docs.gitlab.com/ee/user/project/canary_deployments.html) +[Read more about Canary Deployments](../canary_deployments.md) ### Pod logs **[ULTIMATE]** diff --git a/doc/user/project/clusters/kubernetes_pod_logs.md b/doc/user/project/clusters/kubernetes_pod_logs.md index d5b60250860..368031070c1 100644 --- a/doc/user/project/clusters/kubernetes_pod_logs.md +++ b/doc/user/project/clusters/kubernetes_pod_logs.md @@ -7,10 +7,10 @@ By displaying the logs directly in GitLab, developers can avoid having to manage ## Overview -[Kubernetes](https://kubernetes.io) pod logs can be viewed directly within GitLab. Logs can be displayed by clicking on a specific pod from [Deploy Boards](https://docs.gitlab.com/ee/user/project/deploy_boards.html): +[Kubernetes](https://kubernetes.io) pod logs can be viewed directly within GitLab. Logs can be displayed by clicking on a specific pod from [Deploy Boards](../deploy_boards.md): 1. Go to **Operations > Environments** and find the environment which contains the desired pod, like `production`. -1. On the **Environments** page, you should see the status of the environment's pods with [Deploy Boards](https://docs.gitlab.com/ee/user/project/deploy_boards.html). +1. On the **Environments** page, you should see the status of the environment's pods with [Deploy Boards](../deploy_boards.md). 1. When mousing over the list of pods, a tooltip will appear with the exact pod name and status.  1. Click on the desired pod to bring up the logs view, which will contain the last 500 lines for that pod. Support for pods with multiple containers is coming [in a future release](https://gitlab.com/gitlab-org/gitlab-ee/issues/6502). @@ -18,4 +18,4 @@ By displaying the logs directly in GitLab, developers can avoid having to manage ## Requirements -[Enabling Deploy Boards](https://docs.gitlab.com/ee/user/project/deploy_boards.html#enabling-deploy-boards) is required in order to be able to use Pod Logs. +[Enabling Deploy Boards](../deploy_boards.md#enabling-deploy-boards) is required in order to be able to use Pod Logs. diff --git a/doc/user/project/clusters/runbooks/index.md b/doc/user/project/clusters/runbooks/index.md index 54c475a1762..6360a01a0ad 100644 --- a/doc/user/project/clusters/runbooks/index.md +++ b/doc/user/project/clusters/runbooks/index.md @@ -34,7 +34,7 @@ for an overview of how this is accomplished in GitLab!** To create an executable runbook, you will need: 1. **Kubernetes** - A Kubernetes cluster is required to deploy the rest of the applications. - The simplest way to get started is to add a cluster using [GitLab's GKE integration](https://docs.gitlab.com/ee/user/project/clusters/#adding-and-creating-a-new-gke-cluster-via-gitlab). + The simplest way to get started is to add a cluster using [GitLab's GKE integration](../index.md#adding-and-creating-a-new-gke-cluster-via-gitlab). 1. **Helm Tiller** - Helm is a package manager for Kubernetes and is required to install all the other applications. It is installed in its own pod inside the cluster which can run the helm CLI in a safe environment. @@ -59,7 +59,7 @@ the components outlined above and the preloaded demo runbook. ### 1. Add a Kubernetes cluster -Follow the steps outlined in [Adding and creating a new GKE cluster via GitLab](https://docs.gitlab.com/ee/user/project/clusters/#adding-and-creating-a-new-gke-cluster-via-gitlab) +Follow the steps outlined in [Adding and creating a new GKE cluster via GitLab](../index.md#adding-and-creating-a-new-gke-cluster-via-gitlab) to add a Kubernetes cluster to your project. ### 2. Install Helm Tiller, Ingress, and JupyterHub @@ -90,7 +90,7 @@ The server will take a couple of seconds to start. ### 4. Configure access In order for the runbook to access your GitLab project, you will need to enter a -[GitLab Access Token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) +[GitLab Access Token](../../../profile/personal_access_tokens.md) as well as your Project ID in the **Setup** section of the demo runbook. Double-click the **DevOps-Runbook-Demo** folder located on the left panel. diff --git a/doc/user/project/container_registry.md b/doc/user/project/container_registry.md index 83b268db967..58b7fe33906 100644 --- a/doc/user/project/container_registry.md +++ b/doc/user/project/container_registry.md @@ -53,7 +53,7 @@ If you visit the **Registry** link under your project's menu, you can see the explicit instructions to login to the Container Registry using your GitLab credentials. -For example if the Registry's URL is `registry.example.com`, the you should be +For example if the Registry's URL is `registry.example.com`, then you should be able to login with: ``` @@ -204,7 +204,7 @@ at the communication between the client and the Registry. The REST API between the Docker client and Registry is [described here](https://docs.docker.com/registry/spec/api/). Normally, one would just use Wireshark or tcpdump to capture the traffic and see where things went -wrong. However, since all communication between Docker clients and servers +wrong. However, since all communications between Docker clients and servers are done over HTTPS, it's a bit difficult to decrypt the traffic quickly even if you know the private key. What can we do instead? diff --git a/doc/user/project/deploy_tokens/index.md b/doc/user/project/deploy_tokens/index.md index 7688508c6ac..92a29b68a22 100644 --- a/doc/user/project/deploy_tokens/index.md +++ b/doc/user/project/deploy_tokens/index.md @@ -5,7 +5,7 @@ Deploy tokens allow to download (through `git clone`), or read the container registry images of a project without the need of having a user and a password. Please note, that the expiration of deploy tokens happens on the date you define, -at midnight UTC and that they can be only managed by [maintainers](https://docs.gitlab.com/ee/user/permissions.html). +at midnight UTC and that they can be only managed by [maintainers](../../permissions.md). ## Creating a Deploy Token diff --git a/doc/user/project/import/gemnasium.md b/doc/user/project/import/gemnasium.md index dc5b3fcd0bb..7f79ebf6353 100644 --- a/doc/user/project/import/gemnasium.md +++ b/doc/user/project/import/gemnasium.md @@ -9,9 +9,9 @@ Gemnasium has been [acquired by GitLab](https://about.gitlab.com/press/releases/ in January 2018. Since May 15, 2018, the services provided by Gemnasium are no longer available. The team behind Gemnasium has joined GitLab as the new Security Products team and is working on a wider range of tools than just Dependency Scanning: -[SAST](https://docs.gitlab.com/ee/user/application_security/sast/index.html), -[DAST](https://docs.gitlab.com/ee/user/application_security/dast/index.html), -[Container Scanning](https://docs.gitlab.com/ee/user/application_security/container_scanning/index.html) and more. +[SAST](../../application_security/sast/index.md), +[DAST](../../application_security/dast/index.md), +[Container Scanning](../../application_security/container_scanning/index.md) and more. If you want to continue monitoring your dependencies, see the [Migrating to GitLab](#migrating-to-gitlab) section below. @@ -45,7 +45,7 @@ Security features are free for public (open-source) projects hosted on GitLab.co You're almost set! If you're already using [Auto DevOps](../../../topics/autodevops/), you are already covered. Otherwise, you must configure your `.gitlab-ci.yml` according to the -[dependency scanning page](https://docs.gitlab.com/ee/user/application_security/dependency_scanning/index.html). +[dependency scanning page](../../application_security/dependency_scanning/index.md). ### If your project is hosted on GitHub (https://github.com / GitHub Enterprise) @@ -81,7 +81,7 @@ back to both GitLab and GitHub when completed. 1. To set up the dependency scanning job, corresponding to what Gemnasium was doing, you must create a `.gitlab-ci.yml` file, or update it according to - the [dependency scanning docs](https://docs.gitlab.com/ee/user/application_security/dependency_scanning/index.html). + the [dependency scanning docs](../../application_security/dependency_scanning/index.md). The mirroring is pull-only by default, so you may create or update the file on GitHub: diff --git a/doc/user/project/import/github.md b/doc/user/project/import/github.md index 63b90dd76fd..8fba892594b 100644 --- a/doc/user/project/import/github.md +++ b/doc/user/project/import/github.md @@ -119,9 +119,9 @@ Depending your GitLab tier, [project mirroring](../../../workflow/repository_mir your imported project in sync with its GitHub copy. Additionally, you can configure GitLab to send pipeline status updates back GitHub with the -[GitHub Project Integration](https://docs.gitlab.com/ee/user/project/integrations/github.html). **[PREMIUM]** +[GitHub Project Integration](../integrations/github.md). **[PREMIUM]** -If you import your project using [CI/CD for external repo](https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/), then both +If you import your project using [CI/CD for external repo](../../../ci/ci_cd_for_external_repos/index.md), then both of the above are automatically configured. **[PREMIUM]** ## Improving the speed of imports on self-hosted instances diff --git a/doc/user/project/import/gitlab_com.md b/doc/user/project/import/gitlab_com.md index 3b37da67a5b..f48a158e2d3 100644 --- a/doc/user/project/import/gitlab_com.md +++ b/doc/user/project/import/gitlab_com.md @@ -1,8 +1,9 @@ # Project importing from GitLab.com to your private GitLab instance You can import your existing GitLab.com projects to your GitLab instance. But keep in mind that it is possible only if -GitLab support is enabled on your GitLab instance. -You can read more about GitLab support [here](http://docs.gitlab.com/ce/integration/gitlab.html) +GitLab.com integration is enabled on your GitLab instance. +[Read more about GitLab.com integration for self-managed GitLab instances](../../../integration/gitlab.md). + To get to the importer page you need to go to "New project" page. >**Note:** diff --git a/doc/user/project/import/index.md b/doc/user/project/import/index.md index ebbc5ca133b..2b6927bd780 100644 --- a/doc/user/project/import/index.md +++ b/doc/user/project/import/index.md @@ -14,12 +14,13 @@ 1. [From repo by URL](repo_by_url.md) 1. [By uploading a manifest file (AOSP)](manifest.md) 1. [From Gemnasium](gemnasium.md) +1. [From Phabricator](phabricator.md) In addition to the specific migration documentation above, you can import any Git repository via HTTP from the New Project page. Be aware that if the repository is too large the import can timeout. -There is also the option of [connecting your external repository to get CI/CD benefits](https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/index.html). **[PREMIUM]** +There is also the option of [connecting your external repository to get CI/CD benefits](../../../ci/ci_cd_for_external_repos/index.md). **[PREMIUM]** ## Migrating from self-hosted GitLab to GitLab.com diff --git a/doc/user/project/import/phabricator.md b/doc/user/project/import/phabricator.md new file mode 100644 index 00000000000..5c624e3aff6 --- /dev/null +++ b/doc/user/project/import/phabricator.md @@ -0,0 +1,29 @@ +# Import Phabricator tasks into a GitLab project + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/60562) in +GitLab 12.0. + +GitLab allows you to import all tasks from a Phabricator instance into +GitLab issues. The import creates a single project with the +repository disabled. + +Currently, only the following basic fields are imported: + +- Title +- Description +- State (open or closed) +- Created at +- Closed at + +## Enabling this feature + +While this feature is incomplete, a feature flag is required to enable it so that +we can gain early feedback before releasing it for everyone. To enable it: + +1. Run the following command in a Rails console: + + ```ruby + Feature.enable(:phabricator_import) + ``` + +1. Enable Phabricator as an [import source](../../admin_area/settings/visibility_and_access_controls.md#import-sources) in the Admin area. diff --git a/doc/user/project/index.md b/doc/user/project/index.md index 6b3b40bf9f8..a24f525253d 100644 --- a/doc/user/project/index.md +++ b/doc/user/project/index.md @@ -37,7 +37,7 @@ When you create a project in GitLab, you'll have access to a large number of - [Multiple Issue Boards](issue_board.md#multiple-issue-boards-starter): Allow your teams to create their own workflows (Issue Boards) for the same project **[STARTER]** - [Merge Requests](merge_requests/index.md): Apply your branching strategy and get reviewed by your team - - [Merge Request Approvals](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html): Ask for approval before + - [Merge Request Approvals](merge_requests/merge_request_approvals.md): Ask for approval before implementing a change **[STARTER]** - [Fix merge conflicts from the UI](merge_requests/resolve_conflicts.md): Your Git diff tool right from GitLab's UI @@ -74,7 +74,7 @@ When you create a project in GitLab, you'll have access to a large number of timeout (defines the maximum amount of time in minutes that a job is able run), custom path for `.gitlab-ci.yml`, test coverage parsing, pipeline's visibility, and much more - [Kubernetes cluster integration](clusters/index.md): Connecting your GitLab project with a Kubernetes cluster - - [Feature Flags](https://docs.gitlab.com/ee/user/project/operations/feature_flags.html): Feature flags allow you to ship a project in + - [Feature Flags](operations/feature_flags.md): Feature flags allow you to ship a project in different flavors by dynamically toggling certain functionality **[PREMIUM]** - [GitLab Pages](pages/index.md): Build, test, and deploy your static website with GitLab Pages @@ -91,10 +91,10 @@ When you create a project in GitLab, you'll have access to a large number of - [Releases](releases/index.md): a way to track deliverables in your project as snapshot in time of the source, build output, and other metadata or artifacts associated with a released version of your code. -- [Maven packages](https://docs.gitlab.com/ee/user/project/packages/maven_repository.html): your private Maven repository in GitLab. **[PREMIUM]** -- [NPM packages](https://docs.gitlab.com/ee/user/project/packages/npm_registry.html): your private NPM package registry in GitLab. **[PREMIUM]** +- [Maven packages](packages/maven_repository.md): your private Maven repository in GitLab. **[PREMIUM]** +- [NPM packages](packages/npm_registry.md): your private NPM package registry in GitLab. **[PREMIUM]** - [Code owners](code_owners.md): specify code owners for certain files **[STARTER]** -- [License Management](https://docs.gitlab.com/ee/user/application_security/license_management/index.html): approve and blacklist licenses for projects. **[ULTIMATE]** +- [License Management](../application_security/license_management/index.md): approve and blacklist licenses for projects. **[ULTIMATE]** ### Project integrations @@ -135,7 +135,7 @@ Read through the documentation on [project settings](settings/index.md). Instead of importing a repository directly to GitLab, you can connect your repository as a CI/CD project. -Read through the documentation on [CI/CD for external repositories](https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/index.html). +Read through the documentation on [CI/CD for external repositories](../../ci/ci_cd_for_external_repos/index.md). ## Project members diff --git a/doc/user/project/insights/index.md b/doc/user/project/insights/index.md index b6cc1862cc2..5154ff38154 100644 --- a/doc/user/project/insights/index.md +++ b/doc/user/project/insights/index.md @@ -14,7 +14,7 @@ requests to be merged and much more.  NOTE: **Note:** -This feature is [also available at the group level](https://docs.gitlab.com/ee/user/group/insights/index.html). +This feature is [also available at the group level](../../group/insights/index.md). ## View your project's Insights @@ -33,7 +33,7 @@ for details about the content of this file. NOTE: **Note:** Once the configuration file is created, you can also -[use it for your project's group](https://docs.gitlab.com/ee/user/group/insights/index.html#configure-your-insights). +[use it for your project's group](../../group/insights/index.md#configure-your-insights). NOTE: **Note:** If the project doesn't have any configuration file, it'll try to use diff --git a/doc/user/project/integrations/project_services.md b/doc/user/project/integrations/project_services.md index f560de427c5..0bfee3bac99 100644 --- a/doc/user/project/integrations/project_services.md +++ b/doc/user/project/integrations/project_services.md @@ -39,7 +39,7 @@ Click on the service links to see further configuration instructions and details | [HipChat](hipchat.md) | Private group chat and IM | | [Irker (IRC gateway)](irker.md) | Send IRC messages, on update, to a list of recipients through an Irker gateway | | [JIRA](jira.md) | JIRA issue tracker | -| [Jenkins](https://docs.gitlab.com/ee/integration/jenkins.html) **[STARTER]** | An extendable open source continuous integration server | +| [Jenkins](../../../integration/jenkins.md) **[STARTER]** | An extendable open source continuous integration server | | JetBrains TeamCity CI | A continuous integration and build server | | [Mattermost slash commands](mattermost_slash_commands.md) | Mattermost chat and ChatOps slash commands | | [Mattermost Notifications](mattermost.md) | Receive event notifications in Mattermost | diff --git a/doc/user/project/integrations/prometheus_library/kubernetes.md b/doc/user/project/integrations/prometheus_library/kubernetes.md index 436129f1dbc..8b1cf1a251a 100644 --- a/doc/user/project/integrations/prometheus_library/kubernetes.md +++ b/doc/user/project/integrations/prometheus_library/kubernetes.md @@ -27,7 +27,7 @@ integration services must be enabled. Prometheus needs to be deployed into the cluster and configured properly in order to gather Kubernetes metrics. GitLab supports two methods for doing so: -- GitLab [integrates with Kubernetes](../../clusters/index.md), and can [deploy Prometheus into a connected cluster](../prometheus.html#managed-prometheus-on-kubernetes). It is automatically configured to collect Kubernetes metrics. +- GitLab [integrates with Kubernetes](../../clusters/index.md), and can [deploy Prometheus into a connected cluster](../prometheus.md#managed-prometheus-on-kubernetes). It is automatically configured to collect Kubernetes metrics. - To configure your own Prometheus server, you can follow the [Prometheus documentation](https://prometheus.io/docs/introduction/overview/). ## Specifying the Environment @@ -40,7 +40,7 @@ Instead, the [Deployment](https://kubernetes.io/docs/concepts/workloads/controll > Introduced in [GitLab 10.2](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/15201). -GitLab also gathers Kubernetes metrics for [canary deployments](https://docs.gitlab.com/ee/user/project/canary_deployments.html), allowing easy comparison between the current deployed version and the canary. +GitLab also gathers Kubernetes metrics for [canary deployments](../../canary_deployments.md), allowing easy comparison between the current deployed version and the canary. These metrics expect the [Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) or [DaemonSet](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) name to begin with `$CI_ENVIRONMENT_SLUG-canary`, to isolate the canary metrics. diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index 10eb3a4f3b7..a0f500a939f 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -1016,7 +1016,7 @@ X-Gitlab-Event: Pipeline Hook "email": "user@gitlab.com" } }, - "jobs":[ + "builds":[ { "id": 380, "stage": "deploy", diff --git a/doc/user/project/issues/create_new_issue.md b/doc/user/project/issues/create_new_issue.md index 5e05846b77f..c2916c79876 100644 --- a/doc/user/project/issues/create_new_issue.md +++ b/doc/user/project/issues/create_new_issue.md @@ -67,7 +67,7 @@ or contacts to continue working._ ## New issue via Service Desk **[PREMIUM]** -Enable [Service Desk](https://docs.gitlab.com/ee/user/project/service_desk.html) to your project and offer email support. +Enable [Service Desk](../service_desk.md) to your project and offer email support. By doing so, when your customer sends a new email, a new issue can be created in the appropriate project and followed up from there. diff --git a/doc/user/project/issues/index.md b/doc/user/project/issues/index.md index c82b7f100d2..94865ad46ee 100644 --- a/doc/user/project/issues/index.md +++ b/doc/user/project/issues/index.md @@ -102,13 +102,13 @@ For more information, see the [Issue Boards](../issue_board.md) page. Epics let you manage your portfolio of projects more efficiently and with less effort by tracking groups of issues that share a theme, across projects and milestones. -For more information, see the [Epics](https://docs.gitlab.com/ee/user/group/epics/) page. +For more information, see the [Epics](../../group/epics/index.md) page. ### Related issues **[STARTER]** You can mark two issues as related, so that when viewing each one, the other is always listed in its Related Issues section. This can help display important context, such as past work, dependencies, or duplicates. -For more information, see [Related Issues](https://docs.gitlab.com/ee/user/project/issues/related_issues.html). +For more information, see [Related Issues](related_issues.md). ### Crosslinking issues @@ -129,7 +129,7 @@ For more information, see [Crosslinking issues](crosslinking_issues.md). - [Bulk edit issues](../bulk_editing.md) - From the Issues List, select multiple issues in order to change their status, assignee, milestone, or labels in bulk. - [Import issues](csv_import.md) -- [Export issues](https://docs.gitlab.com/ee/user/project/issues/csv_export.html) **[STARTER]** +- [Export issues](csv_export.md) **[STARTER]** - [Issues API](../../../api/issues.md) - Configure an [external issue tracker](../../../integration/external-issue-tracker.md) such as Jira, Redmine, or Bugzilla. diff --git a/doc/user/project/issues/issue_data_and_actions.md b/doc/user/project/issues/issue_data_and_actions.md index ef9fcaec3e6..fc11c0251e0 100644 --- a/doc/user/project/issues/issue_data_and_actions.md +++ b/doc/user/project/issues/issue_data_and_actions.md @@ -90,7 +90,7 @@ If a label doesn't exist yet, you can click **Edit**, and it opens a dropdown me - Assign a weight. Larger values are used to indicate more effort is required to complete the issue. Only positive values or zero are allowed. -Learn more in the [Issue Weight documentation](https://docs.gitlab.com/ee/workflow/issue_weight.html). +Learn more in the [Issue Weight documentation](../../../workflow/issue_weight.md). #### 9. Participants @@ -103,7 +103,7 @@ Learn more in the [Issue Weight documentation](https://docs.gitlab.com/ee/workfl - Unsubscribe: if you are receiving notifications on that issue but no longer want to receive them, unsubscribe from it. -Read more in the [notifications documentation](https://docs.gitlab.com/ee/workflow/notifications.html#issue--epics--merge-request-events). +Read more in the [notifications documentation](../../../workflow/notifications.md#issue--epics--merge-request-events). #### 11. Reference diff --git a/doc/user/project/issues/multiple_assignees_for_issues.md b/doc/user/project/issues/multiple_assignees_for_issues.md index 8781ebdd5b0..d1db0790d69 100644 --- a/doc/user/project/issues/multiple_assignees_for_issues.md +++ b/doc/user/project/issues/multiple_assignees_for_issues.md @@ -22,7 +22,7 @@ Consider a team formed by frontend developers, backend developers, UX designers, QA testers, and a product manager working together to bring an idea to market. -Multiple Assignees for Issues makes collaboration smother, +Multiple Assignees for Issues makes collaboration smoother, and allows shared responsibilities to be clearly displayed. All assignees are shown across your team's workflows and receive notifications (as they would as single assignees), simplifying communication and ownership. diff --git a/doc/user/project/issues/related_issues.md b/doc/user/project/issues/related_issues.md index db0ab65b442..e679ebf86e6 100644 --- a/doc/user/project/issues/related_issues.md +++ b/doc/user/project/issues/related_issues.md @@ -1,6 +1,6 @@ # Related issues **[STARTER]** -> [Introduced][ee-1797] in [GitLab Starter][ee] 9.4. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/1797) in [GitLab Starter](https://about.gitlab.com/pricing/) 9.4. Related issues are a bi-directional relationship between any two issues and appear in a block below the issue description. Issues can be across groups @@ -35,11 +35,6 @@ will no longer appear in either issue.  -Please access our [permissions] page for more information. +Please access our [permissions](../../permissions.md) page for more information. -Additionally, you are also able to manage related issues through [our API]. - -[ee]: https://about.gitlab.com/pricing/ -[ee-1797]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/1797 -[permissions]: ../../permissions.md -[Our API]: https://docs.gitlab.com/ee/api/issue_links.html +Additionally, you are also able to manage related issues through [our API](../../../api/issue_links.md). diff --git a/doc/user/project/maven_packages.md b/doc/user/project/maven_packages.md index d32d6084b38..48835a2dac7 100644 --- a/doc/user/project/maven_packages.md +++ b/doc/user/project/maven_packages.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/project/packages/maven_repository.html' +redirect_to: 'packages/maven_repository.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/user/project/packages/maven_repository.html). +This document was moved to [another location](packages/maven_repository.md). diff --git a/doc/user/project/merge_requests/code_quality_diff.md b/doc/user/project/merge_requests/code_quality_diff.md index 890058eec6f..ccc694672a6 100644 --- a/doc/user/project/merge_requests/code_quality_diff.md +++ b/doc/user/project/merge_requests/code_quality_diff.md @@ -1,5 +1,5 @@ --- -redirect_from: 'https://docs.gitlab.com/ee/user/project/merge_requests/code_quality_diff.html' +redirect_from: 'code_quality_diff.md' redirect_to: 'code_quality.md' --- diff --git a/doc/user/project/merge_requests/container_scanning.md b/doc/user/project/merge_requests/container_scanning.md index 4d41e424f4a..a062731ea35 100644 --- a/doc/user/project/merge_requests/container_scanning.md +++ b/doc/user/project/merge_requests/container_scanning.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/application_security/container_scanning/index.html' +redirect_to: '../../application_security/container_scanning/index.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/user/application_security/container_scanning/index.html). +This document was moved to [another location](../../application_security/container_scanning/index.md). diff --git a/doc/user/project/merge_requests/dast.md b/doc/user/project/merge_requests/dast.md index b676c661267..98a2906e560 100644 --- a/doc/user/project/merge_requests/dast.md +++ b/doc/user/project/merge_requests/dast.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/application_security/dast/index.html' +redirect_to: '../../application_security/dast/index.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/user/application_security/dast/index.html). +This document was moved to [another location](../../application_security/dast/index.md). diff --git a/doc/user/project/merge_requests/dependency_scanning.md b/doc/user/project/merge_requests/dependency_scanning.md index 3a8b53b425c..bdc1c355016 100644 --- a/doc/user/project/merge_requests/dependency_scanning.md +++ b/doc/user/project/merge_requests/dependency_scanning.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/application_security/dependency_scanning/index.html' +redirect_to: '../../application_security/dependency_scanning/index.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/user/application_security/dependency_scanning/index.html). +This document was moved to [another location](../../application_security/dependency_scanning/index.md). diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index 09736c7fc7e..4cfe59b808a 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -33,15 +33,15 @@ With GitLab merge requests, you can: With **[GitLab Enterprise Edition][ee]**, you can also: -- Prepare a full review and submit it once it's ready with [Merge Request Reviews](https://docs.gitlab.com/ee/user/discussions/index.md#merge-request-reviews-premium) **[PREMIUM]** -- View the deployment process across projects with [Multi-Project Pipelines](https://docs.gitlab.com/ee/ci/multi_project_pipelines.md) **[PREMIUM]** +- Prepare a full review and submit it once it's ready with [Merge Request Reviews](../../discussions/index.md#merge-request-reviews-premium) **[PREMIUM]** +- View the deployment process across projects with [Multi-Project Pipelines](../../../ci/multi_project_pipelines.md) **[PREMIUM]** - Request [approvals](merge_request_approvals.md) from your managers **[STARTER]** - Analyze the impact of your changes with [Code Quality reports](code_quality.md) **[STARTER]** -- Manage the licenses of your dependencies with [License Management](https://docs.gitlab.com/ee/user/application_security/license_management/index.md) **[ULTIMATE]** -- Analyze your source code for vulnerabilities with [Static Application Security Testing](https://docs.gitlab.com/ee/user/application_security/sast/index.md) **[ULTIMATE]** -- Analyze your running web applications for vulnerabilities with [Dynamic Application Security Testing](https://docs.gitlab.com/ee/user/application_security/dast/index.md) **[ULTIMATE]** -- Analyze your dependencies for vulnerabilities with [Dependency Scanning](https://docs.gitlab.com/ee/user/application_security/dependency_scanning/index.md) **[ULTIMATE]** -- Analyze your Docker images for vulnerabilities with [Container Scanning](https://docs.gitlab.com/ee/user/application_security/container_scanning/index.md) **[ULTIMATE]** +- Manage the licenses of your dependencies with [License Management](../../application_security/license_management/index.md) **[ULTIMATE]** +- Analyze your source code for vulnerabilities with [Static Application Security Testing](../../application_security/sast/index.md) **[ULTIMATE]** +- Analyze your running web applications for vulnerabilities with [Dynamic Application Security Testing](../../application_security/dast/index.md) **[ULTIMATE]** +- Analyze your dependencies for vulnerabilities with [Dependency Scanning](../../application_security/dependency_scanning/index.md) **[ULTIMATE]** +- Analyze your Docker images for vulnerabilities with [Container Scanning](../../application_security/container_scanning/index.md) **[ULTIMATE]** - Determine the performance impact of changes with [Browser Performance Testing](#browser-performance-testing-premium) **[PREMIUM]** ## Use cases @@ -174,7 +174,7 @@ Start a review in order to create multiple comments on a diff and publish them o Starting a review allows you to get all your thoughts in order and ensure you haven't missed anything before submitting all your comments. -[Learn more about Merge Request Reviews](https://docs.gitlab.com/ee/user/discussions/index.html#merge-request-reviews-premium) +[Learn more about Merge Request Reviews](../../discussions/index.md#merge-request-reviews-premium) ## Squash and merge @@ -395,7 +395,7 @@ GitLab runs the [Sitespeed.io container][sitespeed-container] and displays the d GitLab can scan and report any vulnerabilities found in your project. -[Read more about security reports.](https://docs.gitlab.com/ee/user/application_security/index.html) +[Read more about security reports.](../../application_security/index.md) ## JUnit test reports diff --git a/doc/user/project/merge_requests/license_management.md b/doc/user/project/merge_requests/license_management.md index 08704425a75..93116ebd7c6 100644 --- a/doc/user/project/merge_requests/license_management.md +++ b/doc/user/project/merge_requests/license_management.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/application_security/license_management/index.html' +redirect_to: '../../application_security/license_management/index.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/user/application_security/license_management/index.html). +This document was moved to [another location](../../application_security/license_management/index.md). diff --git a/doc/user/project/merge_requests/merge_request_approvals.md b/doc/user/project/merge_requests/merge_request_approvals.md index d0291c4cef5..2e9db949890 100644 --- a/doc/user/project/merge_requests/merge_request_approvals.md +++ b/doc/user/project/merge_requests/merge_request_approvals.md @@ -3,7 +3,7 @@ > Introduced in [GitLab Enterprise Edition 7.12](https://about.gitlab.com/2015/06/22/gitlab-7-12-released/#merge-request-approvers-ee-only). NOTE: **Note:** -If you are running a self-managed instance, the new interface shown on +Prior to 12.0, if you are running a self-managed instance, the new interface shown on this page will not be available unless the feature flag `approval_rules` is enabled, which can be done from the Rails console by instance administrators. @@ -105,7 +105,7 @@ any [eligible approver](#eligible-approvers) may approve. The following can approve merge requests: - Users being added as approvers at project or merge request level. -- [Code owners](https://docs.gitlab.com/ee/user/project/code_owners.html) related to the merge request ([introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/7933) in [GitLab Starter](https://about.gitlab.com/pricing/) 11.5). +- [Code owners](../code_owners.md) related to the merge request ([introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/7933) in [GitLab Starter](https://about.gitlab.com/pricing/) 11.5). An individual user can be added as an approver for a project if they are a member of: @@ -168,7 +168,7 @@ or a [failed CI/CD pipeline](merge_when_pipeline_succeeds.md). > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/4418) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.9. It is possible to require at least one approval for each entry in the -[`CODEOWNERS` file](https://docs.gitlab.com/ee/user/project/code_owners.html) that matches a file changed in +[`CODEOWNERS` file](../code_owners.md) that matches a file changed in the merge request. To enable this feature: 1. Navigate to your project's **Settings > General** and expand diff --git a/doc/user/project/merge_requests/sast.md b/doc/user/project/merge_requests/sast.md index 688cc79d0f6..165290eb114 100644 --- a/doc/user/project/merge_requests/sast.md +++ b/doc/user/project/merge_requests/sast.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/application_security/sast/index.html' +redirect_to: '../../application_security/sast/index.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/user/application_security/sast/index.html). +This document was moved to [another location](../../application_security/sast/index.md). diff --git a/doc/user/project/merge_requests/sast_docker.md b/doc/user/project/merge_requests/sast_docker.md index 4d41e424f4a..a062731ea35 100644 --- a/doc/user/project/merge_requests/sast_docker.md +++ b/doc/user/project/merge_requests/sast_docker.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/application_security/container_scanning/index.html' +redirect_to: '../../application_security/container_scanning/index.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/user/application_security/container_scanning/index.html). +This document was moved to [another location](../../application_security/container_scanning/index.md). diff --git a/doc/user/project/milestones/burndown_charts.md b/doc/user/project/milestones/burndown_charts.md index 0ad08da8ff5..7ffeb032d7f 100644 --- a/doc/user/project/milestones/burndown_charts.md +++ b/doc/user/project/milestones/burndown_charts.md @@ -52,7 +52,7 @@ and select a milestone from your current ones, while for group's, access the **G select a group, and go through **Issues > Milestones** on the sidebar. NOTE: **Note:** -You're able to [promote project](https://docs.gitlab.com/ee/user/project/milestones/#promoting-project-milestones-to-group-milestones) to group milestones and still see the **Burndown Chart** for them, respecting license limitations. +You're able to [promote project](index.md#promoting-project-milestones-to-group-milestones) to group milestones and still see the **Burndown Chart** for them, respecting license limitations. The chart indicates the project's progress throughout that milestone (for issues assigned to it). diff --git a/doc/user/project/milestones/index.md b/doc/user/project/milestones/index.md index 0d8ee0a6cd2..6cd866b5c0d 100644 --- a/doc/user/project/milestones/index.md +++ b/doc/user/project/milestones/index.md @@ -138,7 +138,7 @@ For group milestones in [GitLab Premium](https://about.gitlab.com/pricing), a [b The milestone sidebar on the milestone view shows the following: -- Percentage complete, which is calculated as number of closed issues plus number of closed/merged merge requests divided by total number issues and merge requests. +- Percentage complete, which is calculated as number of closed issues divided by total number of issues. - The start date and due date. - The total time spent on all issues that have the milestone assigned. diff --git a/doc/user/project/packages/maven_repository.md b/doc/user/project/packages/maven_repository.md index 94785eb6aec..9b7af738696 100644 --- a/doc/user/project/packages/maven_repository.md +++ b/doc/user/project/packages/maven_repository.md @@ -12,7 +12,7 @@ project can have its own space to store its Maven artifacts. NOTE: **Note:** This option is available only if your GitLab administrator has -[enabled support for the Maven repository](https://docs.gitlab.com/ee/administration/packages.html).**[PREMIUM ONLY]** +[enabled support for the Maven repository](../../../administration/packages.md).**[PREMIUM ONLY]** After the Packages feature is enabled, the Maven Repository will be available for all new projects by default. To enable it for existing projects, or if you want diff --git a/doc/user/project/packages/npm_registry.md b/doc/user/project/packages/npm_registry.md index 9f4c01c9a0a..2e274573434 100644 --- a/doc/user/project/packages/npm_registry.md +++ b/doc/user/project/packages/npm_registry.md @@ -20,7 +20,7 @@ within a subgroup is not supported yet. NOTE: **Note:** This option is available only if your GitLab administrator has -[enabled support for the NPM registry](https://docs.gitlab.com/ee/administration/packages.html).**[PREMIUM ONLY]** +[enabled support for the NPM registry](../../../administration/packages.md).**[PREMIUM ONLY]** After the NPM registry is enabled, it will be available for all new projects by default. To enable it for existing projects, or if you want to disable it: diff --git a/doc/user/project/pages/getting_started_part_two.md b/doc/user/project/pages/getting_started_part_two.md index faf51154e3b..b74520e6556 100644 --- a/doc/user/project/pages/getting_started_part_two.md +++ b/doc/user/project/pages/getting_started_part_two.md @@ -88,7 +88,7 @@ website from your project's **Settings > Pages**. You can also take some **optional** further steps: -- _Remove the fork relationship._ The fork relashionship is necessary to contribute back to the project you originally forked from. If you don't have any intentions to do so, you can remove it. To do so, navigate to your project's **Settings**, expand **Advanced settings**, and scroll down to **Remove fork relationship**: +- _Remove the fork relationship._ The fork relationship is necessary to contribute back to the project you originally forked from. If you don't have any intentions to do so, you can remove it. To do so, navigate to your project's **Settings**, expand **Advanced settings**, and scroll down to **Remove fork relationship**:  diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md index 15eb862b431..1d640966013 100644 --- a/doc/user/project/quick_actions.md +++ b/doc/user/project/quick_actions.md @@ -35,7 +35,7 @@ discussions, and descriptions: | `/label ~label1 ~label2` | Add label(s). Label names can also start without ~ but mixed syntax is not supported. | ✓ | ✓ | | `/unlabel ~label1 ~label2` | Remove all or specific label(s)| ✓ | ✓ | | `/relabel ~label1 ~label2` | Replace label | ✓ | ✓ | -| <code>/copy_metadata #issue | !merge_request</code> | Copy labels and milestone from other issue or merge request | ✓ | ✓ | +| <code>/copy_metadata #issue | !merge_request</code> | Copy labels and milestone from other issue or merge request in the project | ✓ | ✓ | | <code>/estimate <1w 3d 2h 14m></code> | Set time estimate | ✓ | ✓ | | `/remove_estimate` | Remove time estimate | ✓ | ✓ | | <code>/spend <time(1h 30m | -1h 5m)> <date(YYYY-MM-DD)></code> | Add or subtract spent time; optionally, specify the date that time was spent on | ✓ | ✓ | diff --git a/doc/user/project/repository/index.md b/doc/user/project/repository/index.md index cb514b76a4e..6fccfd40987 100644 --- a/doc/user/project/repository/index.md +++ b/doc/user/project/repository/index.md @@ -123,7 +123,7 @@ You can live preview changes submitted to a new branch with [Review Apps](../../../ci/review_apps/index.md). With [GitLab Starter](https://about.gitlab.com/pricing/), you can also request -[approval](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html) from your managers. +[approval](../merge_requests/merge_request_approvals.md) from your managers. To create, delete, and view [branches](branches/index.md) via GitLab's UI: @@ -154,7 +154,7 @@ Via command line, you can commit multiple times before pushing. you will trigger a pipeline per push, not per commit. - **Skip pipelines:** You can add to you commit message the keyword - [`[ci skip]`](../../../ci/yaml/README.html#skipping-jobs) + [`[ci skip]`](../../../ci/yaml/README.md#skipping-jobs) and GitLab CI will skip that pipeline. - **Cross-link issues and merge requests:** [Cross-linking](../issues/crosslinking_issues.md#from-commit-messages) @@ -226,7 +226,7 @@ Find it under your project's **Repository > Compare**. ## Locked files **[PREMIUM]** -Use [File Locking](https://docs.gitlab.com/ee/user/project/file_lock.html) to +Use [File Locking](../file_lock.md) to lock your files to prevent any conflicting changes. ## Repository's API diff --git a/doc/user/project/repository/reducing_the_repo_size_using_git.md b/doc/user/project/repository/reducing_the_repo_size_using_git.md index 2339759ecc8..e3d771524ce 100644 --- a/doc/user/project/repository/reducing_the_repo_size_using_git.md +++ b/doc/user/project/repository/reducing_the_repo_size_using_git.md @@ -1,6 +1,6 @@ # Reducing the repository size using Git -A GitLab Enterprise Edition administrator can set a [repository size limit][admin-repo-size] +A GitLab Enterprise Edition administrator can set a [repository size limit](../../admin_area/settings/account_and_limit_settings.md) which will prevent you from exceeding it. When a project has reached its size limit, you will not be able to push to it, @@ -14,7 +14,8 @@ move some blobs to LFS, or remove some old dependency updates from history. Unfortunately, it's not so easy and that workflow won't work. Deleting files in a commit doesn't actually reduce the size of the repo since the earlier commits and blobs are still around. What you need to do is rewrite history with Git's -[`filter-branch` option][gitscm], or a tool like the [BFG Repo-Cleaner][bfg]. +[`filter-branch` option](https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History#The-Nuclear-Option:-filter-branch), +or a tool like the [BFG Repo-Cleaner](https://rtyley.github.io/bfg-repo-cleaner/). Note that even with that method, until `git gc` runs on the GitLab side, the "removed" commits and blobs will still be around. You also need to be able to @@ -137,7 +138,3 @@ remove some of them, but it should not be depended on for security purposes! ``` Your repository should now be below the size limit. - -[admin-repo-size]: https://docs.gitlab.com/ee/user/admin_area/settings/account_and_limit_settings.html#repository-size-limit -[bfg]: https://rtyley.github.io/bfg-repo-cleaner/ -[gitscm]: https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History#The-Nuclear-Option:-filter-branch diff --git a/doc/user/project/security_dashboard.md b/doc/user/project/security_dashboard.md index 43e910b29fe..a3da1ec97d3 100644 --- a/doc/user/project/security_dashboard.md +++ b/doc/user/project/security_dashboard.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/application_security/security_dashboard/index.html' +redirect_to: '../application_security/security_dashboard/index.md' --- -This document was moved to [another location](https://docs.gitlab.com/ee/user/application_security/security_dashboard/index.html). +This document was moved to [another location](../application_security/security_dashboard/index.md). diff --git a/doc/user/project/settings/img/import_export_download_export.png b/doc/user/project/settings/img/import_export_download_export.png Binary files differindex 668254073e8..ab81c87bf5f 100644 --- a/doc/user/project/settings/img/import_export_download_export.png +++ b/doc/user/project/settings/img/import_export_download_export.png diff --git a/doc/user/project/settings/img/import_export_export_button.png b/doc/user/project/settings/img/import_export_export_button.png Binary files differindex 7f21bb2335b..9e368739695 100644 --- a/doc/user/project/settings/img/import_export_export_button.png +++ b/doc/user/project/settings/img/import_export_export_button.png diff --git a/doc/user/project/settings/img/import_export_mail_link.png b/doc/user/project/settings/img/import_export_mail_link.png Binary files differindex 48ef42855bc..985c37650d3 100644 --- a/doc/user/project/settings/img/import_export_mail_link.png +++ b/doc/user/project/settings/img/import_export_mail_link.png diff --git a/doc/user/project/settings/img/import_export_new_project.png b/doc/user/project/settings/img/import_export_new_project.png Binary files differindex b335700c5be..fc1f73c5d6e 100644 --- a/doc/user/project/settings/img/import_export_new_project.png +++ b/doc/user/project/settings/img/import_export_new_project.png diff --git a/doc/user/project/settings/img/import_export_select_file.png b/doc/user/project/settings/img/import_export_select_file.png Binary files differindex e1e5e031d81..e3e1a5ef980 100644 --- a/doc/user/project/settings/img/import_export_select_file.png +++ b/doc/user/project/settings/img/import_export_select_file.png diff --git a/doc/user/project/settings/img/settings_edit_button.png b/doc/user/project/settings/img/settings_edit_button.png Binary files differdeleted file mode 100644 index 32bcda03c7e..00000000000 --- a/doc/user/project/settings/img/settings_edit_button.png +++ /dev/null diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index 89008fd15b9..819515d7a4c 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -2,10 +2,11 @@ >**Notes:** > -> - [Introduced][ce-3050] in GitLab 8.9. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/3050) in GitLab 8.9. > - Importing will not be possible if the import instance version differs from > that of the exporter. -> - For GitLab admins, please read through [Project import/export administration](../../../administration/raketasks/project_import_export.md). +> - For GitLab admins, please read through +> [Project import/export administration](../../../administration/raketasks/project_import_export.md). > - For existing installations, the project import option has to be enabled in > application settings (`/admin/application_settings`) under 'Import sources'. > Ask your administrator if you don't see the **GitLab export** button when @@ -14,15 +15,15 @@ > on the GitLab instance in application settings (`/admin/application_settings`) > under 'Visibility and Access Controls'. > - You can find some useful raketasks if you are an administrator in the -> [import_export](../../../administration/raketasks/project_import_export.md) -> raketask. -> - The exports are stored in a temporary [shared directory][tmp] and are deleted -> every 24 hours by a specific worker. +> [import_export](../../../administration/raketasks/project_import_export.md) raketask. +> - The exports are stored in a temporary [shared directory](../../../development/shared_files.md) +> and are deleted every 24 hours by a specific worker. > - Group members will get exported as project members, as long as the user has > maintainer or admin access to the group where the exported project lives. An admin > in the import side is required to map the users, based on email or username. > Otherwise, a supplementary comment is left to mention the original author and > the MRs, notes or issues will be owned by the importer. +> - Project members with owner access will get imported as maintainers. > - Control project Import/Export with the [API](../../../api/project_import_export.md). > - If an imported project contains merge requests originated from forks, > then new branches associated with such merge requests will be created @@ -76,9 +77,9 @@ The following items will NOT be exported: ## Exporting a project and its data -1. Go to the project settings page by clicking on **Edit Project**: +1. Go to your project's homepage. -  +1. Click **Settings** in the sidebar. 1. Scroll down to find the **Export project** button: @@ -97,19 +98,14 @@ The following items will NOT be exported: ## Importing the project -1. The new GitLab project import feature is at the far right of the import - options when creating a New Project. Make sure you are in the right namespace - and you have entered a project name. Click on **GitLab export**: +1. The GitLab project import feature is the first import option when creating a + new project. Click on **GitLab export**:  -1. You can see where the project will be imported to. You can now select file - exported previously: +1. Enter your project name and URL. Then select the file you exported previously:  1. Click on **Import project** to begin importing. Your newly imported project page will appear soon. - -[ce-3050]: https://gitlab.com/gitlab-org/gitlab-ce/issues/3050 -[tmp]: ../../../development/shared_files.md diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md index 99dd018a3ba..e3502a632d9 100644 --- a/doc/user/project/settings/index.md +++ b/doc/user/project/settings/index.md @@ -36,7 +36,7 @@ Set up your project's merge request settings: - Set up the merge request method (merge commit, [fast-forward merge](../merge_requests/fast_forward_merge.html)). - Merge request [description templates](../description_templates.md#description-templates). -- Enable [merge request approvals](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html#merge-request-approvals). **[STARTER]** +- Enable [merge request approvals](../merge_requests/merge_request_approvals.md). **[STARTER]** - Enable [merge only of pipeline succeeds](../merge_requests/merge_when_pipeline_succeeds.md). - Enable [merge only when all discussions are resolved](../../discussions/index.md#only-allow-merge-requests-to-be-merged-if-all-discussions-are-resolved). @@ -44,7 +44,7 @@ Set up your project's merge request settings: ### Service Desk **[PREMIUM]** -Enable [Service Desk](https://docs.gitlab.com/ee/user/project/service_desk.html) for your project to offer customer support. +Enable [Service Desk](../service_desk.md) for your project to offer customer support. ### Export project @@ -100,9 +100,9 @@ Only project Owners and Admin users have the [permissions] to transfer a project You can transfer an existing project into a [group](../../group/index.md) if: -1. you have at least **Maintainer** [permissions] to that group -1. you are an **Owner** of the project. - +1. You have at least **Maintainer** [permissions] to that group. +1. The project is in a subgroup you own. +1. You are at least a **Maintainer** of the project under your personal namespace. Similarly, if you are an owner of a group, you can transfer any of its projects under your own user. diff --git a/doc/user/project/web_ide/index.md b/doc/user/project/web_ide/index.md index 2a2507d98a3..a634a8b2f54 100644 --- a/doc/user/project/web_ide/index.md +++ b/doc/user/project/web_ide/index.md @@ -8,7 +8,7 @@ projects by providing an advanced editor with commit staging. ## Open the Web IDE -The Web IDE can be opened when viewing a file, from the repository file list, +You can open the Web IDE when viewing a file, from the repository file list, and from merge requests.  @@ -45,7 +45,7 @@ Single file editing is based on the [Ace Editor](https://ace.c9.io). ## Stage and commit changes -After making your changes, click the Commit button in the bottom left to +After making your changes, click the **Commit** button in the bottom left to review the list of changed files. Click on each file to review the changes and click the tick icon to stage the file. @@ -67,10 +67,11 @@ shows you a preview of the merge request diff if you commit your changes. > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/19279) in [GitLab Core][ce] 11.0. -The Web IDE can be used to quickly fix failing tests by opening the branch or -merge request in the Web IDE and opening the logs of the failed job. The status -of all jobs for the most recent pipeline and job traces for the current commit -can be accessed by clicking the **Pipelines** button in the top right. +You can use the Web IDE to quickly fix failing tests by opening +the branch or merge request in the Web IDE and opening the logs of the failed +job. You can access the status of all jobs for the most recent pipeline and job +traces for the current commit by clicking the **Pipelines** button in the top +right. The pipeline status is also shown at all times in the status bar in the bottom left. @@ -79,31 +80,31 @@ left. > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/19318) in [GitLab Core][ce] 11.0. -Switching between your authored and assigned merge requests can be done without -leaving the Web IDE. Click the dropdown in the top of the sidebar to open a list -of merge requests. You will need to commit or discard all your changes before -switching to a different merge request. +To switch between your authored and assigned merge requests, click the +dropdown in the top of the sidebar to open a list of merge requests. You will +need to commit or discard all your changes before switching to a different merge +request. ## Switching branches > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/20850) in [GitLab Core][ce] 11.2. -Switching between branches of the current project repository can be done without -leaving the Web IDE. Click the dropdown in the top of the sidebar to open a list -of branches. You will need to commit or discard all your changes before -switching to a different branch. +To switch between branches of the current project repository, click the dropdown +in the top of the sidebar to open a list of branches. +You will need to commit or discard all your changes before switching to a +different branch. ## Client Side Evaluation > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/19764) in [GitLab Core][ce] 11.2. -The Web IDE can be used to preview JavaScript projects right in the browser. +You can use the Web IDE to preview JavaScript projects right in the browser. This feature uses CodeSandbox to compile and bundle the JavaScript used to preview the web application.  -Additionally, for public projects an `Open in CodeSandbox` button is available +Additionally, for public projects an **Open in CodeSandbox** button is available to transfer the contents of the project into a public CodeSandbox project to quickly share your project with others. @@ -115,9 +116,9 @@ GitLab.com  -Once it has been enabled in application settings, projects with a -`package.json` file and a `main` entry point can be previewed inside of the Web -IDE. An example `package.json` is below. +Once you have done that, you can preview projects with a `package.json` file and +a `main` entry point inside the Web IDE. An example `package.json` is shown +below. ```json { diff --git a/doc/user/search/advanced_global_search.md b/doc/user/search/advanced_global_search.md index 38b26f31417..f80f4183802 100644 --- a/doc/user/search/advanced_global_search.md +++ b/doc/user/search/advanced_global_search.md @@ -2,7 +2,7 @@ > - [Introduced][ee-109] in GitLab [Starter][ee] 8.4. > - This is the user documentation. To install and configure Elasticsearch, -> visit the [administrator documentation](https://docs.gitlab.com/ee/integration/elasticsearch.html). +> visit the [administrator documentation](../../integration/elasticsearch.md). NOTE: **Note** Advanced Global Search (powered by Elasticsearch) is not yet available on GitLab.com. We are working on adding it. [Follow this epic for the latest updates](https://gitlab.com/groups/gitlab-org/-/epics/153). diff --git a/doc/user/search/advanced_search_syntax.md b/doc/user/search/advanced_search_syntax.md index 8d4aac5f502..d302cb7a809 100644 --- a/doc/user/search/advanced_search_syntax.md +++ b/doc/user/search/advanced_search_syntax.md @@ -3,7 +3,7 @@ > **Notes:** > - Introduced in [GitLab Enterprise Starter][ee] 9.2 > - This is the user documentation. To install and configure Elasticsearch, -> visit the [administrator documentation](https://docs.gitlab.com/ee/integration/elasticsearch.html). +> visit the [administrator documentation](../../integration/elasticsearch.md). NOTE: **Note** Advanced Global Search (powered by Elasticsearch) is not yet available on GitLab.com. We are working on adding it. [Follow this epic for the latest updates](https://gitlab.com/groups/gitlab-org/-/epics/153). diff --git a/doc/workflow/README.md b/doc/workflow/README.md index 84ab7840140..40e2486ace5 100644 --- a/doc/workflow/README.md +++ b/doc/workflow/README.md @@ -13,15 +13,15 @@ comments: false - [Groups](../user/group/index.md) - Issues - The GitLab Issue Tracker is an advanced and complete tool for tracking the evolution of a new idea or the process of solving a problem. - - [Exporting Issues](https://docs.gitlab.com/ee/user/project/issues/csv_export.html) **[STARTER]** Export issues as a CSV, emailed as an attachment. + - [Exporting Issues](../user/project/issues/csv_export.md) **[STARTER]** Export issues as a CSV, emailed as an attachment. - [Confidential issues](../user/project/issues/confidential_issues.md) - [Due date for issues](../user/project/issues/due_dates.md) - [Issue Board](../user/project/issue_board.md) - [Keyboard shortcuts](shortcuts.md) - [File finder](file_finder.md) -- [File lock](https://docs.gitlab.com/ee/user/project/file_lock.html) **[PREMIUM]** +- [File lock](../user/project/file_lock.md) **[PREMIUM]** - [Labels](../user/project/labels.md) -- [Issue weight](https://docs.gitlab.com/ee/workflow/issue_weight.html) **[STARTER]** +- [Issue weight](issue_weight.md) **[STARTER]** - [Notification emails](notifications.md) - [Projects](../user/project/index.md) - [Project forking workflow](forking_workflow.md) @@ -44,9 +44,9 @@ comments: false - [Merge requests versions](../user/project/merge_requests/versions.md) - ["Work In Progress" merge requests](../user/project/merge_requests/work_in_progress_merge_requests.md) - [Fast-forward merge requests](../user/project/merge_requests/fast_forward_merge.md) - - [Merge request approvals](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html) **[STARTER]** + - [Merge request approvals](../user/project/merge_requests/merge_request_approvals.md) **[STARTER]** - [Repository mirroring](repository_mirroring.md) **[STARTER]** -- [Service Desk](https://docs.gitlab.com/ee/user/project/service_desk.html) **[PREMIUM]** +- [Service Desk](../user/project/service_desk.md) **[PREMIUM]** - [Manage large binaries with Git LFS](lfs/manage_large_binaries_with_git_lfs.md) - [Importing from SVN, GitHub, Bitbucket, etc](importing/README.md) - [Todos](todos.md) diff --git a/doc/workflow/gitlab_flow.md b/doc/workflow/gitlab_flow.md index 1b9fb504b15..7d0abb93262 100644 --- a/doc/workflow/gitlab_flow.md +++ b/doc/workflow/gitlab_flow.md @@ -166,7 +166,7 @@ This branch is the place for any work related to this change. NOTE: **Note:** The name of a branch might be dictated by organizational standards. -For example, in GitLab, any branches in GitLab EE that are equivalent to branches in GitLab CE [must end in `-ee`](https://docs.gitlab.com/ee/development/automatic_ce_ee_merge.html#cherry-picking-from-ce-to-ee). +For example, in GitLab, any branches in GitLab EE that are equivalent to branches in GitLab CE [must end in `-ee`](../development/automatic_ce_ee_merge.md#cherry-picking-from-ce-to-ee). When you are done or want to discuss the code, open a merge request. A merge request is an online place to discuss the change and review the code. diff --git a/doc/workflow/img/notification_global_settings.png b/doc/workflow/img/notification_global_settings.png Binary files differindex 8a5494d16a8..72f7418f1f8 100644 --- a/doc/workflow/img/notification_global_settings.png +++ b/doc/workflow/img/notification_global_settings.png diff --git a/doc/workflow/img/repository_mirroring_pull_settings_upper.png b/doc/workflow/img/repository_mirroring_pull_settings_upper.png Binary files differindex c60354fdca7..8e15b5a0784 100644 --- a/doc/workflow/img/repository_mirroring_pull_settings_upper.png +++ b/doc/workflow/img/repository_mirroring_pull_settings_upper.png diff --git a/doc/workflow/lfs/migrate_from_git_annex_to_git_lfs.md b/doc/workflow/lfs/migrate_from_git_annex_to_git_lfs.md index 1aeab5980a3..71c73e3dffe 100644 --- a/doc/workflow/lfs/migrate_from_git_annex_to_git_lfs.md +++ b/doc/workflow/lfs/migrate_from_git_annex_to_git_lfs.md @@ -42,7 +42,7 @@ Annex to Git LFS. ### TL; DR If you know what you are doing and want to skip the reading, this is what you -need to do (we assume you have [git-annex enabled][annex-gitlab-use] in your +need to do (we assume you have [git-annex enabled](../git_annex.md#using-gitlab-git-annex) in your repository and that you have made backups in case something goes wrong). Fire up a terminal, navigate to your Git repository and: @@ -82,7 +82,7 @@ Make sure the you read about the [`direct` mode][annex-direct] as it contains useful information that may fit in your use case. Note that `annex direct` is deprecated in Git Annex version 6, so you may need to upgrade your repository if the server also has Git Annex 6 installed. Read more in the -[Git Annex troubleshooting tips][annex-tips] section. +[Git Annex troubleshooting tips](../git_annex.md#troubleshooting-tips) section. 1. Backup your repository @@ -166,44 +166,45 @@ GitLab.com), therefore, you don't need to do anything server-side. 1. First, make sure you have `git-lfs` installed locally: - ```bash - git lfs help - ``` + ```bash + git lfs help + ``` - If the terminal doesn't prompt you with a full response on `git-lfs` commands, - [install the Git LFS client][install-lfs] first. + If the terminal doesn't prompt you with a full response on `git-lfs` commands, + [install the Git LFS client][install-lfs] first. 1. Inside the repo, run the following command to initiate LFS: - ```bash - git lfs install - ``` + ```bash + git lfs install + ``` 1. Enable `git-lfs` for the group of files you want to track. You can track specific files, all files containing the same extension, or an entire directory: - ```bash - git lfs track images/01.png # per file - git lfs track **/*.png # per extension - git lfs track images/ # per directory - ``` + ```bash + git lfs track images/01.png # per file + git lfs track **/*.png # per extension + git lfs track images/ # per directory + ``` - Once you do that, run `git status` and you'll see `.gitattributes` added - to your repo. It collects all file patterns that you chose to track via - `git-lfs`. + Once you do that, run `git status` and you'll see `.gitattributes` added + to your repo. It collects all file patterns that you chose to track via + `git-lfs`. 1. Add the files, commit and push them to GitLab: - ```bash - git add . - git commit -m "commit message" - git push - ``` + ```bash + git add . + git commit -m "commit message" + git push + ``` - If your remote is set up with HTTP, you will be asked to enter your login - credentials. If you have [2FA enabled][2fa], make sure to use a - [personal access token][token] instead of your password. + If your remote is set up with HTTP, you will be asked to enter your login + credentials. If you have [2FA enabled](../../user/profile/account/two_factor_authentication.md), make sure to use a + [personal access token](../../user/profile/account/two_factor_authentication.md#personal-access-tokens) + instead of your password. ## Removing the Git Annex branches @@ -238,18 +239,11 @@ git annex uninit - (Blog Post) [Getting Started with Git FLS][post-1] - (Blog Post) [Announcing LFS Support in GitLab][post-2] - (Blog Post) [GitLab Annex Solves the Problem of Versioning Large Binaries with Git][post-3] -- (GitLab Docs) [Git Annex][doc-1] -- (GitLab Docs) [Git LFS][doc-2] +- (GitLab Docs) [Git Annex](../git_annex.md) +- (GitLab Docs) [Git LFS](manage_large_binaries_with_git_lfs.md) -[2fa]: ../../user/profile/account/two_factor_authentication.md -[token]: ../../user/profile/account/two_factor_authentication.html#personal-access-tokens -[annex-tips]: ../git_annex.html#troubleshooting-tips [annex-direct]: https://git-annex.branchable.com/direct_mode/ -[annex-gitlab-use]: ../git_annex.md#using-gitlab-git-annex -[annex-ee]: https://docs.gitlab.com/ee/workflow/git_annex.html [bkp-ext-drive]: https://www.thomas-krenn.com/en/wiki/Git-annex_Repository_on_an_External_Hard_Drive -[doc-1]: https://docs.gitlab.com/ee/workflow/git_annex.html -[doc-2]: https://docs.gitlab.com/ee/workflow/lfs/manage_large_binaries_with_git_lfs.html [Git Annex]: http://git-annex.branchable.com/ [Git LFS]: https://git-lfs.github.com/ [install-lfs]: https://git-lfs.github.com/ diff --git a/doc/workflow/merge_request_approvals.md b/doc/workflow/merge_request_approvals.md index f8a99ec3d57..bfcd8faf236 100644 --- a/doc/workflow/merge_request_approvals.md +++ b/doc/workflow/merge_request_approvals.md @@ -1,5 +1,5 @@ --- -redirect_to: 'https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html' +redirect_to: '../user/project/merge_requests/merge_request_approvals.md' --- -This document was moved to [user/project/merge_requests/merge_request_approvals](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html). +This document was moved to [another location](../user/project/merge_requests/merge_request_approvals.md). diff --git a/doc/workflow/notifications.md b/doc/workflow/notifications.md index 0eb4f85e0ec..5d560f2e000 100644 --- a/doc/workflow/notifications.md +++ b/doc/workflow/notifications.md @@ -16,12 +16,16 @@ Notification settings are divided into three groups: Each of these settings have levels of notification: +- Global: For groups and projects, notifications as per global settings. - Watch: Receive notifications for any activity. -- On Mention: Receive notifications when `@mentioned` in comments. - Participate: Receive notifications for threads you have participated in. +- On Mention: Receive notifications when `@mentioned` in comments. - Disabled: Turns off notifications. - Custom: Receive notifications for custom selected events. -- Global: For groups and projects, notifications as per global settings. + +> Introduced in GitLab 12.0 + +You can also select an email address to receive notifications for each group you belong to. ### Global Settings diff --git a/doc/workflow/repository_mirroring.md b/doc/workflow/repository_mirroring.md index 2f8f1545b84..9772bd385ba 100644 --- a/doc/workflow/repository_mirroring.md +++ b/doc/workflow/repository_mirroring.md @@ -99,6 +99,7 @@ The repository will push soon. To force a push, click the appropriate button. ## Pulling from a remote repository **[STARTER]** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/51) in GitLab Enterprise Edition 8.2. +> [Added Git LFS support](https://gitlab.com/gitlab-org/gitlab-ee/issues/10871) in [GitLab Starter](https://about.gitlab.com/pricing/) 11.11. You can set up a repository to automatically have its branches, tags, and commits updated from an upstream repository. @@ -282,10 +283,10 @@ project mirroring again by [Forcing an update](#forcing-an-update-core). [GitLab Starter](https://about.gitlab.com/pricing/) 10.3. Pull mirroring uses polling to detect new branches and commits added upstream, often minutes -afterwards. If you notify GitLab by [API](https://docs.gitlab.com/ee/api/projects.html#start-the-pull-mirroring-process-for-a-project), +afterwards. If you notify GitLab by [API](https://docs.gitlab.com/ee/api/projects.html#start-the-pull-mirroring-process-for-a-project-starter), updates will be pulled immediately. -For more information, see [Start the pull mirroring process for a Project](https://docs.gitlab.com/ee/api/projects.html#start-the-pull-mirroring-process-for-a-project). +For more information, see [Start the pull mirroring process for a Project](https://docs.gitlab.com/ee/api/projects.html#start-the-pull-mirroring-process-for-a-project-starter). ## Forcing an update **[CORE]** diff --git a/doc/workflow/shortcuts.md b/doc/workflow/shortcuts.md index 6714d08f2f7..5068b5d4d20 100644 --- a/doc/workflow/shortcuts.md +++ b/doc/workflow/shortcuts.md @@ -1,6 +1,6 @@ # GitLab keyboard shortcuts -You can see GitLab's keyboard shortcuts by using 'shift + ?' +You can see GitLab's keyboard shortcuts by using <kbd>shift</kbd> + <kbd>?</kbd> ## Global Shortcuts |
