diff options
| author | Ryan Cobb <rcobb@gitlab.com> | 2019-05-29 10:56:38 -0600 |
|---|---|---|
| committer | Ryan Cobb <rcobb@gitlab.com> | 2019-05-29 10:56:38 -0600 |
| commit | 0393c5059de2ce03f706fbf28056e1d80304b73a (patch) | |
| tree | f47c4daaf9315c89cc6b039988d57e1fbd14c322 /doc | |
| parent | 4fae62b9efa985e4cf6a09cb6bd4b9cf665e6b32 (diff) | |
| parent | 70d1537dda66da8b319ceefde36195047b26a8fd (diff) | |
| download | gitlab-ce-0393c5059de2ce03f706fbf28056e1d80304b73a.tar.gz | |
Merge branch 'master' into 61964-unicorn-instrumentation
Diffstat (limited to 'doc')
130 files changed, 1668 insertions, 1472 deletions
diff --git a/doc/administration/geo/disaster_recovery/background_verification.md b/doc/administration/geo/disaster_recovery/background_verification.md index c7299b6e196..d4c8c2d3624 100644 --- a/doc/administration/geo/disaster_recovery/background_verification.md +++ b/doc/administration/geo/disaster_recovery/background_verification.md @@ -29,12 +29,7 @@ the node more time before scheduling a planned failover. Run the following commands in a Rails console on the **primary** node: ```sh -# Omnibus GitLab gitlab-rails console - -# Installation from source -cd /home/git/gitlab -sudo -u git -H bin/rails console RAILS_ENV=production ``` To check if automatic background verification is enabled: @@ -102,12 +97,7 @@ disable if you need. Run the following commands in a Rails console on the **primary** node: ```sh -# Omnibus GitLab gitlab-rails console - -# Installation from source -cd /home/git/gitlab -sudo -u git -H bin/rails console RAILS_ENV=production ``` To disable automatic background re-verification: @@ -131,31 +121,15 @@ to be resynced without the backoff period: For repositories: -- Omnibus Installation - - ```sh - sudo gitlab-rake geo:verification:repository:reset - ``` - -- Source Installation - - ```sh - sudo -u git -H bundle exec rake geo:verification:repository:reset RAILS_ENV=production - ``` +```sh +sudo gitlab-rake geo:verification:repository:reset +``` For wikis: -- Omnibus Installation - - ```sh - sudo gitlab-rake geo:verification:wiki:reset - ``` - -- Source Installation - - ```sh - sudo -u git -H bundle exec rake geo:verification:wiki:reset RAILS_ENV=production - ``` +```sh +sudo gitlab-rake geo:verification:wiki:reset +``` ## Reconcile differences with checksum mismatches @@ -169,7 +143,7 @@ If the **primary** and **secondary** nodes have a checksum verification mismatch 1. On the project admin page get the **Gitaly storage name**, and **Gitaly relative path**:  -1. Navigate to the project's repository directory on both **primary** and **secondary** nodes. For an installation from source, the path is usually `/home/git/repositories`. For Omnibus installs, the path is usually `/var/opt/gitlab/git-data/repositories`. Note that if `git_data_dirs` is customized, check the directory layout on your server to be sure. +1. Navigate to the project's repository directory on both **primary** and **secondary** nodes (the path is usually `/var/opt/gitlab/git-data/repositories`). Note that if `git_data_dirs` is customized, check the directory layout on your server to be sure. ```sh cd /var/opt/gitlab/git-data/repositories diff --git a/doc/administration/geo/replication/configuration.md b/doc/administration/geo/replication/configuration.md index 57735b21cda..3d4f69d3abe 100644 --- a/doc/administration/geo/replication/configuration.md +++ b/doc/administration/geo/replication/configuration.md @@ -1,9 +1,4 @@ -# Geo configuration (GitLab Omnibus) **[PREMIUM ONLY]** - -NOTE: **Note:** -This is the documentation for the Omnibus GitLab packages. For installations -from source, follow the [**Geo nodes configuration for installations -from source**][configuration-source] guide. +# Geo configuration **[PREMIUM ONLY]** ## Configuring a new **secondary** node @@ -303,7 +298,6 @@ See the [updating the Geo nodes document](updating_the_geo_nodes.md). See the [troubleshooting document](troubleshooting.md). -[configuration-source]: configuration_source.md [setup-geo-omnibus]: index.md#using-omnibus-gitlab [Hashed Storage]: ../../repository_storage_types.md [Disaster Recovery]: ../disaster_recovery/index.md diff --git a/doc/administration/geo/replication/configuration_source.md b/doc/administration/geo/replication/configuration_source.md deleted file mode 100644 index 10dd9405b4b..00000000000 --- a/doc/administration/geo/replication/configuration_source.md +++ /dev/null @@ -1,172 +0,0 @@ -# Geo configuration (source) **[PREMIUM ONLY]** - -NOTE: **Note:** -This documentation applies to GitLab source installations. In GitLab 11.5, this documentation was deprecated and will be removed in a future release. -Please consider [migrating to GitLab Omnibus install](https://docs.gitlab.com/omnibus/update/convert_to_omnibus.html). For installations -using the Omnibus GitLab packages, follow the -[**Omnibus Geo nodes configuration**][configuration] guide. - -## Configuring a new **secondary** node - -NOTE: **Note:** -This is the final step in setting up a **secondary** node. Stages of the setup -process must be completed in the documented order. Before attempting the steps -in this stage, [complete all prior stages](index.md#using-gitlab-installed-from-source-deprecated). - -The basic steps of configuring a **secondary** node are to: - -- Replicate required configurations between the **primary** and **secondary** nodes. -- Configure a tracking database on each **secondary** node. -- Start GitLab on the **secondary** node. - -You are encouraged to first read through all the steps before executing them -in your testing/production environment. - -NOTE: **Note:** -**Do not** set up any custom authentication on **secondary** nodes, this will be handled by the **primary** node. - -NOTE: **Note:** -**Do not** add anything in the **secondary** node's admin area (**Admin Area > Geo**). This is handled solely by the **primary** node. - -### Step 1. Manually replicate secret GitLab values - -GitLab stores a number of secret values in the `/home/git/gitlab/config/secrets.yml` -file which *must* match between the **primary** and **secondary** nodes. Until there is -a means of automatically replicating these between nodes (see [gitlab-org/gitlab-ee#3789]), they must -be manually replicated to **secondary** nodes. - -1. SSH into the **primary** node, and execute the command below: - - ```sh - sudo cat /home/git/gitlab/config/secrets.yml - ``` - - This will display the secrets that need to be replicated, in YAML format. - -1. SSH into the **secondary** node and login as the `git` user: - - ```sh - sudo -i -u git - ``` - -1. Make a backup of any existing secrets: - - ```sh - mv /home/git/gitlab/config/secrets.yml /home/git/gitlab/config/secrets.yml.`date +%F` - ``` - -1. Copy `/home/git/gitlab/config/secrets.yml` from the **primary** node to the **secondary** node, or - copy-and-paste the file contents between nodes: - - ```sh - sudo editor /home/git/gitlab/config/secrets.yml - - # paste the output of the `cat` command you ran on the primary - # save and exit - ``` - -1. Ensure the file permissions are correct: - - ```sh - chown git:git /home/git/gitlab/config/secrets.yml - chmod 0600 /home/git/gitlab/config/secrets.yml - ``` - -1. Restart GitLab - - ```sh - service gitlab restart - ``` - -Once restarted, the **secondary** node will automatically start replicating missing data -from the **primary** node in a process known as backfill. Meanwhile, the **primary** node -will start to notify the **secondary** node of any changes, so that the **secondary** node can -act on those notifications immediately. - -Make sure the **secondary** node is running and accessible. You can login to -the **secondary** node with the same credentials as used for the **primary** node. - -### Step 2. Manually replicate the **primary** node's SSH host keys - -Read [Manually replicate the **primary** node's SSH host keys](configuration.md#step-2-manually-replicate-the-primary-nodes-ssh-host-keys) - -### Step 3. Add the **secondary** GitLab node - -1. Navigate to the **primary** node's **Admin Area > Geo** - (`/admin/geo/nodes`) in your browser. -1. Add the **secondary** node by providing its full URL. **Do NOT** check the - **This is a primary node** checkbox. -1. Optionally, choose which namespaces should be replicated by the - **secondary** node. Leave blank to replicate all. Read more in - [selective synchronization](#selective-synchronization). -1. Click the **Add node** button. -1. SSH into your GitLab **secondary** server and restart the services: - - ```sh - service gitlab restart - ``` - - Check if there are any common issue with your Geo setup by running: - - ```sh - bundle exec rake gitlab:geo:check - ``` - -1. SSH into your GitLab **primary** server and login as root to verify the - **secondary** node is reachable or there are any common issue with your Geo setup: - - ```sh - bundle exec rake gitlab:geo:check - ``` - -Once reconfigured, the **secondary** node will automatically start -replicating missing data from the **primary** node in a process known as backfill. -Meanwhile, the **primary** node will start to notify the **secondary** node of any changes, so -that the **secondary** node can act on those notifications immediately. - -Make sure the **secondary** node is running and accessible. -You can log in to the **secondary** node with the same credentials as used for the **primary** node. - -### Step 4. Enabling Hashed Storage - -Read [Enabling Hashed Storage](configuration.md#step-4-enabling-hashed-storage). - -### Step 5. (Optional) Configuring the secondary to trust the primary - -You can safely skip this step if your **primary** node uses a CA-issued HTTPS certificate. - -If your **primary** node is using a self-signed certificate for *HTTPS* support, you will -need to add that certificate to the **secondary** node's trust store. Retrieve the -certificate from the **primary** node and follow your distribution's instructions for -adding it to the **secondary** node's trust store. In Debian/Ubuntu, you would follow these steps: - -```sh -sudo -i -cp <primary_node_certification_file> /usr/local/share/ca-certificates -update-ca-certificates -``` - -### Step 6. Enable Git access over HTTP/HTTPS - -Geo synchronizes repositories over HTTP/HTTPS, and therefore requires this clone -method to be enabled. Navigate to **Admin Area > Settings** -(`/admin/application_settings`) on the **primary** node, and set -`Enabled Git access protocols` to `Both SSH and HTTP(S)` or `Only HTTP(S)`. - -### Step 7. Verify proper functioning of the secondary node - -Read [Verify proper functioning of the secondary node][configuration-verify-node]. - -## Selective synchronization - -Read [Selective synchronization][configuration-selective-replication]. - -## Troubleshooting - -Read the [troubleshooting document][troubleshooting]. - -[gitlab-org/gitlab-ee#3789]: https://gitlab.com/gitlab-org/gitlab-ee/issues/3789 -[configuration]: configuration.md -[configuration-selective-replication]: configuration.md#selective-synchronization -[configuration-verify-node]: configuration.md#step-7-verify-proper-functioning-of-the-secondary-node -[troubleshooting]: troubleshooting.md diff --git a/doc/administration/geo/replication/database.md b/doc/administration/geo/replication/database.md index e57583a3bf9..a0c2cf0eced 100644 --- a/doc/administration/geo/replication/database.md +++ b/doc/administration/geo/replication/database.md @@ -1,9 +1,7 @@ -# Geo database replication (GitLab Omnibus) **[PREMIUM ONLY]** +# Geo database replication **[PREMIUM ONLY]** NOTE: **Note:** -This is the documentation for the Omnibus GitLab packages. For installations -from source, follow the -[Geo database replication (source)](database_source.md) guide. +The following steps are for Omnibus installs only. Using Geo with source-based installs was **deprecated** in GitLab 11.5. NOTE: **Note:** If your GitLab installation uses external (not managed by Omnibus) PostgreSQL @@ -102,10 +100,15 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o else. If you are using an external database not managed by Omnibus GitLab, you need - to create the replicator user and define a password to it manually. - For information on how to create a replication user, refer to the - [appropriate step](database_source.md#step-1-configure-the-primary-server) - in [Geo database replication (source)](database_source.md). + to create the replicator user and define a password to it manually: + + ```sql + --- Create a new user 'replicator' + CREATE USER gitlab_replicator; + + --- Set/change a password and grants replication privilege + ALTER USER gitlab_replicator WITH REPLICATION ENCRYPTED PASSWORD '<replication_password>'; + ``` 1. Configure PostgreSQL to listen on network interfaces: @@ -340,7 +343,7 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o ## ## Secondary address - ## - replace '<secondary_node_ip>' with the public or VPC address of your Geo secondary node + ## - replace '<secondary_node_ip>' with the public or VPC address of your Geo secondary node ## postgresql['listen_address'] = '<secondary_node_ip>' postgresql['md5_auth_cidr_addresses'] = ['<secondary_node_ip>/32'] @@ -383,8 +386,7 @@ the database on the **primary** node, replicates the database, and creates the needed files for streaming replication. The directories used are the defaults that are set up in Omnibus. If you have -changed any defaults or are using a source installation, configure it as you -see fit replacing the directories and paths. +changed any defaults, configure it as you see fit replacing the directories and paths. CAUTION: **Warning:** Make sure to run this on the **secondary** server as it removes all PostgreSQL's diff --git a/doc/administration/geo/replication/database_source.md b/doc/administration/geo/replication/database_source.md deleted file mode 100644 index 67cf8b6535f..00000000000 --- a/doc/administration/geo/replication/database_source.md +++ /dev/null @@ -1,439 +0,0 @@ -# Geo database replication (source) **[PREMIUM ONLY]** - -NOTE: **Note:** -This documentation applies to GitLab source installations. In GitLab 11.5, this documentation was deprecated and will be removed in a future release. -Please consider [migrating to GitLab Omnibus install](https://docs.gitlab.com/omnibus/update/convert_to_omnibus.html). For installations -using the Omnibus GitLab packages, follow the -[**database replication for Omnibus GitLab**][database] guide. - -NOTE: **Note:** -The stages of the setup process must be completed in the documented order. -Before attempting the steps in this stage, [complete all prior stages](index.md#using-gitlab-installed-from-source-deprecated). - -This document describes the minimal steps you have to take in order to -replicate your **primary** GitLab database to a **secondary** node's database. You may -have to change some values according to your database setup, how big it is, etc. - -You are encouraged to first read through all the steps before executing them -in your testing/production environment. - -## PostgreSQL replication - -The GitLab **primary** node where the write operations happen will connect to -**primary** database server, and the **secondary** ones which are read-only will -connect to **secondary** database servers (which are read-only too). - -NOTE: **Note:** -In many databases' documentation, you will see "**primary**" being referenced as "master" -and "**secondary**" as either "slave" or "standby" server (read-only). - -We recommend using [PostgreSQL replication slots][replication-slots-article] -to ensure the **primary** node retains all the data necessary for the secondaries to -recover. See below for more details. - -The following guide assumes that: - -- You are using PostgreSQL 9.6 or later which includes the - [`pg_basebackup` tool][pgback] and improved [Foreign Data Wrapper][FDW] support. -- You have a **primary** node already set up (the GitLab server you are - replicating from), running PostgreSQL 9.6 or later, and - you have a new **secondary** server set up with the same versions of the OS, - PostgreSQL, and GitLab on all nodes. -- The IP of the **primary** server for our examples is `198.51.100.1`, whereas the - **secondary** node's IP is `198.51.100.2`. Note that the **primary** and **secondary** servers - **must** be able to communicate over these addresses. These IP addresses can either - be public or private. - -CAUTION: **Warning:** -Geo works with streaming replication. Logical replication is not supported at this time. -There is an [issue where support is being discussed](https://gitlab.com/gitlab-org/gitlab-ee/issues/7420). - -### Step 1. Configure the **primary** server - -1. SSH into your GitLab **primary** server and login as root: - - ```sh - sudo -i - ``` - -1. Add this node as the Geo **primary** by running: - - ```sh - bundle exec rake geo:set_primary_node - ``` - -1. Create a [replication user] named `gitlab_replicator`: - - ```sql - --- Create a new user 'replicator' - CREATE USER gitlab_replicator; - - --- Set/change a password and grants replication privilege - ALTER USER gitlab_replicator WITH REPLICATION ENCRYPTED PASSWORD '<replication_password>'; - ``` - -1. Make sure your the `gitlab` database user has a password defined: - - ```sh - sudo \ - -u postgres psql \ - -d template1 \ - -c "ALTER USER gitlab WITH ENCRYPTED PASSWORD '<database_password>';" - ``` - -1. Edit the content of `database.yml` in `production:` and add the password like the example below: - - ```yaml - # - # PRODUCTION - # - production: - adapter: postgresql - encoding: unicode - database: gitlabhq_production - pool: 10 - username: gitlab - password: <database_password> - host: /var/opt/gitlab/geo-postgresql - ``` - -1. Set up TLS support for the PostgreSQL **primary** server: - - CAUTION: **Warning**: - Only skip this step if you **know** that PostgreSQL traffic - between the **primary** and **secondary** nodes will be secured through some other - means, e.g., a known-safe physical network path or a site-to-site VPN that - you have configured. - - If you are replicating your database across the open Internet, it is - **essential** that the connection is TLS-secured. Correctly configured, this - provides protection against both passive eavesdroppers and active - "man-in-the-middle" attackers. - - To generate a self-signed certificate and key, run this command: - - ```sh - openssl req \ - -nodes \ - -batch \ - -x509 \ - -newkey rsa:4096 \ - -keyout server.key \ - -out server.crt \ - -days 3650 - ``` - - This will create two files - `server.key` and `server.crt` - that you can - use for authentication. - - Copy them to the correct location for your PostgreSQL installation: - - ```sh - # Copying a self-signed certificate and key - install -o postgres -g postgres -m 0400 -T server.crt ~postgres/9.x/main/data/server.crt - install -o postgres -g postgres -m 0400 -T server.key ~postgres/9.x/main/data/server.key - ``` - - Add this configuration to `postgresql.conf`, removing any existing - configuration for `ssl_cert_file` or `ssl_key_file`: - - ``` - ssl = on - ssl_cert_file='server.crt' - ssl_key_file='server.key' - ``` - -1. Edit `postgresql.conf` to configure the **primary** server for streaming replication - (for Debian/Ubuntu that would be `/etc/postgresql/9.x/main/postgresql.conf`): - - ``` - listen_address = '<primary_node_ip>' - wal_level = hot_standby - max_wal_senders = 5 - min_wal_size = 80MB - max_wal_size = 1GB - max_replicaton_slots = 1 # Number of Geo secondary nodes - wal_keep_segments = 10 - hot_standby = on - ``` - - NOTE: **Note**: - Be sure to set `max_replication_slots` to the number of Geo **secondary** - nodes that you may potentially have (at least 1). - - For security reasons, PostgreSQL by default only listens on the local - interface (e.g. 127.0.0.1). However, Geo needs to communicate - between the **primary** and **secondary** nodes over a common network, such as a - corporate LAN or the public Internet. For this reason, we need to - configure PostgreSQL to listen on more interfaces. - - The `listen_address` option opens PostgreSQL up to external connections - with the interface corresponding to the given IP. See [the PostgreSQL - documentation][pg-docs-runtime-conn] for more details. - - You may also want to edit the `wal_keep_segments` and `max_wal_senders` to - match your database replication requirements. Consult the - [PostgreSQL - Replication documentation][pg-docs-runtime-replication] for more information. - -1. Set the access control on the **primary** node to allow TCP connections using the - server's public IP and set the connection from the **secondary** node to require a - password. Edit `pg_hba.conf` (for Debian/Ubuntu that would be - `/etc/postgresql/9.x/main/pg_hba.conf`): - - ```sh - host all all <primary_node_ip>/32 md5 - host replication gitlab_replicator <secondary_node_ip>/32 md5 - ``` - - If you want to add another secondary, add one more row like the replication - one and change the IP address: - - ```sh - host all all <primary_node_ip>/32 md5 - host replication gitlab_replicator <secondary_node_ip>/32 md5 - host replication gitlab_replicator <another_secondary_node_ip>/32 md5 - ``` - -1. Restart PostgreSQL for the changes to take effect. - -1. Choose a database-friendly name to use for your secondary to use as the - replication slot name. For example, if your domain is - `secondary.geo.example.com`, you may use `secondary_example` as the slot - name. - -1. Create the replication slot on the **primary** node: - - ```sh - $ sudo -u postgres psql -c "SELECT * FROM pg_create_physical_replication_slot('secondary_example');" - slot_name | xlog_position - ------------------+--------------- - secondary_example | - (1 row) - ``` - -1. Now that the PostgreSQL server is set up to accept remote connections, run - `netstat -plnt` to make sure that PostgreSQL is listening to the server's - public IP. - -### Step 2. Configure the secondary server - -Follow the first steps in ["configure the secondary server"][database-replication] and note that since you are installing from source, the username and -group listed as `gitlab-psql` in those steps should be replaced by `postgres` -instead. After completing the "Test that the `gitlab-psql` user can connect to -the **primary** node's database" step, continue here: - -1. Edit `postgresql.conf` to configure the secondary for streaming replication - (for Debian/Ubuntu that would be `/etc/postgresql/9.*/main/postgresql.conf`): - - ```sh - wal_level = hot_standby - max_wal_senders = 5 - checkpoint_segments = 10 - wal_keep_segments = 10 - hot_standby = on - ``` - -1. Restart PostgreSQL for the changes to take effect. - -#### Enable tracking database on the secondary server - -Geo secondary nodes use a tracking database to keep track of replication status -and recover automatically from some replication issues. Follow the steps below to create -the tracking database. - -1. On the secondary node, run the following command to create `database_geo.yml` with the - information of your secondary PostgreSQL instance: - - ```sh - sudo cp /home/git/gitlab/config/database_geo.yml.postgresql /home/git/gitlab/config/database_geo.yml - ``` - -1. Edit the content of `database_geo.yml` in `production:` as in the example below: - - ```yaml - # - # PRODUCTION - # - production: - adapter: postgresql - encoding: unicode - database: gitlabhq_geo_production - pool: 10 - username: gitlab_geo - # password: - host: /var/opt/gitlab/geo-postgresql - ``` - -1. Create the database `gitlabhq_geo_production` on the PostgreSQL instance of the **secondary** node. - -1. Set up the Geo tracking database: - - ```sh - bundle exec rake geo:db:migrate - ``` - -1. Configure the [PostgreSQL FDW][FDW] connection and credentials: - - Save the script below in a file, ex. `/tmp/geo_fdw.sh` and modify the connection - params to match your environment. Execute it to set up the FDW connection. - - ```sh - #!/bin/bash - - # Secondary Database connection params: - DB_HOST="/var/opt/gitlab/postgresql" # change to the public IP or VPC private IP if its an external server - DB_NAME="gitlabhq_production" - DB_USER="gitlab" - DB_PORT="5432" - - # Tracking Database connection params: - GEO_DB_HOST="/var/opt/gitlab/geo-postgresql" # change to the public IP or VPC private IP if its an external server - GEO_DB_NAME="gitlabhq_geo_production" - GEO_DB_USER="gitlab_geo" - GEO_DB_PORT="5432" - - query_exec () { - gitlab-psql -h $GEO_DB_HOST -d $GEO_DB_NAME -p $GEO_DB_PORT -c "${1}" - } - - query_exec "CREATE EXTENSION postgres_fdw;" - query_exec "CREATE SERVER gitlab_secondary FOREIGN DATA WRAPPER postgres_fdw OPTIONS (host '${DB_HOST}', dbname '${DB_NAME}', port '${DB_PORT}');" - query_exec "CREATE USER MAPPING FOR ${GEO_DB_USER} SERVER gitlab_secondary OPTIONS (user '${DB_USER}');" - query_exec "CREATE SCHEMA gitlab_secondary;" - query_exec "GRANT USAGE ON FOREIGN SERVER gitlab_secondary TO ${GEO_DB_USER};" - ``` - - And edit the content of `database_geo.yml` and to add `fdw: true` to - the `production:` block. - -### Step 3. Initiate the replication process - -Below we provide a script that connects the database on the **secondary** node to -the database on the **primary** node, replicates the database, and creates the -needed files for streaming replication. - -The directories used are the defaults for Debian/Ubuntu. If you have changed -any defaults, configure it as you see fit replacing the directories and paths. - -CAUTION: **Warning:** -Make sure to run this on the **secondary** server as it removes all PostgreSQL's -data before running `pg_basebackup`. - -1. SSH into your GitLab **secondary** server and login as root: - - ```sh - sudo -i - ``` - -1. Save the snippet below in a file, let's say `/tmp/replica.sh`. Modify the - embedded paths if necessary: - - ``` - #!/bin/bash - - PORT="5432" - USER="gitlab_replicator" - echo --------------------------------------------------------------- - echo WARNING: Make sure this script is run from the secondary server - echo --------------------------------------------------------------- - echo - echo Enter the IP or FQDN of the primary PostgreSQL server - read HOST - echo Enter the password for $USER@$HOST - read -s PASSWORD - echo Enter the required sslmode - read SSLMODE - - echo Stopping PostgreSQL and all GitLab services - sudo service gitlab stop - sudo service postgresql stop - - echo Backing up postgresql.conf - sudo -u postgres mv /var/opt/gitlab/postgresql/data/postgresql.conf /var/opt/gitlab/postgresql/ - - echo Cleaning up old cluster directory - sudo -u postgres rm -rf /var/opt/gitlab/postgresql/data - - echo Starting base backup as the replicator user - echo Enter the password for $USER@$HOST - sudo -u postgres /opt/gitlab/embedded/bin/pg_basebackup -h $HOST -D /var/opt/gitlab/postgresql/data -U gitlab_replicator -v -x -P - - echo Writing recovery.conf file - sudo -u postgres bash -c "cat > /var/opt/gitlab/postgresql/data/recovery.conf <<- _EOF1_ - standby_mode = 'on' - primary_conninfo = 'host=$HOST port=$PORT user=$USER password=$PASSWORD sslmode=$SSLMODE' - _EOF1_ - " - - echo Restoring postgresql.conf - sudo -u postgres mv /var/opt/gitlab/postgresql/postgresql.conf /var/opt/gitlab/postgresql/data/ - - echo Starting PostgreSQL - sudo service postgresql start - ``` - -1. Run it with: - - ```sh - bash /tmp/replica.sh - ``` - - When prompted, enter the IP/FQDN of the **primary** node, and the password you set up - for the `gitlab_replicator` user in the first step. - - You should use `verify-ca` for the `sslmode`. You can use `disable` if you - are happy to skip PostgreSQL TLS authentication altogether (e.g., you know - the network path is secure, or you are using a site-to-site VPN). This is - **not** safe over the public Internet! - - You can read more details about each `sslmode` in the - [PostgreSQL documentation][pg-docs-ssl]; - the instructions above are carefully written to ensure protection against - both passive eavesdroppers and active "man-in-the-middle" attackers. - -The replication process is now over. - -## PGBouncer support (optional) - -1. First, enter the PostgreSQL console as an admin user. - -1. Then create the read-only user: - - ```sql - -- NOTE: Use the password defined earlier - CREATE USER gitlab_geo_fdw WITH password '<your_password_here>'; - GRANT CONNECT ON DATABASE gitlabhq_production to gitlab_geo_fdw; - GRANT USAGE ON SCHEMA public TO gitlab_geo_fdw; - GRANT SELECT ON ALL TABLES IN SCHEMA public TO gitlab_geo_fdw; - GRANT SELECT ON ALL SEQUENCES IN SCHEMA public TO gitlab_geo_fdw; - - -- Tables created by "gitlab" should be made read-only for "gitlab_geo_fdw" - -- automatically. - ALTER DEFAULT PRIVILEGES FOR USER gitlab IN SCHEMA public GRANT SELECT ON TABLES TO gitlab_geo_fdw; - ALTER DEFAULT PRIVILEGES FOR USER gitlab IN SCHEMA public GRANT SELECT ON SEQUENCES TO gitlab_geo_fdw; - ``` - -1. Enter the PostgreSQL console on the **secondary** tracking database and change the user mapping to this new user: - - ``` - ALTER USER MAPPING FOR gitlab_geo SERVER gitlab_secondary OPTIONS (SET user 'gitlab_geo_fdw') - ``` - -## MySQL replication - -MySQL replication is not supported for Geo. - -## Troubleshooting - -Read the [troubleshooting document](troubleshooting.md). - -[replication-slots-article]: https://medium.com/@tk512/replication-slots-in-postgresql-b4b03d277c75 -[pgback]: http://www.postgresql.org/docs/9.6/static/app-pgbasebackup.html -[replication user]:https://wiki.postgresql.org/wiki/Streaming_Replication -[FDW]: https://www.postgresql.org/docs/9.6/static/postgres-fdw.html -[database]: database.md -[add-geo-node]: configuration.md#step-3-add-the-secondary-gitlab-node -[database-replication]: database.md#step-2-configure-the-secondary-server -[pg-docs-ssl]: https://www.postgresql.org/docs/9.6/static/libpq-ssl.html#LIBPQ-SSL-PROTECTION -[pg-docs-runtime-conn]: https://www.postgresql.org/docs/9.6/static/runtime-config-connection.html -[pg-docs-runtime-replication]: https://www.postgresql.org/docs/9.6/static/runtime-config-replication.html diff --git a/doc/administration/geo/replication/index.md b/doc/administration/geo/replication/index.md index 6abea2cf271..b2f71d82cfc 100644 --- a/doc/administration/geo/replication/index.md +++ b/doc/administration/geo/replication/index.md @@ -186,30 +186,13 @@ If you installed GitLab using the Omnibus packages (highly recommended): 1. Optional: [Configure a secondary LDAP server](../../auth/ldap.md) for the **secondary** node. See [notes on LDAP](#ldap). 1. [Follow the "Using a Geo Server" guide](using_a_geo_server.md). -### Using GitLab installed from source (Deprecated) - -NOTE: **Note:** -In GitLab 11.5, support for using Geo in GitLab source installations was deprecated and will be removed in a future release. Please consider [migrating to GitLab Omnibus install](https://docs.gitlab.com/omnibus/update/convert_to_omnibus.html). - -If you installed GitLab from source: - -1. [Install GitLab Enterprise Edition](../../../install/installation.md) on the server that will serve as the **secondary** node. Do not create an account or log in to the new **secondary** node. -1. [Upload the GitLab License](https://docs.gitlab.com/ee/user/admin_area/license.html) on the **primary** node to unlock Geo. The license must be for [GitLab Premium](https://about.gitlab.com/pricing/) or higher. -1. [Set up the database replication](database_source.md) (`primary (read-write) <-> secondary (read-only)` topology). -1. [Configure fast lookup of authorized SSH keys in the database](../../operations/fast_ssh_key_lookup.md). Do this step for **both** **primary** and **secondary** nodes. -1. [Configure GitLab](configuration_source.md) to set the **primary** and **secondary** nodes. -1. [Follow the "Using a Geo Server" guide](using_a_geo_server.md). - ## Post-installation documentation After installing GitLab on the **secondary** nodes and performing the initial configuration, see the following documentation for post-installation information. ### Configuring Geo -For information on configuring Geo, see: - -- [Geo configuration (GitLab Omnibus)](configuration.md). -- [Geo configuration (source)](configuration_source.md). Configuring Geo in GitLab source installations was **deprecated** in GitLab 11.5. +For information on configuring Geo, see [Geo configuration](configuration.md). ### Updating Geo diff --git a/doc/administration/geo/replication/security_review.md b/doc/administration/geo/replication/security_review.md index 46d3e68ab63..cd54e2dc8c4 100644 --- a/doc/administration/geo/replication/security_review.md +++ b/doc/administration/geo/replication/security_review.md @@ -120,9 +120,7 @@ questions from [owasp.org](https://www.owasp.org). ### What details regarding required OS components and lock‐down needs have been defined? -- The recommended installation method (Omnibus) packages most components itself. - A from-source installation method exists. Both are documented at - <https://docs.gitlab.com/ee/administration/geo/replication/index.html> +- The supported installation method (Omnibus) packages most components itself. - There are significant dependencies on the system-installed OpenSSH daemon (Geo requires users to set up custom authentication methods) and the omnibus or system-provided PostgreSQL daemon (it must be configured to listen on TCP, diff --git a/doc/administration/geo/replication/troubleshooting.md b/doc/administration/geo/replication/troubleshooting.md index 9c95720487d..c5bdd36ba70 100644 --- a/doc/administration/geo/replication/troubleshooting.md +++ b/doc/administration/geo/replication/troubleshooting.md @@ -23,6 +23,8 @@ to help identify if something is wrong:  +For information on how to resolve common errors reported from the UI, see [common errors](#common-errors). + If the UI is not working, or you are unable to log in, you can run the Geo health check manually to get this information as well as a few more details. This rake task can be run on an app node in the **primary** or **secondary** @@ -40,7 +42,8 @@ Checking Geo ... GitLab Geo is available ... yes GitLab Geo is enabled ... yes GitLab Geo secondary database is correctly configured ... yes -Using database streaming replication? ... yes +Database replication enabled? ... yes +Database replication working? ... yes GitLab Geo tracking database is configured to use Foreign Data Wrapper? ... yes GitLab Geo tracking database Foreign Data Wrapper schema is up-to-date? ... yes GitLab Geo HTTP(S) connectivity ... @@ -68,22 +71,22 @@ Example output: ``` http://secondary.example.com/ ----------------------------------------------------- - GitLab Version: 11.8.1-ee + GitLab Version: 11.10.4-ee Geo Role: Secondary Health Status: Healthy - Repositories: 190/190 (100%) - Verified Repositories: 190/190 (100%) - Wikis: 190/190 (100%) - Verified Wikis: 190/190 (100%) - LFS Objects: 35/35 (100%) - Attachments: 528/528 (100%) - CI job artifacts: 477/477 (100%) - Repositories Checked: 0/190 (0%) + Repositories: 289/289 (100%) + Verified Repositories: 289/289 (100%) + Wikis: 289/289 (100%) + Verified Wikis: 289/289 (100%) + LFS Objects: 8/8 (100%) + Attachments: 5/5 (100%) + CI job artifacts: 0/0 (0%) + Repositories Checked: 0/289 (0%) Sync Settings: Full Database replication lag: 0 seconds - Last event ID seen from primary: 2158 (about 2 minute ago) - Last event ID processed by cursor: 2158 (about 2 minute ago) - Last status report was: 4 minutes ago + Last event ID seen from primary: 10215 (about 2 minutes ago) + Last event ID processed by cursor: 10215 (about 2 minutes ago) + Last status report was: 2 minutes ago ``` ## Is Postgres replication working? @@ -307,7 +310,7 @@ same host on different ports. That is, 5432 and 5431 respectively. ### Checking configuration NOTE: **Note:** -The following steps are for Omnibus installs only. Using Geo with source-based installs [is deprecated](index.md#using-gitlab-installed-from-source-deprecated). +The following steps are for Omnibus installs only. Using Geo with source-based installs was **deprecated** in GitLab 11.5. To check the configuration: @@ -455,3 +458,57 @@ reload of the FDW schema. To manually reload the FDW schema: [database-start-replication]: database.md#step-3-initiate-the-replication-process [database-pg-replication]: database.md#postgresql-replication + +## Common errors + +This section documents common errors reported in the admin UI and how to fix them. + +### Geo database configuration file is missing + +GitLab cannot find or doesn't have permission to access the `database_geo.yml` configuration file. + +In an Omnibus GitLab installation, the file should be in `/var/opt/gitlab/gitlab-rails/etc`. +If it doesn't exist or inadvertent changes have been made to it, run `sudo gitlab-ctl reconfigure` to restore it to its correct state. + + +If this path is mounted on a remote volume, please check your volume configuration and that it has correct permissions. + +### Geo node has a database that is writable which is an indication it is not configured for replication with the primary node. + +This error refers to a problem with the database replica on a **secondary** node, +which Geo expects to have access to. It usually means, either: + +- An unsupported replication method was used (for example, logical replication). +- The instructions to setup a [Geo database replication](database.md) were not followed correctly. + +A common source of confusion with **secondary** nodes is that it requires two separate +PostgreSQL instances: + +- A read-only replica of the **primary** node. +- A regular, writable instance that holds replication metadata. That is, the Geo tracking database. + +### Geo node does not appear to be replicating the database from the primary node. + +The most common problems that prevent the database from replicating correctly are: + +- **Secondary** nodes cannot reach the **primary** node. Check credentials, firewall rules, etc. +- SSL certificate problems. Make sure you copied `/etc/gitlab/gitlab-secrets.json` from the **primary** node. +- Database storage disk is full. +- Database replication slot is misconfigured. +- Database is not using a replication slot or another alternative and cannot catch-up because WAL files were purged. + +Make sure you follow the [Geo database replication](database.md) instructions for supported configuration. + +### Geo database version (...) does not match latest migration (...) + +If you are using GitLab Omnibus installation, something might have failed during upgrade. You can: + +- Run `sudo gitlab-ctl reconfigure`. +- Manually trigger the database migration by running: `sudo gitlab-rake geo:db:migrate` as root on the **secondary** node. + +### Geo database is not configured to use Foreign Data Wrapper + +This error means the Geo Tracking Database doesn't have the FDW server and credentials +configured. + +See [How do I fix a "Foreign Data Wrapper (FDW) is not configured" error?](#how-do-i-fix-a-foreign-data-wrapper-fdw-is-not-configured-error). diff --git a/doc/administration/geo/replication/updating_the_geo_nodes.md b/doc/administration/geo/replication/updating_the_geo_nodes.md index 66728366e24..933a75c47d8 100644 --- a/doc/administration/geo/replication/updating_the_geo_nodes.md +++ b/doc/administration/geo/replication/updating_the_geo_nodes.md @@ -337,6 +337,53 @@ is prepended with the relevant node for better clarity: 1. **[secondary]** Create the `replica.sh` script as described in the [database configuration document][database-source-replication]. + 1. 1. **[secondary]** Save the snippet below in a file, let's say `/tmp/replica.sh`. Modify the + embedded paths if necessary: + + ``` + #!/bin/bash + + PORT="5432" + USER="gitlab_replicator" + echo --------------------------------------------------------------- + echo WARNING: Make sure this script is run from the secondary server + echo --------------------------------------------------------------- + echo + echo Enter the IP or FQDN of the primary PostgreSQL server + read HOST + echo Enter the password for $USER@$HOST + read -s PASSWORD + echo Enter the required sslmode + read SSLMODE + + echo Stopping PostgreSQL and all GitLab services + sudo service gitlab stop + sudo service postgresql stop + + echo Backing up postgresql.conf + sudo -u postgres mv /var/opt/gitlab/postgresql/data/postgresql.conf /var/opt/gitlab/postgresql/ + + echo Cleaning up old cluster directory + sudo -u postgres rm -rf /var/opt/gitlab/postgresql/data + + echo Starting base backup as the replicator user + echo Enter the password for $USER@$HOST + sudo -u postgres /opt/gitlab/embedded/bin/pg_basebackup -h $HOST -D /var/opt/gitlab/postgresql/data -U gitlab_replicator -v -x -P + + echo Writing recovery.conf file + sudo -u postgres bash -c "cat > /var/opt/gitlab/postgresql/data/recovery.conf <<- _EOF1_ + standby_mode = 'on' + primary_conninfo = 'host=$HOST port=$PORT user=$USER password=$PASSWORD sslmode=$SSLMODE' + _EOF1_ + " + + echo Restoring postgresql.conf + sudo -u postgres mv /var/opt/gitlab/postgresql/postgresql.conf /var/opt/gitlab/postgresql/data/ + + echo Starting PostgreSQL + sudo service postgresql start + ``` + 1. **[secondary]** Run the recovery script using the credentials from the previous step: @@ -396,8 +443,6 @@ and it is required since 10.0. [update]: ../../../update/README.md [database]: database.md -[database-replication]: database.md#step-3-initiate-the-replication-process -[database-source-replication]: database_source.md#step-3-initiate-the-replication-process [Hashed Storage]: ../../repository_storage_types.md [hashed-migration]: ../../raketasks/storage.md [ssh-fast-lookup]: ../../operations/fast_ssh_key_lookup.md diff --git a/doc/administration/high_availability/README.md b/doc/administration/high_availability/README.md index 24db1c28778..e5701525077 100644 --- a/doc/administration/high_availability/README.md +++ b/doc/administration/high_availability/README.md @@ -64,8 +64,8 @@ larger one. - 1 PostgreSQL node - 1 Redis node -- 2 or more GitLab application nodes (Unicorn, Workhorse, Sidekiq) - 1 NFS/Gitaly storage server +- 2 or more GitLab application nodes (Unicorn, Workhorse, Sidekiq) #### Installation Instructions @@ -88,9 +88,9 @@ in size, indicating that there is contention or not enough resources. - 1 PostgreSQL node - 1 Redis node -- 2 or more GitLab application nodes (Unicorn, Workhorse) -- 2 or more Sidekiq nodes - 2 or more NFS/Gitaly storage servers +- 2 or more Sidekiq nodes +- 2 or more GitLab application nodes (Unicorn, Workhorse) ## High Availability Architecture Examples @@ -135,7 +135,7 @@ the contention. - 2 or more GitLab application nodes (Unicorn, Workhorse, Sidekiq, PGBouncer) - 1 NFS/Gitaly server - + ### Hybrid @@ -153,7 +153,7 @@ contention due to certain workloads. - 1 or more NFS/Gitaly servers - 1 Monitoring node (Prometheus, Grafana) - + #### Reference Architecture @@ -194,7 +194,7 @@ with the added complexity of many more nodes to configure, manage and monitor. - 2 or more Web nodes (All other web requests) - 2 or more NFS/Gitaly servers - + The following pages outline the steps necessary to configure each component separately: diff --git a/doc/administration/high_availability/database.md b/doc/administration/high_availability/database.md index 1648b6b848a..a446ed9637c 100644 --- a/doc/administration/high_availability/database.md +++ b/doc/administration/high_availability/database.md @@ -134,7 +134,7 @@ otherwise the networks will become a single point of failure. #### Architecture - + Database nodes run two services with PostgreSQL: diff --git a/doc/administration/high_availability/img/fully-distributed.png b/doc/administration/high_availability/img/fully-distributed.png Binary files differnew file mode 100644 index 00000000000..ad23207134e --- /dev/null +++ b/doc/administration/high_availability/img/fully-distributed.png diff --git a/doc/administration/high_availability/img/horizontal.png b/doc/administration/high_availability/img/horizontal.png Binary files differnew file mode 100644 index 00000000000..c3bd489d96f --- /dev/null +++ b/doc/administration/high_availability/img/horizontal.png diff --git a/doc/administration/high_availability/img/hybrid.png b/doc/administration/high_availability/img/hybrid.png Binary files differnew file mode 100644 index 00000000000..7d4a56bf0ea --- /dev/null +++ b/doc/administration/high_availability/img/hybrid.png diff --git a/doc/administration/high_availability/pg_ha_architecture.png b/doc/administration/high_availability/img/pg_ha_architecture.png Binary files differindex ef870f652ae..ef870f652ae 100644 --- a/doc/administration/high_availability/pg_ha_architecture.png +++ b/doc/administration/high_availability/img/pg_ha_architecture.png diff --git a/doc/administration/high_availability/nfs.md b/doc/administration/high_availability/nfs.md index caadec3ac4e..d1233d815ed 100644 --- a/doc/administration/high_availability/nfs.md +++ b/doc/administration/high_availability/nfs.md @@ -39,13 +39,8 @@ options: ### Improving NFS performance with GitLab -NOTE: **Note:** -This is only available starting in certain versions of GitLab: - * 11.5.11 - * 11.6.11 - * 11.7.12 - * 11.8.8 - * 11.9.0 and up (e.g. 11.10, 11.11, etc.) +NOTE: **Note:** This is only available starting in certain versions of GitLab: 11.5.11, +11.6.11, 11.7.12, 11.8.8, 11.9.0 and up (e.g. 11.10, 11.11, etc.) If you are using NFS to share Git data, we recommend that you enable a number of feature flags that will allow GitLab application processes to @@ -107,6 +102,11 @@ stored on a local volume. For more details on another person's experience with EFS, see [Amazon's Elastic File System: Burst Credits](https://rawkode.com/2017/04/16/amazons-elastic-file-system-burst-credits/) +## Avoid using CephFS and GlusterFS + +GitLab strongly recommends against using CephFS and GlusterFS. +These distributed file systems are not well-suited for GitLab's input/output access patterns because git uses many small files and access times and file locking times to propagate will make git activity very slow. + ## Avoid using PostgreSQL with NFS GitLab strongly recommends against running your PostgreSQL database diff --git a/doc/administration/integration/plantuml.md b/doc/administration/integration/plantuml.md index b7b820abb40..82e0c14ffc2 100644 --- a/doc/administration/integration/plantuml.md +++ b/doc/administration/integration/plantuml.md @@ -27,7 +27,7 @@ own PlantUML server is easy in Debian/Ubuntu distributions using Tomcat. First you need to create a `plantuml.war` file from the source code: ``` -sudo apt-get install graphviz openjdk-7-jdk git-core maven +sudo apt-get install graphviz openjdk-8-jdk git-core maven git clone https://github.com/plantuml/plantuml-server.git cd plantuml-server mvn package diff --git a/doc/administration/job_artifacts.md b/doc/administration/job_artifacts.md index e7792106f81..ef370573a98 100644 --- a/doc/administration/job_artifacts.md +++ b/doc/administration/job_artifacts.md @@ -100,6 +100,9 @@ artifacts, you can use an object storage like AWS S3 instead. This configuration relies on valid AWS credentials to be configured already. Use an object storage option like AWS S3 to store job artifacts. +DANGER: **Danger:** +If you're enabling S3 in [GitLab HA](high_availability/README.md), you will need to have an [NFS mount set up for CI traces and artifacts](high_availability/nfs.md#a-single-nfs-mount) or enable [live tracing](job_traces.md#new-live-trace-architecture). If these settings are not set, you will risk job traces disappearing or not being saved. + ### Object Storage Settings For source installations the following settings are nested under `artifacts:` and then `object_store:`. On omnibus installs they are prefixed by `artifacts_object_store_`. diff --git a/doc/administration/logs.md b/doc/administration/logs.md index 3d40cda491a..a7e57e44e86 100644 --- a/doc/administration/logs.md +++ b/doc/administration/logs.md @@ -280,6 +280,14 @@ installations from source. Currently it logs the progress of project imports from the Bitbucket Server importer. Future importers may use this file. +## `auth.log` + +Introduced in GitLab 12.0. This file lives in `/var/log/gitlab/gitlab-rails/auth.log` for +Omnibus GitLab packages or in `/home/git/gitlab/log/auth.log` for +installations from source. + +It logs information whenever [Rack Attack] registers an abusive request. + ## Reconfigure Logs Reconfigure log files live in `/var/log/gitlab/reconfigure` for Omnibus GitLab @@ -298,3 +306,4 @@ Omnibus GitLab packages or in `/home/git/gitlab/log/sidekiq_exporter.log` for installations from source. [repocheck]: repository_checks.md +[Rack Attack]: ../security/rack_attack.md diff --git a/doc/administration/monitoring/prometheus/gitlab_metrics.md b/doc/administration/monitoring/prometheus/gitlab_metrics.md index 48bd709a2b7..46a0ac980e1 100644 --- a/doc/administration/monitoring/prometheus/gitlab_metrics.md +++ b/doc/administration/monitoring/prometheus/gitlab_metrics.md @@ -108,6 +108,24 @@ Some basic Ruby runtime metrics are available: [GC.stat]: https://ruby-doc.org/core-2.3.0/GC.html#method-c-stat +## Puma Metrics **[EXPERIMENTAL]** + +When Puma is used instead of Unicorn, following metrics are available: + +| Metric | Type | Since | Description | +|:-------------------------------------------- |:------- |:----- |:----------- | +| puma_workers | Gauge | 12.0 | Total number of workers | +| puma_running_workers | Gauge | 12.0 | Number of booted workers | +| puma_stale_workers | Gauge | 12.0 | Number of old workers | +| puma_phase | Gauge | 12.0 | Phase number (increased during phased restarts) | +| puma_running | Gauge | 12.0 | Number of running threads | +| puma_queued_connections | Gauge | 12.0 | Number of connections in that worker's "todo" set waiting for a worker thread | +| puma_active_connections | Gauge | 12.0 | Number of threads processing a request | +| puma_pool_capacity | Gauge | 12.0 | Number of requests the worker is capable of taking right now | +| puma_max_threads | Gauge | 12.0 | Maximum number of worker threads | +| puma_idle_threads | Gauge | 12.0 | Number of spawned threads which are not processing a request | + + ## Metrics shared directory GitLab's Prometheus client requires a directory to store metrics data shared between multi-process services. diff --git a/doc/administration/pages/index.md b/doc/administration/pages/index.md index 1373bd56fe3..3a7ca517d56 100644 --- a/doc/administration/pages/index.md +++ b/doc/administration/pages/index.md @@ -291,6 +291,20 @@ Pages access control is disabled by default. To enable it: 1. [Reconfigure GitLab][reconfigure]. 1. Users can now configure it in their [projects' settings](../../user/project/pages/introduction.md#gitlab-pages-access-control-core-only). +### Running behind a proxy + +Like the rest of GitLab, Pages can be used in those environments where external +internet connectivity is gated by a proxy. In order to use a proxy for GitLab +pages: + +1. Configure in `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_pages['http_proxy'] = 'http://example:8080' + ``` + +1. [Reconfigure Gitlab][reconfigure] for the changes to take effect. + ## Activate verbose logging for daemon Verbose logging was [introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/2533) in @@ -376,6 +390,7 @@ Follow the steps below to configure GitLab Pages in a separate server. gitaly['enable'] = false alertmanager['enable'] = false node_exporter['enable'] = false + gitlab_rails['auto_migrate'] = false ``` 1. Run `sudo gitlab-ctl reconfigure`. 1. On `app1` apply the following changes to `/etc/gitlab/gitlab.rb`: diff --git a/doc/api/README.md b/doc/api/README.md index 7ec7955c596..439f58c1253 100644 --- a/doc/api/README.md +++ b/doc/api/README.md @@ -186,6 +186,8 @@ curl "https://gitlab.example.com/api/v4/projects" The API uses JSON to serialize data. You don't need to specify `.json` at the end of an API URL. +All of the API endpoints that use the `POST`, `PUT` or `PATCH` method support params in the request body, with `Content-Type` `application/x-www-form-urlencoded`, `multipart/form-data` or `application/json`. + ## Authentication Most API requests require authentication, or will only return public data when diff --git a/doc/api/epic_links.md b/doc/api/epic_links.md index 619ae6ea2dc..d6e43ae7074 100644 --- a/doc/api/epic_links.md +++ b/doc/api/epic_links.md @@ -125,7 +125,7 @@ POST /groups/:id/epics/:epic_iid/epics | --------------- | -------------- | -------- | ------------------------------------------------------------------------------------------------------------------ | | `id` | integer/string | yes | The ID or [URL-encoded path of the group](README.md#namespaced-path-encoding) owned by the authenticated user | | `epic_iid` | integer | yes | The internal ID of the (future parent) epic. | -| `title` | integer | yes | The global ID of the child epic. Internal ID can't be used because they can conflict with epics from other groups. | +| `title` | string | yes | The title of a newly created epic. | ```bash curl --header POST "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/groups/1/epics/5/epics?title=Newpic @@ -141,6 +141,7 @@ Example response: "group_id": 49, "parent_id": 23, "has_children": false, + "has_issues": false, "reference": "&2", "url": "http://localhost/groups/group16/-/epics/2", "relation_url": "http://localhost/groups/group16/-/epics/1/links/24" diff --git a/doc/api/graphql/index.md b/doc/api/graphql/index.md index cf02bbd9c92..9195ba4cdf1 100644 --- a/doc/api/graphql/index.md +++ b/doc/api/graphql/index.md @@ -16,6 +16,20 @@ added to the API without creating breaking changes. This allows us to have a versionless API as described in [the GraphQL documentation](https://graphql.org/learn/best-practices/#versioning). +## Vision + +We want the GraphQL API to be the **primary** means of interacting +programmatically with GitLab. To achieve this, it needs full coverage - anything +possible in the REST API should also be possible in the GraphQL API. + +To help us meet this vision, the frontend should use GraphQL in preference to +the REST API for new features, although the alpha status of GraphQL may prevent +this from being a possibility at times. + +There are no plans to deprecate the REST API. To reduce the technical burden of +supporting two APIs in parallel, they should share implementations as much as +possible. + ## Enabling the GraphQL feature The GraphQL API itself is currently in Alpha, and therefore hidden behind a @@ -32,9 +46,16 @@ curl --data "value=100" --header "PRIVATE-TOKEN: <your_access_token>" https://gi A first iteration of a GraphQL API includes the following queries 1. `project` : Within a project it is also possible to fetch a `mergeRequest` by IID. - 1. `group` : Only basic group information is currently supported. +### Multiplex queries + +GitLab supports batching queries into a single request using +[apollo-link-batch-http](https://www.apollographql.com/docs/link/links/batch-http). More +info about multiplexed queries is also available for +[graphql-ruby](https://graphql-ruby.org/queries/multiplex.html) the +library GitLab uses on the backend. + ## GraphiQL The API can be explored by using the GraphiQL IDE, it is available on your diff --git a/doc/api/jobs.md b/doc/api/jobs.md index 877cd99723a..d51451b9c83 100644 --- a/doc/api/jobs.md +++ b/doc/api/jobs.md @@ -32,6 +32,7 @@ Example of response "title": "Test the CI integration." }, "coverage": null, + "allow_failure": false, "created_at": "2015-12-24T15:51:21.727Z", "started_at": "2015-12-24T17:54:24.729Z", "finished_at": "2015-12-24T17:54:24.921Z", @@ -81,6 +82,7 @@ Example of response "title": "Test the CI integration." }, "coverage": null, + "allow_failure": false, "created_at": "2015-12-24T15:51:21.802Z", "started_at": "2015-12-24T17:54:27.722Z", "finished_at": "2015-12-24T17:54:27.895Z", @@ -165,6 +167,7 @@ Example of response "title": "Test the CI integration." }, "coverage": null, + "allow_failure": false, "created_at": "2015-12-24T15:51:21.727Z", "started_at": "2015-12-24T17:54:24.729Z", "finished_at": "2015-12-24T17:54:24.921Z", @@ -214,6 +217,7 @@ Example of response "title": "Test the CI integration." }, "coverage": null, + "allow_failure": false, "created_at": "2015-12-24T15:51:21.802Z", "started_at": "2015-12-24T17:54:27.722Z", "finished_at": "2015-12-24T17:54:27.895Z", @@ -296,6 +300,7 @@ Example of response "title": "Test the CI integration." }, "coverage": null, + "allow_failure": false, "created_at": "2015-12-24T15:51:21.880Z", "started_at": "2015-12-24T17:54:30.733Z", "finished_at": "2015-12-24T17:54:31.198Z", @@ -528,6 +533,7 @@ Example of response "title": "Test the CI integration." }, "coverage": null, + "allow_failure": false, "created_at": "2016-01-11T10:13:33.506Z", "started_at": "2016-01-11T10:14:09.526Z", "finished_at": null, @@ -576,6 +582,7 @@ Example of response "title": "Test the CI integration." }, "coverage": null, + "allow_failure": false, "created_at": "2016-01-11T10:13:33.506Z", "started_at": null, "finished_at": null, @@ -628,6 +635,7 @@ Example of response "title": "Test the CI integration." }, "coverage": null, + "allow_failure": false, "download_url": null, "id": 42, "name": "rubocop", @@ -681,6 +689,7 @@ Example response: "title": "Test the CI integration." }, "coverage": null, + "allow_failure": false, "download_url": null, "id": 42, "name": "rubocop", @@ -757,6 +766,7 @@ Example of response "title": "Test the CI integration." }, "coverage": null, + "allow_failure": false, "created_at": "2016-01-11T10:13:33.506Z", "started_at": null, "finished_at": null, diff --git a/doc/api/members.md b/doc/api/members.md index 0593d2c20ea..8784d577f99 100644 --- a/doc/api/members.md +++ b/doc/api/members.md @@ -62,7 +62,9 @@ Example response: ## List all members of a group or project including inherited members Gets a list of group or project members viewable by the authenticated user, including inherited members through ancestor groups. -Returns multiple times the same user (with different member attributes) when the user is a member of the project/group and of one or more ancestor group. +When a user is a member of the project/group and of one or more ancestor groups the user is returned only once with the project access_level (if exists) +or the access_level for the user in the first group which he belongs to in the project groups ancestors chain. +**Note:** We plan to [change](https://gitlab.com/gitlab-org/gitlab-ce/issues/62284) this behavior to return highest access_level instead. ``` GET /groups/:id/members/all diff --git a/doc/api/project_level_variables.md b/doc/api/project_level_variables.md index 4a6f5624394..3b00f6f140e 100644 --- a/doc/api/project_level_variables.md +++ b/doc/api/project_level_variables.md @@ -52,7 +52,9 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a { "key": "TEST_VARIABLE_1", "variable_type": "env_var", - "value": "TEST_1" + "value": "TEST_1", + "protected": false, + "masked": true } ``` @@ -71,6 +73,7 @@ POST /projects/:id/variables | `value` | string | yes | The `value` of a variable | | `variable_type` | string | no | The type of a variable. Available types are: `env_var` (default) and `file` | | `protected` | boolean | no | Whether the variable is protected | +| `masked` | boolean | no | Whether the variable is masked | ``` curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/variables" --form "key=NEW_VARIABLE" --form "value=new value" @@ -81,7 +84,8 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitla "key": "NEW_VARIABLE", "value": "new value", "variable_type": "env_var", - "protected": false + "protected": false, + "masked": false } ``` @@ -100,6 +104,7 @@ PUT /projects/:id/variables/:key | `value` | string | yes | The `value` of a variable | | `variable_type` | string | no | The type of a variable. Available types are: `env_var` (default) and `file` | | `protected` | boolean | no | Whether the variable is protected | +| `masked` | boolean | no | Whether the variable is masked | ``` curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/variables/NEW_VARIABLE" --form "value=updated value" @@ -110,7 +115,8 @@ curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab "key": "NEW_VARIABLE", "value": "updated value", "variable_type": "env_var", - "protected": true + "protected": true, + "masked": false } ``` diff --git a/doc/api/repository_storage_health.md b/doc/api/repository_storage_health.md deleted file mode 100644 index edf4b04acea..00000000000 --- a/doc/api/repository_storage_health.md +++ /dev/null @@ -1,5 +0,0 @@ -# Circuitbreaker API - -NOTE: **Deprecated:** -Support of the circuit breaker is removed, as Gitaly can be configured to -to work without NFS and [communicate solely over HTTP](../administration/gitaly/index.md). diff --git a/doc/api/services.md b/doc/api/services.md index 742abccb69e..01df2a50198 100644 --- a/doc/api/services.md +++ b/doc/api/services.md @@ -754,6 +754,7 @@ Parameters: | `recipients` | string | yes | Comma-separated list of recipient email addresses | | `add_pusher` | boolean | no | Add pusher to recipients list | | `notify_only_broken_pipelines` | boolean | no | Notify only broken pipelines | +| `notify_only_default_branch` | boolean | no | Send notifications only for the default branch ([introduced in GitLab 12.0](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/28271)) | ### Delete Pipeline-Emails service diff --git a/doc/ci/caching/index.md b/doc/ci/caching/index.md index 3f72fe3e9eb..d703e106e76 100644 --- a/doc/ci/caching/index.md +++ b/doc/ci/caching/index.md @@ -100,7 +100,7 @@ From the perspective of the Runner, in order for cache to work effectively, one of the following must be true: - Use a single Runner for all your jobs. -- Use multiple Runners (in autoscale mode or not) that use. +- Use multiple Runners (in autoscale mode or not) that use [distributed caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching), where the cache is stored in S3 buckets (like shared Runners on GitLab.com). - Use multiple Runners (not in autoscale mode) of the same architecture that diff --git a/doc/ci/docker/using_docker_build.md b/doc/ci/docker/using_docker_build.md index 5222cc45bc4..dd112dadc40 100644 --- a/doc/ci/docker/using_docker_build.md +++ b/doc/ci/docker/using_docker_build.md @@ -142,9 +142,12 @@ In order to do that, follow the steps: # The 'docker' hostname is the alias of the service container as described at # https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services # - # Note that if you're using Kubernetes executor, the variable should be set to - # tcp://localhost:2375 because of how Kubernetes executor connects services + # Note that if you're using the Kubernetes executor, the variable should be set to + # tcp://localhost:2375/ because of how the Kubernetes executor connects services # to the job container + # DOCKER_HOST: tcp://localhost:2375/ + # + # For non-Kubernetes executors, we use tcp://docker:2375/ DOCKER_HOST: tcp://docker:2375/ # When using dind, it's wise to use the overlayfs driver for # improved performance. diff --git a/doc/ci/multi_project_pipelines.md b/doc/ci/multi_project_pipelines.md index 556059c01b6..e9deabf27f8 100644 --- a/doc/ci/multi_project_pipelines.md +++ b/doc/ci/multi_project_pipelines.md @@ -134,6 +134,34 @@ staging: The `ENVIRONMENT` variable will be passed to every job defined in a downstream pipeline. It will be available as an environment variable when GitLab Runner picks a job. +In the following configuration, the `MY_VARIABLE` variable will be passed +downstream, because jobs inherit variables declared in top-level `variables`: + +```yaml +variables: + MY_VARIABLE: my-value + +my-pipeline: + variables: + ENVIRONMENT: something + trigger: my/project +``` + +You might want to pass some information about the upstream pipeline using, for +example, predefined variables. In order to do that, you can use interpolation +to pass any variable. For example: + +```yaml +my-pipeline: + variables: + UPSTREAM_BRANCH: $CI_COMMIT_REF_NAME + trigger: my/project +``` + +In this scenario, the `UPSTREAM_BRANCH` variable with a value related to the +upstream pipeline will be passed to a `downstream` job, and will be available +within the context of all downstream builds. + ### Limitations Because bridge jobs are a little different to regular jobs, it is not diff --git a/doc/ci/pipelines.md b/doc/ci/pipelines.md index 342c2ab972a..4dbe1a85588 100644 --- a/doc/ci/pipelines.md +++ b/doc/ci/pipelines.md @@ -330,6 +330,19 @@ GitLab provides API endpoints to: - [Triggering pipelines through the API](triggers/README.md). - [Pipeline triggers API](../api/pipeline_triggers.md). +### Start multiple manual actions in a stage + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/27188) in GitLab 11.11. + +Multiple manual actions in a single stage can be started at the same time using the "Play all manual" button. +Once the user clicks this button, each individual manual action will be triggered and refreshed +to an updated status. + +This functionality is only available: + +- For users with at least Developer access. +- If the the stage contains [manual actions](#manual-actions-from-pipeline-graphs). + ## Security on protected branches A strict security model is enforced when pipelines are executed on diff --git a/doc/ci/variables/README.md b/doc/ci/variables/README.md index 1ba22070abe..d33275cae4f 100644 --- a/doc/ci/variables/README.md +++ b/doc/ci/variables/README.md @@ -58,8 +58,29 @@ the need to specify the value itself. There are two types of variables supported by GitLab: -- `env_var`: the runner will create environment variable named same as the variable key and set its value to the variable value. -- `file`: the runner will write the variable value to a temporary file and set the path to this file as the value of an environment variable named same as the variable key. +- "Variable": the Runner will create an environment variable named same as the variable key and set its value to the variable value. +- "File": the Runner will write the variable value to a temporary file and set the path to this file as the value of an environment variable named same as the variable key. + +Many tools (like [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html) and [kubectl](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/#the-kubeconfig-environment-variable)) provide the ability to customise configuration using files by either providing the file path as a command line argument or an environment variable. Prior to the introduction of variable types, the common pattern was to use the value of a CI variable, save it in a file, and then use the newly created file in your script: + +```bash +# Save the content of variable in a file +echo "$KUBE_CA_PEM" > "$(pwd)/kube.ca.pem" + # Use the newly created file +kubectl config set-cluster e2e --server="$KUBE_URL" --certificate-authority="$(pwd)/kube.ca.pem" +``` + +This can be simplified by creating a variable of type "File" and using it directly. For example, let's say we have the following variables. + + + +We can then call them from `.gitlab-ci.yml` like this: + +```bash +kubectl config set-cluster e2e --server="$KUBE_URL" --certificate-authority="$KUBE_CA_PEM" +``` + +Variable types can be set via the [UI](#via-the-ui) or the [API](../../api/project_level_variables.md#create-variable), but not in `.gitlab-ci.yml`. #### Masked variables @@ -591,8 +612,8 @@ $'\''git'\'' "checkout" "-f" "-q" "dd648b2e48ce6518303b0bb580b2ee32fadaf045" Running on runner-8a2f473d-project-1796893-concurrent-0 via runner-8a2f473d-machine-1480971377-317a7d0f-digital-ocean-4gb... ++ export CI=true ++ CI=true -++ export CI_API_V4_API_URL=https://example.com:3000/api/v4 -++ CI_API_V4_API_URL=https://example.com:3000/api/v4 +++ export CI_API_V4_URL=https://example.com:3000/api/v4 +++ CI_API_V4_URL=https://example.com:3000/api/v4 ++ export CI_DEBUG_TRACE=false ++ CI_DEBUG_TRACE=false ++ export CI_COMMIT_SHA=dd648b2e48ce6518303b0bb580b2ee32fadaf045 @@ -631,8 +652,8 @@ Running on runner-8a2f473d-project-1796893-concurrent-0 via runner-8a2f473d-mach ++ GITLAB_CI=true ++ export CI=true ++ CI=true -++ export CI_API_V4_API_URL=https://example.com:3000/api/v4 -++ CI_API_V4_API_URL=https://example.com:3000/api/v4 +++ export CI_API_V4_URL=https://example.com:3000/api/v4 +++ CI_API_V4_URL=https://example.com:3000/api/v4 ++ export GITLAB_CI=true ++ GITLAB_CI=true ++ export CI_JOB_ID=7046507 diff --git a/doc/ci/variables/img/new_custom_variables_example.png b/doc/ci/variables/img/new_custom_variables_example.png Binary files differindex 4b78e0ff587..efe104efe4c 100644 --- a/doc/ci/variables/img/new_custom_variables_example.png +++ b/doc/ci/variables/img/new_custom_variables_example.png diff --git a/doc/ci/variables/img/variable_types_usage_example.png b/doc/ci/variables/img/variable_types_usage_example.png Binary files differnew file mode 100644 index 00000000000..0e8bde891fe --- /dev/null +++ b/doc/ci/variables/img/variable_types_usage_example.png diff --git a/doc/ci/yaml/README.md b/doc/ci/yaml/README.md index 31ff56e06f8..18c85618b1b 100644 --- a/doc/ci/yaml/README.md +++ b/doc/ci/yaml/README.md @@ -386,17 +386,12 @@ job: - branches@gitlab-org/gitlab-ce except: - master@gitlab-org/gitlab-ce - - release/.*@gitlab-org/gitlab-ce + - /^release/.*$/@gitlab-org/gitlab-ce ``` The above example will run `job` for all branches on `gitlab-org/gitlab-ce`, except `master` and those with names prefixed with `release/`. -NOTE: **Note:** -Because `@` is used to denote the beginning of a ref's repository path, -matching a ref name containing the `@` character in a regular expression -requires the use of the hex character code match `\x40`. - If a job does not have an `only` rule, `only: ['branches', 'tags']` is set by default. If it doesn't have an `except` rule, it is empty. @@ -415,6 +410,28 @@ job: only: ['branches', 'tags'] ``` +#### Regular expressions + +Because `@` is used to denote the beginning of a ref's repository path, +matching a ref name containing the `@` character in a regular expression +requires the use of the hex character code match `\x40`. + +Only the tag or branch name can be matched by a regular expression. +The repository path, if given, is always matched literally. + +If a regular expression shall be used to match the tag or branch name, +the entire ref name part of the pattern has to be a regular expression, +and must be surrounded by `/`. +(With regular expression flags appended after the closing `/`.) +So `issue-/.*/` won't work to match all tag names or branch names +that begin with `issue-`. + +TIP: **Tip** +Use anchors `^` and `$` to avoid the regular expression +matching only a substring of the tag name or branch name. +For example, `/^issue-.*$/` is equivalent to `/^issue-/`, +while just `/issue/` would also match a branch called `severe-issues`. + ### Supported `only`/`except` regexp syntax CAUTION: **Warning:** @@ -1986,7 +2003,7 @@ production: - deploy environment: name: production - url: https://$CI_PROJECT_PATH_SLUG.$AUTO_DEVOPS_DOMAIN + url: https://$CI_PROJECT_PATH_SLUG.$KUBE_INGRESS_BASE_DOMAIN only: - master ``` diff --git a/doc/development/README.md b/doc/development/README.md index 2ff38d68a47..624665a42d1 100644 --- a/doc/development/README.md +++ b/doc/development/README.md @@ -59,6 +59,7 @@ description: 'Learn how to contribute to GitLab.' - [DeclarativePolicy framework](policies.md) - [How Git object deduplication works in GitLab](git_object_deduplication.md) - [Geo development](geo.md) +- [Routing](routing.md) ## Performance guides diff --git a/doc/development/architecture.md b/doc/development/architecture.md index 9a012f4299b..4b76d5f9c5b 100644 --- a/doc/development/architecture.md +++ b/doc/development/architecture.md @@ -106,43 +106,43 @@ Component statuses are linked to configuration documentation for each component. ### Component list -| Component | Description | [Omnibus GitLab](https://docs.gitlab.com/omnibus/README.html) | [GitLab chart](https://docs.gitlab.com/charts/) | [Minikube Minimal](https://docs.gitlab.com/charts/development/minikube/#deploying-gitlab-with-minimal-settings) | [GitLab.com](https://gitlab.com) | CE/EE | -| --------- | ----------- |:--------------------:|:------------------:|:-----:|:--------:|:--------:| -| [NGINX](#nginx) | Routes requests to appropriate components, terminates SSL | [✅][nginx-omnibus] | [✅][nginx-charts] | [⚙][nginx-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | CE & EE | -| [Unicorn (GitLab Rails)](#unicorn) | Handles requests for the web interface and API | [✅][unicorn-omnibus] | [✅][unicorn-charts] | [✅][unicorn-charts] | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#unicorn) | CE & EE | -| [Sidekiq](#sidekiq) | Background jobs processor | [✅][sidekiq-omnibus] | [✅][sidekiq-charts] | [✅](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/index.html) | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#sidekiq) | CE & EE | -| [Gitaly](#gitaly) | Git RPC service for handling all git calls made by GitLab | [✅][gitaly-omnibus] | [✅][gitaly-charts] | [✅][gitaly-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | CE & EE | -| [GitLab Workhorse](#gitlab-workhorse) | Smart reverse proxy, handles large HTTP requests | [✅][workhorse-omnibus] | [✅][workhorse-charts] | [✅][workhorse-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | CE & EE | -| [GitLab Shell](#gitlab-shell) | Handles `git` over SSH sessions | [✅][shell-omnibus] | [✅][shell-charts] | [✅][shell-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | CE & EE | -| [GitLab Pages](#gitlab-pages) | Hosts static websites | [⚙][pages-omnibus] | [❌][pages-charts] | [❌][pages-charts] | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#gitlab-pages) | CE & EE | -| [Registry](#registry) | Container registry, allows pushing and pulling of images | [⚙][registry-omnibus] | [✅][registry-charts] | [✅][registry-charts] | [✅](https://docs.gitlab.com/ee/user/project/container_registry.html#build-and-push-images) | CE & EE | -| [Redis](#redis) | Caching service | [✅][redis-omnibus] | [✅][redis-omnibus] | [✅][redis-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | CE & EE | -| [PostgreSQL](#postgresql) | Database | [✅][postgres-omnibus] | [✅][postgres-charts] | [✅][postgres-charts] | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#postgresql) | CE & EE | -| [PgBouncer](#pgbouncer) | Database connection pooling, failover | [⚙][pgbouncer-omnibus] | [❌][pgbouncer-charts] | [❌][pgbouncer-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#database-architecture) | EE Only | -| [Consul](#consul) | Database node discovery, failover | [⚙][consul-omnibus] | [❌][consul-charts] | [❌][consul-charts] | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#consul) | EE Only | -| [GitLab self-monitoring: Prometheus](#prometheus) | Time-series database, metrics collection, and query service | [✅][prometheus-omnibus] | [✅][prometheus-charts] | [⚙][prometheus-charts] | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#prometheus) | CE & EE | -| [GitLab self-monitoring: Alertmanager](#alertmanager) | Deduplicates, groups, and routes alerts from Prometheus | [✅][alertmanager-omnibus] | [✅][alertmanager-charts] | [⚙][alertmanager-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | CE & EE | -| [GitLab self-monitoring: Grafana](#grafana) | Metrics dashboard | [⚙][grafana-omnibus] | [⤓][grafana-charts] | [⤓][grafana-charts] | [✅](https://dashboards.gitlab.com/d/RZmbBr7mk/gitlab-triage?refresh=30s) | CE & EE | -| [GitLab self-monitoring: Sentry](#sentry) | Track errors generated by the GitLab instance | [⤓][sentry-omnibus] | [❌][sentry-charts] | [❌][sentry-charts] | [✅](https://about.gitlab.com/handbook/support/workflows/services/gitlab_com/500_errors.html#searching-sentry) | CE & EE | -| [GitLab self-monitoring: Jaeger](#jaeger) | View traces generated by the GitLab instance | [❌][jaeger-omnibus] | [❌][jaeger-charts] | [❌][jaeger-charts] | [❌](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/4104) | CE & EE | -| [Redis Exporter](#redis-exporter) | Prometheus endpoint with Redis metrics | [✅][redis-exporter-omnibus] | [✅][redis-exporter-charts] | [✅][redis-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | CE & EE | -| [Postgres Exporter](#postgres-exporter) | Prometheus endpoint with PostgreSQL metrics | [✅][postgres-exporter-omnibus] | [✅][postgres-exporter-charts] | [✅][postgres-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | CE & EE | -| [PgBouncer Exporter](#pgbouncer-exporter) | Prometheus endpoint with PgBouncer metrics | [⚙][pgbouncer-exporter-omnibus] | [❌][pgbouncer-exporter-charts] | [❌][pgbouncer-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | CE & EE | -| [GitLab Monitor](#gitlab-monitor) | Generates a variety of GitLab metrics | [✅][gitlab-monitor-omnibus] | [❌][gitab-monitor-charts] | [❌][gitab-monitor-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | CE & EE | -| [Node Exporter](#node-exporter) | Prometheus endpoint with system metrics | [✅][node-exporter-omnibus] | [❌][node-exporter-charts] | [❌][node-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | CE & EE | -| [Mattermost](#mattermost) | Open-source Slack alternative | [⚙][mattermost-omnibus] | [⤓][mattermost-charts] | [⤓][mattermost-charts] | [⤓](https://docs.gitlab.com/ee/user/project/integrations/mattermost_slash_commands.html#manual-configuration), [⤓](https://docs.gitlab.com/ee/user/project/integrations/mattermost.html) | CE & EE | -| [Minio](#minio) | Object storage service | [⤓][minio-omnibus] | [✅][minio-charts] | [✅][minio-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#storage-architecture) | CE & EE | -| [Runner](#gitlab-runner) | Executes GitLab CI jobs | [⤓][runner-omnibus] | [✅][runner-charts] | [⚙][runner-charts] | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#shared-runners) | CE & EE | -| [Database Migrations](#database-migrations) | Database migrations | [✅][database-migrations-omnibus] | [✅]() | [✅][database-migrations-charts] | [✅][database-migrations-charts] | CE & EE | -| [Certificate Management](#certificate-management) | TLS Settings, Let's Encrypt | [✅][certificate-management-omnibus] | [✅][certificate-management-charts] | [⚙][certificate-management-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#secrets-management) | CE & EE | -| [GitLab Geo Node](#gitlab-geo) | Geographically distributed GitLab nodes | [⚙][geo-omnibus] | [❌][geo-charts] | [❌][geo-charts] | ✅ | EE Only | -| [LDAP Authentication](#ldap-authentication) | Authenticate users against centralized LDAP directory | [⤓][ldap-omnibus] | [⤓][ldap-charts] | [⤓][ldap-charts] | [❌](https://about.gitlab.com/pricing/#gitlab-com) | CE & EE | -| [Outbound email (SMTP)](#outbound-email) | Send email messages to users | [⤓][outbound-email-omnibus] | [⤓][outbound-email-charts] | [⤓][outbound-email-charts] | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#mail-configuration) | CE & EE | -| [Inbound email (SMTP)](#inbound-email) | Receive messages to update issues | [⤓][inbound-email-omnibus] | [⤓][inbound-email-charts] | [⤓][inbound-email-charts] | [✅](https://docs.gitlab.com/ee/user/gitlab_com/#mail-configuration) | CE & EE | -| [ElasticSearch](#elasticsearch) | Improved search within GitLab | [⤓][elasticsearch-omnibus] | [⤓][elasticsearch-charts] | [⤓][elasticsearch-charts] | [❌](https://gitlab.com/groups/gitlab-org/-/epics/153) | EE Only | -| [Sentry integration](#sentry) | Error tracking for deployed apps | [⤓][sentry-integration] | [⤓][sentry-integration] | [⤓][sentry-integration] | [⤓][sentry-integration] | CE & EE | -| [Jaeger integration](#jaeger) | Distributed tracing for deployed apps | [⤓][jaeger-integration] | [⤓][jaeger-integration] | [⤓][jaeger-integration] | [⤓][jaeger-integration] | EE Only | -| [Kubernetes cluster apps](#kubernetes-cluster-apps) | Deploy [Helm](https://docs.helm.sh/), [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/), [Cert-Manager](https://docs.cert-manager.io/en/latest/), [Prometheus](https://prometheus.io/docs/introduction/overview/), a [Runner](https://docs.gitlab.com/runner/), [JupyterHub](http://jupyter.org/), [Knative](https://cloud.google.com/knative) to a cluster | [⤓][managed-k8s-apps] | [⤓][managed-k8s-apps] | [⤓][managed-k8s-apps] | [⤓][managed-k8s-apps] | CE & EE | +| Component | Description | [Omnibus GitLab](https://docs.gitlab.com/omnibus/) | [GitLab chart](https://docs.gitlab.com/charts/) | [Minikube Minimal](https://docs.gitlab.com/charts/development/minikube/#deploying-gitlab-with-minimal-settings) | [GitLab.com](https://gitlab.com) | [Source](../install/installation.md) | [GDK](https://gitlab.com/gitlab-org/gitlab-development-kit) | CE/EE | +| --------- | ----------- |:--------------------:|:------------------:|:-----:|:--------:|:--------:|:-------:|:-------:| +| [NGINX](#nginx) | Routes requests to appropriate components, terminates SSL | [✅][nginx-omnibus] | [✅][nginx-charts] | [⚙][nginx-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | [⤓][nginx-source] | ❌ | CE & EE | +| [Unicorn (GitLab Rails)](#unicorn) | Handles requests for the web interface and API | [✅][unicorn-omnibus] | [✅][unicorn-charts] | [✅][unicorn-charts] | [✅](../user/gitlab_com/index.md#unicorn) | [⚙][unicorn-source] | [✅][gitlab-yml] | CE & EE | +| [Sidekiq](#sidekiq) | Background jobs processor | [✅][sidekiq-omnibus] | [✅][sidekiq-charts] | [✅](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/index.html) | [✅](../user/gitlab_com/index.md#sidekiq) | [✅][gitlab-yml] | [✅][gitlab-yml] | CE & EE | +| [Gitaly](#gitaly) | Git RPC service for handling all git calls made by GitLab | [✅][gitaly-omnibus] | [✅][gitaly-charts] | [✅][gitaly-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | [⚙][gitaly-source] | ✅ | CE & EE | +| [GitLab Workhorse](#gitlab-workhorse) | Smart reverse proxy, handles large HTTP requests | [✅][workhorse-omnibus] | [✅][workhorse-charts] | [✅][workhorse-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | [⚙][workhorse-source] | ✅ | CE & EE | +| [GitLab Shell](#gitlab-shell) | Handles `git` over SSH sessions | [✅][shell-omnibus] | [✅][shell-charts] | [✅][shell-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | [⚙][shell-source] | [✅][gitlab-yml] | CE & EE | +| [GitLab Pages](#gitlab-pages) | Hosts static websites | [⚙][pages-omnibus] | [❌][pages-charts] | [❌][pages-charts] | [✅](../user/gitlab_com/index.md#gitlab-pages) | [⚙][pages-source] | [⚙][pages-gdk] | CE & EE | +| [Registry](#registry) | Container registry, allows pushing and pulling of images | [⚙][registry-omnibus] | [✅][registry-charts] | [✅][registry-charts] | [✅](../user/project/container_registry.md#build-and-push-images) | [⤓][registry-source] | [⚙][registry-gdk] | CE & EE | +| [Redis](#redis) | Caching service | [✅][redis-omnibus] | [✅][redis-omnibus] | [✅][redis-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#service-architecture) | [⤓][redis-source] | ✅ | CE & EE | +| [PostgreSQL](#postgresql) | Database | [✅][postgres-omnibus] | [✅][postgres-charts] | [✅][postgres-charts] | [✅](../user/gitlab_com/index.md#postgresql) | [⤓][postgres-source] | ✅ | CE & EE | +| [PgBouncer](#pgbouncer) | Database connection pooling, failover | [⚙][pgbouncer-omnibus] | [❌][pgbouncer-charts] | [❌][pgbouncer-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#database-architecture) | ❌ | ❌ | EE Only | +| [Consul](#consul) | Database node discovery, failover | [⚙][consul-omnibus] | [❌][consul-charts] | [❌][consul-charts] | [✅](../user/gitlab_com/index.md#consul) | ❌ | ❌ | EE Only | +| [GitLab self-monitoring: Prometheus](#prometheus) | Time-series database, metrics collection, and query service | [✅][prometheus-omnibus] | [✅][prometheus-charts] | [⚙][prometheus-charts] | [✅](../user/gitlab_com/index.md#prometheus) | ❌ | ❌ | CE & EE | +| [GitLab self-monitoring: Alertmanager](#alertmanager) | Deduplicates, groups, and routes alerts from Prometheus | [✅][alertmanager-omnibus] | [✅][alertmanager-charts] | [⚙][alertmanager-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | +| [GitLab self-monitoring: Grafana](#grafana) | Metrics dashboard | [⚙][grafana-omnibus] | [⤓][grafana-charts] | [⤓][grafana-charts] | [✅](https://dashboards.gitlab.com/d/RZmbBr7mk/gitlab-triage?refresh=30s) | ❌ | ❌ | CE & EE | +| [GitLab self-monitoring: Sentry](#sentry) | Track errors generated by the GitLab instance | [⤓][sentry-omnibus] | [❌][sentry-charts] | [❌][sentry-charts] | [✅](https://about.gitlab.com/handbook/support/workflows/services/gitlab_com/500_errors.html#searching-sentry) | [⤓][gitlab-yml] | [⤓][gitlab-yml] | CE & EE | +| [GitLab self-monitoring: Jaeger](#jaeger) | View traces generated by the GitLab instance | [❌][jaeger-omnibus] | [❌][jaeger-charts] | [❌][jaeger-charts] | [❌](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/4104) | [⤓][jaeger-source] | [⚙][jaeger-gdk] | CE & EE | +| [Redis Exporter](#redis-exporter) | Prometheus endpoint with Redis metrics | [✅][redis-exporter-omnibus] | [✅][redis-exporter-charts] | [✅][redis-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | +| [Postgres Exporter](#postgres-exporter) | Prometheus endpoint with PostgreSQL metrics | [✅][postgres-exporter-omnibus] | [✅][postgres-exporter-charts] | [✅][postgres-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | +| [PgBouncer Exporter](#pgbouncer-exporter) | Prometheus endpoint with PgBouncer metrics | [⚙][pgbouncer-exporter-omnibus] | [❌][pgbouncer-exporter-charts] | [❌][pgbouncer-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | +| [GitLab Monitor](#gitlab-monitor) | Generates a variety of GitLab metrics | [✅][gitlab-monitor-omnibus] | [❌][gitab-monitor-charts] | [❌][gitab-monitor-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | +| [Node Exporter](#node-exporter) | Prometheus endpoint with system metrics | [✅][node-exporter-omnibus] | [❌][node-exporter-charts] | [❌][node-exporter-charts] | [✅](https://about.gitlab.com/handbook/engineering/monitoring/) | ❌ | ❌ | CE & EE | +| [Mattermost](#mattermost) | Open-source Slack alternative | [⚙][mattermost-omnibus] | [⤓][mattermost-charts] | [⤓][mattermost-charts] | [⤓](../user/project/integrations/mattermost_slash_commands.md#manual-configuration), [⤓](../user/project/integrations/mattermost.html) | ❌ | ❌ | CE & EE | +| [MinIO](#minio) | Object storage service | [⤓][minio-omnibus] | [✅][minio-charts] | [✅][minio-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#storage-architecture) | ❌ | [⚙][minio-gdk] | CE & EE | +| [Runner](#gitlab-runner) | Executes GitLab CI jobs | [⤓][runner-omnibus] | [✅][runner-charts] | [⚙][runner-charts] | [✅](../user/gitlab_com/index.md#shared-runners) | [⚙][runner-source] | [⚙][runner-gdk] | CE & EE | +| [Database Migrations](#database-migrations) | Database migrations | [✅][database-migrations-omnibus] | [✅][database-migrations-charts] | [✅][database-migrations-charts] | ✅ | [⚙][database-migrations-source] | ✅ | CE & EE | +| [Certificate Management](#certificate-management) | TLS Settings, Let's Encrypt | [✅][certificate-management-omnibus] | [✅][certificate-management-charts] | [⚙][certificate-management-charts] | [✅](https://about.gitlab.com/handbook/engineering/infrastructure/production-architecture/#secrets-management) | [⚙][certificate-management-source] | [⚙][certificate-management-gdk] | CE & EE | +| [GitLab Geo Node](#gitlab-geo) | Geographically distributed GitLab nodes | [⚙][geo-omnibus] | [❌][geo-charts] | [❌][geo-charts] | ✅ | ❌ | [⚙][geo-gdk] | EE Only | +| [LDAP Authentication](#ldap-authentication) | Authenticate users against centralized LDAP directory | [⤓][ldap-omnibus] | [⤓][ldap-charts] | [⤓][ldap-charts] | [❌](https://about.gitlab.com/pricing/#gitlab-com) | [⤓][gitlab-yml] | [⤓][ldap-gdk] | CE & EE | +| [Outbound email (SMTP)](#outbound-email) | Send email messages to users | [⤓][outbound-email-omnibus] | [⤓][outbound-email-charts] | [⤓][outbound-email-charts] | [✅](../user/gitlab_com/index.md#mail-configuration) | [⤓][gitlab-yml] | [⤓][gitlab-yml] | CE & EE | +| [Inbound email (SMTP)](#inbound-email) | Receive messages to update issues | [⤓][inbound-email-omnibus] | [⤓][inbound-email-charts] | [⤓][inbound-email-charts] | [✅](../user/gitlab_com/index.md#mail-configuration) | [⤓][gitlab-yml] | [⤓][gitlab-yml] | CE & EE | +| [ElasticSearch](#elasticsearch) | Improved search within GitLab | [⤓][elasticsearch-omnibus] | [⤓][elasticsearch-charts] | [⤓][elasticsearch-charts] | [❌](https://gitlab.com/groups/gitlab-org/-/epics/153) | [⤓][elasticsearch-source] | [⤓][elasticsearch-gdk] | EE Only | +| [Sentry integration](#sentry) | Error tracking for deployed apps | [⤓][sentry-integration] | [⤓][sentry-integration] | [⤓][sentry-integration] | [⤓][sentry-integration] | [⤓][sentry-integration] | [⤓][sentry-integration] | CE & EE | +| [Jaeger integration](#jaeger) | Distributed tracing for deployed apps | [⤓][jaeger-integration] | [⤓][jaeger-integration] | [⤓][jaeger-integration] | [⤓][jaeger-integration] | [⤓][jaeger-integration] | [⤓][jaeger-integration] | EE Only | +| [Kubernetes cluster apps](#kubernetes-cluster-apps) | Deploy [Helm](https://docs.helm.sh/), [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/), [Cert-Manager](https://docs.cert-manager.io/en/latest/), [Prometheus](https://prometheus.io/docs/introduction/overview/), a [Runner](https://docs.gitlab.com/runner/), [JupyterHub](http://jupyter.org/), [Knative](https://cloud.google.com/knative) to a cluster | [⤓][managed-k8s-apps] | [⤓][managed-k8s-apps] | [⤓][managed-k8s-apps] | [⤓][managed-k8s-apps] | [⤓][managed-k8s-apps] | [⤓][managed-k8s-apps] | CE & EE | ### Component details @@ -164,13 +164,14 @@ GitLab can be considered to have two layers from a process perspective: - [Project page](https://github.com/prometheus/alertmanager/blob/master/README.md) - Configuration: [Omnibus][alertmanager-omnibus], [Charts][alertmanager-charts] - Layer: Monitoring +- Process: `alertmanager` [Alert manager](https://prometheus.io/docs/alerting/alertmanager/) is a tool provided by Prometheus that _"handles alerts sent by client applications such as the Prometheus server. It takes care of deduplicating, grouping, and routing them to the correct receiver integration such as email, PagerDuty, or OpsGenie. It also takes care of silencing and inhibition of alerts."_ You can read more in [issue gitlab-ce#45740](https://gitlab.com/gitlab-org/gitlab-ce/issues/45740) about what we will be alerting on. #### Certificate management - Project page: [Omnibus](https://github.com/certbot/certbot/blob/master/README.rst), [Charts](https://github.com/jetstack/cert-manager/blob/master/README.md) -- Configuration: [Omnibus][certificate-management-omnibus], [Charts][certificate-management-charts] +- Configuration: [Omnibus][certificate-management-omnibus], [Charts][certificate-management-charts], [Source][certificate-management-source], [GDK][certificate-management-gdk] - Layer: Core Service (Processor) #### Consul @@ -183,13 +184,13 @@ Consul is a tool for service discovery and configuration. Consul is distributed, #### Database migrations -- Configuration: [Omnibus][registry-omnibus], [Charts][registry-charts] +- Configuration: [Omnibus][registry-omnibus], [Charts][registry-charts], [Source][database-migrations-source] - Layer: Core Service (Data) #### Elasticsearch - [Project page](https://github.com/elastic/elasticsearch/blob/master/README.textile) -- Configuration: [Omnibus][elasticsearch-omnibus], [Charts][elasticsearch-charts] +- Configuration: [Omnibus][elasticsearch-omnibus], [Charts][elasticsearch-charts], [Source][elasticsearch-source], [GDK][elasticsearch-gdk] - Layer: Core Service (Data) Elasticsearch is a distributed RESTful search engine built for the cloud. @@ -197,14 +198,15 @@ Elasticsearch is a distributed RESTful search engine built for the cloud. #### Gitaly - [Project page](https://gitlab.com/gitlab-org/gitaly/blob/master/README.md) -- Configuration: [Omnibus][gitaly-omnibus], [Charts][gitaly-charts] +- Configuration: [Omnibus][gitaly-omnibus], [Charts][gitaly-charts], [Source][gitaly-source] - Layer: Core Service (Data) +- Process: `gitaly` Gitaly is a service designed by GitLab to remove our need for NFS for Git storage in distributed deployments of GitLab (think GitLab.com or High Availability Deployments). As of 11.3.0, this service handles all Git level access in GitLab. You can read more about the project [in the project's readme](https://gitlab.com/gitlab-org/gitaly). #### Gitlab Geo -- Configuration: [Omnibus][geo-omnibus], [Charts][geo-charts] +- Configuration: [Omnibus][geo-omnibus], [Charts][geo-charts], [GDK][geo-gdk] - Layer: Core Service (Processor) #### Gitlab Monitor @@ -212,12 +214,13 @@ Gitaly is a service designed by GitLab to remove our need for NFS for Git storag - [Project page](https://gitlab.com/gitlab-org/gitlab-monitor) - Configuration: [Omnibus][gitlab-monitor-omnibus], [Charts][gitlab-monitor-charts] - Layer: Monitoring +- Process: `gitlab-monitor` GitLab Monitor is a process designed in house that allows us to export metrics about GitLab application internals to Prometheus. You can read more [in the project's readme](https://gitlab.com/gitlab-org/gitlab-monitor). #### Gitlab Pages -- Configuration: [Omnibus][pages-omnibus], [Charts][pages-charts] +- Configuration: [Omnibus][pages-omnibus], [Charts][pages-charts], [Source][pages-source], [GDK][pages-gdk] - Layer: Core Service (Processor) GitLab Pages is a feature that allows you to publish static websites directly from a repository in GitLab. @@ -227,7 +230,7 @@ You can use it either for personal or business websites, such as portfolios, doc #### Gitlab Runner - [Project page](https://gitlab.com/gitlab-org/gitlab-runner/blob/master/README.md) -- Configuration: [Omnibus][runner-omnibus], [Charts][runner-charts] +- Configuration: [Omnibus][runner-omnibus], [Charts][runner-charts], [Source][runner-source], [GDK][runner-gdk] - Layer: Core Service (Processor) GitLab Runner runs tests and sends the results to GitLab. @@ -237,7 +240,7 @@ GitLab CI is the open-source continuous integration service included with GitLab #### Gitlab Shell - [Project page](https://gitlab.com/gitlab-org/gitlab-shell/blob/master/README.md) -- Configuration: [Omnibus][shell-omnibus], [Charts][shell-charts] +- Configuration: [Omnibus][shell-omnibus], [Charts][shell-charts], [Source][shell-source], [GDK][gitlab-yml] - Layer: Core Service (Processor) [GitLab Shell](https://gitlab.com/gitlab-org/gitlab-shell) is a program designed at GitLab to handle ssh-based `git` sessions, and modifies the list of authorized keys. GitLab Shell is not a Unix shell nor a replacement for Bash or Zsh. @@ -245,8 +248,9 @@ GitLab CI is the open-source continuous integration service included with GitLab #### Gitlab Workhorse - [Project page](https://gitlab.com/gitlab-org/gitlab-workhorse/blob/master/README.md) -- Configuration: [Omnibus][gitlab-workhorse-omnibus], [Charts][gitlab-workhorse-charts] +- Configuration: [Omnibus][gitlab-workhorse-omnibus], [Charts][gitlab-workhorse-charts], [Source][workhorse-source] - Layer: Core Service (Processor) +- Process: `gitlab-workhorse` [GitLab Workhorse](https://gitlab.com/gitlab-org/gitlab-workhorse) is a program designed at GitLab to help alleviate pressure from Unicorn. You can read more about the [historical reasons for developing](https://about.gitlab.com/2016/04/12/a-brief-history-of-gitlab-workhorse/). It's designed to act as a smart reverse proxy to help speed up GitLab as a whole. @@ -261,7 +265,7 @@ Grafana is an open source, feature rich metrics dashboard and graph editor for G #### Jaeger - [Project page](https://github.com/jaegertracing/jaeger/blob/master/README.md) -- Configuration: [Omnibus][jaeger-omnibus], [Charts][jaeger-charts] +- Configuration: [Omnibus][jaeger-omnibus], [Charts][jaeger-charts], [Source][jaeger-source], [GDK][jaeger-gdk] - Layer: Monitoring Jaeger, inspired by Dapper and OpenZipkin, is a distributed tracing system. It can be used for monitoring microservices-based distributed systems. @@ -271,6 +275,7 @@ Jaeger, inspired by Dapper and OpenZipkin, is a distributed tracing system. It c - [Project page](https://github.com/logrotate/logrotate/blob/master/README.md) - Configuration: [Omnibus](https://docs.gitlab.com/omnibus/settings/logs.html#logrotate) - Layer: Core Service +- Process: `logrotate` GitLab is comprised of a large number of services that all log. We started bundling our own logrotate as of 7.4 to make sure we were logging responsibly. This is just a packaged version of the common open source offering. @@ -285,7 +290,7 @@ Mattermost is an open source, private cloud, Slack-alternative from https://matt #### MinIO - [Project page](https://github.com/minio/minio/blob/master/README.md) -- Configuration: [Omnibus][minio-omnibus], [Charts][minio-charts] +- Configuration: [Omnibus][minio-omnibus], [Charts][minio-charts], [GDK][minio-gdk] - Layer: Core Service (Data) MinIO is an object storage server released under Apache License v2.0. It is compatible with Amazon S3 cloud storage service. It is best suited for storing unstructured data such as photos, videos, log files, backups and container / VM images. Size of an object can range from a few KBs to a maximum of 5TB. @@ -293,8 +298,9 @@ MinIO is an object storage server released under Apache License v2.0. It is comp #### NGINX - Project page: [Omnibus](https://github.com/nginx/nginx), [Charts](https://github.com/kubernetes/ingress-nginx/blob/master/README.md) -- Configuration: [Omnibus][nginx-omnibus], [Charts][nginx-charts] +- Configuration: [Omnibus][nginx-omnibus], [Charts][nginx-charts], [Source][nginx-source] - Layer: Core Service (Processor) +- Process: `nginx` Nginx as an ingress port for all HTTP requests and routes them to the approriate sub-systems within GitLab. We are bundling an unmodified version of the popular open source webserver. @@ -303,6 +309,7 @@ Nginx as an ingress port for all HTTP requests and routes them to the approriate - [Project page](https://github.com/prometheus/node_exporter/blob/master/README.md) - Configuration: [Omnibus][node-exporter-omnibus], [Charts][node-exporter-charts] - Layer: Monitoring +- Process: `node-exporter` [Node Exporter](https://github.com/prometheus/node_exporter) is a Prometheus tool that gives us metrics on the underlying machine (think CPU/Disk/Load). It's just a packaged version of the common open source offering from the Prometheus project. @@ -325,8 +332,9 @@ Prometheus exporter for PgBouncer. Exports metrics at 9127/metrics. #### Postgresql - [Project page](https://github.com/postgres/postgres/blob/master/README) -- Configuration: [Omnibus][postgres-omnibus], [Charts][postgres-charts] +- Configuration: [Omnibus][postgres-omnibus], [Charts][postgres-charts], [Source][postgres-source] - Layer: Core Service (Data) +- Process: `postgresql` GitLab packages the popular Database to provide storage for Application meta data and user information. @@ -335,6 +343,7 @@ GitLab packages the popular Database to provide storage for Application meta dat - [Project page](https://github.com/wrouesnel/postgres_exporter/blob/master/README.md) - Configuration: [Omnibus][postgres-exporter-omnibus], [Charts][postgres-exporter-charts] - Layer: Monitoring +- Process: `postgres-exporter` [Postgres-exporter](https://github.com/wrouesnel/postgres_exporter) is the community provided Prometheus exporter that will deliver data about Postgres to Prometheus for use in Grafana Dashboards. @@ -343,14 +352,16 @@ GitLab packages the popular Database to provide storage for Application meta dat - [Project page](https://github.com/prometheus/prometheus/blob/master/README.md) - Configuration: [Omnibus][prometheus-omnibus], [Charts][prometheus-charts] - Layer: Monitoring +- Process: `prometheus` Prometheus is a time-series tool that helps GitLab administrators expose metrics about the individual processes used to provide GitLab the service. #### Redis - [Project page](https://github.com/antirez/redis/blob/unstable/README.md) -- Configuration: [Omnibus][redis-omnibus], [Charts][redis-charts] +- Configuration: [Omnibus][redis-omnibus], [Charts][redis-charts], [Source][redis-source] - Layer: Core Service (Data) +- Process: `redis` Redis is packaged to provide a place to store: @@ -363,13 +374,14 @@ Redis is packaged to provide a place to store: - [Project page](https://github.com/oliver006/redis_exporter/blob/master/README.md) - Configuration: [Omnibus][redis-exporter-omnibus], [Charts][redis-exporter-charts] - Layer: Monitoring +- Process: `redis-exporter` [Redis Exporter](https://github.com/oliver006/redis_exporter) is designed to give specific metrics about the Redis process to Prometheus so that we can graph these metrics in Grafana. #### Registry - [Project page](https://github.com/docker/distribution/blob/master/README.md) -- Configuration: [Omnibus][registry-omnibus], [Charts][registry-charts] +- Configuration: [Omnibus][registry-omnibus], [Charts][registry-charts], [Source][registry-source], [GDK][registry-gdk] - Layer: Core Service (Processor) The registry is what users use to store their own Docker images. The bundled @@ -385,7 +397,7 @@ An external registry can also be configured to use GitLab as an auth endpoint. #### Sentry - [Project page](https://github.com/getsentry/sentry/blob/master/README.rst) -- Configuration: [Omnibus][sentry-omnibus], [Charts][sentry-charts] +- Configuration: [Omnibus][sentry-omnibus], [Charts][sentry-charts], [Source][gitlab-yml], [GDK][gitlab-yml] - Layer: Monitoring Sentry fundamentally is a service that helps you monitor and fix crashes in realtime. The server is in Python, but it contains a full API for sending events from any language, in any application. @@ -393,40 +405,42 @@ Sentry fundamentally is a service that helps you monitor and fix crashes in real #### Sidekiq - [Project page](https://github.com/mperham/sidekiq/blob/master/README.md) -- Configuration: [Omnibus][sidekiq-omnibus], [Charts][sidekiq-charts] +- Configuration: [Omnibus][sidekiq-omnibus], [Charts][sidekiq-charts], [Source][gitlab-yml], [GDK][gitlab-yml] - Layer: Core Service (Processor) +- Process: `sidekiq` Sidekiq is a Ruby background job processor that pulls jobs from the redis queue and processes them. Background jobs allow GitLab to provide a faster request/response cycle by moving work into the background. #### Unicorn - [Project page](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/README.md) -- Configuration: [Omnibus][unicorn-omnibus], [Charts][unicorn-charts] +- Configuration: [Omnibus][unicorn-omnibus], [Charts][unicorn-charts], [Source][unicorn-source], [GDK][gitlab-yml] - Layer: Core Service (Processor) +- Process: `unicorn` [Unicorn](https://bogomips.org/unicorn/) is a Ruby application server that is used to run the core Rails Application that provides the user facing features in GitLab. Often process output you will see this as `bundle` or `config.ru` depending on the GitLab version. #### LDAP Authentication -- Configuration: [Omnibus][ldap-omnibus], [Charts][ldap-charts] +- Configuration: [Omnibus][ldap-omnibus], [Charts][ldap-charts], [Source][gitlab-yml], [GDK][ldap-gdk] - Layer: Core Service (Processor) #### Outbound Email -- Configuration: [Omnibus][outbound-email-omnibus], [Charts][outbound-email-charts] +- Configuration: [Omnibus][outbound-email-omnibus], [Charts][outbound-email-charts], [Source][gitlab-yml], [GDK][gitlab-yml] - Layer: Core Service (Processor) #### Inbound Email -- Configuration: [Omnibus][inbound-email-omnibus], [Charts][inbound-email-charts] +- Configuration: [Omnibus][inbound-email-omnibus], [Charts][inbound-email-charts], [Source][gitlab-yml], [GDK][gitlab-yml] - Layer: Core Service (Processor) #### Kubernetes Cluster Apps -- Configuration: [Omnibus][managed-k8s-apps], [Charts][managed-k8s-apps] +- Configuration: [Omnibus][managed-k8s-apps], [Charts][managed-k8s-apps], [Source][managed-k8s-apps], [GDK][managed-k8s-apps] - Layer: Core Service (Processor) -GitLab provides [GitLab Managed Apps](https://docs.gitlab.com/ee/user/project/clusters/#installing-applications), a one-click install for various applications which can be added directly to your configured cluster. These applications are needed for Review Apps and deployments when using Auto DevOps. You can install them after you create a cluster. +GitLab provides [GitLab Managed Apps](../user/project/clusters/index.md#installing-applications), a one-click install for various applications which can be added directly to your configured cluster. These applications are needed for Review Apps and deployments when using Auto DevOps. You can install them after you create a cluster. ## GitLab by Request Type @@ -595,68 +609,92 @@ We've also detailed [our architecture of GitLab.com](https://about.gitlab.com/ha [alertmanager-omnibus]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template [alertmanager-charts]: https://github.com/helm/charts/tree/master/stable/prometheus -[nginx-omnibus]: https://docs.gitlab.com/omnibus/settings/nginx.html -[nginx-charts]: https://docs.gitlab.com/charts/charts/nginx/index.html +[nginx-omnibus]: https://docs.gitlab.com/omnibus/settings/ +[nginx-charts]: https://docs.gitlab.com/charts/charts/nginx/ +[nginx-source]: ../install/installation.md#9-nginx [unicorn-omnibus]: https://docs.gitlab.com/omnibus/settings/unicorn.html -[unicorn-charts]: https://docs.gitlab.com/charts/charts/gitlab/unicorn/index.html +[unicorn-charts]: https://docs.gitlab.com/charts/charts/gitlab/unicorn/ +[unicorn-source]: ../install/installation.md#configure-it +[gitlab-yml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/config/gitlab.yml.example [sidekiq-omnibus]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template -[sidekiq-charts]: https://docs.gitlab.com/charts/charts/gitlab/sidekiq/index.html -[gitaly-omnibus]: https://docs.gitlab.com/ee/administration/gitaly/ -[gitaly-charts]: https://docs.gitlab.com/charts/charts/gitlab/gitaly/index.html +[sidekiq-charts]: https://docs.gitlab.com/charts/charts/gitlab/sidekiq/ +[gitaly-omnibus]: ../administration/gitaly/index.md +[gitaly-charts]: https://docs.gitlab.com/charts/charts/gitlab/gitaly/ +[gitaly-source]: ../install/installation.md#install-gitaly [workhorse-omnibus]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template -[workhorse-charts]: https://docs.gitlab.com/charts/charts/gitlab/unicorn/index.html +[workhorse-charts]: https://docs.gitlab.com/charts/charts/gitlab/unicorn/ +[workhorse-source]: ../install/installation.md#install-gitlab-workhorse [shell-omnibus]: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template -[shell-charts]: https://docs.gitlab.com/charts/charts/gitlab/gitlab-shell/index.html -[pages-omnibus]: https://docs.gitlab.com/ee/administration/pages/ +[shell-charts]: https://docs.gitlab.com/charts/charts/gitlab/gitlab-shell/ +[shell-source]: ../install/installation.md#install-gitlab-shell +[pages-omnibus]: ../administration/pages/index.md [pages-charts]: https://gitlab.com/charts/gitlab/issues/37 -[registry-omnibus]: https://docs.gitlab.com/ee/administration/container_registry.html#container-registry-domain-configuration -[registry-charts]: https://docs.gitlab.com/charts/charts/registry/index.html +[pages-source]: ../install/installation.md#install-gitlab-pages +[pages-gdk]: https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/pages.md +[registry-omnibus]: ../administration/container_registry.md#container-registry-domain-configuration +[registry-charts]: https://docs.gitlab.com/charts/charts/registry/ +[registry-source]: ../administration/container_registry.md#enable-the-container-registry +[registry-gdk]: https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/registry.md [redis-omnibus]: https://docs.gitlab.com/omnibus/settings/redis.html -[redis-charts]: https://docs.gitlab.com/charts/charts/redis/index.html +[redis-charts]: https://docs.gitlab.com/charts/charts/redis/ +[redis-source]: ../install/installation.md#7-redis [postgres-omnibus]: https://docs.gitlab.com/omnibus/settings/database.html [postgres-charts]: https://github.com/helm/charts/tree/master/stable/postgresql -[pgbouncer-omnibus]: https://docs.gitlab.com/ee/administration/high_availability/pgbouncer.html +[postgres-source]: ../install/installation.md#6-database +[pgbouncer-omnibus]: ../administration/high_availability/pgbouncer.md [pgbouncer-charts]: https://docs.gitlab.com/charts/installation/deployment.html#postgresql -[consul-omnibus]: https://docs.gitlab.com/ee/administration/high_availability/consul.html +[consul-omnibus]: ../administration/high_availability/consul.md [consul-charts]: https://docs.gitlab.com/charts/installation/deployment.html#postgresql -[prometheus-omnibus]: https://docs.gitlab.com/ee/administration/monitoring/prometheus/ +[prometheus-omnibus]: ../administration/monitoring/prometheus/index.md [prometheus-charts]: https://github.com/helm/charts/tree/master/stable/prometheus -[grafana-omnibus]: https://docs.gitlab.com/ee/administration/monitoring/performance/grafana_configuration.html +[grafana-omnibus]: ../administration/monitoring/performance/grafana_configuration.md [grafana-charts]: https://github.com/helm/charts/tree/master/stable/grafana [sentry-omnibus]: https://docs.gitlab.com/omnibus/settings/configuration.html#error-reporting-and-logging-with-sentry [sentry-charts]: https://gitlab.com/charts/gitlab/issues/1319 [jaeger-omnibus]: https://gitlab.com/gitlab-org/omnibus-gitlab/issues/4104 [jaeger-charts]: https://gitlab.com/charts/gitlab/issues/1320 -[redis-exporter-omnibus]: https://docs.gitlab.com/ee/administration/monitoring/prometheus/redis_exporter.html -[redis-exporter-charts]: https://docs.gitlab.com/charts/charts/redis/index.html -[postgres-exporter-omnibus]: https://docs.gitlab.com/ee/administration/monitoring/prometheus/postgres_exporter.html +[jaeger-source]: ../development/distributed_tracing.md#enabling-distributed-tracing +[jaeger-gdk]: ../development/distributed_tracing.html#using-jaeger-in-the-gitlab-development-kit +[redis-exporter-omnibus]: ../administration/monitoring/prometheus/redis_exporter.md +[redis-exporter-charts]: https://docs.gitlab.com/charts/charts/redis/ +[postgres-exporter-omnibus]: ../administration/monitoring/prometheus/postgres_exporter.md [postgres-exporter-charts]: https://github.com/helm/charts/tree/master/stable/postgresql -[pgbouncer-exporter-omnibus]: https://docs.gitlab.com/ee/administration/monitoring/prometheus/pgbouncer_exporter.html +[pgbouncer-exporter-omnibus]: ../administration/monitoring/prometheus/pgbouncer_exporter.md [pgbouncer-exporter-charts]: https://docs.gitlab.com/charts/installation/deployment.html#postgresql -[gitlab-monitor-omnibus]: https://docs.gitlab.com/ee/administration/monitoring/prometheus/gitlab_monitor_exporter.html +[gitlab-monitor-omnibus]: ../administration/monitoring/prometheus/gitlab_monitor_exporter.md [gitab-monitor-charts]: https://gitlab.com/charts/gitlab/issues/319 -[node-exporter-omnibus]: https://docs.gitlab.com/ee/administration/monitoring/prometheus/node_exporter.html +[node-exporter-omnibus]: ../administration/monitoring/prometheus/node_exporter.md [node-exporter-charts]: https://gitlab.com/charts/gitlab/issues/1332 [mattermost-omnibus]: https://docs.gitlab.com/omnibus/gitlab-mattermost/ [mattermost-charts]: https://docs.mattermost.com/install/install-mmte-helm-gitlab-helm.html [minio-omnibus]: https://min.io/download -[minio-charts]: https://docs.gitlab.com/charts/charts/minio/index.html +[minio-charts]: https://docs.gitlab.com/charts/charts/minio/ +[minio-gdk]: https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/object_storage.md [runner-omnibus]: https://docs.gitlab.com/runner/ [runner-charts]: https://docs.gitlab.com/runner/install/kubernetes.html +[runner-source]: https://docs.gitlab.com/runner/ +[runner-gdk]: https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/runner.md [database-migrations-omnibus]: https://docs.gitlab.com/omnibus/settings/database.html#disabling-automatic-database-migration -[database-migrations-charts]: https://docs.gitlab.com/charts/charts/gitlab/migrations/index.html +[database-migrations-charts]: https://docs.gitlab.com/charts/charts/gitlab/migrations/ +[database-migrations-source]: ../update/upgrading_from_source.md#13-install-libs-migrations-etc [certificate-management-omnibus]: https://docs.gitlab.com/omnibus/settings/ssl.html [certificate-management-charts]: https://docs.gitlab.com/charts/installation/tls.html -[geo-omnibus]: https://docs.gitlab.com/ee/administration/geo/replication/index.html#setup-instructions +[certificate-management-source]: ../install/installation.md#using-https +[certificate-management-gdk]: https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/https.md +[geo-omnibus]: ../administration/geo/replication/index.md#setup-instructions [geo-charts]: https://gitlab.com/charts/gitlab/issues/8 -[ldap-omnibus]: https://docs.gitlab.com/ee/administration/auth/ldap.html +[geo-gdk]: https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/geo.md +[ldap-omnibus]: ../administration/auth/ldap.md [ldap-charts]: https://docs.gitlab.com/charts/charts/globals.html#ldap +[ldap-gdk]: https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/ldap.md [outbound-email-omnibus]: https://docs.gitlab.com/omnibus/settings/smtp.html [outbound-email-charts]: https://docs.gitlab.com/charts/installation/command-line-options.html#outgoing-email-configuration -[inbound-email-omnibus]: https://docs.gitlab.com/ee/administration/incoming_email.html +[inbound-email-omnibus]: ../administration/incoming_email.md [inbound-email-charts]: https://docs.gitlab.com/charts/installation/command-line-options.html#incoming-email-configuration -[elasticsearch-omnibus]: https://docs.gitlab.com/ee/integration/elasticsearch.html -[elasticsearch-charts]: https://docs.gitlab.com/ee/integration/elasticsearch.html -[sentry-integration]: https://docs.gitlab.com/ee/user/project/operations/error_tracking.html -[jaeger-integration]: https://docs.gitlab.com/ee/user/project/operations/tracing.html -[managed-k8s-apps]: https://docs.gitlab.com/ee/user/project/clusters/#installing-applications +[elasticsearch-omnibus]: ../integration/elasticsearch.md +[elasticsearch-charts]: ../integration/elasticsearch.md +[elasticsearch-source]: ../integration/elasticsearch.md +[elasticsearch-gdk]: https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/elasticsearch.md +[sentry-integration]: ../user/project/operations/error_tracking.md +[jaeger-integration]: ../user/project/operations/tracing.md +[managed-k8s-apps]: ../user/project/clusters/index.md#installing-applications diff --git a/doc/development/changelog.md b/doc/development/changelog.md index 273a7fceaf5..45b3d5a23a1 100644 --- a/doc/development/changelog.md +++ b/doc/development/changelog.md @@ -35,6 +35,7 @@ the `author` field. GitLab team members **should not**. - Any user-facing change **should** have a changelog entry. Example: "GitLab now uses system fonts for all text." +- Any change behind a feature flag **should not** have a changelog entry. The entry should be added [in the merge request removing the feature flags](https://docs.gitlab.com/ee/development/feature_flags.html#developing-with-feature-flags). - A fix for a regression introduced and then fixed in the same release (i.e., fixing a bug introduced during a monthly release candidate) **should not** have a changelog entry. diff --git a/doc/development/code_review.md b/doc/development/code_review.md index c4e5995714d..29e2aa1a581 100644 --- a/doc/development/code_review.md +++ b/doc/development/code_review.md @@ -27,7 +27,7 @@ Depending on the areas your merge request touches, it must be **approved** by on or more [maintainers](https://about.gitlab.com/handbook/engineering/workflow/code-review/#maintainer): For approvals, we use the approval functionality found in the merge request -widget. Reviewers can add their approval by [approving additionally](https://docs.gitlab.com/ee/user/project/merge_requests/merge_request_approvals.html#adding-or-removing-an-approval). +widget. Reviewers can add their approval by [approving additionally](../user/project/merge_requests/merge_request_approvals.md#adding-or-removing-an-approval). Getting your merge request **merged** also requires a maintainer. If it requires more than one approval, the last maintainer to review and approve it will also merge it. @@ -43,7 +43,7 @@ It picks reviewers and maintainers from the list at the [engineering projects](https://about.gitlab.com/handbook/engineering/projects/) page, with these behaviours: -1. It will not pick people whose [GitLab status](../user/profile/#current-status) +1. It will not pick people whose [GitLab status](../user/profile/index.md#current-status) contains the string 'OOO'. 2. [Trainee maintainers](https://about.gitlab.com/handbook/engineering/workflow/code-review/#trainee-maintainer) are three times as likely to be picked as other reviewers. @@ -152,7 +152,7 @@ required approvers. Maintainers must check before merging if the merge request is introducing new vulnerabilities, by inspecting the list in the Merge Request [Security -Widget](https://docs.gitlab.com/ee/user/project/merge_requests/#security-reports-ultimate). +Widget](../user/project/merge_requests/index.md#security-reports-ultimate). When in doubt, a [Security Engineer][team] can be involved. The list of detected vulnerabilities must be either empty or containing: @@ -286,7 +286,7 @@ experience, refactors the existing code). Then: author has already set this option or if the merge request clearly contains a messy commit history that is intended to be squashed. -[squash-and-merge]: https://docs.gitlab.com/ee/user/project/merge_requests/squash_and_merge.html#squash-and-merge +[squash-and-merge]: ../user/project/merge_requests/squash_and_merge.md#squash-and-merge ### The right balance @@ -319,7 +319,7 @@ reviewee. GitLab is used in a lot of places. Many users use our [Omnibus packages](https://about.gitlab.com/installation/), but some use the [Docker images](https://docs.gitlab.com/omnibus/docker/), some are -[installed from source](https://docs.gitlab.com/ce/install/installation.html), +[installed from source](../install/installation.md), and there are other installation methods available. GitLab.com itself is a large Enterprise Edition instance. This has some implications: diff --git a/doc/development/contributing/index.md b/doc/development/contributing/index.md index 8b1d014e101..59cf5014da4 100644 --- a/doc/development/contributing/index.md +++ b/doc/development/contributing/index.md @@ -87,7 +87,7 @@ Sometimes style guides will be followed but the code will lack structural integr GitLab will do its best to review community contributions as quickly as possible. Specially appointed developers review community contributions daily. You may take a look at the [team page](https://about.gitlab.com/team/) for the merge request coach who specializes in the type of code you have written and mention them in the merge request. For example, if you have written some JavaScript in your code then you should mention the frontend merge request coach. If your code has multiple disciplines you may mention multiple merge request coaches. -GitLab receives a lot of community contributions, so if your code has not been reviewed within 4 days of its initial submission feel free to re-mention the appropriate merge request coach. +GitLab receives a lot of community contributions, so if your code has not been reviewed within two days (excluding weekend and public holidays) of its initial submission feel free to re-mention the appropriate merge request coach. When submitting code to GitLab, you may feel that your contribution requires the aid of an external library. If your code includes an external library please provide a link to the library, as well as reasons for including it. diff --git a/doc/development/contributing/issue_workflow.md b/doc/development/contributing/issue_workflow.md index 0e1ab8663ed..e3a1dc711fd 100644 --- a/doc/development/contributing/issue_workflow.md +++ b/doc/development/contributing/issue_workflow.md @@ -7,7 +7,7 @@ scheduling into milestones. Labelling is a task for everyone. Most issues will have labels for at least one of the following: - Type: ~feature, ~bug, ~customer, etc. -- Subject: ~wiki, ~"container registry", ~ldap, ~api, ~frontend, etc. +- Subject: ~wiki, ~"Container Registry", ~ldap, ~api, ~frontend, etc. - Team: ~Plan, ~Manage, ~Quality, etc. - Stage: ~"devops:plan", ~"devops:create", etc. - Release Scoping: ~Deliverable, ~Stretch, ~"Next Patch Release" @@ -44,7 +44,7 @@ Subject labels are labels that define what area or feature of GitLab this issue hits. They are not always necessary, but very convenient. Examples of subject labels are ~wiki, ~ldap, ~api, -~issues, ~"merge requests", ~labels, and ~"container registry". +~issues, ~"merge requests", ~labels, and ~"Container Registry". If you are an expert in a particular area, it makes it easier to find issues to work on. You can also subscribe to those labels to receive an email each time an @@ -92,20 +92,21 @@ Stage labels specify which [DevOps stage][devops-stages] the issue belongs to. The current stage labels are: -- ~"devops:manage" -- ~"devops:plan" -- ~"devops:create" -- ~"devops:verify" -- ~"devops:package" -- ~"devops:release" -- ~"devops:configure" -- ~"devops:monitor" -- ~"devops:secure" -- ~"devops:defend" -- ~"devops:enablement" - -These labels should be mutually exclusive. If an issue belongs to multiple -stages, the most relevant should be used. +- ~"devops::manage" +- ~"devops::plan" +- ~"devops::create" +- ~"devops::verify" +- ~"devops::package" +- ~"devops::release" +- ~"devops::configure" +- ~"devops::monitor" +- ~"devops::secure" +- ~"devops::defend" +- ~"devops::growth" +- ~"devops::enablement" + +These labels are [scoped labels](../../user/project/labels.md#scoped-labels-premium) +and thus are mutually exclusive. They differ from the [Team labels](#team-labels) because teams may work on issues outside their stage. @@ -121,6 +122,25 @@ The Stage labels are used to generate the [direction pages][direction-pages] aut [devops-stages]: https://about.gitlab.com/direction/#devops-stages [direction-pages]: https://about.gitlab.com/direction/ +## Group labels + +Group labels specify which [groups][structure-groups] the issue belongs to. + +Examples include: + +- ~"group::control" +- ~"group::editor" + +These labels are [scoped labels](../../user/project/labels.md#scoped-labels-premium) +and thus are mutually exclusive. + +Groups are nested beneath a particular stage, so only one stage label and one group label +can be applied to a single issue. You can find the groups listed in the +[Product Categories pages][product-categories]. + +[structure-groups]: https://about.gitlab.com/company/team/structure/#groups +[product-categories]: https://about.gitlab.com/handbook/product/categories/ + ## Release Scoping labels Release Scoping labels help us clearly communicate expectations of the work for the diff --git a/doc/development/distributed_tracing.md b/doc/development/distributed_tracing.md index 038e3de10d7..bfce7488a8d 100644 --- a/doc/development/distributed_tracing.md +++ b/doc/development/distributed_tracing.md @@ -34,7 +34,7 @@ GITLAB_TRACING=opentracing://<driver>?<param_name>=<param_value>&<param_name_2>= In this example, we have the following hypothetical values: - `driver`: the driver. [GitLab supports - `jaeger`](https://docs.gitlab.com/ee/user/project/operations/tracing.html). In future, other + `jaeger`](../user/project/operations/tracing.md). In future, other tracing implementations may also be supported. - `param_name`, `param_value`: these are driver specific configuration values. Configuration parameters for Jaeger are documented [further on in this diff --git a/doc/development/documentation/feature-change-workflow.md b/doc/development/documentation/feature-change-workflow.md index 1f68b6a6a70..ca29353ecbe 100644 --- a/doc/development/documentation/feature-change-workflow.md +++ b/doc/development/documentation/feature-change-workflow.md @@ -86,7 +86,7 @@ Everyone is encouraged to draft the requirements in the issue, but a product man do the following: - When the issue is assigned a release milestone, review and update the Documentation details. -- By the kickoff, finalizie the Documentation details. +- By the kickoff, finalize the Documentation details. ### Developer and maintainer roles @@ -117,7 +117,7 @@ Follow the process below unless otherwise agreed with the product manager and te #### Reviews and merging -All reviewers can help ensure accuracy, clarity, completeness, and adherence to the plans in the issue, as well as the [Documentation Guidelines](https://docs.gitlab.com/ee/development/documentation/) and [Style Guide](https://docs.gitlab.com/ee/development/documentation/styleguide.html). +All reviewers can help ensure accuracy, clarity, completeness, and adherence to the plans in the issue, as well as the [Documentation Guidelines](index.md) and [Style Guide](styleguide.md). - **Prior to merging**, documentation changes committed by the developer must be reviewed by: @@ -136,7 +136,7 @@ All reviewers can help ensure accuracy, clarity, completeness, and adherence to 1. **The maintainer** who is assigned to merge the MR, to verify clarity, completeness, and quality, to the best of their ability. - Upon merging, if a technical writer review has not been performed and there is not yet a linked issue for a follow-up review, the maintainer should [create an issue using the Doc Review template](https://gitlab.com/gitlab-org/gitlab-ce/issues/new?issuable_template=Doc%20Review), link it from the MR, and - mention the original MR author in the new issue. Alternatively, the mainitainer can ask the MR author to create and link this issue before the MR is merged. + mention the original MR author in the new issue. Alternatively, the maintainer can ask the MR author to create and link this issue before the MR is merged. - After merging, documentation changes are reviewed by: @@ -157,14 +157,14 @@ All reviewers can help ensure accuracy, clarity, completeness, and adherence to #### Collaboration By default, the developer will work on documentation changes independently, but -the developer, PM, or technicial writer can propose a broader collaboration for +the developer, PM, or technical writer can propose a broader collaboration for any given issue. Additionally, technical writers are available for questions at any time. #### Review -- Techncial writers provide non-blocking reviews of all documentation changes, +- Technical writers provide non-blocking reviews of all documentation changes, before or after the change is merged. However, if the docs are ready in the MR while there's time before the freeze, the technical writer's review can commence early, on request. - The technical writer will confirm that the doc is clear, grammatically correct, @@ -173,7 +173,7 @@ Additionally, technical writers are available for questions at any time. the developer and code reviewer should have already made a good-faith effort to ensure: - Clarity. - Adherence to the plans and goals in the issue. - - Location (make sure the docs are in the correct directorkes and has the correct name). + - Location (make sure the docs are in the correct directories and has the correct name). - Syntax, typos, and broken links. - Improvements to the content. - Accordance with the [Documentation Style Guide](styleguide.md), and [Structure and Template](structure.md) doc. diff --git a/doc/development/documentation/index.md b/doc/development/documentation/index.md index 4bf8401c0e8..6dfd3b2a690 100644 --- a/doc/development/documentation/index.md +++ b/doc/development/documentation/index.md @@ -466,7 +466,7 @@ If you want to know the in-depth details, here's what's really happening: The following GitLab features are used among others: - [Manual actions](../../ci/yaml/README.md#whenmanual) -- [Multi project pipelines](https://docs.gitlab.com/ee/ci/multi_project_pipeline_graphs.html) +- [Multi project pipelines](../../ci/multi_project_pipeline_graphs.md) - [Review Apps](../../ci/review_apps/index.md) - [Artifacts](../../ci/yaml/README.md#artifacts) - [Specific Runner](../../ci/runners/README.md#locking-a-specific-runner-from-being-enabled-for-other-projects) diff --git a/doc/development/documentation/structure.md b/doc/development/documentation/structure.md index 95b5fcd99a1..fe676efa94d 100644 --- a/doc/development/documentation/structure.md +++ b/doc/development/documentation/structure.md @@ -77,8 +77,8 @@ for use (e.g. variations on the main use case), but if that's not applicable, th Examples of use cases on feature pages: - CE and EE: [Issues](../../user/project/issues/index.md#use-cases) - CE and EE: [Merge Requests](../../user/project/merge_requests/index.md) -- EE-only: [Geo](https://docs.gitlab.com/ee/administration/geo/replication/index.html) -- EE-only: [Jenkins integration](https://docs.gitlab.com/ee/integration/jenkins.html) +- EE-only: [Geo](../../administration/geo/replication/index.md) +- EE-only: [Jenkins integration](../../integration/jenkins.md) ## Requirements @@ -121,8 +121,8 @@ but commented out to help encourage others to add to it in the future. --> Notes: -- (1): Apply the [tier badges](https://docs.gitlab.com/ee/development/documentation/styleguide.html#product-badges) accordingly -- (2): Apply the correct format for the [GitLab version introducing the feature](https://docs.gitlab.com/ee/development/documentation/styleguide.html#gitlab-versions-and-tiers) +- (1): Apply the [tier badges](styleguide.md#product-badges) accordingly +- (2): Apply the correct format for the [GitLab version introducing the feature](styleguide.md#gitlab-versions-and-tiers) ``` ## Help and feedback section diff --git a/doc/development/ee_features.md b/doc/development/ee_features.md index bc472bb5b0a..857595330aa 100644 --- a/doc/development/ee_features.md +++ b/doc/development/ee_features.md @@ -557,40 +557,56 @@ due to `prepend`, but Grape is complex internally and we couldn't easily do that, so we'll follow regular object-oriented practices that we define the interface first here. -For example, suppose we have a few more optional params for EE, given this CE -API code: +For example, suppose we have a few more optional params for EE. We can move the +params out of the `Grape::API` class to a helper module, so we can `prepend` it +before it would be used in the class. ```ruby module API - class MergeRequests < Grape::API - # EE::API::MergeRequests would override the following helpers - helpers do - params :optional_params_ee do + class Projects < Grape::API + helpers Helpers::ProjectsHelpers + end +end +``` + +Given this CE API `params`: + +```ruby +module API + module Helpers + module ProjectsHelpers + extend ActiveSupport::Concern + extend Grape::API::Helpers + + params :optional_project_params_ce do + # CE specific params go here... end - end - params :optional_params do - # CE specific params go here... + params :optional_project_params_ee do + end - use :optional_params_ee + params :optional_project_params do + use :optional_project_params_ce + use :optional_project_params_ee + end end end end -API::MergeRequests.prepend(EE::API::MergeRequests) +API::Helpers::ProjectsHelpers.prepend(EE::API::Helpers::ProjectsHelpers) ``` -And then we could override it in EE module: +We could override it in EE module: ```ruby module EE module API - module MergeRequests - extend ActiveSupport::Concern + module Helpers + module ProjectsHelpers + extend ActiveSupport::Concern - prepended do - helpers do - params :optional_params_ee do + prepended do + params :optional_project_params_ee do # EE specific params go here... end end @@ -600,9 +616,6 @@ module EE end ``` -This way, the only difference between CE and EE for that API file would be -`prepend EE::API::MergeRequests`. - #### EE helpers To make it easy for an EE module to override the CE helpers, we need to define @@ -933,7 +946,7 @@ export default { ``` #### For JS code that is EE only, like props, computed properties, methods, etc, we will keep the current approach - - Since we [can't async load a mixin](https://github.com/vuejs/vue-loader/issues/418#issuecomment-254032223) we will use the [`ee_else_ce`](https://docs.gitlab.com/ee/development/ee_features.html#javascript-code-in-assetsjavascripts) alias we already have for webpack. + - Since we [can't async load a mixin](https://github.com/vuejs/vue-loader/issues/418#issuecomment-254032223) we will use the [`ee_else_ce`](../development/ee_features.md#javascript-code-in-assetsjavascripts) alias we already have for webpack. - This means all the EE specific props, computed properties, methods, etc that are EE only should be in a mixin in the `ee/` folder and we need to create a CE counterpart of the mixin ##### Example: @@ -960,7 +973,7 @@ import mixin from 'ee_else_ce/path/mixin'; ### Non Vue Files For regular JS files, the approach is similar. -1. We will keep using the [`ee_else_ce`](https://docs.gitlab.com/ee/development/ee_features.html#javascript-code-in-assetsjavascripts) helper, this means that EE only code should be inside the `ee/` folder. +1. We will keep using the [`ee_else_ce`](../development/ee_features.md#javascript-code-in-assetsjavascripts) helper, this means that EE only code should be inside the `ee/` folder. 1. An EE file should be created with the EE only code, and it should extend the CE counterpart. 1. For code inside functions that can't be extended, the code should be moved into a new file and we should use `ee_else_ce` helper: diff --git a/doc/development/elasticsearch.md b/doc/development/elasticsearch.md index 0c9e7908713..8b0f4f02d19 100644 --- a/doc/development/elasticsearch.md +++ b/doc/development/elasticsearch.md @@ -2,7 +2,7 @@ This area is to maintain a compendium of useful information when working with elasticsearch. -Information on how to enable ElasticSearch and perform the initial indexing is kept in https://docs.gitlab.com/ee/integration/elasticsearch.html#enabling-elasticsearch +Information on how to enable ElasticSearch and perform the initial indexing is kept in ../integration/elasticsearch.md#enabling-elasticsearch ## Initial installation on OS X @@ -16,7 +16,7 @@ and use `docker stop elastic56` and `docker start elastic56` to stop/start it. ### Installing on the host -We currently only support Elasticsearch [5.6 to 6.x](https://docs.gitlab.com/ee/integration/elasticsearch.html#requirements) +We currently only support Elasticsearch [5.6 to 6.x](../integration/elasticsearch.md#version-requirements) Version 5.6 is available on homebrew and is the recommended version to use in order to test compatibility. @@ -43,7 +43,7 @@ this adds `gitlab-elasticsearch-indexer` to `$GOPATH/bin`, please make sure that - `gitlab:elastic:test:index_size`: Tells you how much space the current index is using, as well as how many documents are in the index. - `gitlab:elastic:test:index_size_change`: Outputs index size, reindexes, and outputs index size again. Useful when testing improvements to indexing size. -Additionally, if you need large repos or multiple forks for testing, please consider [following these instructions](https://docs.gitlab.com/ee/development/rake_tasks.html#extra-project-seed-options) +Additionally, if you need large repos or multiple forks for testing, please consider [following these instructions](rake_tasks.md#extra-project-seed-options) ## How does it work? diff --git a/doc/development/fe_guide/style_guide_js.md b/doc/development/fe_guide/style_guide_js.md index 060cd8baf7f..94dfdbdd073 100644 --- a/doc/development/fe_guide/style_guide_js.md +++ b/doc/development/fe_guide/style_guide_js.md @@ -544,14 +544,24 @@ Please check this [rules][eslint-plugin-vue-rules] for more documentation. <component @click="eventHandler"/> ``` -1. Shorthand `:` is preferable over `v-bind` +2. Shorthand `:` is preferable over `v-bind` ```javascript // bad <component v-bind:class="btn"/> // good - <component :class="btsn"/> + <component :class="btn"/> + ``` + +3. Shorthand `#` is preferable over `v-slot` + + ```javascript + // bad + <template v-slot:header></template> + + // good + <template #header></template> ``` #### Closing tags diff --git a/doc/development/fe_guide/style_guide_scss.md b/doc/development/fe_guide/style_guide_scss.md index 36880dd746d..b25dce65ffe 100644 --- a/doc/development/fe_guide/style_guide_scss.md +++ b/doc/development/fe_guide/style_guide_scss.md @@ -9,20 +9,44 @@ easy to maintain, and performant for the end-user. As part of the effort for [cleaning up our CSS and moving our components into GitLab-UI](https://gitlab.com/groups/gitlab-org/-/epics/950) led by the [GitLab UI WG](https://gitlab.com/gitlab-com/www-gitlab-com/merge_requests/20623) we prefer the use of utility classes over adding new CSS. However, complex CSS can be addressed by adding component classes. -We have a few internal utility classes in [`common.scss`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/assets/stylesheets/framework/common.scss) -and we use [Bootstrap's Utility Classes](https://getbootstrap.com/docs/4.3/utilities/) +#### Where are utility classes defined? + +- [Bootstrap's Utility Classes](https://getbootstrap.com/docs/4.3/utilities/) +- [`common.scss`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/assets/stylesheets/framework/common.scss) (old) +- [`utilities.scss`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/assets/stylesheets/utilities.scss) (new) + +#### Where should I put new utility classes? New utility classes should be added to [`utilities.scss`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/assets/stylesheets/utilities.scss). Existing classes include: -**Background color**: `.bg-variant-shade` e.g. `.bg-warning-400` -**Text color**: `.text-variant-shade` e.g. `.text-success-500` +| Name | Pattern | Example | +|------|---------|---------| +| Background color | `.bg-{variant}-{shade}` | `.bg-warning-400` | +| Text color | `.text-{variant}-{shade}` | `.text-success-500` | +| Font size | `.text-{size}` | `.text-2` | + +- `{variant}` is one of 'primary', 'secondary', 'success', 'warning', 'error' +- `{shade}` is on of the shades listed on [colors](https://design.gitlab.com/foundations/colors/) +- `{size}` is a number from 1-6 from our [Type scale](https://design.gitlab.com/foundations/typography) + +#### When should I create component classes? + +We recommend a "utility-first" approach. + +1. Start with utility classes. +2. If composing utility classes into a component class removes code duplication and encapsulates a clear responsibility, do it. + +This encourages an organic growth of component classes and prevents the creation of one-off unreusable classes. Also, the kind of classes that emerge from "utility-first" tend to be design-centered (e.g. `.button`, `.alert`, `.card`) rather than domain-centered (e.g. `.security-report-widget`, `.commit-header-icon`). + +Examples of component classes that were created using "utility-first" include: -- variant is one of 'primary', 'secondary', 'success', 'warning', 'error' -- shade is on of the shades listed on [colors](https://design.gitlab.com/foundations/colors/) +- [`.circle-icon-container`](https://gitlab.com/gitlab-org/gitlab-ce/blob/579fa8b8ec7eb38d40c96521f517c9dab8c3b97a/app/assets/stylesheets/framework/icons.scss#L85) +- [`.d-flex-center`](https://gitlab.com/gitlab-org/gitlab-ce/blob/900083d89cd6af391d26ab7922b3f64fa2839bef/app/assets/stylesheets/framework/common.scss#L425) -**Font size**: `.text-size` e.g. `.text-2` +Inspiration: -- **size** is number from 1-6 from our [Type scale](https://design.gitlab.com/foundations/typography) +- https://tailwindcss.com/docs/utility-first +- https://tailwindcss.com/docs/extracting-components ### Naming diff --git a/doc/development/feature_flags.md b/doc/development/feature_flags.md index 82b09cc0224..c871015aaf6 100644 --- a/doc/development/feature_flags.md +++ b/doc/development/feature_flags.md @@ -28,7 +28,7 @@ the time, you should execute `/chatops run feature set my_feature_flag 50`. This document only covers feature flags used in the development of GitLab itself. Feature flags in deployed user applications can be found at -[Feature Flags](https://docs.gitlab.com/ee/user/project/operations/feature_flags.html) +[Feature Flags](../user/project/operations/feature_flags.md) ## Developing with feature flags @@ -111,7 +111,7 @@ Whenever a feature flag is present, make sure to test _both_ states of the feature flag. See the -[testing guide](testing_guide/best_practices.html#feature-flags-in-tests) +[testing guide](testing_guide/best_practices.md#feature-flags-in-tests) for information and examples on how to stub feature flags in tests. ## Enabling a feature flag (in development) diff --git a/doc/development/geo.md b/doc/development/geo.md index c8e6a86eb52..87ec34ec5c4 100644 --- a/doc/development/geo.md +++ b/doc/development/geo.md @@ -90,7 +90,7 @@ projects that need updating. Those projects can be: timestamp that is more recent than the `last_repository_successful_sync_at` timestamp in the `Geo::ProjectRegistry` model. - Manual: The admin can manually flag a repository to resync in the - [Geo admin panel](https://docs.gitlab.com/ee/user/admin_area/geo_nodes.html). + [Geo admin panel](../user/admin_area/geo_nodes.md). When we fail to fetch a repository on the secondary `RETRIES_BEFORE_REDOWNLOAD` times, Geo does a so-called _redownload_. It will do a clean clone @@ -299,7 +299,7 @@ basically hashes all Git refs together and stores that hash in the The **secondary** node does the same to calculate the hash of its clone, and compares the hash with the value the **primary** node calculated. If there is a mismatch, Geo will mark this as a mismatch -and the administrator can see this in the [Geo admin panel](https://docs.gitlab.com/ee/user/admin_area/geo_nodes.html). +and the administrator can see this in the [Geo admin panel](../user/admin_area/geo_nodes.md). ## Glossary diff --git a/doc/development/go_guide/index.md b/doc/development/go_guide/index.md index cf78b792ffd..4dad8815fcb 100644 --- a/doc/development/go_guide/index.md +++ b/doc/development/go_guide/index.md @@ -40,7 +40,7 @@ of possible security breaches in our code: - SQL injections Remember to run -[SAST](https://docs.gitlab.com/ee/user/application_security/sast/index) +[SAST](../../user/application_security/sast/index.md) **[ULTIMATE]** on your project (or at least the [gosec analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/gosec)), and to follow our [Security @@ -94,9 +94,9 @@ become available, you will be able to share job templates like this Dependencies should be kept to the minimum. The introduction of a new dependency should be argued in the merge request, as per our [Approval Guidelines](../code_review.md#approval-guidelines). Both [License -Management](https://docs.gitlab.com/ee/user/project/merge_requests/license_management.html) +Management](../../user/project/merge_requests/license_management.md) **[ULTIMATE]** and [Dependency -Scanning](https://docs.gitlab.com/ee/user/application_security/dependency_scanning/index) +Scanning](../../user/application_security/dependency_scanning/index.md) **[ULTIMATE]** should be activated on all projects to ensure new dependencies security status and license compatibility. diff --git a/doc/development/i18n/externalization.md b/doc/development/i18n/externalization.md index 38d56322de8..38425674567 100644 --- a/doc/development/i18n/externalization.md +++ b/doc/development/i18n/externalization.md @@ -195,6 +195,7 @@ For example use `%{created_at}` in Ruby but `%{createdAt}` in JavaScript. Sometimes you need to add some context to the text that you want to translate (if the word occurs in a sentence and/or the word is ambiguous). +Namespaces should be PascalCase. - In Ruby/HAML: diff --git a/doc/development/i18n/merging_translations.md b/doc/development/i18n/merging_translations.md index 85284d8c714..eadf1cd74c5 100644 --- a/doc/development/i18n/merging_translations.md +++ b/doc/development/i18n/merging_translations.md @@ -32,7 +32,7 @@ clicking `Pause sync` on the [Crowdin integration settings page](https://translate.gitlab.com/project/gitlab-ee/settings#integration). When all failures are resolved, the translations need to be double -checked once more as discussed in [confidential issue](https://docs.gitlab.com/ee/user/project/issues/confidential_issues.html) `https://gitlab.com/gitlab-org/gitlab-ce/issues/37850`. +checked once more as discussed in [confidential issue](../../user/project/issues/confidential_issues.md) `https://gitlab.com/gitlab-org/gitlab-ce/issues/37850`. ## Merging translations diff --git a/doc/development/i18n/proofreader.md b/doc/development/i18n/proofreader.md index faea8d5982f..fb5cfb6c157 100644 --- a/doc/development/i18n/proofreader.md +++ b/doc/development/i18n/proofreader.md @@ -27,7 +27,7 @@ are very appreciative of the work done by translators and proofreaders! - Czech - Proofreaders needed. - Danish - - Proofreaders needed. + - Saederup92 - [GitLab](https://gitlab.com/Saederup92), [Crowdin](https://crowdin.com/profile/Saederup92) - Dutch - Emily Hendle - [GitLab](https://gitlab.com/pundachan), [Crowdin](https://crowdin.com/profile/pandachan) - Esperanto @@ -56,7 +56,6 @@ are very appreciative of the work done by translators and proofreaders! - Italian - Paolo Falomo - [GitLab](https://gitlab.com/paolofalomo), [Crowdin](https://crowdin.com/profile/paolo.falomo) - Japanese - - Yamana Tokiuji - [GitLab](https://gitlab.com/tokiuji), [Crowdin](https://crowdin.com/profile/yamana) - Hiroyuki Sato - [GitLab](https://gitlab.com/hiroponz), [Crowdin](https://crowdin.com/profile/hiroponz) - Tomo Dote - [Gitlab](https://gitlab.com/fu7mu4), [Crowdin](https://crowdin.com/profile/fu7mu4) - Korean @@ -72,6 +71,7 @@ are very appreciative of the work done by translators and proofreaders! - Maksymilian Roman - [GitLab](https://gitlab.com/villaincandle), [Crowdin](https://crowdin.com/profile/villaincandle) - Portuguese - Proofreaders needed. + - Diogo Trindade - [GitLab](https://gitlab.com/luisdiogo2071317), [Crowdin](https://crowdin.com/profile/ldiogotrindade) - Portuguese, Brazilian - Paulo George Gomes Bezerra - [GitLab](https://gitlab.com/paulobezerra), [Crowdin](https://crowdin.com/profile/paulogomes.rep) - André Gama - [GitLab](https://gitlab.com/andregamma), [Crowdin](https://crowdin.com/profile/ToeOficial) diff --git a/doc/development/import_export.md b/doc/development/import_export.md index f7f48b03651..fd067b80c16 100644 --- a/doc/development/import_export.md +++ b/doc/development/import_export.md @@ -27,7 +27,7 @@ Read through the current performance problems using the Import/Export below. ### OOM errors -Out of memory (OOM) errors are normally caused by the [Sidekiq Memory Killer](https://docs.gitlab.com/ee/administration/operations/sidekiq_memory_killer.html): +Out of memory (OOM) errors are normally caused by the [Sidekiq Memory Killer](../administration/operations/sidekiq_memory_killer.md): ```bash SIDEKIQ_MEMORY_KILLER_MAX_RSS = 2GB in GitLab.com diff --git a/doc/development/licensed_feature_availability.md b/doc/development/licensed_feature_availability.md index 1657d73e0c9..6f3dd59b2c3 100644 --- a/doc/development/licensed_feature_availability.md +++ b/doc/development/licensed_feature_availability.md @@ -7,8 +7,8 @@ on-premise or GitLab.com plans and features. ## Restricting features scoped by namespaces or projects GitLab.com plans are persisted on user groups and namespaces, therefore, if you're adding a -feature such as [Related issues](https://docs.gitlab.com/ee/user/project/issues/related_issues.html) or -[Service desk](https://docs.gitlab.com/ee/user/project/service_desk.html), +feature such as [Related issues](../user/project/issues/related_issues.md) or +[Service desk](../user/project/service_desk.md), it should be restricted on namespace scope. 1. Add the feature symbol on `EES_FEATURES`, `EEP_FEATURES` or `EEU_FEATURES` constants in @@ -22,8 +22,8 @@ project.feature_available?(:feature_symbol) ## Restricting global features (instance) -However, for features such as [Geo](https://docs.gitlab.com/ee/administration/geo/replication/index.html) and -[Load balancing](https://docs.gitlab.com/ee/administration/database_load_balancing.html), which cannot be restricted +However, for features such as [Geo](../administration/geo/replication/index.md) and +[Load balancing](../administration/database_load_balancing.md), which cannot be restricted to only a subset of projects or namespaces, the check will be made directly in the instance license. diff --git a/doc/development/migration_style_guide.md b/doc/development/migration_style_guide.md index 0c326eeb851..9b26f691b55 100644 --- a/doc/development/migration_style_guide.md +++ b/doc/development/migration_style_guide.md @@ -186,7 +186,11 @@ end When adding a foreign-key constraint to either an existing or new column remember to also add a index on the column. -This is _required_ for all foreign-keys. +This is **required** for all foreign-keys, e.g., to support efficient cascading +deleting: when a lot of rows in a table get deleted, the referenced records need +to be deleted too. The database has to look for corresponding records in the +referenced table. Without an index, this will result in a sequential scan on the +table which can take a long time. Here's an example where we add a new column with a foreign key constraint. Note it includes `index: true` to create an index for it. diff --git a/doc/development/new_fe_guide/development/components.md b/doc/development/new_fe_guide/development/components.md index 8ae58d30c35..963ce53423b 100644 --- a/doc/development/new_fe_guide/development/components.md +++ b/doc/development/new_fe_guide/development/components.md @@ -17,5 +17,5 @@ D3 is very popular across many projects outside of GitLab: Within GitLab, D3 has been used for the following notable features -- [Prometheus graphs](https://docs.gitlab.com/ee/user/project/integrations/prometheus.html) +- [Prometheus graphs](../../../user/project/integrations/prometheus.md) - Contribution calendars diff --git a/doc/development/new_fe_guide/tips.md b/doc/development/new_fe_guide/tips.md index 889a5aab2b7..4564f678ec0 100644 --- a/doc/development/new_fe_guide/tips.md +++ b/doc/development/new_fe_guide/tips.md @@ -10,16 +10,16 @@ yarn clean ## Creating feature flags in development -The process for creating a feature flag is the same as [enabling a feature flag in development](https://docs.gitlab.com/ee/development/feature_flags.html#enabling-a-feature-flag-in-development). +The process for creating a feature flag is the same as [enabling a feature flag in development](../feature_flags.md#enabling-a-feature-flag-in-development). Your feature flag can now be: -- [made available to the frontend](https://docs.gitlab.com/ee/development/feature_flags.html#frontend) via the `gon` -- queried in [tests](https://docs.gitlab.com/ee/development/feature_flags.html#specs) +- [made available to the frontend](../feature_flags.md#frontend) via the `gon` +- queried in [tests](../feature_flags.md#specs) - queried in HAML templates and ruby files via the `Feature.enabled?(:my_shiny_new_feature_flag)` method ### More on feature flags -- [Deleting a feature flag](https://docs.gitlab.com/ee/api/features.html#delete-a-feature) -- [Manage feature flags](https://docs.gitlab.com/ee/development/feature_flags.html) -- [Feature flags API](https://docs.gitlab.com/ee/api/features.html) +- [Deleting a feature flag](../../api/features.md#delete-a-feature) +- [Manage feature flags](../feature_flags.md) +- [Feature flags API](../../api/features.md) diff --git a/doc/development/packages.md b/doc/development/packages.md index a3b891d7834..ab0c5f9904d 100644 --- a/doc/development/packages.md +++ b/doc/development/packages.md @@ -1,8 +1,8 @@ # Packages **[PREMIUM]** -This document will guide you through adding another [package management system](https://docs.gitlab.com/ee/administration/packages.html) support to GitLab. +This document will guide you through adding another [package management system](../administration/packages.md) support to GitLab. -See already supported package types in [Packages documentation](https://docs.gitlab.com/ee/administration/packages.html) +See already supported package types in [Packages documentation](../administration/packages.md) Since GitLab packages' UI is pretty generic, it is possible to add new package system support by solely backend changes. This guide is superficial and does @@ -46,7 +46,7 @@ Group-level and instance-level endpoints are good to have but are optional. NOTE: **Note:** To avoid name conflict for instance-level endpoints we use -[the package naming convention](https://docs.gitlab.com/ee/user/project/packages/npm_registry.html#package-naming-convention) +[the package naming convention](../user/project/packages/npm_registry.md#package-naming-convention) ## Configuration diff --git a/doc/development/rake_tasks.md b/doc/development/rake_tasks.md index 27fc3231218..28a12572961 100644 --- a/doc/development/rake_tasks.md +++ b/doc/development/rake_tasks.md @@ -31,7 +31,7 @@ project. #### Seeding issues for Insights charts **[ULTIMATE]** You can seed issues specifically for working with the -[Insights charts](https://docs.gitlab.com/ee/user/group/insights/index.html) with the +[Insights charts](../user/group/insights/index.md) with the `gitlab:seed:insights:issues` task: ```shell @@ -108,11 +108,13 @@ To make sure that indices still fit. You could find great details in: In order to run the test you can use the following commands: -- `rake spec` to run the rspec suite -- `rake karma` to run the karma test suite -- `rake gitlab:test` to run all the tests +- `bin/rake spec` to run the rspec suite +- `bin/rake spec:unit` to run the only the unit tests +- `bin/rake spec:integration` to run the only the integration tests +- `bin/rake spec:system` to run the only the system tests +- `bin/rake karma` to run the karma test suite -Note: `rake spec` takes significant time to pass. +Note: `bin/rake spec` takes significant time to pass. Instead of running full test suite locally you can save a lot of time by running a single test or directory related to your changes. After you submit merge request CI will run full test suite for you. Green CI status in the merge request means @@ -121,6 +123,9 @@ full test suite is passed. Note: You can't run `rspec .` since this will try to run all the `_spec.rb` files it can find, also the ones in `/tmp` +Note: You can pass RSpec command line options to the `spec:unit`, +`spec:integration`, and `spec:system` tasks, e.g. `bin/rake "spec:unit[--tag ~geo --dry-run]"`. + To run a single test file you can use: - `bin/rspec spec/controllers/commit_controller_spec.rb` for a rspec test diff --git a/doc/development/rolling_out_changes_using_feature_flags.md b/doc/development/rolling_out_changes_using_feature_flags.md index 8e4e07c4319..1b4c89ba2a8 100644 --- a/doc/development/rolling_out_changes_using_feature_flags.md +++ b/doc/development/rolling_out_changes_using_feature_flags.md @@ -102,7 +102,7 @@ GitLab's feature library (using [Flipper](https://github.com/jnunemaker/flipper), and covered in the [Feature Flags](feature_flags.md) guide) supports rolling out changes to a percentage of users. This in turn can be controlled using [GitLab -chatops](https://docs.gitlab.com/ee/ci/chatops/). +chatops](../ci/chatops/README.md). For an up to date list of feature flag commands please see [the source code](https://gitlab.com/gitlab-com/chatops/blob/master/lib/chatops/commands/feature.rb). diff --git a/doc/development/routing.md b/doc/development/routing.md new file mode 100644 index 00000000000..e9c0ad8d4e8 --- /dev/null +++ b/doc/development/routing.md @@ -0,0 +1,63 @@ +# Routing + +The GitLab backend is written primarily with Rails so it uses [Rails +routing](https://guides.rubyonrails.org/routing.html). Beside Rails best +practices, there are few rules unique to the GitLab application. To +support subgroups, GitLab project and group routes use the wildcard +character to match project and group routes. For example, we might have +a path such as: + + /gitlab-com/customer-success/north-america/west/customerA + +However, paths can be ambiguous. Consider the following example: + + /gitlab-com/edit + +It's ambiguous whether there is a subgroup named `edit` or whether +this is a special endpoint to edit the `gitlab-com` group. + +To eliminate the ambiguity and to make the backend easier to maintain, +we introduced the `/-/` scope. The purpose of it is to separate group or +project paths from the rest of the routes. Also it helps to reduce the +number of [reserved names](../user/reserved_names.md). + +## Global routes + +We have a number of global routes. For example: + + /-/health + /-/metrics + +## Group routes + +Every group route must be under the `/-/` scope. + +Examples: + + gitlab-org/-/edit + gitlab-org/-/activity + gitlab-org/-/security/dashboard + gitlab-org/serverless/-/activity + +To achieve that, use the `scope '-'` method. + +## Project routes + +Every project route must be under the `/-/` scope, except cases where a Git +client or other software requires something different. + +Examples: + + gitlab-org/gitlab-ce/-/activity + gitlab-org/gitlab-ce/-/jobs/123 + gitlab-org/gitlab-ce/-/settings/repository + gitlab-org/serverless/runtimes/-/settings/repository + +Currently, only some project routes are placed under the `/-/` scope. However, +you can help us migrate more of them! To migrate project routes: + +1. Modify existing routes by adding `-` scope. +1. Add redirects for legacy routes by using `Gitlab::Routing.redirect_legacy_paths`. +1. Create a technical debt issue to remove deprecated routes in later releases. + +To get started, see an [example merge request](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/28435). diff --git a/doc/development/testing_guide/best_practices.md b/doc/development/testing_guide/best_practices.md index 63ec9755462..82439c94c5a 100644 --- a/doc/development/testing_guide/best_practices.md +++ b/doc/development/testing_guide/best_practices.md @@ -2,19 +2,29 @@ ## Test Design -Testing at GitLab is a first class citizen, not an afterthought. It's important we consider the design of our tests -as we do the design of our features. +Testing at GitLab is a first class citizen, not an afterthought. It's important we consider the design of our tests +as we do the design of our features. -When implementing a feature, we think about developing the right capabilities the right way, which helps us -narrow our scope to a manageable level. When implementing tests for a feature, we must think about developing -the right tests, but then cover _all_ the important ways the test may fail, which can quickly widen our scope to +When implementing a feature, we think about developing the right capabilities the right way, which helps us +narrow our scope to a manageable level. When implementing tests for a feature, we must think about developing +the right tests, but then cover _all_ the important ways the test may fail, which can quickly widen our scope to a level that is difficult to manage. -Test heuristics can help solve this problem. They concisely address many of the common ways bugs -manifest themselves within our code. When designing our tests, take time to review known test heuristics to inform -our test design. We can find some helpful heuristics documented in the Handbook in the +Test heuristics can help solve this problem. They concisely address many of the common ways bugs +manifest themselves within our code. When designing our tests, take time to review known test heuristics to inform +our test design. We can find some helpful heuristics documented in the Handbook in the [Test Design](https://about.gitlab.com/handbook/engineering/quality/guidelines/test-engineering/test-design/) section. +## Run tests against MySQL + +By default, tests are only run againts PostgreSQL, but you can run them on +demand against MySQL by following one of the following conventions: + +| Convention | Valid example | +|:----------------------|:-----------------------------| +| Include `mysql` in your branch name | `enhance-mysql-support` | +| Include `[run mysql]` in your commit message | `Fix MySQL support<br><br>[run mysql]` | + ## Test speed GitLab has a massive test suite that, without [parallelization], can take hours @@ -184,11 +194,11 @@ instead of 30+ seconds in case of a regular `spec_helper`. ### `let` variables GitLab's RSpec suite has made extensive use of `let`(along with it strict, non-lazy -version `let!`) variables to reduce duplication. However, this sometimes [comes at the cost of clarity][lets-not], +version `let!`) variables to reduce duplication. However, this sometimes [comes at the cost of clarity][lets-not], so we need to set some guidelines for their use going forward: - `let!` variables are preferable to instance variables. `let` variables - are preferable to `let!` variables. Local variables are preferable to + are preferable to `let!` variables. Local variables are preferable to `let` variables. - Use `let` to reduce duplication throughout an entire spec file. - Don't use `let` to define variables used by a single test; define them as @@ -199,8 +209,8 @@ so we need to set some guidelines for their use going forward: - Try to avoid overriding the definition of one `let` variable with another. - Don't define a `let` variable that's only used by the definition of another. Use a helper method instead. -- `let!` variables should be used only in case if strict evaluation with defined - order is required, otherwise `let` will suffice. Remember that `let` is lazy and won't +- `let!` variables should be used only in case if strict evaluation with defined + order is required, otherwise `let` will suffice. Remember that `let` is lazy and won't be evaluated until it is referenced. [lets-not]: https://robots.thoughtbot.com/lets-not diff --git a/doc/development/testing_guide/frontend_testing.md b/doc/development/testing_guide/frontend_testing.md index 9bd99e80357..4c9d1684c00 100644 --- a/doc/development/testing_guide/frontend_testing.md +++ b/doc/development/testing_guide/frontend_testing.md @@ -15,10 +15,8 @@ information on general testing practices at GitLab. ## Jest -GitLab has started to migrate tests to the [Jest](https://jestjs.io) -testing framework. You can read a [detailed evaluation](https://gitlab.com/gitlab-org/gitlab-ce/issues/49171) -of Jest compared to our use of Karma and Jasmine. In summary, it will allow us -to improve the performance and consistency of our frontend tests. +We have started to migrate frontend tests to the [Jest](https://jestjs.io) testing framework (see also the corresponding +[epic](https://gitlab.com/groups/gitlab-org/-/epics/895)). Jest tests can be found in `/spec/frontend` and `/ee/spec/frontend` in EE. @@ -26,6 +24,17 @@ It is not yet a requirement to use Jest. You can view the [epic](https://gitlab.com/groups/gitlab-org/-/epics/873) of issues we need to solve before being able to use Jest for all our needs. +### Differences to Karma + +- Jest runs in a Node.js environment, not in a browser. Support for running Jest tests in a browser [is planned](https://gitlab.com/gitlab-org/gitlab-ce/issues/58205). +- Because Jest runs in a Node.js environment, it uses [jsdom](https://github.com/jsdom/jsdom) by default. +- All calls to `setTimeout` and `setInterval` are mocked away. See also [Jest Timer Mocks](https://jestjs.io/docs/en/timer-mocks). +- `rewire` is not required because Jest supports mocking modules. See also [Manual Mocks](https://jestjs.io/docs/en/manual-mocks). +- The following will cause tests to fail in Jest: + - Unmocked requests. + - Unhandled Promise rejections. + - Calls to `console.warn`, including warnings from libraries like Vue. + ### Debugging Jest tests Running `yarn jest-debug` will run Jest in debug mode, allowing you to debug/inspect as described in the [Jest docs](https://jestjs.io/docs/en/troubleshooting#tests-are-failing-and-you-don-t-know-why). @@ -58,9 +67,6 @@ Remember that the performance of each test depends on the environment. GitLab uses the [Karma][karma] test runner with [Jasmine] as its test framework for our JavaScript unit and integration tests. -We generate HTML and JSON fixtures from backend views and controllers -using RSpec (see `spec/javascripts/fixtures/*.rb` for examples). -Fixtures are served during testing by the [jasmine-jquery][jasmine-jquery] plugin. JavaScript tests live in `spec/javascripts/`, matching the folder structure of `app/assets/javascripts/`: `app/assets/javascripts/behaviors/autosize.js` @@ -412,7 +418,7 @@ See this [section][vue-test]. For running the frontend tests, you need the following commands: -- `rake karma:fixtures` (re-)generates fixtures. +- `rake karma:fixtures` (re-)generates [fixtures](#frontend-test-fixtures). - `yarn test` executes the tests. As long as the fixtures don't change, `yarn test` is sufficient (and saves you some time). @@ -460,6 +466,48 @@ yarn karma -f 'spec/javascripts/ide/**/file_spec.js' Information on setting up and running RSpec integration tests with [Capybara] can be found in the [Testing Best Practices](best_practices.md). +## Frontend test fixtures + +Code that is added to HAML templates (in `app/views/`) or makes Ajax requests to the backend has tests that require HTML or JSON from the backend. +Fixtures for these tests are located at: + +- `spec/javascripts/fixtures/`, for running tests in CE. +- `ee/spec/javascripts/fixtures/`, for running tests in EE. + +Fixture files in: + +- The Karma test suite are served by [jasmine-jquery](https://github.com/velesin/jasmine-jquery). +- Jest use `spec/frontend/helpers/fixtures.js`. + +The following are examples of tests that work for both Karma and Jest: + +```javascript +it('makes a request', () => { + const responseBody = getJSONFixture('some/fixture.json'); // loads spec/javascripts/fixtures/some/fixture.json + axiosMock.onGet(endpoint).reply(200, responseBody); + + myButton.click(); + + // ... +}); + +it('uses some HTML element', () => { + loadFixtures('some/page.html'); // loads spec/javascripts/fixtures/some/page.html and adds it to the DOM + + const element = document.getElementById('#my-id'); + + // ... +}); +``` + +HTML and JSON fixtures are generated from backend views and controllers using RSpec (see `spec/javascripts/fixtures/*.rb`). + +For each fixture, the content of the `response` variable is stored in the output file. +This variable gets automagically set if the test is marked as `type: :request` or `type: :controller`. +Fixtures are regenerated using the `bin/rake karma:fixtures` command but you can also generate them individually, +for example `bin/rspec spec/javascripts/fixtures/merge_requests.rb`. +When creating a new fixture, it often makes sense to take a look at the corresponding tests for the endpoint in `(ee/)spec/controllers/` or `(ee/)spec/requests/`. + ## Gotchas ### RSpec errors due to JavaScript @@ -492,7 +540,6 @@ end ``` [jasmine-focus]: https://jasmine.github.io/2.5/focused_specs.html -[jasmine-jquery]: https://github.com/velesin/jasmine-jquery [karma]: http://karma-runner.github.io/ [vue-test]: https://docs.gitlab.com/ce/development/fe_guide/vue.html#testing-vue-components [rspec]: https://github.com/rspec/rspec-rails#feature-specs diff --git a/doc/development/testing_guide/review_apps.md b/doc/development/testing_guide/review_apps.md index f34793c11f4..b5fde92a23d 100644 --- a/doc/development/testing_guide/review_apps.md +++ b/doc/development/testing_guide/review_apps.md @@ -113,7 +113,7 @@ You can also manually start the `review-qa-all`: it runs the full QA suite. On every [pipeline][gitlab-pipeline] in the `qa` stage, the `review-performance` job is automatically started: this job does basic browser performance testing using a -[Sitespeed.io Container](https://docs.gitlab.com/ee/user/project/merge_requests/browser_performance_testing.html). +[Sitespeed.io Container](../../user/project/merge_requests/browser_performance_testing.md). ## How to: @@ -203,7 +203,7 @@ find a way to limit it to only us.** [automated_cleanup.rb]: https://gitlab.com/gitlab-org/gitlab-ee/blob/master/scripts/review_apps/automated_cleanup.rb [Auto-DevOps.gitlab-ci.yml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml [gitlab-ci-yml]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/.gitlab-ci.yml -[gitlab-k8s-integration]: https://docs.gitlab.com/ee/user/project/clusters/index.html +[gitlab-k8s-integration]: ../../user/project/clusters/index.md [password-bug]: https://gitlab.com/gitlab-org/gitlab-ce/issues/53621 --- diff --git a/doc/development/testing_guide/testing_levels.md b/doc/development/testing_guide/testing_levels.md index 1fa6e38ea5a..b5155b6b7fa 100644 --- a/doc/development/testing_guide/testing_levels.md +++ b/doc/development/testing_guide/testing_levels.md @@ -4,12 +4,14 @@ _This diagram demonstrates the relative priority of each test type we use. `e2e` stands for end-to-end._ -As of 2019-04-16, we have the following distribution of tests per level: +As of 2019-05-01, we have the following distribution of tests per level: -- 67 black-box tests at the system level (aka end-to-end or QA tests) in CE, 98 in EE. This represents 0.3% of all the CE tests (0.3% in EE). -- 5,457 white-box tests at the system level (aka system or feature tests) in CE, 6,585 in EE. This represents 24.6% of all the CE tests (20.3% in EE). -- 8,298 integration tests in CE, 10,633 in EE: 0.3% of all the CE tests (0.3% in EE). This represents 37.2% of all the CE tests (32.8% in EE). -- 8,403 unit tests in CE, 15,090 in EE: 0.3% of all the CE tests (0.3% in EE). This represents 37.8% of all the CE tests (46.6% in EE). +| Test level | Community Edition | Enterprise Edition | Community + Enterprise Edition | +| --------- | ---------- | -------------- | ----- | +| Black-box tests at the system level (aka end-to-end or QA tests) | 68 (0.14%) | 31 (0.2%) | 99 (0.17%) | +| White-box tests at the system level (aka system or feature tests) | 5,471 (11.9%) | 969 (7.4%) | 6440 (10.9%) | +| Integration tests | 8,333 (18.2%) | 2,244 (17.2%) | 10,577 (17.9%) | +| Unit tests | 32,031 (69.7%) | 9,778 (75.1%) | 41,809 (71%) | ## Unit tests diff --git a/doc/gitlab-basics/README.md b/doc/gitlab-basics/README.md index 28ba20bec09..00ac6d6b650 100644 --- a/doc/gitlab-basics/README.md +++ b/doc/gitlab-basics/README.md @@ -15,7 +15,7 @@ This documentation is split into the following groups: The following are guides to basic GitLab functionality: -- [Create and add your SSH Keys](create-your-ssh-keys.md), for enabling Git over SSH. +- [Create and add your SSH public key](create-your-ssh-keys.md), for enabling Git over SSH. - [Create a project](create-project.md), to start using GitLab. - [Create a group](../user/group/index.md#create-a-new-group), to combine and administer projects together. - [Create a branch](create-branch.md), to make changes to files stored in a project's repository. diff --git a/doc/gitlab-basics/create-your-ssh-keys.md b/doc/gitlab-basics/create-your-ssh-keys.md index 8fecdc6948e..aac73d4c9c5 100644 --- a/doc/gitlab-basics/create-your-ssh-keys.md +++ b/doc/gitlab-basics/create-your-ssh-keys.md @@ -1,22 +1,22 @@ -# How to create your SSH keys +# Create and add your SSH public key -This topic describes how to create SSH keys. You do this to use Git over SSH instead of Git over HTTP. +This topic describes how to: -## Creating your SSH keys +- Create an SSH key pair to use with GitLab. +- Add the SSH public key to your GitLab account. -1. Go to your [command line](start-using-git.md) and follow the [instructions](../ssh/README.md) to generate your SSH key pair. -1. Log in to GitLab. -1. In the upper-right corner, click your avatar and select **Settings**. -1. On the **User Settings** menu, select **SSH keys**. -1. Paste the **public** key generated in the first step in the **Key** - text field. -1. Optionally, give it a descriptive title so that you can recognize it in the - event you add multiple keys. -1. Finally, click the **Add key** button to add it to GitLab. You will be able to see - its fingerprint, title, and creation date. +You do this to use [Git over SSH instead of Git over HTTP](https://git-scm.com/book/en/v2/Git-on-the-Server-The-Protocols). -  +## Creating your SSH key pair + +1. Go to your [command line](start-using-git.md). +1. Follow the [instructions](../ssh/README.md#generating-a-new-ssh-key-pair) to generate your SSH key pair. + +## Adding your SSH public key to GitLab + +To add the SSH public key to GitLab, +see [Adding an SSH key to your GitLab account](../ssh/README.md#adding-an-ssh-key-to-your-gitlab-account). NOTE: **Note:** Once you add a key, you cannot edit it. If the paste -didn't work, you need to remove the offending key and re-add it. +[didn't work](../ssh/README.md#testing-that-everything-is-set-up-correctly), you need to remove the key and re-add it. diff --git a/doc/gitlab-basics/img/profile_settings_ssh_keys_single_key.png b/doc/gitlab-basics/img/profile_settings_ssh_keys_single_key.png Binary files differdeleted file mode 100644 index 8014f1d5301..00000000000 --- a/doc/gitlab-basics/img/profile_settings_ssh_keys_single_key.png +++ /dev/null diff --git a/doc/gitlab-geo/configuration_source.md b/doc/gitlab-geo/configuration_source.md index f1aab86fadc..b46a2caea4a 100644 --- a/doc/gitlab-geo/configuration_source.md +++ b/doc/gitlab-geo/configuration_source.md @@ -1,5 +1,5 @@ --- -redirect_to: '../administration/geo/replication/configuration_source.md' +redirect_to: '../administration/geo/replication/configuration.md' --- -This document was moved to [another location](../administration/geo/replication/configuration_source.md). +This document was moved to [another location](../administration/geo/replication/configuration.md). diff --git a/doc/gitlab-geo/database_source.md b/doc/gitlab-geo/database_source.md index 3392d0f02c0..b4156dc4ec6 100644 --- a/doc/gitlab-geo/database_source.md +++ b/doc/gitlab-geo/database_source.md @@ -1,5 +1,5 @@ --- -redirect_to: '../administration/geo/replication/database_source.md' +redirect_to: '../administration/geo/replication/database.md' --- -This document was moved to [another location](../administration/geo/replication/database_source.md). +This document was moved to [another location](../administration/geo/replication/database.md). diff --git a/doc/integration/elasticsearch.md b/doc/integration/elasticsearch.md index d1d12dfd064..15176ede733 100644 --- a/doc/integration/elasticsearch.md +++ b/doc/integration/elasticsearch.md @@ -340,10 +340,30 @@ curl --request PUT localhost:9200/gitlab-production/_settings --data '{ } }' ``` -A force merge should be called after enabling the refreshing above: +A force merge should be called after enabling the refreshing above. + +For Elasticsearch 6.x, before proceeding with the force merge, the index should be in read-only mode: + +```bash +curl --request PUT localhost:9200/gitlab-production/_settings --data '{ + "settings": { + "index.blocks.write": true + } }' +``` + +Then, initiate the force merge: ```bash -curl --request POST 'http://localhost:9200/_forcemerge?max_num_segments=5' +curl --request POST 'http://localhost:9200/gitlab-production/_forcemerge?max_num_segments=5' +``` + +After this, if your index is in read-only, switch back to read-write: + +```bash +curl --request PUT localhost:9200/gitlab-production/_settings --data '{ + "settings": { + "index.blocks.write": false + } }' ``` Enable Elasticsearch search in **Admin > Settings > Integrations**. That's it. Enjoy it! diff --git a/doc/integration/salesforce.md b/doc/integration/salesforce.md index 8a99641a256..1ef43cfcece 100644 --- a/doc/integration/salesforce.md +++ b/doc/integration/salesforce.md @@ -1,27 +1,28 @@ -# SalesForce OmniAuth Provider +# Salesforce OmniAuth Provider -You can integrate your GitLab instance with [SalesForce](https://www.salesforce.com/) to enable users to login to your GitLab instance with their SalesForce account. +You can integrate your GitLab instance with [Salesforce](https://www.salesforce.com/) to enable users to log in to your GitLab instance with their Salesforce account. -## Create SalesForce Application +## Create a Salesforce Connected App -To enable SalesForce OmniAuth provider, you must use SalesForce's credentials for your GitLab instance. -To get the credentials (a pair of Client ID and Client Secret), you must register an application on SalesForces. +To enable Salesforce OmniAuth provider, you must use Salesforce's credentials for your GitLab instance. +To get the credentials (a pair of Client ID and Client Secret), you must [create a Connected App](https://help.salesforce.com/articleView?id=connected_app_create.htm&type=5) on Salesforce. -1. Sign in to [SalesForce](https://www.salesforce.com/). +1. Sign in to [Salesforce](https://login.salesforce.com/). -1. Navigate to **Platform Tools/Apps/App Manager** and click on **New Connected App**. +1. In Setup, enter `App Manager` in the Quick Find box, click **App Manager**, then click **New Connected App**. 1. Fill in the application details into the following fields: - **Connected App Name** and **API Name**: Set to any value but consider something like `<Organization>'s GitLab`, `<Your Name>'s GitLab`, or something else that is descriptive. + - **Contact Email**: Enter the contact email for Salesforce to use when contacting you or your support team. - **Description**: Description for the application. -  +  1. Select **API (Enable OAuth Settings)** and click on **Enable OAuth Settings**. 1. Fill in the application details into the following fields: - - **Callback URL**: The call callback URL. For example, `https://gitlab.example.com/users/auth/salesforce/callback`. + - **Callback URL**: The callback URL of your GitLab installation. For example, `https://gitlab.example.com/users/auth/salesforce/callback`. - **Selected OAuth Scopes**: Move **Access your basic information (id, profile, email, address, phone)** and **Allow access to your unique identifier (openid)** to the right column. -  +  1. Click **Save**. 1. On your GitLab server, open the configuration file. @@ -63,17 +64,16 @@ To get the credentials (a pair of Client ID and Client Secret), you must registe app_secret: 'SALESFORCE_CLIENT_SECRET' } ``` -1. Change `SALESFORCE_CLIENT_ID` to the Consumer Key from the SalesForce connected application page. -1. Change `SALESFORCE_CLIENT_SECRET` to the Consumer Secret from the SalesForce connected application page. -  +1. Change `SALESFORCE_CLIENT_ID` to the Consumer Key from the Salesforce connected application page. +1. Change `SALESFORCE_CLIENT_SECRET` to the Consumer Secret from the Salesforce connected application page. +  1. Save the configuration file. -1. [Reconfigure GitLab]( ../administration/restart_gitlab.md#omnibus-gitlab-reconfigure ) or [restart GitLab]( ../administration/restart_gitlab.md#installations-from-source ) for the changes to take effect if you - installed GitLab via Omnibus or from source respectively. +1. [Reconfigure GitLab]( ../administration/restart_gitlab.md#omnibus-gitlab-reconfigure ) or [restart GitLab]( ../administration/restart_gitlab.md#installations-from-source ) for the changes to take effect if you installed GitLab via Omnibus or from source respectively. -On the sign in page, there should now be a SalesForce icon below the regular sign in form. -Click the icon to begin the authentication process. SalesForce will ask the user to sign in and authorize the GitLab application. +On the sign in page, there should now be a Salesforce icon below the regular sign in form. +Click the icon to begin the authentication process. Salesforce will ask the user to sign in and authorize the GitLab application. If everything goes well, the user will be returned to GitLab and will be signed in. NOTE: **Note:** -GitLab requires the email address of each new user. Once the user is logged in using SalesForce, GitLab will redirect the user to the profile page where they will have to provide the email and verify the email. +GitLab requires the email address of each new user. Once the user is logged in using Salesforce, GitLab will redirect the user to the profile page where they will have to provide the email and verify the email.
\ No newline at end of file diff --git a/doc/raketasks/import.md b/doc/raketasks/import.md index bb316df5b9a..b59c06a24ea 100644 --- a/doc/raketasks/import.md +++ b/doc/raketasks/import.md @@ -16,7 +16,7 @@ The new folder needs to have git user ownership and read/write/execute access for git user and its group: ``` -sudo -u git mkdir /var/opt/gitlab/git-data/repository-import-<date>/new_group +sudo -u git mkdir -p /var/opt/gitlab/git-data/repository-import-<date>/new_group ``` ### Copy your bare repositories inside this newly created folder: diff --git a/doc/security/rack_attack.md b/doc/security/rack_attack.md index ad83dc05a93..66081d7e376 100644 --- a/doc/security/rack_attack.md +++ b/doc/security/rack_attack.md @@ -94,7 +94,7 @@ In case you want to remove a blocked IP, follow these steps: 1. Find the IPs that have been blocked in the production log: ```sh - grep "Rack_Attack" /var/log/gitlab/gitlab-rails/production.log + grep "Rack_Attack" /var/log/gitlab/gitlab-rails/auth.log ``` 1. Since the blacklist is stored in Redis, you need to open up `redis-cli`: diff --git a/doc/ssh/README.md b/doc/ssh/README.md index 9c4a391e8da..3bfebfc5d9b 100644 --- a/doc/ssh/README.md +++ b/doc/ssh/README.md @@ -55,7 +55,7 @@ As an admin, you can restrict By default, all keys are permitted, which is also the case for [GitLab.com](../user/gitlab_com/index.md#ssh-host-keys-fingerprints). -## ED25519 SSH keys +### ED25519 SSH keys Following [best practices](https://linux-audit.com/using-ed25519-openssh-keys-instead-of-dsa-rsa-ecdsa/), you should always favor [ED25519](https://ed25519.cr.yp.to/) SSH keys, since they @@ -65,7 +65,7 @@ They were introduced in OpenSSH 6.5, so any modern OS should include the option to create them. If for any reason your OS or the GitLab instance you interact with doesn't support this, you can fallback to RSA. -## RSA SSH keys +### RSA SSH keys RSA keys are the most common ones and therefore the most compatible with servers that may have an old OpenSSH version. Use them if the GitLab server @@ -166,12 +166,13 @@ Now, it's time to add the newly created public key to your GitLab account. NOTE: **Note:** If you opted to create an RSA key, the name might differ. -1. Add your public SSH key to your GitLab account by clicking your avatar - in the upper right corner and selecting **Settings**. From there on, - navigate to **SSH Keys** and paste your public key in the "Key" section. - If you created the key with a comment, this will appear under "Title". - If not, give your key an identifiable title like _Work Laptop_ or - _Home Workstation_, and click **Add key**. +1. Add your **public** SSH key to your GitLab account by: + 1. Clicking your avatar in the upper right corner and selecting **Settings**. + 1. Navigating to **SSH Keys** and pasting your **public** key in the **Key** field. If you: + + - Created the key with a comment, this will appear in the **Title** field. + - Created the key without a comment, give your key an identifiable title like _Work Laptop_ or _Home Workstation_. + 1. Click the **Add key** button. NOTE: **Note:** If you manually copied your public SSH key make sure you copied the entire @@ -305,7 +306,7 @@ who needs to know and configure the private key. GitLab administrators set up Global Deploy keys in the Admin area under the section **Deploy Keys**. Ensure keys have a meaningful title as that will be the primary way for project maintainers and owners to identify the correct Global -Deploy key to add. For instance, if the key gives access to a SaaS CI instance, +Deploy key to add. For instance, if the key gives access to a SaaS CI instance, use the name of that service in the key name if that is all it is used for. When creating Global Shared Deploy keys, give some thought to the granularity of keys - they could be of very narrow usage such as just a specific service or diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index 5a8744d71f9..4ceccaabf86 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -126,10 +126,6 @@ Auto Deploy, and Auto Monitoring will be silently skipped. ## Auto DevOps base domain -NOTE: **Note** -`AUTO_DEVOPS_DOMAIN` environment variable is deprecated and -[is scheduled to be removed](https://gitlab.com/gitlab-org/gitlab-ce/issues/56959). - The Auto DevOps base domain is required if you want to make use of [Auto Review Apps](#auto-review-apps) and [Auto Deploy](#auto-deploy). It can be defined in any of the following places: @@ -162,6 +158,12 @@ Auto DevOps base domain to `1.2.3.4.nip.io`. Once set up, all requests will hit the load balancer, which in turn will route them to the Kubernetes pods that run your application(s). +NOTE: **Note:** +From GitLab 11.8, `KUBE_INGRESS_BASE_DOMAIN` replaces `AUTO_DEVOPS_DOMAIN`. +Support for `AUTO_DEVOPS_DOMAIN` was [removed in GitLab +12.0](https://gitlab.com/gitlab-org/gitlab-ce/issues/56959). + + ## Using multiple Kubernetes clusters **[PREMIUM]** When using Auto DevOps, you may want to deploy different environments to @@ -209,10 +211,6 @@ and verifying that your app is deployed as a review app in the Kubernetes cluster with the `review/*` environment scope. Similarly, you can check the other environments. -NOTE: **Note:** -From GitLab 11.8, `KUBE_INGRESS_BASE_DOMAIN` replaces `AUTO_DEVOPS_DOMAIN`. -`AUTO_DEVOPS_DOMAIN` [is scheduled to be removed](https://gitlab.com/gitlab-org/gitlab-ce/issues/56959). - ## Enabling/Disabling Auto DevOps When first using Auto Devops, review the [requirements](#requirements) to ensure all necessary components to make @@ -734,7 +732,6 @@ also be customized, and you can easily use a [custom buildpack](#custom-buildpac | **Variable** | **Description** | | ------------ | --------------- | -| `AUTO_DEVOPS_DOMAIN` | The [Auto DevOps domain](#auto-devops-base-domain). By default, set automatically by the [Auto DevOps setting](#enablingdisabling-auto-devops). This variable is deprecated and [is scheduled to be removed](https://gitlab.com/gitlab-org/gitlab-ce/issues/56959). Use `KUBE_INGRESS_BASE_DOMAIN` instead. | | `AUTO_DEVOPS_CHART` | The Helm Chart used to deploy your apps; defaults to the one [provided by GitLab](https://gitlab.com/gitlab-org/charts/auto-deploy-app). | | `AUTO_DEVOPS_CHART_REPOSITORY` | The Helm Chart repository used to search for charts; defaults to `https://charts.gitlab.io`. | | `AUTO_DEVOPS_CHART_REPOSITORY_NAME` | From Gitlab 11.11, this variable can be used to set the name of the helm repository; defaults to "gitlab" | diff --git a/doc/university/README.md b/doc/university/README.md index cf13246067f..61ed72d25fb 100644 --- a/doc/university/README.md +++ b/doc/university/README.md @@ -51,10 +51,10 @@ The curriculum is composed of GitLab videos, screencasts, presentations, project #### 1.5. Migrating from other Source Control -1. [Migrating from BitBucket/Stash](https://docs.gitlab.com/ee/user/project/import/bitbucket.html) -1. [Migrating from GitHub](https://docs.gitlab.com/ee/user/project/import/github.html) -1. [Migrating from SVN](https://docs.gitlab.com/ee/user/project/import/svn.html) -1. [Migrating from Fogbugz](https://docs.gitlab.com/ee/user/project/import/fogbugz.html) +1. [Migrating from BitBucket/Stash](../user/project/import/bitbucket.md) +1. [Migrating from GitHub](../user/project/import/github.md) +1. [Migrating from SVN](../user/project/import/svn.md) +1. [Migrating from Fogbugz](../user/project/import/fogbugz.md) #### 1.6. GitLab Inc. @@ -92,7 +92,7 @@ The curriculum is composed of GitLab videos, screencasts, presentations, project 1. [Using any Static Site Generator with GitLab Pages](https://about.gitlab.com/2016/06/17/ssg-overview-gitlab-pages-part-3-examples-ci/) 1. [Securing GitLab Pages with SSL](https://about.gitlab.com/2016/06/24/secure-gitlab-pages-with-startssl/) -1. [GitLab Pages Documentation](https://docs.gitlab.com/ce/user/project/pages/) +1. [GitLab Pages Documentation](../user/project/pages/index.md) #### 2.2. GitLab Issues @@ -131,7 +131,7 @@ The curriculum is composed of GitLab videos, screencasts, presentations, project 1. [GitLab Flow vs Forking in GitLab - Video](https://www.youtube.com/watch?v=UGotqAUACZA) 1. [GitLab Flow Overview](https://about.gitlab.com/2014/09/29/gitlab-flow/) 1. [Always Start with an Issue](https://about.gitlab.com/2016/03/03/start-with-an-issue/) -1. [GitLab Flow Documentation](https://docs.gitlab.com/ee/workflow/gitlab_flow.html) +1. [GitLab Flow Documentation](../workflow/gitlab_flow.md) #### 2.5. GitLab Comparisons @@ -191,10 +191,10 @@ The curriculum is composed of GitLab videos, screencasts, presentations, project #### 3.9. Integrations 1. [How to Integrate JIRA and Jenkins with GitLab - Video](https://gitlabmeetings.webex.com/gitlabmeetings/ldr.php?RCID=44b548147a67ab4d8a62274047146415) -1. [How to Integrate Jira with GitLab](https://docs.gitlab.com/ce/user/project/integrations/jira.html) -1. [How to Integrate Jenkins with GitLab](https://docs.gitlab.com/ee/integration/jenkins.html) -1. [How to Integrate Bamboo with GitLab](https://docs.gitlab.com/ce/user/project/integrations/bamboo.html) -1. [How to Integrate Slack with GitLab](https://docs.gitlab.com/ce/user/project/integrations/slack.html) +1. [How to Integrate Jira with GitLab](../user/project/integrations/jira.md) +1. [How to Integrate Jenkins with GitLab](../integration/jenkins.md) +1. [How to Integrate Bamboo with GitLab](../user/project/integrations/bamboo.md) +1. [How to Integrate Slack with GitLab](../user/project/integrations/slack.md) 1. [How to Integrate Convox with GitLab](https://about.gitlab.com/2016/06/09/continuous-delivery-with-gitlab-and-convox/) 1. [Getting Started with GitLab and Shippable CI](https://about.gitlab.com/2016/05/05/getting-started-gitlab-and-shippable/) diff --git a/doc/university/glossary/README.md b/doc/university/glossary/README.md index 0af2f8d2f54..f15b0107de5 100644 --- a/doc/university/glossary/README.md +++ b/doc/university/glossary/README.md @@ -62,7 +62,7 @@ Entry level [subscription](https://about.gitlab.com/pricing/) for GitLab EE curr ### Bitbucket -Atlassian's web hosting service for Git and Mercurial Projects. Read about [migrating](https://docs.gitlab.com/ce/workflow/importing/import_projects_from_bitbucket.html) from BitBucket to a GitLab instance. +Atlassian's web hosting service for Git and Mercurial Projects. Read about [migrating](../../user/project/import/bitbucket.md) from BitBucket to a GitLab instance. ### Branch @@ -70,10 +70,10 @@ A branch is a parallel version of a repository. This allows you to work on the r ### Branded Login -Having your own logo on [your GitLab instance login page](https://docs.gitlab.com/ee/customization/branded_login_page.html) instead of the GitLab logo. +Having your own logo on [your GitLab instance login page](../../customization/branded_login_page.md) instead of the GitLab logo. ### Job triggers (Build Triggers) -These protect your code base against breaks, for instance when a team is working on the same project. Learn about [setting up](https://docs.gitlab.com/ce/ci/triggers/README.html) job triggers. +These protect your code base against breaks, for instance when a team is working on the same project. Learn about [setting up](../../ci/triggers/README.md) job triggers. ### CEPH @@ -149,7 +149,7 @@ As in "specify [dependencies](https://gitlab.com/gitlab-org/gitlab-ce/issues/147 ### Deploy Keys -A [SSH key](https://docs.gitlab.com/ce/gitlab-basics/create-your-ssh-keys.html)stored on your server that grants access to a single GitLab repository. This is used by a GitLab runner to clone a project's code so that tests can be run against the checked out code. +A [SSH key](../../gitlab-basics/create-your-ssh-keys.md)stored on your server that grants access to a single GitLab repository. This is used by a GitLab runner to clone a project's code so that tests can be run against the checked out code. ### Developer @@ -169,7 +169,7 @@ A folder used for storing multiple files. ### Docker Container Registry -A [feature](https://docs.gitlab.com/ce/user/project/container_registry.html) of [GitLab projects](https://about.gitlab.com/2016/05/23/gitlab-container-registry/). Containers wrap up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, system libraries – anything you can install on a server. This guarantees that it will always run the same, regardless of the environment it is running in. +A [feature](../../user/project/container_registry.md) of [GitLab projects](https://about.gitlab.com/2016/05/23/gitlab-container-registry/). Containers wrap up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, system libraries – anything you can install on a server. This guarantees that it will always run the same, regardless of the environment it is running in. ### Dynamic Environment (review apps) @@ -191,7 +191,7 @@ First Byte (sometimes referred to as time to first byte or [TTFB](https://en.wik ### Fork -Your [own copy](https://docs.gitlab.com/ce/workflow/forking_workflow.html) of a repository that allows you to make changes to the repository without affecting the original. +Your [own copy](../../workflow/forking_workflow.md) of a repository that allows you to make changes to the repository without affecting the original. ### Funnel, or: TOFU, MOFU, BOFU @@ -221,7 +221,7 @@ A single-tenant solution that provides GitLab CE or EE as a managed service. Git ### GitHub -A web-based Git repository hosting service with an enterprise offering. Its main features are: issue tracking, pull request with code review, abundancy of integrations and wiki. It offers free public repos, private repos and enterprise services are paid. Read about [importing a project](https://docs.gitlab.com/ce/workflow/importing/import_projects_from_github.html) from GitHub to GitLab. +A web-based Git repository hosting service with an enterprise offering. Its main features are: issue tracking, pull request with code review, abundancy of integrations and wiki. It offers free public repos, private repos and enterprise services are paid. Read about [importing a project](../../workflow/importing/import_projects_from_github.md) from GitHub to GitLab. ### GitLab CE @@ -241,7 +241,7 @@ Our free SaaS for public and private repositories. ### GitLab Geo -Allows you to replicate your GitLab instance to other geographical locations as a read-only fully operational version. It [can be used](https://docs.gitlab.com/ee/administration/geo/replication/index.html) for cloning and fetching projects, in addition to reading any data. This will make working with large repositories over large distances much faster. +Allows you to replicate your GitLab instance to other geographical locations as a read-only fully operational version. It [can be used](../../administration/geo/replication/index.md) for cloning and fetching projects, in addition to reading any data. This will make working with large repositories over large distances much faster. ### GitLab High Availability @@ -299,15 +299,15 @@ An [application layer protocol](http://www.irchelp.org/) that facilitates commun ### Issue Tracker -A [tool](https://docs.gitlab.com/ee/integration/external-issue-tracker.html) used to manage, organize, and maintain a list of issues, making it easier for an organization to manage. +A [tool](../../integration/external-issue-tracker.md) used to manage, organize, and maintain a list of issues, making it easier for an organization to manage. ### Jenkins -An Open Source CI tool written using the Java programming language. [Jenkins](https://jenkins.io/) does the same job as GitLab CI, Bamboo, and Travis CI. It is extremely popular. Related [documentation](https://docs.gitlab.com/ee/integration/jenkins.html). +An Open Source CI tool written using the Java programming language. [Jenkins](https://jenkins.io/) does the same job as GitLab CI, Bamboo, and Travis CI. It is extremely popular. Related [documentation](../../integration/jenkins.md). ### Jira -Atlassian's [project management software](https://www.atlassian.com/software/jira), i.e. a complex issue tracker. GitLab [can be configured](https://docs.gitlab.com/ee/project_services/jira.html) to interact with JIRA Core either using an on-premise instance or the SaaS solution that Atlassian offers. +Atlassian's [project management software](https://www.atlassian.com/software/jira), i.e. a complex issue tracker. GitLab [can be configured](../../project_services/jira.md) to interact with JIRA Core either using an on-premise instance or the SaaS solution that Atlassian offers. ### JUnit @@ -323,7 +323,7 @@ An open source container cluster manager originally designed by Google. It's bas ### Labels -An [identifier](https://docs.gitlab.com/ce/user/project/labels.html) to describe a group of one or more specific file revisions. +An [identifier](../../user/project/labels.md) to describe a group of one or more specific file revisions. ### Lightweight Directory Access Protocol (LDAP) @@ -331,7 +331,7 @@ An [identifier](https://docs.gitlab.com/ce/user/project/labels.html) to describe ### LDAP User Authentication -GitLab [integrates](https://docs.gitlab.com/ce/administration/auth/ldap.html) with LDAP to support user authentication. This enables GitLab to sign in people from an LDAP server (i.e., allowing people whose names are on the electronic user directory server to be able to use their LDAP accounts to login.) +GitLab [integrates](../../administration/auth/ldap.md) with LDAP to support user authentication. This enables GitLab to sign in people from an LDAP server (i.e., allowing people whose names are on the electronic user directory server to be able to use their LDAP accounts to login.) ### LDAP Group Sync @@ -381,9 +381,9 @@ Takes changes from one branch, and [applies them](https://git-scm.com/docs/git-m [Arises](https://about.gitlab.com/2016/09/06/resolving-merge-conflicts-from-the-gitlab-ui/) when a merge can't be performed cleanly between two versions of the same file. -#### Merge Request +#### Merge Request (MR) -[Takes changes](https://docs.gitlab.com/ce/gitlab-basics/add-merge-request.html) from one branch, and applies them into another branch. +[Takes changes](../../gitlab-basics/add-merge-request.md) from one branch, and applies them into another branch. ### Meteor @@ -395,11 +395,11 @@ Allow you to [organize issues](../../user/project/milestones/index.md) and merge ### Mirror Repositories -A project that is set up to automatically have its branches, tags, and commits [updated from an upstream repository](https://docs.gitlab.com/ee/workflow/repository_mirroring.html). This is useful when a repository you're interested in is located on a different server, and you want to be able to browse its content and activity using the familiar GitLab interface. +A project that is set up to automatically have its branches, tags, and commits [updated from an upstream repository](../../workflow/repository_mirroring.md). This is useful when a repository you're interested in is located on a different server, and you want to be able to browse its content and activity using the familiar GitLab interface. ### MIT License -A type of software license. It lets people do anything with your code with proper attribution and without warranty. It is the most common license for open source applications written in Ruby on Rails. GitLab CE is issued under this [license](https://docs.gitlab.com/ce/development/licensing.html). This means you can download the code, modify it as you want, and even build a new commercial product using the underlying code and it's not illegal. The only condition is that there is no form of warranty provided by GitLab so whatever happens when you use the code is your own problem. +A type of software license. It lets people do anything with your code with proper attribution and without warranty. It is the most common license for open source applications written in Ruby on Rails. GitLab CE is issued under this [license](../../development/licensing.md). This means you can download the code, modify it as you want, and even build a new commercial product using the underlying code and it's not illegal. The only condition is that there is no form of warranty provided by GitLab so whatever happens when you use the code is your own problem. ### Mondo Rescue @@ -427,7 +427,7 @@ A web [server](https://www.nginx.com/resources/wiki/) (pronounced "engine x"). [ ### OAuth -An open standard for authorization, commonly used as a way for internet users to log into third party websites using their Microsoft, Google, Facebook or Twitter accounts without exposing their password. GitLab [is](https://docs.gitlab.com/ce/integration/oauth_provider.html) an OAuth2 authentication service provider. +An open standard for authorization, commonly used as a way for internet users to log into third party websites using their Microsoft, Google, Facebook or Twitter accounts without exposing their password. GitLab [is](../../integration/oauth_provider.md) an OAuth2 authentication service provider. ### Omnibus Packages @@ -479,11 +479,11 @@ An [object-relational](https://en.wikipedia.org/wiki/PostgreSQL) database. Toute ### Protected Branches -A [feature](https://docs.gitlab.com/ce/user/project/protected_branches.html) that protects branches from unauthorized pushes, force pushing or deletion. +A [feature](../../user/project/protected_branches.md) that protects branches from unauthorized pushes, force pushing or deletion. ### Protected Tags -A [feature](https://docs.gitlab.com/ce/user/project/protected_tags.html) that protects tags from unauthorized creation, update or deletion +A [feature](../../user/project/protected_tags.md) that protects tags from unauthorized creation, update or deletion ### Pull @@ -547,7 +547,7 @@ Actual build machines/containers that [run and execute tests](https://gitlab.com ### Sidekiq -The background job processor GitLab [uses](https://docs.gitlab.com/ce/administration/troubleshooting/sidekiq.html) to asynchronously run tasks. +The background job processor GitLab [uses](../../administration/troubleshooting/sidekiq.md) to asynchronously run tasks. ### Software as a service (SaaS) @@ -567,7 +567,7 @@ The board used to track the status and progress of each of the sprint backlog it ### Shell -Terminal on Mac OSX, GitBash on Windows, or Linux Terminal on Linux. You [use git](https://docs.gitlab.com/ce/gitlab-basics/start-using-git.html) and make changes to GitLab projects in your shell. You [use git](https://docs.gitlab.com/ce/gitlab-basics/start-using-git.html) and make changes to GitLab projects in your shell. +Terminal on Mac OSX, GitBash on Windows, or Linux Terminal on Linux. You [use git](../../gitlab-basics/start-using-git.md) and make changes to GitLab projects in your shell. You [use git](../../gitlab-basics/start-using-git.md) and make changes to GitLab projects in your shell. ### Shell command runner @@ -577,7 +577,7 @@ The tenant purchases their own copy of the software and the software can be cust ### Slack -Real time messaging app for teams that is used internally by GitLab team members. GitLab users can enable [Slack integration](https://docs.gitlab.com/ce/project_services/slack.html) to trigger push, issue, and merge request events among others. +Real time messaging app for teams that is used internally by GitLab team members. GitLab users can enable [Slack integration](../../project_services/slack.md) to trigger push, issue, and merge request events among others. ### Slash commands @@ -595,7 +595,7 @@ Program code as typed by a computer programmer (i.e. it has not yet been compile ### SSH Key -A unique identifier of a computer. It is used to identify computers without the need for a password (e.g., On GitLab I have [added the ssh key](https://docs.gitlab.com/ce/gitlab-basics/create-your-ssh-keys.html) of all my work machines so that the GitLab instance knows that it can accept code pushes and pulls from this trusted machines whose keys are I have added.) +A unique identifier of a computer. It is used to identify computers without the need for a password (e.g., On GitLab I have [added the ssh key](../../gitlab-basics/create-your-ssh-keys.md) of all my work machines so that the GitLab instance knows that it can accept code pushes and pulls from this trusted machines whose keys are I have added.) ### Single Sign On (SSO) @@ -627,11 +627,11 @@ A program that allows you to perform superuser/administrator actions on Unix Ope ### Subversion (SVN) -An open source version control system. Read about [migrating from SVN](https://docs.gitlab.com/ce/workflow/importing/migrating_from_svn.html) to GitLab using SubGit. +An open source version control system. Read about [migrating from SVN](../../workflow/importing/migrating_from_svn.md) to GitLab using SubGit. ### Tag -[Represents](https://docs.gitlab.com/ce/api/tags.html) a version of a particular branch at a moment in time. +[Represents](../../api/tags.md) a version of a particular branch at a moment in time. ### Tenancy @@ -673,7 +673,7 @@ Version control is a system that records changes to a file or set of files over ### Virtual Private Cloud (VPC) -A [VPC](https://docs.gitlab.com/ce/university/glossary/README.html#virtual-private-cloud-vpc) is an on demand configurable pool of shared computing resources allocated within a public cloud environment, providing some isolation between the different users using the resources. GitLab users need to create a new Amazon VPC in order to [set up High Availability](https://docs.gitlab.com/ce/university/high-availability/aws/). +A [VPC](#virtual-private-cloud-vpc) is an on demand configurable pool of shared computing resources allocated within a public cloud environment, providing some isolation between the different users using the resources. GitLab users need to create a new Amazon VPC in order to [set up High Availability](../../install/aws/index.md). ### Virtual private server (VPS) @@ -689,7 +689,7 @@ A [model](http://www.umsl.edu/~hugheyd/is6840/waterfall.html) of building softwa ### Webhooks -A way for an app to [provide](https://docs.gitlab.com/ce/user/project/integrations/webhooks.html) other applications with real-time information (e.g., send a message to a slack channel when a commit is pushed.) Read about setting up [custom git hooks](https://gitlab.com/help/administration/custom_hooks.md) for when webhooks are insufficient. +A way for an app to [provide](../../user/project/integrations/webhooks.md) other applications with real-time information (e.g., send a message to a slack channel when a commit is pushed.) Read about setting up [custom git hooks](../../administration/custom_hooks.md) for when webhooks are insufficient. ### Wiki diff --git a/doc/university/high-availability/aws/README.md b/doc/university/high-availability/aws/README.md index 01d5073ab7e..fa04e988042 100644 --- a/doc/university/high-availability/aws/README.md +++ b/doc/university/high-availability/aws/README.md @@ -390,5 +390,5 @@ some redundancy options but it might also imply Geographic replication. There is a lot of ground yet to cover so have a read through these other resources and feel free to open an issue to request additional material. -- [GitLab High Availability](http://docs.gitlab.com/ce/administration/high_availability/README.html#sts=High%20Availability) -- [GitLab Geo](https://docs.gitlab.com/ee/administration/geo/replication/index.html) +- [GitLab High Availability](../../../administration/high_availability/README.md) +- [GitLab Geo](../../../administration/geo/replication/index.md) diff --git a/doc/university/support/README.md b/doc/university/support/README.md index feb90ae9bad..c8ade54a77c 100644 --- a/doc/university/support/README.md +++ b/doc/university/support/README.md @@ -61,28 +61,27 @@ Sometimes we need to upgrade customers from old versions of GitLab to latest, so - Users - Groups - Projects - - [Backup using our Backup rake task](https://docs.gitlab.com/ce/raketasks/backup_restore.html#create-a-backup-of-the-gitlab-system) + - [Backup using our Backup rake task](../../raketasks/backup_restore.md#creating-a-backup-of-the-gitlab-system) - [Upgrade to 5.0 source using our Upgrade documentation](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/doc/update/4.2-to-5.0.md) - [Upgrade to 5.1 source](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/doc/update/5.0-to-5.1.md) - [Upgrade to 6.0 source](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/doc/update/5.1-to-6.0.md) - [Upgrade to 7.14 source](https://gitlab.com/gitlab-org/gitlab-ee/blob/master/doc/update/6.x-or-7.x-to-7.14.md) - - [Backup using our Backup rake task](https://docs.gitlab.com/ce/raketasks/backup_restore.html#create-a-backup-of-the-gitlab-system) - - [Perform the MySQL to PostgreSQL migration to convert your backup](https://docs.gitlab.com/ce/update/mysql_to_postgresql.html#converting-a-gitlab-backup-file-from-mysql-to-postgres) + - [Perform the MySQL to PostgreSQL migration to convert your backup](../../update/mysql_to_postgresql.md) - [Upgrade to Omnibus 7.14](https://docs.gitlab.com/omnibus/update/README.html#upgrading-from-a-non-omnibus-installation-to-an-omnibus-installation) - - [Restore backup using our Restore rake task](https://docs.gitlab.com/ce/raketasks/backup_restore.html#restore-a-previously-created-backup) + - [Restore backup using our Restore rake task](../../raketasks/backup_restore.md#restore) - [Upgrade to latest EE](https://about.gitlab.com/downloads-ee) - (GitLab inc. only) Acquire and apply a license for the Enterprise Edition product, ask in #support -- Perform a downgrade from [EE to CE](https://docs.gitlab.com/ee/downgrade_ee_to_ce/README.html) +- Perform a downgrade from [EE to CE](../../downgrade_ee_to_ce/README.md) #### Start to learn about some of the integrations that we support Our integrations add great value to GitLab. User questions often relate to integrating GitLab with existing external services and the configuration involved - Learn about our Integrations (specially, not only): - - [LDAP](https://docs.gitlab.com/ee/integration/ldap.html) - - [JIRA](https://docs.gitlab.com/ee/project_services/jira.html) - - [Jenkins](https://docs.gitlab.com/ee/integration/jenkins.html) - - [SAML](https://docs.gitlab.com/ce/integration/saml.html) + - [LDAP](../../integration/ldap.md) + - [JIRA](../../project_services/jira.md) + - [Jenkins](../../integration/jenkins.md) + - [SAML](../../integration/saml.md) #### Goals @@ -94,8 +93,8 @@ Our integrations add great value to GitLab. User questions often relate to integ #### Understand the gathering of diagnostics for GitLab instances - Learn about the GitLab checks that are available - - [Environment Information and maintenance checks](https://docs.gitlab.com/ce/raketasks/maintenance.html) - - [GitLab check](https://docs.gitlab.com/ce/raketasks/check.html) + - [Environment Information and maintenance checks](../../raketasks/maintenance.md) + - [GitLab check](../../raketasks/check.md) - Omnibus commands - [Status](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/maintenance/README.md#get-service-status) - [Starting and stopping services](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/maintenance/README.md#starting-and-stopping) @@ -170,11 +169,11 @@ Some tickets need specific knowledge or a deep understanding of a particular com Move on to understanding some of GitLab's more advanced features. You can make use of GitLab.com to understand the features from an end-user perspective and then use your own instance to understand setup and configuration of the feature from an Administrative perspective -- Set up and try [Git LFS](https://docs.gitlab.com/ee/workflow/lfs/manage_large_binaries_with_git_lfs.html) -- Get to know the [GitLab API](https://docs.gitlab.com/ee/api/README.html), its capabilities and shortcomings -- Learn how to [migrate from SVN to Git](https://docs.gitlab.com/ee/workflow/importing/migrating_from_svn.html) -- Set up [GitLab CI](https://docs.gitlab.com/ee/ci/quick_start/README.html) -- Create your first [GitLab Page](https://docs.gitlab.com/ce/administration/pages/) +- Set up and try [Git LFS](../../workflow/lfs/manage_large_binaries_with_git_lfs.md) +- Get to know the [GitLab API](../../api/README.md), its capabilities and shortcomings +- Learn how to [migrate from SVN to Git](../../user/project/import/svn.md) +- Set up [GitLab CI](../../ci/quick_start/README.md) +- Create your first [GitLab Page](../../administration/pages/index.md) - Get to know the GitLab Codebase by reading through the source code: - Find the differences between the [EE codebase](https://gitlab.com/gitlab-org/gitlab-ce) and the [CE codebase](https://gitlab.com/gitlab-org/gitlab-ce) diff --git a/doc/user/admin_area/geo_nodes.md b/doc/user/admin_area/geo_nodes.md index 776ab139c64..fb0f9a3285d 100644 --- a/doc/user/admin_area/geo_nodes.md +++ b/doc/user/admin_area/geo_nodes.md @@ -1,9 +1,13 @@ +--- +type: howto +--- + # Geo nodes admin area **[PREMIUM ONLY]** -For more information about setting up GitLab Geo, read the -[Geo documentation](https://docs.gitlab.com/ee/administration/geo/replication/index.html). +You can configure various settings for GitLab Geo nodes. For more information, see +[Geo documentation](https://docs.gitlab.com/ee/administration/geo/replication/index.md). -When you're done, you can navigate to **Admin area > Geo** (`/admin/geo/nodes`). +On the primary node, go to **Admin area > Geo**. On secondary nodes, go to **Admin area > Geo > Nodes**. ## Common settings @@ -68,3 +72,15 @@ a unique `name` is set for each Geo node. The `gitlab.rb` setting The load balancer must use sticky sessions in order to avoid authentication failures and cross site request errors. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. -->
\ No newline at end of file diff --git a/doc/user/admin_area/index.md b/doc/user/admin_area/index.md index dd4e96c1f4a..4ed1287abbc 100644 --- a/doc/user/admin_area/index.md +++ b/doc/user/admin_area/index.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # GitLab Admin Area **[CORE ONLY]** The Admin Area provides a web UI for administering some features of GitLab self-managed instances. @@ -16,7 +20,7 @@ The Admin Area is made up of the following sections: | Section | Description | |:---------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------| -| Overview | View your GitLab [Dashboard](#admin-dashboard), and administer [projects](#administer-projects), users, groups, jobs, runners, and Gitaly servers. | +| Overview | View your GitLab [Dashboard](#admin-dashboard), and administer [projects](#administer-projects), [users](#administer-users), groups, jobs, runners, and Gitaly servers. | | Monitoring | View GitLab system information, and information on background jobs, logs, [health checks](monitoring/health_check.md), request profiles, and audit logs. | | Messages | Send and manage [broadcast messages](broadcast_messages.md) for your users. | | System Hooks | Configure [system hooks](../../system_hooks/system_hooks.md) for many events. | @@ -61,16 +65,52 @@ Click the **All**, **Private**, **Internal**, or **Public** tab to list only pro criteria. By default, all projects are listed, in reverse order of when they were last updated. For each -project, the name, namespace, description, and size is listed, also options to **Edit** or -**Delete** it. +project, the following information is listed: + +- Name. +- Namespace. +- Description. +- Size, updated every 15 minutes at most. + +Projects can be edited or deleted. + +The list of projects can be sorted by: -Sort projects by **Name**, **Last created**, **Oldest created**, **Last updated**, **Oldest -updated**, **Owner**, and choose to hide or show archived projects. +- Name. +- Last created. +- Oldest created. +- Last updated. +- Oldest updated. +- Owner. + +A user can choose to hide or show archived projects in the list. In the **Filter by name** field, type the project name you want to find, and GitLab will filter them as you type. Select from the **Namespace** dropdown to filter only projects in that namespace. -You can combine the filter options. For example, click the **Public** tab, and enter `score` in -the **Filter by name...** input box to list only public projects with `score` in their name. +You can combine the filter options. For example, to list only public projects with `score` in their name: + +1. Click the **Public** tab. +1. Enter `score` in the **Filter by name...** input box. + +## Administer Users + +You can administer all users in the GitLab instance from the Admin Area's Users page. + +To access the Users page, go to **Admin Area > Overview > Users**. + +Click the **Active**, **Admins**, **2FA Enabled**, or **2FA Disabled**, **External**, or +**Without projects** tab to list only users of that criteria. + +For each user, their username, email address, are listed, also the date their account was +created and the date of last activity. To edit a user, click the **Edit** button in that user's +row. To delete the user, or delete the user and their contributions, click the cog dropdown in +that user's row, and select the desired option. + +To change the sort order, click the sort dropdown and select the desired order. By default the sort dropdown shows **Name**. + +To search for users, enter your criteria in the search field. The user search is case +insensitive, and applies partial matching to name and username. To search for an email address, +you must provide the complete email address. diff --git a/doc/user/admin_area/labels.md b/doc/user/admin_area/labels.md index e383142c33e..eba27548f86 100644 --- a/doc/user/admin_area/labels.md +++ b/doc/user/admin_area/labels.md @@ -1,9 +1,25 @@ +--- +type: reference +--- + # Labels administration **[CORE ONLY]** -## Default Labels +In the Admin Area, you can manage labels for the GitLab instance. For more details, see [Labels](../project/labels.md). -### Define your own default Label Set +## Default Labels -Labels that are created within the Labels view on the Admin Dashboard will be automatically added to each new project. +Labels created in the Admin Area become available to each _new_ project.  + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/admin_area/license.md b/doc/user/admin_area/license.md index 49959a9daef..1e8ce04da92 100644 --- a/doc/user/admin_area/license.md +++ b/doc/user/admin_area/license.md @@ -1,3 +1,7 @@ +--- +type: howto +--- + # Activate all GitLab Enterprise Edition functionality with a license **[STARTER ONLY]** To activate all GitLab Enterprise Edition (EE) functionality, you need to upload @@ -108,3 +112,15 @@ but only the latest license will be used as the active license. [free trial]: https://about.gitlab.com/free-trial/ [pricing]: https://about.gitlab.com/pricing/ + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. -->
\ No newline at end of file diff --git a/doc/user/admin_area/monitoring/health_check.md b/doc/user/admin_area/monitoring/health_check.md index e183898dfb1..ff056490653 100644 --- a/doc/user/admin_area/monitoring/health_check.md +++ b/doc/user/admin_area/monitoring/health_check.md @@ -1,12 +1,14 @@ -# Health Check +--- +type: concepts, howto +--- -> **Notes:** +# Health Check -> - Liveness and readiness probes were [introduced][ce-10416] in GitLab 9.1. -> - The `health_check` endpoint was [introduced][ce-3888] in GitLab 8.8 and was -> be deprecated in GitLab 9.1. -> - [Access token](#access-token-deprecated) has been deprecated in GitLab 9.4 -> in favor of [IP whitelist](#ip-whitelist) +> - Liveness and readiness probes were [introduced][ce-10416] in GitLab 9.1. +> - The `health_check` endpoint was [introduced][ce-3888] in GitLab 8.8 and was +> be deprecated in GitLab 9.1. +> - [Access token](#access-token-deprecated) has been deprecated in GitLab 9.4 +> in favor of [IP whitelist](#ip-whitelist). GitLab provides liveness and readiness probes to indicate service health and reachability to required services. These probes report on the status of the @@ -17,8 +19,7 @@ traffic until the system is ready or restart the container as needed. ## IP whitelist To access monitoring resources, the requesting client IP needs to be included in a whitelist. - -[Read how to add IPs to a whitelist for the monitoring endpoints][admin]. +For details, see [how to add IPs to a whitelist for the monitoring endpoints](../../../administration/monitoring/ip_whitelist.md). ## Using the endpoints @@ -30,7 +31,7 @@ With default whitelist settings, the probes can be accessed from localhost using The first endpoint, `health`, only checks whether the application server is running. It does not verify the database or other services are running. A successful response will return a 200 status code with the following message: -``` +```text GitLab OK ``` @@ -87,9 +88,8 @@ will return a valid successful HTTP status code, and a `success` message. ## Access token (Deprecated) ->**Note:** -Access token has been deprecated in GitLab 9.4 -in favor of [IP whitelist](#ip-whitelist) +> NOTE: **Note:** +> Access token has been deprecated in GitLab 9.4 in favor of [IP whitelist](#ip-whitelist). An access token needs to be provided while accessing the probe endpoints. The current accepted token can be found under the **Admin area ➔ Monitoring ➔ Health check** @@ -99,14 +99,25 @@ accepted token can be found under the **Admin area ➔ Monitoring ➔ Health che The access token can be passed as a URL parameter: -``` +```text https://gitlab.example.com/-/readiness?token=ACCESS_TOKEN ``` +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> + [ce-10416]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10416 [ce-3888]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3888 [pingdom]: https://www.pingdom.com [nagios-health]: https://nagios-plugins.org/doc/man/check_http.html [newrelic-health]: https://docs.newrelic.com/docs/alerts/alert-policies/downtime-alerts/availability-monitoring [kubernetes]: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/ -[admin]: ../../../administration/monitoring/ip_whitelist.md diff --git a/doc/user/admin_area/settings/account_and_limit_settings.md b/doc/user/admin_area/settings/account_and_limit_settings.md index 4a5bfb6b677..1d355824760 100644 --- a/doc/user/admin_area/settings/account_and_limit_settings.md +++ b/doc/user/admin_area/settings/account_and_limit_settings.md @@ -1,25 +1,29 @@ +--- +type: reference +--- + # Account and limit settings ## Repository size limit **[STARTER]** -> [Introduced][ee-740] in [GitLab Enterprise Edition 8.12][ee-8.12]. +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/740) in [GitLab Enterprise Edition 8.12](https://about.gitlab.com/2016/09/22/gitlab-8-12-released/#limit-project-size-ee). +> Available in [GitLab Starter](https://about.gitlab.com/pricing/). Repositories within your GitLab instance can grow quickly, especially if you are -using LFS. Their size can grow exponentially and eat up your storage device quite -fast. +using LFS. Their size can grow exponentially, rapidly consuming available storage. -In order to avoid this from happening, you can set a hard limit for your -repositories' size. This limit can be set globally, per group, or per project, -with per project limits taking the highest priority. +To avoid this from happening, you can set a hard limit for your repositories' size. +This limit can be set globally, per group, or per project, with per project limits +taking the highest priority. -There are numerous cases where you'll need to set up a limit for repository size. +There are numerous use cases where you might set up a limit for repository size. For instance, consider the following workflow: -1. Your team develops apps which demand large files to be stored in +1. Your team develops apps which require large files to be stored in the application repository. -1. Although you have enabled [Git LFS](../../../workflow/lfs/manage_large_binaries_with_git_lfs.html#git-lfs) +1. Although you have enabled [Git LFS](../../../workflow/lfs/manage_large_binaries_with_git_lfs.md#git-lfs) to your project, your storage has grown significantly. -1. Before you blow your storage limit up, you set up a limit of 10 GB +1. Before you exceed available storage, you set up a limit of 10 GB per repository. ### How it works @@ -42,12 +46,19 @@ subsequent push will be denied. LFS objects, however, can be checked on first push and **will** be rejected if the sum of their sizes exceeds the maximum allowed repository size. -For more manually purging the files, read the docs on -[reducing the repository size using Git][repo-size]. +For details on manually purging files, see [reducing the repository size using Git](../../project/repository/reducing_the_repo_size_using_git.md). + +NOTE: **Note:** +For GitLab.com, the repository size limit is 10 GB. + +<!-- ## Troubleshooting -> **Note:** -> For GitLab.com, the repository size limit is 10 GB. +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. -[ee-740]: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/740 -[repo-size]: ../../project/repository/reducing_the_repo_size_using_git.md -[ee-8.12]: https://about.gitlab.com/2016/09/22/gitlab-8-12-released/#limit-project-size-ee +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> diff --git a/doc/user/admin_area/settings/continuous_integration.md b/doc/user/admin_area/settings/continuous_integration.md index 9dd476656ed..6c4abce83c2 100644 --- a/doc/user/admin_area/settings/continuous_integration.md +++ b/doc/user/admin_area/settings/continuous_integration.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # Continuous Integration and Deployment Admin settings **[CORE ONLY]** In this area, you will find settings for Auto DevOps, Runners and job artifacts. @@ -145,3 +149,15 @@ To set the duration for which the jobs will be considered as old and expired: Once that time passes, the jobs will be archived and no longer able to be retried. Make it empty to never expire jobs. It has to be no less than 1 day, for example: <code>15 days</code>, <code>1 month</code>, <code>2 years</code>. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. -->
\ No newline at end of file diff --git a/doc/user/admin_area/settings/email.md b/doc/user/admin_area/settings/email.md index 01a98cf15dc..912c2cff481 100644 --- a/doc/user/admin_area/settings/email.md +++ b/doc/user/admin_area/settings/email.md @@ -1,12 +1,18 @@ +--- +type: reference +--- + # Email +You can customize some of the content in emails sent from your GitLab instance. + ## Custom logo The logo in the header of some emails can be customized, see the [logo customization section](../../../customization/branded_page_and_email_header.md). ## Custom additional text **[PREMIUM ONLY]** ->[Introduced][ee-5031] in [GitLab Premium][eep] 10.7. +> [Introduced][ee-5031] in [GitLab Premium][eep] 10.7. The additional text will appear at the bottom of any email and can be used for legal/auditing/compliance reasons. @@ -24,8 +30,8 @@ legal/auditing/compliance reasons. > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/22560) in GitLab 11.5. -This configuration option sets the email hostname for [private commit emails](../../profile/index.md#private-commit-email), -and it's, by default, set to `users.noreply.YOUR_CONFIGURED_HOSTNAME`. +This configuration option sets the email hostname for [private commit emails](../../profile/index.md#private-commit-email). + By default it is set to `users.noreply.YOUR_CONFIGURED_HOSTNAME`. In order to change this option: @@ -36,3 +42,15 @@ In order to change this option: NOTE: **Note**: Once the hostname gets configured, every private commit email using the previous hostname, will not get recognized by GitLab. This can directly conflict with certain [Push rules](https://docs.gitlab.com/ee/push_rules/push_rules.html) such as `Check whether author is a GitLab user` and `Check whether committer is the current authenticated user`. + +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. -->
\ No newline at end of file diff --git a/doc/user/admin_area/settings/external_authorization.md b/doc/user/admin_area/settings/external_authorization.md index 06e00e02f3d..11c0867da17 100644 --- a/doc/user/admin_area/settings/external_authorization.md +++ b/doc/user/admin_area/settings/external_authorization.md @@ -1,3 +1,7 @@ +--- +type: reference +--- + # External authorization control **[CORE ONLY]** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/4216) in @@ -108,5 +112,17 @@ The label will be shown on all project pages in the upper right corner.  +<!-- ## Troubleshooting + +Include any troubleshooting steps that you can foresee. If you know beforehand what issues +one might have when setting this up, or when something is changed, or on upgrading, it's +important to describe those, too. Think of things that may go wrong and include them here. +This is important to minimize requests for support, and to avoid doc comments with +questions that you know someone might ask. + +Each scenario can be a third-level heading, e.g. `### Getting error message X`. +If you have none to add when creating a doc, leave this section in place +but commented out to help encourage others to add to it in the future. --> + [omnibus-ssl-docs]: https://docs.gitlab.com/omnibus/settings/ssl.html [omnibus-log-docs]: https://docs.gitlab.com/omnibus/settings/logs.html diff --git a/doc/user/admin_area/settings/usage_statistics.md b/doc/user/admin_area/settings/usage_statistics.md index 8b5d80efb0d..01d1eb1cd0e 100644 --- a/doc/user/admin_area/settings/usage_statistics.md +++ b/doc/user/admin_area/settings/usage_statistics.md @@ -4,7 +4,7 @@ GitLab Inc. will periodically collect information about your instance in order to perform various actions. All statistics are opt-out, you can enable/disable them from the admin panel -under **Admin area > Settings > Usage statistics**. +under **Admin area > Settings > Metrics and profiling > Usage statistics**. ## Version check **[CORE ONLY]** diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md index f3b7d7fd471..abc6e771b0f 100644 --- a/doc/user/application_security/dast/index.md +++ b/doc/user/application_security/dast/index.md @@ -47,10 +47,7 @@ applications while you are developing and testing your applications. ## Requirements To run a DAST job, you need GitLab Runner with the -[`docker`](https://docs.gitlab.com/runner/executors/docker.html#use-docker-in-docker-with-privileged-mode) or -[`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html#running-privileged-containers-for-the-runners) -executor running in privileged mode. If you're using the shared Runners on GitLab.com, -this is enabled by default. +[`docker` executor](https://docs.gitlab.com/runner/executors/docker.html). ## Configuring DAST diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index 02c115b7f22..db328262aba 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -63,7 +63,7 @@ The following table shows which languages, package managers and frameworks are s | Javascript | [ESLint security plugin](https://github.com/nodesecurity/eslint-plugin-security) | 11.8 | | Node.js | [NodeJsScan](https://github.com/ajinabraham/NodeJsScan) | 11.1 | | PHP | [phpcs-security-audit](https://github.com/FloeDesignTechnologies/phpcs-security-audit) | 10.8 | -| Python ([pip](https://pip.pypa.io/en/stable/)) | [bandit](https://github.com/openstack/bandit) | 10.3 | +| Python ([pip](https://pip.pypa.io/en/stable/)) | [bandit](https://github.com/PyCQA/bandit) | 10.3 | | Ruby on Rails | [brakeman](https://brakemanscanner.org) | 10.3 | | Scala ([Ant](https://ant.apache.org/), [Gradle](https://gradle.org/), [Maven](https://maven.apache.org/) and [SBT](https://www.scala-sbt.org/)) | [SpotBugs](https://spotbugs.github.io/) with the [find-sec-bugs](https://find-sec-bugs.github.io/) plugin | 11.0 (SBT) & 11.9 (Ant, Gradle, Maven) | | Typescript | [TSLint config security](https://github.com/webschik/tslint-config-security/) | 11.9 | diff --git a/doc/user/application_security/security_dashboard/img/dashboard.png b/doc/user/application_security/security_dashboard/img/dashboard.png Binary files differindex d52a6dacdbf..a75168b1ce4 100644 --- a/doc/user/application_security/security_dashboard/img/dashboard.png +++ b/doc/user/application_security/security_dashboard/img/dashboard.png diff --git a/doc/user/application_security/security_dashboard/index.md b/doc/user/application_security/security_dashboard/index.md index ca31e15c65f..19eeb06a259 100644 --- a/doc/user/application_security/security_dashboard/index.md +++ b/doc/user/application_security/security_dashboard/index.md @@ -54,6 +54,7 @@ First, navigate to the Security Dashboard found under your group's Once you're on the dashboard, at the top you should see a series of filters for: - Severity +- Confidence - Report type - Project diff --git a/doc/user/clusters/applications.md b/doc/user/clusters/applications.md new file mode 100644 index 00000000000..97abe99fe62 --- /dev/null +++ b/doc/user/clusters/applications.md @@ -0,0 +1,263 @@ +# GitLab Managed Apps + +GitLab provides **GitLab Managed Apps**, a one-click install for various applications which can +be added directly to your configured cluster. These applications are +needed for [Review Apps](../../ci/review_apps/index.md) and +[deployments](../../ci/environments.md) when using [Auto DevOps](../../topics/autodevops/index.md). +You can install them after you +[create a cluster](../project/clusters/index.md#adding-and-creating-a-new-gke-cluster-via-gitlab). + +## Installing applications + +Applications managed by GitLab will be installed onto the `gitlab-managed-apps` namespace. +This namespace: + +- Is different from the namespace used for project deployments. +- Is created once. +- Has a non-configurable name. + +To see a list of available applications to install: + +1. For a: + - Project-level cluster, navigate to your project's **Operations > Kubernetes**. + - Group-level cluster, navigate to your group's **Kubernetes** page. + +Install Helm first as it's used to install other applications. + +NOTE: **Note:** +As of GitLab 11.6, Helm will be upgraded to the latest version supported +by GitLab before installing any of the applications. + +The following applications can be installed: + +- [Helm](#helm) +- [Ingress](#ingress) +- [Cert-Manager](#cert-manager) +- [Prometheus](#prometheus) +- [GitLab Runner](#gitlab-runner) +- [JupyterHub](#jupyterhub) +- [Knative](#knative) + +With the exception of Knative, the applications will be installed in a dedicated +namespace called `gitlab-managed-apps`. + +NOTE: **Note:** +Some applications are installable only for a project-level cluster. +Support for installing these applications in a group-level cluster is +planned for future releases. +For updates, see [the issue tracking +progress](https://gitlab.com/gitlab-org/gitlab-ce/issues/51989). + +CAUTION: **Caution:** +If you have an existing Kubernetes cluster with Helm already installed, +you should be careful as GitLab cannot detect it. In this case, installing +Helm via the applications will result in the cluster having it twice, which +can lead to confusion during deployments. + +### Helm + +> - Available for project-level clusters since GitLab 10.2. +> - Available for group-level clusters since GitLab 11.6. + +[Helm](https://docs.helm.sh/) is a package manager for Kubernetes and is +required to install all the other applications. It is installed in its +own pod inside the cluster which can run the `helm` CLI in a safe +environment. + +### Cert-Manager + +> - Available for project-level clusters since GitLab 11.6. +> - Available for group-level clusters since GitLab 11.6. + +[Cert-Manager](https://docs.cert-manager.io/en/latest/) is a native +Kubernetes certificate management controller that helps with issuing +certificates. Installing Cert-Manager on your cluster will issue a +certificate by [Let's Encrypt](https://letsencrypt.org/) and ensure that +certificates are valid and up-to-date. + +NOTE: **Note:** +The +[stable/cert-manager](https://github.com/helm/charts/tree/master/stable/cert-manager) +chart is used to install this application with a +[`values.yaml`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/vendor/cert_manager/values.yaml) +file. + +### GitLab Runner + +> - Available for project-level clusters since GitLab 10.6. +> - Available for group-level clusters since GitLab 11.10. + +[GitLab Runner](https://docs.gitlab.com/runner/) is the open source +project that is used to run your jobs and send the results back to +GitLab. It is used in conjunction with [GitLab +CI/CD](../../ci/README.md), the open-source continuous integration +service included with GitLab that coordinates the jobs. When installing +the GitLab Runner via the applications, it will run in **privileged +mode** by default. Make sure you read the [security +implications](../project/clusters/index.md/#security-implications) before doing so. + +NOTE: **Note:** +The +[runner/gitlab-runner](https://gitlab.com/charts/gitlab-runner) +chart is used to install this application with a +[`values.yaml`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/vendor/runner/values.yaml) +file. + +### Ingress + +> - Available for project-level clusters since GitLab 10.2. +> - Available for group-level clusters since GitLab 11.6. + +[Ingress](https://kubernetes.github.io/ingress-nginx/) can provide load +balancing, SSL termination, and name-based virtual hosting. It acts as a +web proxy for your applications and is useful if you want to use [Auto +DevOps] or deploy your own web apps. + +NOTE: **Note:** +The +[stable/nginx-ingress](https://github.com/helm/charts/tree/master/stable/nginx-ingress) +chart is used to install this application with a +[`values.yaml`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/vendor/ingress/values.yaml) +file. + +### JupyterHub + +> Available for project-level clusters since GitLab 11.0. + +[JupyterHub](https://jupyterhub.readthedocs.io/en/stable/) is a +multi-user service for managing notebooks across a team. [Jupyter +Notebooks](https://jupyter-notebook.readthedocs.io/en/latest/) provide a +web-based interactive programming environment used for data analysis, +visualization, and machine learning. + +Authentication will be enabled only for [project +members](../project/members/index.md) with [Developer or +higher](../permissions.md) access to the project. + +We use a [custom Jupyter +image](https://gitlab.com/gitlab-org/jupyterhub-user-image/blob/master/Dockerfile) +that installs additional useful packages on top of the base Jupyter. You +will also see ready-to-use DevOps Runbooks built with Nurtch's [Rubix library](https://github.com/amit1rrr/rubix). + +More information on +creating executable runbooks can be found in [our Nurtch +documentation](../project/clusters/runbooks/index.md#nurtch-executable-runbooks). Note that +Ingress must be installed and have an IP address assigned before +JupyterHub can be installed. + +NOTE: **Note:** +The +[jupyter/jupyterhub](https://jupyterhub.github.io/helm-chart/) +chart is used to install this application with a +[`values.yaml`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/vendor/jupyter/values.yaml) +file. + +### Knative + +> Available for project-level clusters since GitLab 11.5. + +[Knative](https://cloud.google.com/knative) provides a platform to +create, deploy, and manage serverless workloads from a Kubernetes +cluster. It is used in conjunction with, and includes +[Istio](https://istio.io) to provide an external IP address for all +programs hosted by Knative. + +You will be prompted to enter a wildcard +domain where your applications will be exposed. Configure your DNS +server to use the external IP address for that domain. For any +application created and installed, they will be accessible as +`<program_name>.<kubernetes_namespace>.<domain_name>`. This will require +your kubernetes cluster to have [RBAC +enabled](../project/clusters/index.md#rbac-cluster-resources). + +NOTE: **Note:** +The +[knative/knative](https://storage.googleapis.com/triggermesh-charts) +chart is used to install this application. + +### Prometheus + +> - Available for project-level clusters since GitLab 10.4. +> - Available for group-level clusters since GitLab 11.11. + +[Prometheus](https://prometheus.io/docs/introduction/overview/) is an +open-source monitoring and alerting system useful to supervise your +deployed applications. + +NOTE: **Note:** +The +[stable/prometheus](https://github.com/helm/charts/tree/master/stable/prometheus) +chart is used to install this application with a +[`values.yaml`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/vendor/prometheus/values.yaml) +file. + +## Upgrading applications + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/24789) +in GitLab 11.8. + +The applications below can be upgraded. + +| Application | GitLab version | +| ----------- | -------------- | +| Runner | 11.8+ | + +To upgrade an application: + +1. For a: + - Project-level cluster, navigate to your project's **Operations > Kubernetes**. + - Group-level cluster, navigate to your group's **Kubernetes** page. +1. Select your cluster. +1. If an upgrade is available, the **Upgrade** button is displayed. Click the button to upgrade. + +NOTE: **Note:** +Upgrades will reset values back to the values built into the `runner` +chart plus the values set by +[`values.yaml`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/vendor/runner/values.yaml) + +## Uninstalling applications + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/60665) in +> GitLab 11.11. + +The applications below can be uninstalled. + +| Application | GitLab version | Notes | +| ----------- | -------------- | ----- | +| Prometheus | 11.11+ | All data will be deleted and cannot be restored. | + +To uninstall an application: + +1. For a: + - Project-level cluster, navigate to your project's **Operations > Kubernetes**. + - Group-level cluster, navigate to your group's **Kubernetes** page. +1. Select your cluster. +1. Click the **Uninstall** button for the application. + +Support for uninstalling all applications is planned for progressive rollout. +To follow progress, see [the relevant +epic](https://gitlab.com/groups/gitlab-org/-/epics/1201). + +## Troubleshooting applications + +Applications can fail with the following error: + +```text +Error: remote error: tls: bad certificate +``` + +To avoid installation errors: + +- Before starting the installation of applications, make sure that time is synchronized + between your GitLab server and your Kubernetes cluster. +- Ensure certificates are not out of sync. When installing applications, GitLab expects a new cluster with no previous installation of Helm. + + You can confirm that the certificates match via `kubectl`: + + ```sh + kubectl get configmaps/values-content-configuration-ingress -n gitlab-managed-apps -o \ + "jsonpath={.data['cert\.pem']}" | base64 -d > a.pem + kubectl get secrets/tiller-secret -n gitlab-managed-apps -o "jsonpath={.data['ca\.crt']}" | base64 -d > b.pem + diff a.pem b.pem + ``` + diff --git a/doc/user/group/clusters/index.md b/doc/user/group/clusters/index.md index ff6aa4f5930..8458b4f5de3 100644 --- a/doc/user/group/clusters/index.md +++ b/doc/user/group/clusters/index.md @@ -12,33 +12,10 @@ your group, enabling you to use the same cluster across multiple projects. ## Installing applications -GitLab provides a one-click install for various applications that can be -added directly to your cluster. - -NOTE: **Note:** -Applications will be installed in a dedicated namespace called -`gitlab-managed-apps`. If you have added an existing Kubernetes cluster -with Tiller already installed, you should be careful as GitLab cannot -detect it. In this event, installing Tiller via the applications will -result in the cluster having it twice. This can lead to confusion during -deployments. - -| Application | GitLab version | Description | Helm Chart | -| ----------- | -------------- | ----------- | ---------- | -| [Helm Tiller](https://docs.helm.sh) | 11.6+ | Helm is a package manager for Kubernetes and is required to install all the other applications. It is installed in its own pod inside the cluster which can run the `helm` CLI in a safe environment. | n/a | -| [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress) | 11.6+ | Ingress can provide load balancing, SSL termination, and name-based virtual hosting. It acts as a web proxy for your applications and is useful if you want to use [Auto DevOps](../../../topics/autodevops/index.md) or deploy your own web apps. | [stable/nginx-ingress](https://github.com/helm/charts/tree/master/stable/nginx-ingress) | -| [Cert-Manager](https://docs.cert-manager.io/en/latest/) | 11.6+ | Cert-Manager is a native Kubernetes certificate management controller that helps with issuing certificates. Installing Cert-Manager on your cluster will issue a certificate by [Let's Encrypt](https://letsencrypt.org/) and ensure that certificates are valid and up-to-date. | [stable/cert-manager](https://github.com/helm/charts/tree/master/stable/cert-manager) | -| [Prometheus](https://prometheus.io/docs/introduction/overview/) | 11.11+ | Prometheus is an open-source monitoring and alerting system useful to supervise your deployed applications. | [stable/prometheus](https://github.com/helm/charts/tree/master/stable/prometheus) | -| [GitLab Runner](https://docs.gitlab.com/runner/) | 11.10+ | GitLab Runner is the open source project that is used to run your jobs and send the results back to GitLab. It is used in conjunction with [GitLab CI/CD](../../../ci/README.md), the open-source continuous integration service included with GitLab that coordinates the jobs. When installing the GitLab Runner via the applications, it will run in **privileged mode** by default. Make sure you read the [security implications](../../project/clusters/index.md#security-implications) before doing so. | [runner/gitlab-runner](https://gitlab.com/charts/gitlab-runner) | - -NOTE: **Note:** -Some [cluster -applications](../../project/clusters/index.md#installing-applications) -are installable only for a project-level cluster. Support for installing these -applications in a group-level cluster is planned for future releases. For updates, see: - -- Support installing [JupyterHub in group-level - clusters](https://gitlab.com/gitlab-org/gitlab-ce/issues/51989) +GitLab can install and manage some applications in your group-level +cluster. For more information on installing, upgrading, uninstalling, +and troubleshooting applications for your group cluster, see +[Gitlab Managed Apps](../../clusters/applications.md). ## RBAC compatibility diff --git a/doc/user/group/index.md b/doc/user/group/index.md index a5e3bfda70e..7493e65e237 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -5,33 +5,35 @@ and grant members access to several projects at once. Groups can also be nested in [subgroups](subgroups/index.md). -Find your groups by clicking **Groups** in the top navigation. +Find your groups by clicking **Groups > Your Groups** in the top navigation.  > The groups dropdown in the top navigation was [introduced][ce-36234] in [GitLab 11.1](https://about.gitlab.com/2018/07/22/gitlab-11-1-released/#groups-dropdown-in-navigation). -The Groups page displays all groups you are a member of, how many projects it holds, -how many members it has, the group visibility, and, if you have enough permissions, -a link to the group settings. By clicking the last button you can leave that group. +The Groups page displays: + +- All groups you are a member of. +- How many projects each group contains. +- How many members a group has. +- The group visibility. +- A link to the group settings if you have sufficient permissions. + +By clicking the last button, you can leave that group. ## Use cases -You can create groups for numerous reasons. To name a few: - -- Organize related projects under the same [namespace](#namespaces), add members to that - group and grant access to all their projects at once -- Create a group, include members of your team, and make it easier to - `@mention` all the team at once in issues and merge requests - - Create a group for your company members, and create [subgroups](subgroups/index.md) - for each individual team. Let's say you create a group called `company-team`, and among others, - you created subgroups in this group for each individual team `backend-team`, - `frontend-team`, and `production-team`: - 1. When you start a new implementation from an issue, you add a comment: +You can create groups for numerous reasons. To name a couple: + +- Grant access to multiple projects and multiple team members in fewer steps by organizing related projects under the same [namespace](#namespaces) and adding members to the top-level group. +- Make it easier to `@mention` all of your team at once in issues and merge requests by creating a group and including the appropriate members. + +For example, you could create a group for your company members, and create a [subgroup](subgroups/index.md) for each individual team. Let's say you create a group called `company-team`, and you create subgroups in this group for the individual teams `backend-team`, `frontend-team`, and `production-team`. + - When you start a new implementation from an issue, you add a comment: _"`@company-team`, let's do it! `@company-team/backend-team` you're good to go!"_ - 1. When your backend team needs help from frontend, they add a comment: + - When your backend team needs help from frontend, they add a comment: _"`@company-team/frontend-team` could you help us here please?"_ - 1. When the frontend team completes their implementation, they comment: + - When the frontend team completes their implementation, they comment: _"`@company-team/backend-team`, it's done! Let's ship it `@company-team/production-team`!"_ ## Namespaces @@ -59,8 +61,8 @@ By doing so: ## Issues and merge requests within a group -Issues and merge requests are part of projects. For a given group, view all the -[issues](../project/issues/index.md#issues-list) and [merge requests](../project/merge_requests/index.md#merge-requests-per-group) across all the projects in that group, +Issues and merge requests are part of projects. For a given group, you can view all of the +[issues](../project/issues/index.md#issues-list) and [merge requests](../project/merge_requests/index.md#merge-requests-per-group) across all projects in that group, together in a single list view. ## Create a new group @@ -68,13 +70,13 @@ together in a single list view. > For a list of words that are not allowed to be used as group names see the > [reserved names](../reserved_names.md). -You can create a group in GitLab from: +To create a new Group, either: -1. The Groups page: from the top menu, click **Groups**, and click the green button **New group**: +- In the top menu, click **Groups** and then **Your Groups**, and click the green button **New group**.  -1. Elsewhere: expand the `plus` sign button on the top navbar and choose **New group**: +- Or, in the top menu, expand the `plus` sign and choose **New group**.  @@ -82,18 +84,18 @@ Add the following information:  -1. The **Group name** will populate the URL automatically. Optionally, you can change it. - This is the name that is displayed in the group views. +1. The **Group name** will automatically populate the URL. Optionally, you can change it. + This is the name that displays in group views. The name can contain only: - - Alphanumeric characters. - - Underscores. - - Dashes and dots. - - Spaces. -1. The **Group URL**, which will be the namespace under which your projects will be hosted. + - Alphanumeric characters + - Underscores + - Dashes and dots + - Spaces +1. The **Group URL** is the namespace under which your projects will be hosted. The URL can contain only: - - Alphanumeric characters. - - Underscores. - - Dashes and dots. It cannot start with dashes or end in dot. + - Alphanumeric characters + - Underscores + - Dashes and dots (it cannot start with dashes or end in a dot) 1. Optionally, you can add a brief description to tell others what this group is about. 1. Optionally, choose an avatar for your group. @@ -101,45 +103,39 @@ Add the following information: ## Add users to a group -Add members to a group by navigating to the group's dashboard, and clicking **Members**: +A benefit of putting multiple projects in one group is that you can +give a user to access to all projects in the group with one action. + +Add members to a group by navigating to the group's dashboard and clicking **Members**.  -Select the [permission level](../permissions.md#permissions) and add the new member. You can also set the expiring -date for that user, from which they will no longer have access to your group. +Select the [permission level](../permissions.md#permissions), and add the new member. You can also set the expiring date for that user; this is the date on which they will no longer have access to your group. -One of the benefits of putting multiple projects in one group is that you can -give a user to access to all projects in the group with one action. +Consider a group with two projects: -Consider we have a group with two projects: - -- On the **Group Members** page we can now add a new user to the group. -- Now because this user is a **Developer** member of the group, he automatically +- On the **Group Members** page, you can now add a new user to the group. +- Now, because this user is a **Developer** member of the group, they automatically gets **Developer** access to **all projects** within that group. -If necessary, you can increase the access level of an individual user for a specific project, -by adding them again as a new member to the project with the new permission levels. +To increase the access level of an existing user for a specific project, +add them again as a new member to the project with the desired permission level. ## Request access to a group -As a group owner you can enable or disable non members to request access to -your group. Go to the group settings and click on **Allow users to request access**. +As a group owner, you can enable or disable the ability for non members to request access to +your group. Go to the group settings, and click **Allow users to request access**. -As a user, you can request to be a member of a group. Go to the group you'd -like to be a member of, and click the **Request Access** button on the right +As a user, you can request to be a member of a group, if that setting is enabled. Go to the group for which you'd like to be a member, and click the **Request Access** button on the right side of your screen.  ---- - Group owners and maintainers will be notified of your request and will be able to approve or decline it on the members page.  ---- - If you change your mind before your request is approved, just click the **Withdraw Access Request** button. @@ -149,29 +145,27 @@ If you change your mind before your request is approved, just click the There are two different ways to add a new project to a group: -- Select a group and then click on the **New project** button. +- Select a group, and then click **New project**. You can then continue [creating your project](../../gitlab-basics/create-project.md).  - You can then continue on [creating a project](../../gitlab-basics/create-project.md). - - While you are creating a project, select a group namespace you've already created from the dropdown menu.  -### Default project creation level +### Default project-creation level > [Introduced][ee-2534] in [GitLab Premium][ee] 10.5. > Brought to [GitLab Starter][ee] in 10.7. > [Moved](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/25975) to [GitLab Core](https://about.gitlab.com/pricing/) in 11.10. -Group owners or administrators can allow users with the +Group owners and administrators can allow users with the Developer role to create projects under groups. -By default, [Developers and Maintainers](../permissions.md#group-members-permissions) can create projects under a group, but this can be changed either within the group settings for a group, or -be set globally by a GitLab administrator in the Admin area -at **Settings > General > Visibility and access controls**. +By default, [Developers and Maintainers](../permissions.md#group-members-permissions) can create projects under a group. You can change this setting for a specific group within the group settings, or +you can set this option globally in the Admin area +at **Settings > General > Visibility and access controls** (you must be a GitLab administrator). Available settings are `No one`, `Maintainers`, or `Developers + Maintainers`. @@ -182,13 +176,13 @@ Learn how to [transfer a project into a group](../project/settings/index.md#tran ## Sharing a project with a group You can [share your projects with a group](../project/members/share_project_with_groups.md) -and give your group members access to the project all at once. +and give all group members access to the project at once. Alternatively, you can [lock the sharing with group feature](#share-with-group-lock). ## Manage group memberships via LDAP -In GitLab Enterprise Edition it is possible to manage GitLab group memberships using LDAP groups. +In GitLab Enterprise Edition, it is possible to manage GitLab group memberships using LDAP groups. See [the GitLab Enterprise Edition documentation](../../integration/ldap.md) for more information. ## Epics **[ULTIMATE]** @@ -211,38 +205,42 @@ Get an overview of the vulnerabilities of all the projects in a group and its su > Introduced in [GitLab Ultimate][ee] 11.9 behind the `insights` feature flag. -Configure the Insights that matter for your groups or projects to explore data -such as triage hygiene, issues created/closed per a given period, average time -for merge requests to be merged and much more. +Configure the Insights that matter for your groups or projects, allowing users to explore data +such as: + +- Triage hygiene +- Issues created/closed per a given period +- Average time for merge requests to be merged +- Much more [Learn more about Insights](insights/index.md). ## Transferring groups -From GitLab 10.5, groups can be transferred in the following ways: +From GitLab 10.5, you can transfer groups in the following ways: -- Top-level groups can be transferred to a group, converting them into subgroups. -- Subgroups can be transferred to a new parent group. -- Subgroups can be transferred out from a parent group, converting them into top-level groups. +- Transfer a subgroup to a new parent group. +- Convert a top-level group into a subgroup by transfering it to the desired group. +- Convert a subgroup into a top-level group by transfering it out of its current group. When transferring groups, note: - Changing a group's parent can have unintended side effects. See [Redirects when changing repository paths](../project/index.md#redirects-when-changing-repository-paths). - You can only transfer groups to groups you manage. -- You will need to update your local repositories to point to the new location. -- If the parent group's visibility is lower than the group's current visibility, visibility levels for subgroups and projects will be changed to match the new parent group's visibility. -- Only explicit group membership is transferred, not inherited membership. If the group's owners have only inherited membership, this would leave the group without an owner. In this case, the user transferring the group becomes the group's owner. +- You must update your local repositories to point to the new location. +- If the parent group's visibility is lower than the group's current visibility, visibility levels for subgroups and projects will change to match the new parent group's visibility. +- Only explicit group membership is transferred, not inherited membership. If the group's owners have only inherited membership, this leaves the group without an owner. In this case, the user transferring the group becomes the group's owner. ## Group settings -Once you have created a group, you can manage its settings by navigating to +After creating a group, you can manage its settings by navigating to the group's dashboard, and clicking **Settings**.  ### General settings -Besides giving you the option to edit any settings you've previously +In addition to editing any settings you previously set when [creating the group](#create-a-new-group), you can also access further configurations for your group. @@ -253,14 +251,14 @@ Changing a group's path can have unintended side effects. Read before proceeding. If you are vacating the path so it can be claimed by another group or user, -you may need to rename the group name as well since both names and paths must +you may need to rename the group, too, since both names and paths must be unique. To change your group path: 1. Navigate to your group's **Settings > General**. -1. Enter a new name under "Group path". -1. Hit **Save group**. +1. Enter a new name under **Group path**. +1. Click **Save group**. CAUTION: **Caution:** It is currently not possible to rename a namespace if it contains a @@ -276,20 +274,17 @@ username, you can create a new group and transfer projects to it. Add a security layer to your group by [enforcing two-factor authentication (2FA)](../../security/two_factor_authentication.md#enforcing-2fa-for-all-users-in-a-group) -to all group members. +for all group members. #### Share with group lock Prevent projects in a group from [sharing -a project with another group](../project/members/share_project_with_groups.md). -This allows for tighter control over project access. +a project with another group](../project/members/share_project_with_groups.md) to enable tighter control over project access. -For example, consider you have two distinct teams (Group A and Group B) -working together in a project. -To inherit the group membership, you share the project between the +For example, let's say you have two distinct teams (Group A and Group B) working together in a project, and to inherit the group membership, you share the project between the two groups A and B. **Share with group lock** prevents any project within -the group from being shared with another group. By doing so, you -guarantee only the right group members have access to those projects. +the group from being shared with another group, +guaranteeing that only the right group members have access to those projects. To enable this feature, navigate to the group settings page. Select **Share with group lock** and **Save the group**. @@ -298,25 +293,22 @@ To enable this feature, navigate to the group settings page. Select #### Member Lock **[STARTER]** -With Member lock, it is possible to lock membership in a project to the -level of members in the group. - -Member lock lets a group owner lock down any new project membership to all the -projects within the group, allowing tighter control over project membership. +Member lock lets a group owner prevent any new project membership to all of the +projects within a group, allowing tighter control over project membership. -For instance, if you want to lock the group for an [Audit Event](https://docs.gitlab.com/ee/administration/audit_events.html), -you enable Member lock to guarantee that membership of a project cannot be modified during that audit. +For example, if you want to lock the group for an [Audit Event](https://docs.gitlab.com/ee/administration/audit_events.html), +enable Member lock to guarantee that project membership cannot be modified during that audit. To enable this feature: 1. Navigate to the group's **Settings > General** page. -1. Expand the **Permissions, LFS, 2FA** section and select **Member lock**. -1. Click the **Save changes** button. +1. Expand the **Permissions, LFS, 2FA** section, and select **Member lock**. +1. Click **Save changes**.  This will disable the option for all users who previously had permissions to -operate project memberships so no new users can be added. Furthermore, any +operate project memberships, so no new users can be added. Furthermore, any request to add a new user to a project through API will not be possible. #### Group file templates **[PREMIUM]** @@ -327,11 +319,11 @@ types with every project in a group. It is analogous to the feature, and the selected project should follow the same naming conventions as are documented on that page. -Only projects that are in the group may be chosen as the source of templates. -This includes projects shared with the group, but **excludes** projects in +You can only choose projects in the group as the template source. +This includes projects shared with the group, but it **excludes** projects in subgroups or parent groups of the group being configured. -This feature may be configured for subgroups as well as parent groups. A project +You can configure this feature for both subgroups and parent groups. A project in a subgroup will have access to the templates for that subgroup, as well as any parent groups. @@ -345,28 +337,28 @@ To enable this feature, navigate to the group settings page, expand the #### Group-level project templates **[PREMIUM]** -Define project templates at a group-level by setting a group as a template source. +Define project templates at a group level by setting a group as the template source. [Learn more about group-level project templates](custom_project_templates.md). ### Advanced settings -- **Projects**: view all projects within that group, add members to each project, - access each project's settings, and remove any project from the same screen. -- **Webhooks**: configure [webhooks](../project/integrations/webhooks.md) to your group. -- **Kubernetes cluster integration**: connect your GitLab group with [Kubernetes clusters](clusters/index.md). -- **Audit Events**: view [Audit Events](https://docs.gitlab.com/ee/administration/audit_events.html) +- **Projects**: View all projects within that group, add members to each project, + access each project's settings, and remove any project, all from the same screen. +- **Webhooks**: Configure [webhooks](../project/integrations/webhooks.md) for your group. +- **Kubernetes cluster integration**: Connect your GitLab group with [Kubernetes clusters](clusters/index.md). +- **Audit Events**: View [Audit Events](https://docs.gitlab.com/ee/administration/audit_events.html) for the group. **[STARTER ONLY]** -- **Pipelines quota**: keep track of the [pipeline quota](../admin_area/settings/continuous_integration.md) for the group. +- **Pipelines quota**: Keep track of the [pipeline quota](../admin_area/settings/continuous_integration.md) for the group. ## User contribution analysis **[STARTER]** -With [GitLab Contribution Analytics](contribution_analytics/index.md) +With [GitLab Contribution Analytics](contribution_analytics/index.md), you have an overview of the contributions (pushes, merge requests, and issues) performed by your group members. ## Issues analytics **[PREMIUM]** -With [GitLab Issues Analytics](issues_analytics/index.md), in groups, you can see a bar chart of the number of issues created each month. +With [GitLab Issues Analytics](issues_analytics/index.md), you can see a bar chart of the number of issues created each month in your groups. ## Dependency Proxy **[PREMIUM]** diff --git a/doc/user/group/insights/img/insights_sidebar_link.png b/doc/user/group/insights/img/insights_sidebar_link.png Binary files differnew file mode 100644 index 00000000000..64bbd1fc4d3 --- /dev/null +++ b/doc/user/group/insights/img/insights_sidebar_link.png diff --git a/doc/user/group/insights/index.md b/doc/user/group/insights/index.md index 5283d8da77d..20f76c54ae7 100644 --- a/doc/user/group/insights/index.md +++ b/doc/user/group/insights/index.md @@ -4,8 +4,8 @@ CAUTION: **Beta:** Insights is considered beta, and is not ready for production use. -Follow [gitlab-org&725](https://gitlab.com/groups/gitlab-org/-/epics/725) for -updates. +Follow [gitlab-org/quality/team-tasks#137](https://gitlab.com/gitlab-org/quality/team-tasks/issues/137#general-availability) +for updates. Configure the Insights that matter for your groups to explore data such as triage hygiene, issues created/closed per a given period, average time for merge @@ -13,6 +13,13 @@ requests to be merged and much more.  +## View your group's Insights + +You can access your group's Insights by clicking the **Overview > Insights** +link in the left sidebar: + + + ## Configure your Insights Navigate to your group's **Settings > General**, expand **Insights**, and choose diff --git a/doc/user/group/saml_sso/index.md b/doc/user/group/saml_sso/index.md index 53116606201..62a3ef52c34 100644 --- a/doc/user/group/saml_sso/index.md +++ b/doc/user/group/saml_sso/index.md @@ -5,17 +5,17 @@ NOTE: **Note:** This topic is for SAML on GitLab.com Silver tier and above. For SAML on self-managed GitLab instances, see [SAML OmniAuth Provider](../../../integration/saml.md). -Currently SAML on GitLab.com can be used to automatically add users to a group, and does not yet sign users into GitLab.com. Users should already have an account on the GitLab instance, or can create one when logging in for the first time. +SAML on GitLab.com allows users to be automatically added to a group, and then allows those users to sign into GitLab.com. Users should already have an account on the GitLab instance, or can create one when logging in for the first time. User synchronization for GitLab.com is partially supported using [SCIM](scim_setup.md). NOTE: **Note:** -SAML SSO for groups is used only as a convenient way to add users and does not sync users between providers without using SCIM. If a group is not using SCIM, group Owners will still need to manage user accounts, such as removing users when necessary. +SAML SSO for GitLab.com groups does not sync users between providers without using SCIM. If a group is not using SCIM, group Owners will still need to manage user accounts (for example, removing users when necessary). ## Configuring your Identity Provider 1. Navigate to the group and click **Settings > SAML SSO**. -1. Configure your SAML server using the **Assertion consumer service URL** and **Issuer**. See [your identity provider's documentation](#providers) for more details. +1. Configure your SAML server using the **Assertion consumer service URL** and **Issuer**. Alternatively GitLab provides [metadata XML configuration](#metadata-configuration). See [your identity provider's documentation](#providers) for more details. 1. Configure the SAML response to include a NameID that uniquely identifies each user. 1. Configure required assertions using the [table below](#assertions). 1. Once the identity provider is set up, move on to [configuring GitLab](#configuring-gitlab). @@ -50,6 +50,14 @@ GitLab.com uses the SAML NameID to identify users. The NameID element: | First Name | `first_name`, `firstname`, `firstName` | | | Last Name | `last_name`, `lastname`, `lastName` | | +## Metadata configuration + +GitLab provides metadata XML that can be used to configure your Identity Provider. + +1. Navigate to the group and click **Settings > SAML SSO**. +1. Copy the provided **GitLab metadata URL** +1. Follow your Identity Provider's documentation and paste the metadata URL when it is requested. + ## Configuring GitLab Once you've set up your identity provider to work with GitLab, you'll need to configure GitLab to use it for authentication: @@ -75,6 +83,23 @@ Once you've set up your identity provider to work with GitLab, you'll need to co | OneLogin | [Use the OneLogin SAML Test Connector](https://onelogin.service-now.com/support?id=kb_article&sys_id=93f95543db109700d5505eea4b96198f) | | Ping Identity | [Add and configure a new SAML application](https://docs.pingidentity.com/bundle/p1_enterpriseConfigSsoSaml_cas/page/enableAppWithoutURL.html) | +## Linking SAML to your existing GitLab.com account + +To link SAML to your existing GitLab.com account: + +1. Sign in to your GitLab.com account. +1. Locate the SSO URL for the group you are signing in to. A group Admin can find this on the group's **Settings > SAML SSO** page. +1. Visit the SSO URL and click **Authorize**. +1. Enter your credentials on the Identity Provider if prompted. +1. You will be redirected back to GitLab.com and should now have access to the group. In the future, you can use SAML to sign in to GitLab.com. + +## Signing in to GitLab.com with SAML + +1. Locate the SSO URL for the group you are signing in to. A group Admin can find this on a group's **Settings > SAML SSO** page. If configured, it might also be possible to sign in to GitLab starting from your Identity Provider. +1. Visit the SSO URL and click the **Sign in with Single Sign-On** button. +1. Enter your credentials on the Identity Provider if prompted. +1. You will be signed in to GitLab.com and redirected to the group. + ## Unlinking accounts Users can unlink SAML for a group from their profile page. This can be helpful if: diff --git a/doc/user/profile/personal_access_tokens.md b/doc/user/profile/personal_access_tokens.md index 4085f3b678c..5166524d13b 100644 --- a/doc/user/profile/personal_access_tokens.md +++ b/doc/user/profile/personal_access_tokens.md @@ -49,6 +49,7 @@ the following table. [2fa]: ../account/two_factor_authentication.md [api]: ../../api/README.md [ce-3749]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3749 +[ce-5951]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5951 [ce-14838]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14838 [container registry]: ../project/container_registry.md [users]: ../../api/users.md diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index e60317bc199..e38e4059117 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -71,7 +71,7 @@ new Kubernetes cluster to your project: - **Number of nodes** - Enter the number of nodes you wish the cluster to have. - **Machine type** - The [machine type](https://cloud.google.com/compute/docs/machine-types) of the Virtual Machine instance that the cluster will be based on. - - **RBAC-enabled cluster** - Leave this checked if using default GKE creation options, see the [RBAC section](#role-based-access-control-rbac) for more information. + - **RBAC-enabled cluster** - Leave this checked if using default GKE creation options, see the [RBAC section](#rbac-cluster-resources) for more information. - **GitLab-managed cluster** - Leave this checked if you want GitLab to manage namespaces and service accounts for this cluster. See the [Managed clusters section](#gitlab-managed-clusters) for more information. 1. Finally, click the **Create Kubernetes cluster** button. @@ -263,65 +263,66 @@ you can either: ## Access controls -When creating a cluster in GitLab, you will be asked if you would like to create an -[Attribute-based access control (ABAC)](https://kubernetes.io/docs/admin/authorization/abac/) cluster, or -a [Role-based access control (RBAC)](https://kubernetes.io/docs/admin/authorization/rbac/) one. +When creating a cluster in GitLab, you will be asked if you would like to create either: -NOTE: **Note:** -[RBAC](#role-based-access-control-rbac) is recommended and the GitLab default. +- An [Attribute-based access control (ABAC)](https://kubernetes.io/docs/admin/authorization/abac/) cluster. +- A [Role-based access control (RBAC)](https://kubernetes.io/docs/admin/authorization/rbac/) cluster. -Whether [ABAC](#attribute-based-access-control-abac) or [RBAC](#role-based-access-control-rbac) is enabled, -GitLab will create the necessary service accounts and privileges in order to install and run -[GitLab managed applications](#installing-applications): +NOTE: **Note:** +[RBAC](#rbac-cluster-resources) is recommended and the GitLab default. -- If GitLab is creating the cluster, a `gitlab` service account with - `cluster-admin` privileges will be created in the `default` namespace, - which will be used by GitLab to manage the newly created cluster. +GitLab creates the necessary service accounts and privileges to install and run +[GitLab managed applications](#installing-applications). When GitLab creates the cluster: +- A `gitlab` service account with `cluster-admin` privileges is created in the `default` namespace + to manage the newly created cluster. - A project service account with [`edit` privileges](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) - will be created in the project namespace (also created by GitLab), which will - be used in [deployment jobs](#deployment-variables). + is created in the GitLab-created project namespace for [deployment jobs](#deployment-variables). NOTE: **Note:** Restricted service account for deployment was [introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/51716) in GitLab 11.5. -- When you install Helm into your cluster, the `tiller` service account - will be created with `cluster-admin` privileges in the `gitlab-managed-apps` - namespace. This service account will be added to the installed Helm Tiller and will - be used by Helm to install and run [GitLab managed applications](#installing-applications). - Helm will also create additional service accounts and other resources for each - installed application. Consult the documentation of the Helm charts for each application - for details. +When you install Helm into your cluster, the `tiller` service account +is created with `cluster-admin` privileges in the `gitlab-managed-apps` +namespace. This service account will be added to the installed Helm Tiller and will +be used by Helm to install and run [GitLab managed applications](#installing-applications). +Helm will also create additional service accounts and other resources for each +installed application. Consult the documentation of the Helm charts for each application +for details. If you are [adding an existing Kubernetes cluster](#adding-an-existing-kubernetes-cluster), ensure the token of the account has administrator privileges for the cluster. -The following sections summarize which resources will be created on ABAC/RBAC clusters. +The resources created by GitLab differ depending on the type of cluster. + +### ABAC cluster resources -### Attribute-based access control (ABAC) +GitLab creates the following resources for ABAC clusters. -| Name | Kind | Details | Created when | -| --- | --- | --- | --- | -| `gitlab` | `ServiceAccount` | `default` namespace | Creating a new GKE Cluster | -| `gitlab-token` | `Secret` | Token for `gitlab` ServiceAccount | Creating a new GKE Cluster | -| `tiller` | `ServiceAccount` | `gitlab-managed-apps` namespace | Installing Helm Tiller | -| `tiller-admin` | `ClusterRoleBinding` | `cluster-admin` roleRef | Installing Helm Tiller | -| Project namespace | `ServiceAccount` | Uses namespace of Project | Deploying to a cluster | -| Project namespace | `Secret` | Token for project ServiceAccount | Deploying to a cluster | +| Name | Type | Details | Created when | +|:------------------|:---------------------|:----------------------------------|:---------------------------| +| `gitlab` | `ServiceAccount` | `default` namespace | Creating a new GKE Cluster | +| `gitlab-token` | `Secret` | Token for `gitlab` ServiceAccount | Creating a new GKE Cluster | +| `tiller` | `ServiceAccount` | `gitlab-managed-apps` namespace | Installing Helm Tiller | +| `tiller-admin` | `ClusterRoleBinding` | `cluster-admin` roleRef | Installing Helm Tiller | +| Project namespace | `ServiceAccount` | Uses namespace of Project | Deploying to a cluster | +| Project namespace | `Secret` | Token for project ServiceAccount | Deploying to a cluster | -### Role-based access control (RBAC) +### RBAC cluster resources -| Name | Kind | Details | Created when | -| --- | --- | --- | --- | -| `gitlab` | `ServiceAccount` | `default` namespace | Creating a new GKE Cluster | -| `gitlab-admin` | `ClusterRoleBinding` | [`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) roleRef | Creating a new GKE Cluster | -| `gitlab-token` | `Secret` | Token for `gitlab` ServiceAccount | Creating a new GKE Cluster | -| `tiller` | `ServiceAccount` | `gitlab-managed-apps` namespace | Installing Helm Tiller | -| `tiller-admin` | `ClusterRoleBinding` | `cluster-admin` roleRef | Installing Helm Tiller | -| Project namespace | `ServiceAccount` | Uses namespace of Project | Deploying to a cluster | -| Project namespace | `Secret` | Token for project ServiceAccount | Deploying to a cluster | -| Project namespace | `RoleBinding` | [`edit`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) roleRef | Deploying to a cluster | +GitLab creates the following resources for RBAC clusters. + +| Name | Type | Details | Created when | +|:------------------|:---------------------|:-----------------------------------------------------------------------------------------------------------|:---------------------------| +| `gitlab` | `ServiceAccount` | `default` namespace | Creating a new GKE Cluster | +| `gitlab-admin` | `ClusterRoleBinding` | [`cluster-admin`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) roleRef | Creating a new GKE Cluster | +| `gitlab-token` | `Secret` | Token for `gitlab` ServiceAccount | Creating a new GKE Cluster | +| `tiller` | `ServiceAccount` | `gitlab-managed-apps` namespace | Installing Helm Tiller | +| `tiller-admin` | `ClusterRoleBinding` | `cluster-admin` roleRef | Installing Helm Tiller | +| Project namespace | `ServiceAccount` | Uses namespace of Project | Deploying to a cluster | +| Project namespace | `Secret` | Token for project ServiceAccount | Deploying to a cluster | +| Project namespace | `RoleBinding` | [`edit`](https://kubernetes.io/docs/reference/access-authn-authz/rbac/#user-facing-roles) roleRef | Deploying to a cluster | NOTE: **Note:** Project-specific resources are only created if your cluster is [managed by GitLab](#gitlab-managed-clusters). @@ -346,111 +347,10 @@ install it manually. ## Installing applications -GitLab provides **GitLab Managed Apps**, a one-click install for various applications which can -be added directly to your configured cluster. These applications are -needed for [Review Apps](../../../ci/review_apps/index.md) and -[deployments](../../../ci/environments.md) when using [Auto DevOps](../../../topics/autodevops/index.md). -You can install them after you -[create a cluster](#adding-and-creating-a-new-gke-cluster-via-gitlab). - -Applications managed by GitLab will be installed onto the `gitlab-managed-apps` namespace. This differrent -from the namespace used for project deployments. It is only created once and its name is not configurable. - -To see a list of available applications to install: - -1. Navigate to your project's **Operations > Kubernetes**. -1. Select your cluster. - -Install Helm first as it's used to install other applications. - -NOTE: **Note:** -As of GitLab 11.6, Helm will be upgraded to the latest version supported -by GitLab before installing any of the applications. - -| Application | GitLab version | Description | Helm Chart | -| ----------- | :------------: | ----------- | --------------- | -| [Helm](https://docs.helm.sh/) | 10.2+ | Helm is a package manager for Kubernetes and is required to install all the other applications. It is installed in its own pod inside the cluster which can run the `helm` CLI in a safe environment. | n/a | -| [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) | 10.2+ | Ingress can provide load balancing, SSL termination, and name-based virtual hosting. It acts as a web proxy for your applications and is useful if you want to use [Auto DevOps] or deploy your own web apps. | [stable/nginx-ingress](https://github.com/helm/charts/tree/master/stable/nginx-ingress) | -| [Cert-Manager](https://docs.cert-manager.io/en/latest/) | 11.6+ | Cert-Manager is a native Kubernetes certificate management controller that helps with issuing certificates. Installing Cert-Manager on your cluster will issue a certificate by [Let's Encrypt](https://letsencrypt.org/) and ensure that certificates are valid and up-to-date. | [stable/cert-manager](https://github.com/helm/charts/tree/master/stable/cert-manager) | -| [Prometheus](https://prometheus.io/docs/introduction/overview/) | 10.4+ | Prometheus is an open-source monitoring and alerting system useful to supervise your deployed applications. | [stable/prometheus](https://github.com/helm/charts/tree/master/stable/prometheus) | -| [GitLab Runner](https://docs.gitlab.com/runner/) | 10.6+ | GitLab Runner is the open source project that is used to run your jobs and send the results back to GitLab. It is used in conjunction with [GitLab CI/CD](../../../ci/README.md), the open-source continuous integration service included with GitLab that coordinates the jobs. When installing the GitLab Runner via the applications, it will run in **privileged mode** by default. Make sure you read the [security implications](#security-implications) before doing so. | [runner/gitlab-runner](https://gitlab.com/charts/gitlab-runner) | -| [JupyterHub](http://jupyter.org/) | 11.0+ | [JupyterHub](https://jupyterhub.readthedocs.io/en/stable/) is a multi-user service for managing notebooks across a team. [Jupyter Notebooks](https://jupyter-notebook.readthedocs.io/en/latest/) provide a web-based interactive programming environment used for data analysis, visualization, and machine learning. We use a [custom Jupyter image](https://gitlab.com/gitlab-org/jupyterhub-user-image/blob/master/Dockerfile) that installs additional useful packages on top of the base Jupyter. Authentication will be enabled only for [project members](../members/index.md) with [Developer or higher](../../permissions.md) access to the project. You will also see ready-to-use DevOps Runbooks built with Nurtch's [Rubix library](https://github.com/amit1rrr/rubix). More information on creating executable runbooks can be found in [our Nurtch documentation](runbooks/index.md#nurtch-executable-runbooks). Note that Ingress must be installed and have an IP address assigned before JupyterHub can be installed. | [jupyter/jupyterhub](https://jupyterhub.github.io/helm-chart/) | -| [Knative](https://cloud.google.com/knative) | 11.5+ | Knative provides a platform to create, deploy, and manage serverless workloads from a Kubernetes cluster. It is used in conjunction with, and includes [Istio](https://istio.io) to provide an external IP address for all programs hosted by Knative. You will be prompted to enter a wildcard domain where your applications will be exposed. Configure your DNS server to use the external IP address for that domain. For any application created and installed, they will be accessible as `<program_name>.<kubernetes_namespace>.<domain_name>`. This will require your kubernetes cluster to have [RBAC enabled](#role-based-access-control-rbac). | [knative/knative](https://storage.googleapis.com/triggermesh-charts) - -With the exception of Knative, the applications will be installed in a dedicated -namespace called `gitlab-managed-apps`. - -CAUTION: **Caution:** -If you have an existing Kubernetes cluster with Helm already installed, -you should be careful as GitLab cannot detect it. In this case, installing -Helm via the applications will result in the cluster having it twice, which -can lead to confusion during deployments. - -### Upgrading applications - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/24789) -in GitLab 11.8. - -Users can perform a one-click upgrade for the GitLab Runner application, -when there is an upgrade available. - -To upgrade the GitLab Runner application: - -1. Navigate to your project's **Operations > Kubernetes**. -1. Select your cluster. -1. Click the **Upgrade** button for the Runnner application. - -The **Upgrade** button will not be shown if there is no upgrade -available. - -NOTE: **Note:** -Upgrades will reset values back to the values built into the `runner` -chart plus the values set by -[`values.yaml`](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/vendor/runner/values.yaml) - -### Uninstalling applications - -> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/60665) in -> GitLab 11.11. - -The applications below can be uninstalled. - -| Application | GitLab version | Notes | -| ----------- | -------------- | ----- | -| Prometheus | 11.11+ | All data will be deleted and cannot be restored. | - -To uninstall an application: - -1. Navigate to your project's **Operations > Kubernetes**. -1. Select your cluster. -1. Click the **Uninstall** button for the application. - -Support for uninstalling all applications will be progressively -introduced (see [related -epic](https://gitlab.com/groups/gitlab-org/-/epics/1201)). - -### Troubleshooting applications - -Applications can fail with the following error: - -```text -Error: remote error: tls: bad certificate -``` - -To avoid installation errors: - -- Before starting the installation of applications, make sure that time is synchronized - between your GitLab server and your Kubernetes cluster. -- Ensure certificates are not out of sync. When installing applications, GitLab expects a new cluster with no previous installation of Helm. - - You can confirm that the certificates match via `kubectl`: - - ```sh - kubectl get configmaps/values-content-configuration-ingress -n gitlab-managed-apps -o \ - "jsonpath={.data['cert\.pem']}" | base64 -d > a.pem - kubectl get secrets/tiller-secret -n gitlab-managed-apps -o "jsonpath={.data['ca\.crt']}" | base64 -d > b.pem - diff a.pem b.pem - ``` +GitLab can install and manage some applications in your project-level +cluster. For more information on installing, upgrading, uninstalling, +and troubleshooting applications for your project cluster, see +[Gitlab Managed Apps](../../clusters/applications.md). ## Getting the external endpoint diff --git a/doc/user/project/insights/img/insights_sidebar_link.png b/doc/user/project/insights/img/insights_sidebar_link.png Binary files differnew file mode 100644 index 00000000000..aadb5745992 --- /dev/null +++ b/doc/user/project/insights/img/insights_sidebar_link.png diff --git a/doc/user/project/insights/index.md b/doc/user/project/insights/index.md index 720f4c6604e..b6cc1862cc2 100644 --- a/doc/user/project/insights/index.md +++ b/doc/user/project/insights/index.md @@ -4,18 +4,25 @@ CAUTION: **Beta:** Insights is considered beta, and is not ready for production use. -Follow [gitlab-org&725](https://gitlab.com/groups/gitlab-org/-/epics/725) for -updates. +Follow [gitlab-org/quality/team-tasks#137](https://gitlab.com/gitlab-org/quality/team-tasks/issues/137#general-availability) +for updates. Configure the Insights that matter for your projects to explore data such as triage hygiene, issues created/closed per a given period, average time for merge requests to be merged and much more. - + NOTE: **Note:** This feature is [also available at the group level](https://docs.gitlab.com/ee/user/group/insights/index.html). +## View your project's Insights + +You can access your project's Insights by clicking the **Project > Insights** +link in the left sidebar: + + + ## Configure your Insights Insights are configured using a YAML file called `.gitlab/insights.yml` within @@ -74,8 +81,7 @@ Each chart definition is made up of a hash composed of key-value pairs. For example, here's single chart definition: ```yaml -monthlyBugsCreated: - title: Monthly Bugs Created (bar) +- title: Monthly Bugs Created (bar) type: bar query: issuable_type: issue diff --git a/doc/user/project/integrations/img/mattermost_configuration.png b/doc/user/project/integrations/img/mattermost_configuration.png Binary files differindex e0b55b23520..75ef0310f2d 100644 --- a/doc/user/project/integrations/img/mattermost_configuration.png +++ b/doc/user/project/integrations/img/mattermost_configuration.png diff --git a/doc/user/project/integrations/img/slack_configuration.png b/doc/user/project/integrations/img/slack_configuration.png Binary files differindex 53b30e0e8cd..a14d2969488 100644 --- a/doc/user/project/integrations/img/slack_configuration.png +++ b/doc/user/project/integrations/img/slack_configuration.png diff --git a/doc/user/project/integrations/mattermost.md b/doc/user/project/integrations/mattermost.md index 8c5461de42f..d7fd75fd728 100644 --- a/doc/user/project/integrations/mattermost.md +++ b/doc/user/project/integrations/mattermost.md @@ -27,9 +27,11 @@ There, you will see a checkbox with the following events that can be triggered: - Confidential issue - Merge request - Note +- Confidential note - Tag push - Pipeline - Wiki page +- Deployment Below each of these event checkboxes, you have an input field to enter which Mattermost channel you want to send that event message. Enter your preferred channel handle (the hash sign `#` is optional). diff --git a/doc/user/project/integrations/prometheus.md b/doc/user/project/integrations/prometheus.md index 40113624430..751e8e44e60 100644 --- a/doc/user/project/integrations/prometheus.md +++ b/doc/user/project/integrations/prometheus.md @@ -168,7 +168,7 @@ Alerts can be used to trigger actions, like open an issue automatically. To conf 1. Optionally, select whether to send an email notification to the developers of the project. 1. Click **Save changes**. -Once enabled, an issue will be opened automatically when an alert is triggered. To further customize the issue, you can add labels, mentions, or any other supported [quick action](../quick_actions.md) in the selected issue template. +Once enabled, an issue will be opened automatically when an alert is triggered. The author of the issue will be the GitLab Alert Bot. To further customize the issue, you can add labels, mentions, or any other supported [quick action](../quick_actions.md) in the selected issue template. If the metric exceeds the threshold of the alert for over 5 minutes, an email will be sent to all [Maintainers and Owners](../../permissions.md#project-members-permissions) of the project. diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index 10eb3a4f3b7..a0f500a939f 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -1016,7 +1016,7 @@ X-Gitlab-Event: Pipeline Hook "email": "user@gitlab.com" } }, - "jobs":[ + "builds":[ { "id": 380, "stage": "deploy", diff --git a/doc/user/project/merge_requests/index.md b/doc/user/project/merge_requests/index.md index 723195224d0..09736c7fc7e 100644 --- a/doc/user/project/merge_requests/index.md +++ b/doc/user/project/merge_requests/index.md @@ -547,13 +547,13 @@ Add the following alias to your `~/.gitconfig`: Now you can check out a particular merge request from any repository and any remote. For example, to check out the merge request with ID 5 as shown in GitLab -from the `upstream` remote, do: +from the `origin` remote, do: ``` -git mr upstream 5 +git mr origin 5 ``` -This will fetch the merge request into a local `mr-upstream-5` branch and check +This will fetch the merge request into a local `mr-origin-5` branch and check it out. #### Checkout locally by modifying `.git/config` for a given repository diff --git a/doc/user/project/pages/getting_started_part_two.md b/doc/user/project/pages/getting_started_part_two.md index faf51154e3b..b74520e6556 100644 --- a/doc/user/project/pages/getting_started_part_two.md +++ b/doc/user/project/pages/getting_started_part_two.md @@ -88,7 +88,7 @@ website from your project's **Settings > Pages**. You can also take some **optional** further steps: -- _Remove the fork relationship._ The fork relashionship is necessary to contribute back to the project you originally forked from. If you don't have any intentions to do so, you can remove it. To do so, navigate to your project's **Settings**, expand **Advanced settings**, and scroll down to **Remove fork relationship**: +- _Remove the fork relationship._ The fork relationship is necessary to contribute back to the project you originally forked from. If you don't have any intentions to do so, you can remove it. To do so, navigate to your project's **Settings**, expand **Advanced settings**, and scroll down to **Remove fork relationship**:  diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md index 15eb862b431..1d640966013 100644 --- a/doc/user/project/quick_actions.md +++ b/doc/user/project/quick_actions.md @@ -35,7 +35,7 @@ discussions, and descriptions: | `/label ~label1 ~label2` | Add label(s). Label names can also start without ~ but mixed syntax is not supported. | ✓ | ✓ | | `/unlabel ~label1 ~label2` | Remove all or specific label(s)| ✓ | ✓ | | `/relabel ~label1 ~label2` | Replace label | ✓ | ✓ | -| <code>/copy_metadata #issue | !merge_request</code> | Copy labels and milestone from other issue or merge request | ✓ | ✓ | +| <code>/copy_metadata #issue | !merge_request</code> | Copy labels and milestone from other issue or merge request in the project | ✓ | ✓ | | <code>/estimate <1w 3d 2h 14m></code> | Set time estimate | ✓ | ✓ | | `/remove_estimate` | Remove time estimate | ✓ | ✓ | | <code>/spend <time(1h 30m | -1h 5m)> <date(YYYY-MM-DD)></code> | Add or subtract spent time; optionally, specify the date that time was spent on | ✓ | ✓ | diff --git a/doc/user/project/repository/index.md b/doc/user/project/repository/index.md index 97ecc4c0d65..cb514b76a4e 100644 --- a/doc/user/project/repository/index.md +++ b/doc/user/project/repository/index.md @@ -173,11 +173,15 @@ Via command line, you can commit multiple times before pushing. ## Repository size -On GitLab.com, your [repository size limit is 10GB](../../gitlab_com/index.md#repository-size-limit) -(including LFS). For other instances, the repository size is limited by your -system administrators. +A project's repository size is reported on the project's **Details** page. The reported size is +updated every 15 minutes at most, so may not reflect recent activity. -You can also [reduce a repository size using Git](reducing_the_repo_size_using_git.md). +The repository size for: + +- GitLab.com [is set by GitLab](../../gitlab_com/index.md#repository-size-limit). +- Self-managed instances is set by your GitLab administrators. + +You can [reduce a repository's size using Git](reducing_the_repo_size_using_git.md). ## Contributors diff --git a/doc/user/project/settings/img/import_export_download_export.png b/doc/user/project/settings/img/import_export_download_export.png Binary files differindex 668254073e8..ab81c87bf5f 100644 --- a/doc/user/project/settings/img/import_export_download_export.png +++ b/doc/user/project/settings/img/import_export_download_export.png diff --git a/doc/user/project/settings/img/import_export_export_button.png b/doc/user/project/settings/img/import_export_export_button.png Binary files differindex 7f21bb2335b..9e368739695 100644 --- a/doc/user/project/settings/img/import_export_export_button.png +++ b/doc/user/project/settings/img/import_export_export_button.png diff --git a/doc/user/project/settings/img/import_export_mail_link.png b/doc/user/project/settings/img/import_export_mail_link.png Binary files differindex 48ef42855bc..985c37650d3 100644 --- a/doc/user/project/settings/img/import_export_mail_link.png +++ b/doc/user/project/settings/img/import_export_mail_link.png diff --git a/doc/user/project/settings/img/import_export_new_project.png b/doc/user/project/settings/img/import_export_new_project.png Binary files differindex b335700c5be..fc1f73c5d6e 100644 --- a/doc/user/project/settings/img/import_export_new_project.png +++ b/doc/user/project/settings/img/import_export_new_project.png diff --git a/doc/user/project/settings/img/import_export_select_file.png b/doc/user/project/settings/img/import_export_select_file.png Binary files differindex e1e5e031d81..e3e1a5ef980 100644 --- a/doc/user/project/settings/img/import_export_select_file.png +++ b/doc/user/project/settings/img/import_export_select_file.png diff --git a/doc/user/project/settings/img/settings_edit_button.png b/doc/user/project/settings/img/settings_edit_button.png Binary files differdeleted file mode 100644 index 32bcda03c7e..00000000000 --- a/doc/user/project/settings/img/settings_edit_button.png +++ /dev/null diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index 89008fd15b9..819515d7a4c 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -2,10 +2,11 @@ >**Notes:** > -> - [Introduced][ce-3050] in GitLab 8.9. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/3050) in GitLab 8.9. > - Importing will not be possible if the import instance version differs from > that of the exporter. -> - For GitLab admins, please read through [Project import/export administration](../../../administration/raketasks/project_import_export.md). +> - For GitLab admins, please read through +> [Project import/export administration](../../../administration/raketasks/project_import_export.md). > - For existing installations, the project import option has to be enabled in > application settings (`/admin/application_settings`) under 'Import sources'. > Ask your administrator if you don't see the **GitLab export** button when @@ -14,15 +15,15 @@ > on the GitLab instance in application settings (`/admin/application_settings`) > under 'Visibility and Access Controls'. > - You can find some useful raketasks if you are an administrator in the -> [import_export](../../../administration/raketasks/project_import_export.md) -> raketask. -> - The exports are stored in a temporary [shared directory][tmp] and are deleted -> every 24 hours by a specific worker. +> [import_export](../../../administration/raketasks/project_import_export.md) raketask. +> - The exports are stored in a temporary [shared directory](../../../development/shared_files.md) +> and are deleted every 24 hours by a specific worker. > - Group members will get exported as project members, as long as the user has > maintainer or admin access to the group where the exported project lives. An admin > in the import side is required to map the users, based on email or username. > Otherwise, a supplementary comment is left to mention the original author and > the MRs, notes or issues will be owned by the importer. +> - Project members with owner access will get imported as maintainers. > - Control project Import/Export with the [API](../../../api/project_import_export.md). > - If an imported project contains merge requests originated from forks, > then new branches associated with such merge requests will be created @@ -76,9 +77,9 @@ The following items will NOT be exported: ## Exporting a project and its data -1. Go to the project settings page by clicking on **Edit Project**: +1. Go to your project's homepage. -  +1. Click **Settings** in the sidebar. 1. Scroll down to find the **Export project** button: @@ -97,19 +98,14 @@ The following items will NOT be exported: ## Importing the project -1. The new GitLab project import feature is at the far right of the import - options when creating a New Project. Make sure you are in the right namespace - and you have entered a project name. Click on **GitLab export**: +1. The GitLab project import feature is the first import option when creating a + new project. Click on **GitLab export**:  -1. You can see where the project will be imported to. You can now select file - exported previously: +1. Enter your project name and URL. Then select the file you exported previously:  1. Click on **Import project** to begin importing. Your newly imported project page will appear soon. - -[ce-3050]: https://gitlab.com/gitlab-org/gitlab-ce/issues/3050 -[tmp]: ../../../development/shared_files.md diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md index 99dd018a3ba..737dea1de6c 100644 --- a/doc/user/project/settings/index.md +++ b/doc/user/project/settings/index.md @@ -100,9 +100,9 @@ Only project Owners and Admin users have the [permissions] to transfer a project You can transfer an existing project into a [group](../../group/index.md) if: -1. you have at least **Maintainer** [permissions] to that group -1. you are an **Owner** of the project. - +1. You have at least **Maintainer** [permissions] to that group. +1. The project is in a subgroup you own. +1. You are at least a **Maintainer** of the project under your personal namespace. Similarly, if you are an owner of a group, you can transfer any of its projects under your own user. diff --git a/doc/user/project/web_ide/index.md b/doc/user/project/web_ide/index.md index 2a2507d98a3..a634a8b2f54 100644 --- a/doc/user/project/web_ide/index.md +++ b/doc/user/project/web_ide/index.md @@ -8,7 +8,7 @@ projects by providing an advanced editor with commit staging. ## Open the Web IDE -The Web IDE can be opened when viewing a file, from the repository file list, +You can open the Web IDE when viewing a file, from the repository file list, and from merge requests.  @@ -45,7 +45,7 @@ Single file editing is based on the [Ace Editor](https://ace.c9.io). ## Stage and commit changes -After making your changes, click the Commit button in the bottom left to +After making your changes, click the **Commit** button in the bottom left to review the list of changed files. Click on each file to review the changes and click the tick icon to stage the file. @@ -67,10 +67,11 @@ shows you a preview of the merge request diff if you commit your changes. > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/19279) in [GitLab Core][ce] 11.0. -The Web IDE can be used to quickly fix failing tests by opening the branch or -merge request in the Web IDE and opening the logs of the failed job. The status -of all jobs for the most recent pipeline and job traces for the current commit -can be accessed by clicking the **Pipelines** button in the top right. +You can use the Web IDE to quickly fix failing tests by opening +the branch or merge request in the Web IDE and opening the logs of the failed +job. You can access the status of all jobs for the most recent pipeline and job +traces for the current commit by clicking the **Pipelines** button in the top +right. The pipeline status is also shown at all times in the status bar in the bottom left. @@ -79,31 +80,31 @@ left. > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/19318) in [GitLab Core][ce] 11.0. -Switching between your authored and assigned merge requests can be done without -leaving the Web IDE. Click the dropdown in the top of the sidebar to open a list -of merge requests. You will need to commit or discard all your changes before -switching to a different merge request. +To switch between your authored and assigned merge requests, click the +dropdown in the top of the sidebar to open a list of merge requests. You will +need to commit or discard all your changes before switching to a different merge +request. ## Switching branches > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/20850) in [GitLab Core][ce] 11.2. -Switching between branches of the current project repository can be done without -leaving the Web IDE. Click the dropdown in the top of the sidebar to open a list -of branches. You will need to commit or discard all your changes before -switching to a different branch. +To switch between branches of the current project repository, click the dropdown +in the top of the sidebar to open a list of branches. +You will need to commit or discard all your changes before switching to a +different branch. ## Client Side Evaluation > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/19764) in [GitLab Core][ce] 11.2. -The Web IDE can be used to preview JavaScript projects right in the browser. +You can use the Web IDE to preview JavaScript projects right in the browser. This feature uses CodeSandbox to compile and bundle the JavaScript used to preview the web application.  -Additionally, for public projects an `Open in CodeSandbox` button is available +Additionally, for public projects an **Open in CodeSandbox** button is available to transfer the contents of the project into a public CodeSandbox project to quickly share your project with others. @@ -115,9 +116,9 @@ GitLab.com  -Once it has been enabled in application settings, projects with a -`package.json` file and a `main` entry point can be previewed inside of the Web -IDE. An example `package.json` is below. +Once you have done that, you can preview projects with a `package.json` file and +a `main` entry point inside the Web IDE. An example `package.json` is shown +below. ```json { diff --git a/doc/workflow/img/repository_mirroring_pull_settings_upper.png b/doc/workflow/img/repository_mirroring_pull_settings_upper.png Binary files differindex c60354fdca7..8e15b5a0784 100644 --- a/doc/workflow/img/repository_mirroring_pull_settings_upper.png +++ b/doc/workflow/img/repository_mirroring_pull_settings_upper.png diff --git a/doc/workflow/repository_mirroring.md b/doc/workflow/repository_mirroring.md index 2f8f1545b84..5bc5cf427ce 100644 --- a/doc/workflow/repository_mirroring.md +++ b/doc/workflow/repository_mirroring.md @@ -99,6 +99,7 @@ The repository will push soon. To force a push, click the appropriate button. ## Pulling from a remote repository **[STARTER]** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/51) in GitLab Enterprise Edition 8.2. +> [Added Git LFS support](https://gitlab.com/gitlab-org/gitlab-ee/issues/10871) in [GitLab Starter](https://about.gitlab.com/pricing/) 11.11. You can set up a repository to automatically have its branches, tags, and commits updated from an upstream repository. diff --git a/doc/workflow/shortcuts.md b/doc/workflow/shortcuts.md index 6714d08f2f7..5068b5d4d20 100644 --- a/doc/workflow/shortcuts.md +++ b/doc/workflow/shortcuts.md @@ -1,6 +1,6 @@ # GitLab keyboard shortcuts -You can see GitLab's keyboard shortcuts by using 'shift + ?' +You can see GitLab's keyboard shortcuts by using <kbd>shift</kbd> + <kbd>?</kbd> ## Global Shortcuts |
