Merge branch 'master' into update-todo-in-uiupdate-todo-in-ui

* master: (435 commits)
  Change occurrence of Sidekiq::Testing.inline!
  Fix order-dependent spec failure in appearance_spec.rb
  Put a failed example from appearance_spec in quarantine
  Cache PerformanceBar.allowed_user_ids list locally and in Redis
  Add Grafana to Admin > Monitoring menu when enabled
  Add changelog entry
  Add salesforce logo
  Move error_tracking_frontend specs to Jest
  Only save Peek session in Redis when Peek is enabled
  Migrate markdown header_spec.js to Jest
  Fix golint command in Go guide doc to be recursive
  Move images to their own dirs
  Gitlab -> GitLab
  Re-align CE and EE API docs
  Rename Release groups in issue_workflow.md
  Update api docs to finish aligning EE and CE docs
  Update locale.pot
  Update TODO: allow_collaboration column renaming
  Show upcoming status for releases
  Rebased and squashed commits
  ...

3 files changed, 238 insertions, 182 deletions

diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md
index c7aa22b11f8..092b4375208 100644
--- a/doc/raketasks/backup_restore.md
+++ b/doc/raketasks/backup_restore.md
@@ -18,16 +18,16 @@ installed on your system.
 
 If you installed GitLab:
 
--   Using the Omnibus package, you're all set.
--   From source, make sure `rsync` is installed:
+- Using the Omnibus package, you're all set.
+- From source, make sure `rsync` is installed:
 
-    ```sh
-    # Debian/Ubuntu
-    sudo apt-get install rsync
+  ```sh
+  # Debian/Ubuntu
+  sudo apt-get install rsync
 
-    # RHEL/CentOS
-    sudo yum install rsync
-    ```
+  # RHEL/CentOS
+  sudo yum install rsync
+  ```
 
 ### Tar
 
@@ -269,17 +269,17 @@ For Omnibus GitLab packages:
 
 1. Add the following to `/etc/gitlab/gitlab.rb`:
 
-    ```ruby
-    gitlab_rails['backup_upload_connection'] = {
-      'provider' => 'AWS',
-      'region' => 'eu-west-1',
-      'aws_access_key_id' => 'AKIAKIAKI',
-      'aws_secret_access_key' => 'secret123'
-      # If using an IAM Profile, don't configure aws_access_key_id & aws_secret_access_key
-      # 'use_iam_profile' => true
-    }
-    gitlab_rails['backup_upload_remote_directory'] = 'my.s3.bucket'
-    ```
+   ```ruby
+   gitlab_rails['backup_upload_connection'] = {
+     'provider' => 'AWS',
+     'region' => 'eu-west-1',
+     'aws_access_key_id' => 'AKIAKIAKI',
+     'aws_secret_access_key' => 'secret123'
+     # If using an IAM Profile, don't configure aws_access_key_id & aws_secret_access_key
+     # 'use_iam_profile' => true
+   }
+   gitlab_rails['backup_upload_remote_directory'] = 'my.s3.bucket'
+   ```
 
 1. [Reconfigure GitLab] for the changes to take effect
 
@@ -289,16 +289,16 @@ This example can be used for a bucket in Amsterdam (AMS3).
 
 1. Add the following to `/etc/gitlab/gitlab.rb`:
 
-    ```ruby
-    gitlab_rails['backup_upload_connection'] = {
-      'provider' => 'AWS',
-      'region' => 'ams3',
-      'aws_access_key_id' => 'AKIAKIAKI',
-      'aws_secret_access_key' => 'secret123',
-      'endpoint'              => 'https://ams3.digitaloceanspaces.com'
-    }
-    gitlab_rails['backup_upload_remote_directory'] = 'my.s3.bucket'
-    ```
+   ```ruby
+   gitlab_rails['backup_upload_connection'] = {
+     'provider' => 'AWS',
+     'region' => 'ams3',
+     'aws_access_key_id' => 'AKIAKIAKI',
+     'aws_secret_access_key' => 'secret123',
+     'endpoint'              => 'https://ams3.digitaloceanspaces.com'
+   }
+   gitlab_rails['backup_upload_remote_directory'] = 'my.s3.bucket'
+   ```
 
 1. [Reconfigure GitLab] for the changes to take effect
 
@@ -321,31 +321,31 @@ For installations from source:
 
 1. Edit `home/git/gitlab/config/gitlab.yml`:
 
-    ```yaml
-      backup:
-        # snip
-        upload:
-          # Fog storage connection settings, see http://fog.io/storage/ .
-          connection:
-            provider: AWS
-            region: eu-west-1
-            aws_access_key_id: AKIAKIAKI
-            aws_secret_access_key: 'secret123'
-            # If using an IAM Profile, leave aws_access_key_id & aws_secret_access_key empty
-            # ie. aws_access_key_id: ''
-            # use_iam_profile: 'true'
-          # The remote 'directory' to store your backups. For S3, this would be the bucket name.
-          remote_directory: 'my.s3.bucket'
-          # Turns on AWS Server-Side Encryption with Amazon S3-Managed Keys for backups, this is optional
-          # encryption: 'AES256'
-          # Turns on AWS Server-Side Encryption with Amazon Customer-Provided Encryption Keys for backups, this is optional
-          #   This should be set to the base64-encoded encryption key for Amazon S3 to use to encrypt or decrypt your data.
-          #   'encryption' must also be set in order for this to have any effect.
-          #   To avoid storing the key on disk, the key can also be specified via the `GITLAB_BACKUP_ENCRYPTION_KEY` environment variable.
-          # encryption_key: '<base64 key>'
-          # Specifies Amazon S3 storage class to use for backups, this is optional
-          # storage_class: 'STANDARD'
-    ```
+   ```yaml
+     backup:
+       # snip
+       upload:
+         # Fog storage connection settings, see http://fog.io/storage/ .
+         connection:
+           provider: AWS
+           region: eu-west-1
+           aws_access_key_id: AKIAKIAKI
+           aws_secret_access_key: 'secret123'
+           # If using an IAM Profile, leave aws_access_key_id & aws_secret_access_key empty
+           # ie. aws_access_key_id: ''
+           # use_iam_profile: 'true'
+         # The remote 'directory' to store your backups. For S3, this would be the bucket name.
+         remote_directory: 'my.s3.bucket'
+         # Turns on AWS Server-Side Encryption with Amazon S3-Managed Keys for backups, this is optional
+         # encryption: 'AES256'
+         # Turns on AWS Server-Side Encryption with Amazon Customer-Provided Encryption Keys for backups, this is optional
+         #   This should be set to the base64-encoded encryption key for Amazon S3 to use to encrypt or decrypt your data.
+         #   'encryption' must also be set in order for this to have any effect.
+         #   To avoid storing the key on disk, the key can also be specified via the `GITLAB_BACKUP_ENCRYPTION_KEY` environment variable.
+         # encryption_key: '<base64 key>'
+         # Specifies Amazon S3 storage class to use for backups, this is optional
+         # storage_class: 'STANDARD'
+   ```
 
 1. [Restart GitLab] for the changes to take effect
 
@@ -417,14 +417,14 @@ For Omnibus GitLab packages:
 
 1. Edit `/etc/gitlab/gitlab.rb`:
 
-    ```ruby
-    gitlab_rails['backup_upload_connection'] = {
-      'provider' => 'Google',
-      'google_storage_access_key_id' => 'Access Key',
-      'google_storage_secret_access_key' => 'Secret'
-    }
-    gitlab_rails['backup_upload_remote_directory'] = 'my.google.bucket'
-    ```
+   ```ruby
+   gitlab_rails['backup_upload_connection'] = {
+     'provider' => 'Google',
+     'google_storage_access_key_id' => 'Access Key',
+     'google_storage_secret_access_key' => 'Secret'
+   }
+   gitlab_rails['backup_upload_remote_directory'] = 'my.google.bucket'
+   ```
 
 1. [Reconfigure GitLab] for the changes to take effect
 
@@ -434,15 +434,15 @@ For installations from source:
 
 1. Edit `home/git/gitlab/config/gitlab.yml`:
 
-    ```yaml
-      backup:
-        upload:
-          connection:
-            provider: 'Google'
-            google_storage_access_key_id: 'Access Key'
-            google_storage_secret_access_key: 'Secret'
-          remote_directory: 'my.google.bucket'
-    ```
+   ```yaml
+     backup:
+       upload:
+         connection:
+           provider: 'Google'
+           google_storage_access_key_id: 'Access Key'
+           google_storage_secret_access_key: 'Secret'
+         remote_directory: 'my.google.bucket'
+   ```
 
 1. [Restart GitLab] for the changes to take effect
 
@@ -477,16 +477,16 @@ For Omnibus GitLab packages:
 
 1. Edit `/etc/gitlab/gitlab.rb`:
 
-    ```ruby
-    gitlab_rails['backup_upload_connection'] = {
-      :provider => 'Local',
-      :local_root => '/mnt/backups'
-    }
+   ```ruby
+   gitlab_rails['backup_upload_connection'] = {
+     :provider => 'Local',
+     :local_root => '/mnt/backups'
+   }
 
-    # The directory inside the mounted folder to copy backups to
-    # Use '.' to store them in the root directory
-    gitlab_rails['backup_upload_remote_directory'] = 'gitlab_backups'
-    ```
+   # The directory inside the mounted folder to copy backups to
+   # Use '.' to store them in the root directory
+   gitlab_rails['backup_upload_remote_directory'] = 'gitlab_backups'
+   ```
 
 1. [Reconfigure GitLab] for the changes to take effect.
 
@@ -496,17 +496,17 @@ For installations from source:
 
 1. Edit `home/git/gitlab/config/gitlab.yml`:
 
-    ```yaml
-    backup:
-      upload:
-        # Fog storage connection settings, see http://fog.io/storage/ .
-        connection:
-          provider: Local
-          local_root: '/mnt/backups'
-        # The directory inside the mounted folder to copy backups to
-        # Use '.' to store them in the root directory
-        remote_directory: 'gitlab_backups'
-    ```
+   ```yaml
+   backup:
+     upload:
+       # Fog storage connection settings, see http://fog.io/storage/ .
+       connection:
+         provider: Local
+         local_root: '/mnt/backups'
+       # The directory inside the mounted folder to copy backups to
+       # Use '.' to store them in the root directory
+       remote_directory: 'gitlab_backups'
+   ```
 
 1. [Restart GitLab] for the changes to take effect.
 
@@ -521,9 +521,9 @@ For Omnibus GitLab packages:
 
 1. Edit `/etc/gitlab/gitlab.rb`:
 
-    ```ruby
-    gitlab_rails['backup_archive_permissions'] = 0644 # Makes the backup archives world-readable
-    ```
+   ```ruby
+   gitlab_rails['backup_archive_permissions'] = 0644 # Makes the backup archives world-readable
+   ```
 
 1. [Reconfigure GitLab] for the changes to take effect.
 
@@ -533,10 +533,10 @@ For installations from source:
 
 1. Edit `/home/git/gitlab/config/gitlab.yml`:
 
-    ```yaml
-    backup:
-      archive_permissions: 0644 # Makes the backup archives world-readable
-    ```
+   ```yaml
+   backup:
+     archive_permissions: 0644 # Makes the backup archives world-readable
+   ```
 
 1. [Restart GitLab] for the changes to take effect.
 
@@ -550,10 +550,10 @@ For Omnibus GitLab packages:
 
 1. Edit `/etc/gitlab/gitlab.rb`:
 
-    ```ruby
-    ## Limit backup lifetime to 7 days - 604800 seconds
-    gitlab_rails['backup_keep_time'] = 604800
-    ```
+   ```ruby
+   ## Limit backup lifetime to 7 days - 604800 seconds
+   gitlab_rails['backup_keep_time'] = 604800
+   ```
 
 1. [Reconfigure GitLab] for the changes to take effect.
 
@@ -586,11 +586,11 @@ For installations from source:
 
 1. Edit `home/git/gitlab/config/gitlab.yml`:
 
-    ```yaml
-    backup:
-      ## Limit backup lifetime to 7 days - 604800 seconds
-      keep_time: 604800
-    ```
+   ```yaml
+   backup:
+     ## Limit backup lifetime to 7 days - 604800 seconds
+     keep_time: 604800
+   ```
 
 1. [Restart GitLab] for the changes to take effect.
 
@@ -840,13 +840,13 @@ columns containing sensitive information. If the key is lost, GitLab will be
 unable to decrypt those columns. This will break a wide range of functionality,
 including (but not restricted to):
 
-* [CI/CD variables](../ci/variables/README.md)
-* [Kubernetes / GCP integration](../user/project/clusters/index.md)
-* [Custom Pages domains](../user/project/pages/getting_started_part_three.md)
-* [Project error tracking](../user/project/operations/error_tracking.md)
-* [Runner authentication](../ci/runners/README.md)
-* [Project mirroring](../workflow/repository_mirroring.md)
-* [Web hooks](../user/project/integrations/webhooks.md)
+- [CI/CD variables](../ci/variables/README.md)
+- [Kubernetes / GCP integration](../user/project/clusters/index.md)
+- [Custom Pages domains](../user/project/pages/getting_started_part_three.md)
+- [Project error tracking](../user/project/operations/error_tracking.md)
+- [Runner authentication](../ci/runners/README.md)
+- [Project mirroring](../workflow/repository_mirroring.md)
+- [Web hooks](../user/project/integrations/webhooks.md)
 
 In cases like CI/CD variables and Runner authentication, you might
 experience some unexpected behavior such as:
@@ -865,72 +865,71 @@ backup beforehand.
 
 #### Reset CI/CD variables
 
-1.  Enter the DB console:
+1. Enter the DB console:
 
-    For Omnibus GitLab packages:
+   For Omnibus GitLab packages:
 
-    ```sh
-    sudo gitlab-rails dbconsole
-    ```
+   ```sh
+   sudo gitlab-rails dbconsole
+   ```
 
-    For installations from source:
+   For installations from source:
 
-    ```sh
-    sudo -u git -H bundle exec rails dbconsole RAILS_ENV=production
-    ```
+   ```sh
+   sudo -u git -H bundle exec rails dbconsole RAILS_ENV=production
+   ```
 
-1.  Check the `ci_group_variables` and `ci_variables` tables:
+1. Check the `ci_group_variables` and `ci_variables` tables:
 
-    ```sql
-    SELECT * FROM public."ci_group_variables";
-    SELECT * FROM public."ci_variables";
-    ```
+   ```sql
+   SELECT * FROM public."ci_group_variables";
+   SELECT * FROM public."ci_variables";
+   ```
 
-    Those are the variables that you need to delete.
+   Those are the variables that you need to delete.
 
-1.  Drop the table:
+1. Drop the table:
 
-    ```sql
-    DELETE FROM ci_group_variables;
-    DELETE FROM ci_variables;
-    ```
+   ```sql
+   DELETE FROM ci_group_variables;
+   DELETE FROM ci_variables;
+   ```
 
 1. You may need to reconfigure or restart GitLab for the changes to take
    effect.
 
-
 #### Reset Runner registration tokens
 
-1.  Enter the DB console:
+1. Enter the DB console:
 
-    For Omnibus GitLab packages:
+   For Omnibus GitLab packages:
 
-    ```sh
-    sudo gitlab-rails dbconsole
-    ```
+   ```sh
+   sudo gitlab-rails dbconsole
+   ```
 
-    For installations from source:
+   For installations from source:
 
-    ```sh
-    sudo -u git -H bundle exec rails dbconsole RAILS_ENV=production
-    ```
+   ```sh
+   sudo -u git -H bundle exec rails dbconsole RAILS_ENV=production
+   ```
 
 1. Clear all the tokens for projects, groups, and the whole instance:
 
-    CAUTION: **Caution:**
-    The last UPDATE operation will stop the runners being able to pick up
-    new jobs. You must register new runners.
-
-    ```sql
-    -- Clear project tokens
-    UPDATE projects SET runners_token = null, runners_token_encrypted = null;
-    -- Clear group tokens
-    UPDATE namespaces SET runners_token = null, runners_token_encrypted = null;
-    -- Clear instance tokens
-    UPDATE application_settings SET runners_registration_token_encrypted = null;
-    -- Clear runner tokens
-    UPDATE ci_runners SET token = null, token_encrypted = null;
-    ```
+   CAUTION: **Caution:**
+   The last UPDATE operation will stop the runners being able to pick up
+   new jobs. You must register new runners.
+
+   ```sql
+   -- Clear project tokens
+   UPDATE projects SET runners_token = null, runners_token_encrypted = null;
+   -- Clear group tokens
+   UPDATE namespaces SET runners_token = null, runners_token_encrypted = null;
+   -- Clear instance tokens
+   UPDATE application_settings SET runners_registration_token_encrypted = null;
+   -- Clear runner tokens
+   UPDATE ci_runners SET token = null, token_encrypted = null;
+   ```
 
 A similar strategy can be employed for the remaining features - by removing the
 data that cannot be decrypted, GitLab can be brought back into working order,
diff --git a/doc/raketasks/cleanup.md b/doc/raketasks/cleanup.md
index f5c788af578..f880f31c39e 100644
--- a/doc/raketasks/cleanup.md
+++ b/doc/raketasks/cleanup.md
@@ -92,3 +92,48 @@ I, [2018-08-02T10:26:47.598424 #45087]  INFO -- : Looking for orphaned remote up
 I, [2018-08-02T10:26:47.753131 #45087]  INFO -- : Moved to lost and found: @hashed/6b/DSC_6152.JPG -> lost_and_found/@hashed/6b/DSC_6152.JPG
 I, [2018-08-02T10:26:47.764356 #45087]  INFO -- : Moved to lost and found: @hashed/79/02/7902699be42c8a8e46fbbb4501726517e86b22c56a189f7625a6da49081b2451/711491b29d3eb08837798c4909e2aa4d/DSC00314.jpg -> lost_and_found/@hashed/79/02/7902699be42c8a8e46fbbb4501726517e86b22c56a189f7625a6da49081b2451/711491b29d3eb08837798c4909e2aa4d/DSC00314.jpg
 ```
+
+## Remove orphan artifact files
+
+When you notice there are more job artifacts files on disk than there
+should be, you can run:
+
+```shell
+gitlab-rake gitlab:cleanup:orphan_job_artifact_files
+```
+
+This command:
+
+- Scans through the entire artifacts folder.
+- Checks which files still have a record in the database.
+- If no database record is found, the file is deleted from disk.
+
+By default, this task does not delete anything but shows what it can
+delete. Run the command with `DRY_RUN=false` if you actually want to
+delete the files:
+
+```shell
+gitlab-rake gitlab:cleanup:orphan_job_artifact_files DRY_RUN=false
+```
+
+You can also limit the number of files to delete with `LIMIT`:
+
+```shell
+gitlab-rake gitlab:cleanup:orphan_job_artifact_files LIMIT=100`
+```
+
+This will only delete up to 100 files from disk. You can use this to
+delete a small set for testing purposes.
+
+If you provide `DEBUG=1`, you'll see the full path of every file that
+is detected as being an orphan.
+
+If `ionice` is installed, the tasks uses it to ensure the command is
+not causing too much load on the disk. You can configure the niceness
+level with `NICENESS`. Below are the valid levels, but consult
+`man 1 ionice` to be sure.
+
+- `0` or `None`
+- `1` or `Realtime`
+- `2` or `Best-effort` (default)
+- `3` or `Idle`
diff --git a/doc/raketasks/web_hooks.md b/doc/raketasks/web_hooks.md
index df3dab118b2..2c6ae0749dd 100644
--- a/doc/raketasks/web_hooks.md
+++ b/doc/raketasks/web_hooks.md
@@ -2,42 +2,54 @@
 
 ## Add a webhook for **ALL** projects:
 
-    # omnibus-gitlab
-    sudo gitlab-rake gitlab:web_hook:add URL="http://example.com/hook"
-    # source installations
-    bundle exec rake gitlab:web_hook:add URL="http://example.com/hook" RAILS_ENV=production
+```sh
+# omnibus-gitlab
+sudo gitlab-rake gitlab:web_hook:add URL="http://example.com/hook"
+# source installations
+bundle exec rake gitlab:web_hook:add URL="http://example.com/hook" RAILS_ENV=production
+```
 
 ## Add a webhook for projects in a given **NAMESPACE**:
 
-    # omnibus-gitlab
-    sudo gitlab-rake gitlab:web_hook:add URL="http://example.com/hook" NAMESPACE=acme
-    # source installations
-    bundle exec rake gitlab:web_hook:add URL="http://example.com/hook" NAMESPACE=acme RAILS_ENV=production
+```sh
+# omnibus-gitlab
+sudo gitlab-rake gitlab:web_hook:add URL="http://example.com/hook" NAMESPACE=acme
+# source installations
+bundle exec rake gitlab:web_hook:add URL="http://example.com/hook" NAMESPACE=acme RAILS_ENV=production
+```
 
 ## Remove a webhook from **ALL** projects using:
 
-    # omnibus-gitlab
-    sudo gitlab-rake gitlab:web_hook:rm URL="http://example.com/hook"
-    # source installations
-    bundle exec rake gitlab:web_hook:rm URL="http://example.com/hook" RAILS_ENV=production
+```sh
+# omnibus-gitlab
+sudo gitlab-rake gitlab:web_hook:rm URL="http://example.com/hook"
+# source installations
+bundle exec rake gitlab:web_hook:rm URL="http://example.com/hook" RAILS_ENV=production
+```
 
 ## Remove a webhook from projects in a given **NAMESPACE**:
 
-    # omnibus-gitlab
-    sudo gitlab-rake gitlab:web_hook:rm URL="http://example.com/hook" NAMESPACE=acme
-    # source installations
-    bundle exec rake gitlab:web_hook:rm URL="http://example.com/hook" NAMESPACE=acme RAILS_ENV=production
+```sh
+# omnibus-gitlab
+sudo gitlab-rake gitlab:web_hook:rm URL="http://example.com/hook" NAMESPACE=acme
+# source installations
+bundle exec rake gitlab:web_hook:rm URL="http://example.com/hook" NAMESPACE=acme RAILS_ENV=production
+```
 
 ## List **ALL** webhooks:
 
-    # omnibus-gitlab
-    sudo gitlab-rake gitlab:web_hook:list
-    # source installations
-    bundle exec rake gitlab:web_hook:list RAILS_ENV=production
+```sh
+# omnibus-gitlab
+sudo gitlab-rake gitlab:web_hook:list
+# source installations
+bundle exec rake gitlab:web_hook:list RAILS_ENV=production
+```
 
 ## List the webhooks from projects in a given **NAMESPACE**:
 
-    # omnibus-gitlab
-    sudo gitlab-rake gitlab:web_hook:list NAMESPACE=acme
-    # source installations
-    bundle exec rake gitlab:web_hook:list NAMESPACE=acme RAILS_ENV=production
+```sh
+# omnibus-gitlab
+sudo gitlab-rake gitlab:web_hook:list NAMESPACE=acme
+# source installations
+bundle exec rake gitlab:web_hook:list NAMESPACE=acme RAILS_ENV=production
+```