Merge branch 'master' of gitlab_gitlab:gitlab-org/gitlab-cerunner-metrics-extractor

16 files changed, 249 insertions, 90 deletions

diff --git a/doc/security/README.md b/doc/security/README.md
index c48d5bc2065..e3fb07c69c2 100644
--- a/doc/security/README.md
+++ b/doc/security/README.md
@@ -5,9 +5,10 @@ type: index
 
 # Security
 
+- [Password storage](password_storage.md)
 - [Password length limits](password_length_limits.md)
 - [Restrict SSH key technologies and minimum length](ssh_keys_restrictions.md)
-- [Rack attack](rack_attack.md)
+- [Rate limits](rate_limits.md)
 - [Webhooks and insecure internal web services](webhooks.md)
 - [Information exclusivity](information_exclusivity.md)
 - [Reset your root password](reset_root_password.md)
diff --git a/doc/security/img/outbound_requests_section.png b/doc/security/img/outbound_requests_section.png
deleted file mode 100644
index f7783f34cdd..00000000000
--- a/doc/security/img/outbound_requests_section.png
+++ /dev/null
diff --git a/doc/security/img/outbound_requests_section_v12_2.png b/doc/security/img/outbound_requests_section_v12_2.png
new file mode 100644
index 00000000000..3dc99868a35
--- /dev/null
+++ b/doc/security/img/outbound_requests_section_v12_2.png
diff --git a/doc/security/img/whitelist.png b/doc/security/img/whitelist.png
new file mode 100644
index 00000000000..897000e804d
--- /dev/null
+++ b/doc/security/img/whitelist.png
diff --git a/doc/security/information_exclusivity.md b/doc/security/information_exclusivity.md
index 62a20d3f257..749ccf924b5 100644
--- a/doc/security/information_exclusivity.md
+++ b/doc/security/information_exclusivity.md
@@ -1,6 +1,7 @@
 ---
 type: concepts
 ---
+
 # Information exclusivity
 
 Git is a distributed version control system (DVCS). This means that everyone
diff --git a/doc/security/password_length_limits.md b/doc/security/password_length_limits.md
index d78293c75c6..9909ef4a8e4 100644
--- a/doc/security/password_length_limits.md
+++ b/doc/security/password_length_limits.md
@@ -1,19 +1,31 @@
 ---
 type: reference, howto
 ---
+
 # Custom password length limits
 
-If you want to enforce longer user passwords you can create an extra Devise
-initializer with the steps below.
+The user password length is set to a minimum of 8 characters by default.
+To change that for installations from source:
+
+1. Edit `devise_password_length.rb`:
+
+   ```sh
+   cd /home/git/gitlab
+   sudo -u git -H cp config/initializers/devise_password_length.rb.example config/initializers/devise_password_length.rb
+   sudo -u git -H editor config/initializers/devise_password_length.rb
+   ```
+
+1. Change the new password length limits:
+
+   ```ruby
+   config.password_length = 12..128
+   ```
 
-If you do not use the `devise_password_length.rb` initializer the password
-length is set to a minimum of 8 characters in `config/initializers/devise.rb`.
+   In this example, the minimum length is 12 characters, and the maximum length
+   is 128 characters.
 
-```bash
-cd /home/git/gitlab
-sudo -u git -H cp config/initializers/devise_password_length.rb.example config/initializers/devise_password_length.rb
-sudo -u git -H editor config/initializers/devise_password_length.rb   # inspect and edit the new password length limits
-```
+1. [Restart GitLab](../administration/restart_gitlab.md#installations-from-source)
+   for the changes to take effect.
 
 <!-- ## Troubleshooting
 
diff --git a/doc/security/password_storage.md b/doc/security/password_storage.md
new file mode 100644
index 00000000000..f4e32f96f7b
--- /dev/null
+++ b/doc/security/password_storage.md
@@ -0,0 +1,13 @@
+---
+type: reference
+---
+
+# Password Storage
+
+GitLab stores user passwords in a hashed format, to prevent passwords from being visible.
+
+GitLab uses the [Devise](https://github.com/plataformatec/devise) authentication library, which handles the hashing of user passwords. Password hashes are created with the following attributes:
+
+- **Hashing**: the [bcrypt](https://en.wikipedia.org/wiki/Bcrypt) hashing function is used to generate the hash of the provided password. This is a strong, industry-standard cryptographic hashing function.
+- **Stretching**: Password hashes are [stretched](https://en.wikipedia.org/wiki/Key_stretching) to harden against brute-force attacks. GitLab uses a stretching factor of 10 by default.
+- **Salting**: A [cryptographic salt](https://en.wikipedia.org/wiki/Salt_(cryptography)) is added to each password to harden against pre-computed hash and dictionary attacks. Each salt is randomly generated for each password, so that no two passwords share a salt, to further increase security.
diff --git a/doc/security/rack_attack.md b/doc/security/rack_attack.md
index 8695b5d2194..b99bfb16829 100644
--- a/doc/security/rack_attack.md
+++ b/doc/security/rack_attack.md
@@ -1,7 +1,10 @@
 ---
 type: reference, howto
 ---
-# Rack Attack
+
+# Rack Attack initializer
+
+## Overview
 
 [Rack Attack](https://github.com/kickstarter/rack-attack), also known as Rack::Attack, is a Ruby gem
 that is meant to protect GitLab with the ability to customize throttling and
@@ -13,19 +16,72 @@ If you find throttling is not enough to protect you against abusive clients,
 Rack Attack offers IP whitelisting, blacklisting, Fail2ban style filtering, and
 tracking.
 
-**Note:** Starting with 11.2, Rack Attack is disabled by default. To continue
-using Rack Attack, please enable it by [configuring `gitlab.rb` as described in Settings](#settings).
+For more information on how to use these options see the [Rack Attack README](https://github.com/kickstarter/rack-attack/blob/master/README.md).
+
+NOTE: **Note:** See
+[User and IP rate limits](../user/admin_area/settings/user_and_ip_rate_limits.md)
+for simpler limits that are configured in the UI.
+
+NOTE: **Note:** Starting with GitLab 11.2, Rack Attack is disabled by default. If your
+instance is not exposed to the public internet, it is recommended that you leave
+Rack Attack disabled.
+
+## Behavior
+
+If set up as described in the [Settings](#settings) section below, two behaviors
+will be enabled:
+
+- Protected paths will be throttled.
+- Failed authentications for Git and container registry requests will trigger a temporary IP ban.
+
+### Protected paths throttle
+
+GitLab responds with HTTP status code `429` to POST requests at protected paths
+that exceed 10 requests per minute per IP address.
+
+By default, protected paths are:
+
+```ruby
+default['gitlab']['gitlab-rails']['rack_attack_protected_paths'] = [
+  '/users/password',
+  '/users/sign_in',
+  '/api/#{API::API.version}/session.json',
+  '/api/#{API::API.version}/session',
+  '/users',
+  '/users/confirmation',
+  '/unsubscribes/',
+  '/import/github/personal_access_token'
+]
+```
+
+This header is included in responses to blocked requests:
+
+```
+Retry-After: 60
+```
+
+For example, the following are limited to a maximum 10 requests per minute:
+
+- User sign-in
+- User sign-up (if enabled)
+- User password reset
+
+After 10 requests, the client must wait a minute before it can
+try again.
+
+### Git and container registry failed authentication ban
+
+GitLab responds with HTTP status code `403` for 1 hour, if 30 failed
+authentication requests were received in a 3-minute period from a single IP address.
 
-By default, user sign-in, user sign-up (if enabled), and user password reset is
-limited to 6 requests per minute. After trying for 6 times, the client will
-have to wait for the next minute to be able to try again.
+This applies only to Git requests and container registry (`/jwt/auth`) requests
+(combined).
 
-If you installed or upgraded GitLab by following the [official guides](../install/README.md),
-Rack Attack should be disabled by default. If your instance is not exposed to any incoming
-connections, it is recommended that you leave Rack Attack disabled.
+This limit is reset by requests that authenticate successfully. For example, 29
+failed authentication requests followed by 1 successful request, followed by 29
+more failed authentication requests would not trigger a ban.
 
-For more information on how to use these options check out
-[rack-attack README](https://github.com/kickstarter/rack-attack/blob/master/README.md).
+No response headers are provided.
 
 ## Settings
 
@@ -34,34 +90,34 @@ For more information on how to use these options check out
 1. Open `/etc/gitlab/gitlab.rb` with your editor
 1. Add the following:
 
-    ```ruby
-    gitlab_rails['rack_attack_git_basic_auth'] = {
-      'enabled' => true,
-      'ip_whitelist' => ["127.0.0.1"],
-      'maxretry' => 10, # Limit the number of Git HTTP authentication attempts per IP
-      'findtime' => 60, # Reset the auth attempt counter per IP after 60 seconds
-      'bantime' => 3600 # Ban an IP for one hour (3600s) after too many auth attempts
-    }
-    ```
+   ```ruby
+   gitlab_rails['rack_attack_git_basic_auth'] = {
+     'enabled' => true,
+     'ip_whitelist' => ["127.0.0.1"],
+     'maxretry' => 10, # Limit the number of Git HTTP authentication attempts per IP
+     'findtime' => 60, # Reset the auth attempt counter per IP after 60 seconds
+     'bantime' => 3600 # Ban an IP for one hour (3600s) after too many auth attempts
+   }
+   ```
 
 1. Reconfigure GitLab:
 
-    ```
-    sudo gitlab-ctl reconfigure
-    ```
+   ```
+   sudo gitlab-ctl reconfigure
+   ```
 
 The following settings can be configured:
 
 - `enabled`: By default this is set to `false`. Set this to `true` to enable Rack Attack.
 - `ip_whitelist`: Whitelist any IPs from being blocked. They must be formatted as strings within a Ruby array.
-   CIDR notation is supported in GitLab v12.1 and up.
-   For example, `["127.0.0.1", "127.0.0.2", "127.0.0.3", "192.168.0.1/24"]`.
+  CIDR notation is supported in GitLab v12.1 and up.
+  For example, `["127.0.0.1", "127.0.0.2", "127.0.0.3", "192.168.0.1/24"]`.
 - `maxretry`: The maximum amount of times a request can be made in the
-   specified time.
+  specified time.
 - `findtime`: The maximum amount of time that failed requests can count against an IP
-   before it's blacklisted (in seconds).
+  before it's blacklisted (in seconds).
 - `bantime`: The total amount of time that a blacklisted IP will be blocked (in
-   seconds).
+  seconds).
 
 **Installations from source**
 
@@ -71,25 +127,25 @@ taken in order to enable protection for your GitLab instance:
 
 1. In `config/application.rb` find and uncomment the following line:
 
-    ```ruby
-    config.middleware.use Rack::Attack
-    ```
+   ```ruby
+   config.middleware.use Rack::Attack
+   ```
 
 1. Copy `config/initializers/rack_attack.rb.example` to `config/initializers/rack_attack.rb`
 1. Open `config/initializers/rack_attack.rb`, review the
    `paths_to_be_protected`, and add any other path you need protecting
 1. Restart GitLab:
 
-    ```sh
-    sudo service gitlab restart
-    ```
+   ```sh
+   sudo service gitlab restart
+   ```
 
 If you want more restrictive/relaxed throttle rules, edit
 `config/initializers/rack_attack.rb` and change the `limit` or `period` values.
 For example, more relaxed throttle rules will be if you set
 `limit: 3` and `period: 1.seconds` (this will allow 3 requests per second).
 You can also add other paths to the protected list by adding to `paths_to_be_protected`
-variable. If you change any of these settings do not forget to restart your
+variable. If you change any of these settings you must restart your
 GitLab instance.
 
 ## Remove blocked IPs from Rack Attack via Redis
@@ -98,28 +154,28 @@ In case you want to remove a blocked IP, follow these steps:
 
 1. Find the IPs that have been blocked in the production log:
 
-    ```sh
-    grep "Rack_Attack" /var/log/gitlab/gitlab-rails/auth.log
-    ```
+   ```sh
+   grep "Rack_Attack" /var/log/gitlab/gitlab-rails/auth.log
+   ```
 
 1. Since the blacklist is stored in Redis, you need to open up `redis-cli`:
 
-    ```sh
-    /opt/gitlab/embedded/bin/redis-cli -s /var/opt/gitlab/redis/redis.socket
-    ```
+   ```sh
+   /opt/gitlab/embedded/bin/redis-cli -s /var/opt/gitlab/redis/redis.socket
+   ```
 
 1. You can remove the block using the following syntax, replacing `<ip>` with
    the actual IP that is blacklisted:
 
-    ```
-    del cache:gitlab:rack::attack:allow2ban:ban:<ip>
-    ```
+   ```
+   del cache:gitlab:rack::attack:allow2ban:ban:<ip>
+   ```
 
 1. Confirm that the key with the IP no longer shows up:
 
-    ```
-    keys *rack::attack*
-    ```
+   ```
+   keys *rack::attack*
+   ```
 
 1. Optionally, add the IP to the whitelist to prevent it from being blacklisted
    again (see [settings](#settings)).
@@ -136,8 +192,8 @@ the load balancer. In that case, you will need to:
 1. Whitelist the load balancer's IP address(es) in the Rack Attack [settings](#settings).
 1. Reconfigure GitLab:
 
-    ```
-    sudo gitlab-ctl reconfigure
-    ```
+   ```
+   sudo gitlab-ctl reconfigure
+   ```
 
 1. [Remove the block via Redis.](#remove-blocked-ips-from-rack-attack-via-redis)
diff --git a/doc/security/rate_limits.md b/doc/security/rate_limits.md
new file mode 100644
index 00000000000..80088da77a0
--- /dev/null
+++ b/doc/security/rate_limits.md
@@ -0,0 +1,32 @@
+---
+type: reference, howto
+---
+
+# Rate limits
+
+NOTE: **Note:**
+For GitLab.com, please see
+[GitLab.com-specific rate limits](../user/gitlab_com/index.md#gitlabcom-specific-rate-limits).
+
+Rate limiting is a common technique used to improve the security and durability
+of a web application.
+
+For example, a simple script can make thousands of web requests per second.
+Whether malicious, apathetic, or just a bug, your application and infrastructure
+may not be able to cope with the load. For more details, see
+[Denial-of-service attack](https://en.wikipedia.org/wiki/Denial-of-service_attack).
+Most cases can be mitigated by limiting the rate of requests from a single IP address.
+
+Most [brute-force attacks](https://en.wikipedia.org/wiki/Brute-force_attack) are
+similarly mitigated by a rate limit.
+
+## Admin Area settings
+
+- [User and IP rate limits](../user/admin_area/settings/user_and_ip_rate_limits.md).
+- [Rate limits on raw endpoints](../user/admin_area/settings/rate_limits_on_raw_endpoints.md)
+
+## Rack Attack initializer
+
+This method of rate limiting is cumbersome, but has some advantages. It allows
+throttling of specific paths, and is also integrated into Git and container
+registry requests. See [Rack Attack initializer](rack_attack.md).
diff --git a/doc/security/reset_root_password.md b/doc/security/reset_root_password.md
index a58d70f0ff2..ec360e2d338 100644
--- a/doc/security/reset_root_password.md
+++ b/doc/security/reset_root_password.md
@@ -1,6 +1,7 @@
 ---
 type: howto
 ---
+
 # How to reset your root password
 
 To reset your root password, first log into your server with root privileges.
@@ -22,7 +23,7 @@ user = User.where(id: 1).first
 or
 
 ```bash
-user = User.find_by(email: 'admin@local.host')
+user = User.find_by(email: 'admin@example.com')
 ```
 
 Now you can change your password:
diff --git a/doc/security/ssh_keys_restrictions.md b/doc/security/ssh_keys_restrictions.md
index ae4cc44519e..4c60daf77f4 100644
--- a/doc/security/ssh_keys_restrictions.md
+++ b/doc/security/ssh_keys_restrictions.md
@@ -1,6 +1,7 @@
 ---
 type: reference, howto
 ---
+
 # Restrict allowed SSH key technologies and minimum length
 
 `ssh-keygen` allows users to create RSA keys with as few as 768 bits, which
diff --git a/doc/security/two_factor_authentication.md b/doc/security/two_factor_authentication.md
index 49dadd5abc2..b08d9ffa26e 100644
--- a/doc/security/two_factor_authentication.md
+++ b/doc/security/two_factor_authentication.md
@@ -1,6 +1,7 @@
 ---
 type: howto
 ---
+
 # Enforce Two-factor Authentication (2FA)
 
 Two-factor Authentication (2FA) provides an additional level of security to your
@@ -48,11 +49,11 @@ need to be administrator or owner of the group.
 The following are important notes about 2FA:
 
 - Projects belonging to a 2FA-enabled group that
-  [is shared](../user/project/members/share_project_with_groups.md) 
+  [is shared](../user/project/members/share_project_with_groups.md)
   with a 2FA-disabled group will *not* require members of the 2FA-disabled group to use
-  2FA for the project. For example, if project *P* belongs to 2FA-enabled group *A* and 
+  2FA for the project. For example, if project *P* belongs to 2FA-enabled group *A* and
   is shared with 2FA-disabled group *B*, members of group *B* can access project *P*
-  without 2FA. To ensure this scenario doesn't occur, 
+  without 2FA. To ensure this scenario doesn't occur,
   [prevent sharing of projects](../user/group/index.md#share-with-group-lock)
   for the 2FA-enabled group.
 - If you add additional members to a project within a group or subgroup that has
diff --git a/doc/security/unlock_user.md b/doc/security/unlock_user.md
index 75cf754e197..d34826c853c 100644
--- a/doc/security/unlock_user.md
+++ b/doc/security/unlock_user.md
@@ -1,38 +1,45 @@
 ---
 type: howto
 ---
-# How to unlock a locked user
 
-To unlock a locked user, first log into your server with root privileges.
+# How to unlock a locked user from the command line
 
-Start a Ruby on Rails console with this command:
+After six failed login attempts a user gets in a locked state.
 
+To unlock a locked user:
 
-```bash
-gitlab-rails console production
-```
+1. SSH into your GitLab server.
+1. Start a Ruby on Rails console:
 
-Wait until the console has loaded.
+   ```sh
+   ## For Omnibus GitLab
+   sudo gitlab-rails console production
 
-There are multiple ways to find your user. You can search for email or username.
+   ## For installations from source
+   sudo -u git -H bundle exec rails console RAILS_ENV=production
+   ```
 
-```bash
-user = User.where(id: 1).first
-```
+1. Find the user to unlock. You can search by email or ID.
 
-or
+   ```ruby
+   user = User.find_by(email: 'admin@local.host')
+   ```
 
-```bash
-user = User.find_by(email: 'admin@local.host')
-```
+   or
 
-Unlock the user:
+   ```ruby
+   user = User.where(id: 1).first
+   ```
 
-```bash
-user.unlock_access!
-```
+1. Unlock the user:
 
-Exit the console, the user should now be able to log in again.
+   ```ruby
+   user.unlock_access!
+   ```
+
+1. Exit the console with <kbd>Ctrl</kbd>+<kbd>d</kbd>
+
+The user should now be able to log in.
 
 <!-- ## Troubleshooting
 
diff --git a/doc/security/user_email_confirmation.md b/doc/security/user_email_confirmation.md
index f0af0a7ac6a..7ba50acbb06 100644
--- a/doc/security/user_email_confirmation.md
+++ b/doc/security/user_email_confirmation.md
@@ -1,6 +1,7 @@
 ---
 type: howto
 ---
+
 # User email confirmation at sign-up
 
 GitLab can be configured to require confirmation of a user's email address when
diff --git a/doc/security/user_file_uploads.md b/doc/security/user_file_uploads.md
index 00a2607b607..9fc8f7ec985 100644
--- a/doc/security/user_file_uploads.md
+++ b/doc/security/user_file_uploads.md
@@ -1,6 +1,7 @@
 ---
 type: reference
 ---
+
 # User File Uploads
 
 Images that are attached to issues, merge requests, or comments
@@ -12,7 +13,7 @@ image contains sensitive information.
 Authentication is not enabled because images must be visible in the body of
 notification emails, which are often read from email clients that are not
 authenticated with GitLab, such as Outlook, Apple Mail, or the Mail app on your
-mobile device. 
+mobile device.
 
 >**Note:**
 Non-image attachments do require authentication to be viewed.
diff --git a/doc/security/webhooks.md b/doc/security/webhooks.md
index d4fa088cb15..e39bc9a9626 100644
--- a/doc/security/webhooks.md
+++ b/doc/security/webhooks.md
@@ -1,6 +1,7 @@
 ---
 type: concepts, reference, howto
 ---
+
 # Webhooks and insecure internal web services
 
 If you have non-GitLab web services running on your GitLab server or within its
@@ -33,15 +34,46 @@ to 127.0.0.1, ::1 and 0.0.0.0, as well as IPv4 10.0.0.0/8, 172.16.0.0/12,
 192.168.0.0/16 and IPv6 site-local (ffc0::/10) addresses won't be allowed.
 
 This behavior can be overridden by enabling the option *"Allow requests to the
-local network from hooks and services"* in the *"Outbound requests"* section
+local network from web hooks and services"* in the *"Outbound requests"* section
 inside the Admin area under **Settings**
 (`/admin/application_settings/network`):
 
-![Outbound requests admin settings](img/outbound_requests_section.png)
+![Outbound requests admin settings](img/outbound_requests_section_v12_2.png)
+
+NOTE: **Note:**
+*System hooks* are enabled to make requests to local network by default since they are
+set up by administrators. However, you can turn this off by disabling the
+**Allow requests to the local network from system hooks** option.
+
+## Whitelist for local requests
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/44496) in GitLab 12.2
+
+You can allow certain domains and IP addresses to be accessible to both *system hooks*
+and *webhooks* even when local requests are not allowed by adding them to the
+whitelist. Navigate to **Admin Area > Settings > Network** (`/admin/application_settings/network`)
+and expand **Outbound requests**:
+
+![Outbound local requests whitelist](img/whitelist.png)
+
+The whilelist entries can be separated by semicolons, commas or whitespaces
+(including newlines) and be in different formats like hostnames, IP addresses and/or
+IP ranges. IPv6 is supported. Hostnames that contain unicode characters should
+use IDNA encoding.
+
+The whitelist can hold a maximum of 1000 entries. Each entry can be a maximum of
+255 characters.
+
+Example:
+
+```text
+example.com;gitlab.example.com
+127.0.0.1,1:0:0:0:0:0:0:1
+127.0.0.0/8 1:0:0:0:0:0:0:0/124
+```
 
->**Note:**
-*System hooks* are exempt from this protection because they are set up by
-admins.
+NOTE: **Note:**
+Wildcards (`*.example.com`) and ports (`127.0.0.1:3000`) are not currently supported.
 
 <!-- ## Troubleshooting