15 files changed, 184 insertions, 25 deletions

diff --git a/doc/administration/container_registry.md b/doc/administration/container_registry.md
index a1ac4a2a57c..b21bfafc096 100644
--- a/doc/administration/container_registry.md
+++ b/doc/administration/container_registry.md
@@ -1,6 +1,7 @@
 # GitLab Container Registry administration
 
 > **Notes:**
+>
 > - [Introduced][ce-4040] in GitLab 8.8.
 > - Container Registry manifest `v1` support was added in GitLab 8.9 to support
 >   Docker versions earlier than 1.10.
diff --git a/doc/administration/custom_hooks.md b/doc/administration/custom_hooks.md
index 60ad4bf4e8f..28afaf84f5a 100644
--- a/doc/administration/custom_hooks.md
+++ b/doc/administration/custom_hooks.md
@@ -51,7 +51,7 @@ Hooks can be also placed in `hooks/<hook_name>.d` (global) or
 execution of the hooks.
 
 NOTE: **Note:** `<hook_name>.d` would need to be either `pre-receive.d`,
-`post-receive.d`, or `update.d` to work properly. Any other names will be ignored. 
+`post-receive.d`, or `update.d` to work properly. Any other names will be ignored.
 
 To look in a different directory for the global custom hooks (those in
 `hooks/<hook_name.d>`), set `custom_hooks_dir` in gitlab-shell config. For
@@ -76,9 +76,21 @@ first script exiting with a non-zero value.
 
 > [Introduced][5073] in GitLab 8.10.
 
-If the commit is declined or an error occurs during the Git hook check,
-the STDERR or STDOUT message of the hook will be present in GitLab's UI.
-STDERR takes precedence over STDOUT.
+To have custom error messages appear in GitLab's UI when the commit is
+declined or an error occurs during the Git hook, your script should:
+
+- Send the custom error messages to either the script's `stdout` or `stderr`.
+- Prefix each message with `GL-HOOK-ERR:` with no characters appearing before the prefix.
+
+### Example custom error message
+
+This hook script written in bash will generate the following message in GitLab's UI:
+
+```bash
+#!/bin/sh
+echo "GL-HOOK-ERR: My custom error message.";
+exit 1
+```
 
 ![Custom message from custom Git hook](img/custom_hooks_error_msg.png)
 
diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md
index 2d4b5c65c46..f1cedb85455 100644
--- a/doc/administration/gitaly/index.md
+++ b/doc/administration/gitaly/index.md
@@ -241,12 +241,24 @@ repository from your GitLab server over HTTP.
 
 > [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/22602) in GitLab 11.8.
 
-Gitaly supports TLS credentials for GRPC authentication. To be able to communicate
+Gitaly supports TLS encryption. To be able to communicate
 with a Gitaly instance that listens for secure connections you will need to use `tls://` url
 scheme in the `gitaly_address` of the corresponding storage entry in the gitlab configuration.
 
 The admin needs to bring their own certificate as we do not provide that automatically.
-The certificate to be used needs to be installed on all Gitaly nodes and on all client nodes that communicate with it following procedures described in [GitLab custom certificate configuration](https://docs.gitlab.com/omnibus/settings/ssl.html#install-custom-public-certificates)
+The certificate to be used needs to be installed on all Gitaly nodes and on all client nodes that communicate with it following procedures described in [GitLab custom certificate configuration](https://docs.gitlab.com/omnibus/settings/ssl.html#install-custom-public-certificates).
+
+Note that it is possible to configure Gitaly servers with both an
+unencrypted listening address `listen_addr` and an encrypted listening
+address `tls_listen_addr` at the same time. This allows you to do a
+gradual transition from unencrypted to encrypted traffic, if necessary.
+
+To observe what type of connections are actually being used in a
+production environment you can use the following Prometheus query:
+
+```
+sum(rate(gitaly_connections_total[5m])) by (type)
+```
 
 ### Example TLS configuration
 
@@ -303,6 +315,66 @@ certificate_path = '/path/to/cert.pem'
 key_path = '/path/to/key.pem'
 ```
 
+## Gitaly-ruby
+
+Gitaly was developed to replace Ruby application code in gitlab-ce/ee.
+In order to save time and/or avoid the risk of rewriting existing
+application logic, in some cases we chose to copy some application code
+from gitlab-ce into Gitaly almost as-is. To be able to run that code, we
+made gitaly-ruby, which is a sidecar process for the main Gitaly Go
+process. Some examples of things that are implemented in gitaly-ruby are
+RPC's that deal with wiki's, and RPC's that create commits on behalf of
+a user, such as merge commits.
+
+### Number of gitaly-ruby workers
+
+Gitaly-ruby has much less capacity than Gitaly itself. If your Gitaly
+server has to handle a lot of request, the default setting of having
+just 1 active gitaly-ruby sidecar might not be enough. If you see
+ResourceExhausted errors from Gitaly it's very likely that you have not
+enough gitaly-ruby capacity.
+
+You can increase the number of gitaly-ruby processes on your Gitaly
+server with the following settings.
+
+Omnibus:
+
+```ruby
+# /etc/gitlab/gitlab.rb
+# Default is 2 workers. The minimum is 2; 1 worker is always reserved as
+# a passive stand-by.
+gitaly['ruby_num_workers'] = 4
+```
+
+Source:
+
+```toml
+# /home/git/gitaly/config.toml
+[gitaly-ruby]
+num_workers = 4
+```
+
+### Observing gitaly-ruby traffic
+
+Gitaly-ruby is a somewhat hidden, internal implementation detail of
+Gitaly. There is not that much visibility into what goes on inside
+gitaly-ruby processes.
+
+If you have Prometheus set up to scrape your Gitaly process, you can see
+request rates and error codes for individual RPC's in gitaly-ruby by
+querying `grpc_client_handled_total`. Strictly speaking this metric does
+not differentiate between gitaly-ruby and other RPC's, but in practice
+(as of GitLab 11.9), all gRPC calls made by Gitaly itself are internal
+calls from the main Gitaly process to one of its gitaly-ruby sidecars.
+
+Assuming your `grpc_client_handled_total` counter only observes Gitaly,
+the following query shows you RPC's are (most likely) internally
+implemented as calls to gitaly-ruby.
+
+```
+sum(rate(grpc_client_handled_total[5m])) by (grpc_method) > 0
+```
+
 ## Disabling or enabling the Gitaly service in a cluster environment
 
 If you are running Gitaly [as a remote
diff --git a/doc/administration/high_availability/nfs.md b/doc/administration/high_availability/nfs.md
index b1aaa3bca13..f406163aea0 100644
--- a/doc/administration/high_availability/nfs.md
+++ b/doc/administration/high_availability/nfs.md
@@ -71,9 +71,8 @@ bug](https://bugzilla.redhat.com/show_bug.cgi?id=1552203) that may be fixed in
 [more recent kernels with this
 commit](https://github.com/torvalds/linux/commit/95da1b3a5aded124dd1bda1e3cdb876184813140).
 
-Users encountering a similar issue may be advised to disable the NFS server
-delegation feature, which is an optimization to reduce the number of network
-round-trips needed to read or write files. To disable NFS server delegations
+GitLab recommends all NFS users disable the NFS server
+delegation feature. To disable NFS server delegations
 on an Linux NFS server, do the following:
 
 1. On the NFS server, run:
diff --git a/doc/administration/high_availability/redis.md b/doc/administration/high_availability/redis.md
index a52bc5c3b02..3daebc4d84b 100644
--- a/doc/administration/high_availability/redis.md
+++ b/doc/administration/high_availability/redis.md
@@ -14,6 +14,7 @@ a hosted cloud solution or you can use the one that comes bundled with
 Omnibus GitLab packages.
 
 > **Notes:**
+>
 > - Redis requires authentication for High Availability. See
 >  [Redis Security](https://redis.io/topics/security) documentation for more
 >  information. We recommend using a combination of a Redis password and tight
@@ -55,6 +56,7 @@ components below.
 ### High Availability with Sentinel
 
 > **Notes:**
+>
 > - Starting with GitLab `8.11`, you can configure a list of Redis Sentinel
 >   servers that will monitor a group of Redis servers to provide failover support.
 > - Starting with GitLab `8.14`, the Omnibus GitLab Enterprise Edition package
@@ -231,6 +233,7 @@ Pick the one that suits your needs.
 This is the section where we install and set up the new Redis instances.
 
 > **Notes:**
+>
 > - We assume that you have installed GitLab and all HA components from scratch. If you
 >   already have it installed and running, read how to
 >   [switch from a single-machine installation to Redis HA](#switching-from-an-existing-single-machine-installation-to-redis-ha).
diff --git a/doc/administration/img/custom_hooks_error_msg.png b/doc/administration/img/custom_hooks_error_msg.png
index 845f0de19ce..4f25c471908 100644
--- a/doc/administration/img/custom_hooks_error_msg.png
+++ b/doc/administration/img/custom_hooks_error_msg.png
diff --git a/doc/administration/job_artifacts.md b/doc/administration/job_artifacts.md
index 8522d046a92..e7792106f81 100644
--- a/doc/administration/job_artifacts.md
+++ b/doc/administration/job_artifacts.md
@@ -1,6 +1,7 @@
 # Jobs artifacts administration
 
 > **Notes:**
+>
 > - Introduced in GitLab 8.2 and GitLab Runner 0.7.0.
 > - Starting with GitLab 8.4 and GitLab Runner 1.0, the artifacts archive format changed to `ZIP`.
 > - Starting with GitLab 8.17, builds are renamed to jobs.
@@ -86,6 +87,7 @@ _The artifacts are stored by default in
 ### Using object storage
 
 > **Notes:**
+>
 > - [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/1762) in
 >   [GitLab Premium](https://about.gitlab.com/pricing/) 9.4.
 > - Since version 9.5, artifacts are [browsable](../user/project/pipelines/job_artifacts.md#browsing-artifacts),
diff --git a/doc/administration/logs.md b/doc/administration/logs.md
index 36dee75bd44..3d40cda491a 100644
--- a/doc/administration/logs.md
+++ b/doc/administration/logs.md
@@ -23,16 +23,19 @@ requests from the API are logged to a separate file in `api_json.log`.
 Each line contains a JSON line that can be ingested by Elasticsearch, Splunk, etc. For example:
 
 ```json
-{"method":"GET","path":"/gitlab/gitlab-ce/issues/1234","format":"html","controller":"Projects::IssuesController","action":"show","status":200,"duration":229.03,"view":174.07,"db":13.24,"time":"2017-08-08T20:15:54.821Z","params":[{"key":"param_key","value":"param_value"}],"remote_ip":"18.245.0.1","user_id":1,"username":"admin","gitaly_calls":76,"queue_duration": 112.47}
+{"method":"GET","path":"/gitlab/gitlab-ce/issues/1234","format":"html","controller":"Projects::IssuesController","action":"show","status":200,"duration":229.03,"view":174.07,"db":13.24,"time":"2017-08-08T20:15:54.821Z","params":[{"key":"param_key","value":"param_value"}],"remote_ip":"18.245.0.1","user_id":1,"username":"admin","gitaly_calls":76,"gitaly_duration":7.41,"queue_duration": 112.47}
 ```
 
-In this example, you can see this was a GET request for a specific issue. Notice each line also contains performance data:
+In this example, you can see this was a GET request for a specific
+issue. Notice each line also contains performance data. All times are in
+milliseconds:
 
-1. `duration`: total time in milliseconds taken to retrieve the request
-1. `queue_duration`: total time in milliseconds that the request was queued inside GitLab Workhorse
+1. `duration`: total time taken to retrieve the request
+1. `queue_duration`: total time that the request was queued inside GitLab Workhorse
 1. `view`: total time taken inside the Rails views
 1. `db`: total time to retrieve data from the database
 1. `gitaly_calls`: total number of calls made to Gitaly
+1. `gitaly_duration`: total time taken by Gitaly calls
 
 User clone/fetch activity using http transport appears in this log as `action: git_upload_pack`.
 
@@ -85,7 +88,7 @@ Introduced in GitLab 10.0, this file lives in
 It helps you see requests made directly to the API. For example:
 
 ```json
-{"time":"2018-10-29T12:49:42.123Z","severity":"INFO","duration":709.08,"db":14.59,"view":694.49,"status":200,"method":"GET","path":"/api/v4/projects","params":[{"key":"action","value":"git-upload-pack"},{"key":"changes","value":"_any"},{"key":"key_id","value":"secret"},{"key":"secret_token","value":"[FILTERED]"}],"host":"localhost","ip":"::1","ua":"Ruby","route":"/api/:version/projects","user_id":1,"username":"root","queue_duration":100.31,"gitaly_calls":30}
+{"time":"2018-10-29T12:49:42.123Z","severity":"INFO","duration":709.08,"db":14.59,"view":694.49,"status":200,"method":"GET","path":"/api/v4/projects","params":[{"key":"action","value":"git-upload-pack"},{"key":"changes","value":"_any"},{"key":"key_id","value":"secret"},{"key":"secret_token","value":"[FILTERED]"}],"host":"localhost","ip":"::1","ua":"Ruby","route":"/api/:version/projects","user_id":1,"username":"root","queue_duration":100.31,"gitaly_calls":30,"gitaly_duration":5.36}
 ```
 
 This entry above shows an access to an internal endpoint to check whether an
diff --git a/doc/administration/merge_request_diffs.md b/doc/administration/merge_request_diffs.md
index c34a9519ace..5e9ba4f640f 100644
--- a/doc/administration/merge_request_diffs.md
+++ b/doc/administration/merge_request_diffs.md
@@ -145,3 +145,47 @@ The connection settings match those provided by [Fog](https://github.com/fog), a
     ```
 
 1. Save the file and [restart GitLab](restart_gitlab.md#installations-from-source) for the changes to take effect.
+
+### Alternative in-database storage
+
+Enabling external diffs may reduce the performance of merge requests, as they
+must be retrieved in a separate operation to other data. A compromise may be
+reached by only storing outdated diffs externally, while keeping current diffs
+in the database.
+
+To enable this feature, perform the following steps:
+
+**In Omnibus installations:**
+
+1.  Edit `/etc/gitlab/gitlab.rb` and add the following line:
+
+    ```ruby
+    gitlab_rails['external_diffs_when'] = 'outdated'
+    ```
+
+1. Save the file and [reconfigure GitLab](restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect.
+
+**In installations from source:**
+
+1. Edit `/home/git/gitlab/config/gitlab.yml` and add or amend the following
+   lines:
+
+    ```yaml
+    external_diffs:
+      enabled: true
+      when: outdated
+    ```
+
+1. Save the file and [restart GitLab](restart_gitlab.md#installations-from-source) for the changes to take effect.
+
+With this feature enabled, diffs will initially stored in the database, rather
+than externally. They will be moved to external storage once any of these
+conditions become true:
+
+- A newer version of the merge request diff exists
+- The merge request was merged more than seven days ago
+- The merge request was closed more than seven day ago
+
+These rules strike a balance between space and performance by only storing
+frequently-accessed diffs in the database. Diffs that are less likely to be
+accessed are moved to external storage instead.
diff --git a/doc/administration/monitoring/index.md b/doc/administration/monitoring/index.md
index d18dddf09c0..fa0459b24ff 100644
--- a/doc/administration/monitoring/index.md
+++ b/doc/administration/monitoring/index.md
@@ -7,4 +7,4 @@ Explore our features to monitor your GitLab instance:
 - [GitHub imports](github_imports.md): Monitor the health and progress of GitLab's GitHub importer with various Prometheus metrics.
 - [Monitoring uptime](../../user/admin_area/monitoring/health_check.md): Check the server status using the health check endpoint.
   - [IP whitelists](ip_whitelist.md): Configure GitLab for monitoring endpoints that provide health check information when probed.
-- [nginx_status](https://docs.gitlab.com/omnibus/settings/nginx.html#enabling-disabling-nginx_status): Monitor your Nginx server status
+- [nginx_status](https://docs.gitlab.com/omnibus/settings/nginx.html#enablingdisabling-nginx_status): Monitor your Nginx server status
diff --git a/doc/administration/monitoring/prometheus/index.md b/doc/administration/monitoring/prometheus/index.md
index 20d7ef9bb74..f2ac155a694 100644
--- a/doc/administration/monitoring/prometheus/index.md
+++ b/doc/administration/monitoring/prometheus/index.md
@@ -1,6 +1,7 @@
 # Monitoring GitLab with Prometheus
 
 > **Notes:**
+>
 > - Prometheus and the various exporters listed in this page are bundled in the
 >   Omnibus GitLab package. Check each exporter's documentation for the timeline
 >   they got added. For installations from source you will have to install them
diff --git a/doc/administration/pages/index.md b/doc/administration/pages/index.md
index 279ad018aed..17d72b96a51 100644
--- a/doc/administration/pages/index.md
+++ b/doc/administration/pages/index.md
@@ -5,6 +5,7 @@ description: 'Learn how to administer GitLab Pages.'
 # GitLab Pages administration
 
 > **Notes:**
+>
 > - [Introduced][ee-80] in GitLab EE 8.3.
 > - Custom CNAMEs with TLS support were [introduced][ee-173] in GitLab EE 8.5.
 > - GitLab Pages [were ported][ce-14605] to Community Edition in GitLab 8.17.
@@ -124,7 +125,7 @@ The Pages daemon doesn't listen to the outside world.
     pages_external_url 'http://example.io'
     ```
 
-1. [Reconfigure GitLab][reconfigure]
+1. [Reconfigure GitLab][reconfigure].
 
 Watch the [video tutorial][video-admin] for this configuration.
 
@@ -156,7 +157,22 @@ outside world.
     where `pages-nginx.crt` and `pages-nginx.key` are the SSL cert and key,
     respectively.
 
-1. [Reconfigure GitLab][reconfigure]
+1. [Reconfigure GitLab][reconfigure].
+
+### Additional configuration for Docker container
+
+The GitLab Pages daemon will not have permissions to bind mounts when it runs
+in a Docker container. To overcome this issue you'll need to change the chroot
+behavior:
+
+1. Edit `/etc/gitlab/gitlab.rb`.
+1. Set the `inplace_chroot` to `true` for GitLab Pages:
+
+    ```shell
+    gitlab_pages['inplace_chroot'] = true
+    ```
+
+1. [Reconfigure GitLab][reconfigure].
 
 ## Advanced configuration
 
@@ -194,7 +210,7 @@ world. Custom domains are supported, but no TLS.
     `192.0.2.2` and `2001::2` are the secondary IPs the GitLab Pages daemon
     listens on. If you don't have IPv6, you can omit the IPv6 address.
 
-1. [Reconfigure GitLab][reconfigure]
+1. [Reconfigure GitLab][reconfigure].
 
 ### Custom domains with TLS support
 
@@ -228,7 +244,7 @@ world. Custom domains and TLS are supported.
     `192.0.2.2` and `2001::2` are the secondary IPs where the GitLab Pages daemon
     listens on. If you don't have IPv6, you can omit the IPv6 address.
 
-1. [Reconfigure GitLab][reconfigure]
+1. [Reconfigure GitLab][reconfigure].
 
 ### Custom domain verification
 
@@ -286,7 +302,7 @@ Follow the steps below to configure verbose logging of GitLab Pages daemon.
     gitlab_pages['log_verbose'] = true
     ```
 
-1. [Reconfigure GitLab][reconfigure]
+1. [Reconfigure GitLab][reconfigure].
 
 ## Change storage path
 
@@ -301,7 +317,7 @@ are stored.
     gitlab_rails['pages_path'] = "/mnt/storage/pages"
     ```
 
-1. [Reconfigure GitLab][reconfigure]
+1. [Reconfigure GitLab][reconfigure].
 
 ## Configure listener for reverse proxy requests
 
@@ -324,7 +340,7 @@ Omnibus GitLab 11.1.
     gitlab_pages['listen_proxy'] = "localhost:10080"
     ```
 
-1. [Reconfigure GitLab][reconfigure]
+1. [Reconfigure GitLab][reconfigure].
 
 ## Set maximum pages size
 
diff --git a/doc/administration/repository_storage_paths.md b/doc/administration/repository_storage_paths.md
index 7f25423171f..1689b0a57d6 100644
--- a/doc/administration/repository_storage_paths.md
+++ b/doc/administration/repository_storage_paths.md
@@ -62,6 +62,8 @@ files and add the full paths of the alternative repository storage paths. In
 the example below, we add two more mountpoints that are named `nfs` and `cephfs`
 respectively.
 
+NOTE: **Note:** This example uses NFS and CephFS. We do not recommend using EFS for storage as it may impact GitLab's performance. See the [relevant documentation](./high_availability/nfs.md#avoid-using-awss-elastic-file-system-efs) for more details.
+
 **For installations from source**
 
 1. Edit `gitlab.yml` and add the storage paths:
diff --git a/doc/administration/repository_storage_types.md b/doc/administration/repository_storage_types.md
index 40f7c5566ac..4e1e363888d 100644
--- a/doc/administration/repository_storage_types.md
+++ b/doc/administration/repository_storage_types.md
@@ -86,6 +86,11 @@ by another folder with the next 2 characters. They are both stored in a special
 
 ### Hashed object pools
 
+CAUTION: **Beta:**
+Hashed objects pools are considered beta, and are not ready for production use.
+Follow [gitaly#1548](https://gitlab.com/gitlab-org/gitaly/issues/1548) for
+updates.
+
 For deduplication of public forks and their parent repository, objects are pooled
 in an object pool. These object pools are a third repository where shared objects
 are stored.
@@ -137,7 +142,7 @@ projects:
 2. Uncheck the **Use hashed storage paths for newly created and renamed projects** checkbox.
 
 To schedule a complete rollback, see the 
-[rake task documentation for storage rollback][rake/rollback-to-legacy] for instructions.
+[rake task documentation for storage rollback](raketasks/storage.md#rollback-from-hashed-storage-to-legacy-storage) for instructions.
 
 The rollback task also supports specifying a range of Project IDs. Here is an example
 of limiting the rollout to Project IDs 50 to 100, in an Omnibus Gitlab installation:
@@ -200,6 +205,5 @@ They are also S3 compatible since **10.0** (GitLab Premium), and available in Gi
 [ce-2821]: https://gitlab.com/gitlab-com/infrastructure/issues/2821
 [ce-28283]: https://gitlab.com/gitlab-org/gitlab-ce/issues/28283
 [rake/migrate-to-hashed]: raketasks/storage.md#migrate-existing-projects-to-hashed-storage
-[rake/rollback-to-legacy]: raketasks/storage.md#rollback
 [storage-paths]: repository_storage_types.md
 [gitaly]: gitaly/index.md
diff --git a/doc/administration/uploads.md b/doc/administration/uploads.md
index 8c0c7a36736..708b59a273b 100644
--- a/doc/administration/uploads.md
+++ b/doc/administration/uploads.md
@@ -18,7 +18,7 @@ below.
 >**Notes:**
 For historical reasons, uploads are stored into a base directory, which by default is `uploads/-/system`. It is strongly discouraged to change this configuration option on an existing GitLab installation.
 
-_The uploads are stored by default in `/var/opt/gitlab/gitlab-rails/uploads/-/system`._
+_The uploads are stored by default in `/var/opt/gitlab/gitlab-rails/uploads`._
 
 1. To change the storage path for example to `/mnt/storage/uploads`, edit
    `/etc/gitlab/gitlab.rb` and add the following line: