Add latest changes from gitlab-org/gitlab@13-3-stable-ee

8 files changed, 105 insertions, 22 deletions

diff --git a/doc/administration/monitoring/github_imports.md b/doc/administration/monitoring/github_imports.md
index 21cc4c708a8..0d79684c951 100644
--- a/doc/administration/monitoring/github_imports.md
+++ b/doc/administration/monitoring/github_imports.md
@@ -6,8 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 
 # Monitoring GitHub imports
 
->**Note:**
-Available since [GitLab 10.2](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/14731).
+> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/14731) in GitLab 10.2.
 
 The GitHub importer exposes various Prometheus metrics that you can use to
 monitor the health and progress of the importer.
diff --git a/doc/administration/monitoring/gitlab_self_monitoring_project/img/self_monitoring_default_dashboard.png b/doc/administration/monitoring/gitlab_self_monitoring_project/img/self_monitoring_overview_dashboard.png
index 1d61823ce41..1d61823ce41 100644
--- a/doc/administration/monitoring/gitlab_self_monitoring_project/img/self_monitoring_default_dashboard.png
+++ b/doc/administration/monitoring/gitlab_self_monitoring_project/img/self_monitoring_overview_dashboard.png
diff --git a/doc/administration/monitoring/gitlab_self_monitoring_project/index.md b/doc/administration/monitoring/gitlab_self_monitoring_project/index.md
index 44ba26296b9..e272cccb7ce 100644
--- a/doc/administration/monitoring/gitlab_self_monitoring_project/index.md
+++ b/doc/administration/monitoring/gitlab_self_monitoring_project/index.md
@@ -55,10 +55,10 @@ panels, provide a regular expression in the **Instance label regex** field.
 The dashboard uses metrics available in
 [Omnibus GitLab](https://docs.gitlab.com/omnibus/) installations.
 
-![GitLab self monitoring default dashboard](img/self_monitoring_default_dashboard.png)
+![GitLab self monitoring overview dashboard](img/self_monitoring_overview_dashboard.png)
 
 You can also
-[create your own dashboards](../../../operations/metrics/dashboards/index.md#defining-custom-dashboards-per-project).
+[create your own dashboards](../../../operations/metrics/dashboards/index.md).
 
 ## Connection to Prometheus
 
@@ -83,8 +83,8 @@ Once the webhook is setup, you can
 
 You can add custom metrics in the self monitoring project by:
 
-1. [Duplicating](../../../operations/metrics/dashboards/index.md#duplicating-a-gitlab-defined-dashboard) the default dashboard.
-1. [Editing](../../../operations/metrics/dashboards/index.md#view-and-edit-the-source-file-of-a-custom-dashboard) the newly created dashboard file and configuring it with [dashboard YAML properties](../../../operations/metrics/dashboards/yaml.md).
+1. [Duplicating](../../../operations/metrics/dashboards/index.md#duplicate-a-gitlab-defined-dashboard) the overview dashboard.
+1. [Editing](../../../operations/metrics/index.md) the newly created dashboard file and configuring it with [dashboard YAML properties](../../../operations/metrics/dashboards/yaml.md).
 
 ## Troubleshooting
 
diff --git a/doc/administration/monitoring/performance/grafana_configuration.md b/doc/administration/monitoring/performance/grafana_configuration.md
index 96f1377fb73..136a2749e80 100644
--- a/doc/administration/monitoring/performance/grafana_configuration.md
+++ b/doc/administration/monitoring/performance/grafana_configuration.md
@@ -68,7 +68,7 @@ repository.
 After setting up Grafana, you can enable a link to access it easily from the
 GitLab sidebar:
 
-1. Navigate to the **{admin}** **Admin Area > Settings > Metrics and profiling**.
+1. Navigate to the **Admin Area > Settings > Metrics and profiling**.
 1. Expand **Metrics - Grafana**.
 1. Check the **Enable access to Grafana** checkbox.
 1. Configure the **Grafana URL**:
@@ -77,7 +77,7 @@ GitLab sidebar:
    - *Otherwise,* enter the full URL of the Grafana instance.
 1. Click **Save changes**.
 
-GitLab displays your link in the **{admin}** **Admin Area > Monitoring > Metrics Dashboard**.
+GitLab displays your link in the **Admin Area > Monitoring > Metrics Dashboard**.
 
 ## Security Update
 
diff --git a/doc/administration/monitoring/performance/performance_bar.md b/doc/administration/monitoring/performance/performance_bar.md
index 8002a9ab296..e247ec3708c 100644
--- a/doc/administration/monitoring/performance/performance_bar.md
+++ b/doc/administration/monitoring/performance/performance_bar.md
@@ -23,7 +23,7 @@ From left to right, it displays:
   details:
   ![Gitaly profiling using the Performance Bar](img/performance_bar_gitaly_calls.png)
 - **Rugged calls**: the time taken (in milliseconds) and the total number of
-  [Rugged](../../high_availability/nfs.md#improving-nfs-performance-with-gitlab) calls.
+  [Rugged](../../nfs.md#improving-nfs-performance-with-gitlab) calls.
   Click to display a modal window with more details:
   ![Rugged profiling using the Performance Bar](img/performance_bar_rugged_calls.png)
 - **Redis calls**: the time taken (in milliseconds) and the total number of
@@ -72,8 +72,8 @@ Requests with warnings display `(!)` after their path in the **Request selector*
 The GitLab Performance Bar is disabled by default. To enable it for a given group:
 
 1. Sign in as a user with Administrator [permissions](../../../user/permissions.md).
-1. In the menu bar, click the **{admin}** **Admin Area** icon.
-1. Navigate to **{settings}** **Settings > Metrics and profiling**
+1. In the menu bar, click **Admin Area**.
+1. Navigate to **Settings > Metrics and profiling**
    (`admin/application_settings/metrics_and_profiling`), and expand the section
    **Profiling - Performance bar**.
 1. Click **Enable access to the Performance Bar**.
diff --git a/doc/administration/monitoring/performance/request_profiling.md b/doc/administration/monitoring/performance/request_profiling.md
index a3b29493d84..5746b95eb44 100644
--- a/doc/administration/monitoring/performance/request_profiling.md
+++ b/doc/administration/monitoring/performance/request_profiling.md
@@ -9,8 +9,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 To profile a request:
 
 1. Sign in to GitLab as a user with Administrator or Maintainer [permissions](../../../user/permissions.md).
-1. In the navigation bar, click **{admin}** **Admin area**.
-1. Navigate to **{monitor}** **Monitoring > Requests Profiles**.
+1. In the navigation bar, click **Admin area**.
+1. Navigate to **Monitoring > Requests Profiles**.
 1. In the **Requests Profiles** section, copy the token.
 1. Pass the headers `X-Profile-Token: <token>` and `X-Profile-Mode: <mode>`(where
    `<mode>` can be `execution` or `memory`) to the request you want to profile. When
@@ -29,7 +29,7 @@ To profile a request:
      Profiled requests can take longer than usual.
 
 After the request completes, you can view the profiling output from the
-**{monitor}** **Monitoring > Requests Profiles** administration page:
+**Monitoring > Requests Profiles** administration page:
 
 ![Profiling output](img/request_profile_result.png)
 
diff --git a/doc/administration/monitoring/prometheus/gitlab_metrics.md b/doc/administration/monitoring/prometheus/gitlab_metrics.md
index fa5e5da80c3..bff689c0c0c 100644
--- a/doc/administration/monitoring/prometheus/gitlab_metrics.md
+++ b/doc/administration/monitoring/prometheus/gitlab_metrics.md
@@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w
 To enable the GitLab Prometheus metrics:
 
 1. Log into GitLab as a user with [administrator permissions](../../../user/permissions.md).
-1. Navigate to **{admin}** **Admin Area > Settings > Metrics and profiling**.
+1. Navigate to **Admin Area > Settings > Metrics and profiling**.
 1. Find the **Metrics - Prometheus** section, and click **Enable Prometheus Metrics**.
 1. [Restart GitLab](../../restart_gitlab.md#omnibus-gitlab-restart) for the changes to take effect.
 
@@ -50,7 +50,8 @@ The following metrics are available:
 | `gitlab_page_out_of_bounds`                                    | Counter   |                   12.8 | Counter for the PageLimiter pagination limit being hit                                              | `controller`, `action`, `bot`                       |
 | `gitlab_rails_queue_duration_seconds`                          | Histogram |                    9.4 | Measures latency between GitLab Workhorse forwarding a request to Rails                             |                                                     |
 | `gitlab_sql_duration_seconds`                                  | Histogram |                   10.2 | SQL execution time, excluding `SCHEMA` operations and `BEGIN` / `COMMIT`                                  |                                                     |
-| `gitlab_transaction_allocated_memory_bytes`                    | Histogram |                   10.2 | Allocated memory for all transactions (`gitlab_transaction_*` metrics)                                |                                                     |
+| `gitlab_ruby_threads_max_expected_threads`                     | Gauge |                       13.3 | Maximum number of threads expected to be running and performing application work                    |
+| `gitlab_ruby_threads_running_threads`                          | Gauge |                       13.3 | Number of running Ruby threads by name                    |
 | `gitlab_transaction_cache_<key>_count_total`                   | Counter   |                   10.2 | Counter for total Rails cache calls (per key)                                                       |                                                     |
 | `gitlab_transaction_cache_<key>_duration_total`                | Counter   |                   10.2 | Counter for total time (seconds) spent in Rails cache calls (per key)                               |                                                     |
 | `gitlab_transaction_cache_count_total`                         | Counter   |                   10.2 | Counter for total Rails cache calls (aggregate)                                                     |                                                     |
@@ -95,8 +96,6 @@ The following metrics are available:
 | `gitlab_transaction_db_count_total`                            | Counter   |                   13.1 | Counter for total number of SQL calls                                                               | `controller`, `action`                              |
 | `gitlab_transaction_db_write_count_total`                      | Counter   |                   13.1 | Counter for total number of write SQL calls                                                         | `controller`, `action`                              |
 | `gitlab_transaction_db_cached_count_total`                     | Counter   |                   13.1 | Counter for total number of cached SQL calls                                                        | `controller`, `action`                              |
-| `http_redis_requests_duration_seconds`                         | Histogram |                   13.1 | Redis requests duration during web transactions                                                     | `controller`, `action`                              |
-| `http_redis_requests_total`                                    | Counter   |                   13.1 | Redis requests count during web transactions                                                        | `controller`, `action`                              |
 | `http_elasticsearch_requests_duration_seconds` **(STARTER)**   | Histogram |                   13.1 | Elasticsearch requests duration during web transactions                                             | `controller`, `action`                              |
 | `http_elasticsearch_requests_total` **(STARTER)**              | Counter   |                   13.1 | Elasticsearch requests count during web transactions                                                | `controller`, `action`                              |
 | `pipelines_created_total`                                      | Counter   |                    9.4 | Counter of pipelines created                                                                        |                                                     |
diff --git a/doc/administration/monitoring/prometheus/index.md b/doc/administration/monitoring/prometheus/index.md
index f0ad0a1a2e6..7d93e9797be 100644
--- a/doc/administration/monitoring/prometheus/index.md
+++ b/doc/administration/monitoring/prometheus/index.md
@@ -109,6 +109,81 @@ prometheus['scrape_configs'] = [
 ]
 ```
 
+### Standalone Prometheus using Omnibus GitLab
+
+The Omnibus GitLab package can be used to configure a standalone Monitoring node running Prometheus and [Grafana](../performance/grafana_configuration.md).
+
+The steps below are the minimum necessary to configure a Monitoring node running Prometheus and Grafana with Omnibus GitLab:
+
+1. SSH into the Monitoring node.
+1. [Install](https://about.gitlab.com/install/) the Omnibus GitLab
+   package you want using **steps 1 and 2** from the GitLab downloads page, but
+   do not follow the remaining steps.
+1. Make sure to collect the IP addresses or DNS records of the Consul server nodes, for the next step.
+1. Edit `/etc/gitlab/gitlab.rb` and add the contents:
+
+   ```ruby
+   external_url 'http://gitlab.example.com'
+
+   # Enable Prometheus
+   prometheus['enable'] = true
+   prometheus['listen_address'] = '0.0.0.0:9090'
+   prometheus['monitor_kubernetes'] = false
+
+   # Enable Login form
+   grafana['disable_login_form'] = false
+
+   # Enable Grafana
+   grafana['enable'] = true
+   grafana['admin_password'] = 'toomanysecrets'
+
+   # Enable service discovery for Prometheus
+   consul['enable'] = true
+   consul['monitoring_service_discovery'] =  true
+
+   # The addresses can be IPs or FQDNs
+   consul['configuration'] = {
+      retry_join: %w(10.0.0.1 10.0.0.2 10.0.0.3),
+   }
+
+   # Disable all other services
+   gitlab_rails['auto_migrate'] = false
+   alertmanager['enable'] = false
+   gitaly['enable'] = false
+   gitlab_exporter['enable'] = false
+   gitlab_workhorse['enable'] = false
+   nginx['enable'] = true
+   postgres_exporter['enable'] = false
+   postgresql['enable'] = false
+   redis['enable'] = false
+   redis_exporter['enable'] = false
+   sidekiq['enable'] = false
+   puma['enable'] = false
+   node_exporter['enable'] = false
+   gitlab_exporter['enable'] = false
+   ```
+
+1. Run `sudo gitlab-ctl reconfigure` to compile the configuration.
+
+The next step is to tell all the other nodes where the monitoring node is:
+
+1. Edit `/etc/gitlab/gitlab.rb`, and add, or find and uncomment the following line:
+
+   ```ruby
+   gitlab_rails['prometheus_address'] = '10.0.0.1:9090'
+   ```
+
+   Where `10.0.0.1:9090` is the IP address and port of the Prometheus node.
+
+1. Save the file and [reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to
+   take effect.
+
+NOTE: **Note:**
+Once monitoring using Service Discovery is enabled with `consul['monitoring_service_discovery'] =  true`,
+ensure that `prometheus['scrape_configs']` is not set in `/etc/gitlab/gitlab.rb`. Setting both
+`consul['monitoring_service_discovery'] = true` and `prometheus['scrape_configs']` in `/etc/gitlab/gitlab.rb`
+will result in errors.
+
 ### Using an external Prometheus server
 
 NOTE: **Note:**
@@ -128,14 +203,24 @@ To use an external Prometheus server:
 1. Set each bundled service's [exporter](#bundled-software-metrics) to listen on a network address, for example:
 
    ```ruby
+   node_exporter['listen_address'] = '0.0.0.0:9100'
+   gitlab_workhorse['prometheus_listen_addr'] = "0.0.0.0:9229"
+
+   # Rails nodes
    gitlab_exporter['listen_address'] = '0.0.0.0'
-   sidekiq['listen_address'] = '0.0.0.0'
    gitlab_exporter['listen_port'] = '9168'
-   node_exporter['listen_address'] = '0.0.0.0:9100'
+
+   # Sidekiq nodes
+   sidekiq['listen_address'] = '0.0.0.0'
+
+   # Redis nodes
    redis_exporter['listen_address'] = '0.0.0.0:9121'
+
+   # PostgreSQL nodes
    postgres_exporter['listen_address'] = '0.0.0.0:9187'
+
+   # Gitaly nodes
    gitaly['prometheus_listen_addr'] = "0.0.0.0:9236"
-   gitlab_workhorse['prometheus_listen_addr'] = "0.0.0.0:9229"
    ```
 
 1. Install and set up a dedicated Prometheus instance, if necessary, using the [official installation instructions](https://prometheus.io/docs/prometheus/latest/installation/).
@@ -227,7 +312,7 @@ To use an external Prometheus server:
 
 You can visit `http://localhost:9090` for the dashboard that Prometheus offers by default.
 
->**Note:**
+NOTE: **Note:**
 If SSL has been enabled on your GitLab instance, you may not be able to access
 Prometheus on the same browser as GitLab if using the same FQDN due to [HSTS](https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security). We plan to
 [provide access via GitLab](https://gitlab.com/gitlab-org/multi-user-prometheus), but in the interim there are