diff options
Diffstat (limited to 'doc/administration/monitoring')
| -rw-r--r-- | doc/administration/monitoring/github_imports.md | 3 | ||||
| -rw-r--r-- | doc/administration/monitoring/gitlab_self_monitoring_project/img/self_monitoring_overview_dashboard.png (renamed from doc/administration/monitoring/gitlab_self_monitoring_project/img/self_monitoring_default_dashboard.png) | bin | 51508 -> 51508 bytes | |||
| -rw-r--r-- | doc/administration/monitoring/gitlab_self_monitoring_project/index.md | 8 | ||||
| -rw-r--r-- | doc/administration/monitoring/performance/grafana_configuration.md | 4 | ||||
| -rw-r--r-- | doc/administration/monitoring/performance/performance_bar.md | 6 | ||||
| -rw-r--r-- | doc/administration/monitoring/performance/request_profiling.md | 6 | ||||
| -rw-r--r-- | doc/administration/monitoring/prometheus/gitlab_metrics.md | 7 | ||||
| -rw-r--r-- | doc/administration/monitoring/prometheus/index.md | 93 |
8 files changed, 105 insertions, 22 deletions
diff --git a/doc/administration/monitoring/github_imports.md b/doc/administration/monitoring/github_imports.md index 21cc4c708a8..0d79684c951 100644 --- a/doc/administration/monitoring/github_imports.md +++ b/doc/administration/monitoring/github_imports.md @@ -6,8 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Monitoring GitHub imports ->**Note:** -Available since [GitLab 10.2](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/14731). +> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/14731) in GitLab 10.2. The GitHub importer exposes various Prometheus metrics that you can use to monitor the health and progress of the importer. diff --git a/doc/administration/monitoring/gitlab_self_monitoring_project/img/self_monitoring_default_dashboard.png b/doc/administration/monitoring/gitlab_self_monitoring_project/img/self_monitoring_overview_dashboard.png Binary files differindex 1d61823ce41..1d61823ce41 100644 --- a/doc/administration/monitoring/gitlab_self_monitoring_project/img/self_monitoring_default_dashboard.png +++ b/doc/administration/monitoring/gitlab_self_monitoring_project/img/self_monitoring_overview_dashboard.png diff --git a/doc/administration/monitoring/gitlab_self_monitoring_project/index.md b/doc/administration/monitoring/gitlab_self_monitoring_project/index.md index 44ba26296b9..e272cccb7ce 100644 --- a/doc/administration/monitoring/gitlab_self_monitoring_project/index.md +++ b/doc/administration/monitoring/gitlab_self_monitoring_project/index.md @@ -55,10 +55,10 @@ panels, provide a regular expression in the **Instance label regex** field. The dashboard uses metrics available in [Omnibus GitLab](https://docs.gitlab.com/omnibus/) installations. - + You can also -[create your own dashboards](../../../operations/metrics/dashboards/index.md#defining-custom-dashboards-per-project). +[create your own dashboards](../../../operations/metrics/dashboards/index.md). ## Connection to Prometheus @@ -83,8 +83,8 @@ Once the webhook is setup, you can You can add custom metrics in the self monitoring project by: -1. [Duplicating](../../../operations/metrics/dashboards/index.md#duplicating-a-gitlab-defined-dashboard) the default dashboard. -1. [Editing](../../../operations/metrics/dashboards/index.md#view-and-edit-the-source-file-of-a-custom-dashboard) the newly created dashboard file and configuring it with [dashboard YAML properties](../../../operations/metrics/dashboards/yaml.md). +1. [Duplicating](../../../operations/metrics/dashboards/index.md#duplicate-a-gitlab-defined-dashboard) the overview dashboard. +1. [Editing](../../../operations/metrics/index.md) the newly created dashboard file and configuring it with [dashboard YAML properties](../../../operations/metrics/dashboards/yaml.md). ## Troubleshooting diff --git a/doc/administration/monitoring/performance/grafana_configuration.md b/doc/administration/monitoring/performance/grafana_configuration.md index 96f1377fb73..136a2749e80 100644 --- a/doc/administration/monitoring/performance/grafana_configuration.md +++ b/doc/administration/monitoring/performance/grafana_configuration.md @@ -68,7 +68,7 @@ repository. After setting up Grafana, you can enable a link to access it easily from the GitLab sidebar: -1. Navigate to the **{admin}** **Admin Area > Settings > Metrics and profiling**. +1. Navigate to the **Admin Area > Settings > Metrics and profiling**. 1. Expand **Metrics - Grafana**. 1. Check the **Enable access to Grafana** checkbox. 1. Configure the **Grafana URL**: @@ -77,7 +77,7 @@ GitLab sidebar: - *Otherwise,* enter the full URL of the Grafana instance. 1. Click **Save changes**. -GitLab displays your link in the **{admin}** **Admin Area > Monitoring > Metrics Dashboard**. +GitLab displays your link in the **Admin Area > Monitoring > Metrics Dashboard**. ## Security Update diff --git a/doc/administration/monitoring/performance/performance_bar.md b/doc/administration/monitoring/performance/performance_bar.md index 8002a9ab296..e247ec3708c 100644 --- a/doc/administration/monitoring/performance/performance_bar.md +++ b/doc/administration/monitoring/performance/performance_bar.md @@ -23,7 +23,7 @@ From left to right, it displays: details:  - **Rugged calls**: the time taken (in milliseconds) and the total number of - [Rugged](../../high_availability/nfs.md#improving-nfs-performance-with-gitlab) calls. + [Rugged](../../nfs.md#improving-nfs-performance-with-gitlab) calls. Click to display a modal window with more details:  - **Redis calls**: the time taken (in milliseconds) and the total number of @@ -72,8 +72,8 @@ Requests with warnings display `(!)` after their path in the **Request selector* The GitLab Performance Bar is disabled by default. To enable it for a given group: 1. Sign in as a user with Administrator [permissions](../../../user/permissions.md). -1. In the menu bar, click the **{admin}** **Admin Area** icon. -1. Navigate to **{settings}** **Settings > Metrics and profiling** +1. In the menu bar, click **Admin Area**. +1. Navigate to **Settings > Metrics and profiling** (`admin/application_settings/metrics_and_profiling`), and expand the section **Profiling - Performance bar**. 1. Click **Enable access to the Performance Bar**. diff --git a/doc/administration/monitoring/performance/request_profiling.md b/doc/administration/monitoring/performance/request_profiling.md index a3b29493d84..5746b95eb44 100644 --- a/doc/administration/monitoring/performance/request_profiling.md +++ b/doc/administration/monitoring/performance/request_profiling.md @@ -9,8 +9,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w To profile a request: 1. Sign in to GitLab as a user with Administrator or Maintainer [permissions](../../../user/permissions.md). -1. In the navigation bar, click **{admin}** **Admin area**. -1. Navigate to **{monitor}** **Monitoring > Requests Profiles**. +1. In the navigation bar, click **Admin area**. +1. Navigate to **Monitoring > Requests Profiles**. 1. In the **Requests Profiles** section, copy the token. 1. Pass the headers `X-Profile-Token: <token>` and `X-Profile-Mode: <mode>`(where `<mode>` can be `execution` or `memory`) to the request you want to profile. When @@ -29,7 +29,7 @@ To profile a request: Profiled requests can take longer than usual. After the request completes, you can view the profiling output from the -**{monitor}** **Monitoring > Requests Profiles** administration page: +**Monitoring > Requests Profiles** administration page:  diff --git a/doc/administration/monitoring/prometheus/gitlab_metrics.md b/doc/administration/monitoring/prometheus/gitlab_metrics.md index fa5e5da80c3..bff689c0c0c 100644 --- a/doc/administration/monitoring/prometheus/gitlab_metrics.md +++ b/doc/administration/monitoring/prometheus/gitlab_metrics.md @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w To enable the GitLab Prometheus metrics: 1. Log into GitLab as a user with [administrator permissions](../../../user/permissions.md). -1. Navigate to **{admin}** **Admin Area > Settings > Metrics and profiling**. +1. Navigate to **Admin Area > Settings > Metrics and profiling**. 1. Find the **Metrics - Prometheus** section, and click **Enable Prometheus Metrics**. 1. [Restart GitLab](../../restart_gitlab.md#omnibus-gitlab-restart) for the changes to take effect. @@ -50,7 +50,8 @@ The following metrics are available: | `gitlab_page_out_of_bounds` | Counter | 12.8 | Counter for the PageLimiter pagination limit being hit | `controller`, `action`, `bot` | | `gitlab_rails_queue_duration_seconds` | Histogram | 9.4 | Measures latency between GitLab Workhorse forwarding a request to Rails | | | `gitlab_sql_duration_seconds` | Histogram | 10.2 | SQL execution time, excluding `SCHEMA` operations and `BEGIN` / `COMMIT` | | -| `gitlab_transaction_allocated_memory_bytes` | Histogram | 10.2 | Allocated memory for all transactions (`gitlab_transaction_*` metrics) | | +| `gitlab_ruby_threads_max_expected_threads` | Gauge | 13.3 | Maximum number of threads expected to be running and performing application work | +| `gitlab_ruby_threads_running_threads` | Gauge | 13.3 | Number of running Ruby threads by name | | `gitlab_transaction_cache_<key>_count_total` | Counter | 10.2 | Counter for total Rails cache calls (per key) | | | `gitlab_transaction_cache_<key>_duration_total` | Counter | 10.2 | Counter for total time (seconds) spent in Rails cache calls (per key) | | | `gitlab_transaction_cache_count_total` | Counter | 10.2 | Counter for total Rails cache calls (aggregate) | | @@ -95,8 +96,6 @@ The following metrics are available: | `gitlab_transaction_db_count_total` | Counter | 13.1 | Counter for total number of SQL calls | `controller`, `action` | | `gitlab_transaction_db_write_count_total` | Counter | 13.1 | Counter for total number of write SQL calls | `controller`, `action` | | `gitlab_transaction_db_cached_count_total` | Counter | 13.1 | Counter for total number of cached SQL calls | `controller`, `action` | -| `http_redis_requests_duration_seconds` | Histogram | 13.1 | Redis requests duration during web transactions | `controller`, `action` | -| `http_redis_requests_total` | Counter | 13.1 | Redis requests count during web transactions | `controller`, `action` | | `http_elasticsearch_requests_duration_seconds` **(STARTER)** | Histogram | 13.1 | Elasticsearch requests duration during web transactions | `controller`, `action` | | `http_elasticsearch_requests_total` **(STARTER)** | Counter | 13.1 | Elasticsearch requests count during web transactions | `controller`, `action` | | `pipelines_created_total` | Counter | 9.4 | Counter of pipelines created | | diff --git a/doc/administration/monitoring/prometheus/index.md b/doc/administration/monitoring/prometheus/index.md index f0ad0a1a2e6..7d93e9797be 100644 --- a/doc/administration/monitoring/prometheus/index.md +++ b/doc/administration/monitoring/prometheus/index.md @@ -109,6 +109,81 @@ prometheus['scrape_configs'] = [ ] ``` +### Standalone Prometheus using Omnibus GitLab + +The Omnibus GitLab package can be used to configure a standalone Monitoring node running Prometheus and [Grafana](../performance/grafana_configuration.md). + +The steps below are the minimum necessary to configure a Monitoring node running Prometheus and Grafana with Omnibus GitLab: + +1. SSH into the Monitoring node. +1. [Install](https://about.gitlab.com/install/) the Omnibus GitLab + package you want using **steps 1 and 2** from the GitLab downloads page, but + do not follow the remaining steps. +1. Make sure to collect the IP addresses or DNS records of the Consul server nodes, for the next step. +1. Edit `/etc/gitlab/gitlab.rb` and add the contents: + + ```ruby + external_url 'http://gitlab.example.com' + + # Enable Prometheus + prometheus['enable'] = true + prometheus['listen_address'] = '0.0.0.0:9090' + prometheus['monitor_kubernetes'] = false + + # Enable Login form + grafana['disable_login_form'] = false + + # Enable Grafana + grafana['enable'] = true + grafana['admin_password'] = 'toomanysecrets' + + # Enable service discovery for Prometheus + consul['enable'] = true + consul['monitoring_service_discovery'] = true + + # The addresses can be IPs or FQDNs + consul['configuration'] = { + retry_join: %w(10.0.0.1 10.0.0.2 10.0.0.3), + } + + # Disable all other services + gitlab_rails['auto_migrate'] = false + alertmanager['enable'] = false + gitaly['enable'] = false + gitlab_exporter['enable'] = false + gitlab_workhorse['enable'] = false + nginx['enable'] = true + postgres_exporter['enable'] = false + postgresql['enable'] = false + redis['enable'] = false + redis_exporter['enable'] = false + sidekiq['enable'] = false + puma['enable'] = false + node_exporter['enable'] = false + gitlab_exporter['enable'] = false + ``` + +1. Run `sudo gitlab-ctl reconfigure` to compile the configuration. + +The next step is to tell all the other nodes where the monitoring node is: + +1. Edit `/etc/gitlab/gitlab.rb`, and add, or find and uncomment the following line: + + ```ruby + gitlab_rails['prometheus_address'] = '10.0.0.1:9090' + ``` + + Where `10.0.0.1:9090` is the IP address and port of the Prometheus node. + +1. Save the file and [reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to + take effect. + +NOTE: **Note:** +Once monitoring using Service Discovery is enabled with `consul['monitoring_service_discovery'] = true`, +ensure that `prometheus['scrape_configs']` is not set in `/etc/gitlab/gitlab.rb`. Setting both +`consul['monitoring_service_discovery'] = true` and `prometheus['scrape_configs']` in `/etc/gitlab/gitlab.rb` +will result in errors. + ### Using an external Prometheus server NOTE: **Note:** @@ -128,14 +203,24 @@ To use an external Prometheus server: 1. Set each bundled service's [exporter](#bundled-software-metrics) to listen on a network address, for example: ```ruby + node_exporter['listen_address'] = '0.0.0.0:9100' + gitlab_workhorse['prometheus_listen_addr'] = "0.0.0.0:9229" + + # Rails nodes gitlab_exporter['listen_address'] = '0.0.0.0' - sidekiq['listen_address'] = '0.0.0.0' gitlab_exporter['listen_port'] = '9168' - node_exporter['listen_address'] = '0.0.0.0:9100' + + # Sidekiq nodes + sidekiq['listen_address'] = '0.0.0.0' + + # Redis nodes redis_exporter['listen_address'] = '0.0.0.0:9121' + + # PostgreSQL nodes postgres_exporter['listen_address'] = '0.0.0.0:9187' + + # Gitaly nodes gitaly['prometheus_listen_addr'] = "0.0.0.0:9236" - gitlab_workhorse['prometheus_listen_addr'] = "0.0.0.0:9229" ``` 1. Install and set up a dedicated Prometheus instance, if necessary, using the [official installation instructions](https://prometheus.io/docs/prometheus/latest/installation/). @@ -227,7 +312,7 @@ To use an external Prometheus server: You can visit `http://localhost:9090` for the dashboard that Prometheus offers by default. ->**Note:** +NOTE: **Note:** If SSL has been enabled on your GitLab instance, you may not be able to access Prometheus on the same browser as GitLab if using the same FQDN due to [HSTS](https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security). We plan to [provide access via GitLab](https://gitlab.com/gitlab-org/multi-user-prometheus), but in the interim there are |
