summaryrefslogtreecommitdiff
path: root/doc/administration/high_availability/monitoring_node.md
blob: 385e7441ac959d7abdc1c220191b1af268f7c776 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
# Configuring a Monitoring node for Scaling and High Availability

> [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/issues/3786) in GitLab 12.0.

## Standalone Monitoring node using GitLab Omnibus

The GitLab Omnibus package can be used to configure a standalone Monitoring node running Prometheus and Grafana.
The monitoring node is not highly available. See [Scaling and High Availability](README.md)
for an overview of GitLab scaling and high availability options.

The steps below are the minimum necessary to configure a Monitoring node running Prometheus and Grafana with
Omnibus:

1. SSH into the Monitoring node.
1. [Download/install](https://about.gitlab.com/install) the Omnibus GitLab
   package you want using **steps 1 and 2** from the GitLab downloads page.
   - Do not complete any other steps on the download page.

1. Make sure to collect [`CONSUL_SERVER_NODES`](database.md#consul-information), which are the IP addresses or DNS records of the Consul server nodes, for the next step. Note they are presented as `Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z`

1. Edit `/etc/gitlab/gitlab.rb` and add the contents:

    ```ruby
    external_url 'http://gitlab.example.com'

    # Enable Prometheus
    prometheus['enable'] = true
    prometheus['listen_address'] = '0.0.0.0:9090'
    prometheus['monitor_kubernetes'] = false

    # Enable Grafana
    grafana['enable'] = true
    grafana['admin_password'] = 'toomanysecrets'

    # Enable service discovery for Prometheus
    consul['enable'] = true
    consul['monitoring_service_discovery'] =  true

    # Replace placeholders
    # Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z
    # with the addresses of the Consul server nodes
    consul['configuration'] = {
       retry_join: %w(Y.Y.Y.Y consul1.gitlab.example.com Z.Z.Z.Z),
    }

    # Disable all other services
    gitlab_rails['auto_migrate'] = false
    alertmanager['enable'] = false
    gitaly['enable'] = false
    gitlab_monitor['enable'] = false
    gitlab_workhorse['enable'] = false
    nginx['enable'] = true
    postgres_exporter['enable'] = false
    postgresql['enable'] = false
    redis['enable'] = false
    redis_exporter['enable'] = false
    sidekiq['enable'] = false
    unicorn['enable'] = false
    node_exporter['enable'] = false
    ```

1. Run `sudo gitlab-ctl reconfigure` to compile the configuration.

## Migrating to Service Discovery

Once monitoring using Service Discovery is enabled with `consul['monitoring_service_discovery'] =  true`,
ensure that `prometheus['scrape_configs']` is not set  in `/etc/gitlab/gitlab.rb`. Setting both
`consul['monitoring_service_discovery'] =  true` and `prometheus['scrape_configs']` in `/etc/gitlab/gitlab.rb`
will result in errors.