1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
|
# Health Check
>**Notes:**
- Liveness and readiness probes were [introduced][ce-10416] in GitLab 9.1.
- The `health_check` endpoint was [introduced][ce-3888] in GitLab 8.8 and will
be deprecated in GitLab 9.1. Read more in the [old behavior](#old-behavior)
section.
- [Access token](#access-token) has been deprecated in GitLab 9.4
in favor of [IP Whitelist](#ip-whitelist)
GitLab provides liveness and readiness probes to indicate service health and
reachability to required services. These probes report on the status of the
database connection, Redis connection, and access to the filesystem. These
endpoints [can be provided to schedulers like Kubernetes][kubernetes] to hold
traffic until the system is ready or restart the container as needed.
## IP Whitelist
To access monitoring resources the client IP needs to be included in the whitelist.
To add or remove hosts or IP ranges from the list you can edit `gitlab.rb` or `gitlab.yml`.
Example whitelist configuration:
```yaml
monitoring:
ip_whitelist:
- 127.0.0.0/8 # by default only local IPs are allowed to access monitoring resources
```
## Access Token (Deprecated)
An access token needs to be provided while accessing the probe endpoints. The current
accepted token can be found under the **Admin area ➔ Monitoring ➔ Health check**
(`admin/health_check`) page of your GitLab instance.
The access token can be passed as a URL parameter:
```
https://gitlab.example.com/-/readiness?token=ACCESS_TOKEN
```
which will then provide a report of system health in JSON format:
```
{
"db_check": {
"status": "ok"
},
"redis_check": {
"status": "ok"
},
"fs_shards_check": {
"status": "ok",
"labels": {
"shard": "default"
}
}
}
```
## Using the Endpoint
With default whitelist settings, the probes can be accessed from localhost:
- `http://localhost/-/readiness`
- `http://localhost/-/liveness`
## Status
On failure, the endpoint will return a `500` HTTP status code. On success, the endpoint
will return a valid successful HTTP status code, and a `success` message.
## Old behavior
>**Notes:**
- Liveness and readiness probes were [introduced][ce-10416] in GitLab 9.1.
- The `health_check` endpoint was [introduced][ce-3888] in GitLab 8.8 and will
be deprecated in GitLab 9.1. Read more in the [old behavior](#old-behavior)
section.
GitLab provides a health check endpoint for uptime monitoring on the `health_check` web
endpoint. The health check reports on the overall system status based on the status of
the database connection, the state of the database migrations, and the ability to write
and access the cache. This endpoint can be provided to uptime monitoring services like
[Pingdom][pingdom], [Nagios][nagios-health], and [NewRelic][newrelic-health].
Once you have the [access token](#access-token) or your client IP is [whitelisted](#ip-whitelist),
health information can be retrieved as plain text, JSON, or XML using the `health_check` endpoint:
- `https://gitlab.example.com/health_check?token=ACCESS_TOKEN`
- `https://gitlab.example.com/health_check.json?token=ACCESS_TOKEN`
- `https://gitlab.example.com/health_check.xml?token=ACCESS_TOKEN`
You can also ask for the status of specific services:
- `https://gitlab.example.com/health_check/cache.json?token=ACCESS_TOKEN`
- `https://gitlab.example.com/health_check/database.json?token=ACCESS_TOKEN`
- `https://gitlab.example.com/health_check/migrations.json?token=ACCESS_TOKEN`
For example, the JSON output of the following health check:
```bash
curl --header "TOKEN: ACCESS_TOKEN" https://gitlab.example.com/health_check.json
```
would be like:
```
{"healthy":true,"message":"success"}
```
On failure, the endpoint will return a `500` HTTP status code. On success, the endpoint
will return a valid successful HTTP status code, and a `success` message. Ideally your
uptime monitoring should look for the success message.
[ce-10416]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10416
[ce-3888]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3888
[pingdom]: https://www.pingdom.com
[nagios-health]: https://nagios-plugins.org/doc/man/check_http.html
[newrelic-health]: https://docs.newrelic.com/docs/alerts/alert-policies/downtime-alerts/availability-monitoring
[kubernetes]: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/
|
