diff options
| author | David Wilkins <dwilkins@gitlab.com> | 2019-08-23 13:10:39 +0000 |
|---|---|---|
| committer | Achilleas Pipinellis <axil@gitlab.com> | 2019-08-23 13:10:39 +0000 |
| commit | 7b917bf9b6f107b16d45ed600df2878f79d03575 (patch) | |
| tree | 26b61ddbba1c1675791ccc6c17457d43a775c97a | |
| parent | fcfd12e0fc51987c9f8c4eb95f080970d0fd64b7 (diff) | |
| download | gitlab-ce-7b917bf9b6f107b16d45ed600df2878f79d03575.tar.gz | |
62514 - update prometheus metrics docs
- Strikeout for metrics that aren't reported anymore
- added :thumbsup: for all the metrics that appear to be still
available
- Added :new: for the new metrics.
- Determine Release when the new metrics appeared
- Add descriptions for the new metrics
- remove the adornments :new: and :thumbsup:
- removce the lines with strikeout
| -rw-r--r-- | doc/administration/monitoring/prometheus/gitlab_metrics.md | 143 |
1 files changed, 110 insertions, 33 deletions
diff --git a/doc/administration/monitoring/prometheus/gitlab_metrics.md b/doc/administration/monitoring/prometheus/gitlab_metrics.md index ec26c0b2e7e..0605fb76e2f 100644 --- a/doc/administration/monitoring/prometheus/gitlab_metrics.md +++ b/doc/administration/monitoring/prometheus/gitlab_metrics.md @@ -19,38 +19,106 @@ it, the client IP needs to be [included in a whitelist][whitelist]. For Omnibus and Chart installations, these metrics are automatically enabled and collected as of [GitLab 9.4](https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/1702). For source installations or earlier versions, these metrics will need to be enabled manually and collected by a Prometheus server. -## Unicorn Metrics available +## Metrics available The following metrics are available: -| Metric | Type | Since | Description | -|:--------------------------------- |:--------- |:----- |:----------- | -| db_ping_timeout | Gauge | 9.4 | Whether or not the last database ping timed out | -| db_ping_success | Gauge | 9.4 | Whether or not the last database ping succeeded | -| db_ping_latency_seconds | Gauge | 9.4 | Round trip time of the database ping | -| filesystem_access_latency_seconds | Gauge | 9.4 | Latency in accessing a specific filesystem | -| filesystem_accessible | Gauge | 9.4 | Whether or not a specific filesystem is accessible | -| filesystem_write_latency_seconds | Gauge | 9.4 | Write latency of a specific filesystem | -| filesystem_writable | Gauge | 9.4 | Whether or not the filesystem is writable | -| filesystem_read_latency_seconds | Gauge | 9.4 | Read latency of a specific filesystem | -| filesystem_readable | Gauge | 9.4 | Whether or not the filesystem is readable | -| gitlab_cache_misses_total | Counter | 10.2 | Cache read miss | -| gitlab_cache_operation_duration_seconds | Histogram | 10.2 | Cache access time | -| gitlab_cache_operations_total | Counter | 12.2 | Cache operations by controller/action | -| http_requests_total | Counter | 9.4 | Rack request count | -| http_request_duration_seconds | Histogram | 9.4 | HTTP response time from rack middleware | -| pipelines_created_total | Counter | 9.4 | Counter of pipelines created | -| rack_uncaught_errors_total | Counter | 9.4 | Rack connections handling uncaught errors count | -| redis_ping_timeout | Gauge | 9.4 | Whether or not the last redis ping timed out | -| redis_ping_success | Gauge | 9.4 | Whether or not the last redis ping succeeded | -| redis_ping_latency_seconds | Gauge | 9.4 | Round trip time of the redis ping | -| user_session_logins_total | Counter | 9.4 | Counter of how many users have logged in | -| upload_file_does_not_exist | Counter | 10.7 in EE, 11.5 in CE | Number of times an upload record could not find its file | -| failed_login_captcha_total | Gauge | 11.0 | Counter of failed CAPTCHA attempts during login | -| successful_login_captcha_total | Gauge | 11.0 | Counter of successful CAPTCHA attempts during login | -| unicorn_active_connections | Gauge | 11.0 | The number of active Unicorn connections (workers) | -| unicorn_queued_connections | Gauge | 11.0 | The number of queued Unicorn connections | -| unicorn_workers | Gauge | 12.0 | The number of Unicorn workers | +| Metric | Type | Since | Description | Labels | +|:-------------------------------------------------------------|:----------|-----------------------:|:----------------------------------------------------------------------------------------------------|:----------------------------------------------------| +| gitlab_banzai_cached_render_real_duration_seconds | Histogram | 9.4 | Duration of rendering markdown into HTML when cached output exists | controller, action | +| gitlab_banzai_cacheless_render_real_duration_seconds | Histogram | 9.4 | Duration of rendering markdown into HTML when cached outupt does not exist | controller, action | +| gitlab_cache_misses_total | Counter | 10.2 | Cache read miss | controller, action | +| gitlab_cache_operation_duration_seconds | Histogram | 10.2 | Cache access time | | +| gitlab_cache_operations_total | Counter | 12.2 | Cache operations by controller/action | controller, action, operation | +| gitlab_database_transaction_seconds | Histogram | 12.1 | Time spent in database transactions, in seconds | | +| gitlab_method_call_duration_seconds | Histogram | 10.2 | Method calls real duration | controller, action, module, method | +| gitlab_rails_queue_duration_seconds | Histogram | 9.4 | Measures latency between gitlab-workhorse forwarding a request to Rails | | +| gitlab_sql_duration_seconds | Histogram | 10.2 | SQL execution time, excluding SCHEMA operations and BEGIN / COMMIT | | +| gitlab_transaction_allocated_memory_bytes | Histogram | 10.2 | Allocated memory for all transactions (gitlab_transaction_* metrics) | | +| gitlab_transaction_cache_<key>_count_total | Counter | 10.2 | Counter for total Rails cache calls (per key) | | +| gitlab_transaction_cache_<key>_duration_total | Counter | 10.2 | Counter for total time (seconds) spent in Rails cache calls (per key) | | +| gitlab_transaction_cache_count_total | Counter | 10.2 | Counter for total Rails cache calls (aggregate) | | +| gitlab_transaction_cache_duration_total | Counter | 10.2 | Counter for total time (seconds) spent in Rails cache calls (aggregate) | | +| gitlab_transaction_cache_read_hit_count_total | Counter | 10.2 | Counter for cache hits for Rails cache calls | controller, action | +| gitlab_transaction_cache_read_miss_count_total | Counter | 10.2 | Counter for cache misses for Rails cache calls | controller, action | +| gitlab_transaction_duration_seconds | Histogram | 10.2 | Duration for all transactions (gitlab_transaction_* metrics) | controller, action | +| gitlab_transaction_event_build_found_total | Counter | 9.4 | Counter for build found for api /jobs/request | | +| gitlab_transaction_event_build_invalid_total | Counter | 9.4 | Counter for build invalid due to concurrency conflict for api /jobs/request | | +| gitlab_transaction_event_build_not_found_cached_total | Counter | 9.4 | Counter for cached response of build not found for api /jobs/request | | +| gitlab_transaction_event_build_not_found_total | Counter | 9.4 | Counter for build not found for api /jobs/request | | +| gitlab_transaction_event_change_default_branch_total | Counter | 9.4 | Counter when default branch is changed for any repository | | +| gitlab_transaction_event_create_repository_total | Counter | 9.4 | Counter when any repository is created | | +| gitlab_transaction_event_etag_caching_cache_hit_total | Counter | 9.4 | Counter for etag cache hit. | endpoint | +| gitlab_transaction_event_etag_caching_header_missing_total | Counter | 9.4 | Counter for etag cache miss - header missing | endpoint | +| gitlab_transaction_event_etag_caching_key_not_found_total | Counter | 9.4 | Counter for etag cache miss - key not found | endpoint | +| gitlab_transaction_event_etag_caching_middleware_used_total | Counter | 9.4 | Counter for etag middleware accessed | endpoint | +| gitlab_transaction_event_etag_caching_resource_changed_total | Counter | 9.4 | Counter for etag cache miss - resource changed | endpoint | +| gitlab_transaction_event_fork_repository_total | Counter | 9.4 | Counter for repository forks (RepositoryForkWorker). Only incremented when source repository exists | | +| gitlab_transaction_event_import_repository_total | Counter | 9.4 | Counter for repository imports (RepositoryImportWorker) | | +| gitlab_transaction_event_push_branch_total | Counter | 9.4 | Counter for all branch pushes | | +| gitlab_transaction_event_push_commit_total | Counter | 9.4 | Counter for commits | branch | +| gitlab_transaction_event_push_tag_total | Counter | 9.4 | Counter for tag pushes | | +| gitlab_transaction_event_rails_exception_total | Counter | 9.4 | Counter for number of rails exceptions | | +| gitlab_transaction_event_receive_email_total | Counter | 9.4 | Counter for recieved emails | handler | +| gitlab_transaction_event_remote_mirrors_failed_total | Counter | 10.8 | Counter for failed remote mirrors | | +| gitlab_transaction_event_remote_mirrors_finished_total | Counter | 10.8 | Counter for finished remote mirrors | | +| gitlab_transaction_event_remote_mirrors_running_total | Counter | 10.8 | Counter for running remote mirrors | | +| gitlab_transaction_event_remove_branch_total | Counter | 9.4 | Counter when a branch is removed for any repository | | +| gitlab_transaction_event_remove_repository_total | Counter | 9.4 | Counter when a repository is removed | | +| gitlab_transaction_event_remove_tag_total | Counter | 9.4 | Counter when a tag is remove for any repository | | +| gitlab_transaction_event_sidekiq_exception_total | Counter | 9.4 | Counter of sidekiq exceptions | | +| gitlab_transaction_event_stuck_import_jobs_total | Counter | 9.4 | Count of stuck import jobs | projects_without_jid_count, projects_with_jid_count | +| gitlab_transaction_event_update_build_total | Counter | 9.4 | Counter for update build for api /jobs/request/:id | | +| gitlab_transaction_new_redis_connections_total | Counter | 9.4 | Counter for new redis connections | | +| gitlab_transaction_queue_duration_total | Counter | 9.4 | Duration jobs were enqueued before processing | | +| gitlab_transaction_rails_queue_duration_total | Counter | 9.4 | Measures latency between gitlab-workhorse forwarding a request to Rails | controller, action | +| gitlab_transaction_view_duration_total | Counter | 9.4 | Duration for views | controller, action, view | +| gitlab_view_rendering_duration_seconds | Histogram | 10.2 | Duration for views (histogram) | controller, action, view | +| http_requests_total | Counter | 9.4 | Rack request count | method | +| http_request_duration_seconds | Histogram | 9.4 | HTTP response time from rack middleware | method, status | +| pipelines_created_total | Counter | 9.4 | Counter of pipelines created | | +| rack_uncaught_errors_total | Counter | 9.4 | Rack connections handling uncaught errors count | | +| user_session_logins_total | Counter | 9.4 | Counter of how many users have logged in | | +| upload_file_does_not_exist | Counter | 10.7 in EE, 11.5 in CE | Number of times an upload record could not find its file | | +| failed_login_captcha_total | Gauge | 11.0 | Counter of failed CAPTCHA attempts during login | | +| successful_login_captcha_total | Gauge | 11.0 | Counter of successful CAPTCHA attempts during login | | + +## Metrics controlled by a feature flag + +The following metrics can be controlled by feature flags: + +| Metric | Feature Flag | +|:-------------------------------------------------------------|:-----------------------------------------------------------------| +| gitlab_method_call_duration_seconds | prometheus_metrics_method_instrumentation | +| gitlab_transaction_allocated_memory_bytes | prometheus_metrics_transaction_allocated_memory | +| gitlab_transaction_event_build_found_total | prometheus_transaction_event_build_found_total | +| gitlab_transaction_event_build_invalid_total | prometheus_transaction_event_build_invalid_total | +| gitlab_transaction_event_build_not_found_cached_total | prometheus_transaction_event_build_not_found_cached_total | +| gitlab_transaction_event_build_not_found_total | prometheus_transaction_event_build_not_found_total | +| gitlab_transaction_event_change_default_branch_total | prometheus_transaction_event_change_default_branch_total | +| gitlab_transaction_event_create_repository_total | prometheus_transaction_event_create_repository_total | +| gitlab_transaction_event_etag_caching_cache_hit_total | prometheus_transaction_event_etag_caching_cache_hit_total | +| gitlab_transaction_event_etag_caching_header_missing_total | prometheus_transaction_event_etag_caching_header_missing_total | +| gitlab_transaction_event_etag_caching_key_not_found_total | prometheus_transaction_event_etag_caching_key_not_found_total | +| gitlab_transaction_event_etag_caching_middleware_used_total | prometheus_transaction_event_etag_caching_middleware_used_total | +| gitlab_transaction_event_etag_caching_resource_changed_total | prometheus_transaction_event_etag_caching_resource_changed_total | +| gitlab_transaction_event_fork_repository_total | prometheus_transaction_event_fork_repository_total | +| gitlab_transaction_event_import_repository_total | prometheus_transaction_event_import_repository_total | +| gitlab_transaction_event_push_branch_total | prometheus_transaction_event_push_branch_total | +| gitlab_transaction_event_push_commit_total | prometheus_transaction_event_push_commit_total | +| gitlab_transaction_event_push_tag_total | prometheus_transaction_event_push_tag_total | +| gitlab_transaction_event_rails_exception_total | prometheus_transaction_event_rails_exception_total | +| gitlab_transaction_event_receive_email_total | prometheus_transaction_event_receive_email_total | +| gitlab_transaction_event_remote_mirrors_failed_total | prometheus_transaction_event_remote_mirrors_failed_total | +| gitlab_transaction_event_remote_mirrors_finished_total | prometheus_transaction_event_remote_mirrors_finished_total | +| gitlab_transaction_event_remote_mirrors_running_total | prometheus_transaction_event_remote_mirrors_running_total | +| gitlab_transaction_event_remove_branch_total | prometheus_transaction_event_remove_branch_total | +| gitlab_transaction_event_remove_repository_total | prometheus_transaction_event_remove_repository_total | +| gitlab_transaction_event_remove_tag_total | prometheus_transaction_event_remove_tag_total | +| gitlab_transaction_event_sidekiq_exception_total | prometheus_transaction_event_sidekiq_exception_total | +| gitlab_transaction_event_stuck_import_jobs_total | prometheus_transaction_event_stuck_import_jobs_total | +| gitlab_transaction_event_update_build_total | prometheus_transaction_event_update_build_total | +| gitlab_view_rendering_duration_seconds | prometheus_metrics_view_instrumentation | ## Sidekiq Metrics available for Geo **(PREMIUM)** @@ -99,17 +167,27 @@ Some basic Ruby runtime metrics are available: | Metric | Type | Since | Description | |:-------------------------------------- |:--------- |:----- |:----------- | -| ruby_gc_duration_seconds_total | Counter | 11.1 | Time spent by Ruby in GC | +| ruby_gc_duration_seconds | Counter | 11.1 | Time spent by Ruby in GC | | ruby_gc_stat_... | Gauge | 11.1 | Various metrics from [GC.stat] | | ruby_file_descriptors | Gauge | 11.1 | File descriptors per process | | ruby_memory_bytes | Gauge | 11.1 | Memory usage by process | -| ruby_sampler_duration_seconds_total | Counter | 11.1 | Time spent collecting stats | +| ruby_sampler_duration_seconds | Counter | 11.1 | Time spent collecting stats | | ruby_process_cpu_seconds_total | Gauge | 12.0 | Total amount of CPU time per process | | ruby_process_max_fds | Gauge | 12.0 | Maximum number of open file descriptors per process | | ruby_process_resident_memory_bytes | Gauge | 12.0 | Memory usage by process, measured in bytes | | ruby_process_start_time_seconds | Gauge | 12.0 | UNIX timestamp of process start time | -[GC.stat]: https://ruby-doc.org/core-2.3.0/GC.html#method-c-stat +[GC.stat]: https://ruby-doc.org/core-2.6.3/GC.html#method-c-stat + +## Unicorn Metrics + +Unicorn specific metrics, when Unicorn is used. + +| Metric | Type | Since | Description | +|:---------------------------|:------|:------|:---------------------------------------------------| +| unicorn_active_connections | Gauge | 11.0 | The number of active Unicorn connections (workers) | +| unicorn_queued_connections | Gauge | 11.0 | The number of queued Unicorn connections | +| unicorn_workers | Gauge | 12.0 | The number of Unicorn workers | ## Puma Metrics **(EXPERIMENTAL)** @@ -126,7 +204,6 @@ When Puma is used instead of Unicorn, following metrics are available: | puma_pool_capacity | Gauge | 12.0 | Number of requests the worker is capable of taking right now | | puma_max_threads | Gauge | 12.0 | Maximum number of worker threads | | puma_idle_threads | Gauge | 12.0 | Number of spawned threads which are not processing a request | -| rack_state_total | Gauge | 12.0 | Number of requests in a given rack state | | puma_killer_terminations_total | Gauge | 12.0 | Number of workers terminated by PumaWorkerKiller | ## Metrics shared directory |