diff options
Diffstat (limited to 'doc/user/project/integrations/prometheus.md')
| -rw-r--r-- | doc/user/project/integrations/prometheus.md | 76 |
1 files changed, 70 insertions, 6 deletions
diff --git a/doc/user/project/integrations/prometheus.md b/doc/user/project/integrations/prometheus.md index aab7131e353..765aa91b00f 100644 --- a/doc/user/project/integrations/prometheus.md +++ b/doc/user/project/integrations/prometheus.md @@ -18,6 +18,7 @@ Once enabled, GitLab will automatically detect metrics from known services in th ## Enabling Prometheus Integration ### Managed Prometheus on Kubernetes + > **Note**: [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/28916) in GitLab 10.5 GitLab can seamlessly deploy and manage Prometheus on a [connected Kubernetes cluster](../clusters/index.md), making monitoring of your apps easy. @@ -39,9 +40,9 @@ Once you have a connected Kubernetes cluster with Helm installed, deploying a ma #### About managed Prometheus deployments -Prometheus is deployed into the `gitlab-managed-apps` namespace, using the [official Helm chart](https://github.com/kubernetes/charts/tree/master/stable/prometheus). Prometheus is only accessible within the cluster, with GitLab communicating through the [Kubernetes API](https://kubernetes.io/docs/concepts/overview/kubernetes-api/). +Prometheus is deployed into the `gitlab-managed-apps` namespace, using the [official Helm chart](https://github.com/helm/charts/tree/master/stable/prometheus). Prometheus is only accessible within the cluster, with GitLab communicating through the [Kubernetes API](https://kubernetes.io/docs/concepts/overview/kubernetes-api/). -The Prometheus server will [automatically detect and monitor](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#%3Ckubernetes_sd_config%3E) nodes, pods, and endpoints. To configure a resource to be monitored by Prometheus, simply set the following [Kubernetes annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/): +The Prometheus server will [automatically detect and monitor](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#kubernetes_sd_config) nodes, pods, and endpoints. To configure a resource to be monitored by Prometheus, simply set the following [Kubernetes annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/): - `prometheus.io/scrape` to `true` to enable monitoring of the resource. - `prometheus.io/port` to define the port of the metrics endpoint. @@ -66,9 +67,9 @@ Integration with Prometheus requires the following: Installing and configuring Prometheus to monitor applications is fairly straight forward. -1. [Install Prometheus](https://prometheus.io/docs/introduction/install/) +1. [Install Prometheus](https://prometheus.io/docs/prometheus/latest/installation/) 1. Set up one of the [supported monitoring targets](prometheus_library/index.md) -1. Configure the Prometheus server to [collect their metrics](https://prometheus.io/docs/operating/configuration/#scrape_config) +1. Configure the Prometheus server to [collect their metrics](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config) #### Configuration in GitLab @@ -93,7 +94,7 @@ GitLab will automatically scan the Prometheus server for metrics from known serv You can view the performance dashboard for an environment by [clicking on the monitoring button](../../../ci/environments.md#monitoring-environments). -### Adding additional metrics **[PREMIUM]** +### Adding additional metrics **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/3799) in [GitLab Premium](https://about.gitlab.com/pricing/) 10.6. @@ -120,6 +121,69 @@ GitLab supports a limited set of [CI variables](../../../ci/variables/README.htm To specify a variable in a query, enclose it in curly braces with a leading percent. For example: `%{ci_environment_slug}`. +### Defining Dashboards for Prometheus Metrics per Project + +All projects include a GitLab-defined system dashboard, which includes a few key metrics. Optionally, additional dashboards can also be defined by including configuration files in the project repository under `.gitlab/dashboards`. Configuration files nested under subdirectories will not be available in the UI. Each file should define the layout of the dashboard and the prometheus queries used to populate data. Dashboards can be selected from the dropdown in the UI. + +#### Relationship to Custom Metrics + +[Custom Metrics](#adding-additional-metrics-premium) are defined through the UI and, at this point, are unique from metrics defined in dashboard configuration files. Custom Metrics will appear on the system dashboard, as well as support alerting, whereas metrics defined in configuration files do not yet support alerts. + +#### Dashboard Configuration + +Dashboards have several components. A dashboard has many panel groups, which are comprised of panels, which support one or more metrics. The dashboard should be saved with the `.yml` extension. + +Sample YML Configuration +``` +dashboard: 'Dashboard Title' +priority: 2 +panel_groups: + - group: 'Group Title' + panels: + - type: area-chart + title: "Chart Title" + y_label: "Y-Axis" + metrics: + - id: metric_of_ages + query_range: 'http_requests_total' + label: "Metric of Ages" + unit: "count" +``` + +The above sample dashboard would display a single area chart. The following sections outline the details of expected properties. + +##### Dashboard Properties +| Property | Type | Required? | Meaning | +| ------ | ------ | ------ | ------ | +| `dashboard` | string | required | Heading for the dashboard. Only one dashboard should be defined per file. | +| `priority` | number | optional, default to definition order | Order to appear in dashboard dropdown, higher priority should be higher in the dropdown. Numbers do not need to be consecutive. | +| `panel_groups` | array | required | The panel groups which should be on the dashboard. | + +##### Panel Group Properties +| Property | Type | Required? | Meaning | +| ------ | ------ | ------ | ------ | +| `group` | string | required | Heading for the panel group. | +| `priority` | number | optional, defaults to order in file | Order to appear on the dashboard, higher priority will be higher on the page. Numbers do not need to be consecutive. | +| `panels` | array | required | The panels which should be in the panel group. | + +##### Panel Properties +| Property | Type | Required? | Meaning | +| ------ | ------ | ------ | ------- | +| `type` | enum | optional, defaults to `area-chart` | Specifies the chart type to use. Only `area-chart` is currently supported. | +| `title` | string | required | Heading for the panel. | +| `y_label` | string | optional, but highly encouraged | Y-Axis label for the panel. | +| `weight` | number | optional, defaults to order in file | Order to appear within the grouping, higher priority will be higher on the page. Numbers do not need to be consecutive. | +| `metrics` | array | required | The metrics which should be displayed in the panel. | + +##### Metric Properties +| Property | Type | Required? | Meaning | +| ------ | ------ | ------ | ------ | +| `id` | string | optional | Used for associating dashboard metrics with database records. Must be unique across dashboard configuration files. Required for [alerting](#setting-up-alerts-for-prometheus-metrics-ultimate) (support not yet enabled, see [relevant issue](https://gitlab.com/gitlab-org/gitlab-ce/issues/60319)). | +| `unit` | string | required | Defines the unit of the query's return data. | +| `label` | string | optional, but highly encouraged | Defines the legend-label for the query. Should be unique within the panel's metrics. | +| `query` | string | required unless `query_range` is defined | Defines the Prometheus query to be used to populate the chart/panel. If defined, the `query` endpoint of the [Prometheus API](https://prometheus.io/docs/prometheus/latest/querying/api/) will be utilized. | +| `query_range` | string | required unless `query` is defined | Defines the Prometheus query to be used to populate the chart/panel. If defined, the `query_range` endpoint of the [Prometheus API](https://prometheus.io/docs/prometheus/latest/querying/api/) will be utilized. | + ### Setting up alerts for Prometheus metrics **[ULTIMATE]** #### Managed Prometheus instances @@ -156,7 +220,7 @@ receivers: ... ``` -### Taking action on incidents **[ULTIMATE]** +### Taking action on incidents **(ULTIMATE)** > [Introduced](https://gitlab.com/gitlab-org/gitlab-ee/issues/4925) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 11.11. |