diff options
Diffstat (limited to 'doc/user/project/integrations')
16 files changed, 307 insertions, 31 deletions
diff --git a/doc/user/project/integrations/builds_emails.md b/doc/user/project/integrations/builds_emails.md deleted file mode 100644 index f769dece242..00000000000 --- a/doc/user/project/integrations/builds_emails.md +++ /dev/null @@ -1,15 +0,0 @@ -# Enabling build emails - -By enabling this service, you will be able to receive e-mail notifications about -the result status of your builds. - -Navigate to the [Integrations page](project_services.md#accessing-the-project-services) -and select the **Builds emails** service to configure it. - -In the _Recipients_ area, provide a list of e-mails separated by comma. - -Check the _Add pusher_ checkbox if you want the committer to also receive -e-mail notifications about each build's status. - -If you enable the _Notify only broken builds_ option, e-mail notifications will -be sent only for failed builds. diff --git a/doc/user/project/integrations/img/mattermost_configuration.png b/doc/user/project/integrations/img/mattermost_configuration.png Binary files differindex 3c5ff5ee317..f52acf4ef3b 100644 --- a/doc/user/project/integrations/img/mattermost_configuration.png +++ b/doc/user/project/integrations/img/mattermost_configuration.png diff --git a/doc/user/project/integrations/img/prometheus_environment_detail_with_metrics.png b/doc/user/project/integrations/img/prometheus_environment_detail_with_metrics.png Binary files differnew file mode 100644 index 00000000000..1f5a44f8820 --- /dev/null +++ b/doc/user/project/integrations/img/prometheus_environment_detail_with_metrics.png diff --git a/doc/user/project/integrations/img/prometheus_gcp_firewall_rule.png b/doc/user/project/integrations/img/prometheus_gcp_firewall_rule.png Binary files differnew file mode 100644 index 00000000000..e30cba211e6 --- /dev/null +++ b/doc/user/project/integrations/img/prometheus_gcp_firewall_rule.png diff --git a/doc/user/project/integrations/img/prometheus_gcp_node_name.png b/doc/user/project/integrations/img/prometheus_gcp_node_name.png Binary files differnew file mode 100644 index 00000000000..ea289431454 --- /dev/null +++ b/doc/user/project/integrations/img/prometheus_gcp_node_name.png diff --git a/doc/user/project/integrations/img/prometheus_service_configuration.png b/doc/user/project/integrations/img/prometheus_service_configuration.png Binary files differnew file mode 100644 index 00000000000..c7dfe874817 --- /dev/null +++ b/doc/user/project/integrations/img/prometheus_service_configuration.png diff --git a/doc/user/project/integrations/img/prometheus_yaml_deploy.png b/doc/user/project/integrations/img/prometheus_yaml_deploy.png Binary files differnew file mode 100644 index 00000000000..978cd7eaa50 --- /dev/null +++ b/doc/user/project/integrations/img/prometheus_yaml_deploy.png diff --git a/doc/user/project/integrations/img/slack_configuration.png b/doc/user/project/integrations/img/slack_configuration.png Binary files differindex fc8e58e686b..527824fc3eb 100644 --- a/doc/user/project/integrations/img/slack_configuration.png +++ b/doc/user/project/integrations/img/slack_configuration.png diff --git a/doc/user/project/integrations/kubernetes.md b/doc/user/project/integrations/kubernetes.md index cc67e667472..2a890acde4d 100644 --- a/doc/user/project/integrations/kubernetes.md +++ b/doc/user/project/integrations/kubernetes.md @@ -49,7 +49,8 @@ GitLab CI build environment: - `KUBE_URL` - equal to the API URL - `KUBE_TOKEN` - `KUBE_NAMESPACE` -- `KUBE_CA_PEM` - only if a custom CA bundle was specified +- `KUBE_CA_PEM_FILE` - only present if a custom CA bundle was specified. Path to a file containing PEM data. +- `KUBE_CA_PEM` (deprecated)- only if a custom CA bundle was specified. Raw PEM data. ## Web terminals diff --git a/doc/user/project/integrations/mattermost.md b/doc/user/project/integrations/mattermost.md index 09ba9994d3a..3e77823a6aa 100644 --- a/doc/user/project/integrations/mattermost.md +++ b/doc/user/project/integrations/mattermost.md @@ -24,23 +24,22 @@ There, you will see a checkbox with the following events that can be triggered: - Push - Issue +- Confidential issue - Merge request - Note - Tag push -- Build +- Pipeline - Wiki page -Bellow each of these event checkboxes, you will have an input field to insert -which Mattermost channel you want to send that event message, with `#town-square` -being the default. The hash sign is optional. +Below each of these event checkboxes, you have an input field to enter +which Mattermost channel you want to send that event message. Enter your preferred channel handle (the hash sign `#` is optional). At the end, fill in your Mattermost details: | Field | Description | | ----- | ----------- | -| **Webhook** | The incoming webhooks which you have to setup on Mattermost, it will be something like: http://mattermost.example/hooks/5xo... | +| **Webhook** | The incoming webhook URL which you have to setup on Mattermost, it will be something like: http://mattermost.example/hooks/5xo… | | **Username** | Optional username which can be on messages sent to Mattermost. Fill this in if you want to change the username of the bot. | -| **Notify only broken builds** | If you choose to enable the **Build** event and you want to be only notified about failed builds. | - +| **Notify only broken pipelines** | If you choose to enable the **Pipeline** event and you want to be only notified about failed pipelines. |  diff --git a/doc/user/project/integrations/mock_ci.md b/doc/user/project/integrations/mock_ci.md new file mode 100644 index 00000000000..6aefe5dbded --- /dev/null +++ b/doc/user/project/integrations/mock_ci.md @@ -0,0 +1,13 @@ +# Mock CI Service + +**NB: This service is only listed if you are in a development environment!** + +To setup the mock CI service server, respond to the following endpoints + +- `commit_status`: `#{project.namespace.path}/#{project.path}/status/#{sha}.json` + - Have your service return `200 { status: ['failed'|'canceled'|'running'|'pending'|'success'|'success_with_warnings'|'skipped'|'not_found'] }` + - If the service returns a 404, it is interpreted as `pending` +- `build_page`: `#{project.namespace.path}/#{project.path}/status/#{sha}` + - Just where the build is linked to, doesn't matter if implemented + +For an example of a mock CI server, see [`gitlab-org/gitlab-mock-ci-service`](https://gitlab.com/gitlab-org/gitlab-mock-ci-service) diff --git a/doc/user/project/integrations/project_services.md b/doc/user/project/integrations/project_services.md index a3a163a4c6b..25400633de5 100644 --- a/doc/user/project/integrations/project_services.md +++ b/doc/user/project/integrations/project_services.md @@ -32,7 +32,6 @@ Click on the service links to see further configuration instructions and details | Assembla | Project Management Software (Source Commits Endpoint) | | [Atlassian Bamboo CI](bamboo.md) | A continuous integration and build server | | Buildkite | Continuous integration and deployments | -| [Builds emails](builds_emails.md) | Email the builds status to a list of recipients | | [Bugzilla](bugzilla.md) | Bugzilla issue tracker | | Campfire | Simple web-based real-time group chat | | Custom Issue Tracker | Custom issue tracker | @@ -48,9 +47,11 @@ Click on the service links to see further configuration instructions and details | [Kubernetes](kubernetes.md) | A containerized deployment service | | [Mattermost slash commands](mattermost_slash_commands.md) | Mattermost chat and ChatOps slash commands | | [Mattermost Notifications](mattermost.md) | Receive event notifications in Mattermost | +| Pipelines emails | Email the pipeline status to a list of recipients | | [Slack Notifications](slack.md) | Receive event notifications in Slack | | [Slack slash commands](slack_slash_commands.md) | Slack chat and ChatOps slash commands | | PivotalTracker | Project Management Software (Source Commits Endpoint) | +| [Prometheus](prometheus.md) | Monitor the performance of your deployed apps | | Pushover | Pushover makes it easy to get real-time notifications on your Android device, iPhone, iPad, and Desktop | | [Redmine](redmine.md) | Redmine issue tracker | diff --git a/doc/user/project/integrations/prometheus.md b/doc/user/project/integrations/prometheus.md new file mode 100644 index 00000000000..676a21e85c4 --- /dev/null +++ b/doc/user/project/integrations/prometheus.md @@ -0,0 +1,196 @@ +# Prometheus integration + +> [Introduced][ce-8935] in GitLab 9.0. + +GitLab offers powerful integration with [Prometheus] for monitoring your apps. +Metrics are retrieved from the configured Prometheus server, and then displayed +within the GitLab interface. + +Each project can be configured with its own specific Prometheus server, see the +[configuration](#configuration) section for more details. If you have a single +Prometheus server which monitors all of your infrastructure, you can pre-fill +the settings page with a default template. To configure the template, see the +[Services templates](services_templates.md) document. + +## Requirements + +Integration with Prometheus requires the following: + +1. GitLab 9.0 or higher +1. Your app must be deployed on [Kubernetes][] +1. Prometheus must be configured to collect Kubernetes metrics +1. Each metric must be have a label to indicate the environment +1. GitLab must have network connectivity to the Prometheus sever + +There are a few steps necessary to set up integration between Prometheus and +GitLab. + +## Configuring Prometheus to collect Kubernetes metrics + +In order for Prometheus to collect Kubernetes metrics, you first must have a +Prometheus server up and running. You have two options here: + +- If you installed Omnibus GitLab inside of Kubernetes, you can simply use the + [bundled version of Prometheus][promgldocs]. In that case, follow the info in the + [Omnibus GitLab section](#configuring-omnibus-gitlab-prometheus-to-monitor-kubernetes) + below. +- If you are using GitLab.com or installed GitLab outside of Kubernetes, you + will likely need to run a Prometheus server within the Kubernetes cluster. + Once installed, the easiest way to monitor Kubernetes is to simply use + Prometheus' support for [Kubernetes Service Discovery][prometheus-k8s-sd]. + In that case, follow the instructions on + [configuring your own Prometheus server within Kubernetes](#configuring-your-own-prometheus-server-within-kubernetes). + +### Configuring Omnibus GitLab Prometheus to monitor Kubernetes + +With Omnibus GitLab running inside of Kubernetes, you can leverage the bundled +version of Prometheus to collect the required metrics. + +1. Read how to configure the bundled Prometheus server in the + [Administration guide][gitlab-prometheus-k8s-monitor]. +1. Now that Prometheus is configured, proceed on + [configuring the Prometheus project service in GitLab](#configuration-in-gitlab). + +### Configuring your own Prometheus server within Kubernetes + +Setting up and configuring Prometheus within Kubernetes is quick and painless. +The Prometheus project provides an [official Docker image][prometheus-docker-image] +which we can use as a starting point. + +To get started quickly, we have provided a [sample YML file][prometheus-yml] +that can be used as a template. This file will create a `prometheus` **Namespace**, +**Service**, **Deployment**, and **ConfigMap** in Kubernetes. You can upload +this file to the Kubernetes dashboard using **+ Create** at the top right. + + + +Or use `kubectl`: + +```bash +kubectl apply -f path/to/prometheus.yml +``` + +Once deployed, you should see the Prometheus service, deployment, and +pod start within the `prometheus` namespace. The server will begin to collect +metrics from each Kubernetes Node in the cluster, based on the configuration +provided in the template. + +Since GitLab is not running within Kubernetes, the template provides external +network access via a `NodePort` running on `30090`. This method allows access +to be controlled using provider firewall rules, like within Google Compute Engine. + +Since a `NodePort` does not automatically have firewall rules created for it, +one will need to be created manually to allow access. In GCP/GKE, you will want +to confirm the Node that the Prometheus pod is running on. This can be done +either by looking at the Pod in the Kubernetes dashboard, or by running: + +```bash +kubectl describe pods -n prometheus +``` + +Next on GKE, we need to get the `tag` of the Node or VM Instance, so we can +create an accurate firewall rule. The easiest way to do this is to go into the +Google Cloud Platform Compute console and select the VM instance that matches +the name of the Node gathered from the step above. In this case, the node tag +needed is `gke-prometheus-demo-5d5ada10-node`. Also make a note of the +**External IP**, which will be the IP address the Prometheus server is reachable +on. + + + +Armed with the proper Node tag, the firewall rule can now be created +specifically for this node. To create the firewall rule, open the Google Cloud +Platform Networking console, and select **Firewall Rules**. + +Create a new rule: + +- Specify the source IP range to match your desired access list, which should + include your GitLab server. A sample of GitLab.com's IP address range is + available [in this issue][gitlab.com-ip-range], but note that GitLab.com's IPs + are subject to change without prior notification. +- Allowed protocol and port should be `tcp:30090`. +- The target tags should match the Node tag identified earlier in this step. + + + +--- + +Now that Prometheus is configured, proceed to +[configure the Prometheus project service in GitLab](##configuration-in-gitlab). + +## Configuration in GitLab + +The actual configuration of Prometheus integration within GitLab is very simple. +All you will need is the DNS or IP address of the Prometheus server you'd like +to integrate with. + +1. Navigate to the [Integrations page](project_services.md#accessing-the-project-services) +1. Click the **Prometheus** service +1. Provide the base URL of the your server, for example `http://prometheus.example.com/`. + The **Test Settings** button can be used to confirm connectivity from GitLab + to the Prometheus server. + + + +## Metrics and Labels + +GitLab retrieves performance data from two metrics, `container_cpu_usage_seconds_total` +and `container_memory_usage_bytes`. These metrics are collected from the +Kubernetes pods via Prometheus, and report CPU and Memory utilization of each +container or Pod running in the cluster. + +In order to isolate and only display relevant metrics for a given environment +however, GitLab needs a method to detect which pods are associated. To do that, +GitLab will specifically request metrics that have an `environment` tag that +matches the [$CI_ENVIRONMENT_SLUG][ci-environment-slug]. + +If you are using [GitLab Auto-Deploy][autodeploy] and one of the methods of +configuring Prometheus above, the `environment` will be automatically added. + +### GitLab Prometheus queries + +The queries utilized by GitLab are shown in the following table. + +| Metric | Query | +| ------ | ----- | +| Average Memory (MB) | `(sum(container_memory_usage_bytes{container_name="app",environment="$CI_ENVIRONMENT_SLUG"}) / count(container_memory_usage_bytes{container_name="app",environment="$CI_ENVIRONMENT_SLUG"})) /1024/1024` | +| Average CPU Utilization (%) | `sum(rate(container_cpu_usage_seconds_total{container_name="app",environment="$CI_ENVIRONMENT_SLUG"}[2m])) / count(container_cpu_usage_seconds_total{container_name="app",environment="$CI_ENVIRONMENT_SLUG"}) * 100` | + +## Monitoring CI/CD Environments + +Once configured, GitLab will attempt to retrieve performance metrics for any +environment which has had a successful deployment. If monitoring data was +successfully retrieved, a metrics button will appear on the environment's +detail page. + + + +Clicking on the metrics button will display a new page, showing up to the last +8 hours of performance data. It may take a minute or two for data to appear +after initial deployment. + +## Troubleshooting + +If the metrics button is not appearing, then one of a few issues may be +occurring: + +- GitLab is not able to reach the Prometheus server. A test request can be sent + to the Prometheus server from the [Prometheus Service](#configuration-in-gitlab) + configuration screen. +- No successful deployments have occurred to this environment. +- Prometheus does not have performance data for this environment, or the metrics + are not labeled correctly. To test this, connect to the Prometheus server and + [run a query](#gitlab-prometheus-queries), replacing `$CI_ENVIRONMENT_SLUG` + with the name of your environment. + +[autodeploy]: ../../../ci/autodeploy/index.md +[kubernetes]: https://kubernetes.io +[prometheus-k8s-sd]: https://prometheus.io/docs/operating/configuration/#<kubernetes_sd_config> +[prometheus]: https://prometheus.io +[gitlab-prometheus-k8s-monitor]: ../../../administration/monitoring/prometheus/index.md#configuring-prometheus-to-monitor-kubernetes +[prometheus-docker-image]: https://hub.docker.com/r/prom/prometheus/ +[prometheus-yml]:samples/prometheus.yml +[gitlab.com-ip-range]: https://gitlab.com/gitlab-com/infrastructure/issues/434 +[ci-environment-slug]: https://docs.gitlab.com/ce/ci/variables/#predefined-variables-environment-variables +[ce-8935]: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/8935 +[promgldocs]: ../../../administration/monitoring/prometheus/index.md diff --git a/doc/user/project/integrations/samples/prometheus.yml b/doc/user/project/integrations/samples/prometheus.yml new file mode 100644 index 00000000000..01bbcaffe1e --- /dev/null +++ b/doc/user/project/integrations/samples/prometheus.yml @@ -0,0 +1,69 @@ +apiVersion: v1 +kind: Namespace +metadata: + name: prometheus +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: prometheus + namespace: prometheus +data: + prometheus.yml: |- + scrape_configs: + - job_name: 'kubernetes-nodes' + scheme: https + tls_config: + ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt + insecure_skip_verify: true + bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token + kubernetes_sd_configs: + - role: node + metric_relabel_configs: + - source_labels: [pod_name] + target_label: environment + regex: (.+)-.+-.+ + replacement: $1 +--- +apiVersion: v1 +kind: Service +metadata: + name: prometheus + namespace: prometheus +spec: + selector: + app: prometheus + ports: + - name: prometheus + protocol: TCP + port: 9090 + nodePort: 30090 + type: NodePort +--- +apiVersion: extensions/v1beta1 +kind: Deployment +metadata: + name: prometheus + namespace: prometheus +spec: + replicas: 1 + template: + metadata: + labels: + app: prometheus + spec: + containers: + - name: prometheus + image: prom/prometheus:latest + args: + - '-config.file=/prometheus-data/prometheus.yml' + ports: + - name: prometheus + containerPort: 9090 + volumeMounts: + - name: data-volume + mountPath: /prometheus-data + volumes: + - name: data-volume + configMap: + name: prometheus diff --git a/doc/user/project/integrations/slack.md b/doc/user/project/integrations/slack.md index 57a9492044b..e8b238351ca 100644 --- a/doc/user/project/integrations/slack.md +++ b/doc/user/project/integrations/slack.md @@ -21,23 +21,23 @@ There, you will see a checkbox with the following events that can be triggered: - Push - Issue +- Confidential issue - Merge request - Note - Tag push -- Build +- Pipeline - Wiki page -Bellow each of these event checkboxes, you will have an input field to insert -which Slack channel you want to send that event message, with `#general` -being the default. Enter your preferred channel **without** the hash sign (`#`). +Below each of these event checkboxes, you have an input field to enter +which Slack channel you want to send that event message. Enter your preferred channel name **without** the hash sign (`#`). At the end, fill in your Slack details: | Field | Description | | ----- | ----------- | | **Webhook** | The [incoming webhook URL][slackhook] which you have to setup on Slack. | -| **Username** | Optional username which can be on messages sent to slack. Fill this in if you want to change the username of the bot. | -| **Notify only broken builds** | If you choose to enable the **Build** event and you want to be only notified about failed builds. | +| **Username** | Optional username which can be on messages sent to Slack. Fill this in if you want to change the username of the bot. | +| **Notify only broken pipelines** | If you choose to enable the **Pipeline** event and you want to be only notified about failed pipelines. | After you are all done, click **Save changes** for the changes to take effect. diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index ed1e867f5fb..dbdc93a77a8 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -250,7 +250,19 @@ X-Gitlab-Event: Issue Hook "name": "User1", "username": "user1", "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=40\u0026d=identicon" - } + }, + "labels": [{ + "id": 206, + "title": "API", + "color": "#ffffff", + "project_id": 14, + "created_at": "2013-12-03T17:15:43Z", + "updated_at": "2013-12-03T17:15:43Z", + "template": false, + "description": "API related issues", + "type": "ProjectLabel", + "group_id": 41 + }] } ``` ### Comment events |
