summaryrefslogtreecommitdiff
path: root/doc/operations/incident_management/incidents.md
diff options
context:
space:
mode:
authorGitLab Bot <gitlab-bot@gitlab.com>2020-10-21 07:08:36 +0000
committerGitLab Bot <gitlab-bot@gitlab.com>2020-10-21 07:08:36 +0000
commit48aff82709769b098321c738f3444b9bdaa694c6 (patch)
treee00c7c43e2d9b603a5a6af576b1685e400410dee /doc/operations/incident_management/incidents.md
parent879f5329ee916a948223f8f43d77fba4da6cd028 (diff)
downloadgitlab-ce-48aff82709769b098321c738f3444b9bdaa694c6.tar.gz
Add latest changes from gitlab-org/gitlab@13-5-stable-eev13.5.0-rc42
Diffstat (limited to 'doc/operations/incident_management/incidents.md')
-rw-r--r--doc/operations/incident_management/incidents.md240
1 files changed, 158 insertions, 82 deletions
diff --git a/doc/operations/incident_management/incidents.md b/doc/operations/incident_management/incidents.md
index 3ff02b3dc6b..3d85fa0ebd8 100644
--- a/doc/operations/incident_management/incidents.md
+++ b/doc/operations/incident_management/incidents.md
@@ -4,16 +4,88 @@ group: Health
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers
---
-# Create and manage incidents in GitLab
+# Incidents
-While no configuration is required to use the [manual features](#create-an-incident-manually)
-of incident management, some simple [configuration](#configure-incidents) is needed to automate incident creation.
+Incidents are critical entities in incident management workflows. They represent a service disruption or outage that needs to be restored urgently. GitLab provides tools for the triage, response, and remediation of incidents.
-For users with at least Developer [permissions](../../user/permissions.md), the
-Incident Management list is available at **Operations > Incidents**
+## Incident Creation
+
+You can create an incident manually or automatically.
+
+### Create incidents manually
+
+If you have at least Guest [permissions](../../user/permissions.md), to create an Incident, you have two options to do this manually.
+
+**From the Incidents List:**
+
+> [Moved](https://gitlab.com/gitlab-org/monitor/health/-/issues/24) to GitLab core in 13.3.
+
+- Navigate to **Operations > Incidents** and click **Create Incident**.
+- Create a new issue using the `incident` template available when creating it.
+- Create a new issue and assign the `incident` label to it.
+
+![Incident List Create](./img/incident_list_create_v13_3.png)
+
+**From the Issues List:**
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/230857) in GitLab 13.4.
+
+- Navigate to **Issues > List** and click **Create Issue**.
+- Create a new issue using the `type` drop-down and select `Incident`.
+- The page refreshes and the page only displays fields relevant to Incidents.
+
+![Incident List Create](./img/new_incident_create_v13_4.png)
+
+### Create incidents automatically
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/4925) in GitLab Ultimate 11.11.
+
+With Maintainer or higher [permissions](../../user/permissions.md), you can enable
+ GitLab to create incident automatically whenever an alert is triggered:
+
+1. Navigate to **Settings > Operations > Incidents** and expand
+ **Incidents**:
+
+ ![Incident Management Settings](./img/incident_management_settings_v13_3.png)
+
+1. Check the **Create an incident**
+ checkbox.
+1. To customize the incident, select an [issue templates](../../user/project/description_templates.md#creating-issue-templates).
+1. To send [an email notification](alert_notifications.md#email-notifications) to users
+ with [Developer permissions](../../user/permissions.md), select
+ **Send a separate email notification to Developers**. Email notifications will also be sent to users with **Maintainer** and **Owner** permissions.
+1. Click **Save changes**.
+
+### Create incidents via the PagerDuty webhook
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/119018) in GitLab 13.3.
+
+You can set up a webhook with PagerDuty to automatically create a GitLab incident
+for each PagerDuty incident. This configuration requires you to make changes
+in both PagerDuty and GitLab:
+
+1. Sign in as a user with Maintainer [permissions](../../user/permissions.md).
+1. Navigate to **Settings > Operations > Incidents** and expand **Incidents**.
+1. Select the **PagerDuty integration** tab:
+
+ ![PagerDuty incidents integration](./img/pagerduty_incidents_integration_v13_3.png)
+
+1. Activate the integration, and save the changes in GitLab.
+1. Copy the value of **Webhook URL** for use in a later step.
+1. Follow the steps described in the
+ [PagerDuty documentation](https://support.pagerduty.com/docs/webhooks)
+ to add the webhook URL to a PagerDuty webhook integration.
+
+To confirm the integration is successful, trigger a test incident from PagerDuty to
+confirm that a GitLab incident is created from the incident.
+
+## Incident list
+
+For users with at least Guest [permissions](../../user/permissions.md), the
+Incident list is available at **Operations > Incidents**
in your project's sidebar. The list contains the following metrics:
-![Incident List](img/incident_list_v13_4.png)
+![Incident List](img/incident_list_v13_5.png)
- **Status** - To filter incidents by their status, click **Open**, **Closed**,
or **All** above the incident list.
@@ -27,8 +99,8 @@ in your project's sidebar. The list contains the following metrics:
- **{severity-low}** **Low - S4**
- **{severity-unknown}** **Unknown**
- NOTE: **Note:**
- Editing incident severity on the incident details page was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/229402) in GitLab 13.4.
+ [Editing incident severity](#change-severity) on the incident details page was
+ [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/229402) in GitLab 13.4.
- **Incident** - The description of the incident, which attempts to capture the
most meaningful data.
@@ -44,113 +116,117 @@ The Incident list displays incidents sorted by incident created date.
To see if a column is sortable, point your mouse at the header. Sortable columns
display an arrow next to the column name.
+Incidents share the [Issues API](../../user/project/issues/index.md).
+
TIP: **Tip:**
For a live example of the incident list in action, visit this
[demo project](https://gitlab.com/gitlab-examples/ops/incident-setup/everyone/tanuki-inc/-/incidents).
-NOTE: **Note:**
-Incidents share the [Issues API](../../user/project/issues/index.md).
+## Incident details
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/230847) in GitLab 13.4.
-## Configure incidents
+Users with at least Reporter [permissions](../../user/permissions.md) can view
+the Incident Details page. Navigate to **Operations > Incidents** in your project's
+sidebar, and select an incident from the list.
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/4925) in GitLab Ultimate 11.11.
+When you take any of these actions on an incident, GitLab logs a system note and
+displays it in the Incident Details view:
-With Maintainer or higher [permissions](../../user/permissions.md), you can enable
-or disable Incident Management features in the GitLab user interface
-to create issues when alerts are triggered:
+- Updating the severity of an incident
+ ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/42358) in GitLab 13.5.)
-1. Navigate to **Settings > Operations > Incidents** and expand
- **Incidents**:
+For live examples of GitLab incidents, visit the `tanuki-inc` project's
+[incident list page](https://gitlab.com/gitlab-examples/ops/incident-setup/everyone/tanuki-inc/-/incidents).
+Click any incident in the list to display its incident details page.
- ![Incident Management Settings](./img/incident_management_settings_v13_3.png)
+### Summary
-1. For GitLab versions 11.11 and greater, you can select the **Create an issue**
- checkbox to create an issue based on your own
- [issue templates](../../user/project/description_templates.md#creating-issue-templates).
- For more information, see
- [Trigger actions from alerts](../metrics/alerts.md#trigger-actions-from-alerts) **(ULTIMATE)**.
-1. To create issues from alerts, select the template in the **Issue Template**
- select box.
-1. To send [separate email notifications](index.md#notify-developers-of-alerts) to users
- with [Developer permissions](../../user/permissions.md), select
- **Send a separate email notification to Developers**.
-1. Click **Save changes**.
+The summary section for incidents provides both critical details about and the
+contents of the issue template (if one was used). The highlighted bar at the top
+of the incident displays from left to right:
-Appropriately configured alerts include an
-[embedded chart](../metrics/embed.md#embedding-metrics-based-on-alerts-in-incident-issues)
-for the query corresponding to the alert. You can also configure GitLab to
-[close issues](../metrics/alerts.md#trigger-actions-from-alerts)
-when you receive notification that the alert is resolved.
+- The link to the original alert.
+- The alert start time.
+- The event count.
-## Create an incident manually
+Beneath the highlight bar, GitLab displays a summary that includes the following fields:
-If you have at least Developer [permissions](../../user/permissions.md), to create an Incident, you have two options.
+- Start time
+- Severity
+- `full_query`
+- Monitoring tool
-### From the Incidents List
+Comments are displayed in threads, but can be displayed chronologically
+[in a timeline view](#timeline-view).
-> [Moved](https://gitlab.com/gitlab-org/monitor/health/-/issues/24) to GitLab core in 13.3.
+### Alert details
-- Navigate to **Operations > Incidents** and click **Create Incident**.
-- Create a new issue using the `incident` template available when creating it.
-- Create a new issue and assign the `incident` label to it.
+Incidents show the details of linked alerts in a separate tab. To populate this
+tab, the incident must have been created with a linked alert. Incidents
+created automatically from alerts have this
+field populated.
-![Incident List Create](./img/incident_list_create_v13_3.png)
+![Incident alert details](./img/incident_alert_details_v13_4.png)
-### From the Issues List
+### Timeline view
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/230857) in GitLab 13.4.
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/227836) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.5.
-- Navigate to **Issues > List** and click **Create Issue**.
-- Create a new issue using the `type` drop-down and select `Incident`.
-- The page refreshes and the page only displays fields relevant to Incidents.
+To quickly see the latest updates on an incident, click
+**{comments}** **Turn timeline view on** in the comment bar to display comments
+un-threaded and ordered chronologically, newest to oldest:
-![Incident List Create](./img/new_incident_create_v13_4.png)
+![Timeline view toggle](./img/timeline_view_toggle_v13_5.png)
-## Configure PagerDuty integration
+### Service Level Agreement countdown timer
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/119018) in GitLab 13.3.
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/241663) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.5.
-You can set up a webhook with PagerDuty to automatically create a GitLab issue
-for each PagerDuty incident. This configuration requires you to make changes
-in both PagerDuty and GitLab:
+After enabling **Incident SLA** in the Incident Management configuration, newly-created
+incidents display a SLA (Service Level Agreement) timer showing the time remaining before
+the SLA period expires. If the incident is not closed before the SLA period ends, GitLab
+adds a `missed::SLA` label to the incident.
-1. Sign in as a user with Maintainer [permissions](../../user/permissions.md).
-1. Navigate to **Settings > Operations > Incidents** and expand **Incidents**.
-1. Select the **PagerDuty integration** tab:
+## Incident Actions
- ![PagerDuty incidents integration](./img/pagerduty_incidents_integration_v13_3.png)
+There are different actions avilable to help triage and respond to incidents.
-1. Activate the integration, and save the changes in GitLab.
-1. Copy the value of **Webhook URL** for use in a later step.
-1. Follow the steps described in the
- [PagerDuty documentation](https://support.pagerduty.com/docs/webhooks)
- to add the webhook URL to a PagerDuty webhook integration.
+### Assign incidents
-To confirm the integration is successful, trigger a test incident from PagerDuty to
-confirm that a GitLab issue is created from the incident.
+Assign incidents to users that are actively responding. Select **Edit** in the right-hand side bar to select or deselect assignees.
-## Incident details
+### Change severity
-> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/230847) in GitLab 13.4.
+See [Incident List](#incident-list) for a full description of the severities available. Select **Edit** in the right-hand side bar to change the severity of an incident.
-### Summary
+### Add a to do
-The summary section for incidents provides both critical details about and the
-contents of the issue template (if one was used). The highlighted bar at the top
-of the incident displays from left to right: the link to the original alert, the
-alert start time, and the event count. Beneath the highlight bar, GitLab
-displays a summary that includes the following fields:
+Add a to-do for incidents that you want to track in your to-do list. Clicke the **Add a to do** button at the top of the right-hand side bar to add a to do.
-- Start time
-- Severity
-- `full_query`
-- Monitoring tool
+### Manage incidents from Slack
-### Alert details
+Slack slash commands allow you to control GitLab and view GitLab content without leaving Slack.
-Incidents show the details of linked alerts in a separate tab. To populate this
-tab, the incident must have been created with a linked alert. Incidents
-[created automatically](#configure-incidents) from alerts will have this
-field populated.
+Learn how to [set up Slack slash commands](../../user/project/integrations/slack_slash_commands.md)
+and how to [use the available slash commands](../../integration/slash_commands.md).
-![Incident alert details](./img/incident_alert_details_v13_4.png)
+### Associate Zoom calls
+
+GitLab enables you to [associate a Zoom meeting with an issue](../../user/project/issues/associate_zoom_meeting.md)
+for synchronous communication during incident management. After starting a Zoom
+call for an incident, you can associate the conference call with an issue. Your
+team members can join the Zoom call without requesting a link.
+
+### Embed metrics in incidents
+
+You can embed metrics anywhere [GitLab Markdown](../../user/markdown.md) is
+used, such as descriptions, comments on issues, and merge requests. Embedding
+metrics helps you share them when discussing incidents or performance issues.
+You can output the dashboard directly into any issue, merge request, epic, or
+any other Markdown text field in GitLab by
+[copying and pasting the link to the metrics dashboard](../metrics/embed.md#embedding-gitlab-managed-kubernetes-metrics).
+
+You can embed both [GitLab-hosted metrics](../metrics/embed.md) and
+[Grafana metrics](../metrics/embed_grafana.md) in incidents and issue
+templates.