summaryrefslogtreecommitdiff
path: root/.gitlab/issue_templates/Acceptance Testing.md
blob: 5289638255433b21764309ae94ae0aec41a7a474 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
## Details
- **Feature Toggle Name**: `FEATURE_NAME`
- **Required GitLab Version**: `vX.X`

--------------------------------------------------------------------------------

## 1. Preparation

- [ ] **Controllers and workers**:
  1. Please link to dashboards of the workers, and the controllers and actions that can be impacted
  2. ...
  3. ...

## 2. Development Trial

#### Check Dev Server Versions
- [ ] GitLab: https://dev.gitlab.org/help

#### Enable on `dev.gitlab.org`:
- [ ] `/chatops feature set FEATURE_NAME true --dev` in [`#dev-gitlab`](https://gitlab.slack.com/messages/C6WQ87MU3)

Then leave running while monitoring and performing some testing through web, api or SSH.

#### Monitor

- [ ] [Monitor Using Grafana](https://dashboards.gitlab.net)
- [ ] [Inspect logs in ELK](https://log.gitlab.net/app/kibana)
- [ ] [Check for errors in GitLab Dev Sentry](https://sentry.gitlab.net/gitlab/devgitlaborg/?query=is%3Aunresolved)

## 3. Staging Trial

#### Check Staging Server Versions
- [ ] GitLab: https://staging.gitlab.com/help

#### Enable on `staging.gitlab.com`
- [ ] `/chatops run feature set FEATURE_NAME true --staging` in [`#development`](https://gitlab.slack.com/messages/C02PF508L/)

Then leave running while monitoring for at least **15 minutes** while performing some testing through web, api or SSH.

#### Monitor

- [ ] [Monitor Using Grafana](https://dashboards.gitlab.net)
- [ ] [Inspect logs in ELK](https://log.gitlab.net/app/kibana)
- [ ] [Check for errors in GitLab Sentry](https://sentry.gitlab.net/gitlab/gitlabcom/?query=is%3Aunresolved)

## 4. Production Server Version Check

- [ ] GitLab: https://gitlab.com/help

## 5. Initial Impact Check

- [ ] Enable for a subset of users, when using percentage gates: 1%.

Then leave running while monitoring for at least **15 minutes** while performing some testing through web, api or SSH.

#### Monitor

- [ ] [Monitor Using Grafana](https://dashboards.gitlab.net)
- [ ] [Inspect logs in ELK](https://log.gitlab.net/app/kibana)
- [ ] [Check for errors in GitLab Sentry](https://sentry.gitlab.net/gitlab/gitlabcom/?query=is%3Aunresolved)

## 6. Low Impact Check

- [ ] Enable for a bigger subset of users, when using percentage gates: 10%.

Then leave running while monitoring for at least **30 minutes** while performing some testing through web, api or SSH.

#### Monitor

- [ ] [Monitor Using Grafana](https://dashboards.gitlab.net)
- [ ] [Inspect logs in ELK](https://log.gitlab.net/app/kibana)
- [ ] [Check for errors in GitLab Sentry](https://sentry.gitlab.net/gitlab/gitlabcom/?query=is%3Aunresolved)

## 7. Mid Impact Trial

- [ ] Enable for a big subset of users, when using percentage gates: 50%.

Then leave running while monitoring for at least **12 hours** while performing some testing through web, api or SSH.

#### Monitor

- [ ] [Monitor Using Grafana](https://dashboards.gitlab.net)
- [ ] [Inspect logs in ELK](https://log.gitlab.net/app/kibana)
- [ ] [Check for errors in GitLab Sentry](https://sentry.gitlab.net/gitlab/gitlabcom/?query=is%3Aunresolved)

## 8. Full Impact Trial

- [ ] Enable for all users: `/chatops run feature set FEATURE_NAME true

Then leave running while monitoring for at least **1 week**.

#### Monitor

- [ ] [Monitor Using Grafana](https://dashboards.gitlab.net)
- [ ] [Inspect logs in ELK](https://log.gitlab.net/app/kibana)
- [ ] [Check for errors in GitLab Dev Sentry](https://sentry.gitlab.net/gitlab/devgitlaborg/?query=is%3Aunresolved)

#### Success?

- [ ] Remove the feature gate from the code, and close this issue with that MR.