diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2021-09-20 13:18:24 +0000 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2021-09-20 13:18:24 +0000 |
commit | 0653e08efd039a5905f3fa4f6e9cef9f5d2f799c (patch) | |
tree | 4dcc884cf6d81db44adae4aa99f8ec1233a41f55 /doc/development/experiment_guide | |
parent | 744144d28e3e7fddc117924fef88de5d9674fe4c (diff) | |
download | gitlab-ce-0653e08efd039a5905f3fa4f6e9cef9f5d2f799c.tar.gz |
Add latest changes from gitlab-org/gitlab@14-3-stable-eev14.3.0-rc42
Diffstat (limited to 'doc/development/experiment_guide')
-rw-r--r-- | doc/development/experiment_guide/experimentation.md | 2 | ||||
-rw-r--r-- | doc/development/experiment_guide/index.md | 30 |
2 files changed, 15 insertions, 17 deletions
diff --git a/doc/development/experiment_guide/experimentation.md b/doc/development/experiment_guide/experimentation.md index ee0f63342f1..b242646c549 100644 --- a/doc/development/experiment_guide/experimentation.md +++ b/doc/development/experiment_guide/experimentation.md @@ -106,7 +106,7 @@ class SomeWorker # Since we cannot access cookies in a worker, we need to bucket models # based on a unique, unchanging attribute instead. - # It is therefore necessery to always provide the same subject. + # It is therefore necessary to always provide the same subject. if Gitlab::Experimentation.in_experiment_group?(:experiment_key, subject: user) # execute experimental code else diff --git a/doc/development/experiment_guide/index.md b/doc/development/experiment_guide/index.md index e4a97091a81..4de272fec20 100644 --- a/doc/development/experiment_guide/index.md +++ b/doc/development/experiment_guide/index.md @@ -8,11 +8,13 @@ info: To determine the technical writer assigned to the Stage/Group associated w Experiments can be conducted by any GitLab team, most often the teams from the [Growth Sub-department](https://about.gitlab.com/handbook/engineering/development/growth/). Experiments are not tied to releases because they primarily target GitLab.com. -Experiments are run as an A/B/n test, and are behind a feature flag to turn the test on or off. Based on the data the experiment generates, the team decides if the experiment had a positive impact and should be made the new default, or rolled back. +Experiments are run as an A/B/n test, and are behind an [experiment feature flag](../feature_flags/#experiment-type) to turn the test on or off. Based on the data the experiment generates, the team decides if the experiment had a positive impact and should be made the new default, or rolled back. -## Experiment tracking issue +## Experiment rollout issue -Each experiment should have an [Experiment tracking](https://gitlab.com/groups/gitlab-org/-/issues?scope=all&state=opened&label_name[]=growth%20experiment&search=%22Experiment+tracking%22) issue to track the experiment from roll-out through to cleanup/removal. The tracking issue is similar to a feature flag rollout issue, and is also used to track the status of an experiment. Immediately after an experiment is deployed, the due date of the issue should be set (this depends on the experiment but can be up to a few weeks in the future). +Each experiment should have an [experiment rollout](https://gitlab.com/groups/gitlab-org/-/boards/1352542) issue to track the experiment from rollout through to cleanup and removal. +The rollout issue is similar to a feature flag rollout issue, and is also used to track the status of an experiment. +When an experiment is deployed, the due date of the issue should be set (this depends on the experiment but can be up to a few weeks in the future). After the deadline, the issue needs to be resolved and either: - It was successful and the experiment becomes the new default. @@ -29,6 +31,10 @@ run) shouldn't impact GitLab availability. To avoid or identify issues, experiments are initially deployed to a small number of users. Regardless, experiments still need tests. +Experiments must have corresponding [frontend or feature tests](../testing_guide/index.md) to ensure they +exist in the application. These tests should help prevent the experiment code from +being removed before the [experiment cleanup process](https://about.gitlab.com/handbook/engineering/development/growth/experimentation/#experiment-cleanup-issue) starts. + If, as a reviewer or maintainer, you find code that would usually fail review but is acceptable for now, mention your concerns with a note that there's no need to change the code. The author can then add a comment to this piece of code @@ -38,22 +44,14 @@ addressed. ## Implementing an experiment -There are currently two options when implementing an experiment. - -One is built into GitLab directly and has been around for a while (this is called -`Exerimentation Module`), and the other is provided by -[`gitlab-experiment`](https://gitlab.com/gitlab-org/gitlab-experiment) and is referred -to as `Gitlab::Experiment` -- GLEX for short. +[`GLEX`](https://gitlab.com/gitlab-org/gitlab-experiment) - or `Gitlab::Experiment`, the `gitlab-experiment` gem - is the preferred option for implementing an experiment in GitLab. -Both approaches use [experiment](../feature_flags/index.md#experiment-type) -feature flags. We recommend using GLEX rather than `Experimentation Module` for new experiments. +For more information, see [Implementing an A/B/n experiment using GLEX](gitlab_experiment.md). -- [Implementing an A/B/n experiment using GLEX](gitlab_experiment.md) -- [Implementing an A/B experiment using `Experimentation Module`](experimentation.md) +There are still some longer running experiments using the [`Exerimentation Module`](experimentation.md). -Historical Context: `Experimentation Module` was built iteratively with the needs that -appeared while implementing Growth sub-department experiments, while GLEX was built -with the findings of the team and an easier to use API. +Both approaches use [experiment](../feature_flags/index.md#experiment-type) feature flags. +`GLEX` is the preferred option for new experiments. ### Add new icons and illustrations for experiments |