1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
|
---
stage: Growth
group: Activation
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
---
# Testing experiments
## Testing experiments with RSpec
In the course of working with experiments, you'll probably want to utilize the RSpec
tooling that's built in. This happens automatically for files in `spec/experiments`, but
for other files and specs you want to include it in, you can specify the `:experiment` type:
```ruby
it "tests experiments nicely", :experiment do
end
```
### Stub helpers
You can stub experiments using `stub_experiments`. Pass it a hash using experiment
names as the keys, and the variants you want each to resolve to, as the values:
```ruby
# Ensures the experiments named `:example` & `:example2` are both "enabled" and
# that each will resolve to the given variant (`:my_variant` and `:control`
# respectively).
stub_experiments(example: :my_variant, example2: :control)
experiment(:example) do |e|
e.enabled? # => true
e.assigned.name # => 'my_variant'
end
experiment(:example2) do |e|
e.enabled? # => true
e.assigned.name # => 'control'
end
```
### Exclusion, segmentation, and behavior matchers
You can also test things like the registered behaviors, the exclusions, and
segmentations using the matchers.
```ruby
class ExampleExperiment < ApplicationExperiment
control { }
candidate { '_candidate_' }
exclude { context.actor.first_name == 'Richard' }
segment(variant: :candidate) { context.actor.username == 'jejacks0n' }
end
excluded = double(username: 'rdiggitty', first_name: 'Richard')
segmented = double(username: 'jejacks0n', first_name: 'Jeremy')
# register_behavior matcher
expect(experiment(:example)).to register_behavior(:control)
expect(experiment(:example)).to register_behavior(:candidate).with('_candidate_')
# exclude matcher
expect(experiment(:example)).to exclude(actor: excluded)
expect(experiment(:example)).not_to exclude(actor: segmented)
# segment matcher
expect(experiment(:example)).to segment(actor: segmented).into(:candidate)
expect(experiment(:example)).not_to segment(actor: excluded)
```
### Tracking matcher
Tracking events is a major aspect of experimentation. We try
to provide a flexible way to ensure your tracking calls are covered.
You can do this on the instance level or at an "any instance" level:
```ruby
subject = experiment(:example)
expect(subject).to track(:my_event)
subject.track(:my_event)
```
You can use the `on_next_instance` chain method to specify that it will happen
on the next instance of the experiment. This helps you if you're calling
`experiment(:example).track` downstream:
```ruby
expect(experiment(:example)).to track(:my_event).on_next_instance
experiment(:example).track(:my_event)
```
A full example of the methods you can chain onto the `track` matcher:
```ruby
expect(experiment(:example)).to track(:my_event, value: 1, property: '_property_')
.on_next_instance
.with_context(foo: :bar)
.for(:variant_name)
experiment(:example, :variant_name, foo: :bar).track(:my_event, value: 1, property: '_property_')
```
## Test with Jest
### Stub Helpers
You can stub experiments using the `stubExperiments` helper defined in `spec/frontend/__helpers__/experimentation_helper.js`.
```javascript
import { stubExperiments } from 'helpers/experimentation_helper';
import { getExperimentData } from '~/experimentation/utils';
describe('when my_experiment is enabled', () => {
beforeEach(() => {
stubExperiments({ my_experiment: 'candidate' });
});
it('sets the correct data', () => {
expect(getExperimentData('my_experiment')).toEqual({ experiment: 'my_experiment', variant: 'candidate' });
});
});
```
NOTE:
This method of stubbing in Jest specs will not automatically un-stub itself at the end of the test. We merge our stubbed experiment in with all the other global data in `window.gl`. If you need to remove the stubbed experiments after your test or ensure a clean global object before your test, you'll need to manage the global object directly yourself:
```javascript
describe('tests that care about global state', () => {
const originalObjects = [];
beforeEach(() => {
// For backwards compatibility for now, we're using both window.gon & window.gl
originalObjects.push(window.gon, window.gl);
});
afterEach(() => {
[window.gon, window.gl] = originalObjects;
});
it('stubs experiment in fresh global state', () => {
stubExperiment({ my_experiment: 'candidate' });
// ...
});
})
```
|
