diff options
Diffstat (limited to 'doc/development/sidekiq_style_guide.md')
| -rw-r--r-- | doc/development/sidekiq_style_guide.md | 53 |
1 files changed, 50 insertions, 3 deletions
diff --git a/doc/development/sidekiq_style_guide.md b/doc/development/sidekiq_style_guide.md index 2793756ff64..c5dfc5731e6 100644 --- a/doc/development/sidekiq_style_guide.md +++ b/doc/development/sidekiq_style_guide.md @@ -64,6 +64,36 @@ the extra jobs will take resources away from jobs from workers that were already there, if the resources available to the Sidekiq process handling the namespace are not adjusted appropriately. +## Versioning + +Version can be specified on each Sidekiq worker class. +This is then sent along when the job is created. + +```ruby +class FooWorker + include ApplicationWorker + + version 2 + + def perform(*args) + if job_version == 2 + foo = args.first['foo'] + else + foo = args.first + end + end +end +``` + +Under this schema, any worker is expected to be able to handle any job that was +enqueued by an older version of that worker. This means that when changing the +arguments a worker takes, you must increment the `version` (or set `version 1` +if this is the first time a worker's arguments are changing), but also make sure +that the worker is still able to handle jobs that were queued with any earlier +version of the arguments. From the worker's `perform` method, you can read +`self.job_version` if you want to specifically branch on job version, or you +can read the number or type of provided arguments. + ## Idempotent Jobs It's known that a job can fail for multiple reasons. For example, network outages or bugs. @@ -145,10 +175,27 @@ authorizations for both projects. GitLab doesn't skip jobs scheduled in the future, as we assume that the state will have changed by the time the job is scheduled to -execute. +execute. If you do want to deduplicate jobs scheduled in the future +this can be specified on the worker as follows: + +```ruby +module AuthorizedProjectUpdate + class UserRefreshOverUserRangeWorker + include ApplicationWorker + + deduplicate :until_executing, including_scheduled: true + idempotent! + + # ... + end +end +``` -More [deduplication strategies have been suggested](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/195). If you are implementing a worker that -could benefit from a different strategy, please comment in the issue. +This strategy is called `until_executing`. More [deduplication +strategies have been +suggested](https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/195). If +you are implementing a worker that could benefit from a different +strategy, please comment in the issue. If the automatic deduplication were to cause issues in certain queues. This can be temporarily disabled by enabling a feature flag |