diff options
| author | GitLab Bot <gitlab-bot@gitlab.com> | 2020-03-01 09:07:42 +0000 |
|---|---|---|
| committer | GitLab Bot <gitlab-bot@gitlab.com> | 2020-03-01 09:07:42 +0000 |
| commit | 9c1fa86a6aa0268eae2fafc7d90908502942b810 (patch) | |
| tree | 511e517bba91df4c60313960ec149835efb78779 /doc/development/elasticsearch.md | |
| parent | b2152c3f869fda3e77874e3ce1c8e1bd489cc9ec (diff) | |
| download | gitlab-ce-9c1fa86a6aa0268eae2fafc7d90908502942b810.tar.gz | |
Add latest changes from gitlab-org/gitlab@master
Diffstat (limited to 'doc/development/elasticsearch.md')
| -rw-r--r-- | doc/development/elasticsearch.md | 6 |
1 files changed, 5 insertions, 1 deletions
diff --git a/doc/development/elasticsearch.md b/doc/development/elasticsearch.md index b8d2a873d8b..69113fe8030 100644 --- a/doc/development/elasticsearch.md +++ b/doc/development/elasticsearch.md @@ -36,7 +36,11 @@ Additionally, if you need large repos or multiple forks for testing, please cons The Elasticsearch integration depends on an external indexer. We ship an [indexer written in Go](https://gitlab.com/gitlab-org/gitlab-elasticsearch-indexer). The user must trigger the initial indexing via a rake task but, after this is done, GitLab itself will trigger reindexing when required via `after_` callbacks on create, update, and destroy that are inherited from [/ee/app/models/concerns/elastic/application_versioned_search.rb](https://gitlab.com/gitlab-org/gitlab/blob/master/ee/app/models/concerns/elastic/application_versioned_search.rb). -All indexing after the initial one is done via `ElasticIndexerWorker` (Sidekiq jobs). +After initial indexing is complete, updates proceed in one of two ways, depending on the `:elastic_bulk_incremental_updates` feature flag. + +If disabled, every create, update, or delete operation on an Elasticsearch-tracked model enqueues a new `ElasticIndexerWorker` Sidekiq job which takes care of updating just that document. This is quite inefficient. + +If the feature flag is enabled, create, update, and delete operations for all models except projects (see [#207494](https://gitlab.com/gitlab-org/gitlab/issues/207494)) are tracked in a Redis [`ZSET`](https://redis.io/topics/data-types#sorted-sets) instead. A regular `sidekiq-cron` `ElasticIndexBulkCronWorker` processes this queue, updating many Elasticsearch documents at a time with the [Bulk Request API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html). Search queries are generated by the concerns found in [ee/app/models/concerns/elastic](https://gitlab.com/gitlab-org/gitlab/tree/master/ee/app/models/concerns/elastic). These concerns are also in charge of access control, and have been a historic source of security bugs so please pay close attention to them! |