summaryrefslogtreecommitdiff
path: root/app/workers
Commit message (Collapse)AuthorAgeFilesLines
* Remove auto ssl feature flagsVladimir Shushlin2019-07-122-3/+3
| | | | | * remove feature flag for admin settings * remove feature flag for domain settings
* Add a rubocop for Rails.loggerMayra Cabrera2019-07-1018-18/+22
| | | | | | Suggests to use a JSON structured log instead Related to https://gitlab.com/gitlab-org/gitlab-ce/issues/54102
* Add queue_size method to ApplicationWorkerdm-queue-mirror-jobs-in-batches-drain-ceDouwe Maan2019-07-051-0/+6
|
* Allow asynchronous rebase operations to be monitoredNick Thomas2019-07-041-0/+2
| | | | | | | | | This MR introduces tracking of the `rebase_jid` for merge requests. As with `merge_ongoing?`, `rebase_in_progress?` will now return true if a rebase is proceeding in sidekiq. After one release, we should remove the Gitaly-based lookup of rebases. It is much better to track this kind of thing via the database.
* Includes logic to persist namespace statisticsMayra Cabrera2019-07-024-0/+102
| | | | | | | | | | | | | | | | | | | | | - Add two new ActiveRecord models: - RootNamespaceStoragestatistics will persist root namespace statistics - NamespaceAggregationSchedule will save information when a new update to the namespace statistics needs to be scheduled - Inject into UpdateProjectStatistics concern a new callback that will call an async job to insert a new row onto NamespaceAggregationSchedule table - When a new row is inserted a new job is scheduled. This job will update call an specific service to update the statistics and after that it will delete thee aggregated scheduled row - The RefresherServices makes heavy use of arel to build composable queries to update Namespace::RootStorageStatistics attributes. - Add an extra worker to traverse pending rows on NAmespace::AggregationSchedule table and schedule a worker for each one of this rows. - Add an extra worker to traverse pending rows on NAmespace::AggregationSchedule table and schedule a worker for each one of this rows
* Use project depended feature flag for pages sslpages-ssl-project-aware-feature-flagVladimir Shushlin2019-06-252-6/+4
| | | | | Also add ::Gitlab::LetsEncrypt.enabled? shortcut and simplify it a lot
* Renew Let's Encrypt certificatesVladimir Shushlin2019-06-243-0/+31
| | | | | | | | Add index for pages domain ssl auto renewal Add PagesDomain.needs_ssl_renewal scope Add cron worker for ssl renewal Add worker for ssl renewal Add pages ssl renewal worker queues settings
* Merge branch 'remove_create_deployment_from_build_success_worker' into 'master'Douwe Maan2019-06-241-12/+0
|\ | | | | | | | | | | | | Remove deprecated call in BuildSuccessWorker Closes #63622 See merge request gitlab-org/gitlab-ce!29958
| * Remove deprecated call in BuildSuccessWorkerremove_create_deployment_from_build_success_workerThong Kuah2019-06-241-12/+0
| | | | | | | | | | We now create_deployment on Ci::Build create. No need to call create_deployment again
* | Merge branch '60617-enable-project-cluster-jit' into 'master'Thong Kuah2019-06-241-2/+0
|\ \ | |/ |/| | | | | Enable JIT Kubernetes resource creation for project level clusters See merge request gitlab-org/gitlab-ce!29515
| * Enable project-level JIT resource creation60617-enable-project-cluster-jitTiger2019-06-181-2/+0
| | | | | | | | | | | | | | | | | | | | Previously this behaviour was only available to group and instance-level clusters, as some project clusters relied on Kubernetes credentials being passed through to the runner instead of having their resources managed by GitLab (which is not available when using JIT). These clusters have been migrated to unmanaged, so resources can be created on demand for the remaining managed clusters.
* | Merge branch 'ce-12014-incremental-es-wiki-updates' into 'master'Nick Thomas2019-06-191-8/+15
|\ \ | | | | | | | | | | | | Backport of 12014-incremental-es-wiki-updates See merge request gitlab-org/gitlab-ce!29531
| * | Backport of 12014-incremental-es-wiki-updatesce-12014-incremental-es-wiki-updatesMario de la Ossa2019-06-181-8/+15
| |/ | | | | | | Bringing in the DB migration and some light changes for CE classes
* | Add 2nd response for container api bulk deleteSteve Abrams2019-06-181-26/+3
|/ | | | | | | The bulk delete api endpoint for container registries can only be called once per hour. If a user calls the endpoint more than once per hour, they will now receive a 400 error with a descriptive message.
* Delete unauthorized Todos when project is privateissue_49897Felipe Artur2019-06-051-2/+2
| | | | | Delete Todos for guest users when project visibility level is updated to private.
* Ensure DB is writable before continuing jobs11925-geo-sidekiq-nodes-try-to-run-jobs-even-thought-db-is-readonlyAsh McKenzie2019-06-052-0/+4
| | | | | | | In the context of a Geo setup, some jobs can be running on a Geo secondary where the database is read-only and therefore we should guard against various jobs attempting to write.
* Merge branch 'introduce-auto-merge-process-worker' into 'master'Thong Kuah2019-06-043-8/+17
|\ | | | | | | | | Introduce sidekiq worker for auto merge process See merge request gitlab-org/gitlab-ce!28780
| * Introduce sidekiq worker for auto merge processintroduce-auto-merge-process-workerShinya Maeda2019-06-043-8/+17
| | | | | | | | | | As we have a central domain for auto merge process today, we should use a single worker for any auto merge process.
* | Merge branch ↵Kamil Trzciński2019-06-042-39/+28
|\ \ | |/ |/| | | | | | | | | | | | | 'set-real-next-run-at-for-preventing-duplciate-pipeline-creations' into 'master' Make pipeline schedule worker resilient Closes gitlab-com/gl-infra/production#805 and #61955 See merge request gitlab-org/gitlab-ce!28407
| * Make pipeline schedule worker resilientset-real-next-run-at-for-preventing-duplciate-pipeline-creationsShinya Maeda2019-06-032-39/+28
| | | | | | | | | | | | | | | | | | | | | | | | | | Currently, pipeline schedule worker is unstable because it's sometimes killed by excessive memory consumption. In order to improve the performance, we add the following fixes: 1. next_run_at is always real_next_run, which means the value always takes into account of worker's cron schedule 1. Remove exlusive lock. This is already covered by real_next_run change. 1. Use RunPipelineScheduleWorker for avoiding memory killer. Memory consumption is spread to the multiple sidekiq worker.
* | Abstract auto merge processesShinya Maeda2019-06-031-3/+4
|/ | | | | | | | | | | | We have one auto merge strategy today - Merge When Pipeline Succeeds. In order to add more strategies for Merge Train feature, we abstract the architecture to be more extensible. Removed arguments Fix spec
* Add wiki size to project statisticsPeter Marko2019-05-292-1/+5
|
* Add GitDeduplicationService for deduplication housekeepingjc-git-deduplication-serviceJohn Cai2019-05-211-1/+3
| | | | | GitDeduplicationService performs idempotent operations on deduplicated projects.
* Expire pipeline cache on finishmc/bug/pipeline-cache-invalidMatija Čupić2019-05-211-0/+1
|
* Remove remove_disabled_domains feature flagVladimir Shushlin2019-05-201-2/+0
|
* Stop configuring group clusters on creation60379-remove-ci-preparing-state-feature-flagTiger2019-05-161-1/+1
| | | | | | | | | | | Immediate configuration is not ideal for group and instance level clusters as projects that may never be deployed would still have Kubernetes namespaces and service accounts created for them. As of https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/25586 we now create only the resources that are required for the project being deployed, at the time of deployment.
* Include MR information if possible when emailing notification of closing an ↵Michał Zając2019-05-161-1/+1
| | | | issue
* Change DetectRepositoryLanguagesWorker to not receive userDiego Silva2019-05-091-4/+3
| | | | Fixes #60425
* Add gitlab-managed option to clusters formMayra Cabrera2019-05-031-1/+1
| | | | | | | | When this option is enabled, GitLab will create namespaces and service accounts as usual. When disabled, GitLab wont create any project specific kubernetes resources Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/56557
* Use git_garbage_collect_worker to run pack_refsJan Provaznik2019-05-021-2/+9
| | | | | | | | PackRefs is not an expensive gitaly call - we want to call it more often (than as part of full `gc`) because it helps to keep number of refs files small - too many refs file may be a problem for deployments with slow storage.
* Remove disabled pages domainsVladimir Shushlin2019-04-302-0/+17
| | | | | Domain will be removed by verification worker after 1 week of being disabled
* DELETE clusters/:id/:application endpointThong Kuah2019-04-292-0/+18
| | | | Add endpoint to delete/uninstall a cluster application
* Services to uninstall cluster applicationThong Kuah2019-04-292-0/+21
| | | | + to monitor progress of uninstallation pod
* Merge branch 'lock-pipeline-schedule-worker' into 'master'Stan Hu2019-04-291-11/+17
|\ | | | | | | | | | | | | Prevent concurrent execution of PipelineScheduleWorker Closes gitlab-com/gl-infra/production#805 See merge request gitlab-org/gitlab-ce!27781
| * Prevent concurrent execution of PipelineScheduleWorkerShinya Maeda2019-04-281-11/+17
| | | | | | | | | | | | Currently, PipelineScheduleWorker is fired in a short period on our production server. We can stop this behavior by locking the execution thread with in_lock method.
* | Port changes for design management to CEBob Van Landuyt2019-04-291-1/+3
| | | | | | | | | | | | This ports the changes from https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/10462/ to CE
* | Add deployment events to chat notification servicesJason Goodman2019-04-262-0/+14
|/ | | | | This enables sending a chat message to Slack or Mattermost upon a successful, failed, or canceled deployment
* Merge branch ↵Kamil Trzciński2019-04-172-5/+3
|\ | | | | | | | | | | | | | | | | '60500-disable-jit-kubernetes-resource-creation-for-project-level-clusters' into 'master' Disable JIT Kubernetes resource creation for project level clusters Closes #60500 See merge request gitlab-org/gitlab-ce!27352
| * Disable JIT resource creation for project clusters60500-disable-jit-kubernetes-resource-creation-for-project-level-clustersTiger2019-04-162-5/+3
| | | | | | | | | | | | | | | | | | | | | | | | JIT resource creation blocks deployments if a user is self-managing their cluster, as it will fail the build if unable to create a namespace and service account. Using a custom namespace and service account was previously supported for project level clusters, so we should preserve this functionality. https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/27352
* | Properly expire all pipeline caches when pipeline is deletedsh-fix-pipeline-delete-cachingStan Hu2019-04-151-50/+1
|/ | | | | | | | | | | When deleting a pipeline, only some of the cache structures were being expired, but not the full pipeline list. We have to synchronously schedule a pipeline cache expiration because the pipeline will be deleted if the Sidekiq expiration job picks it up. To do this, properly extract all the logic buried in the Sidekiq worker into a service, and then call the service. Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/60469
* Merge branch 'limit-amount-of-created-pipelines' into 'master'Grzegorz Bizon2019-04-111-12/+15
|\ | | | | | | | | Process at most 4 pipelines during push See merge request gitlab-org/gitlab-ce!27205
| * Process at most 4 pipelines during pushlimit-amount-of-created-pipelinesKamil Trzciński2019-04-101-12/+15
| | | | | | | | | | | | | | | | This adds a limitation that we will try to create pipeline for at most 4 first changes (branches and tags). This does not affect processing of Pipelines for Merge Requests, as each updated MR will have associated pipeline created.
* | Externalize strings in app/workersMartin Wortschack2019-04-103-16/+16
|/ | | | | | - email_receiver_worker - object_storage/migrate_uploads_worker - Update PO file
* Merge branch '43263-git-push-option-to-create-mr' into 'master'Nick Thomas2019-04-091-1/+1
|\ | | | | | | | | | | | | Git push options to create a merge request, set target_branch and set merge when pipeline succeeds Closes #53198 and #43263 See merge request gitlab-org/gitlab-ce!26752
| * Use Gitlab::PushOptions for `ci.skip` push optionLuke Duncalfe2019-04-091-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously the raw push option Array was sent to Pipeline::Chain::Skip. This commit updates this class (and the chain of classes that pass the push option parameters from the API internal `post_receive` endpoint to that class) to treat push options as a Hash of options parsed by GitLab::PushOptions. The GitLab::PushOptions class takes options like this: -o ci.skip -o merge_request.create -o merge_request.target=branch and turns them into a Hash like this: { ci: { skip: true }, merge_request: { create: true, target: 'branch' } } This now how Pipeline::Chain::Skip is determining if the `ci.skip` push option was used.
* | Merge branch 'delay-update-statictics' into 'master'Douwe Maan2019-04-093-6/+33
|\ \ | |/ |/| | | | | Fix the bug that the project statistics is not updated See merge request gitlab-org/gitlab-ce!26854
| * Refactor: extract duplicate steps to a service classHiroyuki Sato2019-04-052-8/+3
| |
| * Update the project statistics immediatellyHiroyuki Sato2019-04-051-0/+7
| |
| * Refactor project_cache_worker_keyHiroyuki Sato2019-04-051-1/+1
| |
| * Fix the bug that the project statistics is not updatedHiroyuki Sato2019-04-053-6/+31
| |