| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
| |
By passing commit data to this worker we remove the need for querying
the Git repository for every job. This in turn reduces the time spent
processing each job.
The migration included migrates jobs from the old format to the new
format. For this to work properly it requires downtime as otherwise
workers may start producing errors until they're using a newer version
of the worker code.
|
| |
|
|
| |
Signed-off-by: Rémy Coutable <remy@rymai.me>
|
| |\
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
'master'
Add `null: true` to timestamps in migrations that does not define it
This is to ensure that migrations will still be consitent when we will
upgrade to Rails 5 which default to `null: false` for timestamps
columns.
Fixes #23666.
See merge request !7791
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This is to ensure that migrations will still be consitent when we will
upgrade to Rails 5 which default to `null: false` for timestamps
columns.
Fixes #23666.
Signed-off-by: Rémy Coutable <remy@rymai.me>
|
| | | |
|
| |/
|
|
|
|
| |
This new global setting will allow admins to specify if HTML emails should be sent or not,
this is basically useful when system administrators want to save some disk space by avoiding
emails in HTML format and using only the Plain Text version.
|
| | |
|
| |
|
|
|
|
| |
There may be more tables but these were the tables that were problematic
for GitLab.com due to foreign key constraints (without cascading
deletes).
|
| |\
| |
| |
| |
| |
| |
| | |
Allow to block JIRA events for commits and merge requests
implements #5541
See merge request !7469
|
| | | |
|
| |\ \
| |/
|/|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Precalculate authorized projects in database
## What does this MR do?
It caches user's authorized projects in database instead of using multiple unions, which should simplify and speed-up things since this operation (getting authorized projects) is used a lot.
## Are there points in the code the reviewer needs to double check?
Did we miss a scenario where we need to refresh the list of projects?
## Does this MR meet the acceptance criteria?
- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [ ] ~~[Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)~~
- [ ] ~~API support added~~
- Tests
- [x] Added for this feature/bug
- [x] All builds are passing
- [x] Conform by the [merge request performance guides](http://docs.gitlab.com/ce/development/merge_request_performance_guidelines.html)
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if it does - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
## What are the relevant issue numbers?
#23150
See merge request !6839
|
| | |
| |
| |
| | |
Closes #23150
|
| |\ \
| | |
| | |
| | |
| | |
| | |
| | | |
Fix migration missing ignored transaction method
Adds missing `disable_ddl_transaction!` to migration
See merge request !7569
|
| | | | |
|
| |\ \ \
| |/ /
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Cycle Analytics: Events per stage
Adds list of events to each stage:
- Issue: list of issues created in the last XX days, that have been labeled or added to a milestone.
- Plan: list of commits that reference for the fist time an issue from the last stage.
- Code: list of MR created in this stage
- Test: List of unique builds triggered by the commits.
- Review: List of MR merged
- Staging: List of deployed builds
- Production: list of issues with the time from idea to production
Fixes #23449
- [x] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG) entry added
- [ ] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- Tests
- [x] Added for this feature/bug
- [x] All builds are passing
- [x] Conform by the [merge request performance guides](http://docs.gitlab.com/ce/development/merge_request_performance_guidelines.html)
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if it does - rebase it please)
- [ ] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
See merge request !6859
|
| | | | |
|
| | | | |
|
| | |/
| |
| |
| | |
worker to populate this field.
|
| | |
| |
| |
| |
| |
| |
| | |
The database should not have any soft-deleted groups. Due to a race
condition (soft-delete completes after the hard-delete), soft-deleted
groups were (incorrectly) left in the database, causing issues while
trying to create a new group with the same name.
|
| |\ \
| | |
| | |
| | |
| | |
| | |
| | | |
Support subscribing to group labels
https://gitlab.com/gitlab-org/gitlab-ce/issues/23586
See merge request !7215
|
| | | | |
|
| | | | |
|
| | | | |
|
| | | | |
|
| | |/ |
|
| |\ \
| |/
|/|
| |
| |
| |
| | |
Allows to authorize chat user against GitLab.
This is needed for: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7438
See merge request !7450
|
| | | |
|
| | |\ |
|
| | | | |
|
| | | |
| | |
| | |
| | | |
them [ci skip]
|
| |\ \ \
| |_|/
|/| |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Disable the "request access" functionality by default for new groups and projects
Currently this feature is enabled by default, and additional action is required to disable it.
Closes #21992
Closes !7011
See merge request !7425
|
| | |/
| |
| |
| |
| | |
Set request access for projects/groups to false by default.
Request access implemented in !5286 8.10 was enabled by default.
|
| |\ \
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Update 8.14-rc1 migrations to minimize downtime and deploy time
## What does this MR do?
Modify 8.14-rc1 migrations and code to minimize downtime and deploy time
## Are there points in the code the reviewer needs to double check?
No
## Why was this MR needed?
8.14-rc1 migrations took too much time in staging, so we didn't want to take too much downtime in production
## Does this MR meet the acceptance criteria?
- Tests
- [x] Added for this feature/bug
- [x] All builds are passing
- [x] Conform by the [merge request performance guides](http://docs.gitlab.com/ce/development/merge_request_performance_guidelines.html)
- [x] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [x] Branch has no merge conflicts with `master` (if it does - rebase it please)
- [x] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
## What are the relevant issue numbers?
Closes #24386
See merge request !7427
|
| | |/
| |
| |
| | |
See https://gitlab.com/gitlab-org/gitlab-ce/issues/24386
|
| | |
| |
| |
| | |
them to be configurable
|
| |/ |
|
| |\
| |
| |
| |
| | |
Use separate email-friendly token for incoming email
See merge request !5914
|
| | |
| |
| |
| | |
email token be reset
|
| |\ \
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Use more than one kind of Git garbage collection
Replaces https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6588 by @jacobvosmaer to get the builds to pass :)
Closes #22729
See merge request !7321
|
| | | | |
|
| |\ \ \
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
Fixups to "Round-robin repository storage"
## What does this MR do?
* Simplifies a method in application_settings.rb
* Correctly marks a migration as needing downtime
* Documents the requirement for renamed columns to be
## Are there points in the code the reviewer needs to double check?
Should any of these changes be split out? Ideally we'd get this into the same point release as !7273
## Why was this MR needed?
Post-facto review of !7273
## Screenshots (if relevant)
## Does this MR meet the acceptance criteria?
- [ ] [CHANGELOG](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CHANGELOG.md) entry added
- [X] [Documentation created/updated](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/doc/development/doc_styleguide.md)
- [X] API support added
- Tests
- [X] Added for this feature/bug
- [x] All builds are passing
- [X] Conform by the [merge request performance guides](http://docs.gitlab.com/ce/development/merge_request_performance_guidelines.html)
- [X] Conform by the [style guides](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md#style-guides)
- [X] Branch has no merge conflicts with `master` (if it does - rebase it please)
- [X] [Squashed related commits together](https://git-scm.com/book/en/Git-Tools-Rewriting-History#Squashing-Commits)
## What are the relevant issue numbers?
Related to #24059
/cc @yorickpeterse @rspeicher
See merge request !7287
|
| | |/ / |
|
| | |/
|/|
| |
| |
| |
| |
| |
| |
| |
| | |
performance
We see that many slow queries on GitLab.com are dominated by finding
the project import data for a specific project. Adding an index is
the most straightforward way of fixing this.
Closes #23748
|
| |/
|
|
|
|
| |
are resolved
Signed-off-by: Rémy Coutable <remy@rymai.me>
|
| |
|
|
| |
round-robin between them
|
| |\
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Use optimistic locking
## What does this MR do?
Removes the usage of pessimistic locking in favor of optimistic which is way cheaper and doesn't block database operation.
Since this is very simple change it should be safe. If we receive `StaleObjectError` message we will reload object a retry operations in lock.
However, I still believe that we need this one: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7005 as this will reduce a load on Database and FS.
This changes a behavior from:
### Pesimistic locking (previous behavior)
#### For updating
1. SELECT * FOR UPDATE (other updates wait on this)
2. we update ci_pipeline
3. latest_build_status
4. enqueue: (use: transition :created -> :pending)
5. [state_machine] we are in state created, we can go to pending
6. [state_machine] ci_pipeline.status = created
7. [state_machine] ci_pipeline.save
8. [state_machine] after_transition: (if for success): PipelineSuccessWorker on Sidekiq
9. release DB lock
#### If no update is required
1. SELECT * FOR UPDATE (other updates wait on this)
2. we update ci_pipeline
3. latest_build_status
4. we are in pending, we can't transition to pending, because it's forbidden
5. release DB lock
### Optimistic locking (implemented by this MR)
#### For updating
1. latest_build_status
2. enqueue: (use `transition :created -> :pending`)
3. [state_machine] we are in state created, we can go to pending
4. [state_machine] ci_pipeline.status = created
5. [state_machine] ci_pipeline.save
6. [state_machine] [save] where(lock_version: ci_pipeline.lock_version).update_all(status: :created, updated_at: Time.now)
7. [state_machine] [save] unless we_updated_row then raise ObjectInconsistentError
#### If no update is required
1. we update ci_pipeline
2. latest_build_status
3. we are in pending, we can't transition to pending, because it's forbidden
## Why was this MR needed?
We have been seeing a number of problems when we migrated Pipeline/Build processing to Sidekiq. Especially we started seeing a lot of blocking queries.
We used a pessimistic locking which doesn't seem to be required. This effectively allows us to fix our issues with blocked queries by using more efficient method of operation.
## What are the relevant issue numbers?
Issues: https://gitlab.com/gitlab-com/infrastructure/issues/623 and https://gitlab.com/gitlab-com/infrastructure/issues/584, but also there's a bunch of Merge Requests that try to improve behavior of scheduled jobs.
cc @pcarranza @yorickpeterse @stanhu
See merge request !7040
|
| | | |
|
| | |
| |
| |
| | |
Code improvements, bug fixes, finish documentation and specs
|
| |/ |
|
| |
|
|
|
|
|
|
|
|
| |
mail_room was configured to deliver mail to the `incoming_email`
queue while `EmailReceiveWorker` was reading the `email_receiver`
queue. Adds a migration that repeats the work of a previous
migration to ensure all mails that wound up in the old
queue get processed.
Closes #23689
|
