| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
| |
Adds import jid to projects
Refactor middleware to set custom expiration time via sidekiq options
Add completed_jids option to sidekiq status and a few other changes
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Dumping too many jobs in the same queue (e.g. the "default" queue) is a
dangerous setup. Jobs that take a long time to process can effectively
block any other work from being performed given there are enough of
these jobs.
Furthermore it becomes harder to monitor the jobs as a single queue
could contain jobs for different workers. In such a setup the only
reliable way of getting counts per job is to iterate over all jobs in a
queue, which is a rather time consuming process.
By using separate queues for various workers we have better control over
throughput, we can add weight to queues, and we can monitor queues
better. Some workers still use the same queue whenever their work is
related. For example, the various CI pipeline workers use the same
"pipeline" queue.
This commit includes a Rails migration that moves Sidekiq jobs from the
old queues to the new ones. This migration also takes care of doing the
inverse if ever needed. This does require downtime as otherwise new jobs
could be scheduled in the old queues after this migration completes.
This commit also includes an RSpec test that blacklists the use of the
"default" queue and ensures cron workers use the "cronjob" queue.
Fixes gitlab-org/gitlab-ce#23370
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
GitLab Performance Monitoring is now able to track custom events not
directly related to application performance. These events include the
number of tags pushed, repositories created, builds registered, etc.
The use of these events is to get a better overview of how a GitLab
instance is used and how that may affect performance. For example, a
large number of Git pushes may have a negative impact on the underlying
storage engine.
Events are stored in the "events" measurement and are not prefixed with
"rails_" or "sidekiq_", this makes it easier to query events with the
same name triggered from different parts of the application. All events
being stored in the same measurement also makes it easier to downsample
data.
Currently the following events are tracked:
* Creating repositories
* Removing repositories
* Changing the default branch of a repository
* Pushing a new tag
* Removing an existing tag
* Pushing a commit (along with the branch being pushed to)
* Pushing a new branch
* Removing an existing branch
* Importing a repository (along with the URL we're importing)
* Forking a repository (along with the source/target path)
* CI builds registered (and when no build could be found)
* CI builds being updated
* Rails and Sidekiq exceptions
Fixes gitlab-org/gitlab-ce#13720
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
| |
This keeps all the cache expiration code in a single file/class instead
of spreading it all across the codebase.
|
| |
|
|
|
|
|
|
| |
This ensures the caches for Repository#empty? and
Repository#has_visible_content? are flushed after a repository has been
imported or forked.
Fixes gitlab-org/gitlab-ce#13505
|
| | |
|
| |
|
|
| |
It should exist in EE only.
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
and request them each session. Pass these tokens to the project import data.
This prevents the need to encrypt these tokens and clear them in case they
expire or get revoked.
For example, if you deleted and re-created OAuth2 keys for Bitbucket, you would get
an Error 500 with no way to recover:
```
Started GET "/import/bitbucket/status" for x.x.x.x at 2015-08-07 05:24:10 +0000
Processing by Import::BitbucketController#status as HTML
Completed 500 Internal Server Error in 607ms (ActiveRecord: 2.3ms)
NameError (uninitialized constant Import::BitbucketController::Unauthorized):
app/controllers/import/bitbucket_controller.rb:77:in `rescue in go_to_bitbucket_for_permissions'
app/controllers/import/bitbucket_controller.rb:74:in `go_to_bitbucket_for_permissions'
app/controllers/import/bitbucket_controller.rb:86:in `bitbucket_unauthorized'
```
Closes #1871
|
| |
|
|
|
| |
This reverts commit 5daf44b7c86e0e2641a902b1da8b01d91fa3dbfa, reversing
changes made to 2f706fbd231cabe7a76a5d17ac44285aaaf8592c.
|
| |
|
|
|
|
|
| |
This reverts commit 957e849f41d96fa9778fcdd06792d2f0274b29ab, reversing
changes made to 6b9dbe9f5a175a8162abf296367f561bab3eea1a.
Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>
|
| |\
| |
| |
| | |
Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>
|
| | |
| |
| |
| | |
Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>
|
| |/ |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>
|
| |
|
|
| |
Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>
|
| |
|
|
| |
Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>
|
|
|
and large repos.
So lets do it async. First create project, then import repo and create
satellite with Sidekiq
|
