| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
| |
Transform `CancelledError` into `JobRetry::Skip`
|
| |
|
|
|
|
|
| |
- Retry connection when it fails
- Properly shutdown daemon
- Stop monitor if the Exception is raised
- Properly guard exception handling
|
| |
|
|
|
|
|
|
| |
If we process message that is not designated to us
previously we would fire a separate Thread for that.
We don't need to do it. We can cheaply check if thread
is available, if it is, we can perform expensive operation
then.
|
| |
|
|
|
|
|
|
|
|
| |
This makes:
- very shallow `Middleware::Monitor` to only request tracking
of sidekiq jobs,
- `SidekiqStatus::Monitor` to be responsible to maintain persistent
connection to receive messages,
- `SidekiqStatus::Monitor` to always use structured logging
and instance variables
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds a middleware to track all threads
for running jobs.
This makes sidekiq to watch for redis-delivered notifications.
This makes be able to send notification to interrupt
running sidekiq jobs.
This does not take into account any native code,
as `Thread.raise` generates exception once the control gets
back to Ruby.
The separate measure should be taken to interrupt gRPC, shellouts,
or anything else that escapes Ruby.
|
| |
|
|
|
|
|
|
| |
- Added Gitlab::Ci::Config::Entry::Rules and Gitlab::Ci::Config::Entry::Rules:Rule
to handle lists of Rule objects to be evalauted for job inclusion
- Added `if:` and `changes:` as available Rules::Rule::Clause classes
- Added Rules handling logic to Seed::Build#included? with extra specs
- Use DisallowedKeysValidator to mutually exclude rules: from only:/except: on job config
|
| |
|
|
|
|
|
|
| |
Current `auth.log` uses `fullpath` and `ip`, while `api_json.log` uses
`remote_ip` and `path` for the same fields. Let's standardize these
namings to make it easier for people working with the data.
Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/66167
|
| |\
| |
| |
| |
| | |
Count notes for commits and merge requests
See merge request gitlab-org/gitlab-ce!31912
|
| | |
| |
| |
| |
| | |
This extends our existing `Gitlab::UsageDataCounters::NoteCounter` to
also count notes on commits and merge requests
|
| | |
| |
| |
| | |
Fix for https://gitlab.com/gitlab-org/gitlab-ce/issues/66319.
|
| | |
| |
| |
| |
| | |
This allows exceptions to advertise their support for sentry
and provide structured data.
|
| |\ \
| | |
| | |
| | |
| | | |
Cache branch and tag names as Redis sets
See merge request gitlab-org/gitlab-ce!30476
|
| | | |
| | |
| | |
| | |
| | |
| | | |
This allows us to check inclusion for the *_exists? methods without
downloading the full list of branch names, which is over 100KiB in size
for gitlab-ce at the moment.
|
| |\ \ \
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
'master'
Allow UsageData.count to use count_by:
See merge request gitlab-org/gitlab-ce!30770
|
| | | | | |
|
| |\ \ \ \
| | | | |
| | | | |
| | | | |
| | | | | |
Migrate legacy uploads rake tasks
See merge request gitlab-org/gitlab-ce!29409
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
- move uploads created by AttachmentUploader
- handle also files created for legacy_diff_notes
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
Previously `ProjectCacheWorker` would be scheduled once per ref, which
would generate unnecessary I/O and load on Sidekiq, especially if many
tags or branches were pushed at once. `ProjectCacheWorker` would expire
three items:
1. Repository size: This only needs to be updated once per push.
2. Commit count: This only needs to be updated if the default branch
is updated.
3. Project method caches: This only needs to be updated if the default
branch changes, but only if certain files change (e.g. README,
CHANGELOG, etc.).
Because the third item requires looking at the actual changes in the
commit deltas, we schedule one `ProjectCacheWorker` to handle the first
two cases, and schedule a separate `ProjectCacheWorker` for the third
case if it is needed. As a result, this brings down the number of
`ProjectCacheWorker` jobs from N to 2.
Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/52046
|
| | |_|_|/
|/| | |
| | | |
| | | | |
This removes the group_overview_security_dashboard feature flag
|
| |\ \ \ \
| |_|_|/
|/| | |
| | | |
| | | |
| | | |
| | | | |
Add `needs:` CI config option CE
Closes gitlab-ee#12334
See merge request gitlab-org/gitlab-ce!31346
|
| | | | |
| | | |
| | | |
| | | |
| | | | |
Ports changes from
https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/12343
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
https://gitlab.com/gitlab-org/gitlab-ce/issues/62971
Adds support for embedding specific charts from the
metrics dashboard. Expected parameters are dashboard,
title, group, and y_label.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
As per https://gitlab.com/gitlab-org/gitlab-ce/issues/46043, project
templates should be squashed before updating, so that repositories
created from these templates don't include the full history of the
backing repository.
|
| | | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
This change lays the foundation for customizable cycle analytics stages.
The main reason for the change is to extract the event definitions to
separate objects (start_event, end_event) so that it could be easily
customized later on.
|
| | | | |
| | | |
| | | |
| | | |
| | | | |
This will avoid Danger to suggest a database review for files inside
db/fixtures
|
| |\ \ \ \
| | | | |
| | | | |
| | | | |
| | | | | |
Remove :puma_phase metrics
See merge request gitlab-org/gitlab-ce!31773
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
We don't use phase restarts, as we use `preload_app`:
https://github.com/puma/puma/blob/master/README.md#clustered-mode
`:puma_phase` values will always be zero.
|
| | | | | |
| | | | |
| | | | |
| | | | | |
- refactor Knative and Prometheus
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
This introduces several changes, but these are all just ported from the
EE project.
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
This change adds a new counter 'cycle_analytics_views' to the usage data
metrics to count the page views for cycle analytics show page.
|
| |\ \ \ \ \
| | | | | |
| | | | | |
| | | | | |
| | | | | | |
Add `ci_dag_limit_needs`
See merge request gitlab-org/gitlab-ce!31803
|
| | |/ / / /
| | | | |
| | | | |
| | | | |
| | | | | |
This makes to limit `needs:` to 5 by default.
Allow to increase the limit to 50 with disable of FF.
|
| |\ \ \ \ \
| |/ / / /
|/| | | |
| | | | |
| | | | |
| | | | |
| | | | | |
Reduce Gitaly calls in PostReceive
Closes #65878
See merge request gitlab-org/gitlab-ce!31741
|
| | | |/ /
| |/| |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | | |
This commit reduces I/O load and memory utilization during PostReceive
for the common case when no project hooks or services are set up.
We saw a Gitaly N+1 issue in `CommitDelta` when many tags or branches
are pushed. We can reduce this overhead in the common case because we
observe that most new projects do not have any Web hooks or services,
especially when they are first created. Previously, `BaseHooksService`
unconditionally iterated through the last 20 commits of each ref to
build the `push_data` structure. The `push_data` structured was used in
numerous places:
1. Building the push payload in `EventCreateService`
2. Creating a CI pipeline
3. Executing project Web or system hooks
4. Executing project services
5. As the return value of `BaseHooksService#execute`
6. `BranchHooksService#invalidated_file_types`
We only need to generate the full `push_data` for items 3, 4, and 6.
Item 1: `EventCreateService` only needs the last commit and doesn't
actually need the commit deltas.
Item 2: In addition, `Ci::CreatePipelineService` only needed a subset of
the parameters.
Item 5: The return value of `BaseHooksService#execute` also wasn't being
used anywhere.
Item 6: This is only used when pushing to the default branch, so if
many tags are pushed we can save significant I/O here.
Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/65878
Fic
|
| |\ \ \ \
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
Require `needs:` to be present
Closes #65839
See merge request gitlab-org/gitlab-ce!31761
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
This changes the `needs:` logic to require
that all jobs to be present. Instead of skipping
do fail the pipeline creation if `needs:` dependency
is not found.
|
| | | | | |
| | | | |
| | | | |
| | | | | |
Source Code Usage Ping for Create SMAU
|
| |\ \ \ \ \
| | | | | |
| | | | | |
| | | | | |
| | | | | | |
Bump Helm to 2.14.3 and kubectl to 1.11.10
See merge request gitlab-org/gitlab-ce!31716
|
| | | | | | | |
|
| | | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | | |
Previously each tag in a push would invoke the Gitaly `FindAllTags` RPC
since the tag cache would be invalidated with every tag.
We can eliminate those extraneous calls by expiring the tag cache once
in `PostReceive` and taking advantage of the cached tags.
Relates to https://gitlab.com/gitlab-org/gitlab-ce/issues/65795
|
| |\ \ \ \ \ \
| |_|/ / / /
|/| | | | |
| | | | | |
| | | | | | |
Only expire branch cache once per push
See merge request gitlab-org/gitlab-ce!31653
|
| | | | | | | |
|
| | | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | | |
Whenever `PostReceive` is enqueued, `UpdateMergeRequestsWorker`
is enqueued and `MergeRequests::RefreshService` is called, it'll
check if the source branch of each MR asssociated to the push exists
or not via `MergeRequest#source_branch_exists?`. The said method will
call `Repository#branch_exists?` which is cached in `Rails.cache`.
When the cache contains outdated data and the source branch actually
exists, the `MergeRequests#RefreshService` job will close associated
MRs which is not correct.
The fix is to expire the branches cache of the project so we have
updated data during the post receive hook which will help in the
accuracy of the check if we need to close associated MRs or not.
|
| | |_|/ / /
|/| | | |
| | | | |
| | | | |
| | | | |
| | | | | |
Since we are unsure what would be the behavior of `stage:`
when we work on DAG. Let's make `stage:` to be required
today with `needs:`.
|
| |\ \ \ \ \
| | | | | |
| | | | | |
| | | | | |
| | | | | | |
Load search result counts asynchronously (CE)
See merge request gitlab-org/gitlab-ce!31663
|
| | | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | | |
Querying all counts for the different search results in the same request
led to timeouts, so we now only calculate the count for the *current*
search results, and request the others in separate asynchronous calls.
|
| |\ \ \ \ \ \
| |/ / / / /
|/| | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | | |
Properly save suggestions in project exports
Closes #65880
See merge request gitlab-org/gitlab-ce!31690
|
| | | |/ / /
| |/| | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
Previously imports would fail if a merge request note included a
suggestion with an
`ActiveRecord::HasManyThroughCantAssociateThroughHasOneOrManyReflection`
exception.
This was happening because suggestions were listed as a descendant of
merge requests, but this doesn't work because suggestions are directly
associated with notes, not merge requests, and that association is lost.
Rails also disallows creating intializing a has_many association through
a different object.
We fix this by making `suggestions` a child of `notes` within a merge
request. This doesn't fix previously broken exported project exports,
but new exports will work.
Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/65880
|
| |\ \ \ \ \
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | | |
Adds highlight to collapsible line in job log
Closes #65790
See merge request gitlab-org/gitlab-ce!31665
|
| | |/ / / /
| | | | |
| | | | |
| | | | |
| | | | | |
In the job log adds a highlight when hovering
the collapsible line
|
