summaryrefslogtreecommitdiff
path: root/spec/lib/gitlab
Commit message (Collapse)AuthorAgeFilesLines
* CE port: allow SRV records in DB service discoverydb_load_balancing_service_discovery_srvTiger2019-08-231-0/+11
|
* Expose namespace storage statistics with GraphQLAlessio Caiazza2019-08-221-0/+18
| | | | | | | | | | | | | | | | | | | Root namespaces have storage statistics. This commit allows namespace owners to get those stats via GraphQL queries like the following one { namespace(fullPath: "a_namespace_path") { rootStorageStatistics { storageSize repositorySize lfsObjectsSize buildArtifactsSize packagesSize wikiSize } } }
* Log time spent on CPU to sidekiq.logBalakumar2019-08-221-1/+36
|
* Merge branch 'feat/smime-signed-notification-emails' into 'master'Sean McGivern2019-08-224-3/+155
|\ | | | | | | | | feat: smime signed notification emails See merge request gitlab-org/gitlab-ce!30644
| * feat: SMIME signed notification emailsDiego Louzán2019-08-204-3/+155
| | | | | | | | | | | | - Add mail interceptor the signs outgoing email with SMIME - Add lib and helpers to work with SMIME data - New configuration params for setting up SMIME key and cert files
* | Merge branch 'sidekiq-interrupt-running-jobs-deadset' into 'master'Stan Hu2019-08-211-1/+9
|\ \ | | | | | | | | | | | | Put cancelled job in DeadSet See merge request gitlab-org/gitlab-ce!32070
| * | Put cancelled job in DeadSetsidekiq-interrupt-running-jobs-deadsetKamil Trzciński2019-08-211-1/+9
| | | | | | | | | | | | | | | This replicates Sidekiq behavior of pushing dead job into DeadSet.
* | | Ensure CI matching operator receives an objectMarius Bobin2019-08-212-0/+56
|/ / | | | | | | | | Ensure the evaluation of right-hand side expression always results in the returning of an object or an empty String
* | Properly handle `sidekiq` skipsidekiq-interrupt-running-jobsKamil Trzciński2019-08-212-7/+19
| | | | | | | | Transform `CancelledError` into `JobRetry::Skip`
* | Improve resillency of monitorKamil Trzciński2019-08-212-15/+90
| | | | | | | | | | | | | | - Retry connection when it fails - Properly shutdown daemon - Stop monitor if the Exception is raised - Properly guard exception handling
* | Perform cheap thread findKamil Trzciński2019-08-211-3/+1
| | | | | | | | | | | | | | | | If we process message that is not designated to us previously we would fire a separate Thread for that. We don't need to do it. We can cheaply check if thread is available, if it is, we can perform expensive operation then.
* | Rework `Sidekiq::JobsThreads` into `Monitor`Kamil Trzciński2019-08-213-83/+235
| | | | | | | | | | | | | | | | | | | | This makes: - very shallow `Middleware::Monitor` to only request tracking of sidekiq jobs, - `SidekiqStatus::Monitor` to be responsible to maintain persistent connection to receive messages, - `SidekiqStatus::Monitor` to always use structured logging and instance variables
* | Allow to interrupt running jobsKamil Trzciński2019-08-211-0/+83
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds a middleware to track all threads for running jobs. This makes sidekiq to watch for redis-delivered notifications. This makes be able to send notification to interrupt running sidekiq jobs. This does not take into account any native code, as `Thread.raise` generates exception once the control gets back to Ruby. The separate measure should be taken to interrupt gRPC, shellouts, or anything else that escapes Ruby.
* | Introducing new Syntax for Ci::Build inclusion rulesdrew2019-08-209-19/+1080
| | | | | | | | | | | | | | | | - Added Gitlab::Ci::Config::Entry::Rules and Gitlab::Ci::Config::Entry::Rules:Rule to handle lists of Rule objects to be evalauted for job inclusion - Added `if:` and `changes:` as available Rules::Rule::Clause classes - Added Rules handling logic to Seed::Build#included? with extra specs - Use DisallowedKeysValidator to mutually exclude rules: from only:/except: on job config
* | Standardize remote_ip and path keys for auth.log and api_json.logStan Hu2019-08-201-2/+2
| | | | | | | | | | | | | | | | Current `auth.log` uses `fullpath` and `ip`, while `api_json.log` uses `remote_ip` and `path` for the same fields. Let's standardize these namings to make it easier for people working with the data. Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/66167
* | Merge branch 'bvl-mr-commit-note-counter' into 'master'Douwe Maan2019-08-202-10/+24
|\ \ | | | | | | | | | | | | Count notes for commits and merge requests See merge request gitlab-org/gitlab-ce!31912
| * | Count comments on notes and merge requestsBob Van Landuyt2019-08-162-10/+24
| | | | | | | | | | | | | | | This extends our existing `Gitlab::UsageDataCounters::NoteCounter` to also count notes on commits and merge requests
* | | Allow measurement for Sidekiq jobs taking > 2.5sAndrew Newdigate2019-08-201-1/+1
| | | | | | | | | | | | Fix for https://gitlab.com/gitlab-org/gitlab-ce/issues/66319.
* | | Add support for sentry_extra_data in exceptionsAlex Kalderimis2019-08-191-2/+27
| | | | | | | | | | | | | | | This allows exceptions to advertise their support for sentry and provide structured data.
* | | Merge branch '64251-branch-name-set-cache' into 'master'Robert Speicher2019-08-192-2/+78
|\ \ \ | | | | | | | | | | | | | | | | Cache branch and tag names as Redis sets See merge request gitlab-org/gitlab-ce!30476
| * | | Cache branch and tag names as Redis setsNick Thomas2019-08-162-2/+78
| | | | | | | | | | | | | | | | | | | | | | | | This allows us to check inclusion for the *_exists? methods without downloading the full list of branch names, which is over 100KiB in size for gitlab-ce at the moment.
* | | | Merge branch '39-count-unique-users-for-more-accurate-smau-reporting' into ↵Nick Thomas2019-08-191-0/+6
|\ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 'master' Allow UsageData.count to use count_by: See merge request gitlab-org/gitlab-ce!30770
| * | | | Allow UsageData.count to use count_by:Ash McKenzie2019-07-181-0/+6
| | | | |
* | | | | Merge branch 'legacy-attachments-migrate-fix' into 'master'Michael Kozono2019-08-162-0/+359
|\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | Migrate legacy uploads rake tasks See merge request gitlab-org/gitlab-ce!29409
| * | | | | Add rake tasks for migrating leacy uploadslegacy-attachments-migrate-fixJarka Košanová2019-08-162-0/+359
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - move uploads created by AttachmentUploader - handle also files created for legacy_diff_notes
* | | | | | Expire project caches once per push instead of once per refStan Hu2019-08-161-1/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously `ProjectCacheWorker` would be scheduled once per ref, which would generate unnecessary I/O and load on Sidekiq, especially if many tags or branches were pushed at once. `ProjectCacheWorker` would expire three items: 1. Repository size: This only needs to be updated once per push. 2. Commit count: This only needs to be updated if the default branch is updated. 3. Project method caches: This only needs to be updated if the default branch changes, but only if certain files change (e.g. README, CHANGELOG, etc.). Because the third item requires looking at the actual changes in the commit deltas, we schedule one `ProjectCacheWorker` to handle the first two cases, and schedule a separate `ProjectCacheWorker` for the third case if it is needed. As a result, this brings down the number of `ProjectCacheWorker` jobs from N to 2. Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/52046
* | | | | | Remove Security Dashboard feature flagrossfuhrman2019-08-161-5/+0
| |_|_|/ / |/| | | | | | | | | | | | | | This removes the group_overview_security_dashboard feature flag
* | | | | Merge branch 'mc/feature/pipeline-tracking-config-ce' into 'master'Kamil Trzciński2019-08-162-5/+24
|\ \ \ \ \ | |_|_|/ / |/| | | | | | | | | | | | | | | | | | | | | | | | Add `needs:` CI config option CE Closes gitlab-ee#12334 See merge request gitlab-org/gitlab-ce!31346
| * | | | Port changes from EEmc/feature/pipeline-tracking-config-ceMatija Čupić2019-08-042-5/+24
| | | | | | | | | | | | | | | | | | | | | | | | | Ports changes from https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/12343
* | | | | Support query parameters in metrics embedsSarah Yasonik2019-08-151-2/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | https://gitlab.com/gitlab-org/gitlab-ce/issues/62971 Adds support for embedding specific charts from the metrics dashboard. Expected parameters are dashboard, title, group, and y_label.
* | | | | Squash project templates on updateHordur Freyr Yngvason2019-08-151-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As per https://gitlab.com/gitlab-org/gitlab-ce/issues/46043, project templates should be squashed before updating, so that repositories created from these templates don't include the full history of the backing repository.
* | | | | Migrations for Cycle Analytics backendAdam Hegyi2019-08-151-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change lays the foundation for customizable cycle analytics stages. The main reason for the change is to extract the event definitions to separate objects (start_event, end_event) so that it could be easily customized later on.
* | | | | Removes db/fixtures from database filesMayra Cabrera2019-08-151-4/+12
| | | | | | | | | | | | | | | | | | | | | | | | | This will avoid Danger to suggest a database review for files inside db/fixtures
* | | | | Merge branch '65278-livesum-puma-phase' into 'master'Kamil Trzciński2019-08-151-2/+0
|\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | Remove :puma_phase metrics See merge request gitlab-org/gitlab-ce!31773
| * | | | | Remove :puma_phase metrics65278-livesum-puma-phaseAleksei Lipniagov2019-08-141-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We don't use phase restarts, as we use `preload_app`: https://github.com/puma/puma/blob/master/README.md#clustered-mode `:puma_phase` values will always be zero.
* | | | | | Make use of Gitlab::KubernetesJoão Cunha2019-08-141-0/+48
| | | | | | | | | | | | | | | | | | | | | | | | - refactor Knative and Prometheus
* | | | | | Migrates Snowplow backend from EE to CEJeremy Jackson2019-08-141-0/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This introduces several changes, but these are all just ported from the EE project.
* | | | | | Track page views for cycle analytics show pageAdam Hegyi2019-08-142-0/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change adds a new counter 'cycle_analytics_views' to the usage data metrics to count the page views for cycle analytics show page.
* | | | | | Merge branch 'limit-amount-of-needs' into 'master'Grzegorz Bizon2019-08-141-3/+33
|\ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add `ci_dag_limit_needs` See merge request gitlab-org/gitlab-ce!31803
| * | | | | | Add `ci_dag_limit_needs` feature flaglimit-amount-of-needsKamil Trzciński2019-08-141-3/+33
| |/ / / / / | | | | | | | | | | | | | | | | | | | | | | | | This makes to limit `needs:` to 5 by default. Allow to increase the limit to 50 with disable of FF.
* | | | | | Merge branch 'sh-optimize-commit-deltas-post-receive' into 'master'Nick Thomas2019-08-141-0/+34
|\ \ \ \ \ \ | |/ / / / / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Reduce Gitaly calls in PostReceive Closes #65878 See merge request gitlab-org/gitlab-ce!31741
| * | | | | Reduce Gitaly calls in PostReceivesh-optimize-commit-deltas-post-receiveStan Hu2019-08-121-0/+34
| | |/ / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit reduces I/O load and memory utilization during PostReceive for the common case when no project hooks or services are set up. We saw a Gitaly N+1 issue in `CommitDelta` when many tags or branches are pushed. We can reduce this overhead in the common case because we observe that most new projects do not have any Web hooks or services, especially when they are first created. Previously, `BaseHooksService` unconditionally iterated through the last 20 commits of each ref to build the `push_data` structure. The `push_data` structured was used in numerous places: 1. Building the push payload in `EventCreateService` 2. Creating a CI pipeline 3. Executing project Web or system hooks 4. Executing project services 5. As the return value of `BaseHooksService#execute` 6. `BranchHooksService#invalidated_file_types` We only need to generate the full `push_data` for items 3, 4, and 6. Item 1: `EventCreateService` only needs the last commit and doesn't actually need the commit deltas. Item 2: In addition, `Ci::CreatePipelineService` only needed a subset of the parameters. Item 5: The return value of `BaseHooksService#execute` also wasn't being used anywhere. Item 6: This is only used when pushing to the default branch, so if many tags are pushed we can save significant I/O here. Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/65878 Fic
* | | | | Merge branch 'require-needs-to-be-present' into 'master'Grzegorz Bizon2019-08-142-2/+25
|\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Require `needs:` to be present Closes #65839 See merge request gitlab-org/gitlab-ce!31761
| * | | | | Require `needs:` to be presentrequire-needs-to-be-presentKamil Trzciński2019-08-132-2/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This changes the `needs:` logic to require that all jobs to be present. Instead of skipping do fail the pipeline creation if `needs:` dependency is not found.
* | | | | | Add usage pings for source code pushesIgor2019-08-132-1/+11
| | | | | | | | | | | | | | | | | | | | | | | | Source Code Usage Ping for Create SMAU
* | | | | | Merge branch 'bump_helm_kubectl_gitlab' into 'master'Douglas Barbosa Alexandre2019-08-131-1/+1
|\ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bump Helm to 2.14.3 and kubectl to 1.11.10 See merge request gitlab-org/gitlab-ce!31716
| * | | | | | Bump Helm to 2.14.3 and kubectl to 1.11.10bump_helm_kubectl_gitlabThong Kuah2019-08-131-1/+1
| | | | | | |
* | | | | | | Only expire tag cache once per pushsh-only-flush-tags-once-per-pushStan Hu2019-08-131-0/+43
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously each tag in a push would invoke the Gitaly `FindAllTags` RPC since the tag cache would be invalidated with every tag. We can eliminate those extraneous calls by expiring the tag cache once in `PostReceive` and taking advantage of the cached tags. Relates to https://gitlab.com/gitlab-org/gitlab-ce/issues/65795
* | | | | | | Merge branch '65803-invalidate-branches-cache-on-refresh' into 'master'Bob Van Landuyt2019-08-131-0/+52
|\ \ \ \ \ \ \ | |_|/ / / / / |/| | | | | | | | | | | | | | | | | | | | Only expire branch cache once per push See merge request gitlab-org/gitlab-ce!31653
| * | | | | | Rename branches_exist? -> includes_branches?65803-invalidate-branches-cache-on-refreshStan Hu2019-08-091-4/+4
| | | | | | |