summaryrefslogtreecommitdiff
path: root/spec/tasks
Commit message (Collapse)AuthorAgeFilesLines
* Enhance performance of counting local UploadsToon Claes2018-11-071-8/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add an index to the `store` column on `uploads`. This makes counting local uploads faster. Also, there is no longer need to check for objects with `store = NULL`. See https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/18557 --- ### Query plans Query: ```sql SELECT COUNT(*) FROM "uploads" WHERE ("uploads"."store" = ? OR "uploads"."store" IS NULL) ``` #### Without index ``` gitlabhq_production=# EXPLAIN ANALYZE SELECT uploads.* FROM uploads WHERE (uploads.store = 1 OR uploads.store IS NULL); QUERY PLAN --------------------------------------------------------------------------------------------------------------- Seq Scan on uploads (cost=0.00..601729.54 rows=578 width=272) (actual time=6.170..2308.256 rows=545 loops=1) Filter: ((store = 1) OR (store IS NULL)) Rows Removed by Filter: 4411957 Planning time: 38.652 ms Execution time: 2308.454 ms (5 rows) ``` #### Add index ``` gitlabhq_production=# create index uploads_tmp1 on uploads (store); CREATE INDEX ``` #### With index ``` gitlabhq_production=# EXPLAIN ANALYZE SELECT uploads.* FROM uploads WHERE (uploads.store = 1 OR uploads.store IS NULL); QUERY PLAN ------------------------------------------------------------------------------------------------------------------------------- Bitmap Heap Scan on uploads (cost=11.46..1238.88 rows=574 width=272) (actual time=0.155..0.577 rows=545 loops=1) Recheck Cond: ((store = 1) OR (store IS NULL)) Heap Blocks: exact=217 -> BitmapOr (cost=11.46..11.46 rows=574 width=0) (actual time=0.116..0.116 rows=0 loops=1) -> Bitmap Index Scan on uploads_tmp1 (cost=0.00..8.74 rows=574 width=0) (actual time=0.095..0.095 rows=545 loops=1) Index Cond: (store = 1) -> Bitmap Index Scan on uploads_tmp1 (cost=0.00..2.44 rows=1 width=0) (actual time=0.020..0.020 rows=0 loops=1) Index Cond: (store IS NULL) Planning time: 0.274 ms Execution time: 0.637 ms (10 rows) ``` Closes https://gitlab.com/gitlab-org/gitlab-ee/issues/6070
* Fix typos in comments and specsGeorge Tsiolis2018-11-011-1/+1
|
* Geo: sync disabled wikis. Stage 2follow-up-geo-sync-disabled-wikisValery Sizov2018-09-251-3/+2
| | | | | | | We started syncing all the wiki regardless of the fact it's disabled or not. We couldn't do that in one stage because of needing of smoth update and deprecating things. This is the second stage that finally removes unused columns in the geo_node_status table.
* Include post migrations when loading the schemaYorick Peterse2018-09-191-0/+33
|\ | | | | See merge request gitlab-org/gitlab-ce!21689
| * Fix the schema load testDJ Mountney2018-09-121-11/+15
| | | | | | | | And added changelog
| * Ensure the schema is loaded with post_migrations includedDJ Mountney2018-09-111-0/+29
| | | | | | | | | | | | If doing a schema load, the post_migrations should also be marked as up, even if SKIP_POST_DEPLOYMENT_MIGRATIONS was set, otherwise future migration runs will be broken.
* | Port cleanup tasks to use GitalyZeger-Jan van de Weg2018-09-071-15/+20
|/ | | | | | | | | | | | | | | | | | | | | | | Rake tasks cleaning up the Git storage were still using direct disk access, which won't work if these aren't attached. To mitigate a migration issue was created. To port gitlab:cleanup:dirs, and gitlab:cleanup:repos, a new RPC was required, ListDirectories. This was implemented in Gitaly, through https://gitlab.com/gitlab-org/gitaly/merge_requests/868. To be able to use the new RPC the Gitaly server was bumped to v0.120. This is an RPC that will not use feature gates, as this doesn't scale on .com so there is no way to test it at scale. Futhermore, we _know_ it doesn't scale, but this might be a useful task for smaller instances. Lastly, the tests are slightly updated to also work when the disk isn't attached. Eventhough this is not planned, it was very little effort and thus I applied the boy scout rule. Closes https://gitlab.com/gitlab-org/gitaly/issues/954 Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/40529
* Refactor SiteStatistics to extract refresh logic into a rake taskGabriel Mazetto2018-08-161-0/+24
|
* Add rake command to migrate archived traces from local storage to object storageShinya Maeda2018-08-161-27/+85
|
* Merge branch 'gitaly-install-path' into 'master'Douwe Maan2018-08-141-51/+16
|\ | | | | | | | | Remove storage path dependency of gitaly install task See merge request gitlab-org/gitlab-ce!21101
| * Remove storage path dependency of gitaly install taskAlejandro Rodríguez2018-08-121-51/+16
| |
* | Add object storage related tests for `gitlab:cleanup:project_uploads` taskMichael Kozono2018-08-031-276/+45
| |
* | Merge branch 'zj-backup-migration-done' into 'master'Sean McGivern2018-08-021-0/+21
|\ \ | | | | | | | | | | | | | | | | | | Remove feature flags from lib/backup Closes gitaly#749, gitaly#1212, and gitaly#1195 See merge request gitlab-org/gitlab-ce!20854
| * | Remove feature flags from lib/backupZeger-Jan van de Weg2018-07-311-0/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Moved to OPT_OUT in 7d14b725a0da41d1ae7c0a8496b5e66832023e3b, Now, by removing the feature gates, this is an mandatory feature. Related issues: - https://gitlab.com/gitlab-org/gitaly/issues/526 - https://gitlab.com/gitlab-org/gitaly/issues/1194 Closes https://gitlab.com/gitlab-org/gitaly/issues/749 Closes https://gitlab.com/gitlab-org/gitaly/issues/1212 Closes https://gitlab.com/gitlab-org/gitaly/issues/1195
* | | Add local project uploads cleanup taskMichael Kozono2018-07-311-1/+316
| |/ |/|
* | Remove git rake tasksZeger-Jan van de Weg2018-07-311-28/+2
|/ | | | | | | | | | These tasks are happening through housekeeping right now, by default ever 10th push. This removes the need for these tasks. Side note, this removes one of my first contributions to GitLab, as back than I introduced these tasks through: 54e6c0045bb13c05cc5478cbdf47d3246bd9fe2b Closes https://gitlab.com/gitlab-org/gitaly/issues/768
* Speed up spec/tasks/gitlab/git_rake_spec.rbRobert Speicher2018-07-031-6/+11
| | | | | | Because no Git repository was actually created at the temporary path we were using, `git fsck` would traverse up until it found a repository, which in our case was the CE or EE repository.
* Fix static analysis failureDJ Mountney2018-06-271-1/+1
|
* Update tests for gitlab:db:configureDJ Mountney2018-06-271-1/+9
|
* Find and mark more Git disk access locations, part 2Jacob Vosmaer2018-06-122-2/+10
|
* Resolve "Hashed Storage: Make possible to migrate single project"Gabriel Mazetto2018-06-071-9/+36
|
* Find and mark more Git disk access locationsJacob Vosmaer (GitLab)2018-06-051-6/+13
|
* Export assigned issues in iCalendar feedImre Farkas2018-05-311-2/+2
|
* Specs covering listing all gitlab:storage:* typesbrodock-refactor-hashed-storage-taskGabriel Mazetto2018-05-291-4/+100
|
* Fix backup creation and restore for specific Rake tasksStan Hu2018-05-241-0/+10
|
* Gitlab::Shell works on shard name, not pathZeger-Jan van de Weg2018-04-251-8/+13
| | | | | | | | | | | Direct disk access is done through Gitaly now, so the legacy path was deprecated. This path was used in Gitlab::Shell however. This required the refactoring in this commit. Added is the removal of direct path access on the project model, as that lookup wasn't needed anymore is most cases. Closes https://gitlab.com/gitlab-org/gitaly/issues/1111
* Allow `rake cache:clear` clearing pipeline status cache44582-clear-pipeline-status-cacheLin Jen-Shin2018-04-101-0/+19
| | | | | | | | | * Use the correct key prefix * Clear old cache keys TODO: At some point we could remove clearing old cache keys.
* fix the rake task from being to strict about GroupMicaël Bergeron2018-04-021-7/+108
|
* apply feedbackMicaël Bergeron2018-03-301-1/+3
|
* fix a future conflictMicaël Bergeron2018-03-291-1/+2
|
* fix rake migration task from ignoring Upload(store: nil)Micaël Bergeron2018-03-291-3/+14
|
* Merge branch '40781-os-to-ce' into 'master'Sean McGivern2018-03-273-0/+183
|\ | | | | | | | | | | | | Bring Object Storage to CE Closes #4171, #4163, #3370, #2841, and #29203 See merge request gitlab-org/gitlab-ce!17358
| * fix the failing specsMicaël Bergeron2018-03-221-1/+1
| |
| * Backport EE changesMicaël Bergeron2018-03-221-0/+0
| |
| * Merge remote-tracking branch 'origin/master' into 40781-os-to-ceMicaël Bergeron2018-03-221-0/+34
| |\
| * \ Merge remote-tracking branch 'origin/master' into 40781-os-to-ceMicaël Bergeron2018-03-083-31/+59
| |\ \
| * | | [cherry-pick] '4862-verify-file-checksums'Sean McGivern2018-03-083-1/+57
| | | | | | | | | | | | | | | | See merge request gitlab-org/gitlab-ee!4753
| * | | another round of fixesMicaël Bergeron2018-03-011-0/+28
| | | |
| * | | Merge remote-tracking branch 'origin/master' into ↵object-storage-ee-to-ce-backportKamil Trzciński2018-02-281-1/+1
| |\ \ \ | | | | | | | | | | | | | | | object-storage-ee-to-ce-backport
| * \ \ \ Merge commit '8af23def1d6' into object-storage-ee-to-ce-backportKamil Trzciński2018-02-285-4/+34
| |\ \ \ \
| * | | | | Merge branch '4163-move-uploads-to-object-storage' into 'master'Sean McGivern2018-02-282-18/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Move uploads to object storage Closes #4163 See merge request gitlab-org/gitlab-ee!3867
| * | | | | Merge commit '4b92efd90cedaa0aff218d11fdce279701128bea' into ↵Kamil Trzciński2018-02-281-0/+38
| |\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | object-storage-ee-to-ce-backport
| * | | | | | Merge branch 'zj-multiple-artifacts-ee' into 'master'Grzegorz Bizon2018-02-281-0/+118
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Multiple artifacts ee See merge request gitlab-org/gitlab-ee!3276
| * | | | | | Merge commit '7fabc892f251740dbd9a4755baede662e6854870' into ↵Kamil Trzciński2018-02-283-8/+78
| |\ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | object-storage-ee-to-ce-backport
| * \ \ \ \ \ \ Merge commit 'f2f58a60b76acd479e37bdbc9246ec9f9b2bea82' into ↵Kamil Trzciński2018-02-285-198/+72
| |\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | object-storage-ee-to-ce-backport
| * \ \ \ \ \ \ \ Merge commit '11c67e7c2f992299ff5918ce67995b73d1e0be6d' into ↵Kamil Trzciński2018-02-283-7/+82
| |\ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | object-storage-ee-to-ce-backport
| * | | | | | | | | Merge branch 'jej/lfs-object-storage' into 'master'Douwe Maan2018-02-281-0/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Can migrate LFS objects to S3 style object storage Closes #2841 See merge request !2760
* | | | | | | | | | Route path lookups through legacy_disk_pathmark-legacy-git-accessJacob Vosmaer2018-03-215-13/+28
| |_|_|_|_|_|_|_|/ |/| | | | | | | |
* | | | | | | | | Implement foreground verification of CI artifacts43949-verify-job-artifactsNick Thomas2018-03-071-0/+34
| |_|_|_|_|_|_|/ |/| | | | | | |
* | | | | | | | Fix tests by latest proposal changesproper-fix-for-artifacts-serviceShinya Maeda2018-03-061-2/+2
| | | | | | | |
View Lorry Controller Status