| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| | |
|
| |
|
|
| |
We don't need to find the filename from the remote URL
|
| |
|
|
|
|
|
|
|
| |
When file_mover_spec.rb ran, it would initialize fog-aws with
`Fog::AWS::Storage::Real` service instead of `Fog::AWS::Storage::Mock`
because `Fog.mock!` was not called. Ensure that we use
`stub_uploads_object_storage` to prevent that from happening.
Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/64083
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
It persist temporary personal snippets under
user/:id namespaces temporarily while creating
a upload record to track it. If an user gets removed
while it's still a tmp upload, it also gets removed.
If the tmp upload is sent, the upload gets moved to
personal_snippets/:id as before. The upload record
also gets updated to the new model type as well.
|
| |
|
|
|
|
| |
We've already migrated all the legacy artifacts to the new realm,
which is ci_job_artifacts table.
It's time to remove the old code base that is no longer used.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/24550 fixed the
case where the wrong path on disk was being searched, but it
inadvertently ommitted the `/uploads/-/system` prefix when rendering the
Markdown for personal snippet uploads when they were stored directly in
object storage.
A personal snippet path is stored using FileUploader#upload_path.
The format for the path:
Local storage: :random_hex/:filename.
Object storage: personal_snippet/:id/:random_hex/:filename.
upload_paths represent the possible paths for a given identifier,
which will vary depending on whether the file is stored in local or
object storage. upload_path should match an element in upload_paths.
base_dir represents the path seen by the user in Markdown, and it
should always be prefixed with uploads/-/system.
store_dirs represent the paths that are actually used on disk. For
object storage, this should omit the prefix /uploads/-/system.
For example, consider the requested path
/uploads/-/system/personal_snippet/172/ff4ad5c2/file.png.
For local storage:
base_dir: uploads/-/system/personal_snippet/172
upload_path: ff4ad5c2/file.png
upload_paths: ["ff4ad5c2/file.png", "personal_snippet/172/ff4ad5c2/file.png"].
store_dirs: {1=>"uploads/-/system/personal_snippet/172/ff4ad5c2",
2=>"personal_snippet/172/ff4ad5c2"}
For object storage:
upload_path: personal_snippet/172/ff4ad5c2/file.png
Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/61671
|
| |
|
|
|
|
|
|
| |
String#underscore isn't particularly slow, but it's possible for us to
call it many times in a users autocomplete request, with mostly-static
values ('User', 'Group', etc.). We can memoise this and save a
surprising amount of time (around 10% of the total request time in some
cases).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Unlike uploads that have been uploaded with Tempfile, the project
import/export archives are stored in a temporary cache directory and
remain there if:
1. Object storage is enabled
2. `move_to_store` is set to `true`.
CarrierWave will leave these files there until disk space runs out or a
clean step is run manually.
If `move_to_store` is set to `false`, CarrierWave will remove the files
after storing them. However, unlike a local file, with object storage,
the file is still copied, so setting `move_to_store` to `true`
doesn't buy us anything.
To ensure files are cleaned up, we can just inherit from the
GitlabUploader implementation of `move_to_store`, which returns `true`
if it's a local file, `false` otherwise.
Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/60656
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We've seen a significant performance penalty when using
`BatchLoader#__replace_with!`. This defines methods on the batch loader
that proxy to the 'real' object using send. The alternative is
`method_missing`, which is slower. However, we've noticed that
`method_missing` can be faster if:
1. The objects being loaded have a large interface.
2. We don't call too many methods on the loaded object.
Avatar uploads meet both criteria above, so let's use the newly-released
feature in https://github.com/exAspArk/batch-loader/pull/45.
Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/60903
|
| |
|
|
|
|
|
| |
- rewords examples starting with 'should'
- rewords examples starting with 'it'
Note: I had to manually fixup "onlies" to "only"
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With object storage enabled, calling `#filename` on an upload does this:
1. Call the `#filename` method on the CarrierWave object.
2. Generate the URL for that object.
3. If the uploader isn't public, do so by generating an authenticated
URL, including signing that request.
That's all correct behaviour, but for the case where we use `#filename`,
it's typically to generate a GitLab URL. That URL doesn't need to be
signed because we do our own auth.
Signing the URLs can be very expensive, especially in batch (say, we
need to get the avatar URLs for 150 users in one request). It's all
unnecessary work. If we used the `RecordsUploads` concern, we have
already recorded a `path` in the database. That `path` is actually
generated from CarrierWave's `#filename` at upload time, so we don't
need to recompute it - we can just use it and strip off the prefix if
it's available.
On a sample users autocomplete URL, at least 10% of the time before this
change went to signing URLs. After this change, we spend no time in URL
signing, and still get the correct results.
|
| |
|
|
|
|
|
|
| |
Set ACL of uploaded objects to not-public (fog-aws) or no ACL at all
(fog-google). Value is ignored by other supported backends (fog-aliyun,
fog-openstack, fog-rackspace)
This allows uploads to GCS with ACL support disabled.
|
| |
|
|
| |
Previously one could move any temp/ sub folder around.
|
| |
|
|
| |
Currently we pass temp file path to FileMover
|
| | |
|
| |
|
|
| |
Signed-off-by: Rémy Coutable <remy@rymai.me>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously, an HTTP request for
`/uploads/-/system/personal_snippet/:snippet_id/:hash/:filename` would
look for an uploader of `PersonalFileUploader` class and use
`PersonalFileUploader#upload_paths` to search the datbase for one of the
following paths:
1. `:hash/:filename`
2. `uploads/-/system/personal_snippet/:id/:hash/:filename`
However, when the upload were stored in object storage,
`PersonalFileUploader#store_dirs` stored the path as:
`personal_snippet/:snippet_id/:hash`
The extraneous `uploads/-/system` prefix prevented the path from being
matched, and uploads in object storage would return a 404 error. Uploads
in local storage would work fine.
To fix this, we set the `#base_dir` properly so that `#upload_paths`
generates the right value for object storage. Note that this also makes
`#store_dirs` do the right thing in `FileUploader`.
Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/52595
|
| |
|
|
|
|
|
|
|
|
|
| |
Specifically, the `ListAllMyBuckets` permission.
This works if you know the directory exists.
See more:
* https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/23981
* https://stackoverflow.com/a/12288581/1992201
|
| |
|
|
|
| |
/app/spec/uploaders/personal_file_uploader_spec.rb:3: warning: already initialized constant IDENTIFIER
/app/spec/uploaders/namespace_file_uploader_spec.rb:3: warning: previous definition of IDENTIFIER was here
|
| |
|
|
|
|
|
| |
Extract code to make it easier reusable
- introduce AttributesRewriter and ContentRewriter
- support group entites when rewriting content
- make Uploader copy_to working for Namespaces
|
| |\
| |
| |
| |
| |
| |
| | |
Fix workhorse temp path for namespace uploads
Closes gitlab-ee#7009
See merge request gitlab-org/gitlab-ce!21650
|
| | | |
|
| |/ |
|
| |\
| |
| |
| |
| | |
Fix NamespaceUploader.base_dir for remote uploads
See merge request gitlab-org/gitlab-ce!21308
|
| | | |
|
| |\ \
| | |
| | |
| | |
| | |
| | |
| | | |
Migrate job artifacts data from `ci_builds` to `ci_job_artifacts` table (with Background migrations)
Closes #46652
See merge request gitlab-org/gitlab-ce!18615
|
| | |/
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
commit 8ea114e5c349ad23f7293cf8141798aa6c9384e3
Author: Shinya Maeda <shinya@gitlab.com>
Date: Fri Aug 17 13:32:14 2018 +0900
Remove redandant parentheses
commit 3eb34f9305980701f0c1528f95fd615c1ca4d50d
Merge: 24308e4d1da 46494f46a13
Author: Shinya Maeda <shinya@gitlab.com>
Date: Fri Aug 17 13:22:05 2018 +0900
Merge branch 'master-ce' into add-background-migration-for-legacy-traces
commit 24308e4d1dab6fbf8437afc56fff7b83cc7ce534
Author: Shinya Maeda <shinya@gitlab.com>
Date: Thu Aug 16 16:39:20 2018 +0900
Fix spec
commit 5a63312d1b36296440da6a874bb667b7dc06869c
Author: Shinya Maeda <shinya@gitlab.com>
Date: Thu Aug 16 14:46:59 2018 +0900
Format SQL in MIgrateLegacyArtifacts class
commit 8894196d2aa2f9bf153cb03fef9603858478db70
Author: Shinya Maeda <shinya@gitlab.com>
Date: Thu Aug 16 14:31:18 2018 +0900
Remove unnecessary index
commit d9753ac058fc48c8f594ee90064f675f7a869b17
Author: Shinya Maeda <shinya@gitlab.com>
Date: Thu Aug 16 14:24:20 2018 +0900
Decouple pertial index to a different migration file
commit 1ce4a9bec6a4f0f981f4139c1a146ca7d887e6ba
Author: Shinya Maeda <shinya@gitlab.com>
Date: Thu Aug 16 14:14:03 2018 +0900
Simplified adding file_location column
commit 47b101fae914470dc4d048f6d33c509ec8270656
Author: Shinya Maeda <shinya@gitlab.com>
Date: Thu Aug 16 14:11:42 2018 +0900
Remove add_column_with_default
commit 80ec4f774f50bbf66e65f283f8fba541c6334e1c
Author: Shinya Maeda <shinya@gitlab.com>
Date: Thu Aug 16 14:02:46 2018 +0900
Simplified job_artifact_uploader
commit d835dab36e85f4fc7a7d0735ede0f0ea081178bf
Author: Shinya Maeda <shinya@gitlab.com>
Date: Thu Aug 16 13:59:03 2018 +0900
Elaborate comments on `file_location`
commit 6e78f23683bbae1372783f19179206369d7fbae7
Merge: fcef07ef361 79fdfec51fa
Author: Shinya Maeda <shinya@gitlab.com>
Date: Thu Aug 16 13:39:28 2018 +0900
Merge branch 'master-ce' into add-background-migration-for-legacy-traces
commit fcef07ef361080823432071722897321b2dd8354
Merge: 1eac3083194 bf9fd9c3fc3
Author: Shinya Maeda <shinya@gitlab.com>
Date: Fri Jul 6 21:46:22 2018 +0900
Merge branch 'master' into add-background-migration-for-legacy-traces
commit 1eac30831947cceec7610be6acc7d3bffadaf617
Author: Shinya Maeda <shinya@gitlab.com>
Date: Fri Jul 6 21:45:49 2018 +0900
Use SQL instead of strip_heredoc
commit 08b17e517722e1e4ef8b60e753d53d697953a293
Merge: c0f48ff243d dc478a82432
Author: Shinya Maeda <shinya@gitlab.com>
Date: Thu Jul 5 23:20:37 2018 +0900
Merge branch 'master' into add-background-migration-for-legacy-traces
commit c0f48ff243ddbe8af6c63aa0170538717191f44a
Author: Shinya Maeda <shinya@gitlab.com>
Date: Thu Jul 5 17:37:40 2018 +0900
Fix flaky spec
commit d082dab7cb456f9b31e76f49d1db46b606ba13ce
Merge: 8d6059d08ff 116955c4533
Author: Shinya Maeda <shinya@gitlab.com>
Date: Thu Jul 5 17:35:06 2018 +0900
Merge branch 'master' into add-background-migration-for-legacy-traces
commit 8d6059d08fffaa99689afe32b66d93b538fb62f2
Author: Shinya Maeda <shinya@gitlab.com>
Date: Wed Jul 4 21:39:09 2018 +0900
Fix static analysys failure
commit c4bdb18d582c520723a73c4c90c522332fa94d36
Author: Shinya Maeda <shinya@gitlab.com>
Date: Wed Jul 4 20:16:16 2018 +0900
Cleanup spec
commit 469e826429feb6a8ce66eb5d5e087f7182eb8813
Author: Shinya Maeda <shinya@gitlab.com>
Date: Wed Jul 4 19:09:12 2018 +0900
Clean up migrate_legacy_artifacts spec
commit dee84be650ad5afc9f18ac05eccb0736e4b1c121
Author: Shinya Maeda <shinya@gitlab.com>
Date: Wed Jul 4 18:13:19 2018 +0900
Set default value at the database level
commit b32cb2f8a7e03238cf97d90f9e05ca11b5c8b687
Merge: 83f7c970ad8 4c1a2a9b997
Author: Shinya Maeda <shinya@gitlab.com>
Date: Wed Jul 4 17:06:02 2018 +0900
Merge branch 'master' into add-background-migration-for-legacy-traces
commit 83f7c970ad821f1b52999398070facf59f320479
Merge: 823629c1d9a 275fbf24b18
Author: Shinya Maeda <shinya@gitlab.com>
Date: Tue Jul 3 18:39:36 2018 +0900
Merge branch 'master' into add-background-migration-for-legacy-traces
commit 823629c1d9aac5e7dcfb6bc393557a72ca3370b4
Author: Shinya Maeda <shinya@gitlab.com>
Date: Mon Jul 2 14:59:54 2018 +0900
Decouple background migrations of filling nullified file_locations
commit 42ec60e8a74478c62962f5c66abfbe3e908d9e49
Author: Shinya Maeda <shinya@gitlab.com>
Date: Mon Jul 2 14:46:38 2018 +0900
Unify partial index migrations
commit c145e9306419f704b14e2a92f51a9cf2d105f7e7
Author: Shinya Maeda <shinya@gitlab.com>
Date: Mon Jul 2 14:24:14 2018 +0900
Make file_location non-null values
commit a442456f6a83327b2736f49b9522084b5675d129
Merge: c2cafa1051c 3cf683629e6
Author: Shinya Maeda <shinya@gitlab.com>
Date: Mon Jul 2 13:16:29 2018 +0900
Merge branch 'master' into add-background-migration-for-legacy-traces
commit c2cafa1051cb7a4582c93b653e0ef3506ee11af4
Merge: 8e5f4f88410 e38db196598
Author: Shinya Maeda <shinya@gitlab.com>
Date: Thu Jun 28 17:13:48 2018 +0900
Merge branch 'master' into add-background-migration-for-legacy-traces
commit 8e5f4f884107c0f574545f043fbacea2698ef3a4
Author: Shinya Maeda <shinya@gitlab.com>
Date: Wed Jun 27 18:54:03 2018 +0900
Remove unneccesary specs
commit 86c1c68b1000770fcf8086fa71801b332fb6df5a
Author: Shinya Maeda <shinya@gitlab.com>
Date: Wed Jun 27 18:05:36 2018 +0900
Add a partial index on ci_builds.id for legacy artifacts search
commit 70aa08c89bb6cd9b12d38b64cbf1838fb919c30f
Merge: 37801122986 292cf668905
Author: Shinya Maeda <shinya@gitlab.com>
Date: Wed Jun 27 17:15:33 2018 +0900
Merge branch 'master' into add-background-migration-for-legacy-traces
commit 378011229864c1e056cf995444f947f6b352172c
Author: Shinya Maeda <shinya@gitlab.com>
Date: Thu Jun 7 14:18:00 2018 +0900
Clean up migration code. Defining migration custom class in only post migration file which requires it for each_batch
commit 5cfe73318b91f377897e9fc50d67145093846459
Author: Shinya Maeda <shinya@gitlab.com>
Date: Wed Jun 6 23:57:03 2018 +0900
Remove indexing for mysql. with_legacy_artifacts targets all archive rows. Enhance tests.
commit 696c030d4a1ae2e1ee5e0eac771e574191fcb477
Author: Shinya Maeda <shinya@gitlab.com>
Date: Wed Jun 6 19:59:35 2018 +0900
Fix static analysys. Split methods. Use `file_types`
commit 86217c650c99d86d5b329229799f250f2ac5fbf6
Author: Shinya Maeda <shinya@gitlab.com>
Date: Wed Jun 6 18:55:15 2018 +0900
Use raw sql for better readability
commit df449404cb167edd61af1e59ca24f25db0a9ca9f
Author: Shinya Maeda <shinya@gitlab.com>
Date: Wed Jun 6 17:47:39 2018 +0900
Revert unneccesary change
commit 764977a1d98b34e34fcd229fca72c9595d62861a
Author: Shinya Maeda <shinya@gitlab.com>
Date: Wed Jun 6 16:09:14 2018 +0900
Remove AddIndexToCiBuildsArtifactsFile. Add temporary index in background migration class.
commit 7728ab3f9dba0af21033e1ca3b0213cacaef85d8
Author: Shinya Maeda <shinya@gitlab.com>
Date: Wed Jun 6 15:28:50 2018 +0900
Try to explicitly separate with postgresql and mysql for addin index
commit b95b47b965413f6590bd7eb38e34b4ef88cd2002
Author: Shinya Maeda <shinya@gitlab.com>
Date: Wed Jun 6 14:29:18 2018 +0900
Dry up code by referring `BackgroundMigration` namespace. Use `BETWEEN` raw SQL to make the edge case explicit.
commit 5b404e46e2a7c0d5a49b9745a8d985a80c9e380a
Author: Shinya Maeda <shinya@gitlab.com>
Date: Wed Jun 6 14:15:37 2018 +0900
Use `store_path` to make spec more explicit
commit 9f80de8e5aed49aae148278b0d27273e92ddbf25
Author: Shinya Maeda <shinya@gitlab.com>
Date: Wed Jun 6 14:03:26 2018 +0900
Use length not limit
commit f78503f3e55cc08d78274767b9381631201262e0
Author: Shinya Maeda <shinya@gitlab.com>
Date: Wed Jun 6 13:48:51 2018 +0900
Add length back for indexing
commit b96a57d5e5c7d2a97a1926ec95d67f0a2a1b8c93
Author: Shinya Maeda <shinya@gitlab.com>
Date: Tue Jun 5 12:42:14 2018 +0000
Update schema.rb
commit 9c25acc9752e5cf87f4062dc9f33fb609e8fb7be
Author: Shinya Maeda <shinya@gitlab.com>
Date: Tue Jun 5 12:40:48 2018 +0000
Add the guard clause in #down method too
commit 2bd5dadbd1b742d15bfb0914ad1be7e2182c1755
Author: Shinya Maeda <shinya@gitlab.com>
Date: Tue Jun 5 19:41:08 2018 +0900
Add a gurad caluse to block adding concurrent index for mysql
commit 9a5e12768ed30f75dd796d7c70f54b5e78aaa746
Author: Shinya Maeda <shinya@gitlab.com>
Date: Tue Jun 5 19:20:33 2018 +0900
Specify length of index in schema.rb manually for resolving mysql errors
commit c0ec68af02706d3c01b20b9f589113a900ff2071
Author: Shinya Maeda <shinya@gitlab.com>
Date: Tue Jun 5 17:51:40 2018 +0900
Separate add_concurrent_index by database type
commit b5f4ed37d9f49719fd8235a7069bb93f3c73c2dc
Author: Shinya Maeda <shinya@gitlab.com>
Date: Tue Jun 5 17:30:19 2018 +0900
Fix length size to use TEXT
commit b38b56056f4de384e32ee9acfbf161397de079fd
Author: Shinya Maeda <shinya@gitlab.com>
Date: Tue Jun 5 17:20:35 2018 +0900
Fix schema version for background migration spec
commit 32c48ef7cba61ceb285c6c7c1f48964a204dff20
Author: Shinya Maeda <shinya@gitlab.com>
Date: Tue Jun 5 16:22:00 2018 +0900
Specify length of indexed column. (This is neccessary for TEXT type)
commit a2e975f2e610ffd5f7f400b29a77d6b337005f1d
Author: Shinya Maeda <shinya@gitlab.com>
Date: Tue Jun 5 15:13:42 2018 +0900
Fix spec; migrate_legacy_artifacts_to_job_artifacts_spec.rb
commit c78c1e3f127b947e46a007bec2d56e0f3a55d7fa
Author: Shinya Maeda <shinya@gitlab.com>
Date: Mon Jun 4 20:15:23 2018 +0900
Separate add_concurrent_index migration from background migration
commit ad55e33bc2cd622b19f78ace64a69c18870d62df
Author: Shinya Maeda <shinya@gitlab.com>
Date: Mon Jun 4 19:59:14 2018 +0900
Revert "Remove index_exists? guard"
This reverts commit d76807838cf06e023b608c16426174a752a8ccf5.
commit b2693be76fb2d643058f7f0fffd0d75ef13c38f0
Author: Shinya Maeda <shinya@gitlab.com>
Date: Mon Jun 4 17:11:47 2018 +0900
Remove index_exists? guard
commit a2eb053d3b3bd8c8ad49958d58b7225658214a72
Author: Shinya Maeda <shinya@gitlab.com>
Date: Mon Jun 4 15:41:54 2018 +0900
Wrap insertion and deletion in a single transaction
commit d4bb2c709ce549eb5d7382c03390bfdf6ec90297
Author: Shinya Maeda <shinya@gitlab.com>
Date: Mon Jun 4 15:08:17 2018 +0900
Add temporary index to ci_builds.artifacts_file to speed up SELECT query
commit 33d62be9ee72a5ee4a61a7bc793df84ea7e85730
Author: Shinya Maeda <shinya@gitlab.com>
Date: Mon May 28 17:31:38 2018 +0900
Add test for legacy path proxy
commit 57ab71a05b464dc4f477837f237eb15e8e51bb84
Author: Shinya Maeda <shinya@gitlab.com>
Date: Mon May 28 15:59:36 2018 +0900
Optimize queries. Add some tests for filtering logic.
commit 8f24dfaac70bc564973bd81745b80421f1a3e1af
Author: Shinya Maeda <shinya@gitlab.com>
Date: Mon May 28 15:30:38 2018 +0900
Fix static analysis
commit ccc60ddd6076f997f63f46cad814e4ac9e4cbdfc
Author: Shinya Maeda <shinya@gitlab.com>
Date: Thu May 24 16:51:59 2018 +0900
Add changelog
commit fdbdb07a40a8343e8292e618f90658e77cc2115e
Author: Shinya Maeda <shinya@gitlab.com>
Date: Thu May 24 16:50:55 2018 +0900
Add spec for migrations
commit 04886af8724258f9971798adcf3eacd5716f20df
Author: Shinya Maeda <shinya@gitlab.com>
Date: Thu May 24 14:48:31 2018 +0900
Revise comments
commit 55bcbee0576b5ac08b4ed461b85d059e0c670159
Author: Shinya Maeda <shinya@gitlab.com>
Date: Thu May 24 14:40:29 2018 +0900
Revise comments
commit db6628675abfe24cc10e682a65bf87ca02504deb
Author: Shinya Maeda <shinya@gitlab.com>
Date: Tue May 15 17:27:12 2018 +0900
Add spec base (Not finished)
commit e92dcc392eb4ad5aac06a3510a5879a3f084b9ee
Author: Shinya Maeda <shinya@gitlab.com>
Date: Tue May 15 17:26:49 2018 +0900
Added artifacts_metadata_store to remove in background migration
commit 91388399d80b58a831b21e6582d5f5ab29e785c7
Author: Shinya Maeda <shinya@gitlab.com>
Date: Tue May 15 17:26:20 2018 +0900
Add a condition to exclude jobs which have job_artifacts already
commit e084ce1181d07d0799714eee058d6c9182bc2fcf
Author: Shinya Maeda <shinya@gitlab.com>
Date: Tue May 15 16:51:15 2018 +0900
Clean up migrate_legacy_artifacts.rb (Fix static analysys)
commit ca4e5d33e27690dc276ba87d08dc24bcf31267f4
Author: Shinya Maeda <shinya@gitlab.com>
Date: Tue May 15 16:32:59 2018 +0900
Remove the culcuration method of metadata size
commit eac4c75be932ae6936702ddfd9202f333f18053f
Author: Shinya Maeda <shinya@gitlab.com>
Date: Tue May 15 16:11:16 2018 +0900
Remove Gitlab::BackgroundMigration:: scope
commit 4a5ca96b628a59eeba25778569e719695ac8b5e8
Author: Shinya Maeda <shinya@gitlab.com>
Date: Tue May 15 16:09:09 2018 +0900
Add limit to file_location column
commit 49ff3d9c98dc145fe3c91cd8ba39d7d71d183fb7
Author: Shinya Maeda <shinya@gitlab.com>
Date: Tue May 15 16:05:22 2018 +0900
Use size.to_i instead of if-condition
commit 526656e6ee5683ec7ea07bd6af3438c3fc32375d
Author: Shinya Maeda <shinya@gitlab.com>
Date: Tue May 15 16:02:57 2018 +0900
Rename current_path to hashed_path
commit 8e6faca7900cf382a35323b2ed0fabede9bf3bf9
Author: Shinya Maeda <shinya@gitlab.com>
Date: Tue May 8 16:53:43 2018 +0900
Rename location_1/2 to current/legacy_path
commit 5034543255963a250e076f37c5e42fbf4cb0fd05
Author: Shinya Maeda <shinya@gitlab.com>
Date: Tue May 1 21:44:10 2018 +0900
Rename path_type to file_location. Fix JobArtifactUploader to look up correct file location.
commit 4f7e3503f13d02d7a70e7b68bf740a8d8bb525db
Author: Shinya Maeda <shinya@gitlab.com>
Date: Fri Apr 27 17:33:15 2018 +0900
Add background migration for legacy artifacts
Add default_value_for to file_location column
Set file_location explicityly with default_values_for
Add file_location to factory
Remove tests of default_values_for
Set file_location nil as default
|
| |/ |
|
| | |
|
| |\
| |
| |
| |
| |
| |
| |
| |
| | |
'48745-project-exports-fail-when-uploads-have-been-migrated-to-object-storage' into 'master'
Resolve "Project exports fail when uploads have been migrated to object storage"
Closes #48745
See merge request gitlab-org/gitlab-ce!20484
|
| | | |
|
| | | |
|
| |/
|
|
| |
Previously we would pull the file, now, we just stream-it as needed from Object Storage
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
See https://gitlab.com/gitlab-org/gitlab-ce/issues/47677 for more information
|
| |\
| |
| |
| |
| |
| |
| |
| |
| | |
'47513-upload-migration-lease-key-is-incorrect-for-non-mounted-uploaders' into 'master'
Resolve "Upload migration lease key is incorrect for non-mounted uploaders"
Closes #47513
See merge request gitlab-org/gitlab-ce!19600
|
| | |
| |
| |
| |
| |
| |
| | |
For FileUploaders it's possible that a model has many uploads
and if lease key is created only from model id, it causes that
the model's uploads can not be migrated in parallel because the
exclusive lease key would be same for all uploads of the model.
|
| |\ \
| |/
|/|
| |
| |
| |
| |
| |
| | |
into 'master'
Resolve "`MigrateUploadsWorker` is doing N+1 queries on migration"
Closes #47408
See merge request gitlab-org/gitlab-ce!19547
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | |
| |
| |
| | |
Signed-off-by: Rémy Coutable <remy@rymai.me>
|
| |\ \
| | |
| | |
| | |
| | | |
Update rspec to 3.7
See merge request gitlab-org/gitlab-ce!19449
|
| | | | |
|
