diff options
Diffstat (limited to 'doc/administration/raketasks/uploads/migrate.md')
| -rw-r--r-- | doc/administration/raketasks/uploads/migrate.md | 108 |
1 files changed, 60 insertions, 48 deletions
diff --git a/doc/administration/raketasks/uploads/migrate.md b/doc/administration/raketasks/uploads/migrate.md index 851305d433f..d58b802b024 100644 --- a/doc/administration/raketasks/uploads/migrate.md +++ b/doc/administration/raketasks/uploads/migrate.md @@ -1,21 +1,26 @@ -# Uploads Migrate Rake Tasks +# Uploads migrate Rake tasks **(CORE ONLY)** -## Migrate to Object Storage +`gitlab:uploads:migrate` migrates uploads between different storage types. -After [configuring the object storage](../../uploads.md#using-object-storage-core-only) for GitLab's uploads, you may use this task to migrate existing uploads from the local storage to the remote storage. +## Migrate to object storage ->**Note:** -All of the processing will be done in a background worker and requires **no downtime**. +After [configuring the object storage](../../uploads.md#using-object-storage-core-only) for GitLab's +uploads, use this task to migrate existing uploads from the local storage to the remote storage. + +Read more about using [object storage with GitLab](../../object_storage.md). -[Read more about using object storage with GitLab](../../object_storage.md). +NOTE: **Note:** +All of the processing will be done in a background worker and requires **no downtime**. ### All-in-one Rake task -GitLab provides a wrapper Rake task that migrates all uploaded files - avatars, -logos, attachments, favicon, etc. - to object storage in one go. Under the hood, -it invokes individual Rake tasks to migrate files falling under each of this -category one by one. The specifications of these individual Rake tasks are -described in the next section. +GitLab provides a wrapper Rake task that migrates all uploaded files (for example avatars, logos, +attachments, and favicon) to object storage in one step. The wrapper task invokes individual Rake +tasks to migrate files falling under each of these categories one by one. + +These [individual Rake tasks](#individual-rake-tasks) are described in the next section. + +To migrate all uploads from local storage to object storage, run: **Omnibus Installation** @@ -31,26 +36,29 @@ sudo RAILS_ENV=production -u git -H bundle exec rake gitlab:uploads:migrate:all ### Individual Rake tasks ->**Note:** -If you already ran the Rake task mentioned above, no need to run these individual Rake tasks as that has been done automatically. +If you already ran the [all-in-one Rake task](#all-in-one-rake-task), there is no need to run these +individual tasks. + +The Rake task uses three parameters to find uploads to migrate: -The Rake task uses 3 parameters to find uploads to migrate. +| Parameter | Type | Description | +|:-----------------|:--------------|:-------------------------------------------------------| +| `uploader_class` | string | Type of the uploader to migrate from. | +| `model_class` | string | Type of the model to migrate from. | +| `mount_point` | string/symbol | Name of the model's column the uploader is mounted on. | -Parameter | Type | Description ---------- | ---- | ----------- -`uploader_class` | string | Type of the uploader to migrate from -`model_class` | string | Type of the model to migrate from -`mount_point` | string/symbol | Name of the model's column on which the uploader is mounted on. +NOTE: **Note:** +These parameters are mainly internal to GitLab's structure, you may want to refer to the task list +instead below. ->**Note:** -These parameters are mainly internal to GitLab's structure, you may want to refer to the task list instead below. +This task also accepts an environment variable which you can use to override +the default batch size: -This task also accepts some environment variables which you can use to override -certain values: +| Variable | Type | Description | +|:---------|:--------|:--------------------------------------------------| +| `BATCH` | integer | Specifies the size of the batch. Defaults to 200. | -Variable | Type | Description --------- | ---- | ----------- -`BATCH` | integer | Specifies the size of the batch. Defaults to 200. +The following shows how to run `gitlab:uploads:migrate` for individual types of uploads. **Omnibus Installation** @@ -76,13 +84,13 @@ gitlab-rake "gitlab:uploads:migrate[PersonalFileUploader, Snippet]" gitlab-rake "gitlab:uploads:migrate[NamespaceFileUploader, Snippet]" gitlab-rake "gitlab:uploads:migrate[FileUploader, MergeRequest]" -# Design Management design thumbnails (EE) +# Design Management design thumbnails gitlab-rake "gitlab:uploads:migrate[DesignManagement::DesignV432x230Uploader, DesignManagement::Action, :image_v432x230]" ``` **Source Installation** ->**Note:** +NOTE: **Note:** Use `RAILS_ENV=production` for every task. ```shell @@ -107,16 +115,14 @@ sudo -u git -H bundle exec rake "gitlab:uploads:migrate[PersonalFileUploader, Sn sudo -u git -H bundle exec rake "gitlab:uploads:migrate[NamespaceFileUploader, Snippet]" sudo -u git -H bundle exec rake "gitlab:uploads:migrate[FileUploader, MergeRequest]" -# Design Management design thumbnails (EE) +# Design Management design thumbnails sudo -u git -H bundle exec rake "gitlab:uploads:migrate[DesignManagement::DesignV432x230Uploader, DesignManagement::Action]" ``` -## Migrate from object storage to local storage +## Migrate to local storage -If you need to disable Object Storage for any reason, first you need to migrate -your data out of Object Storage and back into your local storage. - -**Before proceeding, it is important to disable both `direct_upload` and `background_upload` under `uploads` settings in `gitlab.rb`** +If you need to disable [object storage](../../object_storage.md) for any reason, you must first +migrate your data out of object storage and back into your local storage. CAUTION: **Warning:** **Extended downtime is required** so no new files are created in object storage during @@ -126,23 +132,29 @@ To follow progress, see the [relevant issue](https://gitlab.com/gitlab-org/gitla ### All-in-one Rake task -GitLab provides a wrapper Rake task that migrates all uploaded files - avatars, -logos, attachments, favicon, etc. - to local storage in one go. Under the hood, -it invokes individual Rake tasks to migrate files falling under each of this -category one by one. For details on these Rake tasks please [refer to the section above](#individual-rake-tasks), +GitLab provides a wrapper Rake task that migrates all uploaded files (for example, avatars, logos, +attachments, and favicon) to local storage in one step. The wrapper task invokes individual Rake +tasks to migrate files falling under each of these categories one by one. + +For details on these Rake tasks, refer to [Individual Rake tasks](#individual-rake-tasks), keeping in mind the task name in this case is `gitlab:uploads:migrate_to_local`. -**Omnibus Installation** +To migrate uploads from object storage to local storage: -```shell -gitlab-rake "gitlab:uploads:migrate_to_local:all" -``` +1. Disable both `direct_upload` and `background_upload` under `uploads` settings in `gitlab.rb`. +1. Run the Rake task: -**Source Installation** + **Omnibus Installation** -```shell -sudo RAILS_ENV=production -u git -H bundle exec rake gitlab:uploads:migrate_to_local:all -``` + ```shell + gitlab-rake "gitlab:uploads:migrate_to_local:all" + ``` + + **Source Installation** + + ```shell + sudo RAILS_ENV=production -u git -H bundle exec rake gitlab:uploads:migrate_to_local:all + ``` -After this is done, you may disable Object Storage by undoing the changes described -in the instructions to [configure object storage](../../uploads.md#using-object-storage-core-only) +After running the Rake task, you can disable object storage by undoing the changes described +in the instructions to [configure object storage](../../uploads.md#using-object-storage-core-only). |