1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
|
# Uploads Migrate Rake Tasks
## Migrate to Object Storage
After [configuring the object storage](../../uploads.md#using-object-storage-core-only) for GitLab's uploads, you may use this task to migrate existing uploads from the local storage to the remote storage.
>**Note:**
All of the processing will be done in a background worker and requires **no downtime**.
[Read more about using object storage with GitLab](../../object_storage.md).
### All-in-one Rake task
GitLab provides a wrapper Rake task that migrates all uploaded files - avatars,
logos, attachments, favicon, etc. - to object storage in one go. Under the hood,
it invokes individual Rake tasks to migrate files falling under each of this
category one by one. The specifications of these individual Rake tasks are
described in the next section.
**Omnibus Installation**
```shell
gitlab-rake "gitlab:uploads:migrate:all"
```
**Source Installation**
```shell
sudo RAILS_ENV=production -u git -H bundle exec rake gitlab:uploads:migrate:all
```
### Individual Rake tasks
>**Note:**
If you already ran the Rake task mentioned above, no need to run these individual Rake tasks as that has been done automatically.
The Rake task uses 3 parameters to find uploads to migrate.
Parameter | Type | Description
--------- | ---- | -----------
`uploader_class` | string | Type of the uploader to migrate from
`model_class` | string | Type of the model to migrate from
`mount_point` | string/symbol | Name of the model's column on which the uploader is mounted on.
>**Note:**
These parameters are mainly internal to GitLab's structure, you may want to refer to the task list instead below.
This task also accepts some environment variables which you can use to override
certain values:
Variable | Type | Description
-------- | ---- | -----------
`BATCH` | integer | Specifies the size of the batch. Defaults to 200.
**Omnibus Installation**
```shell
# gitlab-rake gitlab:uploads:migrate[uploader_class, model_class, mount_point]
# Avatars
gitlab-rake "gitlab:uploads:migrate[AvatarUploader, Project, :avatar]"
gitlab-rake "gitlab:uploads:migrate[AvatarUploader, Group, :avatar]"
gitlab-rake "gitlab:uploads:migrate[AvatarUploader, User, :avatar]"
# Attachments
gitlab-rake "gitlab:uploads:migrate[AttachmentUploader, Note, :attachment]"
gitlab-rake "gitlab:uploads:migrate[AttachmentUploader, Appearance, :logo]"
gitlab-rake "gitlab:uploads:migrate[AttachmentUploader, Appearance, :header_logo]"
# Favicon
gitlab-rake "gitlab:uploads:migrate[FaviconUploader, Appearance, :favicon]"
# Markdown
gitlab-rake "gitlab:uploads:migrate[FileUploader, Project]"
gitlab-rake "gitlab:uploads:migrate[PersonalFileUploader, Snippet]"
gitlab-rake "gitlab:uploads:migrate[NamespaceFileUploader, Snippet]"
gitlab-rake "gitlab:uploads:migrate[FileUploader, MergeRequest]"
# Design Management design thumbnails (EE)
gitlab-rake "gitlab:uploads:migrate[DesignManagement::DesignV432x230Uploader, DesignManagement::Action, :image_v432x230]"
```
**Source Installation**
>**Note:**
Use `RAILS_ENV=production` for every task.
```shell
# sudo -u git -H bundle exec rake gitlab:uploads:migrate
# Avatars
sudo -u git -H bundle exec rake "gitlab:uploads:migrate[AvatarUploader, Project, :avatar]"
sudo -u git -H bundle exec rake "gitlab:uploads:migrate[AvatarUploader, Group, :avatar]"
sudo -u git -H bundle exec rake "gitlab:uploads:migrate[AvatarUploader, User, :avatar]"
# Attachments
sudo -u git -H bundle exec rake "gitlab:uploads:migrate[AttachmentUploader, Note, :attachment]"
sudo -u git -H bundle exec rake "gitlab:uploads:migrate[AttachmentUploader, Appearance, :logo]"
sudo -u git -H bundle exec rake "gitlab:uploads:migrate[AttachmentUploader, Appearance, :header_logo]"
# Favicon
sudo -u git -H bundle exec rake "gitlab:uploads:migrate[FaviconUploader, Appearance, :favicon]"
# Markdown
sudo -u git -H bundle exec rake "gitlab:uploads:migrate[FileUploader, Project]"
sudo -u git -H bundle exec rake "gitlab:uploads:migrate[PersonalFileUploader, Snippet]"
sudo -u git -H bundle exec rake "gitlab:uploads:migrate[NamespaceFileUploader, Snippet]"
sudo -u git -H bundle exec rake "gitlab:uploads:migrate[FileUploader, MergeRequest]"
# Design Management design thumbnails (EE)
sudo -u git -H bundle exec rake "gitlab:uploads:migrate[DesignManagement::DesignV432x230Uploader, DesignManagement::Action]"
```
## Migrate legacy uploads out of deprecated paths
> Introduced in GitLab 12.3.
To migrate all uploads created by legacy uploaders, run:
**Omnibus Installation**
```shell
gitlab-rake gitlab:uploads:legacy:migrate
```
**Source Installation**
```shell
bundle exec rake gitlab:uploads:legacy:migrate
```
## Migrate from object storage to local storage
If you need to disable Object Storage for any reason, first you need to migrate
your data out of Object Storage and back into your local storage.
**Before proceeding, it is important to disable both `direct_upload` and `background_upload` under `uploads` settings in `gitlab.rb`**
CAUTION: **Warning:**
**Extended downtime is required** so no new files are created in object storage during
the migration. A configuration setting will be added soon to allow migrating
from object storage to local files with only a brief moment of downtime for configuration changes.
To follow progress, see the [relevant issue](https://gitlab.com/gitlab-org/gitlab/issues/30979).
### All-in-one Rake task
GitLab provides a wrapper Rake task that migrates all uploaded files - avatars,
logos, attachments, favicon, etc. - to local storage in one go. Under the hood,
it invokes individual Rake tasks to migrate files falling under each of this
category one by one. For details on these Rake tasks please [refer to the section above](#individual-rake-tasks),
keeping in mind the task name in this case is `gitlab:uploads:migrate_to_local`.
**Omnibus Installation**
```shell
gitlab-rake "gitlab:uploads:migrate_to_local:all"
```
**Source Installation**
```shell
sudo RAILS_ENV=production -u git -H bundle exec rake gitlab:uploads:migrate_to_local:all
```
After this is done, you may disable Object Storage by undoing the changes described
in the instructions to [configure object storage](../../uploads.md#using-object-storage-core-only)
|
