diff options
Diffstat (limited to 'doc/raketasks/backup_restore.md')
| -rw-r--r-- | doc/raketasks/backup_restore.md | 252 |
1 files changed, 140 insertions, 112 deletions
diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index 65fcfc77ab1..5be6053b76e 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -1,39 +1,52 @@ -# Backup restore +# Backing up and restoring GitLab  An application data backup creates an archive file that contains the database, all repositories and all attachments. -This archive will be saved in `backup_path`, which is specified in the -`config/gitlab.yml` file. -The filename will be `[TIMESTAMP]_gitlab_backup.tar`, where `TIMESTAMP` -identifies the time at which each backup was created. - -> In GitLab 8.15 we changed the timestamp format from `EPOCH` (`1393513186`) -> to `EPOCH_YYYY_MM_DD` (`1393513186_2014_02_27`) -You can only restore a backup to exactly the same version of GitLab on which it -was created. The best way to migrate your repositories from one server to +You can only restore a backup to **exactly the same version** of GitLab on which +it was created. The best way to migrate your repositories from one server to another is through backup restore. -To restore a backup, you will also need to restore `/etc/gitlab/gitlab-secrets.json` -(for omnibus packages) or `/home/git/gitlab/.secret` (for installations -from source). This file contains the database encryption key and CI secret -variables used for two-factor authentication. If you fail to restore this -encryption key file along with the application data backup, users with two-factor -authentication enabled will lose access to your GitLab server. +## Backup + +GitLab provides a simple command line interface to backup your whole installation, +and is flexible enough to fit your needs. + +### Backup timestamp + +>**Note:** +In GitLab 9.2 the timestamp format was changed from `EPOCH_YYYY_MM_DD` to +`EPOCH_YYYY_MM_DD_GitLab version`, for example `1493107454_2017_04_25` +would become `1493107454_2017_04_25_9.1.0`. + +The backup archive will be saved in `backup_path`, which is specified in the +`config/gitlab.yml` file. +The filename will be `[TIMESTAMP]_gitlab_backup.tar`, where `TIMESTAMP` +identifies the time at which each backup was created, plus the GitLab version. +The timestamp is needed if you need to restore GitLab and multiple backups are +available. + +For example, if the backup name is `1493107454_2017_04_25_9.1.0_gitlab_backup.tar`, +then the timestamp is `1493107454_2017_04_25_9.1.0`. -## Create a backup of the GitLab system +### Creating a backup of the GitLab system Use this command if you've installed GitLab with the Omnibus package: + ``` sudo gitlab-rake gitlab:backup:create ``` + Use this if you've installed GitLab from source: + ``` sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production ``` + If you are running GitLab within a Docker container, you can run the backup from the host: + ``` docker exec -t <container name> gitlab-rake gitlab:backup:create ``` @@ -67,9 +80,9 @@ Deleting tmp directories...[DONE] Deleting old backups... [SKIPPING] ``` -## Backup Strategy Option +### Backup strategy option -> **Note:** Introduced as an option in 8.17 +> **Note:** Introduced as an option in GitLab 8.17. The default backup strategy is to essentially stream data from the respective data locations to the backup using the Linux command `tar` and `gzip`. This works @@ -89,7 +102,7 @@ To use the `copy` strategy instead of the default streaming strategy, specify `STRATEGY=copy` in the Rake task command. For example, `sudo gitlab-rake gitlab:backup:create STRATEGY=copy`. -## Exclude specific directories from the backup +### Excluding specific directories from the backup You can choose what should be backed up by adding the environment variable `SKIP`. The available options are: @@ -113,7 +126,7 @@ sudo gitlab-rake gitlab:backup:create SKIP=db,uploads sudo -u git -H bundle exec rake gitlab:backup:create SKIP=db,uploads RAILS_ENV=production ``` -## Upload backups to remote (cloud) storage +### Uploading backups to a remote (cloud) storage Starting with GitLab 7.4 you can let the backup script upload the '.tar' file it creates. It uses the [Fog library](http://fog.io/) to perform the upload. @@ -257,7 +270,7 @@ For installations from source: remote_directory: 'gitlab_backups' ``` -## Backup archive permissions +### Backup archive permissions The backup archives created by GitLab (`1393513186_2014_02_27_gitlab_backup.tar`) will have owner/group git:git and 0600 permissions by default. @@ -275,11 +288,11 @@ gitlab_rails['backup_archive_permissions'] = 0644 # Makes the backup archives wo archive_permissions: 0644 # Makes the backup archives world-readable ``` -## Storing configuration files +### Storing configuration files Please be informed that a backup does not store your configuration -files. One reason for this is that your database contains encrypted -information for two-factor authentication. Storing encrypted +files. One reason for this is that your database contains encrypted +information for two-factor authentication. Storing encrypted information along with its key in the same place defeats the purpose of using encryption in the first place! @@ -292,11 +305,74 @@ At the very **minimum** you should backup `/etc/gitlab/gitlab.rb` and `/home/git/gitlab/config/secrets.yml` (source) to preserve your database encryption key. -## Restore a previously created backup +### Configuring cron to make daily backups -You can only restore a backup to exactly the same version of GitLab that you created it on, for example 7.2.1. +>**Note:** +The following cron jobs do not [backup your GitLab configuration files](#storing-configuration-files) +or [SSH host keys](https://superuser.com/questions/532040/copy-ssh-keys-from-one-server-to-another-server/532079#532079). -### Prerequisites +**For Omnibus installations** + +To schedule a cron job that backs up your repositories and GitLab metadata, use the root user: + +``` +sudo su - +crontab -e +``` + +There, add the following line to schedule the backup for everyday at 2 AM: + +``` +0 2 * * * /opt/gitlab/bin/gitlab-rake gitlab:backup:create CRON=1 +``` + +You may also want to set a limited lifetime for backups to prevent regular +backups using all your disk space. To do this add the following lines to +`/etc/gitlab/gitlab.rb` and reconfigure: + +``` +# limit backup lifetime to 7 days - 604800 seconds +gitlab_rails['backup_keep_time'] = 604800 +``` + +Note that the `backup_keep_time` configuration option only manages local +files. GitLab does not automatically prune old files stored in a third-party +object storage (e.g., AWS S3) because the user may not have permission to list +and delete files. We recommend that you configure the appropriate retention +policy for your object storage. For example, you can configure [the S3 backup +policy as described here](http://stackoverflow.com/questions/37553070/gitlab-omnibus-delete-backup-from-amazon-s3). + +**For installation from source** + +``` +cd /home/git/gitlab +sudo -u git -H editor config/gitlab.yml # Enable keep_time in the backup section to automatically delete old backups +sudo -u git crontab -e # Edit the crontab for the git user +``` + +Add the following lines at the bottom: + +``` +# Create a full backup of the GitLab repositories and SQL database every day at 4am +0 4 * * * cd /home/git/gitlab && PATH=/usr/local/bin:/usr/bin:/bin bundle exec rake gitlab:backup:create RAILS_ENV=production CRON=1 +``` + +The `CRON=1` environment setting tells the backup script to suppress all progress output if there are no errors. +This is recommended to reduce cron spam. + +## Restore + +GitLab provides a simple command line interface to backup your whole installation, +and is flexible enough to fit your needs. + +The [restore prerequisites section](#restore-prerequisites) includes crucial +information. Make sure to read and test the whole restore process at least once +before attempting to perform it in a production environment. + +You can only restore a backup to **exactly the same version** of GitLab that +you created it on, for example 9.1.0. + +### Restore prerequisites You need to have a working GitLab installation before you can perform a restore. This is mainly because the system user performing the @@ -305,13 +381,23 @@ the SQL database it needs to import data into ('gitlabhq_production'). All existing data will be either erased (SQL) or moved to a separate directory (repositories, uploads). -If some or all of your GitLab users are using two-factor authentication (2FA) -then you must also make sure to restore `/etc/gitlab/gitlab.rb` and -`/etc/gitlab/gitlab-secrets.json` (Omnibus), or -`/home/git/gitlab/config/secrets.yml` (installations from source). Note that you -need to run `gitlab-ctl reconfigure` after changing `gitlab-secrets.json`. +To restore a backup, you will also need to restore `/etc/gitlab/gitlab-secrets.json` +(for Omnibus packages) or `/home/git/gitlab/.secret` (for installations +from source). This file contains the database encryption key, +[CI secret variables](../ci/variables/README.md#secret-variables), and +secret variables used for [two-factor authentication](../user/profile/account/two_factor_authentication.md). +If you fail to restore this encryption key file along with the application data +backup, users with two-factor authentication enabled and GitLab Runners will +lose access to your GitLab server. + +Depending on your case, you might want to run the restore command with one or +more of the following options: + +- `BACKUP=timestamp_of_backup` - Required if more than one backup exists. + Read what the [backup timestamp is about](#backup-timestamp). +- `force=yes` - Do not ask if the authorized_keys file should get regenerated. -### Installation from source +### Restore for installation from source ``` # Stop processes that are connected to the database @@ -320,13 +406,6 @@ sudo service gitlab stop bundle exec rake gitlab:backup:restore RAILS_ENV=production ``` -Options: - -``` -BACKUP=timestamp_of_backup (required if more than one backup exists) -force=yes (do not ask if the authorized_keys file should get regenerated) -``` - Example output: ``` @@ -358,13 +437,13 @@ Restoring repositories: Deleting tmp directories...[DONE] ``` -### Omnibus installations +### Restore for Omnibus installations This procedure assumes that: -- You have installed the exact same version of GitLab Omnibus with which the - backup was created -- You have run `sudo gitlab-ctl reconfigure` at least once +- You have installed the **exact same version** of GitLab Omnibus with which the + backup was created. +- You have run `sudo gitlab-ctl reconfigure` at least once. - GitLab is running. If not, start it using `sudo gitlab-ctl start`. First make sure your backup tar file is in the backup directory described in the @@ -372,7 +451,7 @@ First make sure your backup tar file is in the backup directory described in the `/var/opt/gitlab/backups`. ```shell -sudo cp 1393513186_2014_02_27_gitlab_backup.tar /var/opt/gitlab/backups/ +sudo cp 1493107454_2017_04_25_9.1.0_gitlab_backup.tar /var/opt/gitlab/backups/ ``` Stop the processes that are connected to the database. Leave the rest of GitLab @@ -390,7 +469,7 @@ restore: ```shell # This command will overwrite the contents of your GitLab database! -sudo gitlab-rake gitlab:backup:restore BACKUP=1393513186_2014_02_27 +sudo gitlab-rake gitlab:backup:restore BACKUP=1493107454_2017_04_25_9.1.0 ``` Restart and check GitLab: @@ -402,59 +481,7 @@ sudo gitlab-rake gitlab:check SANITIZE=true If there is a GitLab version mismatch between your backup tar file and the installed version of GitLab, the restore command will abort with an error. Install the -[correct GitLab version](https://about.gitlab.com/downloads/archives/) and try again. - -## Configure cron to make daily backups - -### For installation from source: -``` -cd /home/git/gitlab -sudo -u git -H editor config/gitlab.yml # Enable keep_time in the backup section to automatically delete old backups -sudo -u git crontab -e # Edit the crontab for the git user -``` - -Add the following lines at the bottom: - -``` -# Create a full backup of the GitLab repositories and SQL database every day at 4am -0 4 * * * cd /home/git/gitlab && PATH=/usr/local/bin:/usr/bin:/bin bundle exec rake gitlab:backup:create RAILS_ENV=production CRON=1 -``` - -The `CRON=1` environment setting tells the backup script to suppress all progress output if there are no errors. -This is recommended to reduce cron spam. - -### For omnibus installations - -To schedule a cron job that backs up your repositories and GitLab metadata, use the root user: - -``` -sudo su - -crontab -e -``` - -There, add the following line to schedule the backup for everyday at 2 AM: - -``` -0 2 * * * /opt/gitlab/bin/gitlab-rake gitlab:backup:create CRON=1 -``` - -You may also want to set a limited lifetime for backups to prevent regular -backups using all your disk space. To do this add the following lines to -`/etc/gitlab/gitlab.rb` and reconfigure: - -``` -# limit backup lifetime to 7 days - 604800 seconds -gitlab_rails['backup_keep_time'] = 604800 -``` - -Note that the `backup_keep_time` configuration option only manages local -files. GitLab does not automatically prune old files stored in a third-party -object storage (e.g. AWS S3) because the user may not have permission to list -and delete files. We recommend that you configure the appropriate retention -policy for your object storage. For example, you can configure [the S3 backup -policy here as described here](http://stackoverflow.com/questions/37553070/gitlab-omnibus-delete-backup-from-amazon-s3). - -NOTE: This cron job does not [backup your omnibus-gitlab configuration](#backup-and-restore-omnibus-gitlab-configuration) or [SSH host keys](https://superuser.com/questions/532040/copy-ssh-keys-from-one-server-to-another-server/532079#532079). +[correct GitLab version](https://packages.gitlab.com/gitlab/) and try again. ## Alternative backup strategies @@ -479,6 +506,19 @@ Example: LVM snapshots + rsync If you are running GitLab on a virtualized server you can possibly also create VM snapshots of the entire GitLab server. It is not uncommon however for a VM snapshot to require you to power down the server, so this approach is probably of limited practical use. +## Additional notes + +This documentation is for GitLab Community and Enterprise Edition. We backup +GitLab.com and make sure your data is secure, but you can't use these methods +to export / backup your data yourself from GitLab.com. + +Issues are stored in the database. They can't be stored in Git itself. + +To migrate your repositories from one server to another with an up-to-date version of +GitLab, you can use the [import rake task](import.md) to do a mass import of the +repository. Note that if you do an import rake task, rather than a backup restore, you +will have all your repositories, but not any other data. + ## Troubleshooting ### Restoring database backup using omnibus packages outputs warnings @@ -488,7 +528,6 @@ If you are using backup restore procedures you might encounter the following war psql:/var/opt/gitlab/backups/db/database.sql:22: ERROR: must be owner of extension plpgsql psql:/var/opt/gitlab/backups/db/database.sql:2931: WARNING: no privileges could be revoked for "public" (two occurrences) psql:/var/opt/gitlab/backups/db/database.sql:2933: WARNING: no privileges were granted for "public" (two occurrences) - ``` Be advised that, backup is successfully restored in spite of these warnings. @@ -497,14 +536,3 @@ The rake task runs this as the `gitlab` user which does not have the superuser a Those objects have no influence on the database backup/restore but they give this annoying warning. For more information see similar questions on postgresql issue tracker[here](http://www.postgresql.org/message-id/201110220712.30886.adrian.klaver@gmail.com) and [here](http://www.postgresql.org/message-id/2039.1177339749@sss.pgh.pa.us) as well as [stack overflow](http://stackoverflow.com/questions/4368789/error-must-be-owner-of-language-plpgsql). - -## Note -This documentation is for GitLab CE. -We backup GitLab.com and make sure your data is secure, but you can't use these methods to export / backup your data yourself from GitLab.com. - -Issues are stored in the database. They can't be stored in Git itself. - -To migrate your repositories from one server to another with an up-to-date version of -GitLab, you can use the [import rake task](import.md) to do a mass import of the -repository. Note that if you do an import rake task, rather than a backup restore, you -will have all your repositories, but not any other data. |