summaryrefslogtreecommitdiff
path: root/lib/backup
Commit message (Collapse)AuthorAgeFilesLines
* Fix backup restorefix-backup-restoreStan Hu2016-07-251-2/+2
| | | | | | Set permissions of backup dir to g+s Closes #20188
* Refactor repository paths handling to allow multiple git mount pointsAlejandro Rodríguez2016-06-291-12/+14
|
* Only create the backup directory if it is localStan Hu2016-06-101-3/+18
| | | | Closes #12710
* Replace colorize gem with rainbow.Connor Shea2016-06-033-30/+30
| | | | | | | | | | | | | Colorize is a gem licensed under the GPLv2, so we can’t use it in GitLab without relicensing GitLab under the terms of the GPL. Rainbow is licensed under the MIT license and does the exact same thing as Colorize, so Rainbow was added in place of Colorize. The syntax is slightly different for Rainbow vs. Colorize, and was updated in accordance. The gem is still a dependency of Spinach, so it’s included in the development/test environments, but won’t be packaged with the actual product, and therefore doesn’t require we relicense the product. An attempt at relicensing Colorize was made, but didn’t succeed as the library owner never responded. Rainbow library: https://github.com/sickill/rainbow Relevant issue regarding licensing in GitLab's gems: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3775
* Create the specified remote directory during backuprs-create-remote-backup-folderRobert Speicher2016-05-311-1/+1
| | | | | | | This is idempotent, so there's no harm calling it if the directory already exists. Closes #12710
* Improve after reviewfix-registry-backupsKamil Trzcinski2016-05-241-5/+5
|
* Fix backups if registry is disabledKamil Trzcinski2016-05-231-2/+11
|
* Added backup of container registryKamil Trzcinski2016-05-162-1/+14
|
* Make sure there is a connection before using ActiveRecordbackup-database-timeout-fixJeroen Nijhof2016-02-081-0/+3
|
* Backup LFS objects same as any upload.Marin Jankovski2015-11-192-1/+14
|
* USe reject.check_if_it_should_be_archived_in_backupMarin Jankovski2015-11-181-1/+1
|
* Check which folders and archives should be packed before passing to tar command.Marin Jankovski2015-11-181-8/+6
|
* Implement Build ArtifactsKamil Trzcinski2015-11-102-1/+14
| | | | | | | - Offloads uploading to GitLab Workhorse - Use /authorize request for fast uploading - Added backup recipes for artifacts - Support download acceleration using X-Sendfile
* Replace all usages of `git` command with configurable binary pathrs-git-bin-pathRobert Speicher2015-11-031-3/+3
| | | | Closes #3311
* Explicitly require backup/filesrs-require-backup-filesRobert Speicher2015-11-022-0/+4
|
* Do no rely on basename of builds, uploadsbackup-improvementsJacob Vosmaer2015-10-163-6/+15
|
* Remove superfluous mkdir -pJacob Vosmaer2015-10-061-2/+0
|
* Remove unused variableJacob Vosmaer2015-10-061-3/+2
|
* Keep old path: db/database.sql.gzJacob Vosmaer2015-10-062-3/+4
| | | | | Documentation elsewhere refers to this internal path, let's keep it.
* Remove old "files" tarball explicitlyJacob Vosmaer2015-10-061-0/+1
|
* Reduce disk IO during SQL backupJacob Vosmaer2015-10-062-31/+26
| | | | | By using light gzip compression we can save a lot of disk IO during the backup.
* Use tar for intermediate backup storageJacob Vosmaer2015-10-064-59/+45
| | | | | | | During the backup we create an intermediate copy of two directories: builds and uploads. Instead of creating many small files with 'cp -r', we now use tar (and fast gzip) to create single intermediate files. This saves on disk IO and disk space while creating a backup.
* Add support for AWS S3 Server-Side Encryption supportPaul Beattie2015-09-251-3/+4
| | | | | | | This adds support for AWS S3 SSE with S3 managed keys, this means the data is encrypted at rest and the encryption is handled transparently to the end user as well as in the AWS Console. This is optional and not required to make S3 uploads work.
* add pg_schema to backup configpg_dump_schemaValery Sizov2015-09-181-2/+6
|
* Fix backup testsKamil Trzcinski2015-09-161-1/+1
|
* Fix builds directory storeKamil Trzcinski2015-09-151-4/+8
|
* Cleanup CI backup => migrate with GitLabKamil Trzcinski2015-09-151-0/+30
|
* Do not delete the SQL dump too earlyJacob Vosmaer2015-08-211-2/+3
| | | | | | | | | The change in baa157926d432f404a41c31ad6514ff8d5366269 broke backup restore fucnctionality. This would not lead to data loss, but it prevented the restore script from working. This bug exists only in 7.14.0 release candidate versions, not in 7.13. Reported in https://github.com/gitlabhq/gitlabhq/issues/9571 .
* Workaround the warnings emitted by MySQL 5.6 regarding password on theTed Strzalkowski2015-08-121-2/+5
| | | | command line.
* Merge branch 'master' of github.com:gitlabhq/gitlabhqDmitriy Zaporozhets2015-08-034-6/+14
|\
| * Stricter mkdir's in 'rake gitlab:backup:create'backup-directory-permissionsJacob Vosmaer2015-07-303-4/+14
| |
| * Set internal backup directory modes on createJacob Vosmaer2015-07-294-5/+3
| | | | | | | | This sidesteps problems with running 'chmod' on some CIFS mounts.
* | Merge branch 'master' of dev.gitlab.org:gitlab/gitlabhq into ↵Jacob Vosmaer2015-07-271-5/+17
|\ \ | |/ | | | | backup-archive-permissions
| * Don't stop if database.sql.gz already existsoverwrite-during-backupJacob Vosmaer2015-07-211-0/+1
| | | | | | | | | | | | | | | | The existing behavior of the backups is to overwrite whatever data was still there in the scratch directories. This broke when we added a 'gzip' step because 'gzip database.sql' will fail if 'database.sql.gz' already exists. Doing 'rm -f database.sql.gz' before the 'gzip' avoids this failure.
| * Merge branch 'improve-postgres-restore-cleaning' into 'master' Dmitriy Zaporozhets2015-07-071-5/+2
| |\ | | | | | | | | | | | | | | | | | | | | | | | | Use native Postgres database cleaning during backup restore We were using hacks to drop tables etc during a Postgres backup restore. With this change, we let pg_dump insert the DROP TABLE statements it needs at the start of the SQL dump. See merge request !1891
| | * Use native Postgres database cleaning during backup restoreJacob Vosmaer2015-07-071-5/+2
| | | | | | | | | | | | | | | | | | We were using hacks to drop tables etc during a Postgres backup restore. With this change, we let pg_dump insert the DROP TABLE statements it needs at the start of the SQL dump.
| * | Compress database backupKamil Trzcinski2015-07-061-0/+14
| |/
* | Allow custom backup archive permissionsJacob Vosmaer2015-07-061-3/+3
|/ | | | | This change helps system administrators who want to replicate GitLab backup files without needing root permissions.
* Merge branch 'restore_uploads_fix' into 'master'Dmitriy Zaporozhets2015-06-221-1/+1
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Avoid "cannot copy directory ... to itself" error on restore (on Docker?) rake gitlab:backup:restore fails for me in my Docker-hosted Gitlab-CE instance; during the restore, any existing "uploads" directory is backed up by [this code](https://gitlab.com/gitlab-org/gitlab-ce/blob/833bc30/lib/backup/uploads.rb#L23) -- ```ruby def backup_existing_uploads_dir timestamped_uploads_path = File.join(app_uploads_dir, '..', "uploads.#{Time.now.to_i}") if File.exists?(app_uploads_dir) FileUtils.mv(app_uploads_dir, timestamped_uploads_path) end end ``` When this executes for me, the ```FileUtils.mv``` parameters are "/home/git/gitlab/public/uploads" and "/home/git/gitlab/public/uploads/../uploads.1407019546"; an exception is raised, producing this double stacktrace: ``` ArgumentError: cannot copy directory /home/git/gitlab/public/uploads to itself /home/git/gitlab/public/uploads/../uploads.1407019546 /home/git/gitlab/lib/backup/uploads.rb:26:in `backup_existing_uploads_dir' /home/git/gitlab/lib/backup/uploads.rb:18:in `restore' /home/git/gitlab/lib/tasks/gitlab/backup.rake:73:in `block (4 levels) in <top (required)>' /home/git/gitlab/lib/tasks/gitlab/backup.rake:30:in `block (3 levels) in <top (required)>' Errno::EXDEV: Invalid cross-device link @ sys_fail2 - (/home/git/gitlab/public/uploads, /home/git/gitlab/public/uploads/../uploads.1407019546) /home/git/gitlab/lib/backup/uploads.rb:26:in `backup_existing_uploads_dir' /home/git/gitlab/lib/backup/uploads.rb:18:in `restore' /home/git/gitlab/lib/tasks/gitlab/backup.rake:73:in `block (4 levels) in <top (required)>' /home/git/gitlab/lib/tasks/gitlab/backup.rake:30:in `block (3 levels) in <top (required)>' Tasks: TOP => gitlab:backup:uploads:restore (See full trace by running task with --trace) ``` I'm guessing from the first message that ```mv``` walks the destination path to ensure that we're not moving the source into itself -- it doesn't get as far as interpreting the '..', but throws when it sees that the destination appears to start with the source path. The second stacktrace I have no clue about - maybe it's AUFS- or Docker-related? I attempted to reproduce this separately with the omnibus distribution in a fresh Ubuntu 14.04 install without Docker involved, and was unable to - backup and restore worked fine. I then tested my theory by FileUtils.expand_path-ing the destination in my own Docker setup code, and that made the problem go away, so that's what this merge request does. (I'm using backups created and restored on gitlab-ce 7-1-stable, at facfec4b2; this is on Ubuntu 14.04 with Docker 1.1.1) I know I'd look askance at a PR without tests for an unreproducable problem, but even if this is rejected, I'm submitting it anyway because maybe someone else will Google it and find it useful. I'm happy to do more work to improve this if you have suggestions. See merge request !165
| * Avoid "cannot copy directory to itself" restore errorBryan Stearns2014-08-031-1/+1
| |
* | use gitlab not gitlabciJeroen Nijhof2015-06-091-1/+1
| |
* | update fog to 1.25.0 and add multipart upload supportJeroen Nijhof2015-06-091-1/+2
| |
* | ability to skip some items in backupbackup_skip_optionValery Sizov2015-04-021-6/+36
| |
* | Don't use chmod_R for backup tarsVinnie Okada2015-03-281-1/+1
| | | | | | | | | | When creating backup tar files, only change permissions on the `db`, `uploads`, and `repositories` directories, not their contents.
* | Change directory when removing old backupsVinnie Okada2015-03-241-7/+8
| |
* | Merge branch 'backup-permissions' into 'master'Dmitriy Zaporozhets2015-03-211-18/+25
|\ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change permissions on backup files - #2 Use more restrictive permissions for backup tar files and for the db, uploads, and repositories directories inside the tar files. See #1894. Now the backup task recursively `chmod`s the `db/`, `uploads/`, and `repositories/` folders with 0700 permissions, and the tar file is created as 0600. This is a followup to !1703, which was reverted because it broke Rspec tests. The test failures were due to the rake task changing directories and not changing back, which I fixed with this commit. cc @sytse See merge request !1716
| * | Call chdir() with a blockVinnie Okada2015-03-191-18/+17
| | |
| * | Change permissions on backup filesVinnie Okada2015-03-171-5/+13
| | | | | | | | | | | | | | | Use more restrictive permissions for backup tar files and for the db, uploads, and repositories directories inside the tar files.
* | | backup repo with tar instead of git bundleValery Sizov2015-03-191-2/+3
|/ /
* | Revert "Merge branch 'backup-permissions' into 'master'"Dmitriy Zaporozhets2015-03-161-4/+0
| | | | | | | | | | This reverts commit c42262b43b009af990e5769840391862d64a1c2d, reversing changes made to c6586b1283a94c8f08bc669f4d8a9384b263073e.
View Lorry Controller Status