| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| | |
|
| |
|
|
| |
This sidesteps problems with running 'chmod' on some CIFS mounts.
|
| |
|
|
|
|
|
|
| |
The existing behavior of the backups is to overwrite whatever data
was still there in the scratch directories. This broke when we added
a 'gzip' step because 'gzip database.sql' will fail if 'database.sql.gz'
already exists. Doing 'rm -f database.sql.gz' before the 'gzip'
avoids this failure.
|
| |\
| |
| |
| |
| |
| |
| |
| |
| | |
Use native Postgres database cleaning during backup restore
We were using hacks to drop tables etc during a Postgres backup
restore. With this change, we let pg_dump insert the DROP TABLE
statements it needs at the start of the SQL dump.
See merge request !1891
|
| | |
| |
| |
| |
| |
| | |
We were using hacks to drop tables etc during a Postgres backup
restore. With this change, we let pg_dump insert the DROP TABLE
statements it needs at the start of the SQL dump.
|
| |/ |
|
| |\
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Avoid "cannot copy directory ... to itself" error on restore (on Docker?)
rake gitlab:backup:restore fails for me in my Docker-hosted Gitlab-CE instance; during the restore, any existing "uploads" directory is backed up by [this code](https://gitlab.com/gitlab-org/gitlab-ce/blob/833bc30/lib/backup/uploads.rb#L23) --
```ruby
def backup_existing_uploads_dir
timestamped_uploads_path = File.join(app_uploads_dir, '..', "uploads.#{Time.now.to_i}")
if File.exists?(app_uploads_dir)
FileUtils.mv(app_uploads_dir, timestamped_uploads_path)
end
end
```
When this executes for me, the ```FileUtils.mv``` parameters are "/home/git/gitlab/public/uploads" and "/home/git/gitlab/public/uploads/../uploads.1407019546"; an exception is raised, producing this double stacktrace:
```
ArgumentError: cannot copy directory /home/git/gitlab/public/uploads to itself /home/git/gitlab/public/uploads/../uploads.1407019546
/home/git/gitlab/lib/backup/uploads.rb:26:in `backup_existing_uploads_dir'
/home/git/gitlab/lib/backup/uploads.rb:18:in `restore'
/home/git/gitlab/lib/tasks/gitlab/backup.rake:73:in `block (4 levels) in <top (required)>'
/home/git/gitlab/lib/tasks/gitlab/backup.rake:30:in `block (3 levels) in <top (required)>'
Errno::EXDEV: Invalid cross-device link @ sys_fail2 - (/home/git/gitlab/public/uploads, /home/git/gitlab/public/uploads/../uploads.1407019546)
/home/git/gitlab/lib/backup/uploads.rb:26:in `backup_existing_uploads_dir'
/home/git/gitlab/lib/backup/uploads.rb:18:in `restore'
/home/git/gitlab/lib/tasks/gitlab/backup.rake:73:in `block (4 levels) in <top (required)>'
/home/git/gitlab/lib/tasks/gitlab/backup.rake:30:in `block (3 levels) in <top (required)>'
Tasks: TOP => gitlab:backup:uploads:restore
(See full trace by running task with --trace)
```
I'm guessing from the first message that ```mv``` walks the destination path to ensure that we're not moving the source into itself -- it doesn't get as far as interpreting the '..', but throws when it sees that the destination appears to start with the source path.
The second stacktrace I have no clue about - maybe it's AUFS- or Docker-related?
I attempted to reproduce this separately with the omnibus distribution in a fresh Ubuntu 14.04 install without Docker involved, and was unable to - backup and restore worked fine. I then tested my theory by FileUtils.expand_path-ing the destination in my own Docker setup code, and that made the problem go away, so that's what this merge request does.
(I'm using backups created and restored on gitlab-ce 7-1-stable, at facfec4b2; this is on Ubuntu 14.04 with Docker 1.1.1)
I know I'd look askance at a PR without tests for an unreproducable problem, but even if this is rejected, I'm submitting it anyway because maybe someone else will Google it and find it useful. I'm happy to do more work to improve this if you have suggestions.
See merge request !165
|
| | | |
|
| | | |
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| | |
When creating backup tar files, only change permissions on the `db`,
`uploads`, and `repositories` directories, not their contents.
|
| | | |
|
| |\ \
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
Change permissions on backup files - #2
Use more restrictive permissions for backup tar files and for the db, uploads, and repositories directories inside the tar files. See #1894. Now the backup task recursively `chmod`s the `db/`, `uploads/`, and `repositories/` folders with 0700 permissions, and the tar file is created as 0600.
This is a followup to !1703, which was reverted because it broke Rspec tests. The test failures were due to the rake task changing directories and not changing back, which I fixed with this commit.
cc @sytse
See merge request !1716
|
| | | | |
|
| | | |
| | |
| | |
| | |
| | | |
Use more restrictive permissions for backup tar files and for the db,
uploads, and repositories directories inside the tar files.
|
| |/ / |
|
| | |
| |
| |
| |
| | |
This reverts commit c42262b43b009af990e5769840391862d64a1c2d, reversing
changes made to c6586b1283a94c8f08bc669f4d8a9384b263073e.
|
| | |
| |
| |
| |
| | |
Use more restrictive permissions for backup tar files and for the db,
uploads, and repositories directories inside the tar files.
|
| | |
| |
| |
| | |
This reverts commit 7d5f86f6cbd187e75a6ba164ad6bfd036977dd07.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
This commit moves the note folder from
/public/uploads/note
to
/uploads/note
and changes the uploader accordingly.
Now it's no longer possible to avoid the access control by modifing the url.
The Avatar upload has been refactored to use an own uploader as well
to cleanly seperate the two upload types.
|
| | |
| |
| |
| |
| |
| | |
ProjectWiki.new() creates a new wiki git repository, so any tries
to bare clone a bundle fail. With this patch we remove the newly
created wiki.git before restoring from the backup bundle.
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| | |
If a project is being restored, but there is no bundle file, the project was
empty when it was backed up. In this case, just use git init --base to create a
new bare repository.
|
| |\ \
| | |
| | |
| | |
| | | |
Conflicts:
CHANGELOG
|
| | | | |
|
| |/ / |
|
| | |
| |
| |
| |
| |
| |
| | |
We need to skip empty repositories when creating a backup. Before this
change, we were asking gollum-lib if the wiki contains any _pages_. Now
we ask gitlab_git if the repository contains _files_. This should
resolve gollum_lib Grit timeouts in the backup script.
|
| | |
| |
| |
| |
| | |
This change also shows the output of failed Git commands during the
backup.
|
| | | |
|
| |/
|
|
|
| |
The rewrite-hooks.sh script is a deprecated wrapper for gitlab-shell's
create-hooks script.
|
| |
|
|
|
| |
This fixes a bug where wiki repositories for projects with an empty main
repository would not get backed up.
|
| |
|
|
|
| |
Fixes a bug with non-UTF8 locales introduced by
2b816075dc71dfe8f6f9e5349fdff7f03ad9dad0.
|
| |
|
|
| |
Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>
|
| |
|
|
|
|
| |
Invoking 'db:schema:load' turned out to be a bad idea: when downgrading
an existing GitLab installation, the schema of the newer version would
be preserved when trying to import the old version.
|
| | |
|
| |\
| |
| |
| | |
Backup Uploads Symlink
|
| | |
| |
| |
| |
| |
| |
| | |
The backup restore code moves any existing uploads directory out of the
way before restoring the copy from the backup. If public/uploads was a
symlink, this move would replace the symlink. This commit avoids this
issue by first resolving any symlinks in the uploads path.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| | |
Previous to this commit, backups were tied to the git revision (SHA1) of
the app at the time the backup:create command was invoked. If the SHA1
at the time of restore was different, the script would refuse to restore
the backup. This commit loosens this tie so that the backup script only
complains if the value of the Gitlab::VERSION constant is different
between the time of backup and the time of restore.
|
| |\ \
| | |
| | |
| | | |
Pg Force Restore
|
| | |/
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The expected behavior during a GitLab backup restore is to overwrite
existing database data. This works for MySQL because the output of
mysqldump contains 'DROP TABLE IF EXISTS' statements. pg_dump on the
other hand assumes that one will restore into an empty database. When
this is not the case, during the restore with psql some of the data will
be skipped if existing data is 'in the way'. By first invoking `rake
db:schema:load` during a Postgres GitLab backup restore, we make sure
that all important data is correctly restored.
|
| |/
|
|
|
|
|
| |
The backup restore code for repositories was assuming that gitlab-shell
is installed in /home/git/gitlab-shell. This commit changes that to use
the configuration setting from gitlab.yml that specifies the path where
gitlab-shell is installed.
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
This reverts commit c46eaca91247ccf8e6fb3b691dad028e1b084ae3.
|
| |
|
|
|
|
|
|
|
| |
- Database name may contain characters which are not shell friendly
- Database password could contain the same
- While we at it there is no harm in escaping generated paths too
- Refactored 2-line system(command)
Signed-off-by: Nigel Kukard <nkukard@lbsd.net>
|
| | |
|
| | |
|
