diff options
Diffstat (limited to 'doc/raketasks')
-rw-r--r-- | doc/raketasks/README.md | 9 | ||||
-rw-r--r-- | doc/raketasks/backup_hrz.png | bin | 21955 -> 0 bytes | |||
-rw-r--r-- | doc/raketasks/backup_restore.md | 240 | ||||
-rw-r--r-- | doc/raketasks/cleanup.md | 23 | ||||
-rw-r--r-- | doc/raketasks/features.md | 20 | ||||
-rw-r--r-- | doc/raketasks/import.md | 68 | ||||
-rw-r--r-- | doc/raketasks/maintenance.md | 178 | ||||
-rw-r--r-- | doc/raketasks/user_management.md | 49 | ||||
-rw-r--r-- | doc/raketasks/web_hooks.md | 45 |
9 files changed, 0 insertions, 632 deletions
diff --git a/doc/raketasks/README.md b/doc/raketasks/README.md deleted file mode 100644 index 770b7a70fe0..00000000000 --- a/doc/raketasks/README.md +++ /dev/null @@ -1,9 +0,0 @@ -# Rake tasks - -- [Backup restore](backup_restore.md) -- [Cleanup](cleanup.md) -- [Features](features.md) -- [Maintenance](maintenance.md) and self-checks -- [User management](user_management.md) -- [Web hooks](web_hooks.md) -- [Import](import.md) of git repositories in bulk diff --git a/doc/raketasks/backup_hrz.png b/doc/raketasks/backup_hrz.png Binary files differdeleted file mode 100644 index 03e50df1d76..00000000000 --- a/doc/raketasks/backup_hrz.png +++ /dev/null diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md deleted file mode 100644 index 2e41fad89e7..00000000000 --- a/doc/raketasks/backup_restore.md +++ /dev/null @@ -1,240 +0,0 @@ -# Backup restore - -![backup banner](backup_hrz.png) - -## Create a backup of the GitLab system - -A backup creates an archive file that contains the database, all repositories and all attachments. -This archive will be saved in backup_path (see `config/gitlab.yml`). -The filename will be `[TIMESTAMP]_gitlab_backup.tar`. This timestamp can be used to restore an specific backup. -You can only restore a backup to exactly the same version of GitLab that you created it on, for example 7.2.1. - -``` -# use this command if you've installed GitLab with the Omnibus package -sudo gitlab-rake gitlab:backup:create - -# if you've installed GitLab from source -sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production -``` - -Also you can choose what should be backed up by adding environment variable SKIP. Available options: db, -uploads (attachments), repositories. Use a comma to specify several options at the same time. - -``` -sudo gitlab-rake gitlab:backup:create SKIP=db,uploads -``` - -Example output: - -``` -Dumping database tables: -- Dumping table events... [DONE] -- Dumping table issues... [DONE] -- Dumping table keys... [DONE] -- Dumping table merge_requests... [DONE] -- Dumping table milestones... [DONE] -- Dumping table namespaces... [DONE] -- Dumping table notes... [DONE] -- Dumping table projects... [DONE] -- Dumping table protected_branches... [DONE] -- Dumping table schema_migrations... [DONE] -- Dumping table services... [DONE] -- Dumping table snippets... [DONE] -- Dumping table taggings... [DONE] -- Dumping table tags... [DONE] -- Dumping table users... [DONE] -- Dumping table users_projects... [DONE] -- Dumping table web_hooks... [DONE] -- Dumping table wikis... [DONE] -Dumping repositories: -- Dumping repository abcd... [DONE] -Creating backup archive: $TIMESTAMP_gitlab_backup.tar [DONE] -Deleting tmp directories...[DONE] -Deleting old backups... [SKIPPING] -``` - -## Upload backups to remote (cloud) storage - -Starting with GitLab 7.4 you can let the backup script upload the '.tar' file it creates. -It uses the [Fog library](http://fog.io/) to perform the upload. -In the example below we use Amazon S3 for storage. -But Fog also lets you use [other storage providers](http://fog.io/storage/). - -For omnibus packages: - -```ruby -gitlab_rails['backup_upload_connection'] = { - 'provider' => 'AWS', - 'region' => 'eu-west-1', - 'aws_access_key_id' => 'AKIAKIAKI', - 'aws_secret_access_key' => 'secret123' -} -gitlab_rails['backup_upload_remote_directory'] = 'my.s3.bucket' -``` - -For installations from source: - -```yaml - backup: - # snip - upload: - # Fog storage connection settings, see http://fog.io/storage/ . - connection: - provider: AWS - region: eu-west-1 - aws_access_key_id: AKIAKIAKI - aws_secret_access_key: 'secret123' - # The remote 'directory' to store your backups. For S3, this would be the bucket name. - remote_directory: 'my.s3.bucket' -``` - -If you are uploading your backups to S3 you will probably want to create a new -IAM user with restricted access rights. To give the upload user access only for -uploading backups create the following IAM profile, replacing `my.s3.bucket` -with the name of your bucket: - -```json -{ - "Version": "2012-10-17", - "Statement": [ - { - "Sid": "Stmt1412062044000", - "Effect": "Allow", - "Action": [ - "s3:AbortMultipartUpload", - "s3:GetBucketAcl", - "s3:GetBucketLocation", - "s3:GetObject", - "s3:GetObjectAcl", - "s3:ListBucketMultipartUploads", - "s3:PutObject", - "s3:PutObjectAcl" - ], - "Resource": [ - "arn:aws:s3:::my.s3.bucket/*" - ] - }, - { - "Sid": "Stmt1412062097000", - "Effect": "Allow", - "Action": [ - "s3:GetBucketLocation", - "s3:ListAllMyBuckets" - ], - "Resource": [ - "*" - ] - }, - { - "Sid": "Stmt1412062128000", - "Effect": "Allow", - "Action": [ - "s3:ListBucket" - ], - "Resource": [ - "arn:aws:s3:::my.s3.bucket" - ] - } - ] -} -``` - -## Storing configuration files - -Please be informed that a backup does not store your configuration files. -If you use an Omnibus package please see the [instructions in the readme to backup your configuration](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md#backup-and-restore-omnibus-gitlab-configuration). -If you have a cookbook installation there should be a copy of your configuration in Chef. -If you have an installation from source, please consider backing up your `gitlab.yml` file, any SSL keys and certificates, and your [SSH host keys](https://superuser.com/questions/532040/copy-ssh-keys-from-one-server-to-another-server/532079#532079). - -## Restore a previously created backup - -You can only restore a backup to exactly the same version of GitLab that you created it on, for example 7.2.1. - -``` -# Omnibus package installation -sudo gitlab-rake gitlab:backup:restore - -# installation from source -bundle exec rake gitlab:backup:restore RAILS_ENV=production -``` - -Options: - -``` -BACKUP=timestamp_of_backup (required if more than one backup exists) -``` - -Example output: - -``` -Unpacking backup... [DONE] -Restoring database tables: --- create_table("events", {:force=>true}) - -> 0.2231s -[...] -- Loading fixture events...[DONE] -- Loading fixture issues...[DONE] -- Loading fixture keys...[SKIPPING] -- Loading fixture merge_requests...[DONE] -- Loading fixture milestones...[DONE] -- Loading fixture namespaces...[DONE] -- Loading fixture notes...[DONE] -- Loading fixture projects...[DONE] -- Loading fixture protected_branches...[SKIPPING] -- Loading fixture schema_migrations...[DONE] -- Loading fixture services...[SKIPPING] -- Loading fixture snippets...[SKIPPING] -- Loading fixture taggings...[SKIPPING] -- Loading fixture tags...[SKIPPING] -- Loading fixture users...[DONE] -- Loading fixture users_projects...[DONE] -- Loading fixture web_hooks...[SKIPPING] -- Loading fixture wikis...[SKIPPING] -Restoring repositories: -- Restoring repository abcd... [DONE] -Deleting tmp directories...[DONE] -``` - -## Configure cron to make daily backups - -For Omnibus package installations, see https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md#scheduling-a-backup . - -For installation from source: -``` -cd /home/git/gitlab -sudo -u git -H editor config/gitlab.yml # Enable keep_time in the backup section to automatically delete old backups -sudo -u git crontab -e # Edit the crontab for the git user -``` - -Add the following lines at the bottom: - -``` -# Create a full backup of the GitLab repositories and SQL database every day at 4am -0 4 * * * cd /home/git/gitlab && PATH=/usr/local/bin:/usr/bin:/bin bundle exec rake gitlab:backup:create RAILS_ENV=production CRON=1 -``` - -The `CRON=1` environment setting tells the backup script to suppress all progress output if there are no errors. -This is recommended to reduce cron spam. - -## Alternative backup strategies - -If your GitLab server contains a lot of Git repository data you may find the GitLab backup script to be too slow. -In this case you can consider using filesystem snapshots as part of your backup strategy. - -Example: Amazon EBS - -> A GitLab server using omnibus-gitlab hosted on Amazon AWS. -> An EBS drive containing an ext4 filesystem is mounted at `/var/opt/gitlab`. -> In this case you could make an application backup by taking an EBS snapshot. -> The backup includes all repositories, uploads and Postgres data. - -Example: LVM snapshots + rsync - -> A GitLab server using omnibus-gitlab, with an LVM logical volume mounted at `/var/opt/gitlab`. -> Replicating the `/var/opt/gitlab` directory using rsync would not be reliable because too many files would change while rsync is running. -> Instead of rsync-ing `/var/opt/gitlab`, we create a temporary LVM snapshot, which we mount as a read-only filesystem at `/mnt/gitlab_backup`. -> Now we can have a longer running rsync job which will create a consistent replica on the remote server. -> The replica includes all repositories, uploads and Postgres data. - -If you are running GitLab on a virtualized server you can possibly also create VM snapshots of the entire GitLab server. -It is not uncommon however for a VM snapshot to require you to power down the server, so this approach is probably of limited practical use. diff --git a/doc/raketasks/cleanup.md b/doc/raketasks/cleanup.md deleted file mode 100644 index 96d67f7b5d6..00000000000 --- a/doc/raketasks/cleanup.md +++ /dev/null @@ -1,23 +0,0 @@ -# Cleanup - -## Remove garbage from filesystem. Important! Data loss! - -Remove namespaces(dirs) from `/home/git/repositories` if they don't exist in GitLab database. - -``` -# omnibus-gitlab -sudo gitlab-rake gitlab:cleanup:dirs - -# installation from source -bundle exec rake gitlab:cleanup:dirs RAILS_ENV=production -``` - -Remove repositories (global only for now) from `/home/git/repositories` if they don't exist in GitLab database. - -``` -# omnibus-gitlab -sudo gitlab-rake gitlab:cleanup:repos - -# installation from source -bundle exec rake gitlab:cleanup:repos RAILS_ENV=production -``` diff --git a/doc/raketasks/features.md b/doc/raketasks/features.md deleted file mode 100644 index f9a46193547..00000000000 --- a/doc/raketasks/features.md +++ /dev/null @@ -1,20 +0,0 @@ -# Features - -## Enable usernames and namespaces for user projects - -This command will enable the namespaces feature introduced in v4.0. It will move every project in its namespace folder. - -Note: - -- Because the **repository location will change**, you will need to **update all your git URLs** to point to the new location. -- Username can be changed at [Profile / Account](/profile/account) - -**Example:** - -Old path: `git@example.org:myrepo.git` - -New path: `git@example.org:username/myrepo.git` or `git@example.org:groupname/myrepo.git` - -``` -bundle exec rake gitlab:enable_namespaces RAILS_ENV=production -``` diff --git a/doc/raketasks/import.md b/doc/raketasks/import.md deleted file mode 100644 index 8a38937062e..00000000000 --- a/doc/raketasks/import.md +++ /dev/null @@ -1,68 +0,0 @@ -# Import bare repositories into your GitLab instance - -## Notes - -- The owner of the project will be the first admin -- The groups will be created as needed -- The owner of the group will be the first admin -- Existing projects will be skipped - -## How to use - -### Create a new folder inside the git repositories path. This will be the name of the new group. - -- For omnibus-gitlab, it is located at: `/var/opt/gitlab/git-data/repositories` by default, unless you changed -it in the `/etc/gitlab/gitlab.rb` file. -- For installations from source, it is usually located at: `/home/git/repositories` or you can see where -your repositories are located by looking at `config/gitlab.yml` under the `gitlab_shell => repos_path` entry. - -New folder needs to have git user ownership and read/write/execute access for git user and its group: - -``` -sudo -u git mkdir /var/opt/gitlab/git-data/repositories/new_group -``` - -If you are using an installation from source, replace `/var/opt/gitlab/git-data` -with `/home/git`. - -### Copy your bare repositories inside this newly created folder: - -``` -sudo cp -r /old/git/foo.git /var/opt/gitlab/git-data/repositories/new_group/ - -# Do this once when you are done copying git repositories -sudo chown -R git:git /var/opt/gitlab/git-data/repositories/new_group/ -``` - -`foo.git` needs to be owned by the git user and git users group. - -If you are using an installation from source, replace `/var/opt/gitlab/git-data` -with `/home/git`. - -### Run the command below depending on your type of installation: - -#### Omnibus Installation - -``` -$ sudo gitlab-rake gitlab:import:repos -``` - -#### Installation from source - -Before running this command you need to change the directory to where your GitLab installation is located: - -``` -$ cd /home/git/gitlab -$ sudo -u git -H bundle exec rake gitlab:import:repos RAILS_ENV=production -``` - -#### Example output - -``` -Processing abcd.git - * Created abcd (abcd.git) -Processing group/xyz.git - * Created Group group (2) - * Created xyz (group/xyz.git) -[...] -``` diff --git a/doc/raketasks/maintenance.md b/doc/raketasks/maintenance.md deleted file mode 100644 index 41a994f3f68..00000000000 --- a/doc/raketasks/maintenance.md +++ /dev/null @@ -1,178 +0,0 @@ -# Maintenance - -## Gather information about GitLab and the system it runs on - -This command gathers information about your GitLab installation and the System it runs on. These may be useful when asking for help or reporting issues. - -``` -# omnibus-gitlab -sudo gitlab-rake gitlab:env:info - -# installation from source -bundle exec rake gitlab:env:info RAILS_ENV=production -``` - -Example output: - -``` -System information -System: Debian 7.8 -Current User: git -Using RVM: no -Ruby Version: 2.1.5p273 -Gem Version: 2.4.3 -Bundler Version: 1.7.6 -Rake Version: 10.3.2 -Sidekiq Version: 2.17.8 - -GitLab information -Version: 7.7.1 -Revision: 41ab9e1 -Directory: /home/git/gitlab -DB Adapter: postgresql -URL: https://gitlab.example.com -HTTP Clone URL: https://gitlab.example.com/some-project.git -SSH Clone URL: git@gitlab.example.com:some-project.git -Using LDAP: no -Using Omniauth: no - -GitLab Shell -Version: 2.4.1 -Repositories: /home/git/repositories/ -Hooks: /home/git/gitlab-shell/hooks/ -Git: /usr/bin/git -``` - -## Check GitLab configuration - -Runs the following rake tasks: - -- `gitlab:env:check` -- `gitlab:gitlab_shell:check` -- `gitlab:sidekiq:check` -- `gitlab:app:check` - -It will check that each component was setup according to the installation guide and suggest fixes for issues found. - -You may also have a look at our [Trouble Shooting Guide](https://github.com/gitlabhq/gitlab-public-wiki/wiki/Trouble-Shooting-Guide). - -``` -# omnibus-gitlab -sudo gitlab-rake gitlab:check - -# installation from source -bundle exec rake gitlab:check RAILS_ENV=production -``` - -NOTE: Use SANITIZE=true for gitlab:check if you want to omit project names from the output. - -Example output: - -``` -Checking Environment ... - -Git configured for git user? ... yes -Has python2? ... yes -python2 is supported version? ... yes - -Checking Environment ... Finished - -Checking GitLab Shell ... - -GitLab Shell version? ... OK (1.2.0) -Repo base directory exists? ... yes -Repo base directory is a symlink? ... no -Repo base owned by git:git? ... yes -Repo base access is drwxrws---? ... yes -post-receive hook up-to-date? ... yes -post-receive hooks in repos are links: ... yes - -Checking GitLab Shell ... Finished - -Checking Sidekiq ... - -Running? ... yes - -Checking Sidekiq ... Finished - -Checking GitLab ... - -Database config exists? ... yes -Database is SQLite ... no -All migrations up? ... yes -GitLab config exists? ... yes -GitLab config outdated? ... no -Log directory writable? ... yes -Tmp directory writable? ... yes -Init script exists? ... yes -Init script up-to-date? ... yes -Projects have satellites? ... yes -Redis version >= 2.0.0? ... yes - -Checking GitLab ... Finished -``` - -## (Re-)Create satellite repositories - -This will create satellite repositories for all your projects. - -If necessary, remove the `repo_satellites` directory and rerun the commands below. - -``` -sudo -u git -H mkdir -p /home/git/gitlab-satellites -sudo -u git -H bundle exec rake gitlab:satellites:create RAILS_ENV=production -sudo chmod u+rwx,g=rx,o-rwx /home/git/gitlab-satellites -``` - -## Rebuild authorized_keys file - -In some case it is necessary to rebuild the `authorized_keys` file. - -For Omnibus-packages: -``` -sudo gitlab-rake gitlab:shell:setup -``` - -For installations from source: -``` -cd /home/git/gitlab -sudo -u git -H bundle exec rake gitlab:shell:setup RAILS_ENV=production -``` - -``` -This will rebuild an authorized_keys file. -You will lose any data stored in authorized_keys file. -Do you want to continue (yes/no)? yes -``` - -## Clear redis cache - -If for some reason the dashboard shows wrong information you might want to -clear Redis' cache. - -For Omnibus-packages: -``` -sudo gitlab-rake cache:clear -``` - -For installations from source: -``` -cd /home/git/gitlab -sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production -``` - -## Precompile the assets - -Sometimes during version upgrades you might end up with some wrong CSS or -missing some icons. In that case, try to precompile the assets again. - -For Omnibus-packages: -``` -sudo gitlab-rake assets:precompile -``` - -For installations from source: -``` -cd /home/git/gitlab -sudo -u git -H bundle exec rake assets:precompile RAILS_ENV=production -``` diff --git a/doc/raketasks/user_management.md b/doc/raketasks/user_management.md deleted file mode 100644 index 80b01ca4043..00000000000 --- a/doc/raketasks/user_management.md +++ /dev/null @@ -1,49 +0,0 @@ -# User management - -## Add user as a developer to all projects - -```bash -# omnibus-gitlab -sudo gitlab-rake gitlab:import:user_to_projects[username@domain.tld] - -# installation from source -bundle exec rake gitlab:import:user_to_projects[username@domain.tld] RAILS_ENV=production -``` - -## Add all users to all projects - -Notes: - -- admin users are added as masters - -```bash -# omnibus-gitlab -sudo gitlab-rake gitlab:import:all_users_to_all_projects - -# installation from source -bundle exec rake gitlab:import:all_users_to_all_projects RAILS_ENV=production -``` - -## Add user as a developer to all groups - -```bash -# omnibus-gitlab -sudo gitlab-rake gitlab:import:user_to_groups[username@domain.tld] - -# installation from source -bundle exec rake gitlab:import:user_to_groups[username@domain.tld] RAILS_ENV=production -``` - -## Add all users to all groups - -Notes: - -- admin users are added as owners so they can add additional users to the group - -```bash -# omnibus-gitlab -sudo gitlab-rake gitlab:import:all_users_to_all_groups - -# installation from source -bundle exec rake gitlab:import:all_users_to_all_groups RAILS_ENV=production -``` diff --git a/doc/raketasks/web_hooks.md b/doc/raketasks/web_hooks.md deleted file mode 100644 index 5a8b94af9b4..00000000000 --- a/doc/raketasks/web_hooks.md +++ /dev/null @@ -1,45 +0,0 @@ -# Web hooks - -## Add a web hook for **ALL** projects: - - # omnibus-gitlab - sudo gitlab-rake gitlab:web_hook:add URL="http://example.com/hook" - # source installations - bundle exec rake gitlab:web_hook:add URL="http://example.com/hook" RAILS_ENV=production - -## Add a web hook for projects in a given **NAMESPACE**: - - # omnibus-gitlab - sudo gitlab-rake gitlab:web_hook:add URL="http://example.com/hook" NAMESPACE=acme - # source installations - bundle exec rake gitlab:web_hook:add URL="http://example.com/hook" NAMESPACE=acme RAILS_ENV=production - -## Remove a web hook from **ALL** projects using: - - # omnibus-gitlab - sudo gitlab-rake gitlab:web_hook:rm URL="http://example.com/hook" - # source installations - bundle exec rake gitlab:web_hook:rm URL="http://example.com/hook" RAILS_ENV=production - -## Remove a web hook from projects in a given **NAMESPACE**: - - # omnibus-gitlab - sudo gitlab-rake gitlab:web_hook:rm URL="http://example.com/hook" NAMESPACE=acme - # source installations - bundle exec rake gitlab:web_hook:rm URL="http://example.com/hook" NAMESPACE=acme RAILS_ENV=production - -## List **ALL** web hooks: - - # omnibus-gitlab - sudo gitlab-rake gitlab:web_hook:list - # source installations - bundle exec rake gitlab:web_hook:list RAILS_ENV=production - -## List the web hooks from projects in a given **NAMESPACE**: - - # omnibus-gitlab - sudo gitlab-rake gitlab:web_hook:list NAMESPACE=/ - # source installations - bundle exec rake gitlab:web_hook:list NAMESPACE=/ RAILS_ENV=production - -> Note: `/` is the global namespace. |