diff options
Diffstat (limited to 'doc/administration/raketasks')
| -rw-r--r-- | doc/administration/raketasks/README.md | 11 | ||||
| -rw-r--r-- | doc/administration/raketasks/backup_hrz.png | bin | 0 -> 21955 bytes | |||
| -rw-r--r-- | doc/administration/raketasks/backup_restore.md | 438 | ||||
| -rw-r--r-- | doc/administration/raketasks/check.md | 63 | ||||
| -rw-r--r-- | doc/administration/raketasks/check_repos_output.png | bin | 0 -> 73786 bytes | |||
| -rw-r--r-- | doc/administration/raketasks/cleanup.md | 24 | ||||
| -rw-r--r-- | doc/administration/raketasks/features.md | 20 | ||||
| -rw-r--r-- | doc/administration/raketasks/import.md | 68 | ||||
| -rw-r--r-- | doc/administration/raketasks/list_repos.md | 30 | ||||
| -rw-r--r-- | doc/administration/raketasks/maintenance.md | 169 | ||||
| -rw-r--r-- | doc/administration/raketasks/user_management.md | 72 | ||||
| -rw-r--r-- | doc/administration/raketasks/web_hooks.md | 45 |
12 files changed, 940 insertions, 0 deletions
diff --git a/doc/administration/raketasks/README.md b/doc/administration/raketasks/README.md new file mode 100644 index 00000000000..a49c43b8ef2 --- /dev/null +++ b/doc/administration/raketasks/README.md @@ -0,0 +1,11 @@ +# Rake tasks + +- [Backup restore](backup_restore.md) +- [Check](check.md) +- [Cleanup](cleanup.md) +- [Features](features.md) +- [Maintenance](maintenance.md) and self-checks +- [User management](user_management.md) +- [Webhooks](web_hooks.md) +- [Import](import.md) of git repositories in bulk +- [Rebuild authorized_keys file](http://docs.gitlab.com/ce/raketasks/maintenance.html#rebuild-authorized_keys-file) task for administrators diff --git a/doc/administration/raketasks/backup_hrz.png b/doc/administration/raketasks/backup_hrz.png Binary files differnew file mode 100644 index 00000000000..03e50df1d76 --- /dev/null +++ b/doc/administration/raketasks/backup_hrz.png diff --git a/doc/administration/raketasks/backup_restore.md b/doc/administration/raketasks/backup_restore.md new file mode 100644 index 00000000000..fa976134341 --- /dev/null +++ b/doc/administration/raketasks/backup_restore.md @@ -0,0 +1,438 @@ +# Backup restore + + + +## Create a backup of the GitLab system + +A backup creates an archive file that contains the database, all repositories and all attachments. +This archive will be saved in backup_path (see `config/gitlab.yml`). +The filename will be `[TIMESTAMP]_gitlab_backup.tar`. This timestamp can be used to restore an specific backup. +You can only restore a backup to exactly the same version of GitLab that you created it +on, for example 7.2.1. The best way to migrate your repositories from one server to +another is through backup restore. + +You need to keep a separate copy of `/etc/gitlab/gitlab-secrets.json` +(for omnibus packages) or `/home/git/gitlab/.secret` (for installations +from source). This file contains the database encryption key used +for two-factor authentication. If you restore a GitLab backup without +restoring the database encryption key, users who have two-factor +authentication enabled will lose access to your GitLab server. + +``` +# use this command if you've installed GitLab with the Omnibus package +sudo gitlab-rake gitlab:backup:create + +# if you've installed GitLab from source +sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production +``` + +Also you can choose what should be backed up by adding environment variable SKIP. Available options: db, +uploads (attachments), repositories, builds(CI build output logs), artifacts (CI build artifacts), lfs (LFS objects). +Use a comma to specify several options at the same time. + +``` +sudo gitlab-rake gitlab:backup:create SKIP=db,uploads +``` + +Example output: + +``` +Dumping database tables: +- Dumping table events... [DONE] +- Dumping table issues... [DONE] +- Dumping table keys... [DONE] +- Dumping table merge_requests... [DONE] +- Dumping table milestones... [DONE] +- Dumping table namespaces... [DONE] +- Dumping table notes... [DONE] +- Dumping table projects... [DONE] +- Dumping table protected_branches... [DONE] +- Dumping table schema_migrations... [DONE] +- Dumping table services... [DONE] +- Dumping table snippets... [DONE] +- Dumping table taggings... [DONE] +- Dumping table tags... [DONE] +- Dumping table users... [DONE] +- Dumping table users_projects... [DONE] +- Dumping table web_hooks... [DONE] +- Dumping table wikis... [DONE] +Dumping repositories: +- Dumping repository abcd... [DONE] +Creating backup archive: $TIMESTAMP_gitlab_backup.tar [DONE] +Deleting tmp directories...[DONE] +Deleting old backups... [SKIPPING] +``` + +## Upload backups to remote (cloud) storage + +Starting with GitLab 7.4 you can let the backup script upload the '.tar' file it creates. +It uses the [Fog library](http://fog.io/) to perform the upload. +In the example below we use Amazon S3 for storage. +But Fog also lets you use [other storage providers](http://fog.io/storage/). + +For omnibus packages: + +```ruby +gitlab_rails['backup_upload_connection'] = { + 'provider' => 'AWS', + 'region' => 'eu-west-1', + 'aws_access_key_id' => 'AKIAKIAKI', + 'aws_secret_access_key' => 'secret123' +} +gitlab_rails['backup_upload_remote_directory'] = 'my.s3.bucket' +``` + +For installations from source: + +```yaml + backup: + # snip + upload: + # Fog storage connection settings, see http://fog.io/storage/ . + connection: + provider: AWS + region: eu-west-1 + aws_access_key_id: AKIAKIAKI + aws_secret_access_key: 'secret123' + # The remote 'directory' to store your backups. For S3, this would be the bucket name. + remote_directory: 'my.s3.bucket' + # Turns on AWS Server-Side Encryption with Amazon S3-Managed Keys for backups, this is optional + # encryption: 'AES256' +``` + +If you are uploading your backups to S3 you will probably want to create a new +IAM user with restricted access rights. To give the upload user access only for +uploading backups create the following IAM profile, replacing `my.s3.bucket` +with the name of your bucket: + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Sid": "Stmt1412062044000", + "Effect": "Allow", + "Action": [ + "s3:AbortMultipartUpload", + "s3:GetBucketAcl", + "s3:GetBucketLocation", + "s3:GetObject", + "s3:GetObjectAcl", + "s3:ListBucketMultipartUploads", + "s3:PutObject", + "s3:PutObjectAcl" + ], + "Resource": [ + "arn:aws:s3:::my.s3.bucket/*" + ] + }, + { + "Sid": "Stmt1412062097000", + "Effect": "Allow", + "Action": [ + "s3:GetBucketLocation", + "s3:ListAllMyBuckets" + ], + "Resource": [ + "*" + ] + }, + { + "Sid": "Stmt1412062128000", + "Effect": "Allow", + "Action": [ + "s3:ListBucket" + ], + "Resource": [ + "arn:aws:s3:::my.s3.bucket" + ] + } + ] +} +``` + +### Uploading to locally mounted shares + +You may also send backups to a mounted share (`NFS` / `CIFS` / `SMB` / etc.) by +using the [`Local`](https://github.com/fog/fog-local#usage) storage provider. +The directory pointed to by the `local_root` key **must** be owned by the `git` +user **when mounted** (mounting with the `uid=` of the `git` user for `CIFS` and +`SMB`) or the user that you are executing the backup tasks under (for omnibus +packages, this is the `git` user). + +The `backup_upload_remote_directory` **must** be set in addition to the +`local_root` key. This is the sub directory inside the mounted directory that +backups will be copied to, and will be created if it does not exist. If the +directory that you want to copy the tarballs to is the root of your mounted +directory, just use `.` instead. + +For omnibus packages: + +```ruby +gitlab_rails['backup_upload_connection'] = { + :provider => 'Local', + :local_root => '/mnt/backups' +} + +# The directory inside the mounted folder to copy backups to +# Use '.' to store them in the root directory +gitlab_rails['backup_upload_remote_directory'] = 'gitlab_backups' +``` + +For installations from source: + +```yaml + backup: + # snip + upload: + # Fog storage connection settings, see http://fog.io/storage/ . + connection: + provider: Local + local_root: '/mnt/backups' + # The directory inside the mounted folder to copy backups to + # Use '.' to store them in the root directory + remote_directory: 'gitlab_backups' +``` + +## Backup archive permissions + +The backup archives created by GitLab (123456_gitlab_backup.tar) will have owner/group git:git and 0600 permissions by default. +This is meant to avoid other system users reading GitLab's data. +If you need the backup archives to have different permissions you can use the 'archive_permissions' setting. + +``` +# In /etc/gitlab/gitlab.rb, for omnibus packages +gitlab_rails['backup_archive_permissions'] = 0644 # Makes the backup archives world-readable +``` + +``` +# In gitlab.yml, for installations from source: + backup: + archive_permissions: 0644 # Makes the backup archives world-readable +``` + +## Storing configuration files + +Please be informed that a backup does not store your configuration +files. One reason for this is that your database contains encrypted +information for two-factor authentication. Storing encrypted +information along with its key in the same place defeats the purpose +of using encryption in the first place! + +If you use an Omnibus package please see the [instructions in the readme to backup your configuration](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md#backup-and-restore-omnibus-gitlab-configuration). +If you have a cookbook installation there should be a copy of your configuration in Chef. +If you have an installation from source, please consider backing up your `.secret` file, `gitlab.yml` file, any SSL keys and certificates, and your [SSH host keys](https://superuser.com/questions/532040/copy-ssh-keys-from-one-server-to-another-server/532079#532079). + +At the very **minimum** you should backup `/etc/gitlab/gitlab-secrets.json` +(Omnibus) or `/home/git/gitlab/.secret` (source) to preserve your +database encryption key. + +## Restore a previously created backup + +You can only restore a backup to exactly the same version of GitLab that you created it on, for example 7.2.1. + +### Prerequisites + +You need to have a working GitLab installation before you can perform +a restore. This is mainly because the system user performing the +restore actions ('git') is usually not allowed to create or delete +the SQL database it needs to import data into ('gitlabhq_production'). +All existing data will be either erased (SQL) or moved to a separate +directory (repositories, uploads). + +If some or all of your GitLab users are using two-factor authentication +(2FA) then you must also make sure to restore +`/etc/gitlab/gitlab-secrets.json` (Omnibus) or `/home/git/gitlab/.secret` +(installations from source). Note that you need to run `gitlab-ctl +reconfigure` after changing `gitlab-secrets.json`. + +### Installation from source + +``` +# Stop processes that are connected to the database +sudo service gitlab stop + +bundle exec rake gitlab:backup:restore RAILS_ENV=production +``` + +Options: + +``` +BACKUP=timestamp_of_backup (required if more than one backup exists) +force=yes (do not ask if the authorized_keys file should get regenerated) +``` + +Example output: + +``` +Unpacking backup... [DONE] +Restoring database tables: +-- create_table("events", {:force=>true}) + -> 0.2231s +[...] +- Loading fixture events...[DONE] +- Loading fixture issues...[DONE] +- Loading fixture keys...[SKIPPING] +- Loading fixture merge_requests...[DONE] +- Loading fixture milestones...[DONE] +- Loading fixture namespaces...[DONE] +- Loading fixture notes...[DONE] +- Loading fixture projects...[DONE] +- Loading fixture protected_branches...[SKIPPING] +- Loading fixture schema_migrations...[DONE] +- Loading fixture services...[SKIPPING] +- Loading fixture snippets...[SKIPPING] +- Loading fixture taggings...[SKIPPING] +- Loading fixture tags...[SKIPPING] +- Loading fixture users...[DONE] +- Loading fixture users_projects...[DONE] +- Loading fixture web_hooks...[SKIPPING] +- Loading fixture wikis...[SKIPPING] +Restoring repositories: +- Restoring repository abcd... [DONE] +Deleting tmp directories...[DONE] +``` + +### Omnibus installations + +This procedure assumes that: + +- You have installed the exact same version of GitLab Omnibus with which the + backup was created +- You have run `sudo gitlab-ctl reconfigure` at least once +- GitLab is running. If not, start it using `sudo gitlab-ctl start`. + +First make sure your backup tar file is in the backup directory described in the +`gitlab.rb` configuration `gitlab_rails['backup_path']`. The default is +`/var/opt/gitlab/backups`. + +```shell +sudo cp 1393513186_gitlab_backup.tar /var/opt/gitlab/backups/ +``` + +Stop the processes that are connected to the database. Leave the rest of GitLab +running: + +```shell +sudo gitlab-ctl stop unicorn +sudo gitlab-ctl stop sidekiq +# Verify +sudo gitlab-ctl status +``` + +Next, restore the backup, specifying the timestamp of the backup you wish to +restore: + +```shell +# This command will overwrite the contents of your GitLab database! +sudo gitlab-rake gitlab:backup:restore BACKUP=1393513186 +``` + +Restart and check GitLab: + +```shell +sudo gitlab-ctl start +sudo gitlab-rake gitlab:check SANITIZE=true +``` + +If there is a GitLab version mismatch between your backup tar file and the installed +version of GitLab, the restore command will abort with an error. Install the +[correct GitLab version](https://www.gitlab.com/downloads/archives/) and try again. + +## Configure cron to make daily backups + +### For installation from source: +``` +cd /home/git/gitlab +sudo -u git -H editor config/gitlab.yml # Enable keep_time in the backup section to automatically delete old backups +sudo -u git crontab -e # Edit the crontab for the git user +``` + +Add the following lines at the bottom: + +``` +# Create a full backup of the GitLab repositories and SQL database every day at 4am +0 4 * * * cd /home/git/gitlab && PATH=/usr/local/bin:/usr/bin:/bin bundle exec rake gitlab:backup:create RAILS_ENV=production CRON=1 +``` + +The `CRON=1` environment setting tells the backup script to suppress all progress output if there are no errors. +This is recommended to reduce cron spam. + +### For omnibus installations + +To schedule a cron job that backs up your repositories and GitLab metadata, use the root user: + +``` +sudo su - +crontab -e +``` + +There, add the following line to schedule the backup for everyday at 2 AM: + +``` +0 2 * * * /opt/gitlab/bin/gitlab-rake gitlab:backup:create CRON=1 +``` + +You may also want to set a limited lifetime for backups to prevent regular +backups using all your disk space. To do this add the following lines to +`/etc/gitlab/gitlab.rb` and reconfigure: + +``` +# limit backup lifetime to 7 days - 604800 seconds +gitlab_rails['backup_keep_time'] = 604800 +``` + +NOTE: This cron job does not [backup your omnibus-gitlab configuration](#backup-and-restore-omnibus-gitlab-configuration) or [SSH host keys](https://superuser.com/questions/532040/copy-ssh-keys-from-one-server-to-another-server/532079#532079). + +## Alternative backup strategies + +If your GitLab server contains a lot of Git repository data you may find the GitLab backup script to be too slow. +In this case you can consider using filesystem snapshots as part of your backup strategy. + +Example: Amazon EBS + +> A GitLab server using omnibus-gitlab hosted on Amazon AWS. +> An EBS drive containing an ext4 filesystem is mounted at `/var/opt/gitlab`. +> In this case you could make an application backup by taking an EBS snapshot. +> The backup includes all repositories, uploads and Postgres data. + +Example: LVM snapshots + rsync + +> A GitLab server using omnibus-gitlab, with an LVM logical volume mounted at `/var/opt/gitlab`. +> Replicating the `/var/opt/gitlab` directory using rsync would not be reliable because too many files would change while rsync is running. +> Instead of rsync-ing `/var/opt/gitlab`, we create a temporary LVM snapshot, which we mount as a read-only filesystem at `/mnt/gitlab_backup`. +> Now we can have a longer running rsync job which will create a consistent replica on the remote server. +> The replica includes all repositories, uploads and Postgres data. + +If you are running GitLab on a virtualized server you can possibly also create VM snapshots of the entire GitLab server. +It is not uncommon however for a VM snapshot to require you to power down the server, so this approach is probably of limited practical use. + +## Troubleshooting + +### Restoring database backup using omnibus packages outputs warnings +If you are using backup restore procedures you might encounter the following warnings: + +``` +psql:/var/opt/gitlab/backups/db/database.sql:22: ERROR: must be owner of extension plpgsql +psql:/var/opt/gitlab/backups/db/database.sql:2931: WARNING: no privileges could be revoked for "public" (two occurrences) +psql:/var/opt/gitlab/backups/db/database.sql:2933: WARNING: no privileges were granted for "public" (two occurrences) + +``` + +Be advised that, backup is successfully restored in spite of these warnings. + +The rake task runs this as the `gitlab` user which does not have the superuser access to the database. When restore is initiated it will also run as `gitlab` user but it will also try to alter the objects it does not have access to. +Those objects have no influence on the database backup/restore but they give this annoying warning. + +For more information see similar questions on postgresql issue tracker[here](http://www.postgresql.org/message-id/201110220712.30886.adrian.klaver@gmail.com) and [here](http://www.postgresql.org/message-id/2039.1177339749@sss.pgh.pa.us) as well as [stack overflow](http://stackoverflow.com/questions/4368789/error-must-be-owner-of-language-plpgsql). + +## Note +This documentation is for GitLab CE. +We backup GitLab.com and make sure your data is secure, but you can't use these methods to export / backup your data yourself from GitLab.com. + +Issues are stored in the database. They can't be stored in Git itself. + +To migrate your repositories from one server to another with an up-to-date version of +GitLab, you can use the [import rake task](import.md) to do a mass import of the +repository. Note that if you do an import rake task, rather than a backup restore, you +will have all your repositories, but not any other data. diff --git a/doc/administration/raketasks/check.md b/doc/administration/raketasks/check.md new file mode 100644 index 00000000000..3ff3fee6a40 --- /dev/null +++ b/doc/administration/raketasks/check.md @@ -0,0 +1,63 @@ +# Check Rake Tasks + +## Repository Integrity + +Even though Git is very resilient and tries to prevent data integrity issues, +there are times when things go wrong. The following Rake tasks intend to +help GitLab administrators diagnose problem repositories so they can be fixed. + +There are 3 things that are checked to determine integrity. + +1. Git repository file system check ([git fsck](https://git-scm.com/docs/git-fsck)). + This step verifies the connectivity and validity of objects in the repository. +1. Check for `config.lock` in the repository directory. +1. Check for any branch/references lock files in `refs/heads`. + +It's important to note that the existence of `config.lock` or reference locks +alone do not necessarily indicate a problem. Lock files are routinely created +and removed as Git and GitLab perform operations on the repository. They serve +to prevent data integrity issues. However, if a Git operation is interrupted these +locks may not be cleaned up properly. + +The following symptoms may indicate a problem with repository integrity. If users +experience these symptoms you may use the rake tasks described below to determine +exactly which repositories are causing the trouble. + +- Receiving an error when trying to push code - `remote: error: cannot lock ref` +- A 500 error when viewing the GitLab dashboard or when accessing a specific project. + +### Check all GitLab repositories + +This task loops through all repositories on the GitLab server and runs the +3 integrity checks described previously. + +``` +# omnibus-gitlab +sudo gitlab-rake gitlab:repo:check + +# installation from source +bundle exec rake gitlab:repo:check RAILS_ENV=production +``` + +### Check repositories for a specific user + +This task checks all repositories that a specific user has access to. This is important +because sometimes you know which user is experiencing trouble but you don't know +which project might be the cause. + +If the rake task is executed without brackets at the end, you will be prompted +to enter a username. + +```bash +# omnibus-gitlab +sudo gitlab-rake gitlab:user:check_repos +sudo gitlab-rake gitlab:user:check_repos[<username>] + +# installation from source +bundle exec rake gitlab:user:check_repos RAILS_ENV=production +bundle exec rake gitlab:user:check_repos[<username>] RAILS_ENV=production +``` + +Example output: + + diff --git a/doc/administration/raketasks/check_repos_output.png b/doc/administration/raketasks/check_repos_output.png Binary files differnew file mode 100644 index 00000000000..916b1685101 --- /dev/null +++ b/doc/administration/raketasks/check_repos_output.png diff --git a/doc/administration/raketasks/cleanup.md b/doc/administration/raketasks/cleanup.md new file mode 100644 index 00000000000..8fbcbb983e9 --- /dev/null +++ b/doc/administration/raketasks/cleanup.md @@ -0,0 +1,24 @@ +# Cleanup + +## Remove garbage from filesystem. Important! Data loss! + +Remove namespaces(dirs) from `/home/git/repositories` if they don't exist in GitLab database. + +``` +# omnibus-gitlab +sudo gitlab-rake gitlab:cleanup:dirs + +# installation from source +bundle exec rake gitlab:cleanup:dirs RAILS_ENV=production +``` + +Rename repositories from `/home/git/repositories` if they don't exist in GitLab database. +The repositories get a `+orphaned+TIMESTAMP` suffix so that they cannot block new repositories from being created. + +``` +# omnibus-gitlab +sudo gitlab-rake gitlab:cleanup:repos + +# installation from source +bundle exec rake gitlab:cleanup:repos RAILS_ENV=production +``` diff --git a/doc/administration/raketasks/features.md b/doc/administration/raketasks/features.md new file mode 100644 index 00000000000..f9a46193547 --- /dev/null +++ b/doc/administration/raketasks/features.md @@ -0,0 +1,20 @@ +# Features + +## Enable usernames and namespaces for user projects + +This command will enable the namespaces feature introduced in v4.0. It will move every project in its namespace folder. + +Note: + +- Because the **repository location will change**, you will need to **update all your git URLs** to point to the new location. +- Username can be changed at [Profile / Account](/profile/account) + +**Example:** + +Old path: `git@example.org:myrepo.git` + +New path: `git@example.org:username/myrepo.git` or `git@example.org:groupname/myrepo.git` + +``` +bundle exec rake gitlab:enable_namespaces RAILS_ENV=production +``` diff --git a/doc/administration/raketasks/import.md b/doc/administration/raketasks/import.md new file mode 100644 index 00000000000..8a38937062e --- /dev/null +++ b/doc/administration/raketasks/import.md @@ -0,0 +1,68 @@ +# Import bare repositories into your GitLab instance + +## Notes + +- The owner of the project will be the first admin +- The groups will be created as needed +- The owner of the group will be the first admin +- Existing projects will be skipped + +## How to use + +### Create a new folder inside the git repositories path. This will be the name of the new group. + +- For omnibus-gitlab, it is located at: `/var/opt/gitlab/git-data/repositories` by default, unless you changed +it in the `/etc/gitlab/gitlab.rb` file. +- For installations from source, it is usually located at: `/home/git/repositories` or you can see where +your repositories are located by looking at `config/gitlab.yml` under the `gitlab_shell => repos_path` entry. + +New folder needs to have git user ownership and read/write/execute access for git user and its group: + +``` +sudo -u git mkdir /var/opt/gitlab/git-data/repositories/new_group +``` + +If you are using an installation from source, replace `/var/opt/gitlab/git-data` +with `/home/git`. + +### Copy your bare repositories inside this newly created folder: + +``` +sudo cp -r /old/git/foo.git /var/opt/gitlab/git-data/repositories/new_group/ + +# Do this once when you are done copying git repositories +sudo chown -R git:git /var/opt/gitlab/git-data/repositories/new_group/ +``` + +`foo.git` needs to be owned by the git user and git users group. + +If you are using an installation from source, replace `/var/opt/gitlab/git-data` +with `/home/git`. + +### Run the command below depending on your type of installation: + +#### Omnibus Installation + +``` +$ sudo gitlab-rake gitlab:import:repos +``` + +#### Installation from source + +Before running this command you need to change the directory to where your GitLab installation is located: + +``` +$ cd /home/git/gitlab +$ sudo -u git -H bundle exec rake gitlab:import:repos RAILS_ENV=production +``` + +#### Example output + +``` +Processing abcd.git + * Created abcd (abcd.git) +Processing group/xyz.git + * Created Group group (2) + * Created xyz (group/xyz.git) +[...] +``` diff --git a/doc/administration/raketasks/list_repos.md b/doc/administration/raketasks/list_repos.md new file mode 100644 index 00000000000..476428eb4f5 --- /dev/null +++ b/doc/administration/raketasks/list_repos.md @@ -0,0 +1,30 @@ +# Listing repository directories + +You can print a list of all Git repositories on disk managed by +GitLab with the following command: + +``` +# Omnibus +sudo gitlab-rake gitlab:list_repos + +# Source +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:list_repos RAILS_ENV=production +``` + +If you only want to list projects with recent activity you can pass +a date with the 'SINCE' environment variable. The time you specify +is parsed by the Rails [TimeZone#parse +function](http://api.rubyonrails.org/classes/ActiveSupport/TimeZone.html#method-i-parse). + +``` +# Omnibus +sudo gitlab-rake gitlab:list_repos SINCE='Sep 1 2015' + +# Source +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:list_repos RAILS_ENV=production SINCE='Sep 1 2015' +``` + +Note that the projects listed are NOT sorted by activity; they use +the default ordering of the GitLab Rails application. diff --git a/doc/administration/raketasks/maintenance.md b/doc/administration/raketasks/maintenance.md new file mode 100644 index 00000000000..d9dce2af480 --- /dev/null +++ b/doc/administration/raketasks/maintenance.md @@ -0,0 +1,169 @@ +# Maintenance + +## Gather information about GitLab and the system it runs on + +This command gathers information about your GitLab installation and the System it runs on. These may be useful when asking for help or reporting issues. + +``` +# omnibus-gitlab +sudo gitlab-rake gitlab:env:info + +# installation from source +bundle exec rake gitlab:env:info RAILS_ENV=production +``` + +Example output: + +``` +System information +System: Debian 7.8 +Current User: git +Using RVM: no +Ruby Version: 2.1.5p273 +Gem Version: 2.4.3 +Bundler Version: 1.7.6 +Rake Version: 10.3.2 +Sidekiq Version: 2.17.8 + +GitLab information +Version: 7.7.1 +Revision: 41ab9e1 +Directory: /home/git/gitlab +DB Adapter: postgresql +URL: https://gitlab.example.com +HTTP Clone URL: https://gitlab.example.com/some-project.git +SSH Clone URL: git@gitlab.example.com:some-project.git +Using LDAP: no +Using Omniauth: no + +GitLab Shell +Version: 2.4.1 +Repositories: /home/git/repositories/ +Hooks: /home/git/gitlab-shell/hooks/ +Git: /usr/bin/git +``` + +## Check GitLab configuration + +Runs the following rake tasks: + +- `gitlab:gitlab_shell:check` +- `gitlab:sidekiq:check` +- `gitlab:app:check` + +It will check that each component was setup according to the installation guide and suggest fixes for issues found. + +You may also have a look at our [Trouble Shooting Guide](https://github.com/gitlabhq/gitlab-public-wiki/wiki/Trouble-Shooting-Guide). + +``` +# omnibus-gitlab +sudo gitlab-rake gitlab:check + +# installation from source +bundle exec rake gitlab:check RAILS_ENV=production +``` + +NOTE: Use SANITIZE=true for gitlab:check if you want to omit project names from the output. + +Example output: + +``` +Checking Environment ... + +Git configured for git user? ... yes +Has python2? ... yes +python2 is supported version? ... yes + +Checking Environment ... Finished + +Checking GitLab Shell ... + +GitLab Shell version? ... OK (1.2.0) +Repo base directory exists? ... yes +Repo base directory is a symlink? ... no +Repo base owned by git:git? ... yes +Repo base access is drwxrws---? ... yes +post-receive hook up-to-date? ... yes +post-receive hooks in repos are links: ... yes + +Checking GitLab Shell ... Finished + +Checking Sidekiq ... + +Running? ... yes + +Checking Sidekiq ... Finished + +Checking GitLab ... + +Database config exists? ... yes +Database is SQLite ... no +All migrations up? ... yes +GitLab config exists? ... yes +GitLab config outdated? ... no +Log directory writable? ... yes +Tmp directory writable? ... yes +Init script exists? ... yes +Init script up-to-date? ... yes +Redis version >= 2.0.0? ... yes + +Checking GitLab ... Finished +``` + +## Rebuild authorized_keys file + +In some case it is necessary to rebuild the `authorized_keys` file. + +For Omnibus-packages: +``` +sudo gitlab-rake gitlab:shell:setup +``` + +For installations from source: +``` +cd /home/git/gitlab +sudo -u git -H bundle exec rake gitlab:shell:setup RAILS_ENV=production +``` + +``` +This will rebuild an authorized_keys file. +You will lose any data stored in authorized_keys file. +Do you want to continue (yes/no)? yes +``` + +## Clear redis cache + +If for some reason the dashboard shows wrong information you might want to +clear Redis' cache. + +For Omnibus-packages: +``` +sudo gitlab-rake cache:clear +``` + +For installations from source: +``` +cd /home/git/gitlab +sudo -u git -H bundle exec rake cache:clear RAILS_ENV=production +``` + +## Precompile the assets + +Sometimes during version upgrades you might end up with some wrong CSS or +missing some icons. In that case, try to precompile the assets again. + +Note that this only applies to source installations and does NOT apply to +omnibus packages. + +For installations from source: +``` +cd /home/git/gitlab +sudo -u git -H bundle exec rake assets:precompile RAILS_ENV=production +``` + +For omnibus versions, the unoptimized assets (JavaScript, CSS) are frozen at +the release of upstream GitLab. The omnibus version includes optimized versions +of those assets. Unless you are modifying the JavaScript / CSS code on your +production machine after installing the package, there should be no reason to redo +rake assets:precompile on the production machine. If you suspect that assets +have been corrupted, you should reinstall the omnibus package. diff --git a/doc/administration/raketasks/user_management.md b/doc/administration/raketasks/user_management.md new file mode 100644 index 00000000000..629d38efc53 --- /dev/null +++ b/doc/administration/raketasks/user_management.md @@ -0,0 +1,72 @@ +# User management + +## Add user as a developer to all projects + +```bash +# omnibus-gitlab +sudo gitlab-rake gitlab:import:user_to_projects[username@domain.tld] + +# installation from source +bundle exec rake gitlab:import:user_to_projects[username@domain.tld] RAILS_ENV=production +``` + +## Add all users to all projects + +Notes: + +- admin users are added as masters + +```bash +# omnibus-gitlab +sudo gitlab-rake gitlab:import:all_users_to_all_projects + +# installation from source +bundle exec rake gitlab:import:all_users_to_all_projects RAILS_ENV=production +``` + +## Add user as a developer to all groups + +```bash +# omnibus-gitlab +sudo gitlab-rake gitlab:import:user_to_groups[username@domain.tld] + +# installation from source +bundle exec rake gitlab:import:user_to_groups[username@domain.tld] RAILS_ENV=production +``` + +## Add all users to all groups + +Notes: + +- admin users are added as owners so they can add additional users to the group + +```bash +# omnibus-gitlab +sudo gitlab-rake gitlab:import:all_users_to_all_groups + +# installation from source +bundle exec rake gitlab:import:all_users_to_all_groups RAILS_ENV=production +``` + +## Maintain tight control over the number of active users on your GitLab installation + +- Enable this setting to keep new users blocked until they have been cleared by the admin (default: false). + + +``` +block_auto_created_users: false +``` + +## Disable Two-factor Authentication (2FA) for all users + +This task will disable 2FA for all users that have it enabled. This can be +useful if GitLab's `.secret` file has been lost and users are unable to login, +for example. + +```bash +# omnibus-gitlab +sudo gitlab-rake gitlab:two_factor:disable_for_all_users + +# installation from source +bundle exec rake gitlab:two_factor:disable_for_all_users RAILS_ENV=production +``` diff --git a/doc/administration/raketasks/web_hooks.md b/doc/administration/raketasks/web_hooks.md new file mode 100644 index 00000000000..2ebf7c48f4e --- /dev/null +++ b/doc/administration/raketasks/web_hooks.md @@ -0,0 +1,45 @@ +# Webhooks + +## Add a webhook for **ALL** projects: + + # omnibus-gitlab + sudo gitlab-rake gitlab:web_hook:add URL="http://example.com/hook" + # source installations + bundle exec rake gitlab:web_hook:add URL="http://example.com/hook" RAILS_ENV=production + +## Add a webhook for projects in a given **NAMESPACE**: + + # omnibus-gitlab + sudo gitlab-rake gitlab:web_hook:add URL="http://example.com/hook" NAMESPACE=acme + # source installations + bundle exec rake gitlab:web_hook:add URL="http://example.com/hook" NAMESPACE=acme RAILS_ENV=production + +## Remove a webhook from **ALL** projects using: + + # omnibus-gitlab + sudo gitlab-rake gitlab:web_hook:rm URL="http://example.com/hook" + # source installations + bundle exec rake gitlab:web_hook:rm URL="http://example.com/hook" RAILS_ENV=production + +## Remove a webhook from projects in a given **NAMESPACE**: + + # omnibus-gitlab + sudo gitlab-rake gitlab:web_hook:rm URL="http://example.com/hook" NAMESPACE=acme + # source installations + bundle exec rake gitlab:web_hook:rm URL="http://example.com/hook" NAMESPACE=acme RAILS_ENV=production + +## List **ALL** webhooks: + + # omnibus-gitlab + sudo gitlab-rake gitlab:web_hook:list + # source installations + bundle exec rake gitlab:web_hook:list RAILS_ENV=production + +## List the webhooks from projects in a given **NAMESPACE**: + + # omnibus-gitlab + sudo gitlab-rake gitlab:web_hook:list NAMESPACE=/ + # source installations + bundle exec rake gitlab:web_hook:list NAMESPACE=/ RAILS_ENV=production + +> Note: `/` is the global namespace. |
