summaryrefslogtreecommitdiff
path: root/doc/raketasks
diff options
context:
space:
mode:
Diffstat (limited to 'doc/raketasks')
-rw-r--r--doc/raketasks/README.md2
-rw-r--r--doc/raketasks/backup_restore.md93
-rw-r--r--doc/raketasks/cleanup.md16
-rw-r--r--doc/raketasks/features.md2
-rw-r--r--doc/raketasks/import.md16
-rw-r--r--doc/raketasks/web_hooks.md17
6 files changed, 99 insertions, 47 deletions
diff --git a/doc/raketasks/README.md b/doc/raketasks/README.md
index dcc96507676..ad86555fc17 100644
--- a/doc/raketasks/README.md
+++ b/doc/raketasks/README.md
@@ -12,7 +12,7 @@ comments: false
- [General Maintenance](../administration/raketasks/maintenance.md) and self-checks
- [User management](user_management.md)
- [Webhooks](web_hooks.md)
-- [Import](import.md) of git repositories in bulk
+- [Import](import.md) of Git repositories in bulk
- [Rebuild authorized_keys file](../administration/raketasks/maintenance.md#rebuild-authorized_keys-file) task for administrators
- [Migrate Uploads](../administration/raketasks/uploads/migrate.md)
- [Sanitize Uploads](../administration/raketasks/uploads/sanitize.md)
diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md
index 092b4375208..336000f6cb9 100644
--- a/doc/raketasks/backup_restore.md
+++ b/doc/raketasks/backup_restore.md
@@ -77,7 +77,7 @@ You are highly advised to [read about storing configuration files](#storing-conf
Use this command if you've installed GitLab with the Omnibus package:
```sh
-sudo gitlab-rake gitlab:backup:create
+sudo gitlab-backup create
```
Use this if you've installed GitLab from source:
@@ -89,12 +89,12 @@ sudo -u git -H bundle exec rake gitlab:backup:create RAILS_ENV=production
If you are running GitLab within a Docker container, you can run the backup from the host:
```sh
-docker exec -t <container name> gitlab-rake gitlab:backup:create
+docker exec -t <container name> gitlab-backup create
```
If you are using the [GitLab helm chart](https://gitlab.com/charts/gitlab) on a
Kubernetes cluster, you can run the backup task using `backup-utility` script on
-the gitlab task runner pod via `kubectl`. Refer to [backing up a GitLab installation](https://gitlab.com/charts/gitlab/blob/master/doc/backup-restore/backup.md#backing-up-a-gitlab-installation) for more details:
+the GitLab task runner pod via `kubectl`. Refer to [backing up a GitLab installation](https://gitlab.com/charts/gitlab/blob/master/doc/backup-restore/backup.md#backing-up-a-gitlab-installation) for more details:
```sh
kubectl exec -it <gitlab task-runner pod> backup-utility
@@ -199,7 +199,7 @@ To use the `copy` strategy instead of the default streaming strategy, specify
`STRATEGY=copy` in the Rake task command. For example:
```sh
-sudo gitlab-rake gitlab:backup:create STRATEGY=copy
+sudo gitlab-backup create STRATEGY=copy
```
### Backup filename
@@ -207,7 +207,7 @@ sudo gitlab-rake gitlab:backup:create STRATEGY=copy
By default a backup file is created according to the specification in [the Backup timestamp](#backup-timestamp) section above. You can however override the `[TIMESTAMP]` part of the filename by setting the `BACKUP` environment variable. For example:
```sh
-sudo gitlab-rake gitlab:backup:create BACKUP=dump
+sudo gitlab-backup create BACKUP=dump
```
The resulting file will then be `dump_gitlab_backup.tar`. This is useful for systems that make use of rsync and incremental backups, and will result in considerably faster transfer speeds.
@@ -219,7 +219,7 @@ To make sure the generated archive is intelligently transferable by rsync, the `
Note that the `--rsyncable` option in `gzip` is not guaranteed to be available on all distributions. To verify that it is available in your distribution you can run `gzip --help` or consult the man pages.
```sh
-sudo gitlab-rake gitlab:backup:create BACKUP=dump GZIP_RSYNCABLE=yes
+sudo gitlab-backup create BACKUP=dump GZIP_RSYNCABLE=yes
```
### Excluding specific directories from the backup
@@ -244,7 +244,7 @@ will be skipped during a backup.
For Omnibus GitLab packages:
```sh
-sudo gitlab-rake gitlab:backup:create SKIP=db,uploads
+sudo gitlab-backup create SKIP=db,uploads
```
For installations from source:
@@ -315,8 +315,6 @@ if you see `411 Length Required` errors after attempting to upload, you may
need to downgrade the `aws_signature_version` value from the default value to
2 [due to this issue](https://github.com/fog/fog-aws/issues/428).
----
-
For installations from source:
1. Edit `home/git/gitlab/config/gitlab.yml`:
@@ -428,8 +426,6 @@ For Omnibus GitLab packages:
1. [Reconfigure GitLab] for the changes to take effect
----
-
For installations from source:
1. Edit `home/git/gitlab/config/gitlab.yml`:
@@ -452,8 +448,8 @@ Note: This option only works for remote storage. If you want to group your backu
you can pass a `DIRECTORY` environment variable:
```
-sudo gitlab-rake gitlab:backup:create DIRECTORY=daily
-sudo gitlab-rake gitlab:backup:create DIRECTORY=weekly
+sudo gitlab-backup create DIRECTORY=daily
+sudo gitlab-backup create DIRECTORY=weekly
```
### Uploading to locally mounted shares
@@ -462,7 +458,7 @@ You may also send backups to a mounted share (`NFS` / `CIFS` / `SMB` / etc.) by
using the Fog [`Local`](https://github.com/fog/fog-local#usage) storage provider.
The directory pointed to by the `local_root` key **must** be owned by the `git`
user **when mounted** (mounting with the `uid=` of the `git` user for `CIFS` and
-`SMB`) or the user that you are executing the backup tasks under (for omnibus
+`SMB`) or the user that you are executing the backup tasks under (for Omnibus
packages, this is the `git` user).
The `backup_upload_remote_directory` **must** be set in addition to the
@@ -490,8 +486,6 @@ For Omnibus GitLab packages:
1. [Reconfigure GitLab] for the changes to take effect.
----
-
For installations from source:
1. Edit `home/git/gitlab/config/gitlab.yml`:
@@ -513,7 +507,7 @@ For installations from source:
### Backup archive permissions
The backup archives created by GitLab (`1393513186_2014_02_27_gitlab_backup.tar`)
-will have owner/group git:git and 0600 permissions by default.
+will have owner/group `git`/`git` and 0600 permissions by default.
This is meant to avoid other system users reading GitLab's data.
If you need the backup archives to have different permissions you can use the 'archive_permissions' setting.
@@ -527,8 +521,6 @@ For Omnibus GitLab packages:
1. [Reconfigure GitLab] for the changes to take effect.
----
-
For installations from source:
1. Edit `/home/git/gitlab/config/gitlab.yml`:
@@ -574,14 +566,12 @@ crontab -e
There, add the following line to schedule the backup for everyday at 2 AM:
```
-0 2 * * * /opt/gitlab/bin/gitlab-rake gitlab:backup:create CRON=1
+0 2 * * * /opt/gitlab/bin/gitlab-backup create CRON=1
```
You may also want to set a limited lifetime for backups to prevent regular
backups using all your disk space.
----
-
For installations from source:
1. Edit `home/git/gitlab/config/gitlab.yml`:
@@ -624,7 +614,7 @@ GitLab that you created it on, for example CE 9.1.0.
You need to have a working GitLab installation before you can perform
a restore. This is mainly because the system user performing the
-restore actions ('git') is usually not allowed to create or delete
+restore actions (`git`) is usually not allowed to create or delete
the SQL database it needs to import data into ('gitlabhq_production').
All existing data will be either erased (SQL) or moved to a separate
directory (repositories, uploads).
@@ -736,14 +726,15 @@ restore:
```shell
# This command will overwrite the contents of your GitLab database!
-sudo gitlab-rake gitlab:backup:restore BACKUP=1493107454_2018_04_25_10.6.4-ce
+sudo gitlab-backup restore BACKUP=1493107454_2018_04_25_10.6.4-ce
```
Next, restore `/etc/gitlab/gitlab-secrets.json` if necessary as mentioned above.
-Restart and check GitLab:
+Reconfigure, restart and check GitLab:
```shell
+sudo gitlab-ctl reconfigure
sudo gitlab-ctl restart
sudo gitlab-rake gitlab:check SANITIZE=true
```
@@ -769,7 +760,7 @@ backup location (default location is `/var/opt/gitlab/backups`).
For docker installations, the restore task can be run from host:
```sh
-docker exec -it <name of container> gitlab-rake gitlab:backup:restore
+docker exec -it <name of container> gitlab-backup restore
```
The GitLab helm chart uses a different process, documented in
@@ -782,14 +773,14 @@ In this case you can consider using filesystem snapshots as part of your backup
Example: Amazon EBS
-> A GitLab server using omnibus-gitlab hosted on Amazon AWS.
+> A GitLab server using Omnibus GitLab hosted on Amazon AWS.
> An EBS drive containing an ext4 filesystem is mounted at `/var/opt/gitlab`.
> In this case you could make an application backup by taking an EBS snapshot.
> The backup includes all repositories, uploads and Postgres data.
Example: LVM snapshots + rsync
-> A GitLab server using omnibus-gitlab, with an LVM logical volume mounted at `/var/opt/gitlab`.
+> A GitLab server using Omnibus GitLab, with an LVM logical volume mounted at `/var/opt/gitlab`.
> Replicating the `/var/opt/gitlab` directory using rsync would not be reliable because too many files would change while rsync is running.
> Instead of rsync-ing `/var/opt/gitlab`, we create a temporary LVM snapshot, which we mount as a read-only filesystem at `/mnt/gitlab_backup`.
> Now we can have a longer running rsync job which will create a consistent replica on the remote server.
@@ -813,7 +804,8 @@ will have all your repositories, but not any other data.
## Troubleshooting
-### Restoring database backup using omnibus packages outputs warnings
+### Restoring database backup using Omnibus packages outputs warnings
+
If you are using backup restore procedures you might encounter the following warnings:
```
@@ -842,7 +834,7 @@ including (but not restricted to):
- [CI/CD variables](../ci/variables/README.md)
- [Kubernetes / GCP integration](../user/project/clusters/index.md)
-- [Custom Pages domains](../user/project/pages/getting_started_part_three.md)
+- [Custom Pages domains](../user/project/pages/custom_domains_ssl_tls_certification/index.md)
- [Project error tracking](../user/project/operations/error_tracking.md)
- [Runner authentication](../ci/runners/README.md)
- [Project mirroring](../workflow/repository_mirroring.md)
@@ -931,6 +923,29 @@ backup beforehand.
UPDATE ci_runners SET token = null, token_encrypted = null;
```
+#### Reset pending pipeline jobs
+
+1. Enter the DB console:
+
+ For Omnibus GitLab packages:
+
+ ```sh
+ sudo gitlab-rails dbconsole
+ ```
+
+ For installations from source:
+
+ ```sh
+ sudo -u git -H bundle exec rails dbconsole RAILS_ENV=production
+ ```
+
+1. Clear all the tokens for pending jobs:
+
+ ```sql
+ -- Clear build tokens
+ UPDATE ci_builds SET token = null, token_encrypted = null;
+ ```
+
A similar strategy can be employed for the remaining features - by removing the
data that cannot be decrypted, GitLab can be brought back into working order,
and the lost data can be manually replaced.
@@ -968,3 +983,21 @@ want to run the chown against your custom location instead of
[reconfigure GitLab]: ../administration/restart_gitlab.md#omnibus-gitlab-reconfigure
[restart GitLab]: ../administration/restart_gitlab.md#installations-from-source
+
+### Backup fails to complete with Gzip error
+
+While running the backup, you may receive a gzip error:
+
+```sh
+sudo /opt/gitlab/bin/gitlab-backup create
+Dumping ...
+...
+gzip: stdout: Input/output error
+
+Backup failed
+```
+
+If this happens, check the following:
+
+1. Confirm there is sufficent diskspace for the gzip operation.
+1. If NFS is being used, check if the mount option `timeo` is set. The default is `600`, and changing this to smaller values have resulted in this error.
diff --git a/doc/raketasks/cleanup.md b/doc/raketasks/cleanup.md
index f880f31c39e..f84d29cca9a 100644
--- a/doc/raketasks/cleanup.md
+++ b/doc/raketasks/cleanup.md
@@ -1,6 +1,10 @@
# Cleanup
-## Remove garbage from filesystem. Important! Data loss!
+## Remove garbage from filesystem
+
+DANGER: **Danger:**
+The commands below will remove data permanently from your GitLab instance. Only use
+these commands if you are 100% certain that it is safe to delete this data.
Remove namespaces(dirs) from all repository storage paths if they don't exist in GitLab database.
@@ -137,3 +141,13 @@ level with `NICENESS`. Below are the valid levels, but consult
- `1` or `Realtime`
- `2` or `Best-effort` (default)
- `3` or `Idle`
+
+## Remove expired ActiveSession lookup keys
+
+```
+# omnibus-gitlab
+sudo gitlab-rake gitlab:cleanup:sessions:active_sessions_lookup_keys
+
+# installation from source
+bundle exec rake gitlab:cleanup:sessions:active_sessions_lookup_keys RAILS_ENV=production
+```
diff --git a/doc/raketasks/features.md b/doc/raketasks/features.md
index 57c16f110e4..c06800a2aa3 100644
--- a/doc/raketasks/features.md
+++ b/doc/raketasks/features.md
@@ -6,7 +6,7 @@ This command will enable the namespaces feature introduced in v4.0. It will move
Note:
-- Because the **repository location will change**, you will need to **update all your git URLs** to point to the new location.
+- Because the **repository location will change**, you will need to **update all your Git URLs** to point to the new location.
- Username can be changed at **Profile ➔ Account**.
**Example:**
diff --git a/doc/raketasks/import.md b/doc/raketasks/import.md
index b59c06a24ea..d93e7241fda 100644
--- a/doc/raketasks/import.md
+++ b/doc/raketasks/import.md
@@ -11,17 +11,17 @@
## How to use
-### Create a new folder to import your Git repositories from.
+### Create a new folder to import your Git repositories from
-The new folder needs to have git user ownership and read/write/execute access for git user and its group:
+The new folder needs to have Git user ownership and read/write/execute access for Git user and its group:
```
sudo -u git mkdir -p /var/opt/gitlab/git-data/repository-import-<date>/new_group
```
-### Copy your bare repositories inside this newly created folder:
+### Copy your bare repositories inside this newly created folder
-- Any .git repositories found on any of the subfolders will be imported as projects
+- Any `.git` repositories found on any of the subfolders will be imported as projects
- Groups will be created as needed, these could be nested folders. Example:
If we copy the repos to `/var/opt/gitlab/git-data/repository-import-<date>`, and repo A needs to be under the groups G1 and G2, it will
@@ -34,11 +34,11 @@ sudo cp -r /old/git/foo.git /var/opt/gitlab/git-data/repository-import-<date>/ne
sudo chown -R git:git /var/opt/gitlab/git-data/repository-import-<date>
```
-`foo.git` needs to be owned by the git user and git users group.
+`foo.git` needs to be owned by the `git` user and `git` users group.
If you are using an installation from source, replace `/var/opt/gitlab/` with `/home/git`.
-### Run the command below depending on your type of installation:
+### Run the command below depending on your type of installation
#### Omnibus Installation
@@ -96,11 +96,11 @@ Importing bare repositories from hashed storage is unsupported.
To support importing bare repositories from hashed storage, GitLab 10.4 and
later stores the full project path with each repository, in a special section of
-the git repository's config file. This section is formatted as follows:
+the Git repository's config file. This section is formatted as follows:
```
[gitlab]
- fullpath = gitlab-org/gitlab-ce
+ fullpath = gitlab-org/gitlab-ce
```
However, existing repositories were not migrated to include this path.
diff --git a/doc/raketasks/web_hooks.md b/doc/raketasks/web_hooks.md
index a498e9793c1..d9a042ec8fe 100644
--- a/doc/raketasks/web_hooks.md
+++ b/doc/raketasks/web_hooks.md
@@ -1,6 +1,6 @@
# Webhooks administration **(CORE ONLY)**
-## Add a webhook for **ALL** projects:
+## Add a webhook for **ALL** projects
```sh
# omnibus-gitlab
@@ -9,7 +9,7 @@ sudo gitlab-rake gitlab:web_hook:add URL="http://example.com/hook"
bundle exec rake gitlab:web_hook:add URL="http://example.com/hook" RAILS_ENV=production
```
-## Add a webhook for projects in a given **NAMESPACE**:
+## Add a webhook for projects in a given **NAMESPACE**
```sh
# omnibus-gitlab
@@ -18,7 +18,7 @@ sudo gitlab-rake gitlab:web_hook:add URL="http://example.com/hook" NAMESPACE=acm
bundle exec rake gitlab:web_hook:add URL="http://example.com/hook" NAMESPACE=acme RAILS_ENV=production
```
-## Remove a webhook from **ALL** projects using:
+## Remove a webhook from **ALL** projects using
```sh
# omnibus-gitlab
@@ -27,7 +27,7 @@ sudo gitlab-rake gitlab:web_hook:rm URL="http://example.com/hook"
bundle exec rake gitlab:web_hook:rm URL="http://example.com/hook" RAILS_ENV=production
```
-## Remove a webhook from projects in a given **NAMESPACE**:
+## Remove a webhook from projects in a given **NAMESPACE**
```sh
# omnibus-gitlab
@@ -36,7 +36,7 @@ sudo gitlab-rake gitlab:web_hook:rm URL="http://example.com/hook" NAMESPACE=acme
bundle exec rake gitlab:web_hook:rm URL="http://example.com/hook" NAMESPACE=acme RAILS_ENV=production
```
-## List **ALL** webhooks:
+## List **ALL** webhooks
```sh
# omnibus-gitlab
@@ -45,7 +45,7 @@ sudo gitlab-rake gitlab:web_hook:list
bundle exec rake gitlab:web_hook:list RAILS_ENV=production
```
-## List the webhooks from projects in a given **NAMESPACE**:
+## List the webhooks from projects in a given **NAMESPACE**
```sh
# omnibus-gitlab
@@ -53,3 +53,8 @@ sudo gitlab-rake gitlab:web_hook:list NAMESPACE=acme
# source installations
bundle exec rake gitlab:web_hook:list NAMESPACE=acme RAILS_ENV=production
```
+
+## Local requests in webhooks
+
+[Requests to local network by webhooks](../security/webhooks.md) can be allowed
+or blocked by an administrator.
View Lorry Controller Status