diff options
Diffstat (limited to 'doc/administration/raketasks')
| -rw-r--r-- | doc/administration/raketasks/check.md | 12 | ||||
| -rw-r--r-- | doc/administration/raketasks/doctor.md | 2 | ||||
| -rw-r--r-- | doc/administration/raketasks/geo.md | 2 | ||||
| -rw-r--r-- | doc/administration/raketasks/github_import.md | 14 | ||||
| -rw-r--r-- | doc/administration/raketasks/ldap.md | 97 | ||||
| -rw-r--r-- | doc/administration/raketasks/maintenance.md | 16 | ||||
| -rw-r--r-- | doc/administration/raketasks/praefect.md | 2 | ||||
| -rw-r--r-- | doc/administration/raketasks/project_import_export.md | 8 | ||||
| -rw-r--r-- | doc/administration/raketasks/storage.md | 15 | ||||
| -rw-r--r-- | doc/administration/raketasks/uploads/migrate.md | 20 | ||||
| -rw-r--r-- | doc/administration/raketasks/uploads/sanitize.md | 12 |
11 files changed, 147 insertions, 53 deletions
diff --git a/doc/administration/raketasks/check.md b/doc/administration/raketasks/check.md index f61a3107b3d..25869fd425d 100644 --- a/doc/administration/raketasks/check.md +++ b/doc/administration/raketasks/check.md @@ -1,7 +1,7 @@ --- -stage: none -group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +stage: Enablement +group: Distribution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Integrity check Rake task **(CORE ONLY)** @@ -56,7 +56,7 @@ sudo -u git -H bundle exec rake gitlab:git:fsck RAILS_ENV=production Various types of files can be uploaded to a GitLab installation by users. These integrity checks can detect missing files. Additionally, for locally stored files, checksums are generated and stored in the database upon upload, -and these checks will verify them against current files. +and these checks verify them against current files. Currently, integrity checks are supported for the following types of file: @@ -137,8 +137,8 @@ Done! ## LDAP check -The LDAP check Rake task will test the bind DN and password credentials -(if configured) and will list a sample of LDAP users. This task is also +The LDAP check Rake task tests the bind DN and password credentials +(if configured) and lists a sample of LDAP users. This task is also executed as part of the `gitlab:check` task, but can run independently. See [LDAP Rake Tasks - LDAP Check](ldap.md#check) for details. diff --git a/doc/administration/raketasks/doctor.md b/doc/administration/raketasks/doctor.md index e6a6751f199..c80f580cce6 100644 --- a/doc/administration/raketasks/doctor.md +++ b/doc/administration/raketasks/doctor.md @@ -1,7 +1,7 @@ --- stage: Enablement group: Geo -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Doctor Rake tasks **(CORE ONLY)** diff --git a/doc/administration/raketasks/geo.md b/doc/administration/raketasks/geo.md index 0ee6fd8fc75..492fe4b52ca 100644 --- a/doc/administration/raketasks/geo.md +++ b/doc/administration/raketasks/geo.md @@ -1,7 +1,7 @@ --- stage: Enablement group: Geo -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Geo Rake Tasks **(PREMIUM ONLY)** diff --git a/doc/administration/raketasks/github_import.md b/doc/administration/raketasks/github_import.md index d549110c32f..5c0bc721a9a 100644 --- a/doc/administration/raketasks/github_import.md +++ b/doc/administration/raketasks/github_import.md @@ -1,7 +1,7 @@ --- -stage: none -group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +stage: Enablement +group: Distribution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # GitHub import **(CORE ONLY)** @@ -10,7 +10,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w To retrieve and import GitHub repositories, you need a [GitHub personal access token](https://github.com/settings/tokens). A username should be passed as the second argument to the Rake task, -which will become the owner of the project. You can resume an import +which becomes the owner of the project. You can resume an import with the same command. Bear in mind that the syntax is very specific. Remove any spaces within the argument block and @@ -20,7 +20,7 @@ before/after the brackets. Also, some shells (for example, `zsh`) can interpret ## Caveats If the GitHub [rate limit](https://developer.github.com/v3/#rate-limiting) is reached while importing, -the importing process will wait (`sleep()`) until it can continue importing. +the importing process waits (`sleep()`) until it can continue importing. ## Importing multiple projects @@ -35,8 +35,8 @@ bundle exec rake "import:github[access_token,root,foo/bar]" RAILS_ENV=production ``` In this case, `access_token` is your GitHub personal access token, `root` -is your GitLab username, and `foo/bar` is the new GitLab namespace/project that -will get created from your GitHub project. Subgroups are also possible: `foo/foo/bar`. +is your GitLab username, and `foo/bar` is the new GitLab namespace/project +created from your GitHub project. Subgroups are also possible: `foo/foo/bar`. ## Importing a single project diff --git a/doc/administration/raketasks/ldap.md b/doc/administration/raketasks/ldap.md index 821a72291fd..7dafda89e3c 100644 --- a/doc/administration/raketasks/ldap.md +++ b/doc/administration/raketasks/ldap.md @@ -1,7 +1,7 @@ --- stage: Manage group: Access -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # LDAP Rake tasks **(CORE ONLY)** @@ -42,7 +42,7 @@ The following task will run a [group sync](../auth/ldap/index.md#group-sync) imm when you'd like to update all configured group memberships against LDAP without waiting for the next scheduled group sync to be run. -NOTE: **Note:** +NOTE: If you'd like to change the frequency at which a group sync is performed, [adjust the cron schedule](../auth/ldap/index.md#adjusting-ldap-group-sync-schedule) instead. @@ -147,3 +147,96 @@ confirmation dialog: ```shell sudo gitlab-rake gitlab:ldap:rename_provider[old_provider,new_provider] force=yes ``` + +## Secrets + +GitLab can use [LDAP configuration secrets](../auth/ldap/index.md#using-encrypted-credentials) to read from an encrypted file. The following Rake tasks are provided for updating the contents of the encrypted file. + +### Show secret + +Show the contents of the current LDAP secrets. + +**Omnibus Installation** + +```shell +sudo gitlab-rake gitlab:ldap:secret:show +``` + +**Source Installation** + +```shell +bundle exec rake gitlab:ldap:secret:show RAILS_ENV=production +``` + +**Example output:** + +```plaintext +main: + password: '123' + user_bn: 'gitlab-adm' +``` + +### Edit secret + +Opens the secret contents in your editor, and writes the resulting content to the encrypted secret file when you exit. + +**Omnibus Installation** + +```shell +sudo gitlab-rake gitlab:ldap:secret:edit EDITOR=vim +``` + +**Source Installation** + +```shell +bundle exec rake gitlab:ldap:secret:edit RAILS_ENV=production EDITOR=vim +``` + +### Write raw secret + +Write new secret content by providing it on STDIN. + +**Omnibus Installation** + +```shell +echo -e "main:\n password: '123'" | sudo gitlab-rake gitlab:ldap:secret:write +``` + +**Source Installation** + +```shell +echo -e "main:\n password: '123'" | bundle exec rake gitlab:ldap:secret:write RAILS_ENV=production +``` + +### Secrets examples + +**Editor example** + +The write task can be used in cases where the edit command does not work with your editor: + +```shell +# Write the existing secret to a plaintext file +sudo gitlab-rake gitlab:ldap:secret:show > ldap.yaml +# Edit the ldap file in your editor +... +# Re-encrypt the file +cat ldap.yaml | sudo gitlab-rake gitlab:ldap:secret:write +# Remove the plaintext file +rm ldap.yaml +``` + +**KMS integration example** + +It can also be used as a receiving application for content encrypted with a KMS: + +```shell +gcloud kms decrypt --key my-key --keyring my-test-kms --plaintext-file=- --ciphertext-file=my-file --location=us-west1 | sudo gitlab-rake gitlab:ldap:secret:write +``` + +**gcloud secret integration example** + +It can also be used as a receiving application for secrets out of gcloud: + +```shell +gcloud secrets versions access latest --secret="my-test-secret" > $1 | sudo gitlab-rake gitlab:ldap:secret:write +``` diff --git a/doc/administration/raketasks/maintenance.md b/doc/administration/raketasks/maintenance.md index b93442be0a1..26381434ad4 100644 --- a/doc/administration/raketasks/maintenance.md +++ b/doc/administration/raketasks/maintenance.md @@ -1,7 +1,7 @@ --- -stage: none -group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +stage: Enablement +group: Distribution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Maintenance Rake tasks **(CORE ONLY)** @@ -107,8 +107,8 @@ The `gitlab:check` Rake task runs the following Rake tasks: - `gitlab:sidekiq:check` - `gitlab:app:check` -It will check that each component was set up according to the installation guide and suggest fixes -for issues found. This command must be run from your application server and will not work correctly on +It checks that each component was set up according to the installation guide and suggest fixes +for issues found. This command must be run from your application server and doesn't work correctly on component servers like [Gitaly](../gitaly/index.md#run-gitaly-on-its-own-server). You may also have a look at our troubleshooting guides for: @@ -272,7 +272,7 @@ clear it. To clear all exclusive leases: -DANGER: **Warning:** +WARNING: Don't run it while GitLab or Sidekiq is running ```shell @@ -300,7 +300,7 @@ To check the status of specific migrations, you can use the following Rake task: sudo gitlab-rake db:migrate:status ``` -This will output a table with a `Status` of `up` or `down` for +This outputs a table with a `Status` of `up` or `down` for each Migration ID. ```shell @@ -313,7 +313,7 @@ database: gitlabhq_production ## Run incomplete database migrations -Database migrations can be stuck in an incomplete state. That is, they'll have a `down` +Database migrations can be stuck in an incomplete state, with a `down` status in the output of the `sudo gitlab-rake db:migrate:status` command. To complete these migrations, use the following Rake task: diff --git a/doc/administration/raketasks/praefect.md b/doc/administration/raketasks/praefect.md index ca9c2065904..17c3e44bb7e 100644 --- a/doc/administration/raketasks/praefect.md +++ b/doc/administration/raketasks/praefect.md @@ -1,7 +1,7 @@ --- stage: Create group: Gitaly -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- diff --git a/doc/administration/raketasks/project_import_export.md b/doc/administration/raketasks/project_import_export.md index d83ab6e5cee..0ea0bb3c28f 100644 --- a/doc/administration/raketasks/project_import_export.md +++ b/doc/administration/raketasks/project_import_export.md @@ -1,7 +1,7 @@ --- -stage: none -group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +stage: Enablement +group: Distribution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Project import/export administration **(CORE ONLY)** @@ -36,7 +36,7 @@ sudo gitlab-rake gitlab:import_export:version bundle exec rake gitlab:import_export:version RAILS_ENV=production ``` -The current list of DB tables that will be exported can be listed by using the following command: +The current list of DB tables to export can be listed by using the following command: ```shell # Omnibus installations diff --git a/doc/administration/raketasks/storage.md b/doc/administration/raketasks/storage.md index 9b15f5ed4de..c7afebf15bd 100644 --- a/doc/administration/raketasks/storage.md +++ b/doc/administration/raketasks/storage.md @@ -1,7 +1,7 @@ --- -stage: none -group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +stage: Enablement +group: Distribution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Repository storage Rake tasks **(CORE ONLY)** @@ -74,7 +74,7 @@ To have a summary and then a list of projects and their attachments using hashed ## Migrate to hashed storage -DANGER: **Deprecated:** +WARNING: In GitLab 13.0, [hashed storage](../repository_storage_types.md#hashed-storage) is enabled by default and the legacy storage is deprecated. Support for legacy storage will be removed in GitLab 14.0. If you're on GitLab @@ -82,8 +82,9 @@ Support for legacy storage will be removed in GitLab 14.0. If you're on GitLab The option to choose between hashed and legacy storage in the admin area has been disabled. -This task will schedule all your existing projects and attachments associated -with it to be migrated to the **Hashed** storage type: +This task must be run on any machine that has Rails/Sidekiq configured and will +schedule all your existing projects and attachments associated with it to be +migrated to the **Hashed** storage type: - **Omnibus installation** @@ -123,7 +124,7 @@ commands below that helps you inspect projects and attachments in both legacy an ## Rollback from hashed storage to legacy storage -DANGER: **Deprecated:** +WARNING: In GitLab 13.0, [hashed storage](../repository_storage_types.md#hashed-storage) is enabled by default and the legacy storage is deprecated. Support for legacy storage will be removed in GitLab 14.0. If you're on GitLab diff --git a/doc/administration/raketasks/uploads/migrate.md b/doc/administration/raketasks/uploads/migrate.md index 075b27fb70d..2f51bf9357f 100644 --- a/doc/administration/raketasks/uploads/migrate.md +++ b/doc/administration/raketasks/uploads/migrate.md @@ -1,7 +1,7 @@ --- -stage: none -group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +stage: Enablement +group: Distribution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Uploads migrate Rake tasks **(CORE ONLY)** @@ -13,10 +13,10 @@ There is a Rake task for migrating uploads between different storage types. ## Migrate to object storage -After [configuring the object storage](../../uploads.md#using-object-storage) for GitLab's -uploads, use this task to migrate existing uploads from the local storage to the remote storage. +After [configuring the object storage](../../uploads.md#using-object-storage) for uploads +to GitLab, use this task to migrate existing uploads from the local storage to the remote storage. -All of the processing will be done in a background worker and requires **no downtime**. +All of the processing is done in a background worker and requires **no downtime**. Read more about using [object storage with GitLab](../../object_storage.md). @@ -55,8 +55,8 @@ The Rake task uses three parameters to find uploads to migrate: | `model_class` | string | Type of the model to migrate from. | | `mount_point` | string/symbol | Name of the model's column the uploader is mounted on. | -NOTE: **Note:** -These parameters are mainly internal to GitLab's structure, you may want to refer to the task list +NOTE: +These parameters are mainly internal to the structure of GitLab, you may want to refer to the task list instead below. This task also accepts an environment variable which you can use to override @@ -131,9 +131,9 @@ sudo -u git -H bundle exec rake "gitlab:uploads:migrate[DesignManagement::Design If you need to disable [object storage](../../object_storage.md) for any reason, you must first migrate your data out of object storage and back into your local storage. -CAUTION: **Warning:** +WARNING: **Extended downtime is required** so no new files are created in object storage during -the migration. A configuration setting will be added soon to allow migrating +the migration. A configuration setting is planned to allow migrating from object storage to local files with only a brief moment of downtime for configuration changes. To follow progress, see the [relevant issue](https://gitlab.com/gitlab-org/gitlab/-/issues/30979). diff --git a/doc/administration/raketasks/uploads/sanitize.md b/doc/administration/raketasks/uploads/sanitize.md index 4f7c99babd8..637992d52ca 100644 --- a/doc/administration/raketasks/uploads/sanitize.md +++ b/doc/administration/raketasks/uploads/sanitize.md @@ -1,7 +1,7 @@ --- -stage: none -group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +stage: Enablement +group: Distribution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Uploads sanitize Rake tasks **(CORE ONLY)** @@ -41,8 +41,8 @@ The Rake task accepts following parameters. | Parameter | Type | Description | |:-------------|:--------|:----------------------------------------------------------------------------------------------------------------------------| -| `start_id` | integer | Only uploads with equal or greater ID will be processed | -| `stop_id` | integer | Only uploads with equal or smaller ID will be processed | +| `start_id` | integer | Only uploads with equal or greater ID are processed | +| `stop_id` | integer | Only uploads with equal or smaller ID are processed | | `dry_run` | boolean | Do not remove EXIF data, only check if EXIF data are present or not. Defaults to `true` | | `sleep_time` | float | Pause for number of seconds after processing each image. Defaults to 0.3 seconds | | `uploader` | string | Run sanitization only for uploads of the given uploader: `FileUploader`, `PersonalFileUploader`, or `NamespaceFileUploader` | @@ -66,7 +66,7 @@ To remove EXIF data on uploads with an ID between 100 and 5000 and pause for 0.1 sudo RAILS_ENV=production -u git -H bundle exec rake gitlab:uploads:sanitize:remove_exif[100,5000,false,0.1] 2>&1 | tee exif.log ``` -The output is written into an `exif.log` file because it will probably be long. +The output is written into an `exif.log` file because it is often long. If sanitization fails for an upload, an error message should be in the output of the Rake task. Typical reasons include that the file is missing in the storage or it's not a valid image. |