diff options
author | GitLab Bot <gitlab-bot@gitlab.com> | 2022-12-20 14:22:11 +0000 |
---|---|---|
committer | GitLab Bot <gitlab-bot@gitlab.com> | 2022-12-20 14:22:11 +0000 |
commit | 0c872e02b2c822e3397515ec324051ff540f0cd5 (patch) | |
tree | ce2fb6ce7030e4dad0f4118d21ab6453e5938cdd /doc/raketasks | |
parent | f7e05a6853b12f02911494c4b3fe53d9540d74fc (diff) | |
download | gitlab-ce-15.7.0-rc42.tar.gz |
Add latest changes from gitlab-org/gitlab@15-7-stable-eev15.7.0-rc42
Diffstat (limited to 'doc/raketasks')
-rw-r--r-- | doc/raketasks/backup_gitlab.md | 2 | ||||
-rw-r--r-- | doc/raketasks/backup_restore.md | 100 | ||||
-rw-r--r-- | doc/raketasks/cleanup.md | 5 | ||||
-rw-r--r-- | doc/raketasks/generate_sample_prometheus_data.md | 2 | ||||
-rw-r--r-- | doc/raketasks/index.md | 12 | ||||
-rw-r--r-- | doc/raketasks/list_repos.md | 2 | ||||
-rw-r--r-- | doc/raketasks/migrate_snippets.md | 2 | ||||
-rw-r--r-- | doc/raketasks/sidekiq_job_migration.md | 11 | ||||
-rw-r--r-- | doc/raketasks/spdx.md | 2 | ||||
-rw-r--r-- | doc/raketasks/user_management.md | 5 | ||||
-rw-r--r-- | doc/raketasks/web_hooks.md | 2 | ||||
-rw-r--r-- | doc/raketasks/x509_signatures.md | 2 |
12 files changed, 117 insertions, 30 deletions
diff --git a/doc/raketasks/backup_gitlab.md b/doc/raketasks/backup_gitlab.md index 47696fc1f99..aea61e36037 100644 --- a/doc/raketasks/backup_gitlab.md +++ b/doc/raketasks/backup_gitlab.md @@ -36,7 +36,7 @@ The backup command requires [additional parameters](backup_restore.md#back-up-an your installation is using PgBouncer, for either performance reasons or when using it with a Patroni cluster. WARNING: -The backup command doesn't verify if another backup is already running, as described in +Before GitLab 15.5.0, the backup command doesn't verify if another backup is already running, as described in [issue 362593](https://gitlab.com/gitlab-org/gitlab/-/issues/362593). We strongly recommend you make sure that all backups are complete before starting a new one. diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index 2ac79a913f3..796cb71321a 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -85,6 +85,9 @@ In the following cases, consider using file system data transfer or snapshots as - Your GitLab instance has a lot of forked projects and the regular backup task duplicates the Git data for all of them. - Your GitLab instance has a problem and using the regular backup and import Rake tasks isn't possible. +WARNING: +Gitaly Cluster [does not support snapshot backups](../administration/gitaly/index.md#snapshot-backup-and-recovery-limitations). + When considering using file system data transfer or snapshots: - Don't use these methods to migrate from one operating system to another. The operating systems of the source and destination should be as similar as possible. For example, @@ -887,7 +890,7 @@ Truncate the filenames in the `uploads` table: - `current_filename`: a filename that is currently more than 246 characters long. - `new_filename`: a filename that has been truncated to 246 characters maximum. - - `new_path`: new path considering the new_filename (truncated). + - `new_path`: new path considering the `new_filename` (truncated). Once you validate the batch results, you must change the batch size (`row_id`) using the following sequence of numbers (10000 to 20000). Repeat this process until you reach the last record in the `uploads` table. @@ -981,3 +984,98 @@ Truncate the filenames on the filesystem. You must manually rename the files in #### Re-run the backup task After following all the previous steps, re-run the backup task. + +### Restoring database backup fails when `pg_stat_statements` was previously enabled + +The GitLab backup of the PostgreSQL database includes all SQL statements required to enable extensions that were +previously enabled in the database. + +The `pg_stat_statements` extension can only be enabled or disabled by a PostgreSQL user with `superuser` role. +As the restore process uses a database user with limited permissions, it can't execute the following SQL statements: + +```sql +DROP EXTENSION IF EXISTS pg_stat_statements; +CREATE EXTENSION IF NOT EXISTS pg_stat_statements WITH SCHEMA public; +``` + +When trying to restore the backup in a PostgreSQL instance that doesn't have the `pg_stats_statements` extension, +the following error message is displayed: + +```plaintext +ERROR: permission denied to create extension "pg_stat_statements" +HINT: Must be superuser to create this extension. +ERROR: extension "pg_stat_statements" does not exist +``` + +When trying to restore in an instance that has the `pg_stats_statements` extension enabled, the cleaning up step +fails with an error message similar to the following: + +```plaintext +rake aborted! +ActiveRecord::StatementInvalid: PG::InsufficientPrivilege: ERROR: must be owner of view pg_stat_statements +/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:42:in `block (4 levels) in <top (required)>' +/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:41:in `each' +/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:41:in `block (3 levels) in <top (required)>' +/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:71:in `block (3 levels) in <top (required)>' +/opt/gitlab/embedded/bin/bundle:23:in `load' +/opt/gitlab/embedded/bin/bundle:23:in `<main>' +Caused by: +PG::InsufficientPrivilege: ERROR: must be owner of view pg_stat_statements +/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:42:in `block (4 levels) in <top (required)>' +/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:41:in `each' +/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:41:in `block (3 levels) in <top (required)>' +/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:71:in `block (3 levels) in <top (required)>' +/opt/gitlab/embedded/bin/bundle:23:in `load' +/opt/gitlab/embedded/bin/bundle:23:in `<main>' +Tasks: TOP => gitlab:db:drop_tables +(See full trace by running task with --trace) +``` + +#### Prevent the dump file to include `pg_stat_statements` + +To prevent the inclusion of the extension in the PostgreSQL dump file that is part of the backup bundle, +enable the extension in any schema except the `public` schema: + +```sql +CREATE SCHEMA adm; +CREATE EXTENSION pg_stat_statements SCHEMA adm; +``` + +If the extension was previously enabled in the `public` schema, move it to a new one: + +```sql +CREATE SCHEMA adm; +ALTER EXTENSION pg_stat_statements SET SCHEMA adm; +``` + +To query the `pg_stat_statements` data after changing the schema, prefix the view name with the new schema: + +```sql +SELECT * FROM adm.pg_stat_statements limit 0; +``` + +To make it compatible with third-party monitoring solutions that expect it to be enabled in the `public` schema, +you need to include it in the `search_path`: + +```sql +set search_path to public,adm; +``` + +#### Fix an existing dump file to remove references to `pg_stat_statements` + +To fix an existing backup file, do the following changes: + +1. Extract from the backup the following file: `db/database.sql.gz`. +1. Decompress the file or use an editor that is capable of handling it compressed. +1. Remove the following lines, or similar ones: + + ```sql + CREATE EXTENSION IF NOT EXISTS pg_stat_statements WITH SCHEMA public; + ``` + + ```sql + COMMENT ON EXTENSION pg_stat_statements IS 'track planning and execution statistics of all SQL statements executed'; + ``` + +1. Save the changes and recompress the file. +1. Update the backup file with the modified `db/database.sql.gz`. diff --git a/doc/raketasks/cleanup.md b/doc/raketasks/cleanup.md index a67fec26a9b..60c81b26c05 100644 --- a/doc/raketasks/cleanup.md +++ b/doc/raketasks/cleanup.md @@ -4,7 +4,7 @@ group: Distribution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Clean up **(FREE SELF)** +# Clean up Rake tasks **(FREE SELF)** GitLab provides Rake tasks for cleaning up GitLab instances. @@ -35,7 +35,8 @@ You can also specify the project with `PROJECT_ID` instead of `PROJECT_PATH`. For example: ```shell -$ sudo gitlab-rake gitlab:cleanup:orphan_lfs_file_references PROJECT_PATH="gitlab-org/gitlab-foss" +$ sudo gitlab-rake gitlab:cleanup:orphan_lfs_file_references PROJECT_ID="13083" + I, [2019-12-13T16:35:31.764962 #82356] INFO -- : Looking for orphan LFS files for project GitLab Org / GitLab Foss I, [2019-12-13T16:35:31.923659 #82356] INFO -- : Removed invalid references: 12 ``` diff --git a/doc/raketasks/generate_sample_prometheus_data.md b/doc/raketasks/generate_sample_prometheus_data.md index ec7f54a41c2..7445065c77e 100644 --- a/doc/raketasks/generate_sample_prometheus_data.md +++ b/doc/raketasks/generate_sample_prometheus_data.md @@ -4,7 +4,7 @@ group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Generate sample Prometheus data **(FREE SELF)** +# Generate sample Prometheus data Rake task **(FREE SELF)** This command runs Prometheus queries for each of the metrics of a specific environment for a series of time intervals to now: diff --git a/doc/raketasks/index.md b/doc/raketasks/index.md index 4a157688a41..0d5a826b0a1 100644 --- a/doc/raketasks/index.md +++ b/doc/raketasks/index.md @@ -7,15 +7,13 @@ comments: false # Rake tasks **(FREE SELF)** -GitLab provides [Rake](https://ruby.github.io/rake/) tasks to assist you with -common administration and operational processes. +GitLab provides [Rake](https://ruby.github.io/rake/) tasks to assist you with common administration and operational +processes. You can perform GitLab Rake tasks by using: -- `gitlab-rake <raketask>` for [Omnibus GitLab](https://docs.gitlab.com/omnibus/index.html) - installations. -- `bundle exec rake <raketask>` for [source](../install/installation.md) - installations. +- `gitlab-rake <raketask>` for [Omnibus GitLab](https://docs.gitlab.com/omnibus/index.html) installations. +- `bundle exec rake <raketask>` for [source](../install/installation.md) installations. ## Available Rake tasks @@ -39,7 +37,7 @@ The following Rake tasks are available for use with GitLab: | [Praefect Rake tasks](../administration/raketasks/praefect.md) | [Praefect](../administration/gitaly/praefect.md)-related tasks. | | [Project import/export](../administration/raketasks/project_import_export.md) | Prepare for [project exports and imports](../user/project/settings/import_export.md). | | [Sample Prometheus data](generate_sample_prometheus_data.md) | Generate sample Prometheus data. | -| [Sidekiq job migration](sidekiq_job_migration.md) | Migrate Sidekiq jobs scheduled for future dates to a new queue. | +| [Sidekiq job migration](../administration/sidekiq/sidekiq_job_migration.md) | Migrate Sidekiq jobs scheduled for future dates to a new queue. | | [SMTP maintenance](../administration/raketasks/smtp.md) | SMTP-related tasks. | | [SPDX license list import](spdx.md) | Import a local copy of the [SPDX license list](https://spdx.org/licenses/) for matching [License Compliance policies](../user/compliance/license_compliance/index.md). | | [Repository storage](../administration/raketasks/storage.md) | List and migrate existing projects and attachments from legacy storage to hashed storage. | diff --git a/doc/raketasks/list_repos.md b/doc/raketasks/list_repos.md index 57d24a2942d..258d41aa190 100644 --- a/doc/raketasks/list_repos.md +++ b/doc/raketasks/list_repos.md @@ -4,7 +4,7 @@ group: Distribution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Listing repository directories **(FREE SELF)** +# List repository directories Rake task **(FREE SELF)** You can print a list of all Git repositories on disk managed by GitLab. diff --git a/doc/raketasks/migrate_snippets.md b/doc/raketasks/migrate_snippets.md index 62d2866e499..e51edc5c133 100644 --- a/doc/raketasks/migrate_snippets.md +++ b/doc/raketasks/migrate_snippets.md @@ -4,7 +4,7 @@ group: Editor info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Migration to versioned snippets **(FREE SELF)** +# Migrate to versioned snippets Rake tasks **(FREE SELF)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/215861) in GitLab 13.0. diff --git a/doc/raketasks/sidekiq_job_migration.md b/doc/raketasks/sidekiq_job_migration.md deleted file mode 100644 index 45a0cbaa267..00000000000 --- a/doc/raketasks/sidekiq_job_migration.md +++ /dev/null @@ -1,11 +0,0 @@ ---- -redirect_to: '../administration/sidekiq/sidekiq_job_migration.md' -remove_date: '2022-11-11' ---- - -This document was moved to [another location](../administration/sidekiq/sidekiq_job_migration.md). - -<!-- This redirect file can be deleted after <2022-11-11>. --> -<!-- Redirects that point to other docs in the same project expire in three months. --> -<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/raketasks/spdx.md b/doc/raketasks/spdx.md index 31f860f45de..608139fa404 100644 --- a/doc/raketasks/spdx.md +++ b/doc/raketasks/spdx.md @@ -4,7 +4,7 @@ group: Composition Analysis info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# SPDX license list import **(ULTIMATE SELF)** +# SPDX license list import Rake task **(ULTIMATE SELF)** GitLab provides a Rake task for uploading a fresh copy of the [SPDX license list](https://spdx.org/licenses/) to a GitLab instance. This list is needed for matching the names of [License Compliance policies](../user/compliance/license_compliance/index.md). diff --git a/doc/raketasks/user_management.md b/doc/raketasks/user_management.md index 84d943e2c21..7756774e432 100644 --- a/doc/raketasks/user_management.md +++ b/doc/raketasks/user_management.md @@ -4,9 +4,10 @@ group: Distribution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# User management **(FREE SELF)** +# User management Rake tasks **(FREE SELF)** -GitLab provides Rake tasks for user management. +GitLab provides Rake tasks for managing users. Administrators can also use the Admin Area to +[manage users](../user/admin_area/index.md#administering-users). ## Add user as a developer to all projects diff --git a/doc/raketasks/web_hooks.md b/doc/raketasks/web_hooks.md index b4d01d21dc7..3bd9d7e2d2e 100644 --- a/doc/raketasks/web_hooks.md +++ b/doc/raketasks/web_hooks.md @@ -4,7 +4,7 @@ group: Distribution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# Webhooks administration **(FREE SELF)** +# Webhooks administration Rake tasks **(FREE SELF)** GitLab provides Rake tasks for webhooks management. diff --git a/doc/raketasks/x509_signatures.md b/doc/raketasks/x509_signatures.md index 3404f6ae9cf..364264ae204 100644 --- a/doc/raketasks/x509_signatures.md +++ b/doc/raketasks/x509_signatures.md @@ -4,7 +4,7 @@ group: Source Code info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments --- -# X.509 signatures **(FREE SELF)** +# X.509 signatures Rake task **(FREE SELF)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/122159) in GitLab 12.10. |