summaryrefslogtreecommitdiff
path: root/doc/raketasks
diff options
context:
space:
mode:
authorGitLab Bot <gitlab-bot@gitlab.com>2022-12-20 14:22:11 +0000
committerGitLab Bot <gitlab-bot@gitlab.com>2022-12-20 14:22:11 +0000
commit0c872e02b2c822e3397515ec324051ff540f0cd5 (patch)
treece2fb6ce7030e4dad0f4118d21ab6453e5938cdd /doc/raketasks
parentf7e05a6853b12f02911494c4b3fe53d9540d74fc (diff)
downloadgitlab-ce-15.7.0-rc42.tar.gz
Add latest changes from gitlab-org/gitlab@15-7-stable-eev15.7.0-rc42
Diffstat (limited to 'doc/raketasks')
-rw-r--r--doc/raketasks/backup_gitlab.md2
-rw-r--r--doc/raketasks/backup_restore.md100
-rw-r--r--doc/raketasks/cleanup.md5
-rw-r--r--doc/raketasks/generate_sample_prometheus_data.md2
-rw-r--r--doc/raketasks/index.md12
-rw-r--r--doc/raketasks/list_repos.md2
-rw-r--r--doc/raketasks/migrate_snippets.md2
-rw-r--r--doc/raketasks/sidekiq_job_migration.md11
-rw-r--r--doc/raketasks/spdx.md2
-rw-r--r--doc/raketasks/user_management.md5
-rw-r--r--doc/raketasks/web_hooks.md2
-rw-r--r--doc/raketasks/x509_signatures.md2
12 files changed, 117 insertions, 30 deletions
diff --git a/doc/raketasks/backup_gitlab.md b/doc/raketasks/backup_gitlab.md
index 47696fc1f99..aea61e36037 100644
--- a/doc/raketasks/backup_gitlab.md
+++ b/doc/raketasks/backup_gitlab.md
@@ -36,7 +36,7 @@ The backup command requires [additional parameters](backup_restore.md#back-up-an
your installation is using PgBouncer, for either performance reasons or when using it with a Patroni cluster.
WARNING:
-The backup command doesn't verify if another backup is already running, as described in
+Before GitLab 15.5.0, the backup command doesn't verify if another backup is already running, as described in
[issue 362593](https://gitlab.com/gitlab-org/gitlab/-/issues/362593). We strongly recommend
you make sure that all backups are complete before starting a new one.
diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md
index 2ac79a913f3..796cb71321a 100644
--- a/doc/raketasks/backup_restore.md
+++ b/doc/raketasks/backup_restore.md
@@ -85,6 +85,9 @@ In the following cases, consider using file system data transfer or snapshots as
- Your GitLab instance has a lot of forked projects and the regular backup task duplicates the Git data for all of them.
- Your GitLab instance has a problem and using the regular backup and import Rake tasks isn't possible.
+WARNING:
+Gitaly Cluster [does not support snapshot backups](../administration/gitaly/index.md#snapshot-backup-and-recovery-limitations).
+
When considering using file system data transfer or snapshots:
- Don't use these methods to migrate from one operating system to another. The operating systems of the source and destination should be as similar as possible. For example,
@@ -887,7 +890,7 @@ Truncate the filenames in the `uploads` table:
- `current_filename`: a filename that is currently more than 246 characters long.
- `new_filename`: a filename that has been truncated to 246 characters maximum.
- - `new_path`: new path considering the new_filename (truncated).
+ - `new_path`: new path considering the `new_filename` (truncated).
Once you validate the batch results, you must change the batch size (`row_id`) using the following sequence of numbers (10000 to 20000). Repeat this process until you reach the last record in the `uploads` table.
@@ -981,3 +984,98 @@ Truncate the filenames on the filesystem. You must manually rename the files in
#### Re-run the backup task
After following all the previous steps, re-run the backup task.
+
+### Restoring database backup fails when `pg_stat_statements` was previously enabled
+
+The GitLab backup of the PostgreSQL database includes all SQL statements required to enable extensions that were
+previously enabled in the database.
+
+The `pg_stat_statements` extension can only be enabled or disabled by a PostgreSQL user with `superuser` role.
+As the restore process uses a database user with limited permissions, it can't execute the following SQL statements:
+
+```sql
+DROP EXTENSION IF EXISTS pg_stat_statements;
+CREATE EXTENSION IF NOT EXISTS pg_stat_statements WITH SCHEMA public;
+```
+
+When trying to restore the backup in a PostgreSQL instance that doesn't have the `pg_stats_statements` extension,
+the following error message is displayed:
+
+```plaintext
+ERROR: permission denied to create extension "pg_stat_statements"
+HINT: Must be superuser to create this extension.
+ERROR: extension "pg_stat_statements" does not exist
+```
+
+When trying to restore in an instance that has the `pg_stats_statements` extension enabled, the cleaning up step
+fails with an error message similar to the following:
+
+```plaintext
+rake aborted!
+ActiveRecord::StatementInvalid: PG::InsufficientPrivilege: ERROR: must be owner of view pg_stat_statements
+/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:42:in `block (4 levels) in <top (required)>'
+/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:41:in `each'
+/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:41:in `block (3 levels) in <top (required)>'
+/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:71:in `block (3 levels) in <top (required)>'
+/opt/gitlab/embedded/bin/bundle:23:in `load'
+/opt/gitlab/embedded/bin/bundle:23:in `<main>'
+Caused by:
+PG::InsufficientPrivilege: ERROR: must be owner of view pg_stat_statements
+/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:42:in `block (4 levels) in <top (required)>'
+/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:41:in `each'
+/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:41:in `block (3 levels) in <top (required)>'
+/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:71:in `block (3 levels) in <top (required)>'
+/opt/gitlab/embedded/bin/bundle:23:in `load'
+/opt/gitlab/embedded/bin/bundle:23:in `<main>'
+Tasks: TOP => gitlab:db:drop_tables
+(See full trace by running task with --trace)
+```
+
+#### Prevent the dump file to include `pg_stat_statements`
+
+To prevent the inclusion of the extension in the PostgreSQL dump file that is part of the backup bundle,
+enable the extension in any schema except the `public` schema:
+
+```sql
+CREATE SCHEMA adm;
+CREATE EXTENSION pg_stat_statements SCHEMA adm;
+```
+
+If the extension was previously enabled in the `public` schema, move it to a new one:
+
+```sql
+CREATE SCHEMA adm;
+ALTER EXTENSION pg_stat_statements SET SCHEMA adm;
+```
+
+To query the `pg_stat_statements` data after changing the schema, prefix the view name with the new schema:
+
+```sql
+SELECT * FROM adm.pg_stat_statements limit 0;
+```
+
+To make it compatible with third-party monitoring solutions that expect it to be enabled in the `public` schema,
+you need to include it in the `search_path`:
+
+```sql
+set search_path to public,adm;
+```
+
+#### Fix an existing dump file to remove references to `pg_stat_statements`
+
+To fix an existing backup file, do the following changes:
+
+1. Extract from the backup the following file: `db/database.sql.gz`.
+1. Decompress the file or use an editor that is capable of handling it compressed.
+1. Remove the following lines, or similar ones:
+
+ ```sql
+ CREATE EXTENSION IF NOT EXISTS pg_stat_statements WITH SCHEMA public;
+ ```
+
+ ```sql
+ COMMENT ON EXTENSION pg_stat_statements IS 'track planning and execution statistics of all SQL statements executed';
+ ```
+
+1. Save the changes and recompress the file.
+1. Update the backup file with the modified `db/database.sql.gz`.
diff --git a/doc/raketasks/cleanup.md b/doc/raketasks/cleanup.md
index a67fec26a9b..60c81b26c05 100644
--- a/doc/raketasks/cleanup.md
+++ b/doc/raketasks/cleanup.md
@@ -4,7 +4,7 @@ group: Distribution
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
-# Clean up **(FREE SELF)**
+# Clean up Rake tasks **(FREE SELF)**
GitLab provides Rake tasks for cleaning up GitLab instances.
@@ -35,7 +35,8 @@ You can also specify the project with `PROJECT_ID` instead of `PROJECT_PATH`.
For example:
```shell
-$ sudo gitlab-rake gitlab:cleanup:orphan_lfs_file_references PROJECT_PATH="gitlab-org/gitlab-foss"
+$ sudo gitlab-rake gitlab:cleanup:orphan_lfs_file_references PROJECT_ID="13083"
+
I, [2019-12-13T16:35:31.764962 #82356] INFO -- : Looking for orphan LFS files for project GitLab Org / GitLab Foss
I, [2019-12-13T16:35:31.923659 #82356] INFO -- : Removed invalid references: 12
```
diff --git a/doc/raketasks/generate_sample_prometheus_data.md b/doc/raketasks/generate_sample_prometheus_data.md
index ec7f54a41c2..7445065c77e 100644
--- a/doc/raketasks/generate_sample_prometheus_data.md
+++ b/doc/raketasks/generate_sample_prometheus_data.md
@@ -4,7 +4,7 @@ group: Respond
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
-# Generate sample Prometheus data **(FREE SELF)**
+# Generate sample Prometheus data Rake task **(FREE SELF)**
This command runs Prometheus queries for each of the metrics of a specific environment
for a series of time intervals to now:
diff --git a/doc/raketasks/index.md b/doc/raketasks/index.md
index 4a157688a41..0d5a826b0a1 100644
--- a/doc/raketasks/index.md
+++ b/doc/raketasks/index.md
@@ -7,15 +7,13 @@ comments: false
# Rake tasks **(FREE SELF)**
-GitLab provides [Rake](https://ruby.github.io/rake/) tasks to assist you with
-common administration and operational processes.
+GitLab provides [Rake](https://ruby.github.io/rake/) tasks to assist you with common administration and operational
+processes.
You can perform GitLab Rake tasks by using:
-- `gitlab-rake <raketask>` for [Omnibus GitLab](https://docs.gitlab.com/omnibus/index.html)
- installations.
-- `bundle exec rake <raketask>` for [source](../install/installation.md)
- installations.
+- `gitlab-rake <raketask>` for [Omnibus GitLab](https://docs.gitlab.com/omnibus/index.html) installations.
+- `bundle exec rake <raketask>` for [source](../install/installation.md) installations.
## Available Rake tasks
@@ -39,7 +37,7 @@ The following Rake tasks are available for use with GitLab:
| [Praefect Rake tasks](../administration/raketasks/praefect.md) | [Praefect](../administration/gitaly/praefect.md)-related tasks. |
| [Project import/export](../administration/raketasks/project_import_export.md) | Prepare for [project exports and imports](../user/project/settings/import_export.md). |
| [Sample Prometheus data](generate_sample_prometheus_data.md) | Generate sample Prometheus data. |
-| [Sidekiq job migration](sidekiq_job_migration.md) | Migrate Sidekiq jobs scheduled for future dates to a new queue. |
+| [Sidekiq job migration](../administration/sidekiq/sidekiq_job_migration.md) | Migrate Sidekiq jobs scheduled for future dates to a new queue. |
| [SMTP maintenance](../administration/raketasks/smtp.md) | SMTP-related tasks. |
| [SPDX license list import](spdx.md) | Import a local copy of the [SPDX license list](https://spdx.org/licenses/) for matching [License Compliance policies](../user/compliance/license_compliance/index.md). |
| [Repository storage](../administration/raketasks/storage.md) | List and migrate existing projects and attachments from legacy storage to hashed storage. |
diff --git a/doc/raketasks/list_repos.md b/doc/raketasks/list_repos.md
index 57d24a2942d..258d41aa190 100644
--- a/doc/raketasks/list_repos.md
+++ b/doc/raketasks/list_repos.md
@@ -4,7 +4,7 @@ group: Distribution
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
-# Listing repository directories **(FREE SELF)**
+# List repository directories Rake task **(FREE SELF)**
You can print a list of all Git repositories on disk managed by GitLab.
diff --git a/doc/raketasks/migrate_snippets.md b/doc/raketasks/migrate_snippets.md
index 62d2866e499..e51edc5c133 100644
--- a/doc/raketasks/migrate_snippets.md
+++ b/doc/raketasks/migrate_snippets.md
@@ -4,7 +4,7 @@ group: Editor
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
-# Migration to versioned snippets **(FREE SELF)**
+# Migrate to versioned snippets Rake tasks **(FREE SELF)**
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/215861) in GitLab 13.0.
diff --git a/doc/raketasks/sidekiq_job_migration.md b/doc/raketasks/sidekiq_job_migration.md
deleted file mode 100644
index 45a0cbaa267..00000000000
--- a/doc/raketasks/sidekiq_job_migration.md
+++ /dev/null
@@ -1,11 +0,0 @@
----
-redirect_to: '../administration/sidekiq/sidekiq_job_migration.md'
-remove_date: '2022-11-11'
----
-
-This document was moved to [another location](../administration/sidekiq/sidekiq_job_migration.md).
-
-<!-- This redirect file can be deleted after <2022-11-11>. -->
-<!-- Redirects that point to other docs in the same project expire in three months. -->
-<!-- Redirects that point to docs in a different project or site (link is not relative and starts with `https:`) expire in one year. -->
-<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html -->
diff --git a/doc/raketasks/spdx.md b/doc/raketasks/spdx.md
index 31f860f45de..608139fa404 100644
--- a/doc/raketasks/spdx.md
+++ b/doc/raketasks/spdx.md
@@ -4,7 +4,7 @@ group: Composition Analysis
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
-# SPDX license list import **(ULTIMATE SELF)**
+# SPDX license list import Rake task **(ULTIMATE SELF)**
GitLab provides a Rake task for uploading a fresh copy of the [SPDX license list](https://spdx.org/licenses/)
to a GitLab instance. This list is needed for matching the names of [License Compliance policies](../user/compliance/license_compliance/index.md).
diff --git a/doc/raketasks/user_management.md b/doc/raketasks/user_management.md
index 84d943e2c21..7756774e432 100644
--- a/doc/raketasks/user_management.md
+++ b/doc/raketasks/user_management.md
@@ -4,9 +4,10 @@ group: Distribution
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
-# User management **(FREE SELF)**
+# User management Rake tasks **(FREE SELF)**
-GitLab provides Rake tasks for user management.
+GitLab provides Rake tasks for managing users. Administrators can also use the Admin Area to
+[manage users](../user/admin_area/index.md#administering-users).
## Add user as a developer to all projects
diff --git a/doc/raketasks/web_hooks.md b/doc/raketasks/web_hooks.md
index b4d01d21dc7..3bd9d7e2d2e 100644
--- a/doc/raketasks/web_hooks.md
+++ b/doc/raketasks/web_hooks.md
@@ -4,7 +4,7 @@ group: Distribution
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
-# Webhooks administration **(FREE SELF)**
+# Webhooks administration Rake tasks **(FREE SELF)**
GitLab provides Rake tasks for webhooks management.
diff --git a/doc/raketasks/x509_signatures.md b/doc/raketasks/x509_signatures.md
index 3404f6ae9cf..364264ae204 100644
--- a/doc/raketasks/x509_signatures.md
+++ b/doc/raketasks/x509_signatures.md
@@ -4,7 +4,7 @@ group: Source Code
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
-# X.509 signatures **(FREE SELF)**
+# X.509 signatures Rake task **(FREE SELF)**
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/122159) in GitLab 12.10.