Add latest changes from gitlab-org/gitlab@master

8 files changed, 111 insertions, 50 deletions

diff --git a/doc/administration/geo/disaster_recovery/index.md b/doc/administration/geo/disaster_recovery/index.md
index e8223c4b57e..4b42d739d7c 100644
--- a/doc/administration/geo/disaster_recovery/index.md
+++ b/doc/administration/geo/disaster_recovery/index.md
@@ -132,7 +132,10 @@ Note the following when promoting a secondary:
    gitlab-ctl promote-to-primary-node
    ```
 
-   If you have already run the [preflight checks](planned_failover.md#preflight-checks), you can skip them with:
+   CAUTION: **Warning:**
+   Skipping preflight checks will promote the secondary to a primary without any further confirmation!
+
+   If you have already run the [preflight checks](planned_failover.md#preflight-checks) or don't want to run them, you can skip preflight checks with:
 
    ```shell
    gitlab-ctl promote-to-primary-node --skip-preflight-check
diff --git a/doc/administration/geo/disaster_recovery/planned_failover.md b/doc/administration/geo/disaster_recovery/planned_failover.md
index 88ebcb847bb..c7aab680814 100644
--- a/doc/administration/geo/disaster_recovery/planned_failover.md
+++ b/doc/administration/geo/disaster_recovery/planned_failover.md
@@ -45,7 +45,7 @@ be found in `/var/opt/gitlab/gitlab-rails/shared/pages` if using Omnibus).
 
 ## Preflight checks
 
-Run this command to list out all preflight checks before scheduling a planned failover to ensure the process will go smoothly:
+Run this command to list out all preflight checks and automatically check if replication and verification are complete before scheduling a planned failover to ensure the process will go smoothly:
 
 ```shell
 gitlab-ctl promotion-preflight-checks
diff --git a/doc/administration/raketasks/doctor.md b/doc/administration/raketasks/doctor.md
new file mode 100644
index 00000000000..2c1b6928663
--- /dev/null
+++ b/doc/administration/raketasks/doctor.md
@@ -0,0 +1,84 @@
+# Doctor Rake tasks **(CORE ONLY)**
+
+This is a collection of tasks to help investigate and repair
+problems caused by data integrity issues.
+
+## Verify database values can be decrypted using the current secrets
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/20069) in GitLab 13.1.
+
+This task runs through all possible encrypted values in the
+database, verifying that they are decryptable using the current
+secrets file (`gitlab-secrets.json`).
+
+Automatic resolution is not yet implemented. If you have values that
+cannot be decrypted, you can follow steps to reset them, see our
+docs on what to do [when the secrets file is lost](../../raketasks/backup_restore.md#when-the-secrets-file-is-lost).
+
+NOTE: **Note:**
+This can take a very long time, depending on the size of your
+database, as it checks all rows in all tables.
+
+**Omnibus Installation**
+
+```shell
+sudo gitlab-rake gitlab:doctor:secrets
+```
+
+**Source Installation**
+
+```shell
+bundle exec rake gitlab:doctor:secrets RAILS_ENV=production
+```
+
+**Example output**
+
+<!-- vale gitlab.SentenceSpacing = NO -->
+```plaintext
+I, [2020-06-11T17:17:54.951815 #27148]  INFO -- : Checking encrypted values in the database
+I, [2020-06-11T17:18:12.677708 #27148]  INFO -- : - ApplicationSetting failures: 0
+I, [2020-06-11T17:18:12.823692 #27148]  INFO -- : - User failures: 0
+[...] other models possibly containing encrypted data
+I, [2020-06-11T17:18:14.938335 #27148]  INFO -- : - Group failures: 1
+I, [2020-06-11T17:18:15.559162 #27148]  INFO -- : - Operations::FeatureFlagsClient failures: 0
+I, [2020-06-11T17:18:15.575533 #27148]  INFO -- : - ScimOauthAccessToken failures: 0
+I, [2020-06-11T17:18:15.575678 #27148]  INFO -- : Total: 1 row(s) affected
+I, [2020-06-11T17:18:15.575711 #27148]  INFO -- : Done!
+```
+<!-- vale gitlab.SentenceSpacing = YES -->
+
+### Verbose mode
+
+In order to get more detailed information about which
+rows and columns cannot be decrypted, you can pass a VERBOSE
+environment variable:
+
+**Omnibus Installation**
+
+```shell
+sudo gitlab-rake gitlab:doctor:secrets VERBOSE=1
+```
+
+**Source Installation**
+
+```shell
+bundle exec rake gitlab:doctor:secrets RAILS_ENV=production VERBOSE=1
+```
+
+**Example verbose output**
+
+<!-- vale gitlab.SentenceSpacing = NO -->
+```plaintext
+I, [2020-06-11T17:17:54.951815 #27148]  INFO -- : Checking encrypted values in the database
+I, [2020-06-11T17:18:12.677708 #27148]  INFO -- : - ApplicationSetting failures: 0
+I, [2020-06-11T17:18:12.823692 #27148]  INFO -- : - User failures: 0
+[...] other models possibly containing encrypted data
+D, [2020-06-11T17:19:53.224344 #27351] DEBUG -- : > Something went wrong for Group[10].runners_token: Validation failed: Route can't be blank
+I, [2020-06-11T17:19:53.225178 #27351]  INFO -- : - Group failures: 1
+D, [2020-06-11T17:19:53.225267 #27351] DEBUG -- :   - Group[10]: runners_token
+I, [2020-06-11T17:18:15.559162 #27148]  INFO -- : - Operations::FeatureFlagsClient failures: 0
+I, [2020-06-11T17:18:15.575533 #27148]  INFO -- : - ScimOauthAccessToken failures: 0
+I, [2020-06-11T17:18:15.575678 #27148]  INFO -- : Total: 1 row(s) affected
+I, [2020-06-11T17:18:15.575711 #27148]  INFO -- : Done!
+```
+<!-- vale gitlab.SentenceSpacing = YES -->
diff --git a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md
index b96f5814f26..a7e34edbfc1 100644
--- a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md
+++ b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md
@@ -334,23 +334,7 @@ end
 
 ### Find mirrors with "bad decrypt" errors
 
-```ruby
-total = 0
-bad = []
-ProjectImportData.find_each do |data|
-  begin
-    total += 1
-    data.credentials
-  rescue => e
-    bad << data
-  end
-end
-
-puts "Bad count: #{bad.count} / #{total}"
-bad.each do |repo|
-  puts Project.find(repo.project_id).full_path
-end; bad.count
-```
+This content has been converted to a Rake task, see the [Doctor Rake tasks docs](../raketasks/doctor.md).
 
 ### Transfer mirror users and tokens to a single service account
 
@@ -769,18 +753,9 @@ area on disk. It remains to be seen exactly how or whether the deletion is usefu
 
 ### Bad Decrypt Script (for encrypted variables)
 
-See <https://gitlab.com/snippets/1730735/raw>.
-
-This script will go through all the encrypted variables and count how many are not able
-to be decrypted. Might be helpful to run on multiple nodes to see which `gitlab-secrets.json`
-file is most up to date:
-
-```shell
-wget -O /tmp/bad-decrypt.rb https://gitlab.com/snippets/1730735/raw
-gitlab-rails runner /tmp/bad-decrypt.rb
-```
+This content has been converted to a Rake task, see the [Doctor Rake tasks docs](../raketasks/doctor.md).
 
-If `ProjectImportData Bad count:` is detected and the decision is made to delete the
+As an example of repairing, if `ProjectImportData Bad count:` is detected and the decision is made to delete the
 encrypted credentials to allow manual reentry:
 
 ```ruby
@@ -811,16 +786,18 @@ encrypted credentials to allow manual reentry:
 If `User OTP Secret Bad count:` is detected. For each user listed disable/enable
 two-factor authentication.
 
-### Decrypt Script for encrypted tokens
-
-This script will search for all encrypted tokens that are causing decryption errors,
-and update or reset as needed:
+The following script will search in some of the tables for encrypted tokens that are
+causing decryption errors, and update or reset as needed:
 
 ```shell
 wget -O /tmp/encrypted-tokens.rb https://gitlab.com/snippets/1876342/raw
 gitlab-rails runner /tmp/encrypted-tokens.rb
 ```
 
+### Decrypt Script for encrypted tokens
+
+This content has been converted to a Rake task, see the [Doctor Rake tasks docs](../raketasks/doctor.md).
+
 ## Geo
 
 ### Artifacts
diff --git a/doc/development/documentation/styleguide.md b/doc/development/documentation/styleguide.md
index 47867417c6d..d6d29cd8eef 100644
--- a/doc/development/documentation/styleguide.md
+++ b/doc/development/documentation/styleguide.md
@@ -1453,7 +1453,7 @@ For GitLab.com only tiers (when the feature is not available for self-managed in
 
 The tier should be ideally added to headers, so that the full badge will be displayed.
 However, it can be also mentioned from paragraphs, list items, and table cells. For these cases,
-the tier mention will be represented by an orange question mark that will show the tiers on hover.
+the tier mention will be represented by an orange info icon **(information)** that will show the tiers on hover.
 
 Use the lowest tier at the page level, even if higher-level tiers exist on the page. For example, you might have a page that is marked as Starter but a section badged as Premium.
 
diff --git a/doc/raketasks/README.md b/doc/raketasks/README.md
index 724046bcd46..b7cfc18534b 100644
--- a/doc/raketasks/README.md
+++ b/doc/raketasks/README.md
@@ -20,6 +20,7 @@ The following are available Rake tasks:
 | [Back up and restore](backup_restore.md)                                                            | Back up, restore, and migrate GitLab instances between servers.                           |
 | [Clean up](cleanup.md)                                                                              | Clean up unneeded items from GitLab instances.                                            |
 | [Development](../development/rake_tasks.md)                                                         | Tasks for GitLab contributors.                                                            |
+| [Doctor tasks](../administration/raketasks/doctor.md)                                               | Checks for data integrity issues.                                                         |
 | [Elasticsearch](../integration/elasticsearch.md#gitlab-elasticsearch-rake-tasks) **(STARTER ONLY)** | Maintain Elasticsearch in a GitLab instance.                                              |
 | [Enable namespaces](features.md)                                                                    | Enable usernames and namespaces for user projects.                                        |
 | [General maintenance](../administration/raketasks/maintenance.md)                                   | General maintenance and self-check tasks.                                                 |
diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md
index f8c9c79a822..79622fdcfd7 100644
--- a/doc/raketasks/backup_restore.md
+++ b/doc/raketasks/backup_restore.md
@@ -26,14 +26,6 @@ installed on your system.
     sudo yum install rsync
     ```
 
-- **Tar**: Backup and restore tasks use `tar` under the hood to create and extract
-  archives. Ensure you have version 1.30 or above of `tar` available in your
-  system. To check the version, run:
-
-  ```shell
-  tar --version
-  ```
-
 ## Backup timestamp
 
 NOTE: **Note:**
@@ -679,7 +671,7 @@ You can only restore a backup to **exactly the same version and type (CE/EE)** o
 GitLab that you created it on, for example CE 9.1.0.
 
 If your backup is a different version than the current installation, you will
-need to [downgrade your GitLab installation](https://docs.gitlab.com/omnibus/update/README.html#downgrading)
+need to [downgrade your GitLab installation](https://docs.gitlab.com/omnibus/update/README.html#downgrade)
 before restoring the backup.
 
 ### Restore prerequisites
@@ -936,7 +928,7 @@ Be advised that, backup is successfully restored in spite of these warnings.
 The Rake task runs this as the `gitlab` user which does not have the superuser access to the database. When restore is initiated it will also run as `gitlab` user but it will also try to alter the objects it does not have access to.
 Those objects have no influence on the database backup/restore but they give this annoying warning.
 
-For more information see similar questions on PostgreSQL issue tracker[here](https://www.postgresql.org/message-id/201110220712.30886.adrian.klaver@gmail.com) and [here](https://www.postgresql.org/message-id/2039.1177339749@sss.pgh.pa.us) as well as [stack overflow](https://stackoverflow.com/questions/4368789/error-must-be-owner-of-language-plpgsql).
+For more information see similar questions on PostgreSQL issue tracker [here](https://www.postgresql.org/message-id/201110220712.30886.adrian.klaver@gmail.com) and [here](https://www.postgresql.org/message-id/2039.1177339749@sss.pgh.pa.us) as well as [stack overflow](https://stackoverflow.com/questions/4368789/error-must-be-owner-of-language-plpgsql).
 
 ### When the secrets file is lost
 
@@ -963,6 +955,9 @@ experience some unexpected behavior such as:
 - Stuck jobs.
 - 500 errors.
 
+You can check whether you have undecryptable values in the database using
+the [Secrets Doctor Rake task](../administration/raketasks/doctor.md).
+
 In this case, you are required to reset all the tokens for CI/CD variables
 and Runner Authentication, which is described in more detail below. After
 resetting the tokens, you should be able to visit your project and the jobs
diff --git a/doc/subscriptions/index.md b/doc/subscriptions/index.md
index 0d77a2fe3c3..64bf2e0b50b 100644
--- a/doc/subscriptions/index.md
+++ b/doc/subscriptions/index.md
@@ -161,16 +161,17 @@ To change payment method or update credit card information:
 
 ### Change the linked account
 
-To change the GitLab.com account associated with a Customers Portal
+To change the GitLab.com account associated with your Customers Portal
 account:
 
 1. Log in to the
    [Customers Portal](https://customers.gitlab.com/customers/sign_in).
-1. Go to [GitLab.com](https://gitlab.com) in a separate browser tab. Ensure you
+1. In a separate browser tab, go to [GitLab.com](https://gitlab.com) and ensure you
    are not logged in.
-1. On the Customers Portal page, select the **My account** drop-down and click on [**Account details**](https://customers.gitlab.com/customers/edit).
-1. Under **Your GitLab.com account**, click **Change linked account** button.
-1. Log in to the [GitLab.com](https://gitlab.com) account you want to link to the Customers Portal.
+1. On the Customers Portal page, click **My account > Account details**.
+1. Under **Your GitLab.com account**, click **Change linked account**.
+1. Log in to the [GitLab.com](https://gitlab.com) account you want to link to the Customers Portal
+   account.
 
 ### Change the associated namespace