Add latest changes from gitlab-org/gitlab@13-10-stable-eev13.10.0-rc40

4 files changed, 90 insertions, 14 deletions

diff --git a/doc/administration/troubleshooting/debug.md b/doc/administration/troubleshooting/debug.md
index a6073e34d58..5a8ee1c5c94 100644
--- a/doc/administration/troubleshooting/debug.md
+++ b/doc/administration/troubleshooting/debug.md
@@ -255,7 +255,7 @@ separate Rails process to debug the issue:
 ### GitLab: API is not accessible
 
 This often occurs when GitLab Shell attempts to request authorization via the
-internal API (e.g., `http://localhost:8080/api/v4/internal/allowed`), and
+[internal API](../../development/internal_api.md) (e.g., `http://localhost:8080/api/v4/internal/allowed`), and
 something in the check fails. There are many reasons why this may happen:
 
 1. Timeout connecting to a database (e.g., PostgreSQL or Redis)
@@ -271,8 +271,8 @@ strace -ttTfyyy -s 1024 -p <PID of unicorn worker> -o /tmp/unicorn.txt
 ```
 
 If you cannot isolate which Unicorn worker is the issue, try to run `strace`
-on all the Unicorn workers to see where the `/internal/allowed` endpoint gets
-stuck:
+on all the Unicorn workers to see where the
+[`/internal/allowed`](../../development/internal_api.md) endpoint gets stuck:
 
 ```shell
 ps auwx | grep unicorn | awk '{ print " -p " $2}' | xargs  strace -ttTfyyy -s 1024 -o /tmp/unicorn.txt
diff --git a/doc/administration/troubleshooting/elasticsearch.md b/doc/administration/troubleshooting/elasticsearch.md
index 0f60c43ef9e..606697b5247 100644
--- a/doc/administration/troubleshooting/elasticsearch.md
+++ b/doc/administration/troubleshooting/elasticsearch.md
@@ -36,7 +36,7 @@ The type of problem will determine what steps to take. The possible troubleshoot
 - Indexing.
 - Integration.
 - Performance.
-- Background Migrations.
+- Advanced Search Migrations.
 
 ### Search Results workflow
 
@@ -148,7 +148,7 @@ graph TD;
   F7(Escalate to<br>GitLab support.)
 ```
 
-### Background Migrations workflow
+### Advanced Search Migrations workflow
 
 ```mermaid
 graph TD;
@@ -180,7 +180,7 @@ Most Elasticsearch troubleshooting can be broken down into 4 categories:
 - [Troubleshooting indexing](#troubleshooting-indexing)
 - [Troubleshooting integration](#troubleshooting-integration)
 - [Troubleshooting performance](#troubleshooting-performance)
-- [Troubleshooting background migrations](#troubleshooting-background-migrations)
+- [Troubleshooting Advanced Search migrations](#troubleshooting-advanced-search-migrations)
 
 Generally speaking, if it does not fall into those four categories, it is either:
 
@@ -294,7 +294,7 @@ Beyond that, you will want to review the error. If it is:
   GitLab support.
 - An OS issue, you will want to reach out to your systems administrator.
 - A `Faraday::TimeoutError (execution expired)` error **and** you're using a proxy,
-  [set a custom  `gitlab_rails['env']` environment variable, called `no_proxy`](https://docs.gitlab.com/omnibus/settings/environment-variables.html)
+  [set a custom `gitlab_rails['env']` environment variable, called `no_proxy`](https://docs.gitlab.com/omnibus/settings/environment-variables.html)
   with the IP address of your Elasticsearch host.
 
 ### Troubleshooting performance
@@ -356,15 +356,16 @@ dig further into these.
 Feel free to reach out to GitLab support, but this is likely to be something a skilled
 Elasticsearch administrator has more experience with.
 
-### Troubleshooting background migrations
+### Troubleshooting Advanced Search migrations
 
-Troubleshooting background migration failures can be difficult and may require contacting 
-an Elasticsearch administrator or GitLab Support.
+Troubleshooting Advanced Search migration failures can be difficult and may
+require contacting an Elasticsearch administrator or GitLab Support.
 
-The best place to start while debugging issues with a background migration is the 
-[`elasticsearch.log` file](../logs.md#elasticsearchlog). Migrations will
-print information while a migration is in progress and any errors encountered.
-Apply fixes for any errors found in the log and retry the migration.
+The best place to start while debugging issues with an Advanced Search
+migration is the [`elasticsearch.log` file](../logs.md#elasticsearchlog).
+Migrations will log information while a migration is in progress and any
+errors encountered. Apply fixes for any errors found in the log and retry
+the migration.
 
 If you still encounter issues after retrying the migration, reach out to GitLab support.
 
diff --git a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md
index 55e707042ba..69eb639a8db 100644
--- a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md
+++ b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md
@@ -1045,6 +1045,67 @@ project = Project.find_by_full_path('<group/project>')
 Geo::RepositorySyncService.new(project).execute
 ```
 
+### Blob types newer than uploads/artifacts/LFS
+
+- `Packages::PackageFile`
+- `Terraform::StateVersion`
+- `MergeRequestDiff`
+
+`Packages::PackageFile` is used in the following examples, but things generally work the same for the other Blob types.
+
+#### The Replicator
+
+The main kinds of classes are Registry, Model, and Replicator. If you have an instance of one of these classes, you can get the others. The Registry and Model mostly manage PostgreSQL DB state. The Replicator knows how to replicate/verify (or it can call a service to do it):
+
+```ruby
+model_record = Packages::PackageFile.last
+model_record.replicator.registry.replicator.model_record # just showing that these methods exist
+```
+
+#### Replicate a package file, synchronously, given an ID
+
+```ruby
+model_record = Packages::PackageFile.find(id)
+model_record.replicator.send(:download)
+```
+
+#### Replicate a package file, synchronously, given a registry ID
+
+```ruby
+registry = Geo::PackageFileRegistry.find(registry_id)
+registry.replicator.send(:download)
+```
+
+### Repository types newer than project/wiki repositories
+
+- `SnippetRepository`
+- `GroupWikiRepository`
+
+`SnippetRepository` is used in the examples below, but things generally work the same for the other Repository types.
+
+#### The Replicator
+
+The main kinds of classes are Registry, Model, and Replicator. If you have an instance of one of these classes, you can get the others. The Registry and Model mostly manage PostgreSQL DB state. The Replicator knows how to replicate/verify (or it can call a service to do it).
+
+```ruby
+model_record = SnippetRepository.last
+model_record.replicator.registry.replicator.model_record # just showing that these methods exist
+```
+
+#### Replicate a snippet repository, synchronously, given an ID
+
+```ruby
+model_record = SnippetRepository.find(id)
+model_record.replicator.send(:sync_repository)
+```
+
+#### Replicate a snippet repository, synchronously, given a registry ID
+
+```ruby
+registry = Geo::SnippetRepositoryRegistry.find(registry_id)
+registry.replicator.send(:sync_repository)
+```
+
 ### Generate usage ping
 
 #### Generate or get the cached usage ping
diff --git a/doc/administration/troubleshooting/log_parsing.md b/doc/administration/troubleshooting/log_parsing.md
index 25300d036ed..2900ce58940 100644
--- a/doc/administration/troubleshooting/log_parsing.md
+++ b/doc/administration/troubleshooting/log_parsing.md
@@ -41,6 +41,20 @@ jq -cR 'fromjson?' file.json | jq <COMMAND>
 By default `jq` will error out when it encounters a line that is not valid JSON.
 This skips over all invalid lines and parses the rest.
 
+#### Print a JSON log's time range
+
+```shell
+cat log.json | (head -1; tail -1) | jq '.time'
+```
+
+Use `zcat` if the file has been rotated and compressed:
+
+```shell
+zcat @400000006026b71d1a7af804.s | (head -1; tail -1) | jq '.time'
+
+zcat some_json.log.25.gz | (head -1; tail -1) | jq '.time'
+```
+
 ### Parsing `production_json.log` and `api_json.log`
 
 #### Find all requests with a 5XX status code