Add latest changes from gitlab-org/gitlab@13-12-stable-eev13.12.0-rc42

7 files changed, 131 insertions, 17 deletions

diff --git a/doc/administration/troubleshooting/defcon.md b/doc/administration/troubleshooting/defcon.md
new file mode 100644
index 00000000000..09e11553d97
--- /dev/null
+++ b/doc/administration/troubleshooting/defcon.md
@@ -0,0 +1,25 @@
+---
+stage: Enablement
+group: Distribution
+info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
+type: reference
+---
+
+# Disaster Recovery
+
+This document describes a feature that allows to easily disable some important but computationally
+expensive parts of the application, in order to relieve stress on the database in an ongoing downtime.
+
+## `ci_queueing_disaster_recovery`
+
+This feature flag, if enabled temporarily disables fair scheduling on shared runners.
+This can help reduce system resource usage on the `jobs/request` endpoint
+by significantly reducing computations being performed.
+
+Side effects:
+
+- In case of a large backlog of jobs, the jobs will be processed in the order
+they were put in the system instead of balancing the jobs across many projects
+- Projects which are out of quota will be run. This affects
+only jobs that were created during the last hour, as prior jobs are canceled
+by a periodic background worker (`StuckCiJobsWorker`).
diff --git a/doc/administration/troubleshooting/elasticsearch.md b/doc/administration/troubleshooting/elasticsearch.md
index 11425d464b9..d04ce23188f 100644
--- a/doc/administration/troubleshooting/elasticsearch.md
+++ b/doc/administration/troubleshooting/elasticsearch.md
@@ -251,13 +251,13 @@ can be made. If the indices:
 - Can be made, Escalate this to GitLab support.
 
 If the issue is not with creating an empty index, the next step is to check for errors
-during the indexing of projects. If errors do occur, they will either stem from the indexing:
+during the indexing of projects. If errors do occur, they stem from either the indexing:
 
 - On the GitLab side. You need to rectify those. If they are not
   something you are familiar with, contact GitLab support for guidance.
 - Within the Elasticsearch instance itself. See if the error is [documented and has a fix](../../integration/elasticsearch.md#troubleshooting). If not, speak with your Elasticsearch administrator.
 
-If the indexing process does not present errors, you will want to check the status of the indexed projects. You can do this via the following Rake tasks:
+If the indexing process does not present errors, check the status of the indexed projects. You can do this via the following Rake tasks:
 
 - [`sudo gitlab-rake gitlab:elastic:index_projects_status`](../../integration/elasticsearch.md#gitlab-advanced-search-rake-tasks) (shows the overall status)
 - [`sudo gitlab-rake gitlab:elastic:projects_not_indexed`](../../integration/elasticsearch.md#gitlab-advanced-search-rake-tasks) (shows specific projects that are not indexed)
@@ -290,11 +290,11 @@ If the issue is:
   regarding the error(s) you are seeing. If you are unsure here, it never hurts to reach
   out to GitLab support.
 
-Beyond that, you will want to review the error. If it is:
+Beyond that, review the error. If it is:
 
 - Specifically from the indexer, this could be a bug/issue and should be escalated to
   GitLab support.
-- An OS issue, you will want to reach out to your systems administrator.
+- An OS issue, you should reach out to your systems administrator.
 - A `Faraday::TimeoutError (execution expired)` error **and** you're using a proxy,
   [set a custom `gitlab_rails['env']` environment variable, called `no_proxy`](https://docs.gitlab.com/omnibus/settings/environment-variables.html)
   with the IP address of your Elasticsearch host.
@@ -365,7 +365,7 @@ require contacting an Elasticsearch administrator or GitLab Support.
 
 The best place to start while debugging issues with an Advanced Search
 migration is the [`elasticsearch.log` file](../logs.md#elasticsearchlog).
-Migrations will log information while a migration is in progress and any
+Migrations log information while a migration is in progress and any
 errors encountered. Apply fixes for any errors found in the log and retry
 the migration.
 
diff --git a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md
index 6b1cf2d1194..588be73e786 100644
--- a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md
+++ b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md
@@ -305,6 +305,42 @@ p.statistics.refresh!
 pp p.statistics  # compare with earlier values
 ```
 
+### Identify deploy keys associated with blocked and non-member users 
+
+When the user who created a deploy key is blocked or removed from the project, the key 
+can no longer be used to push to protected branches in a private project (see [issue #329742](https://gitlab.com/gitlab-org/gitlab/-/issues/329742)). 
+The following script identifies unusable deploy keys:
+
+```ruby
+ghost_user_id = User.ghost.id
+
+DeployKeysProject.with_write_access.find_each do |deploy_key_mapping|
+  project = deploy_key_mapping.project
+  deploy_key = deploy_key_mapping.deploy_key
+  user = deploy_key.user
+
+  access_checker = Gitlab::DeployKeyAccess.new(deploy_key, container: project)
+
+  # can_push_for_ref? tests if deploy_key can push to default branch, which is likely to be protected
+  can_push = access_checker.can_do_action?(:push_code)
+  can_push_to_default = access_checker.can_push_for_ref?(project.repository.root_ref)
+  
+  next if access_checker.allowed? && can_push && can_push_to_default
+
+  if user.nil? || user.id == ghost_user_id
+    username = 'none'
+    state = '-'
+  else
+    username = user.username
+    user_state = user.state
+  end
+
+  puts "Deploy key: #{deploy_key.id}, Project: #{project.full_path}, Can push?: " + (can_push ? 'YES' : 'NO') +
+       ", Can push to default branch #{project.repository.root_ref}?: " + (can_push_to_default ? 'YES' : 'NO') +
+       ", User: #{username}, User state: #{user_state}"
+end
+```
+
 ## Wikis
 
 ### Recreate
@@ -537,7 +573,7 @@ inactive_users.each do |user|
 end
 ```
 
-### Find Max permissions for project/group
+### Find a user's max permissions for project/group
 
 ```ruby
 user = User.find_by_username 'username'
diff --git a/doc/administration/troubleshooting/log_parsing.md b/doc/administration/troubleshooting/log_parsing.md
index 2900ce58940..a0f71960e14 100644
--- a/doc/administration/troubleshooting/log_parsing.md
+++ b/doc/administration/troubleshooting/log_parsing.md
@@ -201,3 +201,42 @@ grep "fatal: " /var/log/gitlab/gitaly/current | \
     jq '."grpc.request.glProjectPath"' | \
     sort | uniq
 ```
+
+### Parsing `gitlab-shell.log`
+
+For investigating Git calls via SSH, from [GitLab 12.10](https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/367).
+
+Find the top 20 calls by project and user:
+
+```shell
+jq --raw-output --slurp '
+  map(
+    select(
+      .username != null and
+      .gl_project_path !=null
+    )
+  )
+  | group_by(.username+.gl_project_path)
+  | sort_by(-length)
+  | limit(20; .[])
+  | "count: \(length)\tuser: \(.[0].username)\tproject: \(.[0].gl_project_path)" ' \
+  /var/log/gitlab/gitlab-shell/gitlab-shell.log
+```
+
+Find the top 20 calls by project, user, and command:
+
+```shell
+jq --raw-output --slurp '
+  map(
+    select(
+      .command  != null and
+      .username != null and
+      .gl_project_path !=null
+    )
+  )
+  | group_by(.username+.gl_project_path+.command)
+  | sort_by(-length)
+  | limit(20; .[])
+  | "count: \(length)\tcommand: \(.[0].command)\tuser: \(.[0].username)\tproject: \(.[0].gl_project_path)" ' \
+  /var/log/gitlab/gitlab-shell/gitlab-shell.log
+```
diff --git a/doc/administration/troubleshooting/sidekiq.md b/doc/administration/troubleshooting/sidekiq.md
index 8d19a8a163b..297a8355036 100644
--- a/doc/administration/troubleshooting/sidekiq.md
+++ b/doc/administration/troubleshooting/sidekiq.md
@@ -11,7 +11,7 @@ tasks. When things go wrong it can be difficult to troubleshoot. These
 situations also tend to be high-pressure because a production system job queue
 may be filling up. Users will notice when this happens because new branches
 may not show up and merge requests may not be updated. The following are some
-troubleshooting steps that will help you diagnose the bottleneck.
+troubleshooting steps to help you diagnose the bottleneck.
 
 GitLab administrators/users should consider working through these
 debug steps with GitLab Support so the backtraces can be analyzed by our team.
@@ -42,7 +42,7 @@ Example log output:
 
 When using [Sidekiq JSON logging](../logs.md#sidekiqlog),
 arguments logs are limited to a maximum size of 10 kilobytes of text;
-any arguments after this limit will be discarded and replaced with a
+any arguments after this limit are discarded and replaced with a
 single argument containing the string `"..."`.
 
 You can set `SIDEKIQ_LOG_ARGUMENTS` [environment variable](https://docs.gitlab.com/omnibus/settings/environment-variables.html)
@@ -58,7 +58,7 @@ In GitLab 13.5 and earlier, set `SIDEKIQ_LOG_ARGUMENTS` to `1` to start logging
 
 ## Thread dump
 
-Send the Sidekiq process ID the `TTIN` signal and it will output thread
+Send the Sidekiq process ID the `TTIN` signal to output thread
 backtraces in the log file.
 
 ```shell
@@ -66,7 +66,7 @@ kill -TTIN <sidekiq_pid>
 ```
 
 Check in `/var/log/gitlab/sidekiq/current` or `$GITLAB_HOME/log/sidekiq.log` for
-the backtrace output. The backtraces will be lengthy and generally start with
+the backtrace output. The backtraces are lengthy and generally start with
 several `WARN` level messages. Here's an example of a single thread's backtrace:
 
 ```plaintext
@@ -88,8 +88,8 @@ Move on to other troubleshooting methods if this happens.
 ## Process profiling with `perf`
 
 Linux has a process profiling tool called `perf` that is helpful when a certain
-process is eating up a lot of CPU. If you see high CPU usage and Sidekiq won't
-respond to the `TTIN` signal, this is a good next step.
+process is eating up a lot of CPU. If you see high CPU usage and Sidekiq isn't
+responding to the `TTIN` signal, this is a good next step.
 
 If `perf` is not installed on your system, install it with `apt-get` or `yum`:
 
@@ -134,8 +134,8 @@ corresponding Ruby code where this is happening.
 `gdb` can be another effective tool for debugging Sidekiq. It gives you a little
 more interactive way to look at each thread and see what's causing problems.
 
-Attaching to a process with `gdb` will suspends the normal operation
-of the process (Sidekiq will not process jobs while `gdb` is attached).
+Attaching to a process with `gdb` suspends the normal operation
+of the process (Sidekiq does not process jobs while `gdb` is attached).
 
 Start by attaching to the Sidekiq PID:
 
@@ -285,7 +285,7 @@ end
 ### Remove Sidekiq jobs for given parameters (destructive)
 
 The general method to kill jobs conditionally is the following command, which
-will remove jobs that are queued but not started. Running jobs will not be killed.
+removes jobs that are queued but not started. Running jobs can not be killed.
 
 ```ruby
 queue = Sidekiq::Queue.new('<queue name>')
@@ -294,7 +294,7 @@ queue.each { |job| job.delete if <condition>}
 
 Have a look at the section below for cancelling running jobs.
 
-In the method above, `<queue-name>` is the name of the queue that contains the job(s) you want to delete and `<condition>` will decide which jobs get deleted.
+In the method above, `<queue-name>` is the name of the queue that contains the job(s) you want to delete and `<condition>` decides which jobs get deleted.
 
 Commonly, `<condition>` references the job arguments, which depend on the type of job in question. To find the arguments for a specific queue, you can have a look at the `perform` function of the related worker file, commonly found at `/app/workers/<queue-name>_worker.rb`.
 
diff --git a/doc/administration/troubleshooting/ssl.md b/doc/administration/troubleshooting/ssl.md
index d7bfd537eca..7c0b745f8c2 100644
--- a/doc/administration/troubleshooting/ssl.md
+++ b/doc/administration/troubleshooting/ssl.md
@@ -238,3 +238,17 @@ remote server that is serving only HTTP.
 One scenario is that you're using [object storage](../object_storage.md), which
 isn't served under HTTPS. GitLab is misconfigured and attempts a TLS handshake,
 but the object storage will respond with plain HTTP.
+
+## `schannel: SEC_E_UNTRUSTED_ROOT`
+
+If you're on Windows and get the following error:
+
+```plaintext
+Fatal: unable to access 'https://gitlab.domain.tld/group/project.git': schannel: SEC_E_UNTRUSTED_ROOT (0x80090325) - The certificate chain was issued by an authority that is not trusted."
+```
+
+You may need to specify that Git should use OpenSSL:
+
+```shell
+git config --system http.sslbackend openssl
+```
diff --git a/doc/administration/troubleshooting/test_environments.md b/doc/administration/troubleshooting/test_environments.md
index 53e51bbfe80..16ef4f83978 100644
--- a/doc/administration/troubleshooting/test_environments.md
+++ b/doc/administration/troubleshooting/test_environments.md
@@ -92,7 +92,7 @@ gitlab_rails['omniauth_providers'] = [
 
 #### GroupSAML for GitLab.com
 
-See [the GDK SAML documentation](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/saml.md).
+See [the GDK SAML documentation](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/main/doc/howto/saml.md).
 
 ### Elasticsearch