Add latest changes from gitlab-org/gitlab@master

3 files changed, 63 insertions, 3 deletions

diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md
index a441dc9c4e1..c12826c80cd 100644
--- a/doc/administration/gitaly/index.md
+++ b/doc/administration/gitaly/index.md
@@ -1006,6 +1006,12 @@ unset http_proxy
 unset https_proxy
 ```
 
+### Gitaly not listening on new address after reconfiguring
+
+When updating the `gitaly['listen_addr']` or `gitaly['prometheus_listen_addr']` values, Gitaly may continue to listen on the old address after a `sudo gitlab-ctl reconfigure`.
+
+When this occurs, performing a `sudo gitlab-ctl restart` will resolve the issue.  This will no longer be necessary after [this issue](https://gitlab.com/gitlab-org/gitaly/issues/2521) is resolved.
+
 ### Praefect
 
 Praefect is an experimental daemon that allows for replication of the Git data.
diff --git a/doc/administration/gitaly/praefect.md b/doc/administration/gitaly/praefect.md
index 783ae1e0317..42437ecd7f4 100644
--- a/doc/administration/gitaly/praefect.md
+++ b/doc/administration/gitaly/praefect.md
@@ -84,6 +84,8 @@ with secure tokens as you complete the setup process.
    Praefect cluster directly; that could lead to data loss.
 1. `PRAEFECT_SQL_PASSWORD`: this password is used by Praefect to connect to
    PostgreSQL.
+1. `GRAFANA_PASSWORD`: this password is used to access the `admin`
+   account in the Grafana dashboards.
 
 We will note in the instructions below where these secrets are required.
 
@@ -184,6 +186,10 @@ application server, or a Gitaly node.
    # Make Praefect accept connections on all network interfaces.
    # Use firewalls to restrict access to this address/port.
    praefect['listen_addr'] = '0.0.0.0:2305'
+
+   # Enable Prometheus metrics access to Praefect. You must use firewalls
+   # to restrict access to this address/port.
+   praefect['prometheus_listen_addr'] = '0.0.0.0:9652'
    ```
 
 1. Configure a strong `auth_token` for **Praefect** by editing
@@ -354,6 +360,10 @@ documentation](index.md#3-gitaly-server-configuration).
    # Make Gitaly accept connections on all network interfaces.
    # Use firewalls to restrict access to this address/port.
    gitaly['listen_addr'] = '0.0.0.0:8075'
+
+   # Enable Prometheus metrics access to Gitaly. You must use firewalls
+   # to restrict access to this address/port.
+   gitaly['prometheus_listen_addr'] = '0.0.0.0:9236'
    ```
 
 1. Configure a strong `auth_token` for **Gitaly** by editing
@@ -453,7 +463,7 @@ Particular attention should be shown to:
 
    You will need to replace:
 
-   - `PRAEFECT_URL_OR_IP` with the IP/host address of the Praefect node
+   - `PRAEFECT_HOST` with the IP address or hostname of the Praefect node
    - `PRAEFECT_EXTERNAL_TOKEN` with the real secret
 
    ```ruby
@@ -462,7 +472,7 @@ Particular attention should be shown to:
        "path" => "/var/opt/gitlab/git-data"
      },
      "praefect" => {
-       "gitaly_address" => "tcp://PRAEFECT_URL_OR_IP:2305",
+       "gitaly_address" => "tcp://PRAEFECT_HOST:2305",
        "gitaly_token" => 'PRAEFECT_EXTERNAL_TOKEN'
      }
    })
@@ -478,6 +488,38 @@ Particular attention should be shown to:
    gitlab_shell['secret_token'] = 'GITLAB_SHELL_SECRET_TOKEN'
    ```
 
+1. Add Prometheus monitoring settings by editing `/etc/gitlab/gitlab.rb`.
+
+   You will need to replace:
+
+   - `PRAEFECT_HOST` with the IP address or hostname of the Praefect node
+   - `GITALY_HOST` with the IP address or hostname of each Gitaly node
+
+   ```ruby
+   prometheus['scrape_configs'] = [
+     {
+       'job_name' => 'praefect',
+       'static_configs' => [
+         'targets' => [
+           'PRAEFECT_HOST:9652' # praefect
+         ]
+       ]
+     },
+     {
+       'job_name' => 'praefect-gitaly',
+       'static_configs' => [
+         'targets' => [
+           'GITALY_HOST:9236', # gitaly-1
+           'GITALY_HOST:9236', # gitaly-2
+           'GITALY_HOST:9236', # gitaly-3
+         ]
+       ]
+     }
+   ]
+
+   grafana['disable_login_form'] = false
+   ```
+
 1. Save the changes to `/etc/gitlab/gitlab.rb` and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure):
 
    ```shell
@@ -490,6 +532,12 @@ Particular attention should be shown to:
    sudo gitlab-rake gitlab:gitaly:check
    ```
 
+1. Set the Grafana admin password. This command will prompt you to enter a new password:
+
+   ```shell
+   sudo gitlab-ctl set-grafana-password
+   ```
+
 1. Update the **Repository storage** settings from **Admin Area > Settings >
    Repository > Repository storage** to make the newly configured Praefect
    cluster the storage location for new Git repositories.
@@ -502,7 +550,12 @@ Particular attention should be shown to:
    repository that viewed. If the project is created, and you can see the
    README file, it works!
 
-Congratulations! You have configured a highly available Praefect cluster, and
+1. Inspect metrics by browsing to `/-/grafana` on your GitLab server.
+   Log in with `admin` / `GRAFANA_PASSWORD`. Go to 'Explore' and query
+   `gitlab_build_info` to verify that you are getting metrics from all your
+   machines.
+
+Congratulations! You have configured a highly available Praefect cluster.
 
 ## Migrating existing repositories to Praefect
 
diff --git a/doc/administration/raketasks/maintenance.md b/doc/administration/raketasks/maintenance.md
index 6dc5542466f..b45f1ae529d 100644
--- a/doc/administration/raketasks/maintenance.md
+++ b/doc/administration/raketasks/maintenance.md
@@ -58,6 +58,7 @@ Runs the following rake tasks:
 - `gitlab:app:check`
 
 It will check that each component was set up according to the installation guide and suggest fixes for issues found.
+This command must be run from your app server and will not work correctly on component servers like [Gitaly](../gitaly/index.md#running-gitaly-on-its-own-server).
 
 You may also have a look at our Troubleshooting Guides: