Merge branch 'master' into 27574-pipelines-empty-state27574-pipelines-empty-state

* master: (209 commits)
  Fix Slack link when on notify
  Use Enumerable#index_by where possible
  Protect against unknown emojis in frequently used list
  Fix config option to disable Prometheus
  Fix double click token name
  fix describe block to make the karma reporter happy
  removes n+1 query from tags and branches indexes
  Fix broken links limit lines to 80 chars
  Add prometheus memory requirements, include additional detail on disabling prometheus in docs.
  Add `requirements: { id: %r{[^/]+} }` for all projects and groups namespaced API routes
  Expand on the good changelog/bad changelog documentation examples
  Add policy for closing stale merge requests
  Fix spec
  Use code icon for Raw
  Fix spec
  Add copy button to blob header and use icon for Raw button
  Fix archive prefix bug for refs containing dots
  Include routes when loading user projects
  Fixed search not working in issue boards modal
  Document a New Branch feature for Bare Projects
  ...

3 files changed, 118 insertions, 50 deletions

diff --git a/doc/administration/monitoring/prometheus/index.md b/doc/administration/monitoring/prometheus/index.md
index 3a394c561db..69b16b7c483 100644
--- a/doc/administration/monitoring/prometheus/index.md
+++ b/doc/administration/monitoring/prometheus/index.md
@@ -3,10 +3,9 @@
 >**Notes:**
 - Prometheus and the various exporters listed in this page are bundled in the
   Omnibus GitLab package. Check each exporter's documentation for the timeline
-  they got added. For installations from source you will have to install
-  them yourself. Over subsequent releases additional GitLab metrics will be
-  captured.
-- Prometheus services are off by default but will be on starting with GitLab 9.0.
+  they got added. For installations from source you will have to install them
+  yourself. Over subsequent releases additional GitLab metrics will be captured.
+- Prometheus services are on by default with GitLab 9.0.
 - Prometheus and its exporters do not authenticate users, and will be available
   to anyone who can access them.
 
@@ -26,26 +25,25 @@ dashboard tool like [Grafana].
 ## Configuring Prometheus
 
 >**Note:**
-Available since Omnibus GitLab 8.16. For installations from source you'll
-have to install and configure it yourself.
+For installations from source you'll have to install and configure it yourself.
 
-To enable Prometheus:
+Prometheus and it's exporters are on by default, starting with GitLab 9.0.
+Prometheus will run as the `gitlab-prometheus` user and listen on
+`http://localhost:9090`. Each exporter will be automatically be set up as a
+monitoring target for Prometheus, unless individually disabled.
+
+To disable Prometheus and all of its exporters, as well as any added in the future:
 
 1. Edit `/etc/gitlab/gitlab.rb`
-1. Add or find and uncomment the following line, making sure it's set to `true`:
+1. Add or find and uncomment the following line, making sure it's set to `false`:
 
     ```ruby
-    prometheus['enable'] = true
+    prometheus_monitoring['enable'] = false
     ```
 
 1. Save the file and [reconfigure GitLab][reconfigure] for the changes to
    take effect
 
-By default, Prometheus will run as the `gitlab-prometheus` user and listen on
-`http://localhost:9090`. If the [node exporter](#node-exporter) service
-has been enabled, it will automatically be set up as a monitoring target for
-Prometheus.
-
 ## Changing the port Prometheus listens on
 
 >**Note:**
@@ -71,16 +69,14 @@ To change the address/port that Prometheus listens on:
 
 ## Viewing performance metrics
 
-After you have [enabled Prometheus](#configuring-prometheus), you can visit
-`http://localhost:9090` for the dashboard that Prometheus offers by default.
+You can visit `http://localhost:9090` for the dashboard that Prometheus offers by default.
 
 >**Note:**
 If SSL has been enabled on your GitLab instance, you may not be able to access
 Prometheus on the same browser as GitLab due to [HSTS][hsts]. We plan to
 [provide access via GitLab][multi-user-prometheus], but in the interim there are
 some workarounds: using a separate browser for Prometheus, resetting HSTS, or
-having [Nginx proxy it][nginx-custom-config]. Follow issue [#27069] for more
-information.
+having [Nginx proxy it][nginx-custom-config].
 
 The performance data collected by Prometheus can be viewed directly in the
 Prometheus console or through a compatible dashboard tool.
@@ -96,6 +92,30 @@ Sample Prometheus queries:
 - **Data transmitted:** `irate(node_network_transmit_bytes[5m])`
 - **Data received:** `irate(node_network_receive_bytes[5m])`
 
+## Configuring Prometheus to monitor Kubernetes
+
+> Introduced in GitLab 9.0.
+
+If your GitLab server is running within Kubernetes, an option is now available
+to monitor the health of each node in the cluster. This is particularly helpful
+if your CI/CD environments run in the same cluster, and you would like enable
+[Prometheus integration][] to monitor them.
+
+When enabled, the bundled Prometheus server monitors Kubernetes and automatically
+[collects metrics][prometheus-cadvisor-metrics] from each Node in the cluster.
+
+To enable the Kubernetes monitoring:
+
+1. Edit `/etc/gitlab/gitlab.rb`
+1. Add or find and uncomment the following line:
+
+    ```ruby
+    prometheus['monitor_kubernetes'] = true
+    ```
+
+1. Save the file and [reconfigure GitLab][reconfigure] for the changes to
+   take effect
+
 ## Prometheus exporters
 
 There are a number of libraries and servers which help in exporting existing
@@ -143,5 +163,6 @@ The GitLab monitor exporter allows you to measure various GitLab metrics.
 [prom-grafana]: https://prometheus.io/docs/visualization/grafana/
 [scrape-config]: https://prometheus.io/docs/operating/configuration/#%3Cscrape_config%3E
 [reconfigure]: ../../restart_gitlab.md#omnibus-gitlab-reconfigure
-[#27069]: https://gitlab.com/gitlab-org/gitlab-ce/issues/27069
 [1261]: https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/1261
+[prometheus integration]: ../../../user/project/integrations/prometheus.md
+[rometheus-cadvisor-metrics]: https://github.com/google/cadvisor/blob/master/docs/storage/prometheus.md
diff --git a/doc/administration/pages/index.md b/doc/administration/pages/index.md
index 62b0468da79..0c63b0b59a7 100644
--- a/doc/administration/pages/index.md
+++ b/doc/administration/pages/index.md
@@ -26,9 +26,9 @@ it works.
 
 ---
 
-In the case of [custom domains](#custom-domains) (but not 
-[wildcard domains](#wildcard-domains)), the Pages daemon needs to listen on 
-ports `80` and/or `443`. For that reason, there is some flexibility in the way 
+In the case of [custom domains](#custom-domains) (but not
+[wildcard domains](#wildcard-domains)), the Pages daemon needs to listen on
+ports `80` and/or `443`. For that reason, there is some flexibility in the way
 which you can set it up:
 
 1. Run the Pages daemon in the same server as GitLab, listening on a secondary IP.
@@ -65,11 +65,13 @@ you need to add a [wildcard DNS A record][wiki-wildcard-dns] pointing to the
 host that GitLab runs. For example, an entry would look like this:
 
 ```
-*.example.io. 1800 IN A 1.1.1.1
+*.example.io. 1800 IN A    1.1.1.1
+*.example.io. 1800 IN AAAA 2001::1
 ```
 
 where `example.io` is the domain under which GitLab Pages will be served
-and `1.1.1.1` is the IP address of your GitLab instance.
+and `1.1.1.1` is the IPv4 address of your GitLab instance and `2001::1` is the
+IPv6 address. If you don't have IPv6, you can omit the AAAA record.
 
 > **Note:**
 You should not use the GitLab domain to serve user pages. For more information
@@ -141,7 +143,8 @@ outside world.
 In addition to the wildcard domains, you can also have the option to configure
 GitLab Pages to work with custom domains. Again, there are two options here:
 support custom domains with and without TLS certificates. The easiest setup is
-that without TLS certificates.
+that without TLS certificates. In either case, you'll need a secondary IP. If
+you have IPv6 as well as IPv4 addresses, you can use them both.
 
 ### Custom domains
 
@@ -163,11 +166,12 @@ world. Custom domains are supported, but no TLS.
     pages_external_url "http://example.io"
     nginx['listen_addresses'] = ['1.1.1.1']
     pages_nginx['enable'] = false
-    gitlab_pages['external_http'] = '1.1.1.2:80'
+    gitlab_pages['external_http'] = ['1.1.1.2:80', '[2001::2]:80']
     ```
 
     where `1.1.1.1` is the primary IP address that GitLab is listening to and
-    `1.1.1.2` the secondary IP where the GitLab Pages daemon listens to.
+    `1.1.1.2` and `2001::2` are the secondary IPs the GitLab Pages daemon
+    listens on. If you don't have IPv6, you can omit the IPv6 address.
 
 1. [Reconfigure GitLab][reconfigure]
 
@@ -194,12 +198,13 @@ world. Custom domains and TLS are supported.
     pages_nginx['enable'] = false
     gitlab_pages['cert'] = "/etc/gitlab/ssl/example.io.crt"
     gitlab_pages['cert_key'] = "/etc/gitlab/ssl/example.io.key"
-    gitlab_pages['external_http'] = '1.1.1.2:80'
-    gitlab_pages['external_https'] = '1.1.1.2:443'
+    gitlab_pages['external_http'] = ['1.1.1.2:80', '[2001::2]:80']
+    gitlab_pages['external_https'] = ['1.1.1.2:443', '[2001::2]:443']
     ```
 
     where `1.1.1.1` is the primary IP address that GitLab is listening to and
-    `1.1.1.2` the secondary IP where the GitLab Pages daemon listens to.
+    `1.1.1.2` and `2001::2` are the secondary IPs where the GitLab Pages daemon
+    listens on. If you don't have IPv6, you can omit the IPv6 address.
 
 1. [Reconfigure GitLab][reconfigure]
 
diff --git a/doc/administration/pages/source.md b/doc/administration/pages/source.md
index f6f50e2c571..a45c3306457 100644
--- a/doc/administration/pages/source.md
+++ b/doc/administration/pages/source.md
@@ -1,5 +1,9 @@
 # GitLab Pages administration for source installations
 
+>**Note:**
+Before attempting to enable GitLab Pages, first make sure you have
+[installed GitLab](../../install/installation.md) successfully.
+
 This is the documentation for configuring a GitLab Pages when you have installed
 GitLab from source and not using the Omnibus packages.
 
@@ -13,7 +17,33 @@ Pages to the latest supported version.
 
 ## Overview
 
-[Read the Omnibus overview section.](index.md#overview)
+GitLab Pages makes use of the [GitLab Pages daemon], a simple HTTP server
+written in Go that can listen on an external IP address and provide support for
+custom domains and custom certificates. It supports dynamic certificates through
+SNI and exposes pages using HTTP2 by default.
+You are encouraged to read its [README][pages-readme] to fully understand how
+it works.
+
+---
+
+In the case of [custom domains](#custom-domains) (but not
+[wildcard domains](#wildcard-domains)), the Pages daemon needs to listen on
+ports `80` and/or `443`. For that reason, there is some flexibility in the way
+which you can set it up:
+
+1. Run the Pages daemon in the same server as GitLab, listening on a secondary IP.
+1. Run the Pages daemon in a separate server. In that case, the
+   [Pages path](#change-storage-path) must also be present in the server that
+   the Pages daemon is installed, so you will have to share it via network.
+1. Run the Pages daemon in the same server as GitLab, listening on the same IP
+   but on different ports. In that case, you will have to proxy the traffic with
+   a loadbalancer. If you choose that route note that you should use TCP load
+   balancing for HTTPS. If you use TLS-termination (HTTPS-load balancing) the
+   pages will not be able to be served with user provided certificates. For
+   HTTP it's OK to use HTTP or TCP load balancing.
+
+In this document, we will proceed assuming the first option. If you are not
+supporting custom domains a secondary IP is not needed.
 
 ## Prerequisites
 
@@ -75,7 +105,7 @@ The Pages daemon doesn't listen to the outside world.
     cd /home/git
     sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-pages.git
     cd gitlab-pages
-    sudo -u git -H git checkout v0.2.4
+    sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_PAGES_VERSION)
     sudo -u git -H make
     ```
 
@@ -100,14 +130,21 @@ The Pages daemon doesn't listen to the outside world.
        https: false
      ```
 
-1. Copy the `gitlab-pages-ssl` Nginx configuration file:
+1. Edit `/etc/default/gitlab` and set `gitlab_pages_enabled` to `true` in
+   order to enable the pages daemon. In `gitlab_pages_options` the
+   `-pages-domain` must match the `host` setting that you set above.
 
-    ```bash
-    sudo cp lib/support/nginx/gitlab-pages-ssl /etc/nginx/sites-available/gitlab-pages-ssl.conf
-    sudo ln -sf /etc/nginx/sites-{available,enabled}/gitlab-pages-ssl.conf
     ```
+    gitlab_pages_enabled=true
+    gitlab_pages_options="-pages-domain example.io -pages-root $app_root/shared/pages -listen-proxy 127.0.0.1:8090
+    ```
+
+1. Copy the `gitlab-pages` Nginx configuration file:
 
-      Replace `gitlab-pages-ssl` with `gitlab-pages` if you are not using SSL.
+    ```bash
+    sudo cp lib/support/nginx/gitlab-pages /etc/nginx/sites-available/gitlab-pages.conf
+    sudo ln -sf /etc/nginx/sites-{available,enabled}/gitlab-pages.conf
+    ```
 
 1. Restart NGINX
 1. [Restart GitLab][restart]
@@ -131,7 +168,7 @@ outside world.
     cd /home/git
     sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-pages.git
     cd gitlab-pages
-    sudo -u git -H git checkout v0.2.4
+    sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_PAGES_VERSION)
     sudo -u git -H make
     ```
 
@@ -149,6 +186,17 @@ outside world.
        https: true
      ```
 
+1. Edit `/etc/default/gitlab` and set `gitlab_pages_enabled` to `true` in
+   order to enable the pages daemon. In `gitlab_pages_options` the
+   `-pages-domain` must match the `host` setting that you set above.
+   The `-root-cert` and `-root-key` settings are the wildcard TLS certificates
+   of the `example.io` domain:
+
+    ```
+    gitlab_pages_enabled=true
+    gitlab_pages_options="-pages-domain example.io -pages-root $app_root/shared/pages -listen-proxy 127.0.0.1:8090 -root-cert /path/to/example.io.crt -root-key /path/to/example.io.key
+    ```
+
 1. Copy the `gitlab-pages-ssl` Nginx configuration file:
 
     ```bash
@@ -156,12 +204,9 @@ outside world.
     sudo ln -sf /etc/nginx/sites-{available,enabled}/gitlab-pages-ssl.conf
     ```
 
-      Replace `gitlab-pages-ssl` with `gitlab-pages` if you are not using SSL.
-
 1. Restart NGINX
 1. [Restart GitLab][restart]
 
-
 ## Advanced configuration
 
 In addition to the wildcard domains, you can also have the option to configure
@@ -189,7 +234,7 @@ world. Custom domains are supported, but no TLS.
     cd /home/git
     sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-pages.git
     cd gitlab-pages
-    sudo -u git -H git checkout v0.2.4
+    sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_PAGES_VERSION)
     sudo -u git -H make
     ```
 
@@ -224,12 +269,10 @@ world. Custom domains are supported, but no TLS.
 1. Copy the `gitlab-pages-ssl` Nginx configuration file:
 
     ```bash
-    sudo cp lib/support/nginx/gitlab-pages-ssl /etc/nginx/sites-available/gitlab-pages-ssl.conf
-    sudo ln -sf /etc/nginx/sites-{available,enabled}/gitlab-pages-ssl.conf
+    sudo cp lib/support/nginx/gitlab-pages /etc/nginx/sites-available/gitlab-pages.conf
+    sudo ln -sf /etc/nginx/sites-{available,enabled}/gitlab-pages.conf
     ```
 
-      Replace `gitlab-pages-ssl` with `gitlab-pages` if you are not using SSL.
-
 1. Edit all GitLab related configs in `/etc/nginx/site-available/` and replace
    `0.0.0.0` with `1.1.1.1`, where `1.1.1.1` the primary IP where GitLab
    listens to.
@@ -257,7 +300,7 @@ world. Custom domains and TLS are supported.
     cd /home/git
     sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-pages.git
     cd gitlab-pages
-    sudo -u git -H git checkout v0.2.4
+    sudo -u git -H git checkout v$(</home/git/gitlab/GITLAB_PAGES_VERSION)
     sudo -u git -H make
     ```
 
@@ -300,8 +343,6 @@ world. Custom domains and TLS are supported.
     sudo ln -sf /etc/nginx/sites-{available,enabled}/gitlab-pages-ssl.conf
     ```
 
-      Replace `gitlab-pages-ssl` with `gitlab-pages` if you are not using SSL.
-
 1. Edit all GitLab related configs in `/etc/nginx/site-available/` and replace
    `0.0.0.0` with `1.1.1.1`, where `1.1.1.1` the primary IP where GitLab
    listens to.
@@ -392,5 +433,6 @@ than GitLab to prevent XSS attacks.
 [pages-userguide]: ../../user/project/pages/index.md
 [reconfigure]: ../restart_gitlab.md#omnibus-gitlab-reconfigure
 [restart]: ../restart_gitlab.md#installations-from-source
-[gitlab-pages]: https://gitlab.com/gitlab-org/gitlab-pages/tree/v0.2.4
+[gitlab-pages]: https://gitlab.com/gitlab-org/gitlab-pages/tree/v0.4.0
+[gl-example]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/lib/support/init.d/gitlab.default.example
 [shared runners]: ../../ci/runners/README.md