From 8b573c94895dc0ac0e1d9d59cf3e8745e8b539ca Mon Sep 17 00:00:00 2001 From: GitLab Bot Date: Thu, 17 Dec 2020 11:59:07 +0000 Subject: Add latest changes from gitlab-org/gitlab@13-7-stable-ee --- .../pages/img/zip_cache_configuration.png | Bin 0 -> 21703 bytes doc/administration/pages/index.md | 152 ++++++++++++++++++--- doc/administration/pages/source.md | 12 +- 3 files changed, 140 insertions(+), 24 deletions(-) create mode 100644 doc/administration/pages/img/zip_cache_configuration.png (limited to 'doc/administration/pages') diff --git a/doc/administration/pages/img/zip_cache_configuration.png b/doc/administration/pages/img/zip_cache_configuration.png new file mode 100644 index 00000000000..a332d8d0eb5 Binary files /dev/null and b/doc/administration/pages/img/zip_cache_configuration.png differ diff --git a/doc/administration/pages/index.md b/doc/administration/pages/index.md index 0ebeb96d247..af0476b72e9 100644 --- a/doc/administration/pages/index.md +++ b/doc/administration/pages/index.md @@ -1,7 +1,7 @@ --- stage: Release -group: Release Management -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +group: Release +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments description: 'Learn how to administer GitLab Pages.' --- @@ -16,7 +16,7 @@ description: 'Learn how to administer GitLab Pages.' GitLab Pages allows for hosting of static sites. It must be configured by an administrator. Separate [user documentation](../../user/project/pages/index.md) is available. -NOTE: **Note:** +NOTE: This guide is for Omnibus GitLab installations. If you have installed GitLab from source, see [GitLab Pages administration for source installations](source.md). @@ -53,7 +53,7 @@ supporting custom domains a secondary IP is not needed. Before proceeding with the Pages configuration, you will need to: -1. Have a domain for Pages that is not a subdomain of your GitLab's instance domain. +1. Have a domain for Pages that is not a subdomain of your GitLab instance domain. | GitLab domain | Pages domain | Does it work? | | :---: | :---: | :---: | @@ -68,7 +68,7 @@ Before proceeding with the Pages configuration, you will need to: so that your users don't have to bring their own. 1. (Only for custom domains) Have a **secondary IP**. -NOTE: **Note:** +NOTE: If your GitLab instance and the Pages daemon are deployed in a private network or behind a firewall, your GitLab Pages websites will only be accessible to devices/users that have access to the private network. ### Add the domain to the Public Suffix List @@ -101,7 +101,7 @@ where `example.io` is the domain under which GitLab Pages will be served and `192.0.2.1` is the IPv4 address of your GitLab instance and `2001::1` is the IPv6 address. If you don't have IPv6, you can omit the AAAA record. -NOTE: **Note:** +NOTE: You should not use the GitLab domain to serve user pages. For more information see the [security section](#security). ## Configuration @@ -181,7 +181,7 @@ behavior: 1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). -NOTE: **Note:** +NOTE: `inplace_chroot` option might not work with the other features, such as [Pages Access Control](#access-control). The [GitLab Pages README](https://gitlab.com/gitlab-org/gitlab-pages#caveats) has more information about caveats and workarounds. @@ -204,13 +204,13 @@ control over how the Pages daemon runs and serves content in your environment. | `artifacts_server_url` | API URL to proxy artifact requests to. Defaults to GitLab `external URL` + `/api/v4`, for example `https://gitlab.com/api/v4`. | `auth_redirect_uri` | Callback URL for authenticating with GitLab. Defaults to project's subdomain of `pages_external_url` + `/auth`. | `auth_secret` | Secret key for signing authentication requests. Leave blank to pull automatically from GitLab during OAuth registration. -| `dir` | Working directory for config and secrets files. +| `dir` | Working directory for configuration and secrets files. | `enable` | Enable or disable GitLab Pages on the current system. | `external_http` | Configure Pages to bind to one or more secondary IP addresses, serving HTTP requests. Multiple addresses can be given as an array, along with exact ports, for example `['1.2.3.4', '1.2.3.5:8063']`. Sets value for `listen_http`. | `external_https` | Configure Pages to bind to one or more secondary IP addresses, serving HTTPS requests. Multiple addresses can be given as an array, along with exact ports, for example `['1.2.3.4', '1.2.3.5:8063']`. Sets value for `listen_https`. | `gitlab_client_http_timeout` | GitLab API HTTP client connection timeout in seconds (default: 10s). | `gitlab_client_jwt_expiry` | JWT Token expiry time in seconds (default: 30s). -| `domain_config_source` | Domain configuration source (default: `disk`) +| `domain_config_source` | Domain configuration source (default: `auto`) | `gitlab_id` | The OAuth application public ID. Leave blank to automatically fill when Pages authenticates with GitLab. | `gitlab_secret` | The OAuth application secret. Leave blank to automatically fill when Pages authenticates with GitLab. | `gitlab_server` | Server to use for authentication when access control is enabled; defaults to GitLab `external_url`. @@ -241,7 +241,7 @@ control over how the Pages daemon runs and serves content in your environment. | `pages_path` | The directory on disk where pages are stored, defaults to `GITLAB-RAILS/shared/pages`. | `pages_nginx[]` | | | `enable` | Include a virtual host `server{}` block for Pages inside NGINX. Needed for NGINX to proxy traffic back to the Pages daemon. Set to `false` if the Pages daemon should directly receive all requests, for example, when using [custom domains](index.md#custom-domains). -| `FF_ENABLE_REDIRECTS` | Feature flag to disable redirects (enabled by default). Read the [redirects documentation](../../user/project/pages/redirects.md#disable-redirects) for more info. | +| `FF_ENABLE_REDIRECTS` | Feature flag to disable redirects (enabled by default). Read the [redirects documentation](../../user/project/pages/redirects.md#disable-redirects) for more information. | --- @@ -375,7 +375,7 @@ Pages access control is disabled by default. To enable it: 1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). 1. Users can now configure it in their [projects' settings](../../user/project/pages/pages_access_control.md). -NOTE: **Important:** +NOTE: For this setting to be effective with multi-node setups, it has to be applied to all the App nodes and Sidekiq nodes. @@ -395,7 +395,7 @@ To do that: 1. Check the **Disable public access to Pages sites** checkbox. 1. Click **Save changes**. -CAUTION: **Warning:** +WARNING: For self-managed installations, all public websites remain private until they are redeployed. This issue will be resolved by [sourcing domain configuration from the GitLab API](https://gitlab.com/gitlab-org/gitlab/-/issues/218357). @@ -427,6 +427,42 @@ Authority (CA) in the system certificate store. For Omnibus, this is fixed by [installing a custom CA in Omnibus GitLab](https://docs.gitlab.com/omnibus/settings/ssl.html#install-custom-public-certificates). +### Zip serving and cache configuration + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-pages/-/merge_requests/392) in GitLab 13.7. + +WARNING: +These are advanced settings. The recommended default values are set inside GitLab Pages. You should +change these settings only if absolutely necessary. Use extreme caution. + +GitLab Pages can serve content from zip archives through object storage (an +[issue](https://gitlab.com/gitlab-org/gitlab-pages/-/issues/485) exists for supporting disk storage +as well). It uses an in-memory cache to increase the performance when serving content from a zip +archive. You can modify the cache behavior by changing the following configuration flags. + +| Setting | Description | +| ------- | ----------- | +| `zip_cache_expiration` | The cache expiration interval of zip archives. Must be greater than zero to avoid serving stale content. Default is 60s. | +| `zip_cache_cleanup` | The interval at which archives are cleaned from memory if they have already expired. Default is 30s. | +| `zip_cache_refresh` | The time interval in which an archive is extended in memory if accessed before `zip_cache_expiration`. This works together with `zip_cache_expiration` to determine if an archive is extended in memory. See the [example below](#zip-cache-refresh-example) for important details. Default is 30s. | +| `zip_open_timeout` | The maximum time allowed to open a zip archive. Increase this time for big archives or slow network connections, as doing so may affect the latency of serving Pages. Default is 30s. | + +#### Zip cache refresh example + +Archives are refreshed in the cache (extending the time they are held in memory) if they're accessed +before `zip_cache_expiration`, and the time left before expiring is less than or equal to +`zip_cache_refresh`. For example, if `archive.zip` is accessed at time 0s, it expires in 60s (the +default for `zip_cache_expiration`). In the example below, if the archive is opened again after 15s +it is **not** refreshed because the time left for expiry (45s) is greater than `zip_cache_refresh` +(default 30s). However, if the archive is accessed again after 45s (from the first time it was +opened) it's refreshed. This extends the time the archive remains in memory from +`45s + zip_cache_expiration (60s)`, for a total of 105s. + +After an archive reaches `zip_cache_expiration`, it's marked as expired and removed on the next +`zip_cache_cleanup` interval. + +![Zip cache configuration](img/zip_cache_configuration.png) + ## Activate verbose logging for daemon Verbose logging was [introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/2533) in @@ -539,7 +575,7 @@ your main application server. To configure GitLab Pages on a separate server: -DANGER: **Warning:** +WARNING: The following procedure includes steps to back up and edit the `gitlab-secrets.json` file. This file contains secrets that control database encryption. Proceed with caution. @@ -632,13 +668,14 @@ Pages server. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/217912) in GitLab 13.3. GitLab Pages can use different sources to get domain configuration. -The default value is `nil`; however, GitLab Pages will default to `disk`. +The default value is `nil`; however, GitLab Pages will default to `auto`. ```ruby gitlab_pages['domain_config_source'] = nil ``` -You can specify `gitlab` to enable [API-based configuration](#gitlab-api-based-configuration). +If left unchanged, GitLab Pages tries to use any available source (either `gitlab` or `disk`). The +preferred source is `gitlab`, which uses [API-based configuration](#gitlab-api-based-configuration). For more details see this [blog post](https://about.gitlab.com/blog/2020/08/03/how-gitlab-pages-uses-the-gitlab-api-to-serve-content/). @@ -655,15 +692,86 @@ was used prior to GitLab 13.0. Follow these steps to enable it: 1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. -If you encounter an issue, you can disable it by choosing `disk` or `nil`: +If you encounter an issue, you can disable it by choosing `disk`: ```ruby -gitlab_pages['domain_config_source'] = nil +gitlab_pages['domain_config_source'] = "disk" ``` For other common issues, see the [troubleshooting section](#failed-to-connect-to-the-internal-gitlab-api) or report an issue. +## Using object storage + +> [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5577) in GitLab 13.6. + +[Read more about using object storage with GitLab](../object_storage.md). + +### Object storage settings + +The following settings are: + +- Nested under `pages:` and then `object_store:` on source installations. +- Prefixed by `pages_object_store_` on Omnibus GitLab installations. + +| Setting | Description | Default | +|---------|-------------|---------| +| `enabled` | Whether object storage is enabled. | `false` | +| `remote_directory` | The name of the bucket where Pages site content is stored. | | +| `connection` | Various connection options described below. | | + +#### S3-compatible connection settings + +See [the available connection settings for different providers](../object_storage.md#connection-settings). + +In Omnibus installations: + +1. Add the following lines to `/etc/gitlab/gitlab.rb` and replace the values with the ones you want: + + ```ruby + gitlab_rails['pages_object_store_enabled'] = true + gitlab_rails['pages_object_store_remote_directory'] = "pages" + gitlab_rails['pages_object_store_connection'] = { + 'provider' => 'AWS', + 'region' => 'eu-central-1', + 'aws_access_key_id' => 'AWS_ACCESS_KEY_ID', + 'aws_secret_access_key' => 'AWS_SECRET_ACCESS_KEY' + } + ``` + + If you use AWS IAM profiles, be sure to omit the AWS access key and secret access key/value + pairs: + + ```ruby + gitlab_rails['pages_object_store_connection'] = { + 'provider' => 'AWS', + 'region' => 'eu-central-1', + 'use_iam_profile' => true + } + ``` + +1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) + for the changes to take effect. + +In installations from source: + +1. Edit `/home/git/gitlab/config/gitlab.yml` and add or amend the following lines: + + ```yaml + pages: + object_store: + enabled: true + remote_directory: "pages" # The bucket name + connection: + provider: AWS # Only AWS supported at the moment + aws_access_key_id: AWS_ACESS_KEY_ID + aws_secret_access_key: AWS_SECRET_ACCESS_KEY + region: eu-central-1 + ``` + +1. Save the file and [restart GitLab](../restart_gitlab.md#installations-from-source) + for the changes to take effect. + ## Backup GitLab Pages are part of the [regular backup](../../raketasks/backup_restore.md), so there is no separate backup to configure. @@ -734,7 +842,7 @@ x509: certificate signed by unknown authority ``` The reason for those errors is that the files `resolv.conf` and `ca-bundle.pem` are missing inside the chroot. -The fix is to copy the host's `/etc/resolv.conf` and GitLab's certificate bundle inside the chroot: +The fix is to copy the host's `/etc/resolv.conf` and the GitLab certificate bundle inside the chroot: ```shell sudo mkdir -p /var/opt/gitlab/gitlab-rails/shared/pages/etc/ssl @@ -811,3 +919,11 @@ Upgrading to an [officially supported operating system](https://about.gitlab.com This problem comes from the permissions of the GitLab Pages OAuth application. To fix it, go to **Admin > Applications > GitLab Pages** and edit the application. Under **Scopes**, ensure that the `api` scope is selected and save your changes. + +### Workaround in case no wildcard DNS entry can be set + +If the wildcard DNS [prerequisite](#prerequisites) can't be met, you can still use GitLab Pages in a limited fashion: + +1. [Move](../../user/project/settings/index.md#transferring-an-existing-project-into-another-namespace) + all projects you need to use Pages with into a single group namespace, for example `pages`. +1. Configure a [DNS entry](#dns-configuration) without the `*.`-wildcard, for example `pages.example.io`. diff --git a/doc/administration/pages/source.md b/doc/administration/pages/source.md index 87217b141a4..c7c25f0f3a7 100644 --- a/doc/administration/pages/source.md +++ b/doc/administration/pages/source.md @@ -1,12 +1,12 @@ --- stage: Release -group: Release Management -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +group: Release +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # GitLab Pages administration for source installations -NOTE: **Note:** +NOTE: Before attempting to enable GitLab Pages, first make sure you have [installed GitLab](../../install/installation.md) successfully. @@ -77,7 +77,7 @@ host that GitLab runs. For example, an entry would look like this: where `example.io` is the domain under which GitLab Pages will be served and `192.0.2.1` is the IP address of your GitLab instance. -NOTE: **Note:** +NOTE: You should not use the GitLab domain to serve user pages. For more information see the [security section](#security). @@ -349,7 +349,7 @@ world. Custom domains and TLS are supported. ## NGINX caveats -NOTE: **Note:** +NOTE: The following information applies only for installations from source. Be extra careful when setting up the domain name in the NGINX configuration. You must @@ -395,7 +395,7 @@ API to check that the user is authorized to read that site. From [GitLab 12.8](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/3689) onward, Access Control parameters for Pages are set in a configuration file, which by convention is named `gitlab-pages-config`. The configuration file is passed to -pages using the `-config flag` or CONFIG environment variable. +pages using the `-config flag` or `CONFIG` environment variable. Pages access control is disabled by default. To enable it: -- cgit v1.2.1