diff options
| author | GitLab Bot <gitlab-bot@gitlab.com> | 2022-02-18 09:45:46 +0000 |
|---|---|---|
| committer | GitLab Bot <gitlab-bot@gitlab.com> | 2022-02-18 09:45:46 +0000 |
| commit | a7b3560714b4d9cc4ab32dffcd1f74a284b93580 (patch) | |
| tree | 7452bd5c3545c2fa67a28aa013835fb4fa071baf /doc/administration | |
| parent | ee9173579ae56a3dbfe5afe9f9410c65bb327ca7 (diff) | |
| download | gitlab-ce-a7b3560714b4d9cc4ab32dffcd1f74a284b93580.tar.gz | |
Add latest changes from gitlab-org/gitlab@14-8-stable-eev14.8.0-rc42
Diffstat (limited to 'doc/administration')
90 files changed, 1330 insertions, 640 deletions
diff --git a/doc/administration/audit_event_streaming.md b/doc/administration/audit_event_streaming.md index eac54416924..3bdc67e5a69 100644 --- a/doc/administration/audit_event_streaming.md +++ b/doc/administration/audit_event_streaming.md @@ -8,12 +8,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/332747) in GitLab 14.5 [with a flag](../administration/feature_flags.md) named `ff_external_audit_events_namespace`. Disabled by default. > - [Enabled on GitLab.com and by default on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/338939) in GitLab 14.7. - -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature per group, ask an administrator to [disable the feature flag](../administration/feature_flags.md) named `ff_external_audit_events_namespace`. On GitLab.com, this feature is available. +> - [Feature flag `ff_external_audit_events_namespace`](https://gitlab.com/gitlab-org/gitlab/-/issues/349588) removed in GitLab 14.8. Event streaming allows owners of top-level groups to set an HTTP endpoint to receive **all** audit events about the group, and its -subgroups and projects. +subgroups and projects as structured JSON. Top-level group owners can manage their audit logs in third-party systems such as Splunk, using the Splunk [HTTP Event Collector](https://docs.splunk.com/Documentation/Splunk/8.2.2/Data/UsetheHTTPEventCollector). Any service that can receive @@ -37,6 +35,7 @@ mutation { externalAuditEventDestination { destinationUrl group { + verificationToken name } } @@ -60,6 +59,7 @@ query { externalAuditEventDestinations { nodes { destinationUrl + verificationToken id } } @@ -68,3 +68,13 @@ query { ``` If the resulting list is empty, then audit event streaming is not enabled for that group. + +## Verify event authenticity + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345424) in GitLab 14.8. + +Each streaming destination has a unique verification token (`verificationToken`) that can be used to verify the authenticity of the event. This +token is generated when the event destination is created and cannot be changed. + +Each streamed event contains a random alphanumeric identifier for the `X-Gitlab-Event-Streaming-Token` HTTP header that can be verified against +the destination's value when [listing streaming destinations](#list-currently-enabled-streaming-destinations). diff --git a/doc/administration/audit_events.md b/doc/administration/audit_events.md index 06ad16bbcba..d4902a18cac 100644 --- a/doc/administration/audit_events.md +++ b/doc/administration/audit_events.md @@ -51,7 +51,10 @@ There are two kinds of events logged: When a user is being [impersonated](../user/admin_area/index.md#user-impersonation), their actions are logged as audit events as usual, with two additional details: 1. Usual audit events include information about the impersonating administrator. These are visible in their respective Audit Event pages depending on their type (Group/Project/User). -1. Extra audit events are recorded for the start and stop of the administrator's impersonation session. These are visible in the instance Audit Events. +1. Extra audit events are recorded for the start and stop of the administrator's impersonation session. These are visible in + the: + - Instance audit events. + - Group audit events for all groups the user belongs to (GitLab 14.8 and later). This is limited to 20 groups for performance reasons.  @@ -103,6 +106,7 @@ From there, you can see the following actions: - Group CI/CD variable added, removed, or protected status changed. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30857) in GitLab 13.3. - Compliance framework created, updated, or deleted. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340649) in GitLab 14.5. - Event streaming destination created, updated, or deleted. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/344664) in GitLab 14.6. +- Instance administrator started or stopped impersonation of a group member. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/300961) in GitLab 14.8. Group events can also be accessed via the [Group Audit Events API](../api/audit_events.md#group-audit-events) @@ -236,10 +240,13 @@ Don't see the event you want in any of the epics linked above? You can either: ### Disabled events -#### Repository push +#### Repository push (DEPRECATED) > [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/337993) in GitLab 14.3. +WARNING: +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/337993) in GitLab 14.3. + The current architecture of audit events is not prepared to receive a very high amount of records. It may make the user interface for your project or audit events very busy, and the disk space consumed by the `audit_events` PostgreSQL table may increase considerably. It's disabled by default diff --git a/doc/administration/auditor_users.md b/doc/administration/auditor_users.md index 5498ea9d4be..12e222290e7 100644 --- a/doc/administration/auditor_users.md +++ b/doc/administration/auditor_users.md @@ -64,7 +64,7 @@ To create an Auditor user: 1. Select **Create user** or **Save changes** if you created a new user or edited an existing one respectively. -To revoke Auditor permissions from a user, make them a regular user by +To revoke Auditor permissions from a user, make them a Regular user by following the previous steps. Additionally users can be set as an Auditor using [SAML groups](../integration/saml.md#auditor-groups). diff --git a/doc/administration/auth/atlassian.md b/doc/administration/auth/atlassian.md index 08d07b13e22..7c6aee2c2da 100644 --- a/doc/administration/auth/atlassian.md +++ b/doc/administration/auth/atlassian.md @@ -1,7 +1,7 @@ --- type: reference stage: Manage -group: Authentication & Authorization +group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/auth/authentiq.md b/doc/administration/auth/authentiq.md index 23735b75991..a4a085a494c 100644 --- a/doc/administration/auth/authentiq.md +++ b/doc/administration/auth/authentiq.md @@ -1,7 +1,7 @@ --- type: reference stage: Manage -group: Authentication & Authorization +group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/auth/cognito.md b/doc/administration/auth/cognito.md index 2509bbb73d7..66f8407894c 100644 --- a/doc/administration/auth/cognito.md +++ b/doc/administration/auth/cognito.md @@ -1,7 +1,7 @@ --- type: concepts, howto stage: Manage -group: Authentication & Authorization +group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/auth/crowd.md b/doc/administration/auth/crowd.md index 9a5f6b0a642..8dfa832a103 100644 --- a/doc/administration/auth/crowd.md +++ b/doc/administration/auth/crowd.md @@ -1,7 +1,7 @@ --- type: reference stage: Manage -group: Authentication & Authorization +group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/auth/index.md b/doc/administration/auth/index.md index d469988e719..d09344629e2 100644 --- a/doc/administration/auth/index.md +++ b/doc/administration/auth/index.md @@ -2,7 +2,7 @@ comments: false type: index stage: Manage -group: Authentication & Authorization +group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -50,3 +50,31 @@ For more information, see the links shown on this page for each external provide | **Authentication** | SAML at top-level group (1 provider) | LDAP (multiple providers)<br>Generic OAuth2<br>SAML (only 1 permitted per unique provider)<br>Kerberos<br>JWT<br>Smartcard<br>OmniAuth Providers (only 1 permitted per unique provider) | | **Provider-to-GitLab Role Sync** | SAML Group Sync | LDAP Group Sync | | **User Removal** | SCIM (remove user from top-level group) | LDAP (Blocking User from Instance) | + +## Change apps or configuration + +When GitLab doesn't support having multiple providers (such as OAuth), GitLab configuration and user identification must be +updated at the same time if the provider or app is changed. + +These instructions apply to all methods of authentication where GitLab stores an `extern_uid` and it is the only data used +for user authentication. + +When changing apps within a provider, if the user `extern_uid` does not change, only the GitLab configuration must be +updated. + +To swap configurations: + +1. Change provider configuration in your `gitlab.rb` file. +1. Update `extern_uid` for all users that have an identity in GitLab for the previous provider. + +To find the `extern_uid`, look at an existing user's current `extern_uid` for an ID that matches the appropriate field in +your current provider for the same user. + +There are two methods to update the `extern_uid`: + +- Using the [Users API](../../api/users.md#user-modification). Pass the provider name and the new `extern_uid`. +- Using the [Rails console](../operations/rails_console.md): + + ```ruby + Identity.where(extern_uid: 'old-id').update!(extern_uid: 'new-id')` + ``` diff --git a/doc/administration/auth/jwt.md b/doc/administration/auth/jwt.md index ac2f699dd5d..98f928fd7ee 100644 --- a/doc/administration/auth/jwt.md +++ b/doc/administration/auth/jwt.md @@ -1,7 +1,7 @@ --- type: reference stage: Manage -group: Authentication & Authorization +group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/auth/ldap/google_secure_ldap.md b/doc/administration/auth/ldap/google_secure_ldap.md index 6b8e699e861..10745c5e2bf 100644 --- a/doc/administration/auth/ldap/google_secure_ldap.md +++ b/doc/administration/auth/ldap/google_secure_ldap.md @@ -1,7 +1,7 @@ --- type: reference stage: Manage -group: Authentication & Authorization +group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/auth/ldap/index.md b/doc/administration/auth/ldap/index.md index b773281b216..2a396c4d53a 100644 --- a/doc/administration/auth/ldap/index.md +++ b/doc/administration/auth/ldap/index.md @@ -1,7 +1,7 @@ --- type: reference stage: Manage -group: Authentication & Authorization +group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -468,7 +468,7 @@ If initially your LDAP configuration looked like: password: '123' ``` -1. Edit `/etc/gitlab/gitlab.rb` and remove the settings for `user_bn` and `password`. +1. Edit `/etc/gitlab/gitlab.rb` and remove the settings for `user_dn` and `password`. 1. [Reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. @@ -502,7 +502,7 @@ If initially your LDAP configuration looked like: password: '123' ``` -1. Edit `config/gitlab.yaml` and remove the settings for `user_bn` and `password`. +1. Edit `config/gitlab.yaml` and remove the settings for `user_dn` and `password`. 1. [Restart GitLab](../../restart_gitlab.md#installations-from-source) for the changes to take effect. diff --git a/doc/administration/auth/ldap/ldap-troubleshooting.md b/doc/administration/auth/ldap/ldap-troubleshooting.md index 97a626f4a40..06fe579e101 100644 --- a/doc/administration/auth/ldap/ldap-troubleshooting.md +++ b/doc/administration/auth/ldap/ldap-troubleshooting.md @@ -1,7 +1,7 @@ --- type: reference stage: Manage -group: Authentication & Authorization +group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -382,7 +382,7 @@ the following are true: - The configured `admin_group` in the `gitlab.rb` is a CN, rather than a DN or an array. - This CN falls under the scope of the configured `group_base`. - The members of the `admin_group` have already signed into GitLab with their LDAP - credentials. GitLab only grants the Administrator role to the users whose + credentials. GitLab only grants administrator access to the users whose accounts are already connected to LDAP. If all the above are true and the users are still not getting access, diff --git a/doc/administration/auth/ldap/ldap_synchronization.md b/doc/administration/auth/ldap/ldap_synchronization.md index 3329ea8e9cc..d7ce54a47c0 100644 --- a/doc/administration/auth/ldap/ldap_synchronization.md +++ b/doc/administration/auth/ldap/ldap_synchronization.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Authentication & Authorization +group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -347,3 +347,7 @@ These metrics are meant to provide a baseline and performance may vary based on any number of factors. This benchmark was extreme and most instances don't have near this many users or groups. Disk speed, database performance, network and LDAP server response time affects these metrics. + +## Troubleshooting + +See our [administrator guide to troubleshooting LDAP](ldap-troubleshooting.md). diff --git a/doc/administration/auth/oidc.md b/doc/administration/auth/oidc.md index efe4b7440ee..a099b9c76f8 100644 --- a/doc/administration/auth/oidc.md +++ b/doc/administration/auth/oidc.md @@ -1,7 +1,7 @@ --- type: reference stage: Manage -group: Authentication & Authorization +group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -461,6 +461,65 @@ To use symmetric key encryption: If after reconfiguring, you see the error `JSON::JWS::VerificationFailed` error message, this means the incorrect secret was specified. +#### Casdoor + +GitLab works with OpenID providers that use HTTPS. To connect to GitLab using OpenID with Casdoor, use HTTPS instead of HTTP. + +For your app, complete the following steps on Casdoor: + +1. Get a client ID and a client secret. +1. Add your GitLab redirect URL. For example, if your GitLab domain is `gitlab.example.com`, ensure the Casdoor app has the following + `Redirect URI`: `https://gitlab.example.com/users/auth/openid_connect/callback`. + +See the [Casdoor documentation](https://casdoor.org/docs/integration/gitlab) for more details. + +Example Omnibus GitLab configuration (file path: `/etc/gitlab/gitlab.rb`): + +```ruby +gitlab_rails['omniauth_providers'] = [ + { + name: "openid_connect", + label: "Casdoor", # optional label for login button, defaults to "Openid Connect" + args: { + name: "openid_connect", + scope: ["openid", "profile", "email"], + response_type: "code", + issuer: "https://<CASDOOR_HOSTNAME>", + client_auth_method: "query", + discovery: true, + uid_field: "sub", + client_options: { + identifier: "<YOUR CLIENT ID>", + secret: "<YOUR CLIENT SECRET>", + redirect_uri: "https://gitlab.example.com/users/auth/openid_connect/callback" + } + } + } +] +``` + +Example installations from source configuration (file path: `config/gitlab.yml`): + +```yaml + - { name: 'openid_connect', + label: 'Casdoor', # optional label for login button, defaults to "Openid Connect" + args: { + name: 'openid_connect', + scope: ['openid','profile','email'], + response_type: 'code', + issuer: 'https://<CASDOOR_HOSTNAME>', + discovery: true, + client_auth_method: 'query', + uid_field: 'sub', + client_options: { + identifier: '<YOUR CLIENT ID>', + secret: '<YOUR CLIENT SECRET>', + redirect_uri: 'https://gitlab.example.com/users/auth/openid_connect/callback' + } + } + } +``` + ## General troubleshooting If you're having trouble, here are some tips: diff --git a/doc/administration/auth/smartcard.md b/doc/administration/auth/smartcard.md index f77d3a36a7b..3f0d95967bf 100644 --- a/doc/administration/auth/smartcard.md +++ b/doc/administration/auth/smartcard.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Authentication & Authorization +group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- diff --git a/doc/administration/clusters/kas.md b/doc/administration/clusters/kas.md index ba0d7280b74..192b636e246 100644 --- a/doc/administration/clusters/kas.md +++ b/doc/administration/clusters/kas.md @@ -4,13 +4,15 @@ group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Install the GitLab Agent Server (KAS) **(FREE SELF)** +# Install the GitLab Agent Server for Kubernetes (KAS) **(FREE SELF)** > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3834) in GitLab 13.10, the GitLab Agent Server (KAS) became available on GitLab.com under `wss://kas.gitlab.com`. -> [Moved](https://gitlab.com/groups/gitlab-org/-/epics/6290) from GitLab Premium to GitLab Free in 14.5. +> - [Moved](https://gitlab.com/groups/gitlab-org/-/epics/6290) from GitLab Premium to GitLab Free in 14.5. -The GitLab Agent Server (KAS) is a GitLab backend service dedicated to -managing the [GitLab Agent](../../user/clusters/agent/index.md). +The GitLab Agent Server for Kubernetes is a GitLab backend service dedicated to +managing the [GitLab Agent for Kubernetes](../../user/clusters/agent/index.md). + +The KAS acronym refers to the former name, Kubernetes Agent Server. The KAS is already installed and available in GitLab.com under `wss://kas.gitlab.com`. This document describes how to install a KAS for GitLab self-managed instances. @@ -94,8 +96,8 @@ For GitLab instances installed through Omnibus packages: ## Troubleshooting -If you face any issues with KAS, you can read the service logs -with the following command: +If you have issues while using the GitLab Agent Server for Kubernetes, view the +service logs by running the following command: ```shell kubectl logs -f -l=app=kas -n <YOUR-GITLAB-NAMESPACE> @@ -103,8 +105,7 @@ kubectl logs -f -l=app=kas -n <YOUR-GITLAB-NAMESPACE> In Omnibus GitLab, find the logs in `/var/log/gitlab/gitlab-kas/`. -See also the [user documentation](../../user/clusters/agent/index.md#troubleshooting) -for troubleshooting problems with individual agents. +You can also [troubleshoot issues with individual Agents](../../user/clusters/agent/troubleshooting.md). ### KAS logs - GitOps: failed to get project information diff --git a/doc/administration/compliance.md b/doc/administration/compliance.md index 843d5d0930a..7c3f36c71c2 100644 --- a/doc/administration/compliance.md +++ b/doc/administration/compliance.md @@ -16,7 +16,7 @@ relevant compliance standards. ## Policy management Organizations have unique policy requirements, either due to organizational -standards or mandates from regulatory bodies. The following features help you +standards or mandates from regulatory bodies. The following features help you define rules and policies to adhere to workflow requirements, separation of duties, and secure supply chain best practices: @@ -31,7 +31,7 @@ and secure supply chain best practices: - [**Merge request approvals**](../user/project/merge_requests/approvals/index.md) (for instances, groups, and projects): Configure approvals required for merge requests. -- [**Push rules**](../push_rules/push_rules.md) (for instances, groups, and +- [**Push rules**](../user/project/repository/push_rules.md) (for instances, groups, and projects): Control pushes to your repositories. - Separation of duties using [**protected branches**](../user/project/protected_branches.md#require-code-owner-approval-on-a-protected-branch) and [**custom CI/CD configuration paths**](../ci/pipelines/settings.md#specify-a-custom-cicd-configuration-file) (for projects): Users can leverage the GitLab cross-project YAML configurations diff --git a/doc/administration/docs_self_host.md b/doc/administration/docs_self_host.md index 007055b5de7..3a4511e5aa4 100644 --- a/doc/administration/docs_self_host.md +++ b/doc/administration/docs_self_host.md @@ -4,77 +4,69 @@ group: Distribution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# How to self-host the docs site **(FREE SELF)** - -If you have a self-managed instance of GitLab, you may not be able to access the -product documentation as hosted on `docs.gitlab.com` from your GitLab instance. - -Be aware of the following items if you self-host the product documentation: - -- You must host the product documentation site under a subdirectory that matches - your installed GitLab version (for example, `14.5/`). The - [Docker images](https://gitlab.com/gitlab-org/gitlab-docs/container_registry/631635) - hosted by the GitLab Docs team provide this by default. We use a - [script](https://gitlab.com/gitlab-org/gitlab-docs/-/blob/2995d1378175803b22fb8806ba77adf63e79f32c/scripts/normalize-links.sh#L28-82) - to normalize the links and prefix them with the respective version. -- The version dropdown will display additional versions that don't exist, selecting - those versions will display a 404 Not Found page. -- Results when using the search box will display results from `docs.gitlab.com` - and not the local documentation. -- When you use the Docker images to serve the product documentation site, by - default the landing page redirects to the respective version (for example, `/14.5/`), - which causes the landing page <https://docs.gitlab.com> to not be displayed. +# How to host the GitLab product documentation **(FREE SELF)** + +If you are not able to access the GitLab product documentation at `docs.gitlab.com`, +you can host the documentation yourself instead. ## Documentation self-hosting options -You can self-host the GitLab product documentation locally using one of these -methods: +To host the GitLab product documentation, you can use: -- Docker +- A Docker container - GitLab Pages -- From your own webserver +- Your own web server + +After you create a website by using one of these methods, you redirect the UI links +in the product to point to your website. -The examples on this page are based on GitLab 14.5. +NOTE: +The website you create must be hosted under a subdirectory that matches +your installed GitLab version (for example, `14.5/`). The +[Docker images](https://gitlab.com/gitlab-org/gitlab-docs/container_registry/631635) +use this version by default. + +The following examples use GitLab 14.5. ### Self-host the product documentation with Docker -The Docker images use a built-in webserver listening on port `4000`, so you need -to expose that. +You can run the GitLab product documentation website in a Docker container: -In the server that you host GitLab, or any other server that your GitLab instance -can talk to, you can use Docker to pull the docs site: +1. Expose port `4000`. The Docker image uses this port for the web server. +1. On the server where you host GitLab, or on any other server that your GitLab instance + can communicate with, pull the docs site: -```shell -docker run -it --rm -p 4000:4000 registry.gitlab.com/gitlab-org/gitlab-docs:14.5 -``` + ```shell + docker run -it --rm -p 4000:4000 registry.gitlab.com/gitlab-org/gitlab-docs:14.5 + ``` -If you use [Docker compose](../install/docker.md#install-gitlab-using-docker-compose) -to host your GitLab instance, add the following to `docker-compose.yaml`: - -```yaml -version: '3.6' -services: - docs: - image: registry.gitlab.com/gitlab-org/gitlab-docs:14.5 - hostname: 'https://gitlab.example.com' - ports: - - '4000:4000' -``` + If you host your GitLab instance using [Docker compose](../install/docker.md#install-gitlab-using-docker-compose), + add the following to `docker-compose.yaml`: + + ```yaml + version: '3.6' + services: + docs: + image: registry.gitlab.com/gitlab-org/gitlab-docs:14.5 + hostname: 'https://gitlab.example.com' + ports: + - '4000:4000' + ``` ### Self-host the product documentation with GitLab Pages -You use GitLab Pages to host the GitLab product documentation locally. +You can use GitLab Pages to host the GitLab product documentation. Prerequisite: -- The Pages site URL must not use a subfolder. Due to the nature of how the docs +- Ensure the Pages site URL does not use a subfolder. Because of how the docs site is pre-compiled, the CSS and JavaScript files are relative to the main domain or subdomain. For example, URLs like `https://example.com/docs/` are not supported. To host the product documentation site with GitLab Pages: -1. [Create a new blank project](../user/project/working_with_projects.md#create-a-blank-project). +1. [Create a blank project](../user/project/working_with_projects.md#create-a-blank-project). 1. Create a new or edit your existing `.gitlab-ci.yml` file, and add the following `pages` job, while ensuring the version is the same as your GitLab installation: @@ -97,13 +89,13 @@ To host the product documentation site with GitLab Pages: | [Project website](../user/project/pages/getting_started_part_one.md#project-website-examples) | Not supported | Supported | | [User or group website](../user/project/pages/getting_started_part_one.md#user-and-group-website-examples) | Supported | Supported | -### Self-host the product documentation on your own webserver +### Self-host the product documentation on your own web server -Because the product documentation site is static, you can grab the directory from -the container (in `/usr/share/nginx/html`) and use your own web server to host -it wherever you want. +Because the product documentation site is static, from the container, you can take the contents +of `/usr/share/nginx/html` and use your own web server to host +the docs wherever you want. -Use the following commands, and replace `<destination>` with the directory where the +Run the following commands, replacing `<destination>` with the directory where the documentation files will be copied to: ```shell @@ -114,18 +106,30 @@ docker rm -f gitlab-docs ## Redirect the `/help` links to the new docs page -After your local product documentation site is running, [redirect the help -links](../user/admin_area/settings/help_page.md#redirect-help-pages) in the GitLab -application to your local site. +After your local product documentation site is running, +[redirect the help links](../user/admin_area/settings/help_page.md#redirect-help-pages) +in the GitLab application to your local site. Be sure to use the fully qualified domain name as the docs URL. For example, if you used the [Docker method](#self-host-the-product-documentation-with-docker), enter `http://0.0.0.0:4000`. -You don't need to append the version, as GitLab will detect it and append it to -any documentation URL requests, as needed. For example, if your GitLab version is -14.5, the GitLab Docs URL becomes `http://0.0.0.0:4000/14.5/`. The link -inside GitLab displays as `<instance_url>/help/user/admin_area/settings/help_page#destination-requirements`, -but when you select it, you are redirected to +You don't need to append the version. GitLab detects it and appends it to +documentation URL requests as needed. For example, if your GitLab version is +14.5: + +- The GitLab Docs URL becomes `http://0.0.0.0:4000/14.5/`. +- The link in GitLab displays as `<instance_url>/help/user/admin_area/settings/help_page#destination-requirements`. +- When you select the link, you are redirected to `http://0.0.0.0:4000/14.5/ee/user/admin_area/settings/help_page/#destination-requirements`. To test the setting, select a **Learn more** link within the GitLab application. + +## Known issues + +If you self-host the product documentation: + +- The version dropdown displays additional versions that don't exist. Selecting + these versions displays a `404 Not Found` page. +- The search displays results from `docs.gitlab.com` and not the local site. +- By default, the landing page redirects to the + respective version (for example, `/14.5/`). This causes the landing page <https://docs.gitlab.com> to not be displayed. diff --git a/doc/administration/feature_flags.md b/doc/administration/feature_flags.md index afbf0759452..85a7304b5e8 100644 --- a/doc/administration/feature_flags.md +++ b/doc/administration/feature_flags.md @@ -36,7 +36,7 @@ For example, data is not recorded and services do not run. If you used a certain feature and identified a bug, a misbehavior, or an error, it's very important that you [**provide feedback**](https://gitlab.com/gitlab-org/gitlab/-/issues/new?issue[title]=Docs%20-%20feature%20flag%20feedback%3A%20Feature%20Name&issue[description]=Describe%20the%20problem%20you%27ve%20encountered.%0A%0A%3C!--%20Don%27t%20edit%20below%20this%20line%20--%3E%0A%0A%2Flabel%20~%22docs%5C-comments%22%20) to GitLab as soon as possible so we can improve or fix it while behind a flag. When you upgrade -GitLab to an earlier version, the feature flag status may change. +GitLab, the feature flag status may change. ## Risks when enabling features still in development @@ -130,11 +130,15 @@ irb(main):001:0> Feature.enable(:my_awesome_feature) => nil ``` -To check if a flag is enabled or disabled you can use `Feature.enabled?` or `Feature.disabled?`. For example, for a fictional feature flag named `my_awesome_feature`: +When the feature is ready, GitLab removes the feature flag, and the option for +enabling and disabling it no longer exists. The feature becomes available in all instances. + +### Check if a feature flag is enabled + +To check if a flag is enabled or disabled, use `Feature.enabled?` or `Feature.disabled?`. +For example, for a feature flag named `my_awesome_feature` that is already enabled: ```ruby -Feature.enable(:my_awesome_feature) -=> nil Feature.enabled?(:my_awesome_feature) => true Feature.disabled?(:my_awesome_feature) @@ -143,3 +147,21 @@ Feature.disabled?(:my_awesome_feature) When the feature is ready, GitLab removes the feature flag, and the option for enabling and disabling it no longer exists. The feature becomes available in all instances. + +### View set feature flags + +You can view all GitLab administrator set feature flags: + +```ruby +Feature.all +=> [#<Flipper::Feature:198220 name="my_awesome_feature", state=:on, enabled_gate_names=[:boolean], adapter=:memoizable>] +``` + +### Unset feature flag + +You can unset a feature flag so that GitLab will fall back to the current defaults for that flag: + +```ruby +Feature.remove(:my_awesome_feature) +=> true +``` diff --git a/doc/administration/geo/glossary.md b/doc/administration/geo/glossary.md index f8769d31ec7..a18e78a5e01 100644 --- a/doc/administration/geo/glossary.md +++ b/doc/administration/geo/glossary.md @@ -25,8 +25,8 @@ these definitions yet. | Site | One or a collection of nodes running a single GitLab application. A site can be single-node or multi-node. | GitLab | deployment, installation instance | | Single-node site | A specific configuration of GitLab that uses exactly one node. | GitLab | single-server, single-instance | Multi-node site | A specific configuration of GitLab that uses more than one node. | GitLab | multi-server, multi-instance, high availability | -| Primary site | A GitLab site that is configured to be read and writable. There can only be a single primary site. | Geo-specific | Geo deployment, Primary node | -| Secondary site(s) | GitLab site that is configured to be read-only. There can be one or more secondary sites. | Geo-specific | Geo deployment, Secondary node | +| Primary site | A GitLab site whose data is being replicated by at least one secondary site. There can only be a single primary site. | Geo-specific | Geo deployment, Primary node | +| Secondary site(s) | A GitLab site that is configured to replicate the data of a primary site. There can be one or more secondary sites. | Geo-specific | Geo deployment, Secondary node | | Geo deployment | A collection of two or more GitLab sites with exactly one primary site being replicated by one or more secondary sites. | Geo-specific | | | Reference architecture(s) | A [specified configuration of GitLab for a number of users](../reference_architectures/index.md), possibly including multiple nodes and multiple sites. | GitLab | | | Promoting | Changing the role of a site from secondary to primary. | Geo-specific | | diff --git a/doc/administration/geo/index.md b/doc/administration/geo/index.md index b5eb0ba5841..c4164284e97 100644 --- a/doc/administration/geo/index.md +++ b/doc/administration/geo/index.md @@ -204,6 +204,7 @@ This list of limitations only reflects the latest version of GitLab. If you are - Real-time updates of issues/merge requests (for example, via long polling) doesn't work on the **secondary** site. - GitLab Runners cannot register with a **secondary** site. Support for this is [planned for the future](https://gitlab.com/gitlab-org/gitlab/-/issues/3294). - [Selective synchronization](replication/configuration.md#selective-synchronization) only limits what repositories and files are replicated. The entire PostgreSQL data is still replicated. Selective synchronization is not built to accommodate compliance / export control use cases. +- [Pages access control](../../user/project/pages/pages_access_control.md) doesn't work on secondaries. See [GitLab issue #9336](https://gitlab.com/gitlab-org/gitlab/-/issues/9336) for details. ### Limitations on replication/verification diff --git a/doc/administration/geo/replication/datatypes.md b/doc/administration/geo/replication/datatypes.md index 0fa08dcc9e0..5beb2479c57 100644 --- a/doc/administration/geo/replication/datatypes.md +++ b/doc/administration/geo/replication/datatypes.md @@ -39,7 +39,7 @@ verification methods: | Blobs | User uploads _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ | | Blobs | LFS objects _(file system)_ | Geo with API | SHA256 checksum | | Blobs | LFS objects _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ | -| Blobs | CI job artifacts _(file system)_ | Geo with API | _Not implemented_ | +| Blobs | CI job artifacts _(file system)_ | Geo with API | SHA256 checksum | | Blobs | CI job artifacts _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ | | Blobs | Archived CI build traces _(file system)_ | Geo with API | _Not implemented_ | | Blobs | Archived CI build traces _(object storage)_ | Geo with API/Managed (*2*) | _Not implemented_ | diff --git a/doc/administration/geo/replication/geo_validation_tests.md b/doc/administration/geo/replication/geo_validation_tests.md index ce1bd8a9d3c..70b3eaf5fea 100644 --- a/doc/administration/geo/replication/geo_validation_tests.md +++ b/doc/administration/geo/replication/geo_validation_tests.md @@ -118,7 +118,7 @@ The following are PostgreSQL upgrade validation tests we performed. [Verify Geo installation with PostgreSQL 13](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6131): - Description: With PostgreSQL 13 available as an opt-in version in GitLab 14.1, we tested fresh installations of GitLab with Geo when PostgreSQL 13 is enabled. -- Outcome: Successfully built an environment with Geo and PostgreSQL 13 using [GitLab Environment Toolkit](https://gitlab.com/gitlab-org/quality/gitlab-environment-toolkit) and performed Geo QA tests against the environment without failures. +- Outcome: Successfully built an environment with Geo and PostgreSQL 13 using [GitLab Environment Toolkit](https://gitlab.com/gitlab-org/gitlab-environment-toolkit) and performed Geo QA tests against the environment without failures. ### September 2020 @@ -200,3 +200,12 @@ The following are additional validation tests we performed. - Description: Tested a Geo deployment with Gitaly clusters configured on both the primary and secondary Geo sites. Triggered automatic Gitaly cluster failover on the primary Geo site, and ran end-to-end Geo tests. Then triggered Gitaly cluster failover on the secondary Geo site, and re-ran the end-to-end Geo tests. - Outcome: Successful end-to-end tests before and after Gitaly cluster failover on the primary site, and before and after Gitaly cluster failover on the secondary site. + +### January 2022 + +[Validate Object storage replication using Azure based object storage](https://gitlab.com/gitlab-org/gitlab/-/issues/348804#note_821294631): + +- Description: Tested the average time it takes for a single image to replicate from the primary object storage location to the secondary when using Azure based object storage replication and [GitLab based object storage replication](object_storage.md#enabling-gitlab-managed-object-storage-replication). This was tested by uploading a 1mb image to a project on the primary site every second for 60 seconds. The time was then measured until a image was available on the secondary site. This was achieved using a [Ruby Script](https://gitlab.com/gitlab-org/quality/geo-replication-tester). +- Outcome: When using Azure based replication the average time for an image to replicate from the primary object storage to the secondary was recorded as 40 seconds, the longest replication time was 70 seconds and the quickest was 11 seconds. When using GitLab based replication the average time for replication to complete was 5 seconds, the longest replication time was 10 seconds and the quickest was 3 seconds. +- Follow up issue: + - [Test and validate object storage replication performance on reference architectures](https://gitlab.com/gitlab-org/gitlab/-/issues/347314) diff --git a/doc/administration/geo/replication/location_aware_git_url.md b/doc/administration/geo/replication/location_aware_git_url.md index ce1af27611e..8689fac987f 100644 --- a/doc/administration/geo/replication/location_aware_git_url.md +++ b/doc/administration/geo/replication/location_aware_git_url.md @@ -7,6 +7,12 @@ type: howto # Location-aware Git remote URL with AWS Route53 **(PREMIUM SELF)** +NOTE: +Since GitLab 14.6, +[GitLab Geo supports a location-aware URL including web UI and API traffic.](../secondary_proxy/location_aware_external_url.md) +This configuration is recommended over the location-aware Git remote URL +described in this document. + You can provide GitLab users with a single remote URL that automatically uses the Geo site closest to them. This means users don't need to update their Git configuration to take advantage of closer Geo sites as they move. @@ -18,12 +24,6 @@ Though these instructions use [AWS Route53](https://aws.amazon.com/route53/), other services such as [Cloudflare](https://www.cloudflare.com/) could be used as well. -NOTE: -You can also use a load balancer to distribute web UI or API traffic to -[multiple Geo **secondary** sites](../../../user/admin_area/geo_nodes.md#multiple-secondary-sites-behind-a-load-balancer). -Importantly, the **primary** site cannot yet be included. See the feature request -[Support putting the **primary** behind a Geo node load balancer](https://gitlab.com/gitlab-org/gitlab/-/issues/10888). - ## Prerequisites In this example, we have already set up: diff --git a/doc/administration/geo/replication/troubleshooting.md b/doc/administration/geo/replication/troubleshooting.md index a6ea41171a9..1371e5d84c8 100644 --- a/doc/administration/geo/replication/troubleshooting.md +++ b/doc/administration/geo/replication/troubleshooting.md @@ -115,35 +115,39 @@ http://secondary.example.com/ To check if PostgreSQL replication is working, check if: -- [Nodes are pointing to the correct database instance](#are-nodes-pointing-to-the-correct-database-instance). -- [Geo can detect the current node correctly](#can-geo-detect-the-current-node-correctly). +- [Sites are pointing to the correct database node](#are-sites-pointing-to-the-correct-database-node). +- [Geo can detect the current site correctly](#can-geo-detect-the-current-site-correctly). -#### Are nodes pointing to the correct database instance? +#### Are sites pointing to the correct database node? -You should make sure your **primary** Geo node points to the instance with -writing permissions. +You should make sure your **primary** Geo [site](../glossary.md) points to +the database node that has write permissions. -Any **secondary** nodes should point only to read-only instances. +Any **secondary** sites should point only to read-only database nodes. -#### Can Geo detect the current node correctly? +#### Can Geo detect the current site correctly? -Geo finds the current machine's Geo node name in `/etc/gitlab/gitlab.rb` by: +Geo finds the current Puma or Sidekiq node's Geo [site](../glossary.md) name in +`/etc/gitlab/gitlab.rb` with the following logic: -- Using the `gitlab_rails['geo_node_name']` setting. -- If that is not defined, using the `external_url` setting. +1. Get the "Geo node name" (there is + [an issue to rename the settings to "Geo site name"](https://gitlab.com/gitlab-org/gitlab/-/issues/335944)): + - Omnibus GitLab: Get the `gitlab_rails['geo_node_name']` setting. + - GitLab Helm Charts: Get the `global.geo.nodeName` setting (see [Charts with GitLab Geo](https://docs.gitlab.com/charts/advanced/geo/index.html)). +1. If that is not defined, then get the `external_url` setting. -This name is used to look up the node with the same **Name** in the **Geo Nodes** +This name is used to look up the Geo site with the same **Name** in the **Geo Sites** dashboard. -To check if the current machine has a node name that matches a node in the +To check if the current machine has a site name that matches a site in the database, run the check task: ```shell sudo gitlab-rake gitlab:geo:check ``` -It displays the current machine's node name and whether the matching database -record is a **primary** or **secondary** node. +It displays the current machine's site name and whether the matching database +record is a **primary** or **secondary** site. ```plaintext This machine's Geo node name matches a database record ... yes, found a secondary node named "Shanghai" @@ -158,6 +162,9 @@ This machine's Geo node name matches a database record ... no doc/administration/geo/replication/troubleshooting.md#can-geo-detect-the-current-node-correctly ``` +Learn more about recommended site names in the description of the Name field in +[Geo Admin Area Common Settings](../../../user/admin_area/geo_nodes.md#common-settings). + ### Message: `WARNING: oldest xmin is far in the past` and `pg_wal` size growing If a replication slot is inactive, @@ -692,6 +699,8 @@ determine the actual replication status of Design repositories. ### Sync failure message: "Verification failed with: Error during verification: File is not checksummable" +#### Missing files on the Geo primary site + In GitLab 14.5 and earlier, certain data types which were missing on the Geo primary site were marked as "synced" on Geo secondary sites. This was because from the perspective of Geo secondary sites, the state matched the primary site and nothing more could be done on secondary sites. Secondaries would regularly try to sync these files again by using the "verification" feature: @@ -745,6 +754,32 @@ This behavior affects only the following data types through GitLab 14.6: to make Geo visibly surface data loss risks. The sync/verification loop is therefore short-circuited. `last_sync_failure` is now set to `The file is missing on the Geo primary site`. +#### Failed syncs with GitLab-managed object storage replication + +There is [an issue in GitLab 14.2 through 14.7](https://gitlab.com/gitlab-org/gitlab/-/issues/299819#note_822629467) +that affects Geo when the GitLab-managed object storage replication is used, causing blob object types to fail synchronization. + +Since GitLab 14.2, verification failures result in synchronization failures and cause +a re-synchronization of these objects. + +As verification is not implemented for files stored in object storage (see +[issue 13845](https://gitlab.com/gitlab-org/gitlab/-/issues/13845) for more details), this +results in a loop that consistently fails for all objects stored in object storage. + +You can work around this by marking the objects as synced and succeeded verification, however +be aware that can also mark objects that may be +[missing from the primary](#missing-files-on-the-geo-primary-site). + +To do that, enter the [Rails console](../../troubleshooting/navigating_gitlab_via_rails_console.md) +and run: + +```ruby +Gitlab::Geo.verification_enabled_replicator_classes.each do |klass| + updated = klass.registry_class.failed.where(last_sync_failure: "Verification failed with: Error during verification: File is not checksummable").update_all(verification_checksum: '0000000000000000000000000000000000000000', verification_state: 2, verification_failure: nil, verification_retry_at: nil, state: 2, last_sync_failure: nil, retry_at: nil, verification_retry_count: 0, retry_count: 0) + pp "Updated #{updated} #{klass.replicable_name_plural}" +end +``` + ## Fixing errors during a failover or when promoting a secondary to a primary node The following are possible error messages that might be encountered during failover or diff --git a/doc/administration/geo/replication/using_a_geo_server.md b/doc/administration/geo/replication/using_a_geo_server.md index 04c30514a89..62562a1149d 100644 --- a/doc/administration/geo/replication/using_a_geo_server.md +++ b/doc/administration/geo/replication/using_a_geo_server.md @@ -1,9 +1,9 @@ --- redirect_to: '../../geo/replication/usage.md' -remove_date: '2022-06-01' +remove_date: '2022-02-01' --- This document was moved to [another location](../../geo/replication/usage.md). -<!-- This redirect file can be deleted after 2022-06-01 --> +<!-- This redirect file can be deleted after 2022-02-01 --> <!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/geo/replication/version_specific_updates.md b/doc/administration/geo/replication/version_specific_updates.md index d3a132a6666..95833a290dd 100644 --- a/doc/administration/geo/replication/version_specific_updates.md +++ b/doc/administration/geo/replication/version_specific_updates.md @@ -8,7 +8,22 @@ info: To determine the technical writer assigned to the Stage/Group associated w Review this page for update instructions for your version. These steps accompany the [general steps](updating_the_geo_sites.md#general-update-steps) -for updating Geo nodes. +for updating Geo sites. + +## Updating to 14.2 through 14.7 + +There is [an issue in GitLab 14.2 through 14.7](https://gitlab.com/gitlab-org/gitlab/-/issues/299819#note_822629467) +that affects Geo when the GitLab-managed object storage replication is used, causing blob object types to fail synchronization. + +Since GitLab 14.2, verification failures result in synchronization failures and cause +a resynchronization of these objects. + +As verification is not yet implemented for files stored in object storage (see +[issue 13845](https://gitlab.com/gitlab-org/gitlab/-/issues/13845) for more details), this +results in a loop that consistently fails for all objects stored in object storage. + +For information on how to fix this, see +[Troubleshooting - Failed syncs with GitLab-managed object storage replication](troubleshooting.md#failed-syncs-with-gitlab-managed-object-storage-replication). ## Updating to 14.4 @@ -18,7 +33,7 @@ There is [an issue in GitLab 14.4.0 through 14.4.2](../../../update/index.md#144 ### Multi-arch images -We found an [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/336013) where the Container Registry replication wasn't fully working if you used multi-arch images. In case of a multi-arch image, only the primary architecture (for example `amd64`) would be replicated to the secondary node. This has been [fixed in GitLab 14.3](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/67624) and was backported to 14.2 and 14.1, but manual steps are required to force a re-sync. +We found an [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/336013) where the Container Registry replication wasn't fully working if you used multi-arch images. In case of a multi-arch image, only the primary architecture (for example `amd64`) would be replicated to the secondary site. This has been [fixed in GitLab 14.3](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/67624) and was backported to 14.2 and 14.1, but manual steps are required to force a re-sync. You can check if you are affected by running: @@ -26,12 +41,12 @@ You can check if you are affected by running: docker manifest inspect <SECONDARY_IMAGE_LOCATION> | jq '.mediaType' ``` -Where `<SECONDARY_IMAGE_LOCATION>` is a container image on your secondary node. +Where `<SECONDARY_IMAGE_LOCATION>` is a container image on your secondary site. If the output matches `application/vnd.docker.distribution.manifest.list.v2+json` (there can be a `mediaType` entry at several levels, we only care about the top level entry), then you don't need to do anything. -Otherwise, on all your **secondary** nodes, in a [Rails console](../../operations/rails_console.md), run the following: +Otherwise, for each **secondary** site, on a Rails application node, open a [Rails console](../../operations/rails_console.md), and run the following: ```ruby list_type = 'application/vnd.docker.distribution.manifest.list.v2+json' @@ -63,7 +78,7 @@ We found an issue where [Primary sites can not be removed from the UI](https://g This bug only exists in the UI and does not block the removal of Primary sites using any other method. -If you are running an affected version and need to remove your Primary site, you can manually remove the Primary site by using the [Geo Nodes API](../../../api/geo_nodes.md#delete-a-geo-node). +If you are running an affected version and need to remove your Primary site, you can manually remove the Primary site by using the [Geo Sites API](../../../api/geo_nodes.md#delete-a-geo-node). ### Geo Admin Area shows 'Unhealthy' after enabling Maintenance Mode @@ -71,9 +86,9 @@ GitLab 13.9 through GitLab 14.3 are affected by a bug in which enabling [GitLab ## Updating to GitLab 13.12 -### Secondary nodes re-download all LFS files upon update +### Secondary sites re-download all LFS files upon update -We found an issue where [secondary nodes re-download all LFS files](https://gitlab.com/gitlab-org/gitlab/-/issues/334550) upon update. This bug: +We found an issue where [secondary sites re-download all LFS files](https://gitlab.com/gitlab-org/gitlab/-/issues/334550) upon update. This bug: - Only applies to Geo secondary sites that have replicated LFS objects. - Is _not_ a data loss risk. @@ -172,7 +187,7 @@ In GitLab 13.3, Geo removed the PostgreSQL [Foreign Data Wrapper](https://www.po dependency for the tracking database. The FDW server, user, and the extension is removed during the upgrade -process on each secondary node. The GitLab settings related to the FDW in the +process on each secondary site. The GitLab settings related to the FDW in the `/etc/gitlab/gitlab.rb` have been deprecated and can be safely removed. There are some scenarios like using an external PostgreSQL instance for the @@ -185,9 +200,9 @@ DROP EXTENSION IF EXISTS postgres_fdw; ``` WARNING: -In GitLab 13.3, promoting a secondary node to a primary while the secondary is +In GitLab 13.3, promoting a secondary site to a primary while the secondary is paused fails. Do not pause replication before promoting a secondary. If the -node is paused, be sure to resume before promoting. To avoid this issue, +site is paused, be sure to resume before promoting. To avoid this issue, upgrade to GitLab 13.4 or later. WARNING: @@ -198,9 +213,9 @@ contain a workaround if you run into errors during the failover. ## Updating to GitLab 13.2 -In GitLab 13.2, promoting a secondary node to a primary while the secondary is +In GitLab 13.2, promoting a secondary site to a primary while the secondary is paused fails. Do not pause replication before promoting a secondary. If the -node is paused, be sure to resume before promoting. To avoid this issue, +site is paused, be sure to resume before promoting. To avoid this issue, upgrade to GitLab 13.4 or later. ## Updating to GitLab 13.0 @@ -375,5 +390,5 @@ For the recommended procedure, see the WARNING: This version is affected by a [bug that results in new LFS objects not being -replicated to Geo secondary nodes](https://gitlab.com/gitlab-org/gitlab/-/issues/32696). +replicated to Geo secondary sites](https://gitlab.com/gitlab-org/gitlab/-/issues/32696). The issue is fixed in GitLab 12.1. Be sure to upgrade to GitLab 12.1 or later. diff --git a/doc/administration/geo/secondary_proxy/index.md b/doc/administration/geo/secondary_proxy/index.md index ebd71757e91..c8dcbb81312 100644 --- a/doc/administration/geo/secondary_proxy/index.md +++ b/doc/administration/geo/secondary_proxy/index.md @@ -140,6 +140,18 @@ for details. To use TLS certificates with Let's Encrypt, you can manually point the domain to one of the Geo sites, generate the certificate, then copy it to all other sites. +## Behavior of secondary sites when the primary Geo site is down + +Considering that web traffic is proxied to the primary, the behavior of the secondary sites differs when the primary +site is inaccessible: + +- UI and API traffic return the same errors as the primary (or fail if the primary is not accessible at all), since they are proxied. +- For repositories that already exist on the specific secondary site being accessed, Git read operations still work as expected, + including authentication through HTTP(s) or SSH. +- Git operations for repositories that are not replicated to the secondary site return the same errors + as the primary site, since they are proxied. +- All Git write operations return the same errors as the primary site, since they are proxied. + ## Features accelerated by secondary Geo sites Most HTTP traffic sent to a secondary Geo site can be proxied to the primary Geo site. With this architecture, diff --git a/doc/administration/geo/setup/database.md b/doc/administration/geo/setup/database.md index 7f4c771c962..a8bebef8f44 100644 --- a/doc/administration/geo/setup/database.md +++ b/doc/administration/geo/setup/database.md @@ -40,7 +40,7 @@ instructions on setting up replication with a Patroni cluster. The GitLab **primary** node where the write operations happen connects to the **primary** database server, and **secondary** nodes -connect to their own database servers (which are also read-only). +connect to their own database servers (which are read-only). We recommend using [PostgreSQL replication slots](https://medium.com/@tk512/replication-slots-in-postgresql-b4b03d277c75) to ensure that the **primary** node retains all the data necessary for the **secondary** nodes to @@ -893,6 +893,9 @@ Instead, follow the instructions below. A production-ready and secure setup requires at least three Consul nodes, two Patroni nodes and one PgBouncer node on the secondary site. +Because of [omnibus-6587](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6587), Consul can't track multiple +services, so these need to be different than the nodes used for the Standby Cluster database. + Be sure to use [password credentials](../../postgresql/replication_and_failover.md#database-authorization-for-patroni) and other database best practices. diff --git a/doc/administration/get_started.md b/doc/administration/get_started.md index 3be3c11947e..f3b4400c8c3 100644 --- a/doc/administration/get_started.md +++ b/doc/administration/get_started.md @@ -62,7 +62,7 @@ You may need to import projects from external sources like GitHub, Bitbucket, or ### Popular project imports -- [GitHub Enterprise to self-managed GitLab](../integration/github.md#enabling-github-oauth): Enabling OAuth makes it easier for developers to find and import their projects. +- [GitHub Enterprise to self-managed GitLab](../integration/github.md): Enabling OAuth makes it easier for developers to find and import their projects. - [Bitbucket Server](../user/project/import/bitbucket_server.md#limitations): There are certain data limitations. For assistance with these data types, contact your GitLab account manager or GitLab Support about our professional migration services. diff --git a/doc/administration/gitaly/configure_gitaly.md b/doc/administration/gitaly/configure_gitaly.md index a0c959d5de9..88efd1885db 100644 --- a/doc/administration/gitaly/configure_gitaly.md +++ b/doc/administration/gitaly/configure_gitaly.md @@ -28,6 +28,12 @@ The following configuration options are also available: - Configuring the [number of `gitaly-ruby` workers](#configure-number-of-gitaly-ruby-workers). - Limiting [RPC concurrency](#limit-rpc-concurrency). +## About the Gitaly token + +The token referred to throughout the Gitaly documentation is just an arbitrary password selected by +the administrator. It is unrelated to tokens created for the GitLab API or other similar web API +tokens. + ## Run Gitaly on its own server By default, Gitaly is run on the same server as Gitaly clients and is @@ -116,11 +122,6 @@ We assume your GitLab installation has three repository storages: You can use as few as one server with one repository storage if desired. -NOTE: -The token referred to throughout the Gitaly documentation is just an arbitrary password selected by -the administrator. It is unrelated to tokens created for the GitLab API or other similar web API -tokens. - ### Install Gitaly Install Gitaly on each Gitaly server using either Omnibus GitLab or install it from source: @@ -476,7 +477,7 @@ example: ```ruby git_data_dirs({ 'default' => { 'gitaly_address' => 'tcp://gitaly1.internal:8075' }, - # Address of the GitLab server that has Gitaly running on it + # Address of the GitLab server that also has Gitaly running on it 'storage1' => { 'gitaly_address' => 'tcp://gitlab.internal:8075', 'path' => '/mnt/gitlab/git-data' }, 'storage2' => { 'gitaly_address' => 'tcp://gitaly2.internal:8075' }, }) @@ -565,12 +566,6 @@ Note the following: - You can configure Gitaly servers with both an unencrypted listening address `listen_addr` and an encrypted listening address `tls_listen_addr` at the same time. This allows you to gradually transition from unencrypted to encrypted traffic if necessary. -- When running Praefect sub-commands such as `dial-nodes` and `list-untracked-repositories` from the command line with Gitaly TLS enabled, you must set - the `SSL_CERT_DIR` or `SSL_CERT_FILE` environment variable so that the Gitaly certificate is trusted. For example: - - ```shell - sudo SSL_CERT_DIR=/etc/gitlab/trusted_certs /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml dial-nodes - ``` To configure Gitaly with TLS: diff --git a/doc/administration/gitaly/faq.md b/doc/administration/gitaly/faq.md index f79b9555c10..b0a88e8adc9 100644 --- a/doc/administration/gitaly/faq.md +++ b/doc/administration/gitaly/faq.md @@ -23,10 +23,10 @@ Gitaly Cluster and [Geo](../geo/index.md) both provide redundancy. However the r The following table outlines the major differences between Gitaly Cluster and Geo: -| Tool | Nodes | Locations | Latency tolerance | Failover | Consistency | Provides redundancy for | -|:---------------|:---------|:----------|:-------------------|:----------------------------------------------------------------------------|:-----------------------------------------|:------------------------| -| Gitaly Cluster | Multiple | Single | Approximately 1 ms | [Automatic](praefect.md#automatic-failover-and-primary-election-strategies) | [Strong](index.md#strong-consistency) | Data storage in Git | -| Geo | Multiple | Multiple | Up to one minute | [Manual](../geo/disaster_recovery/index.md) | Eventual | Entire GitLab instance | +| Tool | Nodes | Locations | Latency tolerance | Failover | Consistency | Provides redundancy for | +|:---------------|:---------|:----------|:------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------|:--------------------------------------|:------------------------| +| Gitaly Cluster | Multiple | Single | [Less than 1 second, ideally single-digit milliseconds](praefect.md#network-latency-and-connectivity) | [Automatic](praefect.md#automatic-failover-and-primary-election-strategies) | [Strong](index.md#strong-consistency) | Data storage in Git | +| Geo | Multiple | Multiple | Up to one minute | [Manual](../geo/disaster_recovery/index.md) | Eventual | Entire GitLab instance | For more information, see: diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md index 4d60832720b..ea24e901943 100644 --- a/doc/administration/gitaly/index.md +++ b/doc/administration/gitaly/index.md @@ -290,7 +290,7 @@ including [horizontally distributing reads](https://gitlab.com/groups/gitlab-org #### Distributed reads -> - Introduced in GitLab 13.1 in [beta](https://about.gitlab.com/handbook/product/gitlab-the-product/#alpha-beta-ga) with feature flag `gitaly_distributed_reads` set to disabled. +> - Introduced in GitLab 13.1 in [beta](../../policy/alpha-beta-support.md#beta-features) with feature flag `gitaly_distributed_reads` set to disabled. > - [Made generally available and enabled by default](https://gitlab.com/gitlab-org/gitaly/-/issues/2951) in GitLab 13.3. > - [Disabled by default](https://gitlab.com/gitlab-org/gitaly/-/issues/3178) in GitLab 13.5. > - [Enabled by default](https://gitlab.com/gitlab-org/gitaly/-/issues/3334) in GitLab 13.8. @@ -316,8 +316,8 @@ You can [monitor distribution of reads](#monitor-gitaly-cluster) using Prometheu #### Strong consistency -> - Introduced in GitLab 13.1 in [alpha](https://about.gitlab.com/handbook/product/gitlab-the-product/#alpha-beta-ga), disabled by default. -> - Entered [beta](https://about.gitlab.com/handbook/product/gitlab-the-product/#alpha-beta-ga) in GitLab 13.2, disabled by default. +> - Introduced in GitLab 13.1 in [alpha](../../policy/alpha-beta-support.md#alpha-features), disabled by default. +> - Entered [beta](../../policy/alpha-beta-support.md#beta-features) in GitLab 13.2, disabled by default. > - In GitLab 13.3, disabled unless primary-wins voting strategy is disabled. > - From GitLab 13.4, enabled by default. > - From GitLab 13.5, you must use Git v2.28.0 or higher on Gitaly nodes to enable strong consistency. @@ -529,6 +529,10 @@ To monitor [strong consistency](#strong-consistency), you can use the following - `gitaly_hook_transaction_voting_delay_seconds`, the client-side delay introduced by waiting for the transaction to be committed. +To monitor the number of repositories that have no healthy, up-to-date replicas: + +- `gitaly_praefect_unavailable_repositories` + You can also monitor the [Praefect logs](../logs.md#praefect-logs). #### Database metrics `/db_metrics` endpoint diff --git a/doc/administration/gitaly/praefect.md b/doc/administration/gitaly/praefect.md index e2db30b8358..8f17835b8a3 100644 --- a/doc/administration/gitaly/praefect.md +++ b/doc/administration/gitaly/praefect.md @@ -39,10 +39,26 @@ NOTE: If not set in GitLab, feature flags are read as false from the console and Praefect uses their default value. The default value depends on the GitLab version. -### Network connectivity +### Network latency and connectivity -Gitaly Cluster [components](index.md#components) need to communicate with each other over many -routes. Your firewall rules must allow the following for Gitaly Cluster to function properly: +Network latency for Gitaly Cluster should ideally be measurable in single-digit milliseconds. Latency is particularly +important for: + +- Gitaly node health checks. Nodes must be able to respond within 1 second. +- Reference transactions that enforce [strong consistency](index.md#strong-consistency). Lower latencies mean Gitaly + nodes can agree on changes faster. + +Achieving acceptable latency between Gitaly nodes: + +- On physical networks generally means high bandwidth, single location connections. +- On the cloud generally means within the same region, including allowing cross availability zone replication. These links + are designed for this type of synchronization. Latency of less than 2ms should be sufficient for Gitaly Cluster. + +If you can't provide low network latencies for replication (for example, between distant locations), consider Geo. For +more information, see [How does Gitaly Cluster compare to Geo](faq.md#how-does-gitaly-cluster-compare-to-geo). + +Gitaly Cluster [components](index.md#components) communicate with each other over many routes. Your firewall rules must +allow the following for Gitaly Cluster to function properly: | From | To | Default port | TLS port | |:-----------------------|:-----------------------|:-------------|:---------| @@ -254,11 +270,11 @@ reads distribution caching is enabled by configuration To reduce PostgreSQL resource consumption, we recommend setting up and configuring [PgBouncer](https://www.pgbouncer.org/) in front of the PostgreSQL instance. To do -this, you must point Praefect to PgBouncer by setting Praefect database parameters: +this, you must point Praefect to PgBouncer by setting database parameters on Praefect configuration: ```ruby praefect['database_host'] = PGBOUNCER_HOST -praefect['database_port'] = 5432 +praefect['database_port'] = 6432 praefect['database_user'] = 'praefect' praefect['database_password'] = PRAEFECT_SQL_PASSWORD praefect['database_dbname'] = 'praefect_production' @@ -273,40 +289,21 @@ Praefect requires an additional connection to the PostgreSQL that supports the this feature is only available with `session` pool mode (`pool_mode = session`). It is not supported in `transaction` pool mode (`pool_mode = transaction`). -For the additional connection, you must either: +To configure the additional connection, you must either: -- Connect Praefect directly to PostgreSQL and bypass PgBouncer. - Configure a new PgBouncer database that uses to the same PostgreSQL database endpoint, but with different pool mode. That is, `pool_mode = session`. +- Connect Praefect directly to PostgreSQL and bypass PgBouncer. -Praefect can be configured to use different connection parameters for direct access -to PostgreSQL. This is the connection that supports the `LISTEN` feature. - -Here is an example of Praefect that bypasses PgBouncer and directly connects to PostgreSQL: - -```ruby -praefect['database_direct_host'] = POSTGRESQL_HOST -praefect['database_direct_port'] = 5432 - -# Use the following to override parameters of direct database connection. -# Comment out where the parameters are the same for both connections. - -praefect['database_direct_user'] = 'praefect' -praefect['database_direct_password'] = PRAEFECT_SQL_PASSWORD -praefect['database_direct_dbname'] = 'praefect_production' -#praefect['database_direct_sslmode'] = '...' -#praefect['database_direct_sslcert'] = '...' -#praefect['database_direct_sslkey'] = '...' -#praefect['database_direct_sslrootcert'] = '...' -``` +#### Configure a new PgBouncer database with `pool_mode = session` -We recommend using PgBouncer with `session` pool mode instead. You can use the +We recommend using PgBouncer with `session` pool mode. You can use the [bundled PgBouncer](../postgresql/pgbouncer.md) or use an external PgBouncer and [configure it manually](https://www.pgbouncer.org/config.html). -The following example uses the bundled PgBouncer and sets up two separate connection pools, +The following example uses the bundled PgBouncer and sets up two separate connection pools on PostgreSQL host, one in `session` pool mode and the other in `transaction` pool mode. For this example to work, -you need to prepare PostgreSQL server with [setup instruction](#manual-database-setup): +you need to prepare PostgreSQL server as documented in [in the setup instructions](#manual-database-setup): ```ruby pgbouncer['databases'] = { @@ -373,6 +370,29 @@ your databases manually and configuring an external PgBouncer, you must include its password in the file used by PgBouncer. For example, `userlist.txt` if the [`auth_file`](https://www.pgbouncer.org/config.html#auth_file) configuration option is set. For more details, consult the PgBouncer documentation. +#### Configure Praefect to connect directly to PostgreSQL + +As an alternative to configuring PgBouncer with `session` pool mode, Praefect can be configured to use different connection parameters for direct access +to PostgreSQL. This is the connection that supports the `LISTEN` feature. + +An example of Praefect configuration that bypasses PgBouncer and directly connects to PostgreSQL: + +```ruby +praefect['database_direct_host'] = POSTGRESQL_HOST +praefect['database_direct_port'] = 5432 + +# Use the following to override parameters of direct database connection. +# Comment out where the parameters are the same for both connections. + +praefect['database_direct_user'] = 'praefect' +praefect['database_direct_password'] = PRAEFECT_SQL_PASSWORD +praefect['database_direct_dbname'] = 'praefect_production' +#praefect['database_direct_sslmode'] = '...' +#praefect['database_direct_sslcert'] = '...' +#praefect['database_direct_sslkey'] = '...' +#praefect['database_direct_sslrootcert'] = '...' +``` + ### Praefect > [Introduced](https://gitlab.com/gitlab-org/gitaly/-/issues/2634) in GitLab 13.4, Praefect nodes can no longer be designated as `primary`. @@ -570,15 +590,15 @@ On the **Praefect** node: edit `/etc/gitlab/gitlab.rb`, remember to run `sudo gitlab-ctl reconfigure` again before trying the `sql-ping` command. -#### Enabling TLS support +#### Enable TLS support > [Introduced](https://gitlab.com/gitlab-org/gitaly/-/issues/1698) in GitLab 13.2. Praefect supports TLS encryption. To communicate with a Praefect instance that listens for secure connections, you must: -- Use a `tls://` URL scheme in the `gitaly_address` of the corresponding storage entry - in the GitLab configuration. +- Ensure Gitaly is [configured for TLS](configure_gitaly.md#enable-tls-support) and use a `tls://` URL scheme in the `gitaly_address` + of the corresponding storage entry in the GitLab configuration. - Bring your own certificates because this isn't provided automatically. The certificate corresponding to each Praefect server must be installed on that Praefect server. @@ -594,6 +614,13 @@ Note the following: - Hostname, you can either use the Common Name field for this, or add it as a Subject Alternative Name. - IP address, you must add it as a Subject Alternative Name to the certificate. +- When running Praefect sub-commands such as `dial-nodes` and `list-untracked-repositories` from the command line with + [Gitaly TLS enabled](configure_gitaly.md#enable-tls-support), you must set the `SSL_CERT_DIR` or `SSL_CERT_FILE` + environment variable so that the Gitaly certificate is trusted. For example: + + ```shell + sudo SSL_CERT_DIR=/etc/gitlab/trusted_certs /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml dial-nodes + ``` - You can configure Praefect servers with both an unencrypted listening address `listen_addr` and an encrypted listening address `tls_listen_addr` at the same time. @@ -639,7 +666,7 @@ To configure Praefect with TLS: ```ruby git_data_dirs({ "default" => { - "gitaly_address" => 'tls://PRAEFECT_LOADBALANCER_HOST:2305', + "gitaly_address" => 'tls://PRAEFECT_LOADBALANCER_HOST:3305', "gitaly_token" => 'PRAEFECT_EXTERNAL_TOKEN' } }) @@ -957,7 +984,10 @@ Particular attention should be shown to: balancer. - `PRAEFECT_EXTERNAL_TOKEN` with the real secret - If you are using TLS, the `gitaly_address` should begin with `tls://`. + If you are using TLS: + + - The `gitaly_address` should begin with `tls://` instead. + - The port should be changed to `3305`. ```ruby git_data_dirs({ diff --git a/doc/administration/gitaly/recovery.md b/doc/administration/gitaly/recovery.md index e1b9a73908d..9210c8f08d3 100644 --- a/doc/administration/gitaly/recovery.md +++ b/doc/administration/gitaly/recovery.md @@ -28,7 +28,7 @@ To minimize data loss in GitLab 13.0 to 14.0, Gitaly Cluster: ### Read-only mode -> - Introduced in GitLab 13.0 as [generally available](https://about.gitlab.com/handbook/product/gitlab-the-product/#generally-available-ga). +> - Introduced in GitLab 13.0 as [generally available](../../policy/alpha-beta-support.md#generally-available-ga). > - Between GitLab 13.0 and GitLab 13.2, read-only mode applied to the whole virtual storage and occurred whenever failover occurred. > - [In GitLab 13.3 and later](https://gitlab.com/gitlab-org/gitaly/-/issues/2862), read-only mode applies on a per-repository basis and only occurs if a new primary is out of date. new primary. If the failed primary contained unreplicated writes, [data loss can occur](#check-for-data-loss). diff --git a/doc/administration/gitaly/troubleshooting.md b/doc/administration/gitaly/troubleshooting.md index fdd281c1a90..ff136b44340 100644 --- a/doc/administration/gitaly/troubleshooting.md +++ b/doc/administration/gitaly/troubleshooting.md @@ -435,6 +435,15 @@ Here are common errors and potential causes: - Praefect cannot reach one or more of its child Gitaly nodes. Try running the Praefect connection checker to diagnose. +### Praefect database experiencing high CPU load + +Some common reasons for the Praefect database to experience elevated CPU usage include: + +- Prometheus metrics scrapes [running an expensive query](https://gitlab.com/gitlab-org/gitaly/-/issues/3796). If you have GitLab 14.2 + or above, set `praefect['separate_database_metrics'] = true` in `gitlab.rb`. +- [Read distribution caching](praefect.md#reads-distribution-caching) is disabled, increasing the number of queries made to the + database when user traffic is high. Ensure read distribution caching is enabled. + ### Determine primary Gitaly node To determine the primary node of a repository: diff --git a/doc/administration/img/custom_hooks_error_msg.png b/doc/administration/img/custom_hooks_error_msg.png Binary files differdeleted file mode 100644 index e7d5c157d08..00000000000 --- a/doc/administration/img/custom_hooks_error_msg.png +++ /dev/null diff --git a/doc/administration/incoming_email.md b/doc/administration/incoming_email.md index 84b30724dff..cd1f39b1295 100644 --- a/doc/administration/incoming_email.md +++ b/doc/administration/incoming_email.md @@ -62,8 +62,8 @@ can reserve your catch-all mailbox for other purposes. ### Dedicated email address -This solution is relatively simple to set up: you just need to create an email -address dedicated to receive your users' replies to GitLab notifications. However, +To set up this solution, you must create a dedicated email +address to receive your users' replies to GitLab notifications. However, this method only supports replies, and not the other features of [incoming email](#incoming-email). ## Accepted headers @@ -75,7 +75,7 @@ Email is processed correctly when a configured email address is present in one o - `Envelope-To` or `X-Envelope-To` In GitLab 14.6 and later, [Service Desk](../user/project/service_desk.md) -also checks these additional headers. +also checks accepted headers. Usually, the "To" field contains the email address of the primary receiver. However, it might not include the configured GitLab email address if: @@ -84,6 +84,14 @@ However, it might not include the configured GitLab email address if: - The address was included when using "Reply all". - The email was forwarded. +## Rejected headers + +To prevent unwanted issue creation from automatic email systems, GitLab ignores all incoming email +containing the following headers: + +- `Auto-Submitted` with a value other than `no` +- `X-Autoreply` with a value of `yes` + ## Set it up If you want to use Gmail / Google Apps for incoming email, make sure you have diff --git a/doc/administration/instance_limits.md b/doc/administration/instance_limits.md index 3bedb6b01bd..614ab5278c5 100644 --- a/doc/administration/instance_limits.md +++ b/doc/administration/instance_limits.md @@ -214,7 +214,7 @@ or `git push --mirror` is used. This limit does not affect any of the updated merge request pipelines. All updated merge requests have a pipeline created when using -[pipelines for merge requests](../ci/pipelines/merge_request_pipelines.md). +[merge request pipelines](../ci/pipelines/merge_request_pipelines.md). ## Retention of activity history @@ -228,7 +228,11 @@ There is a limit when embedding metrics in GitLab Flavored Markdown (GFM) for pe - **Max limit**: 100 embeds. -## Number of webhooks +## Webhook limits + +Also see [Webhook rate limits](#webhook-rate-limit). + +### Number of webhooks To set the maximum number of group or project webhooks for a self-managed installation, run the following in the [GitLab Rails console](operations/rails_console.md#starting-a-rails-console-session): @@ -246,11 +250,33 @@ Plan.default.actual_limits.update!(group_hooks: 100) Set the limit to `0` to disable it. -- **Default maximum number of webhooks**: `100` per project, `50` per group -- **Maximum payload size**: 25 MB +The default maximum number of webhooks is `100` per project, `50` per group. For GitLab.com, see the [webhook limits for GitLab.com](../user/gitlab_com/index.md#webhooks). +### Webhook payload size + +The maximum webhook payload size is 25 MB. + +### Recursive webhooks + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/329743) in GitLab 14.8. + +GitLab detects and blocks webhooks that are recursive or that exceed the limit +of webhooks that can be triggered from other webhooks. This enables GitLab to +continue to support workflows that use webhooks to call the API non-recursively, or that +do not trigger an unreasonable number of other webhooks. + +Recursion can happen when a webhook is configured to make a call +to its own GitLab instance (for example, the API). The call then triggers the same +webhook and creates an infinite loop. + +The maximum number of requests to an instance made by a series of webhooks that +trigger other webhooks is 100. When the limit is reached, GitLab blocks any further +webhooks that would be triggered by the series. + +Blocked recursive webhook calls are logged in `auth.log` with the message `"Recursive webhook blocked from executing"`. + ## Pull Mirroring Interval > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/237891) in GitLab 13.7. diff --git a/doc/administration/job_artifacts.md b/doc/administration/job_artifacts.md index 62c403bfe43..8f824fcefb3 100644 --- a/doc/administration/job_artifacts.md +++ b/doc/administration/job_artifacts.md @@ -1,6 +1,6 @@ --- stage: Verify -group: Testing +group: Pipeline Insights info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -440,10 +440,7 @@ To change the number of projects listed, change the number in `limit(50)`. #### Delete job artifacts from jobs completed before a specific date WARNING: -These commands remove data permanently from the database and from disk. We -highly recommend running them only under the guidance of a Support Engineer, or -running them in a test environment with a backup of the instance ready to be -restored, just in case. +These commands remove data permanently from both the database and from disk. Before running them, we highly recommend seeking guidance from a Support Engineer, or running them in a test environment with a backup of the instance ready to be restored, just in case. If you need to manually remove job artifacts associated with multiple jobs while **retaining their job logs**, this can be done from the Rails console (`sudo gitlab-rails console`): @@ -490,10 +487,7 @@ If you need to manually remove job artifacts associated with multiple jobs while #### Delete job artifacts and logs from jobs completed before a specific date WARNING: -These commands remove data permanently from both the database and from disk. We -highly recommend running them only under the guidance of a Support Engineer, or -running them in a test environment with a backup of the instance ready to be -restored, just in case. +These commands remove data permanently from both the database and from disk. Before running them, we highly recommend seeking guidance from a Support Engineer, or running them in a test environment with a backup of the instance ready to be restored, just in case. If you need to manually remove **all** job artifacts associated with multiple jobs, **including job logs**, this can be done from the Rails console (`sudo gitlab-rails console`): diff --git a/doc/administration/load_balancer.md b/doc/administration/load_balancer.md index 60293e56202..45fd6e65078 100644 --- a/doc/administration/load_balancer.md +++ b/doc/administration/load_balancer.md @@ -2,7 +2,6 @@ stage: Enablement group: Distribution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference --- # Load Balancer for multi-node GitLab **(FREE SELF)** @@ -21,38 +20,38 @@ How do you want to handle SSL in your multi-node environment? There are several options: - Each application node terminates SSL -- The load balancer(s) terminate SSL and communication is not secure between - the load balancer(s) and the application nodes -- The load balancer(s) terminate SSL and communication is *secure* between the - load balancer(s) and the application nodes +- The load balancers terminate SSL and communication is not secure between + the load balancers and the application nodes +- The load balancers terminate SSL and communication is *secure* between the + load balancers and the application nodes ### Application nodes terminate SSL -Configure your load balancer(s) to pass connections on port 443 as 'TCP' rather +Configure your load balancers to pass connections on port 443 as 'TCP' rather than 'HTTP(S)' protocol. This passes the connection to the application nodes NGINX service untouched. NGINX has the SSL certificate and listen on port 443. See [NGINX HTTPS documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-https) for details on managing SSL certificates and configuring NGINX. -### Load Balancer(s) terminate SSL without backend SSL +### Load Balancers terminate SSL without backend SSL -Configure your load balancer(s) to use the 'HTTP(S)' protocol rather than 'TCP'. -The load balancer(s) is be responsible for managing SSL certificates and +Configure your load balancers to use the 'HTTP(S)' protocol rather than 'TCP'. +The load balancers is be responsible for managing SSL certificates and terminating SSL. -Since communication between the load balancer(s) and GitLab isn't secure, +Since communication between the load balancers and GitLab isn't secure, there is some additional configuration needed. See [NGINX Proxied SSL documentation](https://docs.gitlab.com/omnibus/settings/nginx.html#supporting-proxied-ssl) for details. -### Load Balancer(s) terminate SSL with backend SSL +### Load Balancers terminate SSL with backend SSL -Configure your load balancer(s) to use the 'HTTP(S)' protocol rather than 'TCP'. -The load balancer(s) is responsible for managing SSL certificates that +Configure your load balancers to use the 'HTTP(S)' protocol rather than 'TCP'. +The load balancers is responsible for managing SSL certificates that end users see. -Traffic is secure between the load balancer(s) and NGINX in this +Traffic is secure between the load balancers and NGINX in this scenario. There is no need to add configuration for proxied SSL since the connection is secure all the way. However, configuration must be added to GitLab to configure SSL certificates. See diff --git a/doc/administration/logs.md b/doc/administration/logs.md index 0d7635405d6..c4e9642d048 100644 --- a/doc/administration/logs.md +++ b/doc/administration/logs.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/github_imports.md b/doc/administration/monitoring/github_imports.md index e91483eb79d..e16e9bb0336 100644 --- a/doc/administration/monitoring/github_imports.md +++ b/doc/administration/monitoring/github_imports.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/gitlab_self_monitoring_project/index.md b/doc/administration/monitoring/gitlab_self_monitoring_project/index.md index 1cf4e5a25ba..911d76a89e3 100644 --- a/doc/administration/monitoring/gitlab_self_monitoring_project/index.md +++ b/doc/administration/monitoring/gitlab_self_monitoring_project/index.md @@ -1,13 +1,17 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Self monitoring project **(FREE SELF)** +# Self-monitoring project (DEPRECATED) **(FREE SELF)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32351) in GitLab 12.7, behind a disabled feature flag (`self_monitoring_project`). -> - The feature flag was removed and the self monitoring project was [made generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/198511) in GitLab 12.8. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32351) in GitLab 12.7 [with a flag](../../feature_flags.md) named `self_monitoring_project`. Disabled by default. +> - Generally available in GitLab 12.8. [Feature flag `self_monitoring_project`](https://gitlab.com/gitlab-org/gitlab/-/issues/198511) removed. +> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/348909) in GitLab 14.8. + +WARNING: +This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/348909) in GitLab 14.8. GitLab provides administrators insights into the health of their GitLab instance. diff --git a/doc/administration/monitoring/index.md b/doc/administration/monitoring/index.md index 4c49efd6bd5..df655053723 100644 --- a/doc/administration/monitoring/index.md +++ b/doc/administration/monitoring/index.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/ip_whitelist.md b/doc/administration/monitoring/ip_whitelist.md index 75b09f8a366..b8347ba3f0d 100644 --- a/doc/administration/monitoring/ip_whitelist.md +++ b/doc/administration/monitoring/ip_whitelist.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/performance/gitlab_configuration.md b/doc/administration/monitoring/performance/gitlab_configuration.md index f316a75a868..128ddad6555 100644 --- a/doc/administration/monitoring/performance/gitlab_configuration.md +++ b/doc/administration/monitoring/performance/gitlab_configuration.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/performance/grafana_configuration.md b/doc/administration/monitoring/performance/grafana_configuration.md index c37a264938e..79612145327 100644 --- a/doc/administration/monitoring/performance/grafana_configuration.md +++ b/doc/administration/monitoring/performance/grafana_configuration.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/performance/index.md b/doc/administration/monitoring/performance/index.md index f3db6ac9f03..20fad8baf91 100644 --- a/doc/administration/monitoring/performance/index.md +++ b/doc/administration/monitoring/performance/index.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/performance/performance_bar.md b/doc/administration/monitoring/performance/performance_bar.md index 14a560223f9..e93dbd3a2d6 100644 --- a/doc/administration/monitoring/performance/performance_bar.md +++ b/doc/administration/monitoring/performance/performance_bar.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -108,7 +108,7 @@ appears next to requests with warnings. The performance bar is disabled by default for non-administrators. To enable it for a given group: -1. Sign in as a user with Administrator [role](../../../user/permissions.md). +1. Sign in as a user with administrator access. 1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Settings > Metrics and profiling** (`admin/application_settings/metrics_and_profiling`), and expand diff --git a/doc/administration/monitoring/performance/request_profiling.md b/doc/administration/monitoring/performance/request_profiling.md index ebdca8d3960..f8a66f6f48f 100644 --- a/doc/administration/monitoring/performance/request_profiling.md +++ b/doc/administration/monitoring/performance/request_profiling.md @@ -1,10 +1,16 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Request Profiling **(FREE SELF)** +# Request profiling (DEPRECATED) **(FREE SELF)** + +> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/352488) in GitLab 14.8, and planned for removal in GitLab 15.0. + +WARNING: +This feature is in its end-of-life process. It is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/352488) +for use in GitLab 14.8, and is planned for removal in GitLab 15.0. To profile a request: diff --git a/doc/administration/monitoring/prometheus/gitlab_exporter.md b/doc/administration/monitoring/prometheus/gitlab_exporter.md index d9852524aec..15ec880533e 100644 --- a/doc/administration/monitoring/prometheus/gitlab_exporter.md +++ b/doc/administration/monitoring/prometheus/gitlab_exporter.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/prometheus/gitlab_metrics.md b/doc/administration/monitoring/prometheus/gitlab_metrics.md index c5b87afd94b..a1bfae678c7 100644 --- a/doc/administration/monitoring/prometheus/gitlab_metrics.md +++ b/doc/administration/monitoring/prometheus/gitlab_metrics.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w To enable the GitLab Prometheus metrics: -1. Log in to GitLab as a user with Administrator [role](../../../user/permissions.md). +1. Log in to GitLab as a user with administrator access. 1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Settings > Metrics and profiling**. 1. Find the **Metrics - Prometheus** section, and select **Add link to Prometheus**. @@ -48,6 +48,7 @@ The following metrics are available: | `gitlab_database_transaction_seconds` | Histogram | 12.1 | Time spent in database transactions, in seconds | | | `gitlab_method_call_duration_seconds` | Histogram | 10.2 | Method calls real duration | `controller`, `action`, `module`, `method` | | `gitlab_page_out_of_bounds` | Counter | 12.8 | Counter for the PageLimiter pagination limit being hit | `controller`, `action`, `bot` | +| `gitlab_rails_boot_time_seconds` | Gauge | 14.8 | Time elapsed for Rails primary process to finish startup | | | `gitlab_rails_queue_duration_seconds` | Histogram | 9.4 | Measures latency between GitLab Workhorse forwarding a request to Rails | | | `gitlab_sql_duration_seconds` | Histogram | 10.2 | SQL execution time, excluding `SCHEMA` operations and `BEGIN` / `COMMIT` | | | `gitlab_sql_<role>_duration_seconds` | Histogram | 13.10 | SQL execution time, excluding `SCHEMA` operations and `BEGIN` / `COMMIT`, grouped by database roles (primary/replica) | | @@ -269,6 +270,16 @@ configuration option in `gitlab.yml`. These metrics are served from the | `geo_pages_deployments_verification_total` | Gauge | 14.6 | Number of pages deployments verifications tried on secondary | `url` | | `geo_pages_deployments_verified` | Gauge | 14.6 | Number of pages deployments verified on secondary | `url` | | `geo_pages_deployments_verification_failed` | Gauge | 14.6 | Number of pages deployments verifications failed on secondary | `url` | +| `geo_job_artifacts` | Gauge | 14.8 | Number of job artifacts on primary | `url` | +| `geo_job_artifacts_checksum_total` | Gauge | 14.8 | Number of job artifacts tried to checksum on primary | `url` | +| `geo_job_artifacts_checksummed` | Gauge | 14.8 | Number of job artifacts successfully checksummed on primary | `url` | +| `geo_job_artifacts_checksum_failed` | Gauge | 14.8 | Number of job artifacts failed to calculate the checksum on primary | `url` | +| `geo_job_artifacts_synced` | Gauge | 14.8 | Number of syncable job artifacts synced on secondary | `url` | +| `geo_job_artifacts_failed` | Gauge | 14.8 | Number of syncable job artifacts failed to sync on secondary | `url` | +| `geo_job_artifacts_registry` | Gauge | 14.8 | Number of job artifacts in the registry | `url` | +| `geo_job_artifacts_verification_total` | Gauge | 14.8 | Number of job artifacts verifications tried on secondary | `url` | +| `geo_job_artifacts_verified` | Gauge | 14.8 | Number of job artifacts verified on secondary | `url` | +| `geo_job_artifacts_verification_failed` | Gauge | 14.8 | Number of job artifacts verifications failed on secondary | `url` | | `limited_capacity_worker_running_jobs` | Gauge | 13.5 | Number of running jobs | `worker` | | `limited_capacity_worker_max_running_jobs` | Gauge | 13.5 | Maximum number of running jobs | `worker` | | `limited_capacity_worker_remaining_work_count` | Gauge | 13.5 | Number of jobs waiting to be enqueued | `worker` | diff --git a/doc/administration/monitoring/prometheus/index.md b/doc/administration/monitoring/prometheus/index.md index acdcdb41dca..9d90c4f3cc7 100644 --- a/doc/administration/monitoring/prometheus/index.md +++ b/doc/administration/monitoring/prometheus/index.md @@ -295,6 +295,36 @@ To use an external Prometheus server: 1. Reload the Prometheus server. +### Configure the storage retention size + +Prometheus has several custom flags to configure local storage: + +- `storage.tsdb.retention.time`: when to remove old data. Defaults to `15d`. Overrides + `storage.tsdb.retention` if this flag is set to anything other than the default. +- `storage.tsdb.retention.size`: [EXPERIMENTAL] the maximum number of bytes of storage blocks to + retain. The oldest data is removed first. Defaults to `0` (disabled). This flag is experimental + and may change in future releases. Units supported: `B`, `KB`, `MB`, `GB`, `TB`, `PB`, `EB`. For + example, `512MB`. + +To configure the storage retention size: + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + prometheus['flags'] = { + 'storage.tsdb.path' => "/var/opt/gitlab/prometheus/data", + 'storage.tsdb.retention.time' => "7d", + 'storage.tsdb.retention.size' => "2GB", + 'config.file' => "/var/opt/gitlab/prometheus/prometheus.yml" + } + ``` + +1. Reconfigure GitLab: + + ```shell + sudo gitlab-ctl reconfigure + ``` + ## Viewing performance metrics You can visit `http://localhost:9090` for the dashboard that Prometheus offers by default. @@ -402,3 +432,35 @@ To disable the monitoring of Kubernetes: 1. Save the file and [reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. + +## Troubleshooting + +### `/var/opt/gitlab/prometheus` consumes too much disk space + +If you are **not** using Prometheus monitoring: + +1. [Disable Prometheus](index.md#configuring-prometheus). +1. Delete the data under `/var/opt/gitlab/prometheus`. + +If you are using Prometheus monitoring: + +1. Stop Prometheus (deleting data while it's running can cause data corruption): + + ```shell + gitlab-ctl stop prometheus + ``` + +1. Delete the data under `/var/opt/gitlab/prometheus/data`. +1. Start the service again: + + ```shell + gitlab-ctl start prometheus + ``` + +1. Verify the service is up and running: + + ```shell + gitlab-ctl status prometheus + ``` + +1. Optional. [Configure the storage retention size](index.md#configure-the-storage-retention-size). diff --git a/doc/administration/monitoring/prometheus/node_exporter.md b/doc/administration/monitoring/prometheus/node_exporter.md index 68d997d7596..d7a4a96cd9a 100644 --- a/doc/administration/monitoring/prometheus/node_exporter.md +++ b/doc/administration/monitoring/prometheus/node_exporter.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/prometheus/pgbouncer_exporter.md b/doc/administration/monitoring/prometheus/pgbouncer_exporter.md index aba1561500a..979a6bcd232 100644 --- a/doc/administration/monitoring/prometheus/pgbouncer_exporter.md +++ b/doc/administration/monitoring/prometheus/pgbouncer_exporter.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/prometheus/postgres_exporter.md b/doc/administration/monitoring/prometheus/postgres_exporter.md index 8a851afe35b..95a6540bd19 100644 --- a/doc/administration/monitoring/prometheus/postgres_exporter.md +++ b/doc/administration/monitoring/prometheus/postgres_exporter.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/prometheus/puma_exporter.md b/doc/administration/monitoring/prometheus/puma_exporter.md index 804c4243cfa..794e2c10b25 100644 --- a/doc/administration/monitoring/prometheus/puma_exporter.md +++ b/doc/administration/monitoring/prometheus/puma_exporter.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/prometheus/redis_exporter.md b/doc/administration/monitoring/prometheus/redis_exporter.md index 6cc262842a1..a5f12bbc52f 100644 --- a/doc/administration/monitoring/prometheus/redis_exporter.md +++ b/doc/administration/monitoring/prometheus/redis_exporter.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/monitoring/prometheus/registry_exporter.md b/doc/administration/monitoring/prometheus/registry_exporter.md index 3a2acd47338..f4fa35c206e 100644 --- a/doc/administration/monitoring/prometheus/registry_exporter.md +++ b/doc/administration/monitoring/prometheus/registry_exporter.md @@ -1,6 +1,6 @@ --- stage: Monitor -group: Monitor +group: Respond info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/nfs.md b/doc/administration/nfs.md index 87a10a0ec04..50c8102f366 100644 --- a/doc/administration/nfs.md +++ b/doc/administration/nfs.md @@ -24,11 +24,9 @@ file system performance, see Starting with GitLab version 14.0, support for NFS to store Git repository data is deprecated. Technical customer support and engineering support is available for the 14.x releases. Engineering is fixing bugs and security vulnerabilities consistent with our [release and maintenance policy](../policy/maintenance.md#security-releases). -At the end of the 14.12 milestone (tentatively June 22nd, 2022) technical and engineering support for using NFS to store Git repository data will be officially at end-of-life. There will be no product changes or troubleshooting provided via Engineering, Security or Paid Support channels. +Upon the release of GitLab 15.0 (tentatively May 22nd, 2022) technical and engineering support for using NFS to store Git repository data will be officially at end-of-life. There will be no product changes or troubleshooting provided via Engineering, Security or Paid Support channels after the release date of 15.0, regardless of your GitLab version. -For those customers still running earlier versions of GitLab, [our support eligibility and maintenance policy applies](https://about.gitlab.com/support/statement-of-support.html#version-support). - -For the 14.x releases, we continue to help with Git related tickets from customers running one or more Gitaly servers with its data stored on NFS. Examples may include: +Until the release of 15.0, for customers running 14.x releases, we continue to help with Git related tickets from customers running one or more Gitaly servers with its data stored on NFS. Examples may include: - Performance issues or timeouts accessing Git data - Commits or branches vanish diff --git a/doc/administration/object_storage.md b/doc/administration/object_storage.md index 0976fc3684e..420f2191ef2 100644 --- a/doc/administration/object_storage.md +++ b/doc/administration/object_storage.md @@ -526,7 +526,7 @@ To migrate existing local data to object storage see the following guides: - [Uploads](raketasks/uploads/migrate.md#migrate-to-object-storage) - [Merge request diffs](merge_request_diffs.md#using-object-storage) - [Packages](packages/index.md#migrating-local-packages-to-object-storage) (optional feature) -- Dependency Proxy - [migration not yet supported](https://gitlab.com/gitlab-org/gitlab/-/issues/343064) +- [Dependency Proxy](packages/dependency_proxy.md#migrate-local-dependency-proxy-blobs-and-manifests-to-object-storage) - [Terraform state files](terraform_state.md#migrate-to-object-storage) - [Pages content](pages/index.md#migrate-pages-deployments-to-object-storage) diff --git a/doc/administration/operations/extra_sidekiq_processes.md b/doc/administration/operations/extra_sidekiq_processes.md index 1c9b98041dc..fd04efe8473 100644 --- a/doc/administration/operations/extra_sidekiq_processes.md +++ b/doc/administration/operations/extra_sidekiq_processes.md @@ -296,7 +296,7 @@ Instead of a queue, a queue namespace can also be provided, to have the process automatically listen on all queues in that namespace without needing to explicitly list all the queue names. For more information about queue namespaces, see the relevant section in the -[Sidekiq style guide](../../development/sidekiq_style_guide.md#queue-namespaces). +[Sidekiq development documentation](../../development/sidekiq/index.md#queue-namespaces). For example, say you want to start 2 extra processes: one to process the `process_commit` queue, and one to process the `post_receive` queue. This can be diff --git a/doc/administration/operations/fast_ssh_key_lookup.md b/doc/administration/operations/fast_ssh_key_lookup.md index ddedb3fe76a..dca99879cc3 100644 --- a/doc/administration/operations/fast_ssh_key_lookup.md +++ b/doc/administration/operations/fast_ssh_key_lookup.md @@ -138,7 +138,7 @@ This is a brief overview. Please refer to the above instructions for more contex > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/299109) in GitLab 14.5. WARNING: -`gitlab-sshd` is in [**Alpha**](https://about.gitlab.com/handbook/product/gitlab-the-product/#alpha-beta-ga). +`gitlab-sshd` is in [**Alpha**](../../policy/alpha-beta-support.md#alpha-features). It is not ready for production use. `gitlab-sshd` is [a standalone SSH server](https://gitlab.com/gitlab-org/gitlab-shell/-/tree/main/internal/sshd) @@ -178,3 +178,22 @@ GitLab supports `authorized_keys` database lookups with [SELinux](https://en.wik Because the SELinux policy is static, GitLab doesn't support the ability to change internal webserver ports at the moment. Administrators would have to create a special `.te` file for the environment, since it isn't generated dynamically. + +## Troubleshooting + +If your SSH traffic is [slow](https://github.com/linux-pam/linux-pam/issues/270) +or causing high CPU load, be sure to check the size of `/var/log/btmp`, and ensure it is rotated on a regular basis. +If this file is very large, GitLab SSH fast lookup can cause the bottleneck to be hit more frequently, thus decreasing performance even further. +If you are able to, you may consider disabling [`UsePAM` in your `sshd_config`](https://linux.die.net/man/5/sshd_config) to avoid reading `/var/log/btmp` altogether. + +Running `strace` and `lsof` on a running `sshd: git` process can return useful debugging information. To get an `strace` on an in-progress Git over SSH connection for IP `x.x.x.x`, run: + +```plaintext +sudo strace -s 10000 -p $(sudo netstat -tp | grep x.x.x.x | egrep 'ssh.*: git' | sed -e 's/.*ESTABLISHED *//' -e 's#/.*##') +``` + +Or get an `lsof` for a running Git over SSH process: + +```plaintext +sudo lsof -p $(sudo netstat -tp | egrep 'ssh.*: git' | head -1 | sed -e 's/.*ESTABLISHED *//' -e 's#/.*##') +``` diff --git a/doc/administration/package_information/supported_os.md b/doc/administration/package_information/supported_os.md index 78f496a3998..71ebc4d3647 100644 --- a/doc/administration/package_information/supported_os.md +++ b/doc/administration/package_information/supported_os.md @@ -23,7 +23,6 @@ The following lists the currently supported OSs and their possible EOL dates. | Debian 9 | GitLab CE / GitLab EE 9.3.0 | amd64 | 2022 | <https://wiki.debian.org/LTS> | | Debian 10 | GitLab CE / GitLab EE 12.2.0 | amd64, arm64 | 2024 | <https://wiki.debian.org/LTS> | | Debian 11 | GitLab CE / GitLab EE 14.6.0 | amd64, arm64 | 2026 | <https://wiki.debian.org/LTS> | -| OpenSUSE 15.2 | GitLab CE / GitLab EE 13.11.0 | x86_64, aarch64 | Dec 2021 | <https://en.opensuse.org/Lifetime> | | OpenSUSE 15.3 | GitLab CE / GitLab EE 14.5.0 | x86_64, aarch64 | Nov 2022 | <https://en.opensuse.org/Lifetime> | | SLES 12 | GitLab EE 9.0.0 | x86_64 | Oct 2027 | <https://www.suse.com/lifecycle/> | | Ubuntu 18.04 | GitLab CE / GitLab EE 10.7.0 | amd64 | April 2023 | <https://wiki.ubuntu.com/Releases> | @@ -81,8 +80,9 @@ release for them can be found below: | Raspbian Stretch | [June 2020](https://downloads.raspberrypi.org/raspbian/images/raspbian-2019-04-09/) | [GitLab CE](https://packages.gitlab.com/app/gitlab/raspberry-pi2/search?q=gitlab-ce_13.2&dist=raspbian%2Fstretch) 13.3 | | Debian Jessie | [June 2020](https://www.debian.org/News/2020/20200709) | [GitLab CE](https://packages.gitlab.com/app/gitlab/gitlab-ce/search?q=gitlab-ce_13.2&dist=debian%2Fjessie) / [GitLab EE](https://packages.gitlab.com/app/gitlab/gitlab-ee/search?q=gitlab-ee_13.2&dist=debian%2Fjessie) 13.3 | | CentOS 6 | [November 2020](https://wiki.centos.org/About/Product) | [GitLab CE](https://packages.gitlab.com/app/gitlab/gitlab-ce/search?q=13.6&filter=all&filter=all&dist=el%2F6) / [GitLab EE](https://packages.gitlab.com/app/gitlab/gitlab-ee/search?q=13.6&filter=all&filter=all&dist=el%2F6) 13.6 | -| OpenSUSE 15.1 | [November 2020](https://en.opensuse.org/Lifetime#Discontinued_distributions) | [GitLab CE](https://packages.gitlab.com/app/gitlab/gitlab-ce/search?q=gitlab-ce-13.12&dist=opensuse%2F15.1) / [GitLab EE](https://packages.gitlab.com/app/gitlab/gitlab-ee/search?q=gitlab-ee-13.12&dist=opensuse%2F15.2) 13.12 | +| OpenSUSE 15.1 | [November 2020](https://en.opensuse.org/Lifetime#Discontinued_distributions) | [GitLab CE](https://packages.gitlab.com/app/gitlab/gitlab-ce/search?q=gitlab-ce-13.12&dist=opensuse%2F15.1) / [GitLab EE](https://packages.gitlab.com/app/gitlab/gitlab-ee/search?q=gitlab-ee-13.12&dist=opensuse%2F15.1) 13.12 | | Ubuntu 16.04 | [April 2021](https://ubuntu.com/info/release-end-of-life) | [GitLab CE](https://packages.gitlab.com/app/gitlab/gitlab-ce/search?q=gitlab-ce_13.12&dist=ubuntu%2Fxenial) / [GitLab EE](https://packages.gitlab.com/app/gitlab/gitlab-ee/search?q=gitlab-ee_13.12&dist=ubuntu%2Fxenial) 13.12 | +| OpenSUSE 15.2 | [December 2021](https://en.opensuse.org/Lifetime#Discontinued_distributions) | [GitLab CE](https://packages.gitlab.com/app/gitlab/gitlab-ce/search?q=gitlab-ce-14.7&dist=opensuse%2F15.2) / [GitLab EE](https://packages.gitlab.com/app/gitlab/gitlab-ee/search?q=gitlab-ee-14.7&dist=opensuse%2F15.2) 14.7 | NOTE: An exception to this deprecation policy is when we are unable to provide diff --git a/doc/administration/packages/container_registry.md b/doc/administration/packages/container_registry.md index 43293385ed9..33a5311709f 100644 --- a/doc/administration/packages/container_registry.md +++ b/doc/administration/packages/container_registry.md @@ -662,14 +662,14 @@ configurable in future releases. The Registry server listens on localhost at port `5000` by default, which is the address for which the Registry server should accept connections. -In the examples below we set the Registry's port to `5001`. +In the examples below we set the Registry's port to `5010`. **Omnibus GitLab** 1. Open `/etc/gitlab/gitlab.rb` and set `registry['registry_http_addr']`: ```ruby - registry['registry_http_addr'] = "localhost:5001" + registry['registry_http_addr'] = "localhost:5010" ``` 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. @@ -681,7 +681,7 @@ In the examples below we set the Registry's port to `5001`. ```yaml http: - addr: localhost:5001 + addr: localhost:5010 ``` 1. Save the file and restart the Registry server. @@ -1379,7 +1379,7 @@ project or branch name. Special characters can include: To get around this, you can [change the group path](../../user/group/index.md#change-a-groups-path), [change the project path](../../user/project/settings/index.md#renaming-a-repository) or change the -branch name. Another option is to create a [push rule](../../push_rules/push_rules.md) to prevent +branch name. Another option is to create a [push rule](../../user/project/repository/push_rules.md) to prevent this at the instance level. ### Image push errors @@ -1548,6 +1548,46 @@ To fix this you can do one of two things: We use a concrete example to illustrate how to diagnose a problem with the S3 setup. +#### Investigate a cleanup policy + +If you're unsure why your cleanup policy did or didn't delete a tag, execute the policy line by line +by running the below script from the [Rails console](../../administration/operations/rails_console.md). +This can help diagnose problems with the policy. + +```ruby +repo = ContainerRepository.find(<project_id>) +policy = repo.project.container_expiration_policy + +tags = repo.tags +tags.map(&:name) + +tags.reject!(&:latest?) +tags.map(&:name) + +regex_delete = ::Gitlab::UntrustedRegexp.new("\\A#{policy.name_regex}\\z") +regex_retain = ::Gitlab::UntrustedRegexp.new("\\A#{policy.name_regex_keep}\\z") + +tags.select! { |tag| regex_delete.match?(tag.name) && !regex_retain.match?(tag.name) } + +tags.map(&:name) + +now = DateTime.current +tags.sort_by! { |tag| tag.created_at || now }.reverse! # Lengthy operation + +tags = tags.drop(policy.keep_n) +tags.map(&:name) + +older_than_timestamp = ChronicDuration.parse(policy.older_than).seconds.ago + +tags.select! { |tag| tag.created_at && tag.created_at < older_than_timestamp } + +tags.map(&:name) +``` + +- The script builds the list of tags to delete (`tags`). +- `tags.map(&:name)` prints a list of tags to remove. This may be a lengthy operation. +- After each filter, check the list of `tags` to see if it contains the intended tags to destroy. + #### Unexpected 403 error during push A user attempted to enable an S3-backed Registry. The `docker login` step went diff --git a/doc/administration/packages/dependency_proxy.md b/doc/administration/packages/dependency_proxy.md index b3dc6ffc2b2..93c6585230f 100644 --- a/doc/administration/packages/dependency_proxy.md +++ b/doc/administration/packages/dependency_proxy.md @@ -199,6 +199,55 @@ This section describes the earlier configuration format. 1. [Restart GitLab](../restart_gitlab.md#installations-from-source "How to restart GitLab") for the changes to take effect. +#### Migrate local Dependency Proxy blobs and manifests to object storage + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/79663) in GitLab 14.8. + +After [configuring object storage](#using-object-storage), +use the following task to migrate existing Dependency Proxy blobs and manifests from local storage +to remote storage. The processing is done in a background worker and requires no downtime. + +For Omnibus GitLab: + +```shell +sudo gitlab-rake "gitlab:dependency_proxy:migrate" +``` + +For installations from source: + +```shell +RAILS_ENV=production sudo -u git -H bundle exec rake gitlab:dependency_proxy:migrate +``` + +You can optionally track progress and verify that all packages migrated successfully using the +[PostgreSQL console](https://docs.gitlab.com/omnibus/settings/database.html#connecting-to-the-bundled-postgresql-database): + +- For Omnibus GitLab instances: `sudo gitlab-rails dbconsole` +- For installations from source: `sudo -u git -H psql -d gitlabhq_production` + +Verify that `objectstg` (where `file_store = '2'`) has the count of all Dependency Proxy blobs and +manifests for each respective query: + +```shell +gitlabhq_production=# SELECT count(*) AS total, sum(case when file_store = '1' then 1 else 0 end) AS filesystem, sum(case when file_store = '2' then 1 else 0 end) AS objectstg FROM dependency_proxy_blobs; + +total | filesystem | objectstg +------+------------+----------- + 22 | 0 | 22 + +gitlabhq_production=# SELECT count(*) AS total, sum(case when file_store = '1' then 1 else 0 end) AS filesystem, sum(case when file_store = '2' then 1 else 0 end) AS objectstg FROM dependency_proxy_manifests; + +total | filesystem | objectstg +------+------------+----------- + 10 | 0 | 10 +``` + +Verify that there are no files on disk in the `dependency_proxy` folder: + +```shell +sudo find /var/opt/gitlab/gitlab-rails/shared/dependency_proxy -type f | grep -v tmp | wc -l +``` + ## Disabling Authentication Authentication was introduced in 13.7 as part of [enabling private groups to use the diff --git a/doc/administration/packages/index.md b/doc/administration/packages/index.md index eea4964efbe..abf5c46114b 100644 --- a/doc/administration/packages/index.md +++ b/doc/administration/packages/index.md @@ -240,7 +240,7 @@ You can optionally track progress and verify that all packages migrated successf - `sudo gitlab-rails dbconsole` for Omnibus GitLab instances. - `sudo -u git -H psql -d gitlabhq_production` for source-installed instances. -Verify `objectstg` below (where `store=2`) has count of all packages: +Verify `objectstg` below (where `file_store = '2'`) has count of all packages: ```shell gitlabhq_production=# SELECT count(*) AS total, sum(case when file_store = '1' then 1 else 0 end) AS filesystem, sum(case when file_store = '2' then 1 else 0 end) AS objectstg FROM packages_package_files; diff --git a/doc/administration/pages/img/pages_deploy_failure_v14_8.png b/doc/administration/pages/img/pages_deploy_failure_v14_8.png Binary files differnew file mode 100644 index 00000000000..ec502b86453 --- /dev/null +++ b/doc/administration/pages/img/pages_deploy_failure_v14_8.png diff --git a/doc/administration/pages/index.md b/doc/administration/pages/index.md index 53510688794..f000824711f 100644 --- a/doc/administration/pages/index.md +++ b/doc/administration/pages/index.md @@ -238,6 +238,7 @@ control over how the Pages daemon runs and serves content in your environment. | `artifacts_server_url` | API URL to proxy artifact requests to. Defaults to GitLab `external URL` + `/api/v4`, for example `https://gitlab.com/api/v4`. When running a [separate Pages server](#running-gitlab-pages-on-a-separate-server), this URL must point to the main GitLab server's API. | | `auth_redirect_uri` | Callback URL for authenticating with GitLab. Defaults to project's subdomain of `pages_external_url` + `/auth`. | | `auth_secret` | Secret key for signing authentication requests. Leave blank to pull automatically from GitLab during OAuth registration. | +| `client_cert_key_pairs` | Client certificates and keys used for mutual TLS with the GitLab API. See [Support mutual TLS when calling the GitLab API](#support-mutual-tls-when-calling-the-gitlab-api) for details. [Introduced](https://gitlab.com/gitlab-org/gitlab-pages/-/issues/548) in GitLab 14.8. | | `dir` | Working directory for configuration and secrets files. | | `enable` | Enable or disable GitLab Pages on the current system. | | `external_http` | Configure Pages to bind to one or more secondary IP addresses, serving HTTP requests. Multiple addresses can be given as an array, along with exact ports, for example `['1.2.3.4', '1.2.3.5:8063']`. Sets value for `listen_http`. | @@ -511,6 +512,22 @@ Authority (CA) in the system certificate store. For Omnibus, this is fixed by [installing a custom CA in Omnibus GitLab](https://docs.gitlab.com/omnibus/settings/ssl.html#install-custom-public-certificates). +### Support mutual TLS when calling the GitLab API + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-pages/-/issues/548) in GitLab 14.8. + +If GitLab has been [configured to require mutual TLS](https://docs.gitlab.com/omnibus/settings/nginx.html#enable-2-way-ssl-client-authentication), you need to add the client certificates to Pages: + +1. Configure in `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_pages['client_cert_key_pairs'] = ['</path/to/cert>:</path/to/key>'] + ``` + + Where `</path/to/cert>` and `</path/to/key>` are the file paths to the client certificate and its respective key file. + Both of these files must be encoded in PEM format. +1. To configure Pages to validate the server certificates, [add the root CA to the system trust store](#using-a-custom-certificate-authority-ca). + ### ZIP serving and cache configuration > [Introduced](https://gitlab.com/gitlab-org/gitlab-pages/-/merge_requests/392) in GitLab 13.7. @@ -688,7 +705,7 @@ To override the global maximum pages size for a specific group: ## Running GitLab Pages on a separate server You can run the GitLab Pages daemon on a separate server to decrease the load on -your main application server. This configuration does not support mutual TLS (mTLS). See the [corresponding feature proposal](https://gitlab.com/gitlab-org/gitlab-pages/-/issues/548) for more information. +your main application server. To configure GitLab Pages on a separate server: @@ -1021,6 +1038,25 @@ Migrate your existing Pages deployments from local storage to object storage: sudo gitlab-rake gitlab:pages:deployments:migrate_to_object_storage ``` +You can track progress and verify that all Pages deployments migrated successfully using the +[PostgreSQL console](https://docs.gitlab.com/omnibus/settings/database.html#connecting-to-the-bundled-postgresql-database): + +- `sudo gitlab-rails dbconsole` for Omnibus GitLab instances. +- `sudo -u git -H psql -d gitlabhq_production` for source-installed instances. + +Verify `objectstg` below (where `store=2`) has count of all Pages deployments: + +```shell +gitlabhq_production=# SELECT count(*) AS total, sum(case when file_store = '1' then 1 else 0 end) AS filesystem, sum(case when file_store = '2' then 1 else 0 end) AS objectstg FROM pages_deployments; + +total | filesystem | objectstg +------+------------+----------- + 10 | 0 | 10 +``` + +After verifying everything is working correctly, +[disable Pages local storage](#disable-pages-local-storage). + ### Rolling Pages deployments back to local storage After the migration to object storage is performed, you can choose to move your Pages deployments back to local storage: @@ -1033,7 +1069,7 @@ sudo gitlab-rake gitlab:pages:deployments:migrate_to_local > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/301159) in GitLab 13.11. -If you use [object storage](#using-object-storage), you can disable local storage: +If you use [object storage](#using-object-storage), you can disable local storage to avoid unnecessary disk usage/writes: 1. Edit `/etc/gitlab/gitlab.rb`: @@ -1064,6 +1100,8 @@ is the default starting from GitLab 14.0. Skip this step if you're already runni 1. If you want to store your pages content in [object storage](#using-object-storage), make sure to configure it. If you want to store the pages content locally or continue using an NFS server, skip this step. 1. [Migrate legacy storage to ZIP storage.](#migrate-legacy-storage-to-zip-storage) +1. If you have configured GitLab to store your pages content in [object storage](#using-object-storage), + [migrate Pages deployments to object storage](#migrate-pages-deployments-to-object-storage) 1. Upgrade GitLab to 14.0. ## Backup @@ -1077,8 +1115,6 @@ than GitLab to prevent XSS attacks. ### Rate limits -> [Introduced](https://gitlab.com/gitlab-org/gitlab-pages/-/issues/631) in GitLab 14.5. - You can enforce rate limits to help minimize the risk of a Denial of Service (DoS) attack. GitLab Pages uses a [token bucket algorithm](https://en.wikipedia.org/wiki/Token_bucket) to enforce rate limiting. By default, requests that exceed the specified limits are reported but not rejected. @@ -1098,6 +1134,8 @@ Rate limits are enforced using the following: #### Enable source-IP rate limits +> [Introduced](https://gitlab.com/gitlab-org/gitlab-pages/-/issues/631) in GitLab 14.5. + 1. Set rate limits in `/etc/gitlab/gitlab.rb`: ```ruby @@ -1116,6 +1154,8 @@ Rate limits are enforced using the following: #### Enable domain rate limits +> [Introduced](https://gitlab.com/gitlab-org/gitlab-pages/-/issues/630) in GitLab 14.7. + 1. Set rate limits in `/etc/gitlab/gitlab.rb`: ```ruby @@ -1495,3 +1535,20 @@ behavior: NOTE: `inplace_chroot` option might not work with the other features, such as [Pages Access Control](#access-control). The [GitLab Pages README](https://gitlab.com/gitlab-org/gitlab-pages#caveats) has more information about caveats and workarounds. + +### GitLab Pages deploy job fails with error "is not a recognized provider" + +If the **pages** job succeeds but the **deploy** job gives the error "is not a recognized provider": + + + +The error message `is not a recognized provider` could be coming from the `fog` gem that GitLab uses to connect to cloud providers for object storage. + +To fix that: + +1. Check your `gitlab.rb` file. If you have `gitlab_rails['pages_object_store_enabled']` enabled, but no bucket details have been configured, either: + + - Configure object storage for your Pages deployments, following the [S3-compatible connection settings](#s3-compatible-connection-settings) guide. + - Store your deployments locally, by commenting out that line. + +1. Save the changes you made to your `gitlab.rb` file, then [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). diff --git a/doc/administration/postgresql/database_load_balancing.md b/doc/administration/postgresql/database_load_balancing.md index b83820dd0b6..1324666c32b 100644 --- a/doc/administration/postgresql/database_load_balancing.md +++ b/doc/administration/postgresql/database_load_balancing.md @@ -147,9 +147,9 @@ When the list of hosts is updated, it might take a while for the old connections to be terminated. The `disconnect_timeout` setting can be used to enforce an upper limit on the time it takes to terminate all old database connections. -### Handling Stale Reads **(PREMIUM SELF)** +### Handling stale reads -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/3526) in GitLab 10.3. +> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/327902) from GitLab Premium to GitLab Free in 14.0. To prevent reading from an outdated secondary the load balancer checks if it is in sync with the primary. If the data is recent enough, the diff --git a/doc/administration/postgresql/replication_and_failover.md b/doc/administration/postgresql/replication_and_failover.md index 5777f35bfcf..8c7151606a5 100644 --- a/doc/administration/postgresql/replication_and_failover.md +++ b/doc/administration/postgresql/replication_and_failover.md @@ -1029,7 +1029,7 @@ Considering these, you should carefully plan your PostgreSQL upgrade: ``` NOTE: - `gitlab-ctl pg-upgrade` tries to detect the role of the node. If for any reason the auto-detection does not work or you believe it did not detect the role correctly, you can use the `--leader` or `--replica` arguments to manually override it. + On a Geo secondary site, the Patroni leader node is called `standby leader`. 1. Stop Patroni **only on replicas**. @@ -1049,6 +1049,11 @@ Considering these, you should carefully plan your PostgreSQL upgrade: sudo gitlab-ctl pg-upgrade -V 12 ``` + NOTE: + `gitlab-ctl pg-upgrade` tries to detect the role of the node. If for any reason the auto-detection + does not work or you believe it did not detect the role correctly, you can use the `--leader` or + `--replica` arguments to manually override it. + 1. Check the status of the leader and cluster. You can proceed only if you have a healthy leader: ```shell diff --git a/doc/administration/raketasks/ldap.md b/doc/administration/raketasks/ldap.md index e6adb98a92a..cae2465fad8 100644 --- a/doc/administration/raketasks/ldap.md +++ b/doc/administration/raketasks/ldap.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Authentication & Authorization +group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index a5c60af47b1..eb1127b5e99 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -12,7 +12,7 @@ full list of reference architectures, see > - **Supported users (approximate):** 10,000 > - **High Availability:** Yes ([Praefect](#configure-praefect-postgresql) needs a third-party PostgreSQL solution for HA) -> - **Estimated Costs:** [GCP](https://cloud.google.com/products/calculator#id=e77713f6-dc0b-4bb3-bcef-cea904ac8efd) +> - **Estimated Costs:** [See cost table](index.md#cost-to-run) > - **Cloud Native Hybrid Alternative:** [Yes](#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) > - **Performance tested daily with the [GitLab Performance Tool](https://gitlab.com/gitlab-org/quality/performance)**: > - **Test requests per second (RPS) rates:** API: 200 RPS, Web: 20 RPS, Git (Pull): 20 RPS, Git (Push): 4 RPS @@ -274,11 +274,11 @@ for details. ### Load balancer terminates SSL with backend SSL -Configure your load balancer(s) to use the 'HTTP(S)' protocol rather than 'TCP'. -The load balancer(s) will be responsible for managing SSL certificates that +Configure your load balancers to use the 'HTTP(S)' protocol rather than 'TCP'. +The load balancers will be responsible for managing SSL certificates that end users will see. -Traffic will also be secure between the load balancer(s) and NGINX in this +Traffic will also be secure between the load balancers and NGINX in this scenario. There is no need to add configuration for proxied SSL since the connection will be secure all the way. However, configuration will need to be added to GitLab to configure SSL certificates. See @@ -2234,7 +2234,7 @@ future with further specific cloud provider details. | Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | 15.5 vCPU, 50 GB memory | | Supporting services such as NGINX, Prometheus | 2 | 4 vCPU, 15 GB memory | `n1-standard-4` | 7.75 vCPU, 25 GB memory | -- For this setup, we **recommend** and regularly [test](index.md#testing-process-and-results) +- For this setup, we **recommend** and regularly [test](index.md#validation-and-test-results) [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine) and [Amazon Elastic Kubernetes Service (EKS)](https://aws.amazon.com/eks/). Other Kubernetes services may also work, but your mileage may vary. - Nodes configuration is shown as it is forced to ensure pod vcpu / memory ratios and avoid scaling during **performance testing**. - In production deployments, there is no need to assign pods to nodes. A minimum of three nodes in three different availability zones is strongly recommended to align with resilient cloud architecture practices. diff --git a/doc/administration/reference_architectures/1k_users.md b/doc/administration/reference_architectures/1k_users.md index ed6fbe84a48..0d0e7681ffd 100644 --- a/doc/administration/reference_architectures/1k_users.md +++ b/doc/administration/reference_architectures/1k_users.md @@ -18,6 +18,7 @@ many organizations. > - **Supported users (approximate):** 1,000 > - **High Availability:** No. For a highly-available environment, you can > follow a modified [3K reference architecture](3k_users.md#supported-modifications-for-lower-user-counts-ha). +> - **Estimated Costs:** [See cost table](index.md#cost-to-run) > - **Cloud Native Hybrid:** No. For a cloud native hybrid environment, you > can follow a [modified hybrid reference architecture](#cloud-native-hybrid-reference-architecture-with-helm-charts). > - **Performance tested daily with the [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance)**: diff --git a/doc/administration/reference_architectures/25k_users.md b/doc/administration/reference_architectures/25k_users.md index 8cc355db951..86819024eeb 100644 --- a/doc/administration/reference_architectures/25k_users.md +++ b/doc/administration/reference_architectures/25k_users.md @@ -12,7 +12,7 @@ full list of reference architectures, see > - **Supported users (approximate):** 25,000 > - **High Availability:** Yes ([Praefect](#configure-praefect-postgresql) needs a third-party PostgreSQL solution for HA) -> - **Estimated Costs:** [GCP](https://cloud.google.com/products/calculator#id=925386e1-c01c-4c0a-8d7d-ebde1824b7b0) +> - **Estimated Costs:** [See cost table](index.md#cost-to-run) > - **Cloud Native Hybrid Alternative:** [Yes](#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) > - **Performance tested weekly with the [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance)**: > - **Test requests per second (RPS) rates:** API: 500 RPS, Web: 50 RPS, Git (Pull): 50 RPS, Git (Push): 10 RPS @@ -277,11 +277,11 @@ for details. ### Load balancer terminates SSL with backend SSL -Configure your load balancer(s) to use the 'HTTP(S)' protocol rather than 'TCP'. -The load balancer(s) will be responsible for managing SSL certificates that +Configure your load balancers to use the 'HTTP(S)' protocol rather than 'TCP'. +The load balancers will be responsible for managing SSL certificates that end users will see. -Traffic will also be secure between the load balancer(s) and NGINX in this +Traffic will also be secure between the load balancers and NGINX in this scenario. There is no need to add configuration for proxied SSL since the connection will be secure all the way. However, configuration will need to be added to GitLab to configure SSL certificates. See @@ -2232,7 +2232,7 @@ future with further specific cloud provider details. | Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | 15.5 vCPU, 50 GB memory | | Supporting services such as NGINX, Prometheus | 2 | 4 vCPU, 15 GB memory | `n1-standard-4` | 7.75 vCPU, 25 GB memory | -- For this setup, we **recommend** and regularly [test](index.md#testing-process-and-results) +- For this setup, we **recommend** and regularly [test](index.md#validation-and-test-results) [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine) and [Amazon Elastic Kubernetes Service (EKS)](https://aws.amazon.com/eks/). Other Kubernetes services may also work, but your mileage may vary. - Nodes configuration is shown as it is forced to ensure pod vcpu / memory ratios and avoid scaling during **performance testing**. - In production deployments, there is no need to assign pods to nodes. A minimum of three nodes in three different availability zones is strongly recommended to align with resilient cloud architecture practices. diff --git a/doc/administration/reference_architectures/2k_users.md b/doc/administration/reference_architectures/2k_users.md index 467c64b8279..f6c484b08b1 100644 --- a/doc/administration/reference_architectures/2k_users.md +++ b/doc/administration/reference_architectures/2k_users.md @@ -13,7 +13,7 @@ For a full list of reference architectures, see > - **Supported users (approximate):** 2,000 > - **High Availability:** No. For a highly-available environment, you can > follow a modified [3K reference architecture](3k_users.md#supported-modifications-for-lower-user-counts-ha). -> - **Estimated Costs:** [GCP](https://cloud.google.com/products/calculator#id=84d11491-d72a-493c-a16e-650931faa658) +> - **Estimated Costs:** [See cost table](index.md#cost-to-run) > - **Cloud Native Hybrid:** [Yes](#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) > - **Performance tested daily with the [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance)**: > - **Test requests per second (RPS) rates:** API: 40 RPS, Web: 4 RPS, Git (Pull): 4 RPS, Git (Push): 1 RPS @@ -1022,7 +1022,7 @@ future with further specific cloud provider details. | Sidekiq | 2 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | 3.9 vCPU, 11.8 GB memory | | Supporting services such as NGINX, Prometheus | 2 | 1 vCPU, 3.75 GB memory | `n1-standard-1` | 1.9 vCPU, 5.5 GB memory | -- For this setup, we **recommend** and regularly [test](index.md#testing-process-and-results) +- For this setup, we **recommend** and regularly [test](index.md#validation-and-test-results) [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine) and [Amazon Elastic Kubernetes Service (EKS)](https://aws.amazon.com/eks/). Other Kubernetes services may also work, but your mileage may vary. - Nodes configuration is shown as it is forced to ensure pod vcpu / memory ratios and avoid scaling during **performance testing**. - In production deployments, there is no need to assign pods to nodes. A minimum of three nodes in three different availability zones is strongly recommended to align with resilient cloud architecture practices. diff --git a/doc/administration/reference_architectures/3k_users.md b/doc/administration/reference_architectures/3k_users.md index 01d9987739b..587303a1f8f 100644 --- a/doc/administration/reference_architectures/3k_users.md +++ b/doc/administration/reference_architectures/3k_users.md @@ -22,7 +22,7 @@ For a full list of reference architectures, see > - **Supported users (approximate):** 3,000 > - **High Availability:** Yes, although [Praefect](#configure-praefect-postgresql) needs a third-party PostgreSQL solution -> - **Estimated Costs:** [GCP](https://cloud.google.com/products/calculator/#id=ac4838e6-9c40-4a36-ac43-6d1bc1843e08) +> - **Estimated Costs:** [See cost table](index.md#cost-to-run) > - **Cloud Native Hybrid Alternative:** [Yes](#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) > - **Performance tested weekly with the [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance)**: > - **Test requests per second (RPS) rates:** API: 60 RPS, Web: 6 RPS, Git (Pull): 6 RPS, Git (Push): 1 RPS @@ -278,11 +278,11 @@ for details. ### Load balancer terminates SSL with backend SSL -Configure your load balancer(s) to use the 'HTTP(S)' protocol rather than 'TCP'. -The load balancer(s) will be responsible for managing SSL certificates that +Configure your load balancers to use the 'HTTP(S)' protocol rather than 'TCP'. +The load balancers will be responsible for managing SSL certificates that end users will see. -Traffic will also be secure between the load balancer(s) and NGINX in this +Traffic will also be secure between the load balancers and NGINX in this scenario. There is no need to add configuration for proxied SSL since the connection will be secure all the way. However, configuration will need to be added to GitLab to configure SSL certificates. See @@ -2191,7 +2191,7 @@ future with further specific cloud provider details. | Sidekiq | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | 11.8 vCPU, 38.9 GB memory | | Supporting services such as NGINX, Prometheus | 2 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | 3.9 vCPU, 11.8 GB memory | -- For this setup, we **recommend** and regularly [test](index.md#testing-process-and-results) +- For this setup, we **recommend** and regularly [test](index.md#validation-and-test-results) [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine) and [Amazon Elastic Kubernetes Service (EKS)](https://aws.amazon.com/eks/). Other Kubernetes services may also work, but your mileage may vary. - Nodes configuration is shown as it is forced to ensure pod vcpu / memory ratios and avoid scaling during **performance testing**. - In production deployments, there is no need to assign pods to nodes. A minimum of three nodes in three different availability zones is strongly recommended to align with resilient cloud architecture practices. diff --git a/doc/administration/reference_architectures/50k_users.md b/doc/administration/reference_architectures/50k_users.md index d5bb9c4ad64..f4bf232d548 100644 --- a/doc/administration/reference_architectures/50k_users.md +++ b/doc/administration/reference_architectures/50k_users.md @@ -12,7 +12,7 @@ full list of reference architectures, see > - **Supported users (approximate):** 50,000 > - **High Availability:** Yes ([Praefect](#configure-praefect-postgresql) needs a third-party PostgreSQL solution for HA) -> - **Estimated Costs:** [GCP](https://cloud.google.com/products/calculator/#id=8006396b-88ee-40cd-a1c8-77cdefa4d3c8) +> - **Estimated Costs:** [See cost table](index.md#cost-to-run) > - **Cloud Native Hybrid Alternative:** [Yes](#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) > - **Performance tested weekly with the [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance)**: > - **Test requests per second (RPS) rates:** API: 1000 RPS, Web: 100 RPS, Git (Pull): 100 RPS, Git (Push): 20 RPS @@ -283,11 +283,11 @@ for details. ### Load balancer terminates SSL with backend SSL -Configure your load balancer(s) to use the 'HTTP(S)' protocol rather than 'TCP'. -The load balancer(s) will be responsible for managing SSL certificates that +Configure your load balancers to use the 'HTTP(S)' protocol rather than 'TCP'. +The load balancers will be responsible for managing SSL certificates that end users will see. -Traffic will also be secure between the load balancer(s) and NGINX in this +Traffic will also be secure between the load balancers and NGINX in this scenario. There is no need to add configuration for proxied SSL since the connection will be secure all the way. However, configuration will need to be added to GitLab to configure SSL certificates. See @@ -2248,7 +2248,7 @@ future with further specific cloud provider details. | Sidekiq | 4 | 4 vCPU, 15 GB memory | `n1-standard-4` | 15.5 vCPU, 50 GB memory | | Supporting services such as NGINX, Prometheus | 2 | 4 vCPU, 15 GB memory | `n1-standard-4` | 7.75 vCPU, 25 GB memory | -- For this setup, we **recommend** and regularly [test](index.md#testing-process-and-results) +- For this setup, we **recommend** and regularly [test](index.md#validation-and-test-results) [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine) and [Amazon Elastic Kubernetes Service (EKS)](https://aws.amazon.com/eks/). Other Kubernetes services may also work, but your mileage may vary. - Nodes configuration is shown as it is forced to ensure pod vcpu / memory ratios and avoid scaling during **performance testing**. - In production deployments, there is no need to assign pods to nodes. A minimum of three nodes in three different availability zones is strongly recommended to align with resilient cloud architecture practices. diff --git a/doc/administration/reference_architectures/5k_users.md b/doc/administration/reference_architectures/5k_users.md index 33ca4e4899f..4014ec04904 100644 --- a/doc/administration/reference_architectures/5k_users.md +++ b/doc/administration/reference_architectures/5k_users.md @@ -19,7 +19,7 @@ costly-to-operate environment by using the > - **Supported users (approximate):** 5,000 > - **High Availability:** Yes ([Praefect](#configure-praefect-postgresql) needs a third-party PostgreSQL solution for HA) -> - **Estimated Costs:** [GCP](https://cloud.google.com/products/calculator/#id=8742e8ea-c08f-4e0a-b058-02f3a1c38a2f) +> - **Estimated Costs:** [See cost table](index.md#cost-to-run) > - **Cloud Native Hybrid Alternative:** [Yes](#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) > - **Performance tested weekly with the [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance)**: > - **Test requests per second (RPS) rates:** API: 100 RPS, Web: 10 RPS, Git (Pull): 10 RPS, Git (Push): 2 RPS @@ -276,11 +276,11 @@ for details. ### Load balancer terminates SSL with backend SSL -Configure your load balancer(s) to use the 'HTTP(S)' protocol rather than 'TCP'. -The load balancer(s) will be responsible for managing SSL certificates that +Configure your load balancers to use the 'HTTP(S)' protocol rather than 'TCP'. +The load balancers will be responsible for managing SSL certificates that end users will see. -Traffic will also be secure between the load balancer(s) and NGINX in this +Traffic will also be secure between the load balancers and NGINX in this scenario. There is no need to add configuration for proxied SSL since the connection will be secure all the way. However, configuration will need to be added to GitLab to configure SSL certificates. See @@ -2167,7 +2167,7 @@ future with further specific cloud provider details. | Sidekiq | 3 | 4 vCPU, 15 GB memory | `n1-standard-4` | 11.8 vCPU, 38.9 GB memory | | Supporting services such as NGINX, Prometheus | 2 | 2 vCPU, 7.5 GB memory | `n1-standard-2` | 3.9 vCPU, 11.8 GB memory | -- For this setup, we **recommend** and regularly [test](index.md#testing-process-and-results) +- For this setup, we **recommend** and regularly [test](index.md#validation-and-test-results) [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine) and [Amazon Elastic Kubernetes Service (EKS)](https://aws.amazon.com/eks/). Other Kubernetes services may also work, but your mileage may vary. - Nodes configuration is shown as it is forced to ensure pod vcpu / memory ratios and avoid scaling during **performance testing**. - In production deployments, there is no need to assign pods to nodes. A minimum of three nodes in three different availability zones is strongly recommended to align with resilient cloud architecture practices. diff --git a/doc/administration/reference_architectures/index.md b/doc/administration/reference_architectures/index.md index bd796600564..815155866e8 100644 --- a/doc/administration/reference_architectures/index.md +++ b/doc/administration/reference_architectures/index.md @@ -59,7 +59,9 @@ provided architecture. ## Available reference architectures -The following reference architectures are available: +The following reference architectures are available. + +### GitLab package (Omnibus) - [Up to 1,000 users](1k_users.md) - [Up to 2,000 users](2k_users.md) @@ -69,6 +71,8 @@ The following reference architectures are available: - [Up to 25,000 users](25k_users.md) - [Up to 50,000 users](50k_users.md) +### Cloud native hybrid + The following Cloud Native Hybrid reference architectures, where select recommended components can be run in Kubernetes, are available: - [Up to 2,000 users](2k_users.md#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) @@ -83,6 +87,204 @@ to get assistance from Support with troubleshooting the [2,000 users](2k_users.m and higher reference architectures. [Read more about our definition of scaled architectures](https://about.gitlab.com/support/#definition-of-scaled-architecture). +### Validation and test results + +The [Quality Engineering - Enablement team](https://about.gitlab.com/handbook/engineering/quality/quality-engineering/) does regular smoke and performance tests for the reference architectures to ensure they remain compliant. + +- Testing occurs against all reference architectures and cloud providers in an automated and ad-hoc fashion. This is done by two tools: + - The [GitLab Environment Toolkit](https://gitlab.com/gitlab-org/gitlab-environment-toolkit) for building the environments. + - The [GitLab Performance Tool](https://gitlab.com/gitlab-org/quality/performance) for performance testing. +- Network latency on the test environments between components on all Cloud Providers were measured at <5ms. Note that this is shared as an observation and not as an implicit recommendation. +- We aim to have a "test smart" approach where architectures tested have a good range that can also apply to others. Testing focuses on 10k Omnibus on GCP as the testing has shown this is a good bellwether for the other architectures and cloud providers as well as Cloud Native Hybrids. +- Testing is done publicly and all results are shared. +- For more information about performance testing at GitLab, read [how our QA team leverages GitLab’s performance testing tool (and you can too)](https://about.gitlab.com/blog/2020/02/18/how-were-building-up-performance-testing-of-gitlab/). + +The following table details the testing done against the reference architectures along with the frequency and results. Additional testing is continuously evaluated, and the table is updated accordingly. + +<style> +table.test-coverage td { + border-left: 1px solid #dbdbdb; + border-right: 1px solid #dbdbdb; + border-bottom: 1px solid #dbdbdb; +} + +table.test-coverage th { + border-left: 1px solid #dbdbdb; + border-right: 1px solid #dbdbdb; + border-bottom: 1px solid #dbdbdb; +} +</style> + +<table class="test-coverage"> + <col> + <colgroup span="2"></colgroup> + <colgroup span="2"></colgroup> + <tr> + <th rowspan="2">Reference<br/>Architecture</th> + <th style="text-align: center" colspan="2" scope="colgroup">GCP (* also proxy for Bare-Metal)</th> + <th style="text-align: center" colspan="2" scope="colgroup">AWS</th> + <th style="text-align: center" colspan="2" scope="colgroup">Azure</th> + </tr> + <tr> + <th scope="col">Omnibus</th> + <th scope="col">Cloud Native Hybrid</th> + <th scope="col">Omnibus</th> + <th scope="col">Cloud Native Hybrid</th> + <th scope="col">Omnibus</th> + <th scope="col">Cloud Native Hybrid</th> + </tr> + <tr> + <th scope="row">1k</th> + <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/1k">Daily</a> (to be moved to Weekly)</td> + <td></td> + <td></td> + <td></td> + <td></td> + <td></td> + </tr> + <tr> + <th scope="row">2k</th> + <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/2k">Daily</a> (to be moved to Weekly)</td> + <td></td> + <td></td> + <td></td> + <td></td> + <td></td> + </tr> + <tr> + <th scope="row">3k</th> + <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/3k">Weekly</a></td> + <td></td> + <td></td> + <td></td> + <td></td> + <td></td> + </tr> + <tr> + <th scope="row">5k</th> + <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/5k">Weekly</a></td> + <td></td> + <td></td> + <td></td> + <td></td> + <td></td> + </tr> + <tr> + <th scope="row">10k</th> + <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k">Daily</a></td> + <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k-Cloud-Native-Hybrid">Ad-Hoc</a></td> + <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k">Ad-Hoc (inc Cloud Services)</a></td> + <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k-Cloud-Native-Hybrid">Ad-Hoc</a></td> + <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k">Ad-Hoc</a></td> + <td></td> + </tr> + <tr> + <th scope="row">25k</th> + <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/25k">Weekly</a></td> + <td></td> + <td></td> + <td></td> + <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/25k">Ad-Hoc</a></td> + <td></td> + </tr> + <tr> + <th scope="row">50k</th> + <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/50k">Weekly</a></td> + <td></td> + <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/50k">Ad-Hoc (inc Cloud Services)</a></td> + <td></td> + <td></td> + <td></td> + </tr> +</table> + +The Standard Reference Architectures are designed to be platform agnostic, with everything being run on VMs via [Omnibus GitLab](https://docs.gitlab.com/omnibus/). While testing occurs primarily on GCP, ad-hoc testing has shown that they perform similarly on equivalently specced hardware on other Cloud Providers or if run on premises (bare-metal). + +### Cost to run + +<table class="test-coverage"> + <col> + <colgroup span="2"></colgroup> + <colgroup span="2"></colgroup> + <tr> + <th rowspan="2">Reference<br/>Architecture</th> + <th style="text-align: center" colspan="2" scope="colgroup">GCP</th> + <th style="text-align: center" colspan="2" scope="colgroup">AWS</th> + <th style="text-align: center" colspan="2" scope="colgroup">Azure</th> + </tr> + <tr> + <th scope="col">Omnibus</th> + <th scope="col">Cloud Native Hybrid</th> + <th scope="col">Omnibus</th> + <th scope="col">Cloud Native Hybrid</th> + <th scope="col">Omnibus</th> + <th scope="col">Cloud Native Hybrid</th> + </tr> + <tr> + <th scope="row">1k</th> + <td><a href="https://cloud.google.com/products/calculator#id=a6d6a94a-c7dc-4c22-85c4-7c5747f272ed">Calculated cost</a></td> + <td></td> + <td></td> + <td></td> + <td></td> + <td></td> + </tr> + <tr> + <th scope="row">2k</th> + <td><a href="https://cloud.google.com/products/calculator#id=84d11491-d72a-493c-a16e-650931faa658">Calculated cost</a></td> + <td></td> + <td></td> + <td></td> + <td></td> + <td></td> + </tr> + <tr> + <th scope="row">3k</th> + <td><a href="https://cloud.google.com/products/calculator/#id=ac4838e6-9c40-4a36-ac43-6d1bc1843e08">Calculated cost</a></td> + <td></td> + <td></td> + <td></td> + <td></td> + <td></td> + </tr> + <tr> + <th scope="row">5k</th> + <td><a href="https://cloud.google.com/products/calculator/#id=8742e8ea-c08f-4e0a-b058-02f3a1c38a2f">Calculated cost</a></td> + <td></td> + <td></td> + <td></td> + <td></td> + <td></td> + </tr> + <tr> + <th scope="row">10k</th> + <td><a href="https://cloud.google.com/products/calculator#id=e77713f6-dc0b-4bb3-bcef-cea904ac8efd">Calculated cost</a></td> + <td></td> + <td></td> + <td></td> + <td></td> + <td></td> + </tr> + <tr> + <th scope="row">25k</th> + <td><a href="https://cloud.google.com/products/calculator#id=925386e1-c01c-4c0a-8d7d-ebde1824b7b0">Calculated cost</a></td> + <td></td> + <td></td> + <td></td> + <td></td> + <td></td> + </tr> + <tr> + <th scope="row">50k</th> + <td><a href="https://cloud.google.com/products/calculator/#id=8006396b-88ee-40cd-a1c8-77cdefa4d3c8">Calculated cost</a></td> + <td></td> + <td></td> + <td></td> + <td></td> + <td></td> + </tr> +</table> + ## Availability Components GitLab comes with the following components for your use, listed from least to @@ -191,33 +393,3 @@ The reference architectures for user counts [3,000](3k_users.md) and up support In the specific case you have the requirement to achieve HA but have a lower user count, select modifications to the [3,000 user](3k_users.md) architecture are supported. For more details, [refer to this section in the architecture's documentation](3k_users.md#supported-modifications-for-lower-user-counts-ha). - -## Testing process and results - -The [Quality Engineering - Enablement team](https://about.gitlab.com/handbook/engineering/quality/quality-engineering/) does regular smoke and performance tests for the reference architectures to ensure they remain compliant. - -In this section, we detail some of the process as well as the results. - -Note the following about the testing process: - -- Testing occurs against all main reference architectures and cloud providers in an automated and ad-hoc fashion. - This is achieved through two tools built by the team: - - The [GitLab Environment Toolkit](https://gitlab.com/gitlab-org/quality/gitlab-environment-toolkit) for building the environments. - - The [GitLab Performance Tool](https://gitlab.com/gitlab-org/quality/performance) for performance testing. -- Network latency on the test environments between components on all Cloud Providers were measured at <5ms. Note that this is shared as an observation and not as an implicit recommendation. -- We aim to have a "test smart" approach where architectures tested have a good range that can also apply to others. Testing focuses on 10k Omnibus on GCP as the testing has shown this is a good bellwether for the other architectures and cloud providers as well as Cloud Native Hybrids. -- Testing is done publicly and all results are shared. - -Τhe following table details the testing done against the reference architectures along with the frequency and results. Note that this list above is non exhaustive. Additional testing is continuously evaluated and iterated on, and the table is updated accordingly. - -| Reference<br/>Architecture<br/>Size | Bare-Metal | GCP | AWS | Azure | -|-----------------------------|------------|-----|-----|-------| -| 1k | <i>Refer to GCP<sup>1</sup></i> | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/1k)<sup>1</sup> | - | - | -| 2k | <i>Refer to GCP<sup>1</sup></i> | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/2k)<sup>1</sup> | - | - | -| 3k | <i>Refer to GCP<sup>1</sup></i> | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/3k)<sup>1</sup> | - | - | -| 5k | <i>Refer to GCP<sup>1</sup></i> | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/5k)<sup>1</sup> | - | - | -| 10k | <i>Refer to GCP<sup>1</sup></i> | [Standard - Daily](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k)<sup>1</sup> <br/> [Standard (inc Cloud Services) - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k) <br/> [Cloud Native Hybrid - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k-Cloud-Native-Hybrid) | [Standard (inc Cloud Services) - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k) <br/> [Cloud Native Hybrid - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k-Cloud-Native-Hybrid) | [Standard - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k) | -| 25k | <i>Refer to GCP<sup>1</sup></i> | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/25k)<sup>1</sup> | - | [Standard - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/25k) | -| 50k | <i>Refer to GCP<sup>1</sup></i> | [Standard - Weekly](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/50k)<sup>1</sup> | [Standard (inc Cloud Services) - Ad-Hoc](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/50k) | - | - -1. The Standard Reference Architectures are designed to be platform agnostic, with everything being run on VMs via [Omnibus GitLab](https://docs.gitlab.com/omnibus/). While testing occurs primarily on GCP, ad-hoc testing has shown that they perform similarly on equivalently specced hardware on other Cloud Providers or if run on premises (bare-metal). diff --git a/doc/administration/restart_gitlab.md b/doc/administration/restart_gitlab.md index 9579d413bf8..88909b2b9d8 100644 --- a/doc/administration/restart_gitlab.md +++ b/doc/administration/restart_gitlab.md @@ -7,7 +7,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # How to restart GitLab **(FREE SELF)** Depending on how you installed GitLab, there are different methods to restart -its service(s). +its services. ## Omnibus installations diff --git a/doc/administration/server_hooks.md b/doc/administration/server_hooks.md index 2a431d17774..9d8bc03e5e9 100644 --- a/doc/administration/server_hooks.md +++ b/doc/administration/server_hooks.md @@ -10,171 +10,135 @@ disqus_identifier: 'https://docs.gitlab.com/ee/administration/custom_hooks.html' > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/196051) in GitLab 12.8 replacing Custom Hooks. -Git supports hooks that are executed on different actions. These hooks run on the server and can be -used to enforce specific commit policies or perform other tasks based on the state of the -repository. +Server hooks run custom logic on the GitLab server. Users can use them to run Git-related tasks such as: -Git supports the following hooks: +- Enforcing specific commit policies. +- Performing tasks based on the state of the repository. -- `pre-receive` -- `post-receive` -- `update` +Server hooks use `pre-receive`, `post-receive`, and `update` +[Git server-side hooks](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks#_server_side_hooks). -See [the Git documentation](https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks#_server_side_hooks) -for more information about each hook type. +GitLab administrators configure server hooks on the file system of the GitLab server. If you don't have file system access, +alternatives to server hooks include: -Server-side Git hooks can be configured for: +- [Webhooks](../user/project/integrations/webhooks.md). +- [GitLab CI/CD](../ci/index.md). +- [Push rules](../push_rules/push_rules.md), for a user-configurable Git hook interface. -- [A single repository](#create-a-server-hook-for-a-repository). -- [All repositories](#create-a-global-server-hook-for-all-repositories). +[Geo](geo/index.md) doesn't replicate server hooks to secondary nodes. -Note the following about server hooks: +## Create a server hook for a single repository -- Server hooks must be configured on the file system of the GitLab server. Only GitLab server - administrators are able to complete these tasks. If you don't have file system access, see - possible alternatives such as: - - [Webhooks](../user/project/integrations/webhooks.md). - - [GitLab CI/CD](../ci/index.md). - - [Push Rules](../push_rules/push_rules.md), for a user-configurable Git hook - interface. -- Server hooks aren't replicated to [Geo](geo/index.md) secondary nodes. +To create a server hook for a single repository: -## Create a server hook for a repository +1. On the top bar, select **Menu > Admin**. +1. Go to **Overview > Projects** and select the project you want to add a server hook to. +1. On the page that appears, locate the value of **Gitaly relative path**. This path is where server hooks must be located. + - If you are using [hashed storage](repository_storage_types.md#hashed-storage), see + [Translate hashed storage paths](repository_storage_types.md#translate-hashed-storage-paths) for information on + interpreting the relative path. + - If you are not using [hashed storage](repository_storage_types.md#hashed-storage): + - For Omnibus GitLab installations, the path is usually `/var/opt/gitlab/git-data/repositories/<group>/<project>.git`. + - For an installation from source, the path is usually `/home/git/repositories/<group>/<project>.git`. +1. On the file system, create a new directory in the correct location called `custom_hooks`. +1. In the new `custom_hooks` directory, create a file with a name that matches the hook type. For example, for a + `pre-receive` server hook, the filename should be `pre-receive` with no extension. +1. Make the server hook file executable and ensure that it's owned by the Git user. +1. Write the code to make the server hook function as expected. Server hooks can be in any programming language. Ensure + the [shebang](https://en.wikipedia.org/wiki/Shebang_(Unix)) at the top reflects the language type. For + example, if the script is in Ruby the shebang is probably `#!/usr/bin/env ruby`. -If you are not using [hashed storage](repository_storage_types.md#hashed-storage), the project's -repository directory might not exactly match the instructions below. In that case: - -- For an installation from source, the path is usually - `/home/git/repositories/<group>/<project>.git`. -- For Omnibus GitLab installs, the path is usually - `/var/opt/gitlab/git-data/repositories/<group>/<project>.git`. - -Follow the steps below to set up a server-side hook for a repository: - -1. Go to **Admin area > Projects** and select the project you want to add a server hook to. -1. Locate the **Gitaly relative path** on the page that appears. This is where the server hook - must be implemented. For information on interpreting the relative path, see - [Translate hashed storage paths](repository_storage_types.md#translate-hashed-storage-paths). -1. On the file system, create a new directory in this location called `custom_hooks`. -1. Inside the new `custom_hooks` directory, create a file with a name matching the hook type. For - example, for a pre-receive hook the filename should be `pre-receive` with no extension. -1. Make the hook file executable and ensure that it's owned by the Git user. -1. Write the code to make the server hook function as expected. Hooks can be in any language. Ensure - the ["shebang"](https://en.wikipedia.org/wiki/Shebang_(Unix)) at the top properly reflects the - language type. For example, if the script is in Ruby the shebang is probably - `#!/usr/bin/env ruby`. - -Assuming the hook code is properly implemented, the hook code is executed as appropriate. +If the server hook code is properly implemented, it should execute when the Git hook is next triggered. ## Create a global server hook for all repositories -To create a Git hook that applies to all of the repositories in your instance, set a global server -hook. The default global server hook directory is in the GitLab Shell directory. Any -hook added there applies to all repositories, including: +To create a Git hook that applies to all repositories, set a global server hook. The default global server hook directory +is in the GitLab Shell directory. Any server hook added there applies to all repositories, including: + +- [Project and group wiki](../user/project/wiki/index.md) repositories. Their storage directory names are in the format + `<id>.wiki.git`. +- [Design management](../user/project/issues/design_management.md) repositories under a project. Their storage directory + names are in the format `<id>.design.git`. -- [Project and group wiki](../user/project/wiki/index.md) repositories, - whose storage directory names are in the format `<id>.wiki.git`. -- [Design management](../user/project/issues/design_management.md) repositories under a - project, whose storage directory names are in the format `<id>.design.git`. +### Choose a server hook directory -The default directory: +Before creating a global server hook, you must choose a directory for it. The default global server hook directory: +- For Omnibus GitLab installations is usually `/opt/gitlab/embedded/service/gitlab-shell/hooks`. - For an installation from source is usually `/home/git/gitlab-shell/hooks`. -- For Omnibus GitLab installs is usually `/opt/gitlab/embedded/service/gitlab-shell/hooks`. -To use a different directory for global server hooks, set `custom_hooks_dir` in Gitaly -configuration: +To use a different directory for global server hooks, set `custom_hooks_dir` in Gitaly configuration: -- For Omnibus installations, this is set in `gitlab.rb`. +- For Omnibus installations, set in `gitlab.rb`. - For source installations, the configuration location depends on the GitLab version. For: - - GitLab 13.0 and earlier, this is set in `gitlab-shell/config.yml`. - - GitLab 13.1 and later, this is set in `gitaly/config.toml` under the `[hooks]` section. - -NOTE: -The `custom_hooks_dir` value in `gitlab-shell/config.yml` is still honored in GitLab 13.1 and later -if the value in `gitaly/config.toml` is blank or non-existent. + - GitLab 13.0 and earlier, set in `gitlab-shell/config.yml`. + - GitLab 13.1 and later, set in `gitaly/config.toml` under the `[hooks]` section. However, GitLab honors the + `custom_hooks_dir` value in `gitlab-shell/config.yml` if the value in `gitaly/config.toml` is blank or non-existent. -Follow the steps below to set up a global server hook for all repositories: +### Create the global server hook -1. On the GitLab server, navigate to the configured global server hook directory. -1. Create a new directory in this location. Depending on the type of hook, it can be either a - `pre-receive.d`, `post-receive.d`, or `update.d` directory. -1. Inside this new directory, add your hook. Hooks can be in any language. Ensure the - ["shebang"](https://en.wikipedia.org/wiki/Shebang_(Unix)) at the top properly reflects the - language type. For example, if the script is in Ruby the shebang is probably - `#!/usr/bin/env ruby`. -1. Make the hook file executable and ensure that it's owned by the Git user. +To create a global server hook for all repositories: -Now test the hook to check whether it is functioning properly. +1. On the GitLab server, go to the configured global server hook directory. +1. Create a new directory in this location called `pre-receive.d`, `post-receive.d`, or `update.d`, depending on the type + of server hook. Any other names are ignored. +1. Inside this new directory, add your server hook. Server hooks can be in any programming language. Ensure the + [shebang](https://en.wikipedia.org/wiki/Shebang_(Unix)) at the top reflects the language type. For example, if the + script is in Ruby the shebang is probably `#!/usr/bin/env ruby`. +1. Make the hook file executable, ensure that it's owned by the Git user, and ensure it does not match the backup file + pattern (`*~`). -## Chained hooks +If the server hook code is properly implemented, it should execute when the Git hook is next triggered. -Server hooks set [per project](#create-a-server-hook-for-a-repository) or -[globally](#create-a-global-server-hook-for-all-repositories) can be executed in a chain. +## Chained server hooks -Server hooks are searched for and executed in the following order of priority: +GitLab can execute server hooks in a chain. GitLab searches for and executes server hooks in the following order: -- Built-in GitLab server hooks. These are not user-customizable. -- `<project>.git/custom_hooks/<hook_name>`: Per-project hooks. This was kept for backwards - compatibility. +- Built-in GitLab server hooks. These server hooks are not customizable by users. +- `<project>.git/custom_hooks/<hook_name>`: Per-project hooks. This location is kept for backwards compatibility. - `<project>.git/custom_hooks/<hook_name>.d/*`: Location for per-project hooks. -- `<custom_hooks_dir>/<hook_name>.d/*`: Location for all executable global hook files - except editor backup files. +- `<custom_hooks_dir>/<hook_name>.d/*`: Location for all executable global hook files except editor backup files. -Within a directory, server hooks: +Within a server hooks directory, hooks: - Are executed in alphabetical order. - Stop executing when a hook exits with a non-zero value. -`<hook_name>.d` must be either `pre-receive.d`, `post-receive.d`, or `update.d` to work properly. -Any other names are ignored. +## Environment variables available to server hooks -Files in `.d` directories must be executable and not match the backup file pattern (`*~`). +You can pass any environment variable to server hooks, but you should only rely on supported environment variables. -For `<project>.git` you need to [translate](repository_storage_types.md#translate-hashed-storage-paths) -your project name into the hashed storage format that GitLab uses. +The following GitLab environment variables are supported for all server hooks: -## Environment Variables +| Environment variable | Description | +|:---------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------| +| `GL_ID` | GitLab identifier of user that initiated the push. For example, `user-2234`. | +| `GL_PROJECT_PATH` | (GitLab 13.2 and later) GitLab project path. | +| `GL_PROTOCOL` | (GitLab 13.2 and later) Protocol used for this change. One of: `http` (Git `push` using HTTP), `ssh` (Git `push` using SSH), or `web` (all other actions). | +| `GL_REPOSITORY` | `project-<id>` where `id` is the ID of the project. | +| `GL_USERNAME` | GitLab username of the user that initiated the push. | -The following set of environment variables are available to server hooks. - -| Environment variable | Description | -|:---------------------|:----------------------------------------------------------------------------| -| `GL_ID` | GitLab identifier of user that initiated the push. For example, `user-2234` | -| `GL_PROJECT_PATH` | (GitLab 13.2 and later) GitLab project path | -| `GL_PROTOCOL` | (GitLab 13.2 and later) Protocol used for this change. One of: `http` (Git Push using HTTP), `ssh` (Git Push using SSH), or `web` (all other actions). | -| `GL_REPOSITORY` | `project-<id>` where `id` is the ID of the project | -| `GL_USERNAME` | GitLab username of the user that initiated the push | - -Pre-receive and post-receive server hooks can also access the following Git environment variables. +The following Git environment variables are supported for `pre-receive` and `post-receive` server hooks: | Environment variable | Description | |:-----------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `GIT_ALTERNATE_OBJECT_DIRECTORIES` | Alternate object directories in the quarantine environment. See [Git `receive-pack` documentation](https://git-scm.com/docs/git-receive-pack#_quarantine_environment). | | `GIT_OBJECT_DIRECTORY` | GitLab project path in the quarantine environment. See [Git `receive-pack` documentation](https://git-scm.com/docs/git-receive-pack#_quarantine_environment). | -| `GIT_PUSH_OPTION_COUNT` | Number of push options. See [Git `pre-receive` documentation](https://git-scm.com/docs/githooks#pre-receive). | -| `GIT_PUSH_OPTION_<i>` | Value of push options where `i` is from `0` to `GIT_PUSH_OPTION_COUNT - 1`. See [Git `pre-receive` documentation](https://git-scm.com/docs/githooks#pre-receive). | - -NOTE: -While other environment variables can be passed to server hooks, your application should not rely on -them as they can change. +| `GIT_PUSH_OPTION_COUNT` | Number of [push options](../user/project/push_options.md). See [Git `pre-receive` documentation](https://git-scm.com/docs/githooks#pre-receive). | +| `GIT_PUSH_OPTION_<i>` | Value of [push options](../user/project/push_options.md) where `i` is from `0` to `GIT_PUSH_OPTION_COUNT - 1`. See [Git `pre-receive` documentation](https://git-scm.com/docs/githooks#pre-receive). | ## Custom error messages -To have custom error messages appear in the GitLab UI when a commit is declined or an error occurs -during the Git hook, your script should: +You can have custom error messages appear in the GitLab UI when a commit is declined or an error occurs during the Git +hook. To display a custom error message, your script must: - Send the custom error messages to either the script's `stdout` or `stderr`. - Prefix each message with `GL-HOOK-ERR:` with no characters appearing before the prefix. -### Example custom error message - -This hook script written in Bash generates the following message in the GitLab UI: +For example: ```shell #!/bin/sh echo "GL-HOOK-ERR: My custom error message."; exit 1 ``` - - diff --git a/doc/administration/sidekiq.md b/doc/administration/sidekiq.md index 989a024d6ab..fea6ac9e554 100644 --- a/doc/administration/sidekiq.md +++ b/doc/administration/sidekiq.md @@ -4,77 +4,131 @@ group: Distribution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Configuring Sidekiq **(FREE SELF)** +# Configure an external Sidekiq instance **(FREE SELF)** -This section discusses how to configure an external Sidekiq instance using the -bundled Sidekiq in the GitLab package. +You can configure an external Sidekiq instance by using the Sidekiq that's +bundled in the GitLab package. Sidekiq requires connection to the Redis, +PostgreSQL, and Gitaly instances. -Sidekiq requires connection to the Redis, PostgreSQL and Gitaly instance. -To configure the Sidekiq node: +## Required configuration + +To configure Sidekiq: 1. SSH into the Sidekiq server. -1. [Download/install](https://about.gitlab.com/install/) the Omnibus GitLab package -you want using steps 1 and 2 from the GitLab downloads page. -**Do not complete any other steps on the download page.** -1. Open `/etc/gitlab/gitlab.rb` with your editor. -1. Generate the Sidekiq configuration: +1. [Download and install](https://about.gitlab.com/install/) the Omnibus GitLab package + using steps 1 and 2. **Do not complete any other steps.** +1. Edit `/etc/gitlab/gitlab.rb` with the following information and make sure + to replace with your values: ```ruby - ## Optional: Enable extra Sidekiq processes - sidekiq_cluster['enable'] = true - sidekiq['queue_groups'] = [ - "elastic_commit_indexer", - "*" - ] - ``` + ## + ## To maintain uniformity of links across nodes, the + ##`external_url` on the Sidekiq server should point to the external URL that users + ## use to access GitLab. This can be either: + ## + ## - The `external_url` set on your application server. + ## - The URL of a external load balancer, which routes traffic to the GitLab application server. + ## -1. Setup Sidekiq's connection to Redis: + external_url 'https://gitlab.example.com' + + ## Prevent database migrations from running on upgrade automatically + gitlab_rails['auto_migrate'] = false + + ######################################## + ##### Services Disabled ### + ######################################## + # + # When running GitLab on just one server, you have a single `gitlab.rb` + # to enable all services you want to run. + # When running GitLab on N servers, you have N `gitlab.rb` files. + # Enable only the services you want to run on each + # specific server, while disabling all others. + # + nginx['enable'] = false + grafana['enable'] = false + prometheus['enable'] = false + gitlab_rails['auto_migrate'] = false + alertmanager['enable'] = false + gitaly['enable'] = false + gitlab_workhorse['enable'] = false + nginx['enable'] = false + postgres_exporter['enable'] = false + postgresql['enable'] = false + redis['enable'] = false + redis_exporter['enable'] = false + puma['enable'] = false + gitlab_exporter['enable'] = false + + ####################################### + ### Sidekiq configuration ### + ####################################### + sidekiq['enable'] = true + sidekiq['listen_address'] = "0.0.0.0" + + ## Set number of Sidekiq queue processes to the same number as available CPUs + sidekiq['queue_groups'] = ['*'] * 4 + + ## Set number of Sidekiq threads per queue process to the recommend number of 10 + sidekiq['max_concurrency'] = 10 + + ######################################## + #### Redis ### + ######################################## - ```ruby ## Must be the same in every sentinel node redis['master_name'] = 'gitlab-redis' ## The same password for Redis authentication you set up for the master node. - redis['master_password'] = 'YOUR_PASSOWORD' - - ## A list of sentinels with `host` and `port` - gitlab_rails['redis_sentinels'] = [ - {'host' => '10.10.1.34', 'port' => 26379}, - {'host' => '10.10.1.35', 'port' => 26379}, - {'host' => '10.10.1.36', 'port' => 26379}, - ] - ``` + redis['master_password'] = '<redis_master_password>' -1. Set up Sidekiq's connection to Gitaly: + ####################################### + ### Gitaly ### + ####################################### - ```ruby + ## Replace <gitaly_token> with the one you set up, see + ## https://docs.gitlab.com/ee/administration/gitaly/configure_gitaly.html#about-the-gitaly-token git_data_dirs({ 'default' => { 'gitaly_address' => 'tcp://gitaly:8075' }, }) - gitlab_rails['gitaly_token'] = 'YOUR_TOKEN' - ``` + gitlab_rails['gitaly_token'] = '<gitaly_token>' -1. Set up Sidekiq's connection to PostgreSQL: + ####################################### + ### Postgres ### + ####################################### - ```ruby - gitlab_rails['db_host'] = '10.10.1.30' - gitlab_rails['db_password'] = 'YOUR_PASSOWORD' + # Replace <database_host> and <database_password> + gitlab_rails['db_host'] = '<database_host>' + gitlab_rails['db_password'] = '<database_password>' gitlab_rails['db_port'] = '5432' gitlab_rails['db_adapter'] = 'postgresql' gitlab_rails['db_encoding'] = 'unicode' gitlab_rails['auto_migrate'] = false + + # Add the Sidekiq nodes to PostgreSQL's trusted addresses. + # In the following example, 10.10.1.30/32 is the private IP + # of the Sidekiq server. + postgresql['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 10.10.1.30/32) ``` - Remember to add the Sidekiq nodes to PostgreSQL's trusted addresses: +1. Reconfigure GitLab: - ```ruby - postgresql['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 10.10.1.30/32 10.10.1.31/32 10.10.1.32/32 10.10.1.33/32 10.10.1.38/32) + ```shell + sudo gitlab-ctl reconfigure ``` -1. If you run multiple Sidekiq nodes with a shared file storage, such as NFS, you must - specify the UIDs and GIDs to ensure they match between servers. Specifying the UIDs - and GIDs prevents permissions issues in the file system. This advice is similar to the - [advice for Geo setups](geo/replication/multiple_servers.md#step-4-configure-the-frontend-application-nodes-on-the-geo-secondary-site): +1. Restart the Sidekiq nodes after completing the process and finishing the database migrations. + +## Configure multiple Sidekiq nodes with shared storage + +If you run multiple Sidekiq nodes with a shared file storage, such as NFS, you must +specify the UIDs and GIDs to ensure they match between servers. Specifying the UIDs +and GIDs prevents permissions issues in the file system. This advice is similar to the +[advice for Geo setups](geo/replication/multiple_servers.md#step-4-configure-the-frontend-application-nodes-on-the-geo-secondary-site). + +To set up multiple Sidekiq nodes: + +1. Edit `/etc/gitlab/gitlab.rb`: ```ruby user['uid'] = 9000 @@ -85,180 +139,87 @@ you want using steps 1 and 2 from the GitLab downloads page. registry['gid'] = 9002 ``` -1. Disable other services: +1. Reconfigure GitLab: - ```ruby - nginx['enable'] = false - grafana['enable'] = false - prometheus['enable'] = false - gitlab_rails['auto_migrate'] = false - alertmanager['enable'] = false - gitaly['enable'] = false - gitlab_monitor['enable'] = false - gitlab_workhorse['enable'] = false - nginx['enable'] = false - postgres_exporter['enable'] = false - postgresql['enable'] = false - redis['enable'] = false - redis_exporter['enable'] = false - puma['enable'] = false - gitlab_exporter['enable'] = false + ```shell + sudo gitlab-ctl reconfigure ``` -1. If you're using the Container Registry and it's running on a different node than Sidekiq, then - configure the registry URL: +## Configure the Container Registry when using an external Sidekiq + +If you're using the Container Registry and it's running on a different +node than Sidekiq, follow the steps below. + +1. Edit `/etc/gitlab/gitlab.rb`, and configure the registry URL: ```ruby registry_external_url 'https://registry.example.com' gitlab_rails['registry_api_url'] = "https://registry.example.com" ``` - - You must also copy the `registry.key` file to each Sidekiq node. -1. Define the `external_url`. To maintain uniformity of links across nodes, the - `external_url` on the Sidekiq server should point to the external URL that users - will use to access GitLab. This will either be the `external_url` set on your - application server or the URL of a external load balancer which will route traffic - to the GitLab application server: +1. Reconfigure GitLab: - ```ruby - external_url 'https://gitlab.example.com' + ```shell + sudo gitlab-ctl reconfigure ``` -1. (Optional) If you want to collect Sidekiq metrics, enable the Sidekiq metrics server. - To make metrics available from `localhost:8082/metrics`, set the following values: +1. In the instance where Container Registry is hosted, copy the `registry.key` + file to the Sidekiq node. + +## Configure the Sidekiq metrics server + +If you want to collect Sidekiq metrics, enable the Sidekiq metrics server. +To make metrics available from `localhost:8082/metrics`: + +To configure the metrics server: + +1. Edit `/etc/gitlab/gitlab.rb`: ```ruby sidekiq['metrics_enabled'] = true sidekiq['listen_address'] = "localhost" sidekiq['listen_port'] = "8082" - + # Optionally log all the metrics server logs to log/sidekiq_exporter.log sidekiq['exporter_log_enabled'] = true ``` -1. (Optional) If you use health check probes to observe Sidekiq, - set a separate port for health checks. - Configuring health checks is only necessary if there is something that actually probes them. - For more information about health checks, see the [Sidekiq health check page](sidekiq_health_check.md). - Enable health checks for Sidekiq: +1. Reconfigure GitLab: + + ```shell + sudo gitlab-ctl reconfigure + ``` + +## Configure health checks + +If you use health check probes to observe Sidekiq, +you can set a separate port for health checks. +Configuring health checks is only necessary if there is something that actually probes them. +For more information about health checks, see the [Sidekiq health check page](sidekiq_health_check.md). + +To enable health checks for Sidekiq: + +1. Edit `/etc/gitlab/gitlab.rb`: ```ruby - sidekiq['health_checks_enabled'] = true - sidekiq['health_checks_listen_address'] = "localhost" - sidekiq['health_checks_listen_port'] = "8092" + sidekiq['health_checks_enabled'] = true + sidekiq['health_checks_listen_address'] = "localhost" + sidekiq['health_checks_listen_port'] = "8092" ``` NOTE: - If health check settings are not set, they will default to the metrics exporter settings. + If health check settings are not set, they default to the metrics exporter settings. This default is deprecated and is set to be removed in [GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/347509). -1. Run `gitlab-ctl reconfigure`. - -You will need to restart the Sidekiq nodes after an update has occurred and database -migrations performed. - -## Example configuration - -Here's what the ending `/etc/gitlab/gitlab.rb` would look like: - -```ruby -######################################## -##### Services Disabled ### -######################################## - -nginx['enable'] = false -grafana['enable'] = false -prometheus['enable'] = false -gitlab_rails['auto_migrate'] = false -alertmanager['enable'] = false -gitaly['enable'] = false -gitlab_workhorse['enable'] = false -nginx['enable'] = false -postgres_exporter['enable'] = false -postgresql['enable'] = false -redis['enable'] = false -redis_exporter['enable'] = false -puma['enable'] = false -gitlab_exporter['enable'] = false - -######################################## -#### Redis ### -######################################## - -## Must be the same in every sentinel node -redis['master_name'] = 'gitlab-redis' - -## The same password for Redis authentication you set up for the master node. -redis['master_password'] = 'YOUR_PASSOWORD' - -## A list of sentinels with `host` and `port` -gitlab_rails['redis_sentinels'] = [ - {'host' => '10.10.1.34', 'port' => 26379}, - {'host' => '10.10.1.35', 'port' => 26379}, - {'host' => '10.10.1.36', 'port' => 26379}, - ] - -####################################### -### Gitaly ### -####################################### - -git_data_dirs({ - 'default' => { 'gitaly_address' => 'tcp://gitaly:8075' }, -}) -gitlab_rails['gitaly_token'] = 'YOUR_TOKEN' - -####################################### -### Postgres ### -####################################### -gitlab_rails['db_host'] = '10.10.1.30' -gitlab_rails['db_password'] = 'YOUR_PASSOWORD' -gitlab_rails['db_port'] = '5432' -gitlab_rails['db_adapter'] = 'postgresql' -gitlab_rails['db_encoding'] = 'unicode' -gitlab_rails['auto_migrate'] = false - -####################################### -### Sidekiq configuration ### -####################################### -sidekiq['metrics_enabled'] = true -sidekiq['exporter_log_enabled'] = false -sidekiq['listen_port'] = "8082" - -sidekiq['health_checks_enabled'] = true -sidekiq['health_checks_listen_address'] = "localhost" -sidekiq['health_checks_listen_port'] = "8092" - -####################################### -### Monitoring configuration ### -####################################### -consul['enable'] = true -consul['monitoring_service_discovery'] = true - -consul['configuration'] = { - bind_addr: '10.10.1.48', - retry_join: %w(10.10.1.34 10.10.1.35 10.10.1.36) -} - -# Set the network addresses that the exporters will listen on -node_exporter['listen_address'] = '10.10.1.48:9100' - -# Rails Status for prometheus -gitlab_rails['monitoring_whitelist'] = ['10.10.1.42', '127.0.0.1'] - -# Container Registry URL for cleanup jobs -registry_external_url 'https://registry.example.com' -gitlab_rails['registry_api_url'] = "https://registry.example.com" - -# External URL (this should match the URL used to access your GitLab instance) -external_url 'https://gitlab.example.com' -``` - -## Further reading - -Related Sidekiq configuration: - -1. [Extra Sidekiq processes](operations/extra_sidekiq_processes.md) -1. [Extra Sidekiq routing](operations/extra_sidekiq_routing.md) -1. [Using the GitLab-Sidekiq chart](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/) -1. [Sidekiq health checks](sidekiq_health_check.md) +1. Reconfigure GitLab: + + ```shell + sudo gitlab-ctl reconfigure + ``` + +## Related topics + +- [Extra Sidekiq processes](operations/extra_sidekiq_processes.md) +- [Extra Sidekiq routing](operations/extra_sidekiq_routing.md) +- [Using the GitLab-Sidekiq chart](https://docs.gitlab.com/charts/charts/gitlab/sidekiq/) +- [Sidekiq health checks](sidekiq_health_check.md) diff --git a/doc/administration/troubleshooting/diagnostics_tools.md b/doc/administration/troubleshooting/diagnostics_tools.md index 53d4810b920..d510df5976c 100644 --- a/doc/administration/troubleshooting/diagnostics_tools.md +++ b/doc/administration/troubleshooting/diagnostics_tools.md @@ -23,3 +23,7 @@ running on. [strace-parser](https://gitlab.com/wchandler/strace-parser) is a small tool to analyze and summarize raw `strace` data. + +## kubesos + +The [`kubesos`](https://gitlab.com/gitlab-com/support/toolbox/kubesos/) utiltity retrieves GitLab cluster configuration and logs from GitLab Cloud Native chart deployments. diff --git a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md index 33a81c12811..265c5278fd6 100644 --- a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md +++ b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md @@ -78,6 +78,43 @@ Notify.test_email(e, "Test email for #{n}", 'Test email').deliver_now Notify.test_email(u.email, "Test email for #{u.name}", 'Test email').deliver_now ``` +## Limiting output + +Adding a semicolon(`;`) and a follow-up statement at the end of a statement prevents the default implicit return output. This is useful if you are already explicitly printing details and potentially have a lot of return output: + +```ruby +puts ActiveRecord::Base.descendants; :ok +Project.select(&:pages_deployed?).each {|p| puts p.pages_url }; true +``` + +## Get or store the result of last operation + +Underscore(`_`) represents the implicit return of the previous statement. You can use this to quickly assign a variable from the output of the previous command: + +```ruby +Project.last +# => #<Project id:2537 root/discard>> +project = _ +# => #<Project id:2537 root/discard>> +project.id +# => 2537 +``` + +## Open object in irb + +Sometimes it is easier to navigate through a method if you are within the context of the object. You can shim into the namespace of `Object` to let you open `irb` within the context of any object: + +```ruby +Object.define_method(:irb) { binding.irb } + +project = Project.last +# => #<Project id:2537 root/discard>> +project.irb +# Notice new context +irb(#<Project>)> web_url +# => "https://gitlab-example/root/discard" +``` + ## Query the database using an ActiveRecord Model ```ruby @@ -818,7 +855,7 @@ conflicting_permanent_redirects = RedirectRoute.matching_path_and_descendants(pa conflicting_permanent_redirects.destroy_all ``` -## Merge Requests +## Merge requests ### Close a merge request properly (if merged but still marked as open) @@ -1287,7 +1324,7 @@ has more information about Service Ping. ### Generate or get the cached Service Ping ```ruby -Gitlab::UsageData.to_json +Gitlab::Usage::ServicePingReport.for(output: :all_metrics_values, cached: true) ``` ### Generate a fresh new Service Ping @@ -1295,7 +1332,7 @@ Gitlab::UsageData.to_json This also refreshes the cached Service Ping displayed in the Admin Area ```ruby -Gitlab::UsageData.to_json(force_refresh: true) +Gitlab::Usage::ServicePingReport.for(output: :all_metrics_values) ``` ### Generate and print @@ -1332,7 +1369,7 @@ cluster = Clusters::Cluster.find_by(name: 'cluster_name') Delete cluster without associated resources: ```ruby -# Find users with the Administrator role +# Find users with the administrator access user = User.find_by(username: 'admin_user') # Find the cluster with the ID @@ -1377,3 +1414,18 @@ Gitlab::CurrentSettings.elasticsearch_url Gitlab::CurrentSettings.elasticsearch_indexing ``` + +#### Changing the Elasticsearch password + +```ruby +es_url = Gitlab::CurrentSettings.current_application_settings + +# Confirm the current ElasticSearch URL +es_url.elasticsearch_url + +# Set the ElasticSearch URL +es_url.elasticsearch_url = "http://<username>:<password>@your.es.host:<port>" + +# Save the change +es_url.save! +``` diff --git a/doc/administration/troubleshooting/group_saml_scim.md b/doc/administration/troubleshooting/group_saml_scim.md index 8ec6b35ec39..a30ade058f6 100644 --- a/doc/administration/troubleshooting/group_saml_scim.md +++ b/doc/administration/troubleshooting/group_saml_scim.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Authentication & Authorization +group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- @@ -40,7 +40,8 @@ User claims and attributes: SCIM mapping: - + + Group Sync: diff --git a/doc/administration/troubleshooting/img/AzureAD-scim_provisioning.png b/doc/administration/troubleshooting/img/AzureAD-scim_provisioning.png Binary files differnew file mode 100644 index 00000000000..b8edcfa31c2 --- /dev/null +++ b/doc/administration/troubleshooting/img/AzureAD-scim_provisioning.png diff --git a/doc/administration/troubleshooting/postgresql.md b/doc/administration/troubleshooting/postgresql.md index e4d1696ea93..47fd424b1fd 100644 --- a/doc/administration/troubleshooting/postgresql.md +++ b/doc/administration/troubleshooting/postgresql.md @@ -98,6 +98,15 @@ This section is for links to information elsewhere in the GitLab documentation. - [Common Geo errors](../geo/replication/troubleshooting.md#fixing-common-errors). +- Mismatch in `pg_dump` and `psql` versions: + + ```plaintext + Dumping PostgreSQL database gitlabhq_production ... pg_dump: error: server version: 13.3; pg_dump version: 14.2 + pg_dump: error: aborting because of server version mismatch + ``` + + To fix this, see [Backup and restore a non-packaged PostgreSQL database](https://docs.gitlab.com/omnibus/settings/database.html#backup-and-restore-a-non-packaged-postgresql-database). + ## Support topics ### Database deadlocks diff --git a/doc/administration/user_settings.md b/doc/administration/user_settings.md index 2c1bb781882..2e879f8789d 100644 --- a/doc/administration/user_settings.md +++ b/doc/administration/user_settings.md @@ -1,6 +1,6 @@ --- stage: Manage -group: Authentication & Authorization +group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- |
