diff options
Diffstat (limited to 'doc/administration')
82 files changed, 1878 insertions, 990 deletions
diff --git a/doc/administration/audit_event_streaming.md b/doc/administration/audit_event_streaming.md new file mode 100644 index 00000000000..cee6cddbbf0 --- /dev/null +++ b/doc/administration/audit_event_streaming.md @@ -0,0 +1,70 @@ +--- +stage: Manage +group: Compliance +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Audit event streaming **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/332747) in GitLab 14.5 [with a flag](../administration/feature_flags.md) named `ff_external_audit_events_namespace`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available per group, ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `ff_external_audit_events_namespace`. On GitLab.com, this feature is not available. +You should not use this feature for production environments. + +Event streaming allows owners of top-level groups to set an HTTP endpoint to receive **all** audit events about the group, and its +subgroups and projects. + +Top-level group owners can manage their audit logs in third-party systems such as Splunk, using the Splunk +[HTTP Event Collector](https://docs.splunk.com/Documentation/Splunk/8.2.2/Data/UsetheHTTPEventCollector). Any service that can receive +structured JSON data can be used as the endpoint. + +NOTE: +GitLab can stream a single event more than once to the same destination. Use the `id` key in the payload to deduplicate incoming data. + +## Add a new event streaming destination + +WARNING: +Event streaming destinations will receive **all** audit event data, which could include sensitive information. Make sure you trust the destination endpoint. + +To enable event streaming, a group owner must add a new event streaming destination using the `externalAuditEventDestinationCreate` mutation +in the GraphQL API. + +```graphql +mutation { + externalAuditEventDestinationCreate(input: { destinationUrl: "https://mydomain.io/endpoint/ingest", groupPath: "my-group" } ) { + errors + externalAuditEventDestination { + destinationUrl + group { + name + } + } + } +} +``` + +Event streaming is enabled if: + +- The returned `errors` object is empty. +- The API responds with `200 OK`. + +## List currently enabled streaming destinations + +Group owners can view a list of event streaming destinations at any time using the `externalAuditEventDesinations` query type. + +```graphql +query { + group(fullPath: "my-group") { + id + externalAuditEventDestinations { + nodes { + destinationUrl + id + } + } + } +} +``` + +If the resulting list is empty, then audit event streaming is not enabled for that group. diff --git a/doc/administration/audit_events.md b/doc/administration/audit_events.md index 572c341f2b2..2062016ef03 100644 --- a/doc/administration/audit_events.md +++ b/doc/administration/audit_events.md @@ -12,7 +12,10 @@ on a [paid plan](https://about.gitlab.com/pricing/). GitLab system administrators can also take advantage of the logs located on the file system. See [the logs system documentation](logs.md#audit_jsonlog) for more details. -You can generate an [Audit report](audit_reports.md) of audit events. +You can: + +- Generate an [audit report](audit_reports.md) of audit events. +- [Stream audit events](audit_event_streaming.md) to an external endpoint. ## Overview @@ -70,6 +73,17 @@ From there, you can see the following actions: - Group changed visibility. - User was added to group and with which [permissions](../user/permissions.md). - User sign-in via [Group SAML](../user/group/saml_sso/index.md). +- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/8071) in GitLab 14.5, changes to the following + [group SAML](../user/group/saml_sso/index.md) configuration: + - Enabled status. + - Enforcing SSO-only authentication for web activity. + - Enforcing SSO-only authentication for Git and Dependency Proxy activity. + - Enforcing users to have dedicated group-managed accounts. + - Prohibiting outer forks. + - Identity provider SSO URL. + - Certificate fingerprint. + - Default membership role. + - SSO-SAML group sync configuration. - Permissions changes of a user assigned to a group. - Removed user from group. - Project repository imported into group. @@ -83,6 +97,7 @@ From there, you can see the following actions: - 2FA enforcement or grace period changed. - Roles allowed to create project changed. - Group CI/CD variable added, removed, or protected status changed. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/30857) in GitLab 13.3. +- Compliance framework created, updated, or deleted. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340649) in GitLab 14.6. Group events can also be accessed via the [Group Audit Events API](../api/audit_events.md#group-audit-events) @@ -186,9 +201,16 @@ successful sign-in events: After upgrading from GitLab Free to a paid tier, successful sign-in events are the only Audit Events visible in Audit Events views until more events are logged. +### "Deleted User" events + +Audit events can be created for a user after the user is deleted. The user name associated with the event is set to +"Deleted User" because the actual user name is unknowable. For example, if a deleted user's access to a project is +removed automatically due to expiration, the audit event is created for "Deleted User". We are [investigating](https://gitlab.com/gitlab-org/gitlab/-/issues/343933) +whether this is avoidable. + ### Missing events -Some events are not tracked in Audit Events. See the following +Some events are not tracked in audit events. See the following epics for more detail on which events are not being tracked, and our progress on adding these events into GitLab: @@ -196,10 +218,12 @@ on adding these events into GitLab: - [Group settings and activity](https://gitlab.com/groups/gitlab-org/-/epics/475) - [Instance-level settings and activity](https://gitlab.com/groups/gitlab-org/-/epics/476) -Don't see the event you want in any of the epics linked above? You can use the **Audit Event -Proposal** issue template to -[create an issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_template=Audit%20Event%20Proposal) -to request it, or you can [add it yourself](../development/audit_event_guide/). +Don't see the event you want in any of the epics linked above? You can either: + +- Use the **Audit Event Proposal** issue template to + [create an issue](https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_template=Audit%20Event%20Proposal) to + request it. +- [Add it yourself](../development/audit_event_guide/). ### Disabled events @@ -240,7 +264,7 @@ The search filters you can see depends on which audit level you are at. | Scope (Instance level) | A specific group, project, or user that the action was scoped to. | | Date range | Either via the date range buttons or pickers (maximum range of 31 days). Default is from the first day of the month to today's date. | - + ## Export to CSV **(PREMIUM SELF)** @@ -251,7 +275,7 @@ Export to CSV allows customers to export the current filter view of your audit e CSV file, which stores tabular data in plain text. The data provides a comprehensive view with respect to audit events. -To export the Audit Events to CSV: +To export the audit events to CSV: 1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Monitoring > Audit Events**. @@ -284,5 +308,5 @@ The first row contains the headers, which are listed in the following table alon ### Limitation -The Audit Events CSV file is limited to a maximum of `100,000` events. +The audit events CSV file is limited to a maximum of `100,000` events. The remaining records are truncated when this limit is reached. diff --git a/doc/administration/auth/atlassian.md b/doc/administration/auth/atlassian.md index b58bbfa8eac..14c48231a3d 100644 --- a/doc/administration/auth/atlassian.md +++ b/doc/administration/auth/atlassian.md @@ -41,7 +41,7 @@ To enable the Atlassian OmniAuth provider for passwordless authentication you mu sudo -u git -H editor /home/git/gitlab/config/gitlab.yml ``` -1. See [Initial OmniAuth Configuration](../../integration/omniauth.md#initial-omniauth-configuration) for initial settings to enable single sign-on and add `atlassian_oauth2` as an OAuth provider. +1. See [Configure initial settings](../../integration/omniauth.md#configure-initial-settings) for initial settings to enable single sign-on and add `atlassian_oauth2` as an OAuth provider. 1. Add the provider configuration for Atlassian: For Omnibus GitLab installations: diff --git a/doc/administration/auth/authentiq.md b/doc/administration/auth/authentiq.md index 835293ff500..19ee143a72a 100644 --- a/doc/administration/auth/authentiq.md +++ b/doc/administration/auth/authentiq.md @@ -27,7 +27,7 @@ Authentiq generates a Client ID and the accompanying Client Secret for you to us sudo -u git -H editor /home/git/gitlab/config/gitlab.yml ``` -1. See [Initial OmniAuth Configuration](../../integration/omniauth.md#initial-omniauth-configuration) for initial settings to enable single sign-on and add Authentiq as an OAuth provider. +1. See [Configure initial settings](../../integration/omniauth.md#configure-initial-settings) for initial settings to enable single sign-on and add Authentiq as an OAuth provider. 1. Add the provider configuration for Authentiq: diff --git a/doc/administration/auth/cognito.md b/doc/administration/auth/cognito.md index 41e77c10e27..d137489a838 100644 --- a/doc/administration/auth/cognito.md +++ b/doc/administration/auth/cognito.md @@ -40,7 +40,7 @@ The following steps enable AWS Cognito as an authentication provider: ## Configure GitLab -1. See [Initial OmniAuth Configuration](../../integration/omniauth.md#initial-omniauth-configuration) for initial settings. +1. See [Configure initial settings](../../integration/omniauth.md#configure-initial-settings) for initial settings. 1. On your GitLab server, open the configuration file. **For Omnibus installations** @@ -88,4 +88,4 @@ Your sign-in page should now display a Cognito button below the regular sign-in To begin the authentication process, click the icon, and AWS Cognito asks the user to sign in and authorize the GitLab application. If successful, the user is redirected and signed in to your GitLab instance. -For more information, see the [Initial OmniAuth Configuration](../../integration/omniauth.md#initial-omniauth-configuration). +For more information, see [Configure initial settings](../../integration/omniauth.md#configure-initial-settings). diff --git a/doc/administration/auth/crowd.md b/doc/administration/auth/crowd.md index f83710ef4c7..466e208a52e 100644 --- a/doc/administration/auth/crowd.md +++ b/doc/administration/auth/crowd.md @@ -36,7 +36,7 @@ this provider also allows Crowd authentication for Git-over-https requests. sudo -u git -H editor config/gitlab.yml ``` -1. See [Initial OmniAuth Configuration](../../integration/omniauth.md#initial-omniauth-configuration) +1. See [Configure initial settings](../../integration/omniauth.md#configure-initial-settings) for initial settings. 1. Add the provider configuration: @@ -84,7 +84,7 @@ could not authorize you from Crowd because invalid credentials ``` Ensure the Crowd users who need to sign in to GitLab are authorized to the -[application](#configure-a-new-crowd-application) in the **Authorisation** step. +[application](#configure-a-new-crowd-application) in the **Authorization** step. This could be verified by trying "Authentication test" for Crowd (as of 2.11). - + diff --git a/doc/administration/auth/jwt.md b/doc/administration/auth/jwt.md index b74b70ee8c0..26e523cb802 100644 --- a/doc/administration/auth/jwt.md +++ b/doc/administration/auth/jwt.md @@ -25,7 +25,7 @@ JWT will provide you with a secret key for you to use. sudo -u git -H editor config/gitlab.yml ``` -1. See [Initial OmniAuth Configuration](../../integration/omniauth.md#initial-omniauth-configuration) for initial settings. +1. See [Configure initial settings](../../integration/omniauth.md#configure-initial-settings) for initial settings. 1. Add the provider configuration. For Omnibus GitLab: diff --git a/doc/administration/auth/ldap/google_secure_ldap.md b/doc/administration/auth/ldap/google_secure_ldap.md index e5af8e8256a..69f0bfdce4c 100644 --- a/doc/administration/auth/ldap/google_secure_ldap.md +++ b/doc/administration/auth/ldap/google_secure_ldap.md @@ -35,7 +35,7 @@ The steps below cover: credentials' and 'Read user information'. Select 'Add LDAP Client' NOTE: - If you plan to use GitLab [LDAP Group Sync](index.md#group-sync) + If you plan to use GitLab [LDAP Group Sync](ldap_synchronization.md#group-sync) , turn on 'Read group information'.  diff --git a/doc/administration/auth/ldap/index.md b/doc/administration/auth/ldap/index.md index 92815f10b92..9047cfae1e9 100644 --- a/doc/administration/auth/ldap/index.md +++ b/doc/administration/auth/ldap/index.md @@ -19,58 +19,52 @@ This integration works with most LDAP-compliant directory servers, including: - Open LDAP. - 389 Server. -Users added through LDAP take a [licensed seat](../../../subscriptions/self_managed/index.md#billable-users). +Users added through LDAP: -## Security +- Take a [licensed seat](../../../subscriptions/self_managed/index.md#billable-users). +- Can authenticate with Git using either their GitLab username or their email and LDAP password, + even if password authentication for Git + [is disabled](../../../user/admin_area/settings/sign_in_restrictions.md#password-authentication-enabled). -GitLab assumes that LDAP users: +The LDAP DN is associated with existing GitLab users when: -- Are not able to change their LDAP `mail`, `email`, or `userPrincipalName` attributes. - An LDAP user allowed to change their email on the LDAP server can potentially - [take over any account](#enable-ldap-sign-in-for-existing-gitlab-users) - on your GitLab server. -- Have unique email addresses. If not, it's possible for LDAP users with the same - email address to share the same GitLab account. +- The existing user signs in to GitLab with LDAP for the first time. +- The LDAP email address is the primary email address of an existing GitLab user. If the LDAP email + attribute isn't found in the GitLab user database, a new user is created. -We recommend against using LDAP integration if your LDAP users are -allowed to change their `mail`, `email` or `userPrincipalName` attributes on -the LDAP server, or share email addresses. +If an existing GitLab user wants to enable LDAP sign-in for themselves, they should: -### User deletion +1. Check that their GitLab email address matches their LDAP email address. +1. Sign in to GitLab by using their LDAP credentials. -Users deleted from the LDAP server are immediately blocked from signing in -to GitLab and [no longer consumes a -license](../../../user/admin_area/moderate_users.md). -However, there's an LDAP check cache time of one hour (which is -[configurable](#adjust-ldap-user-sync-schedule) for GitLab Premium users). -This means users already signed-in or who are using Git over SSH can access -GitLab for up to one hour. Manually block the user in the GitLab Admin Area -to immediately block all access. +## Security -## Git password authentication +GitLab has multiple mechanisms to verify a user is still active in LDAP. If the user is no longer active in +LDAP, they are placed in an `ldap_blocked` status and are signed out. They are unable to sign in using any authentication provider until they are +reactivated in LDAP. -LDAP-enabled users can authenticate with Git using their GitLab username or -email and LDAP password, even if password authentication for Git is disabled -in the application settings. +Users are considered inactive in LDAP when they: -## Enable LDAP sign-in for existing GitLab users +- Are removed from the directory completely. +- Reside outside the configured `base` DN or `user_filter` search. +- Are marked as disabled or deactivated in Active Directory through the user account control attribute. This means attribute + `userAccountControl:1.2.840.113556.1.4.803` has bit 2 set. -When a user signs in to GitLab with LDAP for the first time and their LDAP -email address is the primary email address of an existing GitLab user, the -LDAP DN is associated with the existing user. If the LDAP email attribute -isn't found in the GitLab user database, a new user is created. +Status is checked for all LDAP users: -In other words, if an existing GitLab user wants to enable LDAP sign-in for -themselves, they should check that their GitLab email address matches their -LDAP email address, and then sign into GitLab by using their LDAP credentials. +- When signing in using any authentication provider. +- Once per hour for active web sessions or Git requests using tokens or SSH keys. +- When performing Git over HTTP requests using LDAP username and password. +- Once per day during [User Sync](ldap_synchronization.md#user-sync). -## Google Secure LDAP +### Security risks -> Introduced in GitLab 11.9. +You should only use LDAP integration if your LDAP users cannot: -[Google Cloud Identity](https://cloud.google.com/identity/) provides a Secure -LDAP service that can be configured with GitLab for authentication and group sync. -See [Google Secure LDAP](google_secure_ldap.md) for detailed configuration instructions. +- Change their `mail`, `email` or `userPrincipalName` attributes on the LDAP server. These + users can potentially take over any account on your GitLab server. +- Share email addresses. LDAP users with the same email address can share the same GitLab + account. ## Configure LDAP @@ -109,7 +103,6 @@ gitlab_rails['ldap_servers'] = { 'verify_certificates' => true, 'bind_dn' => '_the_full_dn_of_the_user_you_will_bind_with', 'password' => '_the_password_of_the_bind_user', - 'verify_certificates' => true, 'tls_options' => { 'ca_file' => '', 'ssl_version' => '', @@ -170,7 +163,7 @@ These configuration settings are available: | `bind_dn` | The full DN of the user you bind with. | **{dotted-circle}** No | `'america\momo'` or `'CN=Gitlab,OU=Users,DC=domain,DC=com'` | | `password` | The password of the bind user. | **{dotted-circle}** No | `'your_great_password'` | | `encryption` | Encryption method. The `method` key is deprecated in favor of `encryption`. | **{check-circle}** Yes | `'start_tls'` or `'simple_tls'` or `'plain'` | -| `verify_certificates` | Enables SSL certificate verification if encryption method is `start_tls` or `simple_tls`. Defaults to true. | **{dotted-circle}** No | boolean | +| `verify_certificates` | Enables SSL certificate verification if encryption method is `start_tls` or `simple_tls`. If set to false, no validation of the LDAP server's SSL certificate is performed. Defaults to true. | **{dotted-circle}** No | boolean | | `timeout` | Set a timeout, in seconds, for LDAP queries. This helps avoid blocking a request if the LDAP server becomes unresponsive. A value of `0` means there is no timeout. (default: `10`) | **{dotted-circle}** No | `10` or `30` | | `active_directory` | This setting specifies if LDAP server is Active Directory LDAP server. For non-AD servers it skips the AD specific queries. If your LDAP server is not AD, set this to false. | **{dotted-circle}** No | boolean | | `allow_username_or_email_login` | If enabled, GitLab ignores everything after the first `@` in the LDAP username submitted by the user on sign-in. If you are using `uid: 'userPrincipalName'` on ActiveDirectory you must disable this setting, because the userPrincipalName contains an `@`. | **{dotted-circle}** No | boolean | @@ -266,7 +259,7 @@ For more information about `LDAP_MATCHING_RULE_IN_CHAIN` filters, see [Search Filter Syntax](https://docs.microsoft.com/en-us/windows/win32/adsi/search-filter-syntax). Support for nested members in the user filter shouldn't be confused with -[group sync nested groups](#supported-ldap-group-typesattributes) support. +[group sync nested groups](ldap_synchronization.md#supported-ldap-group-typesattributes) support. GitLab does not support the custom filter syntax used by OmniAuth LDAP. @@ -347,7 +340,7 @@ sync, while also allowing your SAML identity provider to handle additional checks like custom 2FA. When LDAP web sign in is disabled, users don't see an **LDAP** tab on the sign-in page. -This does not disable [using LDAP credentials for Git access](#git-password-authentication). +This does not disable using LDAP credentials for Git access. **Omnibus configuration** @@ -458,26 +451,6 @@ If initially your LDAP configuration looked like: 1. [Restart GitLab](../../restart_gitlab.md#installations-from-source) for the changes to take effect. -## Encryption - -### TLS server authentication - -`simple_tls` and `start_tls` are the two available encryption methods. - -For either encryption method, if setting `verify_certificates: false`, TLS -encryption is established with the LDAP server before any LDAP-protocol data is -exchanged but no validation of the LDAP server's SSL certificate is performed. - -### Limitations - -#### TLS client authentication - -Not implemented by `Net::LDAP`. - -You should disable anonymous LDAP authentication and enable simple or Simple Authentication -and Security Layer (SASL) authentication. The TLS client authentication setting in your LDAP server -cannot be mandatory and clients cannot be authenticated with the TLS protocol. - ## Multiple LDAP servers **(PREMIUM SELF)** With GitLab, you can configure multiple LDAP servers that your GitLab instance @@ -528,342 +501,43 @@ If you configure multiple LDAP servers, use a unique naming convention for the `label` section of each entry. That label is used as the display name of the tab shown on the sign-in page. -## User sync **(PREMIUM SELF)** - -Once per day, GitLab runs a worker to check and update GitLab -users against LDAP. - -The process executes the following access checks: - -- Ensure the user is still present in LDAP. -- If the LDAP server is Active Directory, ensure the user is active (not - blocked/disabled state). This check is performed only if - `active_directory: true` is set in the LDAP configuration. - -In Active Directory, a user is marked as disabled/blocked if the user -account control attribute (`userAccountControl:1.2.840.113556.1.4.803`) -has bit 2 set. - -<!-- vale gitlab.Spelling = NO --> - -For more information, see [Bitmask Searches in LDAP](https://ctovswild.com/2009/09/03/bitmask-searches-in-ldap/). - -<!-- vale gitlab.Spelling = YES --> +## Disable anonymous LDAP authentication -The user is set to an `ldap_blocked` state in GitLab if the previous conditions -fail. This means the user cannot sign in or push or pull code. +GitLab doesn't support TLS client authentication. Complete these steps on your LDAP server. -The process also updates the following user information: - -- Email address -- SSH public keys (if `sync_ssh_keys` is set) -- Kerberos identity (if Kerberos is enabled) - -The LDAP sync process: - -- Updates existing users. -- Creates new users on first sign in. - -### Adjust LDAP user sync schedule **(PREMIUM SELF)** - -By default, GitLab runs a worker once per day at 01:30 a.m. server time to -check and update GitLab users against LDAP. - -You can manually configure LDAP user sync times by setting the -following configuration values, in cron format. If needed, you can -use a [crontab generator](http://www.crontabgenerator.com). -The example below shows how to set LDAP user -sync to run once every 12 hours at the top of the hour. - -**Omnibus installations** - -1. Edit `/etc/gitlab/gitlab.rb`: - - ```ruby - gitlab_rails['ldap_sync_worker_cron'] = "0 */12 * * *" - ``` - -1. [Reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. - -**Source installations** - -1. Edit `config/gitlab.yaml`: - - ```yaml - cron_jobs: - ldap_sync_worker_cron: - "0 */12 * * *" - ``` - -1. [Restart GitLab](../../restart_gitlab.md#installations-from-source) for the changes to take effect. +1. Disable anonymous authentication. +1. Enable one of the following authentication types: + - Simple authentication. + - Simple Authentication and Security Layer (SASL) authentication. -## Group Sync **(PREMIUM SELF)** +The TLS client authentication setting in your LDAP server cannot be mandatory and clients cannot be +authenticated with the TLS protocol. -If your LDAP supports the `memberof` property, when the user signs in for the -first time GitLab triggers a sync for groups the user should be a member of. -That way they don't have to wait for the hourly sync to be granted -access to their groups and projects. +## Deleting users -A group sync process runs every hour on the hour, and `group_base` must be set -in LDAP configuration for LDAP synchronizations based on group CN to work. This allows -GitLab group membership to be automatically updated based on LDAP group members. +Users deleted from the LDAP server: -The `group_base` configuration should be a base LDAP 'container', such as an -'organization' or 'organizational unit', that contains LDAP groups that should -be available to GitLab. For example, `group_base` could be -`ou=groups,dc=example,dc=com`. In the configuration file it looks like the -following. +- Are immediately blocked from signing in to GitLab. +- [No longer consume a license](../../../user/admin_area/moderate_users.md). -**Omnibus configuration** - -1. Edit `/etc/gitlab/gitlab.rb`: - - ```ruby - gitlab_rails['ldap_servers'] = { - 'main' => { - # snip... - 'group_base' => 'ou=groups,dc=example,dc=com', - } - } - ``` - -1. [Apply your changes to GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure). - -**Source configuration** - -1. Edit `/home/git/gitlab/config/gitlab.yml`: - - ```yaml - production: - ldap: - servers: - main: - # snip... - group_base: ou=groups,dc=example,dc=com - ``` - -1. [Restart GitLab](../../restart_gitlab.md#installations-from-source) for the changes to take effect. - -To take advantage of group sync, group owners or maintainers must [create one -or more LDAP group links](#add-group-links). - -### Add group links **(PREMIUM SELF)** - -For information on adding group links by using CNs and filters, refer to the -[GitLab groups documentation](../../../user/group/index.md#manage-group-memberships-via-ldap). - -### Administrator sync **(PREMIUM SELF)** - -As an extension of group sync, you can automatically manage your global GitLab -administrators. Specify a group CN for `admin_group` and all members of the -LDAP group are given administrator privileges. The configuration looks -like the following. - -NOTE: -Administrators are not synced unless `group_base` is also -specified alongside `admin_group`. Also, only specify the CN of the `admin_group`, -as opposed to the full DN. -Additionally, if an LDAP user has an `admin` role, but is not a member of the `admin_group` -group, GitLab revokes their `admin` role when syncing. - -**Omnibus configuration** - -1. Edit `/etc/gitlab/gitlab.rb`: - - ```ruby - gitlab_rails['ldap_servers'] = { - 'main' => { - # snip... - 'group_base' => 'ou=groups,dc=example,dc=com', - 'admin_group' => 'my_admin_group', - } - } - ``` +However, these users can continue to use Git with SSH until the next time the +[LDAP check cache runs](ldap_synchronization.md#adjust-ldap-user-sync-schedule). -1. [Apply your changes to GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure). +To delete the account immediately, you can manually +[block the user](../../../user/admin_area/moderate_users.md#block-a-user). -**Source configuration** - -1. Edit `/home/git/gitlab/config/gitlab.yml`: - - ```yaml - production: - ldap: - servers: - main: - # snip... - group_base: ou=groups,dc=example,dc=com - admin_group: my_admin_group - ``` - -1. [Restart GitLab](../../restart_gitlab.md#installations-from-source) for the changes to take effect. - -### Global group memberships lock **(PREMIUM SELF)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/1793) in GitLab 12.0. - -"Lock memberships to LDAP synchronization" setting allows instance administrators -to lock down user abilities to invite new members to a group. - -When enabled, the following applies: - -- Only administrator can manage memberships of any group including access levels. -- Users are not allowed to share project with other groups or invite members to - a project created in a group. - -To enable it, you must: - -1. [Configure LDAP](#configure-ldap). -1. On the top bar, select **Menu > Admin**. -1. On the left sidebar, select **Settings > General**. -1. Expand the **Visibility and access controls** section. -1. Ensure the **Lock memberships to LDAP synchronization** checkbox is selected. - -### Adjust LDAP group sync schedule **(PREMIUM SELF)** - -By default, GitLab runs a group sync process every hour, on the hour. -The values shown are in cron format. If needed, you can use a -[Crontab Generator](http://www.crontabgenerator.com). - -WARNING: -Do not start the sync process too frequently as this -could lead to multiple syncs running concurrently. This concern is primarily -for installations with a large number of LDAP users. Review the -[LDAP group sync benchmark metrics](#benchmarks) to see how -your installation compares before proceeding. - -You can manually configure LDAP group sync times by setting the -following configuration values. The example below shows how to set group -sync to run once every two hours at the top of the hour. - -**Omnibus installations** - -1. Edit `/etc/gitlab/gitlab.rb`: - - ```ruby - gitlab_rails['ldap_group_sync_worker_cron'] = "0 */2 * * * *" - ``` - -1. [Reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. - -**Source installations** - -1. Edit `config/gitlab.yaml`: - - ```yaml - cron_jobs: - ldap_group_sync_worker_cron: - "*/30 * * * *" - ``` - -1. [Restart GitLab](../../restart_gitlab.md#installations-from-source) for the changes to take effect. - -### External groups **(PREMIUM SELF)** - -Using the `external_groups` setting allows you to mark all users belonging -to these groups as [external users](../../../user/permissions.md#external-users). -Group membership is checked periodically through the `LdapGroupSync` background -task. - -**Omnibus configuration** - -1. Edit `/etc/gitlab/gitlab.rb`: - - ```ruby - gitlab_rails['ldap_servers'] = { - 'main' => { - # snip... - 'external_groups' => ['interns', 'contractors'], - } - } - ``` - -1. [Apply your changes to GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure). - -**Source configuration** - -1. Edit `config/gitlab.yaml`: - - ```yaml - production: - ldap: - servers: - main: - # snip... - external_groups: ['interns', 'contractors'] - ``` - -1. [Restart GitLab](../../restart_gitlab.md#installations-from-source) for the changes to take effect. - -### Group sync technical details - -This section outlines what LDAP queries are executed and what behavior you -can expect from group sync. - -Group member access are downgraded from a higher level if their LDAP group -membership changes. For example, if a user the Owner role in a group and the -next group sync reveals they should only have the Developer role, their -access is adjusted accordingly. The only exception is if the user is the -last owner in a group. Groups need at least one owner to fulfill -administrative duties. - -#### Supported LDAP group types/attributes - -GitLab supports LDAP groups that use member attributes: - -- `member` -- `submember` -- `uniquemember` -- `memberof` -- `memberuid` - -This means group sync supports (at least) LDAP groups with the following object -classes: - -- `groupOfNames` -- `posixGroup` -- `groupOfUniqueNames` - -Other object classes should work if members are defined as one of the -mentioned attributes. - -Active Directory supports nested groups. Group sync recursively resolves -membership if `active_directory: true` is set in the configuration file. - -##### Nested group memberships - -Nested group memberships are resolved only if the nested group -is found in the configured `group_base`. For example, if GitLab sees a -nested group with DN `cn=nested_group,ou=special_groups,dc=example,dc=com` but -the configured `group_base` is `ou=groups,dc=example,dc=com`, `cn=nested_group` -is ignored. - -#### Queries - -- Each LDAP group is queried a maximum of one time with base `group_base` and - filter `(cn=<cn_from_group_link>)`. -- If the LDAP group has the `memberuid` attribute, GitLab executes another - LDAP query per member to obtain each user's full DN. These queries are - executed with base `base`, scope 'base object', and a filter depending on - whether `user_filter` is set. Filter may be `(uid=<uid_from_group>)` or a - joining of `user_filter`. - -#### Benchmarks +## Google Secure LDAP -Group sync was written to be as performant as possible. Data is cached, database -queries are optimized, and LDAP queries are minimized. The last benchmark run -revealed the following metrics: +> Introduced in GitLab 11.9. -For 20,000 LDAP users, 11,000 LDAP groups, and 1,000 GitLab groups with 10 -LDAP group links each: +[Google Cloud Identity](https://cloud.google.com/identity/) provides a Secure +LDAP service that can be configured with GitLab for authentication and group sync. +See [Google Secure LDAP](google_secure_ldap.md) for detailed configuration instructions. -- Initial sync (no existing members assigned in GitLab) took 1.8 hours -- Subsequent syncs (checking membership, no writes) took 15 minutes +## Synchronize users and groups -These metrics are meant to provide a baseline and performance may vary based on -any number of factors. This benchmark was extreme and most instances don't -have near this many users or groups. Disk speed, database performance, -network and LDAP server response time affects these metrics. +For more information on synchronizing users and groups between LDAP and GitLab, see +[LDAP synchronization](ldap_synchronization.md). ## Troubleshooting diff --git a/doc/administration/auth/ldap/ldap-troubleshooting.md b/doc/administration/auth/ldap/ldap-troubleshooting.md index 4757725d0bd..aa40060c4c1 100644 --- a/doc/administration/auth/ldap/ldap-troubleshooting.md +++ b/doc/administration/auth/ldap/ldap-troubleshooting.md @@ -229,7 +229,7 @@ ldapsearch -H ldaps://$host:$port -D "$bind_dn" -y bind_dn_password.txt -b "$ba #### Sync all users **(PREMIUM SELF)** -The output from a manual [user sync](index.md#user-sync) can show you what happens when +The output from a manual [user sync](ldap_synchronization.md#user-sync) can show you what happens when GitLab tries to sync its users against LDAP. Enter the [rails console](#rails-console) and then run: @@ -239,8 +239,7 @@ Rails.logger.level = Logger::DEBUG LdapSyncWorker.new.perform ``` -Next, [learn how to read the -output](#example-console-output-after-a-user-sync). +Next, [learn how to read the output](#example-console-output-after-a-user-sync). ##### Example console output after a user sync **(PREMIUM SELF)** @@ -342,9 +341,8 @@ LDAP group sync, but for some reason it's not happening. There are several things to check to debug the situation. - Ensure LDAP configuration has a `group_base` specified. - [This configuration](index.md#group-sync) is required for group sync to work properly. -- Ensure the correct [LDAP group link is added to the GitLab - group](index.md#add-group-links). + [This configuration](ldap_synchronization.md#group-sync) is required for group sync to work properly. +- Ensure the correct [LDAP group link is added to the GitLab group](ldap_synchronization.md#add-group-links). - Check that the user has an LDAP identity: 1. Sign in to GitLab as an administrator user. 1. On the top bar, select **Menu > Admin**. @@ -354,7 +352,7 @@ things to check to debug the situation. 1. Select the **Identities** tab. There should be an LDAP identity with an LDAP DN as the 'Identifier'. If not, this user hasn't signed in with LDAP yet and must do so first. -- You've waited an hour or [the configured interval](index.md#adjust-ldap-group-sync-schedule) for +- You've waited an hour or [the configured interval](ldap_synchronization.md#adjust-ldap-group-sync-schedule) for the group to sync. To speed up the process, either go to the GitLab group **Group information > Members** and press **Sync now** (sync one group) or [run the group sync Rake task](../../raketasks/ldap.md#run-a-group-sync) (sync all groups). @@ -366,8 +364,7 @@ the rails console. 1. Choose a GitLab group to test with. This group should have an LDAP group link already configured. 1. [Enable debug logging, find the above GitLab group, and sync it with LDAP](#sync-one-group). -1. Look through the output of the sync. See [example log - output](#example-console-output-after-a-group-sync) +1. Look through the output of the sync. See [example log output](#example-console-output-after-a-group-sync) for how to read the output. 1. If you still aren't able to see why the user isn't being added, [query the LDAP group directly](#query-a-group-in-ldap) to see what members are listed. @@ -377,20 +374,20 @@ the rails console. #### Administrator privileges not granted -When [Administrator sync](index.md#administrator-sync) has been configured +When [Administrator sync](ldap_synchronization.md#administrator-sync) has been configured but the configured users aren't granted the correct administrator privileges, confirm the following are true: -- A [`group_base` is also configured](index.md#group-sync). +- A [`group_base` is also configured](ldap_synchronization.md#group-sync). - The configured `admin_group` in the `gitlab.rb` is a CN, rather than a DN or an array. - This CN falls under the scope of the configured `group_base`. - The members of the `admin_group` have already signed into GitLab with their LDAP credentials. GitLab only grants the Administrator role to the users whose accounts are already connected to LDAP. -If all the above are true and the users are still not getting access, [run a manual -group sync](#sync-all-groups) in the rails console and [look through the -output](#example-console-output-after-a-group-sync) to see what happens when +If all the above are true and the users are still not getting access, +[run a manual group sync](#sync-all-groups) in the rails console and +[look through the output](#example-console-output-after-a-group-sync) to see what happens when GitLab syncs the `admin_group`. #### Sync all groups @@ -399,7 +396,7 @@ NOTE: To sync all groups manually when debugging is unnecessary, [use the Rake task](../../raketasks/ldap.md#run-a-group-sync) instead. -The output from a manual [group sync](index.md#group-sync) can show you what happens +The output from a manual [group sync](ldap_synchronization.md#group-sync) can show you what happens when GitLab syncs its LDAP group memberships against LDAP. ```ruby @@ -494,7 +491,7 @@ this line indicates the sync is finished: Finished syncing admin users for 'ldapmain' provider ``` -If [administrator sync](index.md#administrator-sync) is not configured, you see a message +If [administrator sync](ldap_synchronization.md#administrator-sync) is not configured, you see a message stating as such: ```shell @@ -583,6 +580,25 @@ end You can then [run a UserSync](#sync-all-users) **(PREMIUM SELF)** to sync the latest DN for each of these users. +## Expired license causes errors with multiple LDAP servers + +Using [multiple LDAP servers](index.md#multiple-ldap-servers) requires a valid license. An expired +license can cause: + +- `502` errors in the web interface. +- The following error in logs (the actual strategy name depends on the name configured in `/etc/gitlab/gitlab.rb`): + + ```plaintext + Could not find a strategy with name `Ldapsecondary'. Please ensure it is required or explicitly set it using the :strategy_class option. (Devise::OmniAuth::StrategyNotFound) + ``` + +To resolve this error, you must apply a new license to the GitLab instance without the web interface: + +1. Remove or comment out the GitLab configuration lines for all non-primary LDAP servers. +1. [Reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) so that it temporarily uses only one LDAP server. +1. Enter the [Rails console and add the license key](../../troubleshooting/gitlab_rails_cheat_sheet.md#add-a-license-through-the-console). +1. Re-enable the additional LDAP servers in the GitLab configuration and reconfigure GitLab again. + ## Debugging Tools ### LDAP check @@ -610,8 +626,7 @@ If a user account is blocked or unblocked due to the LDAP configuration, a message is [logged to `application.log`](../../logs.md#applicationlog). If there is an unexpected error during an LDAP lookup (configuration error, -timeout), the sign-in is rejected and a message is [logged to -`production.log`](../../logs.md#productionlog). +timeout), the sign-in is rejected and a message is [logged to `production.log`](../../logs.md#productionlog). ### ldapsearch diff --git a/doc/administration/auth/ldap/ldap_synchronization.md b/doc/administration/auth/ldap/ldap_synchronization.md new file mode 100644 index 00000000000..2673a8374ec --- /dev/null +++ b/doc/administration/auth/ldap/ldap_synchronization.md @@ -0,0 +1,349 @@ +--- +stage: Manage +group: Access +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# LDAP synchronization **(PREMIUM SELF)** + +If you have [configured LDAP to work with GitLab](index.md), GitLab can automatically synchronize +users and groups. This process updates user and group information. + +You can change when synchronization occurs. + +## User sync + +Once per day, GitLab runs a worker to check and update GitLab +users against LDAP. + +The process executes the following access checks: + +- Ensure the user is still present in LDAP. +- If the LDAP server is Active Directory, ensure the user is active (not + blocked/disabled state). This check is performed only if + `active_directory: true` is set in the LDAP configuration. + +In Active Directory, a user is marked as disabled/blocked if the user +account control attribute (`userAccountControl:1.2.840.113556.1.4.803`) +has bit 2 set. + +<!-- vale gitlab.Spelling = NO --> + +For more information, see [Bitmask Searches in LDAP](https://ctovswild.com/2009/09/03/bitmask-searches-in-ldap/). + +<!-- vale gitlab.Spelling = YES --> + +The user is set to an `ldap_blocked` state in GitLab if the previous conditions +fail. This means the user cannot sign in or push or pull code. + +The process also updates the following user information: + +- Email address +- SSH public keys (if `sync_ssh_keys` is set) +- Kerberos identity (if Kerberos is enabled) + +The LDAP sync process: + +- Updates existing users. +- Creates new users on first sign in. + +### Adjust LDAP user sync schedule + +By default, GitLab runs a worker once per day at 01:30 a.m. server time to +check and update GitLab users against LDAP. + +You can manually configure LDAP user sync times by setting the +following configuration values, in cron format. If needed, you can +use a [crontab generator](http://www.crontabgenerator.com). +The example below shows how to set LDAP user +sync to run once every 12 hours at the top of the hour. + +**Omnibus installations** + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['ldap_sync_worker_cron'] = "0 */12 * * *" + ``` + +1. [Reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. + +**Source installations** + +1. Edit `config/gitlab.yaml`: + + ```yaml + cron_jobs: + ldap_sync_worker_cron: + "0 */12 * * *" + ``` + +1. [Restart GitLab](../../restart_gitlab.md#installations-from-source) for the changes to take effect. + +## Group sync + +If your LDAP supports the `memberof` property, when the user signs in for the +first time GitLab triggers a sync for groups the user should be a member of. +That way they don't have to wait for the hourly sync to be granted +access to their groups and projects. + +A group sync process runs every hour on the hour, and `group_base` must be set +in LDAP configuration for LDAP synchronizations based on group CN to work. This allows +GitLab group membership to be automatically updated based on LDAP group members. + +The `group_base` configuration should be a base LDAP 'container', such as an +'organization' or 'organizational unit', that contains LDAP groups that should +be available to GitLab. For example, `group_base` could be +`ou=groups,dc=example,dc=com`. In the configuration file it looks like the +following. + +**Omnibus configuration** + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['ldap_servers'] = { + 'main' => { + # snip... + 'group_base' => 'ou=groups,dc=example,dc=com', + } + } + ``` + +1. [Apply your changes to GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure). + +**Source configuration** + +1. Edit `/home/git/gitlab/config/gitlab.yml`: + + ```yaml + production: + ldap: + servers: + main: + # snip... + group_base: ou=groups,dc=example,dc=com + ``` + +1. [Restart GitLab](../../restart_gitlab.md#installations-from-source) for the changes to take effect. + +To take advantage of group sync, group owners or maintainers must [create one +or more LDAP group links](#add-group-links). + +### Add group links + +For information on adding group links by using CNs and filters, refer to the +[GitLab groups documentation](../../../user/group/index.md#manage-group-memberships-via-ldap). + +### Administrator sync + +As an extension of group sync, you can automatically manage your global GitLab +administrators. Specify a group CN for `admin_group` and all members of the +LDAP group are given administrator privileges. The configuration looks +like the following. + +NOTE: +Administrators are not synced unless `group_base` is also +specified alongside `admin_group`. Also, only specify the CN of the `admin_group`, +as opposed to the full DN. +Additionally, if an LDAP user has an `admin` role, but is not a member of the `admin_group` +group, GitLab revokes their `admin` role when syncing. + +**Omnibus configuration** + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['ldap_servers'] = { + 'main' => { + # snip... + 'group_base' => 'ou=groups,dc=example,dc=com', + 'admin_group' => 'my_admin_group', + } + } + ``` + +1. [Apply your changes to GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure). + +**Source configuration** + +1. Edit `/home/git/gitlab/config/gitlab.yml`: + + ```yaml + production: + ldap: + servers: + main: + # snip... + group_base: ou=groups,dc=example,dc=com + admin_group: my_admin_group + ``` + +1. [Restart GitLab](../../restart_gitlab.md#installations-from-source) for the changes to take effect. + +### Global group memberships lock + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/1793) in GitLab 12.0. + +"Lock memberships to LDAP synchronization" setting allows instance administrators +to lock down user abilities to invite new members to a group. + +When enabled, the following applies: + +- Only administrator can manage memberships of any group including access levels. +- Users are not allowed to share project with other groups or invite members to + a project created in a group. + +To enable it, you must: + +1. [Configure LDAP](index.md#configure-ldap). +1. On the top bar, select **Menu > Admin**. +1. On the left sidebar, select **Settings > General**. +1. Expand the **Visibility and access controls** section. +1. Ensure the **Lock memberships to LDAP synchronization** checkbox is selected. + +### Adjust LDAP group sync schedule + +By default, GitLab runs a group sync process every hour, on the hour. +The values shown are in cron format. If needed, you can use a +[Crontab Generator](http://www.crontabgenerator.com). + +WARNING: +Do not start the sync process too frequently as this +could lead to multiple syncs running concurrently. This concern is primarily +for installations with a large number of LDAP users. Review the +[LDAP group sync benchmark metrics](#benchmarks) to see how +your installation compares before proceeding. + +You can manually configure LDAP group sync times by setting the +following configuration values. The example below shows how to set group +sync to run once every two hours at the top of the hour. + +**Omnibus installations** + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['ldap_group_sync_worker_cron'] = "0 */2 * * * *" + ``` + +1. [Reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. + +**Source installations** + +1. Edit `config/gitlab.yaml`: + + ```yaml + cron_jobs: + ldap_group_sync_worker_cron: + "*/30 * * * *" + ``` + +1. [Restart GitLab](../../restart_gitlab.md#installations-from-source) for the changes to take effect. + +### External groups + +Using the `external_groups` setting allows you to mark all users belonging +to these groups as [external users](../../../user/permissions.md#external-users). +Group membership is checked periodically through the `LdapGroupSync` background +task. + +**Omnibus configuration** + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['ldap_servers'] = { + 'main' => { + # snip... + 'external_groups' => ['interns', 'contractors'], + } + } + ``` + +1. [Apply your changes to GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure). + +**Source configuration** + +1. Edit `config/gitlab.yaml`: + + ```yaml + production: + ldap: + servers: + main: + # snip... + external_groups: ['interns', 'contractors'] + ``` + +1. [Restart GitLab](../../restart_gitlab.md#installations-from-source) for the changes to take effect. + +### Group sync technical details + +This section outlines what LDAP queries are executed and what behavior you +can expect from group sync. + +Group member access are downgraded from a higher level if their LDAP group +membership changes. For example, if a user the Owner role in a group and the +next group sync reveals they should only have the Developer role, their +access is adjusted accordingly. The only exception is if the user is the +last owner in a group. Groups need at least one owner to fulfill +administrative duties. + +#### Supported LDAP group types/attributes + +GitLab supports LDAP groups that use member attributes: + +- `member` +- `submember` +- `uniquemember` +- `memberof` +- `memberuid` + +This means group sync supports (at least) LDAP groups with the following object +classes: + +- `groupOfNames` +- `posixGroup` +- `groupOfUniqueNames` + +Other object classes should work if members are defined as one of the +mentioned attributes. + +Active Directory supports nested groups. Group sync recursively resolves +membership if `active_directory: true` is set in the configuration file. + +##### Nested group memberships + +Nested group memberships are resolved only if the nested group +is found in the configured `group_base`. For example, if GitLab sees a +nested group with DN `cn=nested_group,ou=special_groups,dc=example,dc=com` but +the configured `group_base` is `ou=groups,dc=example,dc=com`, `cn=nested_group` +is ignored. + +#### Queries + +- Each LDAP group is queried a maximum of one time with base `group_base` and + filter `(cn=<cn_from_group_link>)`. +- If the LDAP group has the `memberuid` attribute, GitLab executes another + LDAP query per member to obtain each user's full DN. These queries are + executed with base `base`, scope 'base object', and a filter depending on + whether `user_filter` is set. Filter may be `(uid=<uid_from_group>)` or a + joining of `user_filter`. + +#### Benchmarks + +Group sync was written to be as performant as possible. Data is cached, database +queries are optimized, and LDAP queries are minimized. The last benchmark run +revealed the following metrics: + +For 20,000 LDAP users, 11,000 LDAP groups, and 1,000 GitLab groups with 10 +LDAP group links each: + +- Initial sync (no existing members assigned in GitLab) took 1.8 hours +- Subsequent syncs (checking membership, no writes) took 15 minutes + +These metrics are meant to provide a baseline and performance may vary based on +any number of factors. This benchmark was extreme and most instances don't +have near this many users or groups. Disk speed, database performance, +network and LDAP server response time affects these metrics. diff --git a/doc/administration/auth/oidc.md b/doc/administration/auth/oidc.md index 12729f2050b..b8c443ae4d4 100644 --- a/doc/administration/auth/oidc.md +++ b/doc/administration/auth/oidc.md @@ -27,7 +27,7 @@ The OpenID Connect provides you with a client's details and secret for you to us sudo -u git -H editor config/gitlab.yml ``` - See [Initial OmniAuth Configuration](../../integration/omniauth.md#initial-omniauth-configuration) for initial settings. + See [Configure initial settings](../../integration/omniauth.md#configure-initial-settings) for initial settings. 1. Add the provider configuration. @@ -40,7 +40,7 @@ The OpenID Connect provides you with a client's details and secret for you to us 'icon' => '<custom_provider_icon>', 'args' => { 'name' => 'openid_connect', - 'scope' => ['openid','profile'], + 'scope' => ['openid','profile','email'], 'response_type' => 'code', 'issuer' => '<your_oidc_url>', 'discovery' => true, @@ -65,7 +65,7 @@ The OpenID Connect provides you with a client's details and secret for you to us icon: '<custom_provider_icon>', args: { name: 'openid_connect', - scope: ['openid','profile'], + scope: ['openid','profile','email'], response_type: 'code', issuer: '<your_oidc_url>', discovery: true, @@ -228,7 +228,7 @@ Azure B2C [offers two ways of defining the business logic for logging in a user] While cumbersome to configure, custom policies are required because standard Azure B2C user flows [do not send the OpenID `email` claim](https://github.com/MicrosoftDocs/azure-docs/issues/16566). In -other words, they do not work with the [`allow_single_sign_on` or `auto_link_user` parameters](../../integration/omniauth.md#initial-omniauth-configuration). +other words, they do not work with the [`allow_single_sign_on` or `auto_link_user` parameters](../../integration/omniauth.md#configure-initial-settings). With a standard Azure B2C policy, GitLab cannot create a new account or link to an existing one with an email address. diff --git a/doc/administration/cicd.md b/doc/administration/cicd.md index 89fc31822ee..d53290f1d5d 100644 --- a/doc/administration/cicd.md +++ b/doc/administration/cicd.md @@ -7,22 +7,18 @@ type: howto # GitLab CI/CD instance configuration **(FREE SELF)** -GitLab CI/CD is enabled by default in all new projects on an instance. You can configure -the instance to have [GitLab CI/CD disabled by default](#disable-gitlab-cicd-in-new-projects) -in new projects. - -You can still choose to [enable GitLab CI/CD in individual projects](../ci/enable_or_disable_ci.md#enable-cicd-in-a-project) -at any time. +GitLab administrators can manage the GitLab CI/CD configuration for their instance. ## Disable GitLab CI/CD in new projects -You can set GitLab CI/CD to be disabled by default in all new projects by modifying the settings in: +GitLab CI/CD is enabled by default in all new projects on an instance. You can set +CI/CD to be disabled by default in new projects by modifying the settings in: - `gitlab.yml` for source installations. - `gitlab.rb` for Omnibus GitLab installations. Existing projects that already had CI/CD enabled are unchanged. Also, this setting only changes -the project default, so project owners can still enable CI/CD in the project settings. +the project default, so project owners [can still enable CI/CD in the project settings](../ci/enable_or_disable_ci.md#enable-cicd-in-a-project). For installations from source: @@ -62,6 +58,19 @@ For Omnibus GitLab installations: sudo gitlab-ctl reconfigure ``` +## Set the `needs:` job limit **(FREE SELF)** + +The maximum number of jobs that can be defined in `needs:` defaults to 50. + +A GitLab administrator with [access to the GitLab Rails console](operations/rails_console.md#starting-a-rails-console-session) +can choose a custom limit. For example, to set the limit to `100`: + +```ruby +Plan.default.actual_limits.update!(ci_needs_size_limit: 100) +``` + +To disable directed acyclic graphs (DAG), set the limit to `0`. + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/administration/clusters/kas.md b/doc/administration/clusters/kas.md index 226710a8911..93b24007de8 100644 --- a/doc/administration/clusters/kas.md +++ b/doc/administration/clusters/kas.md @@ -10,7 +10,7 @@ The Kubernetes Agent Server (KAS) is a GitLab backend service dedicated to managing [Kubernetes Agents](../../user/clusters/agent/index.md). The KAS is already installed and available in GitLab.com under `wss://kas.gitlab.com`. -See [how to use GitLab.com's KAS](../../user/clusters/agent/index.md#set-up-the-kubernetes-agent-server). +See [how to use GitLab.com's KAS](../../user/clusters/agent/install/index.md#set-up-the-kubernetes-agent-server). This document describes how to install a KAS for GitLab self-managed instances. ## Installation options diff --git a/doc/administration/compliance.md b/doc/administration/compliance.md index 1761af1ffa1..a05495c024e 100644 --- a/doc/administration/compliance.md +++ b/doc/administration/compliance.md @@ -6,29 +6,81 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Compliance features **(FREE)** -You can configure the following GitLab features to help ensure that your GitLab -instance meets common compliance standards. Click a feature name for additional -documentation. - -The [security features](../security/index.md) in GitLab may also help you meet -relevant compliance standards. - -| Feature | GitLab tier | GitLab SaaS | Product level | -|----------|:-----------:|:-----------:|:-------------:| -|**[Restrict SSH Keys](../security/ssh_keys_restrictions.md)**<br>Control the technology and key length of SSH keys used to access GitLab. | Free+ | **{dotted-circle}** No | Instance | -|**[Granular user roles and flexible permissions](../user/permissions.md)**<br>Manage access and permissions with five different user roles and settings for external users. Set permissions according to people's role, rather than either read or write access to a repository. Don't share the source code with people that only need access to the issue tracker. | Free+ | **{check-circle}** Yes | Instance, Group, Project | -|**[Generate reports on permission levels of users](../user/admin_area/index.md#user-permission-export)**<br>Administrators can generate a report listing all users' access permissions for groups and projects in the instance. | Premium+ | **{dotted-circle}** No | Instance | -|**[Enforce TOS acceptance](../user/admin_area/settings/terms.md)**<br>Enforce your users accepting new terms of service by blocking GitLab traffic. | Free+ | **{dotted-circle}** No | Instance | -|**[Email all users of a project, group, or entire server](../tools/email.md)**<br>An administrator can email groups of users based on project or group membership, or email everyone using the GitLab instance. This is great for scheduled maintenance or upgrades. | Premium+ | **{dotted-circle}** No | Instance | -|**[Omnibus package supports log forwarding](https://docs.gitlab.com/omnibus/settings/logs.html#udp-log-forwarding)**<br>Forward your logs to a central system. | Premium+ | **{dotted-circle}** No | Instance | -|**[Lock project membership to group](../user/group/index.md#prevent-members-from-being-added-to-a-group)**<br>Group owners can prevent new members from being added to projects within a group. | Premium+ | **{check-circle}** Yes | Group | -|**[LDAP group sync](auth/ldap/index.md#group-sync)**<br>GitLab Enterprise Edition gives administrators the ability to automatically sync groups and manage SSH keys, permissions, and authentication, so you can focus on building your product, not configuring your tools. | Premium+ | **{dotted-circle}** No | Instance | -|**[LDAP group sync filters](auth/ldap/index.md#group-sync)**<br>GitLab Enterprise Edition Premium gives more flexibility to synchronize with LDAP based on filters, meaning you can leverage LDAP attributes to map GitLab permissions. | Premium+ | **{dotted-circle}** No | Instance | -|**[Audit events](audit_events.md)**<br>To maintain the integrity of your code, GitLab Enterprise Edition Premium gives administrators the ability to view any modifications made within the GitLab server in an advanced audit events system, so you can control, analyze, and track every change. | Premium+ | **{check-circle}** Yes | Instance, Group, Project | -|**[Auditor users](auditor_users.md)**<br>Auditor users are users who are given read-only access to all projects, groups, and other resources on the GitLab instance. | Premium+ | **{dotted-circle}** No | Instance | -|**[Credentials inventory](../user/admin_area/credentials_inventory.md)**<br>With a credentials inventory, GitLab administrators can keep track of the credentials used by all of the users in their GitLab instance. | Ultimate | **{dotted-circle}** No | Instance | -|**Separation of Duties using [Protected branches](../user/project/protected_branches.md#require-code-owner-approval-on-a-protected-branch) and [custom CI Configuration Paths](../ci/pipelines/settings.md#specify-a-custom-cicd-configuration-file)**<br> GitLab Premium users can leverage the GitLab cross-project YAML configurations to define deployers of code and developers of code. View the [Separation of Duties Deploy Project](https://gitlab.com/guided-explorations/separation-of-duties-deploy/blob/master/README.md) and [Separation of Duties Project](https://gitlab.com/guided-explorations/separation-of-duties/blob/master/README.md) to see how to use this set up to define these roles. | Premium+ | **{check-circle}** Yes | Project | -|**[Compliance frameworks](../user/project/settings/index.md#compliance-frameworks)**<br>Create a custom compliance framework at the group level to describe the type of compliance requirements any child project needs to follow. | Premium+ | **{check-circle}** Yes | Group | -|**[Compliance pipelines](../user/project/settings/index.md#compliance-pipeline-configuration)**<br>Define a pipeline configuration to run for any projects with a given compliance framework. | Ultimate | **{check-circle}** Yes | Group | -|**[Compliance report](../user/compliance/compliance_report/index.md)**<br>Quickly get visibility into the compliance posture of your organization. | Ultimate | **{check-circle}** Yes | Group | -|**[External Status Checks](../user/project/merge_requests/status_checks.md)**<br>Interface with third party systems you already use during development to ensure you remain compliant. | Ultimate | **{check-circle}** Yes | Project | +These GitLab features can help ensure that your GitLab instance meets common compliance standards. For more information +about compliance management, see the compliance management [solutions page](https://about.gitlab.com/solutions/compliance/). + +The [security features](../security/index.md) in GitLab may also help you meet relevant compliance standards. + +## Policy management + +Organizations have unique policy requirements, either due to organizational standards or mandates from regulatory bodies. +The following features help you define rules and policies to adhere to workflow requirements, separation of duties, and +secure supply chain best practices: + +- [**Credentials inventory**](../user/admin_area/credentials_inventory.md) (for instances): With a credentials inventory, + GitLab administrators can keep track of the credentials used by all of the users in their GitLab instance. +- [**Granular user roles and flexible permissions**](../user/permissions.md) (for instances, groups, and projects): Manage + access and permissions with five different user roles and settings for external users. Set permissions according to people's + role, rather than either read or write access to a repository. Don't share the source code with people that only need + access to the issue tracker. +- [**Merge request approvals**](../user/project/merge_requests/approvals/index.md) (for instances, groups, and projects): + Configure approvals required for merge requests. +- [**Push rules**](../push_rules/push_rules.md) (for instances, groups, and projects): Control pushes to your + repositories. +- Separation of duties using [**protected branches**](../user/project/protected_branches.md#require-code-owner-approval-on-a-protected-branch) + and [**custom CI/CD configuration paths**](../ci/pipelines/settings.md#specify-a-custom-cicd-configuration-file) (for projects): + Users can leverage the GitLab cross-project YAML configurations to define deployers of code and developers of code. + See how to use this setup to define these roles in: + - The [Separation of Duties deploy project](https://gitlab.com/guided-explorations/separation-of-duties-deploy/blob/master/README.md). + - The [Separation of Duties project](https://gitlab.com/guided-explorations/separation-of-duties/blob/master/README.md). + +## Compliant workflow automation + +It is important for compliance teams to be confident that their controls and requirements are set up correctly, but also that they _stay_ set up correctly. One way of doing this is manually checking settings periodically, but this is error prone and time consuming. A better approach is to use single-source-of-truth settings and automation to ensure that whatever a compliance team has configured, stays configured and working correctly. These features can help you automate compliance: + +- [**Compliance frameworks**](../user/project/settings/index.md#compliance-frameworks) (for groups): Create a custom + compliance framework at the group level to describe the type of compliance requirements any child project needs to follow. +- [**Compliance pipelines**](../user/project/settings/index.md#compliance-pipeline-configuration) (for groups): Define a + pipeline configuration to run for any projects with a given compliance framework. + +## Audit management + +An important part of any compliance program is being able to go back and understand what happened, when +it happened, and who was responsible. This is useful in audit situations as well as for understanding +the root cause of issues when they occur. It is useful to have both low-level, raw lists of audit data +as well as high-level, summary lists of audit data. Between these two, compliance teams can quickly +identify if problems exist and then drill down into the specifics of those issues. These features can help provide visibility into GitLab and audit what is happening: + +- [**Audit events**](audit_events.md) (for instances, groups, and projects): To maintain the integrity of your code, + audit events give administrators the ability to view any modifications made within the GitLab + server in an advanced audit events system, so you can control, analyze, and track every change. +- [**Auditor users**](auditor_users.md) (for instances): Auditor users are users who are given read-only access to all + projects, groups, and other resources on the GitLab instance. +- [**Compliance report**](../user/compliance/compliance_report/index.md) (for groups): Quickly get visibility into the + compliance posture of your organization. + +## Other compliance features + +These features can also help with compliance requirements: + +- [**Email all users of a project, group, or entire server**](../tools/email.md) (for instances): An administrator can + email groups of users based on project or group membership, or email everyone using the GitLab instance. These emails + are great for scheduled maintenance or upgrades. +- [**Enforce ToS acceptance**](../user/admin_area/settings/terms.md) (for instances): Enforce your users accepting new + terms of service by blocking GitLab traffic. +- [**External Status Checks**](../user/project/merge_requests/status_checks.md) (for projects): Interface with third-party + systems you already use during development to ensure you remain compliant. +- [**Generate reports on permission levels of users**](../user/admin_area/index.md#user-permission-export) (for + instances): Administrators can generate a report listing all users' access permissions for groups and projects in the + instance. +- [**Lock project membership to group**](../user/group/index.md#prevent-members-from-being-added-to-a-group) (for + groups): Group owners can prevent new members from being added to projects within a group. +- [**LDAP group sync**](auth/ldap/ldap_synchronization.md#group-sync) (for instances): Gives administrators the ability + to automatically sync groups and manage SSH keys, permissions, and authentication, so you can focus on building your + product, not configuring your tools. +- [**LDAP group sync filters**](auth/ldap/ldap_synchronization.md#group-sync) (for instances): Gives more flexibility to + synchronize with LDAP based on filters, meaning you can leverage LDAP attributes to map GitLab permissions. +- [**Omnibus GitLab package supports log forwarding**](https://docs.gitlab.com/omnibus/settings/logs.html#udp-log-forwarding) + (for instances): Forward your logs to a central system. +- [**Restrict SSH Keys**](../security/ssh_keys_restrictions.md) (for instances): Control the technology and key length + of SSH keys used to access GitLab. diff --git a/doc/administration/environment_variables.md b/doc/administration/environment_variables.md index 3af80363916..21e32d145bd 100644 --- a/doc/administration/environment_variables.md +++ b/doc/administration/environment_variables.md @@ -20,7 +20,7 @@ You can use the following environment variables to override certain values: | Variable | Type | Description | |--------------------------------------------|---------|---------------------------------------------------------------------------------------------------------| | `DATABASE_URL` | string | The database URL; is of the form: `postgresql://localhost/blog_development`. | -| `ENABLE_BOOTSNAP` | string | Enables Bootsnap for speeding up initial Rails boot (`1` to enable). | +| `ENABLE_BOOTSNAP` | string | Toggles [Bootsnap](https://github.com/Shopify/bootsnap) for speeding up initial Rails boot. Enabled by default for non-production environments. Set to `0` to disable. | | `EXTERNAL_URL` | string | Specify the external URL at the [time of installation](https://docs.gitlab.com/omnibus/settings/configuration.html#specifying-the-external-url-at-the-time-of-installation). | | `EXTERNAL_VALIDATION_SERVICE_TIMEOUT` | integer | Timeout, in seconds, for an [external CI/CD pipeline validation service](external_pipeline_validation.md). Default is `5`. | | `EXTERNAL_VALIDATION_SERVICE_URL` | string | URL to an [external CI/CD pipeline validation service](external_pipeline_validation.md). | diff --git a/doc/administration/feature_flags.md b/doc/administration/feature_flags.md index f2067e7a2d1..afbf0759452 100644 --- a/doc/administration/feature_flags.md +++ b/doc/administration/feature_flags.md @@ -42,11 +42,15 @@ GitLab to an earlier version, the feature flag status may change. Features that are disabled by default may change or be removed without notice in a future version of GitLab. -Data corruption, stability degradation, or performance degradation might occur if +Data corruption, stability degradation, performance degradation, or security issues might occur if you enable a feature that's disabled by default. Problems caused by using a default disabled feature aren't covered by GitLab support, unless you were directed by GitLab to enable the feature. +Security issues found in features that are disabled by default are patched in regular releases +and do not follow our regular [maintenance policy](../policy/maintenance.md#security-releases) +with regards to backporting the fix. + ## Risks when disabling released features In most cases, the feature flag code is removed in a future version of GitLab. diff --git a/doc/administration/geo/disaster_recovery/bring_primary_back.md b/doc/administration/geo/disaster_recovery/bring_primary_back.md index d06f11bbe09..64673b9ee8d 100644 --- a/doc/administration/geo/disaster_recovery/bring_primary_back.md +++ b/doc/administration/geo/disaster_recovery/bring_primary_back.md @@ -5,27 +5,27 @@ info: To determine the technical writer assigned to the Stage/Group associated w type: howto --- -# Bring a demoted primary node back online **(PREMIUM SELF)** +# Bring a demoted primary site back online **(PREMIUM SELF)** -After a failover, it is possible to fail back to the demoted **primary** node to +After a failover, it is possible to fail back to the demoted **primary** site to restore your original configuration. This process consists of two steps: -1. Making the old **primary** node a **secondary** node. -1. Promoting a **secondary** node to a **primary** node. +1. Making the old **primary** site a **secondary** site. +1. Promoting a **secondary** site to a **primary** site. WARNING: -If you have any doubts about the consistency of the data on this node, we recommend setting it up from scratch. +If you have any doubts about the consistency of the data on this site, we recommend setting it up from scratch. -## Configure the former **primary** node to be a **secondary** node +## Configure the former **primary** site to be a **secondary** site -Since the former **primary** node will be out of sync with the current **primary** node, the first step is to bring the former **primary** node up to date. Note, deletion of data stored on disk like -repositories and uploads will not be replayed when bringing the former **primary** node back +Since the former **primary** site will be out of sync with the current **primary** site, the first step is to bring the former **primary** site up to date. Note, deletion of data stored on disk like +repositories and uploads will not be replayed when bringing the former **primary** site back into sync, which may result in increased disk usage. Alternatively, you can [set up a new **secondary** GitLab instance](../setup/index.md) to avoid this. -To bring the former **primary** node up to date: +To bring the former **primary** site up to date: -1. SSH into the former **primary** node that has fallen behind. +1. SSH into the former **primary** site that has fallen behind. 1. Make sure all the services are up: ```shell @@ -33,36 +33,36 @@ To bring the former **primary** node up to date: ``` NOTE: - If you [disabled the **primary** node permanently](index.md#step-2-permanently-disable-the-primary-node), + If you [disabled the **primary** site permanently](index.md#step-2-permanently-disable-the-primary-site), you need to undo those steps now. For Debian/Ubuntu you just need to run `sudo systemctl enable gitlab-runsvdir`. For CentOS 6, you need to install - the GitLab instance from scratch and set it up as a **secondary** node by + the GitLab instance from scratch and set it up as a **secondary** site by following [Setup instructions](../setup/index.md). In this case, you don't need to follow the next step. NOTE: If you [changed the DNS records](index.md#step-4-optional-updating-the-primary-domain-dns-record) - for this node during disaster recovery procedure you may need to [block - all the writes to this node](planned_failover.md#prevent-updates-to-the-primary-node) + for this site during disaster recovery procedure you may need to [block + all the writes to this site](planned_failover.md#prevent-updates-to-the-primary-node) during this procedure. -1. [Set up database replication](../setup/database.md). In this case, the **secondary** node - refers to the former **primary** node. - 1. If [PgBouncer](../../postgresql/pgbouncer.md) was enabled on the **current secondary** node - (when it was a primary node) disable it by editing `/etc/gitlab/gitlab.rb` +1. [Set up database replication](../setup/database.md). In this case, the **secondary** site + refers to the former **primary** site. + 1. If [PgBouncer](../../postgresql/pgbouncer.md) was enabled on the **current secondary** site + (when it was a primary site) disable it by editing `/etc/gitlab/gitlab.rb` and running `sudo gitlab-ctl reconfigure`. - 1. You can then set up database replication on the **secondary** node. + 1. You can then set up database replication on the **secondary** site. -If you have lost your original **primary** node, follow the -[setup instructions](../setup/index.md) to set up a new **secondary** node. +If you have lost your original **primary** site, follow the +[setup instructions](../setup/index.md) to set up a new **secondary** site. -## Promote the **secondary** node to **primary** node +## Promote the **secondary** site to **primary** site -When the initial replication is complete and the **primary** node and **secondary** node are +When the initial replication is complete and the **primary** site and **secondary** site are closely in sync, you can do a [planned failover](planned_failover.md). -## Restore the **secondary** node +## Restore the **secondary** site -If your objective is to have two nodes again, you need to bring your **secondary** -node back online as well by repeating the first step -([configure the former **primary** node to be a **secondary** node](#configure-the-former-primary-node-to-be-a-secondary-node)) -for the **secondary** node. +If your objective is to have two sites again, you need to bring your **secondary** +site back online as well by repeating the first step +([configure the former **primary** site to be a **secondary** site](#configure-the-former-primary-site-to-be-a-secondary-site)) +for the **secondary** site. diff --git a/doc/administration/geo/disaster_recovery/index.md b/doc/administration/geo/disaster_recovery/index.md index f6f88e9b193..bf28eb76ffd 100644 --- a/doc/administration/geo/disaster_recovery/index.md +++ b/doc/administration/geo/disaster_recovery/index.md @@ -16,36 +16,36 @@ For the latest updates, check the [Disaster Recovery epic for complete maturity] Multi-secondary configurations require the complete re-synchronization and re-configuration of all non-promoted secondaries and causes downtime. -## Promoting a **secondary** Geo node in single-secondary configurations +## Promoting a **secondary** Geo site in single-secondary configurations We don't currently provide an automated way to promote a Geo replica and do a failover, but you can do it manually if you have `root` access to the machine. -This process promotes a **secondary** Geo node to a **primary** node. To regain -geographic redundancy as quickly as possible, you should add a new **secondary** node +This process promotes a **secondary** Geo site to a **primary** site. To regain +geographic redundancy as quickly as possible, you should add a new **secondary** site immediately after following these instructions. ### Step 1. Allow replication to finish if possible -If the **secondary** node is still replicating data from the **primary** node, follow +If the **secondary** site is still replicating data from the **primary** site, follow [the planned failover docs](planned_failover.md) as closely as possible in order to avoid unnecessary data loss. -### Step 2. Permanently disable the **primary** node +### Step 2. Permanently disable the **primary** site WARNING: -If the **primary** node goes offline, there may be data saved on the **primary** node -that have not been replicated to the **secondary** node. This data should be treated +If the **primary** site goes offline, there may be data saved on the **primary** site +that have not been replicated to the **secondary** site. This data should be treated as lost if you proceed. -If an outage on the **primary** node happens, you should do everything possible to +If an outage on the **primary** site happens, you should do everything possible to avoid a split-brain situation where writes can occur in two different GitLab instances, complicating recovery efforts. So to prepare for the failover, we -must disable the **primary** node. +must disable the **primary** site. - If you have SSH access: - 1. SSH into the **primary** node to stop and disable GitLab: + 1. SSH into the **primary** site to stop and disable GitLab: ```shell sudo gitlab-ctl stop @@ -57,35 +57,35 @@ must disable the **primary** node. sudo systemctl disable gitlab-runsvdir ``` -- If you do not have SSH access to the **primary** node, take the machine offline and +- If you do not have SSH access to the **primary** site, take the machine offline and prevent it from rebooting by any means at your disposal. You might need to: - Reconfigure the load balancers. - Change DNS records (for example, point the primary DNS record to the - **secondary** node to stop usage of the **primary** node). + **secondary** site to stop usage of the **primary** site). - Stop the virtual servers. - Block traffic through a firewall. - - Revoke object storage permissions from the **primary** node. + - Revoke object storage permissions from the **primary** site. - Physically disconnect a machine. If you plan to [update the primary domain DNS record](#step-4-optional-updating-the-primary-domain-dns-record), you may wish to lower the TTL now to speed up propagation. -### Step 3. Promoting a **secondary** node +### Step 3. Promoting a **secondary** site WARNING: -In GitLab 13.2 and 13.3, promoting a secondary node to a primary while the +In GitLab 13.2 and 13.3, promoting a secondary site to a primary while the secondary is paused fails. Do not pause replication before promoting a -secondary. If the node is paused, be sure to resume before promoting. +secondary. If the secondary site is paused, be sure to resume before promoting. This issue has been fixed in GitLab 13.4 and later. Note the following when promoting a secondary: -- If replication was paused on the secondary node (for example as a part of +- If replication was paused on the secondary site (for example as a part of upgrading, while you were running a version of GitLab earlier than 13.4), you - _must_ [enable the node by using the database](../replication/troubleshooting.md#message-activerecordrecordinvalid-validation-failed-enabled-geo-primary-node-cannot-be-disabled) - before proceeding. If the secondary node + _must_ [enable the site by using the database](../replication/troubleshooting.md#message-activerecordrecordinvalid-validation-failed-enabled-geo-primary-node-cannot-be-disabled) + before proceeding. If the secondary site [has been paused](../../geo/index.md#pausing-and-resuming-replication), the promotion performs a point-in-time recovery to the last known state. Data that was created on the primary while the secondary was paused is lost. @@ -99,7 +99,32 @@ Note the following when promoting a secondary: for more information, see this [troubleshooting advice](../replication/troubleshooting.md#errors-when-using---skip-preflight-checks-or---force). -#### Promoting a **secondary** node running on a single machine +#### Promoting a **secondary** site running on a single node running GitLab 14.5 and later + +1. SSH in to your **secondary** node and execute: + + - To promote the secondary node to primary: + + ```shell + sudo gitlab-ctl geo promote + ``` + + - To promote the secondary node to primary **without any further confirmation**: + + ```shell + sudo gitlab-ctl geo promote --force + ``` + +1. Verify you can connect to the newly-promoted **primary** site using the URL used + previously for the **secondary** site. +1. If successful, the **secondary** site is now promoted to the **primary** site. + +#### Promoting a **secondary** site running on a single node running GitLab 14.4 and earlier + +WARNING: +The `gitlab-ctl promote-to-primary-node` and `gitlab-ctl promoted-db` commands are +deprecated in GitLab 14.5 and later, and are scheduled to [be removed in GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/345207). +Use `gitlab-ctl geo promote` instead. 1. SSH in to your **secondary** node and login as root: @@ -116,7 +141,7 @@ Note the following when promoting a secondary: roles ['geo_secondary_role'] ``` -1. Promote the **secondary** node to the **primary** node: +1. Promote the **secondary** site to the **primary** site: - To promote the secondary node to primary along with [preflight checks](planned_failover.md#preflight-checks): @@ -146,18 +171,57 @@ Note the following when promoting a secondary: gitlab-ctl promote-to-primary-node --force ``` -1. Verify you can connect to the newly-promoted **primary** node using the URL used - previously for the **secondary** node. -1. If successful, the **secondary** node is now promoted to the **primary** node. +1. Verify you can connect to the newly-promoted **primary** site using the URL used + previously for the **secondary** site. +1. If successful, the **secondary** site is now promoted to the **primary** site. -#### Promoting a **secondary** node with multiple servers +#### Promoting a **secondary** site with multiple nodes running GitLab 14.5 and later + +1. SSH to every Sidekiq, PostgresSQL, and Gitaly node in the **secondary** site and run one of the following commands: + + - To promote the secondary node to primary: + + ```shell + sudo gitlab-ctl geo promote + ``` + + - To promote the secondary node to primary **without any further confirmation**: + + ```shell + sudo gitlab-ctl geo promote --force + ``` + +1. SSH into each Rails node on your **secondary** site and run one of the following commands: + + - To promote the secondary node to primary: + + ```shell + sudo gitlab-ctl geo promote + ``` + + - To promote the secondary node to primary **without any further confirmation**: + + ```shell + sudo gitlab-ctl geo promote --force + ``` + +1. Verify you can connect to the newly-promoted **primary** site using the URL used + previously for the **secondary** site. +1. If successful, the **secondary** site is now promoted to the **primary** site. + +#### Promoting a **secondary** site with multiple nodes running GitLab 14.4 and earlier + +WARNING: +The `gitlab-ctl promote-to-primary-node` and `gitlab-ctl promoted-db` commands are +deprecated in GitLab 14.5 and later, and are scheduled to [be removed in GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/345207). +Use `gitlab-ctl geo promote` instead. The `gitlab-ctl promote-to-primary-node` command cannot be used yet in -conjunction with multiple servers, as it can only -perform changes on a **secondary** with only a single machine. Instead, you must +conjunction with multiple nodes, as it can only perform changes on +a **secondary** with only a single node. Instead, you must do this manually. -1. SSH in to the database node in the **secondary** and trigger PostgreSQL to +1. SSH in to the database node in the **secondary** site and trigger PostgreSQL to promote to read-write: ```shell @@ -187,16 +251,54 @@ do this manually. 1. Verify you can connect to the newly-promoted **primary** using the URL used previously for the **secondary**. -1. If successful, the **secondary** node is now promoted to the **primary** node. +1. If successful, the **secondary** site is now promoted to the **primary** site. + +#### Promoting a **secondary** site with a Patroni standby cluster running GitLab 14.5 and later + +1. SSH to every Sidekiq, PostgresSQL, and Gitaly node in the **secondary** site and run one of the following commands: + + - To promote the secondary node to primary: + + ```shell + sudo gitlab-ctl geo promote + ``` + + - To promote the secondary node to primary **without any further confirmation**: + + ```shell + sudo gitlab-ctl geo promote --force + ``` + +1. SSH into each Rails node on your **secondary** site and run one of the following commands: -#### Promoting a **secondary** node with a Patroni standby cluster + - To promote the secondary node to primary: + + ```shell + sudo gitlab-ctl geo promote + ``` + + - To promote the secondary node to primary **without any further confirmation**: + + ```shell + sudo gitlab-ctl geo promote --force + ``` + +1. Verify you can connect to the newly-promoted **primary** site using the URL used + previously for the **secondary** site. +1. If successful, the **secondary** site is now promoted to the **primary** site. + +#### Promoting a **secondary** site with a Patroni standby cluster running GitLab 14.4 and earlier + +WARNING: +The `gitlab-ctl promote-to-primary-node` and `gitlab-ctl promoted-db` commands are +deprecated in GitLab 14.5 and later, and are scheduled to [be removed in GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/345207). +Use `gitlab-ctl geo promote` instead. The `gitlab-ctl promote-to-primary-node` command cannot be used yet in -conjunction with a Patroni standby cluster, as it can only -perform changes on a **secondary** with only a single machine. Instead, you must -do this manually. +conjunction with a Patroni standby cluster, as it can only perform changes on +a **secondary** with only a single node. Instead, you must do this manually. -1. SSH in to the Standby Leader database node in the **secondary** and trigger PostgreSQL to +1. SSH in to the Standby Leader database node in the **secondary** site and trigger PostgreSQL to promote to read-write: ```shell @@ -230,9 +332,81 @@ do this manually. 1. Verify you can connect to the newly-promoted **primary** using the URL used previously for the **secondary**. -1. If successful, the **secondary** node is now promoted to the **primary** node. +1. If successful, the **secondary** site is now promoted to the **primary** site. + +#### Promoting a **secondary** site with an external PostgreSQL database running GitLab 14.5 and later + +The `gitlab-ctl geo promote` command can be used in conjunction with +an external PostgreSQL database, but it can only perform changes on +a **secondary** PostgreSQL database managed by Omnibus. +You must promote the replica database associated with the **secondary** +site first. + +1. Promote the replica database associated with the **secondary** site. This + sets the database to read-write. The instructions vary depending on where your database is hosted: + - [Amazon RDS](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ReadRepl.html#USER_ReadRepl.Promote) + - [Azure PostgreSQL](https://docs.microsoft.com/en-us/azure/postgresql/howto-read-replicas-portal#stop-replication) + - [Google Cloud SQL](https://cloud.google.com/sql/docs/mysql/replication/manage-replicas#promote-replica) + - For other external PostgreSQL databases, save the following script in your + secondary node, for example `/tmp/geo_promote.sh`, and modify the connection + parameters to match your environment. Then, execute it to promote the replica: + + ```shell + #!/bin/bash -#### Promoting a **secondary** node with an external PostgreSQL database + PG_SUPERUSER=postgres + + # The path to your pg_ctl binary. You may need to adjust this path to match + # your PostgreSQL installation + PG_CTL_BINARY=/usr/lib/postgresql/10/bin/pg_ctl + + # The path to your PostgreSQL data directory. You may need to adjust this + # path to match your PostgreSQL installation. You can also run + # `SHOW data_directory;` from PostgreSQL to find your data directory + PG_DATA_DIRECTORY=/etc/postgresql/10/main + + # Promote the PostgreSQL database and allow read/write operations + sudo -u $PG_SUPERUSER $PG_CTL_BINARY -D $PG_DATA_DIRECTORY promote + ``` + +1. SSH to every Sidekiq, PostgresSQL, and Gitaly node in the **secondary** site and run one of the following commands: + + - To promote the secondary node to primary: + + ```shell + sudo gitlab-ctl geo promote + ``` + + - To promote the secondary node to primary **without any further confirmation**: + + ```shell + sudo gitlab-ctl geo promote --force + ``` + +1. SSH into each Rails node on your **secondary** site and run one of the following commands: + + - To promote the secondary node to primary: + + ```shell + sudo gitlab-ctl geo promote + ``` + + - To promote the secondary node to primary **without any further confirmation**: + + ```shell + sudo gitlab-ctl geo promote --force + ``` + +1. Verify you can connect to the newly-promoted **primary** site using the URL used + previously for the **secondary** site. +1. If successful, the **secondary** site is now promoted to the **primary** site. + +#### Promoting a **secondary** site with an external PostgreSQL database running GitLab 14.4 and earlier + +WARNING: +The `gitlab-ctl promote-to-primary-node` and `gitlab-ctl promoted-db` commands are +deprecated in GitLab 14.5 and later, and are scheduled to [be removed in GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/345207). +Use `gitlab-ctl geo promote` instead. The `gitlab-ctl promote-to-primary-node` command cannot be used in conjunction with an external PostgreSQL database, as it can only perform changes on a **secondary** @@ -287,23 +461,23 @@ required: 1. Verify you can connect to the newly-promoted **primary** using the URL used previously for the **secondary**. -1. If successful, the **secondary** node is now promoted to the **primary** node. +1. If successful, the **secondary** site is now promoted to the **primary** site. ### Step 4. (Optional) Updating the primary domain DNS record -Updating the DNS records for the primary domain to point to the **secondary** node +Updating the DNS records for the primary domain to point to the **secondary** site to prevent the need to update all references to the primary domain to the secondary domain, like changing Git remotes and API URLs. -1. SSH into the **secondary** node and login as root: +1. SSH into the **secondary** site and login as root: ```shell sudo -i ``` 1. Update the primary domain's DNS record. After updating the primary domain's - DNS records to point to the **secondary** node, edit `/etc/gitlab/gitlab.rb` on the - **secondary** node to reflect the new URL: + DNS records to point to the **secondary** site, edit `/etc/gitlab/gitlab.rb` on the + **secondary** site to reflect the new URL: ```ruby # Change the existing external_url configuration @@ -314,13 +488,13 @@ secondary domain, like changing Git remotes and API URLs. Changing `external_url` does not prevent access via the old secondary URL, as long as the secondary DNS records are still intact. -1. Reconfigure the **secondary** node for the change to take effect: +1. Reconfigure the **secondary** site for the change to take effect: ```shell gitlab-ctl reconfigure ``` -1. Execute the command below to update the newly promoted **primary** node URL: +1. Execute the command below to update the newly promoted **primary** site URL: ```shell gitlab-rake geo:update_primary_node_url @@ -335,14 +509,14 @@ secondary domain, like changing Git remotes and API URLs. To determine if you need to do this, search for the `gitlab_rails["geo_node_name"]` setting in your `/etc/gitlab/gitlab.rb` file. If it is commented out with `#` or not found at all, then you - need to update the **primary** node's name in the database. You can search for it + need to update the **primary** site's name in the database. You can search for it like so: ```shell grep "geo_node_name" /etc/gitlab/gitlab.rb ``` - To update the **primary** node's name in the database: + To update the **primary** site's name in the database: ```shell gitlab-rails runner 'Gitlab::Geo.primary_node.update!(name: GeoNode.current_node_name)' @@ -352,12 +526,12 @@ secondary domain, like changing Git remotes and API URLs. If you updated the DNS records for the primary domain, these changes may not have yet propagated depending on the previous DNS records TTL. -### Step 5. (Optional) Add **secondary** Geo node to a promoted **primary** node +### Step 5. (Optional) Add **secondary** Geo site to a promoted **primary** site -Promoting a **secondary** node to **primary** node using the process above does not enable -Geo on the new **primary** node. +Promoting a **secondary** site to **primary** site using the process above does not enable +Geo on the new **primary** site. -To bring a new **secondary** node online, follow the [Geo setup instructions](../index.md#setup-instructions). +To bring a new **secondary** site online, follow the [Geo setup instructions](../index.md#setup-instructions). ### Step 6. (Optional) Removing the secondary's tracking database @@ -376,13 +550,13 @@ for the changes to take effect. ## Promoting secondary Geo replica in multi-secondary configurations -If you have more than one **secondary** node and you need to promote one of them, we suggest you follow -[Promoting a **secondary** Geo node in single-secondary configurations](#promoting-a-secondary-geo-node-in-single-secondary-configurations) +If you have more than one **secondary** site and you need to promote one of them, we suggest you follow +[Promoting a **secondary** Geo site in single-secondary configurations](#promoting-a-secondary-geo-site-in-single-secondary-configurations) and after that you also need two extra steps. -### Step 1. Prepare the new **primary** node to serve one or more **secondary** nodes +### Step 1. Prepare the new **primary** site to serve one or more **secondary** sites -1. SSH into the new **primary** node and login as root: +1. SSH into the new **primary** site and login as root: ```shell sudo -i @@ -442,13 +616,13 @@ and after that you also need two extra steps. ### Step 2. Initiate the replication process -Now we need to make each **secondary** node listen to changes on the new **primary** node. To do that you need +Now we need to make each **secondary** site listen to changes on the new **primary** site. To do that you need to [initiate the replication process](../setup/database.md#step-3-initiate-the-replication-process) again but this time -for another **primary** node. All the old replication settings are overwritten. +for another **primary** site. All the old replication settings are overwritten. ## Promoting a secondary Geo cluster in GitLab Cloud Native Helm Charts -When updating a Cloud Native Geo deployment, the process for updating any node that is external to the secondary Kubernetes cluster does not differ from the non Cloud Native approach. As such, you can always defer to [Promoting a secondary Geo node in single-secondary configurations](#promoting-a-secondary-geo-node-in-single-secondary-configurations) for more information. +When updating a Cloud Native Geo deployment, the process for updating any node that is external to the secondary Kubernetes cluster does not differ from the non Cloud Native approach. As such, you can always defer to [Promoting a secondary Geo site in single-secondary configurations](#promoting-a-secondary-geo-site-in-single-secondary-configurations) for more information. The following sections assume you are using the `gitlab` namespace. If you used a different namespace when setting up your cluster, you should also replace `--namespace gitlab` with your namespace. @@ -489,13 +663,45 @@ must disable the **primary** site: - Revoke object storage permissions from the **primary** site. - Physically disconnect a machine. -### Step 2. Promote all **secondary** nodes external to the cluster +### Step 2. Promote all **secondary** sites external to the cluster WARNING: If the secondary site [has been paused](../../geo/index.md#pausing-and-resuming-replication), this performs a point-in-time recovery to the last known state. Data that was created on the primary while the secondary was paused is lost. +If you are running GitLab 14.5 and later: + +1. SSH to every Sidekiq, PostgresSQL, and Gitaly node in the **secondary** site and run one of the following commands: + + - To promote the secondary node to primary: + + ```shell + sudo gitlab-ctl geo promote + ``` + + - To promote the secondary node to primary **without any further confirmation**: + + ```shell + sudo gitlab-ctl geo promote --force + ``` + +1. SSH into each Rails node on your **secondary** site and run one of the following commands: + + - To promote the secondary node to primary: + + ```shell + sudo gitlab-ctl geo promote + ``` + + - To promote the secondary node to primary **without any further confirmation**: + + ```shell + sudo gitlab-ctl geo promote --force + ``` + +If you are running GitLab 14.4 and earlier: + 1. SSH in to the database node in the **secondary** and trigger PostgreSQL to promote to read-write: @@ -522,8 +728,6 @@ Data that was created on the primary while the secondary was paused is lost. After making these changes, [reconfigure GitLab](../../restart_gitlab.md#omnibus-gitlab-reconfigure) on the database node. -### Step 3. Promote the **secondary** cluster - 1. Find the task runner pod: ```shell @@ -536,6 +740,8 @@ Data that was created on the primary while the secondary was paused is lost. kubectl --namespace gitlab exec -ti gitlab-geo-task-runner-XXX -- gitlab-rake geo:set_secondary_as_primary ``` +### Step 3. Promote the **secondary** cluster + 1. Update the existing cluster configuration. You can retrieve the existing configuration with Helm: diff --git a/doc/administration/geo/disaster_recovery/planned_failover.md b/doc/administration/geo/disaster_recovery/planned_failover.md index 6312ed669ae..939b4aec968 100644 --- a/doc/administration/geo/disaster_recovery/planned_failover.md +++ b/doc/administration/geo/disaster_recovery/planned_failover.md @@ -204,4 +204,4 @@ in the loss of any data uploaded to the new **primary** in the meantime. Don't forget to remove the broadcast message after the failover is complete. -Finally, you can bring the [old site back as a secondary](bring_primary_back.md#configure-the-former-primary-node-to-be-a-secondary-node). +Finally, you can bring the [old site back as a secondary](bring_primary_back.md#configure-the-former-primary-site-to-be-a-secondary-site). diff --git a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md index 3eb7bc2a8e0..3c7af309f78 100644 --- a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md +++ b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md @@ -66,13 +66,13 @@ promote a Geo replica and perform a failover. NOTE: GitLab 13.9 through GitLab 14.3 are affected by a bug in which the Geo secondary site statuses will appear to stop updating and become unhealthy. For more information, see [Geo Admin Area shows 'Unhealthy' after enabling Maintenance Mode](../../replication/troubleshooting.md#geo-admin-area-shows-unhealthy-after-enabling-maintenance-mode). -On the **secondary** node: +On the **secondary** site: 1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Geo > Nodes** to see its status. Replicated objects (shown in green) should be close to 100%, and there should be no failures (shown in red). If a large proportion of - objects aren't yet replicated (shown in gray), consider giving the node more + objects aren't yet replicated (shown in gray), consider giving the site more time to complete.  @@ -85,20 +85,20 @@ You can use the [Geo status API](../../../../api/geo_nodes.md#retrieve-project-sync-or-verification-failures-that-occurred-on-the-current-node) to review failed objects and the reasons for failure. A common cause of replication failures is the data being missing on the -**primary** node - you can resolve these failures by restoring the data from backup, +**primary** site - you can resolve these failures by restoring the data from backup, or removing references to the missing data. The maintenance window won't end until Geo replication and verification is completely finished. To keep the window as short as possible, you should ensure these processes are close to 100% as possible during active use. -If the **secondary** node is still replicating data from the **primary** node, +If the **secondary** site is still replicating data from the **primary** site, follow these steps to avoid unnecessary data loss: 1. Until a [read-only mode](https://gitlab.com/gitlab-org/gitlab/-/issues/14609) is implemented, updates must be prevented from happening manually to the - **primary**. Your **secondary** node still needs read-only - access to the **primary** node during the maintenance window: + **primary**. Your **secondary** site still needs read-only + access to the **primary** site during the maintenance window: 1. At the scheduled time, using your cloud provider or your node's firewall, block all HTTP, HTTPS and SSH traffic to/from the **primary** node, **except** for your IP and @@ -121,18 +121,18 @@ follow these steps to avoid unnecessary data loss: ``` From this point, users are unable to view their data or make changes on the - **primary** node. They are also unable to log in to the **secondary** node. + **primary** site. They are also unable to log in to the **secondary** site. However, existing sessions need to work for the remainder of the maintenance period, and so public data is accessible throughout. - 1. Verify the **primary** node is blocked to HTTP traffic by visiting it in browser via + 1. Verify the **primary** site is blocked to HTTP traffic by visiting it in browser via another IP. The server should refuse connection. - 1. Verify the **primary** node is blocked to Git over SSH traffic by attempting to pull an + 1. Verify the **primary** site is blocked to Git over SSH traffic by attempting to pull an existing Git repository with an SSH remote URL. The server should refuse connection. - 1. On the **primary** node: + 1. On the **primary** site: 1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Monitoring > Background Jobs**. 1. On the Sidekiq dhasboard, select **Cron**. @@ -150,7 +150,7 @@ follow these steps to avoid unnecessary data loss: 1. If you are manually replicating any [data not managed by Geo](../../replication/datatypes.md#limitations-on-replicationverification), trigger the final replication process now. - 1. On the **primary** node: + 1. On the **primary** site: 1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Monitoring > Background Jobs**. 1. On the Sidekiq dashboard, select **Queues**, and wait for all queues except @@ -165,7 +165,7 @@ follow these steps to avoid unnecessary data loss: - Database replication lag is 0ms. - The Geo log cursor is up to date (0 events behind). - 1. On the **secondary** node: + 1. On the **secondary** site: 1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Monitoring > Background Jobs**. 1. On the Sidekiq dashboard, select **Queues**, and wait for all the `geo` @@ -173,14 +173,14 @@ follow these steps to avoid unnecessary data loss: 1. [Run an integrity check](../../../raketasks/check.md) to verify the integrity of CI artifacts, LFS objects, and uploads in file storage. - At this point, your **secondary** node contains an up-to-date copy of everything the - **primary** node has, meaning nothing is lost when you fail over. + At this point, your **secondary** site contains an up-to-date copy of everything the + **primary** site has, meaning nothing is lost when you fail over. -1. In this final step, you need to permanently disable the **primary** node. +1. In this final step, you need to permanently disable the **primary** site. WARNING: - When the **primary** node goes offline, there may be data saved on the **primary** node - that has not been replicated to the **secondary** node. This data should be treated + When the **primary** site goes offline, there may be data saved on the **primary** site + that has not been replicated to the **secondary** site. This data should be treated as lost if you proceed. NOTE: @@ -189,9 +189,9 @@ follow these steps to avoid unnecessary data loss: When performing a failover, we want to avoid a split-brain situation where writes can occur in two different GitLab instances. So to prepare for the - failover, you must disable the **primary** node: + failover, you must disable the **primary** site: - - If you have SSH access to the **primary** node, stop and disable GitLab: + - If you have SSH access to the **primary** site, stop and disable GitLab: ```shell sudo gitlab-ctl stop @@ -214,19 +214,58 @@ follow these steps to avoid unnecessary data loss: from starting if the machine reboots as `root` with `initctl stop gitlab-runsvvdir && echo 'manual' > /etc/init/gitlab-runsvdir.override && initctl reload-configuration`. - - If you do not have SSH access to the **primary** node, take the machine offline and + - If you do not have SSH access to the **primary** site, take the machine offline and prevent it from rebooting. Since there are many ways you may prefer to accomplish this, we avoid a single recommendation. You may need to: - Reconfigure the load balancers. - Change DNS records (for example, point the **primary** DNS record to the - **secondary** node to stop using the **primary** node). + **secondary** site to stop using the **primary** site). - Stop the virtual servers. - Block traffic through a firewall. - - Revoke object storage permissions from the **primary** node. + - Revoke object storage permissions from the **primary** site. - Physically disconnect a machine. -### Promoting the **secondary** node +### Promoting the **secondary** site running GitLab 14.5 and later + +1. SSH to every Sidekiq, PostgresSQL, and Gitaly node in the **secondary** site and run one of the following commands: + + - To promote the secondary node to primary: + + ```shell + sudo gitlab-ctl geo promote + ``` + + - To promote the secondary node to primary **without any further confirmation**: + + ```shell + sudo gitlab-ctl geo promote --force + ``` + +1. SSH into each Rails node on your **secondary** site and run one of the following commands: + + - To promote the secondary node to primary: + + ```shell + sudo gitlab-ctl geo promote + ``` + + - To promote the secondary node to primary **without any further confirmation**: + + ```shell + sudo gitlab-ctl geo promote --force + ``` + +1. Verify you can connect to the newly promoted **primary** site using the URL used + previously for the **secondary** site. +1. If successful, the **secondary** site is now promoted to the **primary** site. + +### Promoting the **secondary** site running GitLab 14.4 and earlier + +WARNING: +The `gitlab-ctl promote-to-primary-node` and `gitlab-ctl promoted-db` commands are +deprecated in GitLab 14.5 and later, and are scheduled to [be removed in GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/345207). +Use `gitlab-ctl geo promote` instead. NOTE: A new **secondary** should not be added at this time. If you want to add a new @@ -243,13 +282,13 @@ perform changes on a **secondary** with only a single machine. Instead, you must do this manually. WARNING: -In GitLab 13.2 and 13.3, promoting a secondary node to a primary while the +In GitLab 13.2 and 13.3, promoting a secondary site to a primary while the secondary is paused fails. Do not pause replication before promoting a -secondary. If the node is paused, be sure to resume before promoting. This +secondary. If the site is paused, be sure to resume before promoting. This issue has been fixed in GitLab 13.4 and later. WARNING: - If the secondary node [has been paused](../../../geo/index.md#pausing-and-resuming-replication), this performs +If the secondary site [has been paused](../../../geo/index.md#pausing-and-resuming-replication), this performs a point-in-time recovery to the last known state. Data that was created on the primary while the secondary was paused is lost. @@ -291,6 +330,6 @@ Data that was created on the primary while the secondary was paused is lost. ### Next steps To regain geographic redundancy as quickly as possible, you should -[add a new **secondary** node](../../setup/index.md). To +[add a new **secondary** site](../../setup/index.md). To do that, you can re-add the old **primary** as a new secondary and bring it back online. diff --git a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md index d4782144df8..8a4f2ed4306 100644 --- a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md +++ b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md @@ -54,10 +54,10 @@ promote a Geo replica and perform a failover. NOTE: GitLab 13.9 through GitLab 14.3 are affected by a bug in which the Geo secondary site statuses will appear to stop updating and become unhealthy. For more information, see [Geo Admin Area shows 'Unhealthy' after enabling Maintenance Mode](../../replication/troubleshooting.md#geo-admin-area-shows-unhealthy-after-enabling-maintenance-mode). -On the **secondary** node, navigate to the **Admin Area > Geo** dashboard to +On the **secondary** site, navigate to the **Admin Area > Geo** dashboard to review its status. Replicated objects (shown in green) should be close to 100%, and there should be no failures (shown in red). If a large proportion of -objects aren't yet replicated (shown in gray), consider giving the node more +objects aren't yet replicated (shown in gray), consider giving the site more time to complete.  @@ -70,20 +70,20 @@ You can use the [Geo status API](../../../../api/geo_nodes.md#retrieve-project-sync-or-verification-failures-that-occurred-on-the-current-node) to review failed objects and the reasons for failure. A common cause of replication failures is the data being missing on the -**primary** node - you can resolve these failures by restoring the data from backup, +**primary** site - you can resolve these failures by restoring the data from backup, or removing references to the missing data. The maintenance window won't end until Geo replication and verification is completely finished. To keep the window as short as possible, you should ensure these processes are close to 100% as possible during active use. -If the **secondary** node is still replicating data from the **primary** node, +If the **secondary** site is still replicating data from the **primary** site, follow these steps to avoid unnecessary data loss: 1. Until a [read-only mode](https://gitlab.com/gitlab-org/gitlab/-/issues/14609) is implemented, updates must be prevented from happening manually to the - **primary**. Your **secondary** node still needs read-only - access to the **primary** node during the maintenance window: + **primary**. Your **secondary** site still needs read-only + access to the **primary** site during the maintenance window: 1. At the scheduled time, using your cloud provider or your node's firewall, block all HTTP, HTTPS and SSH traffic to/from the **primary** node, **except** for your IP and @@ -106,18 +106,18 @@ follow these steps to avoid unnecessary data loss: ``` From this point, users are unable to view their data or make changes on the - **primary** node. They are also unable to log in to the **secondary** node. + **primary** site. They are also unable to log in to the **secondary** site. However, existing sessions need to work for the remainder of the maintenance period, and so public data is accessible throughout. - 1. Verify the **primary** node is blocked to HTTP traffic by visiting it in browser via + 1. Verify the **primary** site is blocked to HTTP traffic by visiting it in browser via another IP. The server should refuse connection. - 1. Verify the **primary** node is blocked to Git over SSH traffic by attempting to pull an + 1. Verify the **primary** site is blocked to Git over SSH traffic by attempting to pull an existing Git repository with an SSH remote URL. The server should refuse connection. - 1. On the **primary** node: + 1. On the **primary** site: 1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Monitoring > Background Jobs**. 1. On the Sidekiq dhasboard, select **Cron**. @@ -135,7 +135,7 @@ follow these steps to avoid unnecessary data loss: 1. If you are manually replicating any [data not managed by Geo](../../replication/datatypes.md#limitations-on-replicationverification), trigger the final replication process now. - 1. On the **primary** node: + 1. On the **primary** site: 1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Monitoring > Background Jobs**. 1. On the Sidekiq dashboard, select **Queues**, and wait for all queues except @@ -143,14 +143,14 @@ follow these steps to avoid unnecessary data loss: These queues contain work that has been submitted by your users; failing over before it is completed, causes the work to be lost. 1. On the left sidebar, select **Geo > Nodes** and wait for the - following conditions to be true of the **secondary** node you are failing over to: + following conditions to be true of the **secondary** site you are failing over to: - All replication meters reach 100% replicated, 0% failures. - All verification meters reach 100% verified, 0% failures. - Database replication lag is 0ms. - The Geo log cursor is up to date (0 events behind). - 1. On the **secondary** node: + 1. On the **secondary** site: 1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Monitoring > Background Jobs**. 1. On the Sidekiq dashboard, select **Queues**, and wait for all the `geo` @@ -158,14 +158,14 @@ follow these steps to avoid unnecessary data loss: 1. [Run an integrity check](../../../raketasks/check.md) to verify the integrity of CI artifacts, LFS objects, and uploads in file storage. - At this point, your **secondary** node contains an up-to-date copy of everything the - **primary** node has, meaning nothing is lost when you fail over. + At this point, your **secondary** site contains an up-to-date copy of everything the + **primary** site has, meaning nothing is lost when you fail over. -1. In this final step, you need to permanently disable the **primary** node. +1. In this final step, you need to permanently disable the **primary** site. WARNING: - When the **primary** node goes offline, there may be data saved on the **primary** node - that has not been replicated to the **secondary** node. This data should be treated + When the **primary** site goes offline, there may be data saved on the **primary** site + that has not been replicated to the **secondary** site. This data should be treated as lost if you proceed. NOTE: @@ -174,9 +174,9 @@ follow these steps to avoid unnecessary data loss: When performing a failover, we want to avoid a split-brain situation where writes can occur in two different GitLab instances. So to prepare for the - failover, you must disable the **primary** node: + failover, you must disable the **primary** site: - - If you have SSH access to the **primary** node, stop and disable GitLab: + - If you have SSH access to the **primary** site, stop and disable GitLab: ```shell sudo gitlab-ctl stop @@ -199,19 +199,19 @@ follow these steps to avoid unnecessary data loss: from starting if the machine reboots as `root` with `initctl stop gitlab-runsvvdir && echo 'manual' > /etc/init/gitlab-runsvdir.override && initctl reload-configuration`. - - If you do not have SSH access to the **primary** node, take the machine offline and + - If you do not have SSH access to the **primary** site, take the machine offline and prevent it from rebooting. Since there are many ways you may prefer to accomplish this, we avoid a single recommendation. You may need to: - Reconfigure the load balancers. - Change DNS records (for example, point the **primary** DNS record to the - **secondary** node to stop using the **primary** node). + **secondary** site to stop using the **primary** site). - Stop the virtual servers. - Block traffic through a firewall. - - Revoke object storage permissions from the **primary** node. + - Revoke object storage permissions from the **primary** site. - Physically disconnect a machine. -### Promoting the **secondary** node +### Promoting the **secondary** site Note the following when promoting a secondary: @@ -222,9 +222,35 @@ Note the following when promoting a secondary: error during this process, read [the troubleshooting advice](../../replication/troubleshooting.md#fixing-errors-during-a-failover-or-when-promoting-a-secondary-to-a-primary-node). -To promote the secondary node: +To promote the secondary site running GitLab 14.5 and later: -1. SSH in to your **secondary** node and login as root: +1. SSH in to your **secondary** node and run one of the following commands: + + - To promote the secondary node to primary: + + ```shell + sudo gitlab-ctl geo promote + ``` + + - To promote the secondary node to primary **without any further confirmation**: + + ```shell + sudo gitlab-ctl geo promote --force + ``` + +1. Verify you can connect to the newly promoted **primary** site using the URL used + previously for the **secondary** site. + + If successful, the **secondary** site is now promoted to the **primary** site. + +To promote the secondary site running GitLab 14.4 and earlier: + +WARNING: +The `gitlab-ctl promote-to-primary-node` and `gitlab-ctl promoted-db` commands are +deprecated in GitLab 14.5 and later, and are scheduled to [be removed in GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/345207). +Use `gitlab-ctl geo promote` instead. + +1. SSH in to your **secondary** site and login as root: ```shell sudo -i @@ -275,20 +301,20 @@ To promote the secondary node: gitlab-ctl promote-to-primary-node --skip-preflight-check ``` - You can also promote the secondary node to primary **without any further confirmation**, even when preflight checks fail: + You can also promote the secondary site to primary **without any further confirmation**, even when preflight checks fail: ```shell sudo gitlab-ctl promote-to-primary-node --force ``` -1. Verify you can connect to the newly promoted **primary** node using the URL used - previously for the **secondary** node. +1. Verify you can connect to the newly promoted **primary** site using the URL used + previously for the **secondary** site. - If successful, the **secondary** node has now been promoted to the **primary** node. + If successful, the **secondary** site is now promoted to the **primary** site. ### Next steps To regain geographic redundancy as quickly as possible, you should -[add a new **secondary** node](../../setup/index.md). To +[add a new **secondary** site](../../setup/index.md). To do that, you can re-add the old **primary** as a new secondary and bring it back online. diff --git a/doc/administration/geo/index.md b/doc/administration/geo/index.md index 48091967189..30d8d765dc5 100644 --- a/doc/administration/geo/index.md +++ b/doc/administration/geo/index.md @@ -129,25 +129,38 @@ and we recommend you use: ### Firewall rules -The following table lists basic ports that must be open between the **primary** and **secondary** sites for Geo. +The following table lists basic ports that must be open between the **primary** and **secondary** sites for Geo. To simplify failovers, we recommend opening ports in both directions. -| **Primary** site | **Secondary** site | Protocol | -|:-----------------|:-------------------|:-------------| -| 80 | 80 | HTTP | -| 443 | 443 | TCP or HTTPS | -| 22 | 22 | TCP | -| 5432 | | PostgreSQL | +| Source site | Source port | Destination site | Destination port | Protocol | +|-------------|-------------|------------------|------------------|-------------| +| Primary | Any | Secondary | 80 | TCP (HTTP) | +| Primary | Any | Secondary | 443 | TCP (HTTPS) | +| Secondary | Any | Primary | 80 | TCP (HTTP) | +| Secondary | Any | Primary | 443 | TCP (HTTPS) | +| Secondary | Any | Primary | 5432 | TCP | See the full list of ports used by GitLab in [Package defaults](../package_information/defaults.md) NOTE: -[Web terminal](../../ci/environments/index.md#web-terminals) support requires your load balancer to correctly handle WebSocket connections. +[Web terminal](../../ci/environments/index.md#web-terminals-deprecated) support requires your load balancer to correctly handle WebSocket connections. When using HTTP or HTTPS proxying, your load balancer must be configured to pass through the `Connection` and `Upgrade` hop-by-hop headers. See the [web terminal](../integration/terminal.md) integration guide for more details. NOTE: When using HTTPS protocol for port 443, you need to add an SSL certificate to the load balancers. If you wish to terminate SSL at the GitLab application server instead, use TCP protocol. +#### Internal URL + +HTTP requests from any Geo secondary site to the primary Geo site use the Internal URL of the primary +Geo site. If this is not explicitly defined in the primary Geo site settings in the Admin Area, the +public URL of the primary site will be used. + +To update the internal URL of the primary Geo site: + +1. On the top bar, go to **Menu > Admin > Geo > Sites**. +1. Select **Edit** on the primary site. +1. Change the **Internal URL**, then select **Save changes**. + ### LDAP We recommend that if you use LDAP on your **primary** site, you also set up secondary LDAP servers on each **secondary** site. Otherwise, users are unable to perform Git operations over HTTP(s) on the **secondary** site using HTTP Basic Authentication. However, Git via SSH and personal access tokens still works. @@ -258,6 +271,10 @@ For information on using Geo in disaster recovery situations to mitigate data-lo For more information on how to replicate the Container Registry, see [Docker Registry for a **secondary** site](replication/docker_registry.md). +### Geo secondary proxy + +For more information on using Geo proxying on secondary nodes, see [Geo proxying for secondary sites](secondary_proxy/index.md). + ### Security Review For more information on Geo security, see [Geo security review](replication/security_review.md). diff --git a/doc/administration/geo/replication/datatypes.md b/doc/administration/geo/replication/datatypes.md index e8e87f92481..c98436157fc 100644 --- a/doc/administration/geo/replication/datatypes.md +++ b/doc/administration/geo/replication/datatypes.md @@ -183,14 +183,13 @@ successfully, you must replicate their data using some other means. |[Application data in PostgreSQL](../../postgresql/index.md) | **Yes** (10.2) | **Yes** (10.2) | No | | |[Project repository](../../../user/project/repository/) | **Yes** (10.2) | **Yes** (10.7) | No | | |[Project wiki repository](../../../user/project/wiki/) | **Yes** (10.2) | **Yes** (10.7) | No | | -|[Group wiki repository](../../../user/project/wiki/index.md#group-wikis) | [**Yes** (13.10)](https://gitlab.com/gitlab-org/gitlab/-/issues/208147) | No | No | Behind feature flag `geo_group_wiki_repository_replication`, enabled by default. | +|[Group wiki repository](../../../user/project/wiki/group.md) | [**Yes** (13.10)](https://gitlab.com/gitlab-org/gitlab/-/issues/208147) | No | No | Behind feature flag `geo_group_wiki_repository_replication`, enabled by default. | |[Uploads](../../uploads.md) | **Yes** (10.2) | [No](https://gitlab.com/groups/gitlab-org/-/epics/1817) | No | Verified only on transfer or manually using [Integrity Check Rake Task](../../raketasks/check.md) on both sites and comparing the output between them. | |[LFS objects](../../lfs/index.md) | **Yes** (10.2) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/8922) | Via Object Storage provider if supported. Native Geo support (Beta). | Verified only on transfer or manually using [Integrity Check Rake Task](../../raketasks/check.md) on both sites and comparing the output between them. GitLab versions 11.11.x and 12.0.x are affected by [a bug that prevents any new LFS objects from replicating](https://gitlab.com/gitlab-org/gitlab/-/issues/32696).<br /><br />Behind feature flag `geo_lfs_object_replication`, enabled by default. | |[Personal snippets](../../../user/snippets.md) | **Yes** (10.2) | **Yes** (10.2) | No | | |[Project snippets](../../../user/snippets.md) | **Yes** (10.2) | **Yes** (10.2) | No | | -|[CI job artifacts (other than Job Logs)](../../../ci/pipelines/job_artifacts.md) | **Yes** (10.4) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/8923) | Via Object Storage provider if supported. Native Geo support (Beta). | Verified only manually using [Integrity Check Rake Task](../../raketasks/check.md) on both sites and comparing the output between them. | +|[CI job artifacts](../../../ci/pipelines/job_artifacts.md) | **Yes** (10.4) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/8923) | Via Object Storage provider if supported. Native Geo support (Beta). | Verified only manually using [Integrity Check Rake Task](../../raketasks/check.md) on both sites and comparing the output between them. Job logs also verified on transfer. | |[CI Pipeline Artifacts](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/models/ci/pipeline_artifact.rb) | [**Yes** (13.11)](https://gitlab.com/gitlab-org/gitlab/-/issues/238464) | [**Yes** (13.11)](https://gitlab.com/gitlab-org/gitlab/-/issues/238464) | Via Object Storage provider if supported. Native Geo support (Beta). | Persists additional artifacts after a pipeline completes | -|[Job logs](../../job_logs.md) | **Yes** (10.4) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/8923) | Via Object Storage provider if supported. Native Geo support (Beta). | Verified only on transfer or manually using [Integrity Check Rake Task](../../raketasks/check.md) on both sites and comparing the output between them. | |[Container Registry](../../packages/container_registry.md) | **Yes** (12.3) | No | No | Disabled by default. See [instructions](docker_registry.md) to enable. | |[Content in object storage (beta)](object_storage.md) | **Yes** (12.4) | [No](https://gitlab.com/gitlab-org/gitlab/-/issues/13845) | No | | |[Infrastructure Registry](../../../user/packages/infrastructure_registry/index.md) | **Yes** (14.0) | [**Yes**](#limitation-of-verification-for-files-in-object-storage) (14.0) | Via Object Storage provider if supported. Native Geo support (Beta). | Behind feature flag `geo_package_file_replication`, enabled by default. | diff --git a/doc/administration/geo/replication/disable_geo.md b/doc/administration/geo/replication/disable_geo.md index ae2597ad649..02a65f0b8e1 100644 --- a/doc/administration/geo/replication/disable_geo.md +++ b/doc/administration/geo/replication/disable_geo.md @@ -28,7 +28,7 @@ To disable Geo, you need to first remove all your secondary Geo sites, which mea anymore on these sites. You can follow our docs to [remove your secondary Geo sites](remove_geo_site.md). If the current site that you want to keep using is a secondary site, you need to first promote it to primary. -You can use our steps on [how to promote a secondary site](../disaster_recovery/#step-3-promoting-a-secondary-node) +You can use our steps on [how to promote a secondary site](../disaster_recovery/#step-3-promoting-a-secondary-site) to do that. ## Remove the primary site from the UI diff --git a/doc/administration/geo/replication/troubleshooting.md b/doc/administration/geo/replication/troubleshooting.md index b7370d32056..432d042608c 100644 --- a/doc/administration/geo/replication/troubleshooting.md +++ b/doc/administration/geo/replication/troubleshooting.md @@ -455,7 +455,7 @@ To solve this: 1. Back up [the `.git` folder](../../repository_storage_types.md#translate-hashed-storage-paths). -1. Optional: [Spot-check](../../troubleshooting/log_parsing.md#find-all-projects-affected-by-a-fatal-git-problem)) +1. Optional: [Spot-check](../../troubleshooting/log_parsing.md#find-all-projects-affected-by-a-fatal-git-problem) a few of those IDs whether they indeed correspond to a project with known Geo replication failures. Use `fatal: 'geo'` as the `grep` term and the following API call: @@ -683,7 +683,7 @@ when promoting a secondary to a primary node with strategies to resolve them. ### Message: ActiveRecord::RecordInvalid: Validation failed: Name has already been taken -When [promoting a **secondary** node](../disaster_recovery/index.md#step-3-promoting-a-secondary-node), +When [promoting a **secondary** site](../disaster_recovery/index.md#step-3-promoting-a-secondary-site), you might encounter the following error: ```plaintext @@ -723,21 +723,35 @@ If you disabled a secondary node, either with the [replication pause task](../in (13.2) or by using the user interface (13.1 and earlier), you must first re-enable the node before you can continue. This is fixed in 13.4. -Run the following command, replacing `https://<secondary url>/` with the URL -for your secondary server, using either `http` or `https`, and ensuring that you -end the URL with a slash (`/`): +This can be fixed in the database. -```shell -sudo gitlab-rails dbconsole +1. Start a database console: -UPDATE geo_nodes SET enabled = true WHERE url = 'https://<secondary url>/' AND enabled = false;" -``` + In [GitLab 14.2 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/341210): + + ```shell + sudo gitlab-rails dbconsole --database main + ``` + + In GitLab 14.1 and earlier: + + ```shell + sudo gitlab-rails dbconsole + ``` + +1. Run the following command, replacing `https://<secondary url>/` with the URL + for your secondary server. You can use either `http` or `https`, but ensure that you + end the URL with a slash (`/`): + + ```sql + UPDATE geo_nodes SET enabled = true WHERE url = 'https://<secondary url>/' AND enabled = false;" + ``` -This should update 1 row. + This should update 1 row. ### Message: ``NoMethodError: undefined method `secondary?' for nil:NilClass`` -When [promoting a **secondary** node](../disaster_recovery/index.md#step-3-promoting-a-secondary-node), +When [promoting a **secondary** site](../disaster_recovery/index.md#step-3-promoting-a-secondary-site), you might encounter the following error: ```plaintext @@ -753,13 +767,13 @@ Tasks: TOP => geo:set_secondary_as_primary (See full trace by running task with --trace) ``` -This command is intended to be executed on a secondary node only, and this error -is displayed if you attempt to run this command on a primary node. +This command is intended to be executed on a secondary site only, and this error +is displayed if you attempt to run this command on a primary site. ### Message: `sudo: gitlab-pg-ctl: command not found` When -[promoting a **secondary** node with multiple servers](../disaster_recovery/index.md#promoting-a-secondary-node-with-multiple-servers), +[promoting a **secondary** site with multiple nodes](../disaster_recovery/index.md#promoting-a-secondary-site-with-multiple-nodes-running-gitlab-144-and-earlier), you need to run the `gitlab-pg-ctl` command to promote the PostgreSQL read-replica database. diff --git a/doc/administration/geo/secondary_proxy/img/single_url_add_traffic_policy_endpoints.png b/doc/administration/geo/secondary_proxy/img/single_url_add_traffic_policy_endpoints.png Binary files differnew file mode 100644 index 00000000000..d1c4afceb04 --- /dev/null +++ b/doc/administration/geo/secondary_proxy/img/single_url_add_traffic_policy_endpoints.png diff --git a/doc/administration/geo/secondary_proxy/img/single_url_create_policy_records_with_traffic_policy.png b/doc/administration/geo/secondary_proxy/img/single_url_create_policy_records_with_traffic_policy.png Binary files differnew file mode 100644 index 00000000000..3bad391f220 --- /dev/null +++ b/doc/administration/geo/secondary_proxy/img/single_url_create_policy_records_with_traffic_policy.png diff --git a/doc/administration/geo/secondary_proxy/index.md b/doc/administration/geo/secondary_proxy/index.md new file mode 100644 index 00000000000..2b8c0d1e6fa --- /dev/null +++ b/doc/administration/geo/secondary_proxy/index.md @@ -0,0 +1,127 @@ +--- +stage: Enablement +group: Geo +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +type: howto +--- + +# Geo proxying for secondary sites **(PREMIUM SELF)** + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5914) in GitLab 14.4 [with a flag](../../feature_flags.md) named `geo_secondary_proxy`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. See below to [Set up a unified URL for Geo sites](#set-up-a-unified-url-for-geo-sites). +The feature is not ready for production use. + +Use Geo proxying to: + +- Have secondary sites serve read-write traffic by proxying to the primary site. +- Selectively accelerate replicated data types by directing read-only operations to the local site instead. + +When enabled, users of the secondary site can use the WebUI as if they were accessing the +primary site's UI. This significantly improves the overall user experience of secondary sites. + +With secondary proxying, web requests to secondary Geo sites are +proxied directly to the primary, and appear to act as a read-write site. + +Proxying is done by the [`gitlab-workhorse`](https://gitlab.com/gitlab-org/gitlab-workhorse) component. +Traffic usually sent to the Rails application on the Geo secondary site is proxied +to the [internal URL](../index.md#internal-url) of the primary Geo site instead. + +Use secondary proxying for use-cases including: + +- Having all Geo sites behind a single URL. +- Geographically load-balancing traffic without worrying about write access. + +## Set up a unified URL for Geo sites + +Secondary sites can transparently serve read-write traffic. You can +use a single external URL so that requests can hit either the primary Geo site +or any secondary Geo sites that use Geo proxying. + +### Configure an external URL to send traffic to both sites + +Follow the [Location-aware public URL](location_aware_external_url.md) steps to create +a single URL used by all Geo sites, including the primary. + +### Update the Geo sites to use the same external URL + +1. On your Geo sites, SSH **into each node running Rails (Puma, Sidekiq, Log-Cursor) + and change the `external_url` to that of the single URL: + + ```shell + sudo editor /etc/gitlab/gitlab.rb + ``` + +1. Reconfigure the updated nodes for the change to take effect if the URL was different than the one already set: + + ```shell + gitlab-ctl reconfigure + ``` + +1. To match the new external URL set on the secondary Geo sites, the primary database + needs to reflect this change. + + In the Geo administration page of the **primary** site, edit each Geo secondary that + is using the secondary proxying and set the `URL` field to the single URL. + Make sure the primary site is also using this URL. + +### Enable secondary proxying + +1. SSH into each application node (serving user traffic directly) on your secondary Geo site + and add the following environment variable: + + ```shell + sudo editor /etc/gitlab/gitlab.rb + ``` + + ```ruby + gitlab_workhorse['env'] = { + "GEO_SECONDARY_PROXY" => "1" + } + ``` + +1. Reconfigure the updated nodes for the change to take effect: + + ```shell + gitlab-ctl reconfigure + ``` + +1. SSH into one node running Rails on your primary Geo site and enable the Geo secondary proxy feature flag: + + ```shell + sudo gitlab-rails runner "Feature.enable(:geo_secondary_proxy)" + ``` + +## Enable Geo proxying with Separate URLs + +The ability to use proxying with separate URLs is still in development. You can follow the +["Geo secondary proxying with separate URLs" epic](https://gitlab.com/groups/gitlab-org/-/epics/6865) +for progress. + +## Features accelerated by secondary Geo sites + +Most HTTP traffic sent to a secondary Geo site can be proxied to the primary Geo site. With this architecture, +secondary Geo sites are able to support write requests. Certain **read** requests are handled locally by secondary +sites for improved latency and bandwidth nearby. All write requests are proxied to the primary site. + +The following table details the components currently tested through the Geo secondary site Workhorse proxy. +It does not cover all data types, more will be added in the future as they are tested. + +| Feature / component | Accelerated reads? | +|:----------------------------------------------------|:-----------------------| +| Project, wiki, design repository (using the web UI) | **{dotted-circle}** No | +| Project, wiki repository (using Git) | **{check-circle}** Yes <sup>1</sup> | +| Project, Personal Snippet (using the web UI) | **{dotted-circle}** No | +| Project, Personal Snippet (using Git) | **{check-circle}** Yes <sup>1</sup> | +| Group wiki repository (using the web UI) | **{dotted-circle}** No | +| Group wiki repository (using Git) | **{check-circle}** Yes <sup>1</sup> | +| User uploads | **{dotted-circle}** No | +| LFS objects (using the web UI) | **{dotted-circle}** No | +| LFS objects (using Git) | **{check-circle}** Yes | +| Pages | **{dotted-circle}** No <sup>2</sup> | +| Advanced search (using the web UI) | **{dotted-circle}** No | + +1. Git reads are served from the local secondary while pushes get proxied to the primary. + Selective sync or cases where repositories don't exist locally on the Geo secondary throw a "not found" error. +1. Pages can use the same URL (without access control), but must be configured separately and are not proxied. diff --git a/doc/administration/geo/secondary_proxy/location_aware_external_url.md b/doc/administration/geo/secondary_proxy/location_aware_external_url.md new file mode 100644 index 00000000000..b5a8d4baa1f --- /dev/null +++ b/doc/administration/geo/secondary_proxy/location_aware_external_url.md @@ -0,0 +1,83 @@ +--- +stage: Enablement +group: Geo +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +type: howto +--- + +# Location-aware public URL **(PREMIUM SELF)** + +With [Geo proxying for secondary sites](index.md), you can provide GitLab users +with a single URL that automatically uses the Geo site closest to them. +Users don't need to use different URLs or worry about read-only operations to take +advantage of closer Geo sites as they move. + +With [Geo proxying for secondary sites](index.md) web and Git requests are proxied +from **secondary** sites to the **primary**. + +Though these instructions use [AWS Route53](https://aws.amazon.com/route53/), +other services such as [Cloudflare](https://www.cloudflare.com/) can be used +as well. + +## Prerequisites + +This example creates a `gitlab.example.com` subdomain that automatically directs +requests: + +- From Europe to a **secondary** site. +- From all other locations to the **primary** site. + +The URLs to access each node by itself are: + +- `primary.example.com` as a Geo **primary** site. +- `secondary.example.com` as a Geo **secondary** site. + +For this example, you need: + +- A working GitLab **primary** site that is accessible at `gitlab.example.com` _and_ `primary.example.com`. +- A working GitLab **secondary** site. +- A Route53 Hosted Zone managing your domain for the Route53 setup. + +If you haven't yet set up a Geo _primary_ site and _secondary_ site, see the +[Geo setup instructions](../index.md#setup-instructions). + +## AWS Route53 + +### Create a traffic policy + +In a Route53 Hosted Zone, traffic policies can be used to set up a variety of +routing configurations. + +1. Go to the +[Route53 dashboard](https://console.aws.amazon.com/route53/home) and select +**Traffic policies**. + +1. Select **Create traffic policy**. +1. Fill in the **Policy Name** field with `Single Git Host` and select **Next**. +1. Leave **DNS type** as `A: IP Address in IPv4 format`. +1. Select **Connect to...**, then **Geolocation rule**. +1. For the first **Location**: + 1. Leave it as `Default`. + 1. Select **Connect to...**, then **New endpoint**. + 1. Choose **Type** `value` and fill it in with `<your **primary** IP address>`. + +1. For the second **Location**: + 1. Choose `Europe`. + 1. Select **Connect to...** and select **New endpoint**. + 1. Choose **Type** `value` and fill it in with `<your **secondary** IP address>`. + +  + +1. Select **Create traffic policy**. +1. Fill in **Policy record DNS name** with `gitlab`. + +  + +1. Select **Create policy records**. + +You have successfully set up a single host, like `gitlab.example.com`, which +distributes traffic to your Geo sites by geolocation. + +## Enable Geo proxying for secondary sites + +After setting up a single URL to use for all Geo sites, continue with the [steps to enable Geo proxying for secondary sites](index.md). diff --git a/doc/administration/geo/setup/database.md b/doc/administration/geo/setup/database.md index d72bb0737ae..7f4c771c962 100644 --- a/doc/administration/geo/setup/database.md +++ b/doc/administration/geo/setup/database.md @@ -7,23 +7,21 @@ type: howto # Geo database replication **(PREMIUM SELF)** -NOTE: -If your GitLab installation uses external (not managed by Omnibus) PostgreSQL -instances, the Omnibus roles are unable to perform all necessary -configuration steps. In this case, -[follow the Geo with external PostgreSQL instances document instead](external_database.md). +This document describes the minimal required steps to replicate your primary +GitLab database to a secondary node's database. You may have to change some +values, based on attributes including your database's setup and size. NOTE: -The stages of the setup process must be completed in the documented order. -Before attempting the steps in this stage, [complete all prior stages](../setup/index.md#using-omnibus-gitlab). +If your GitLab installation uses external (not managed by Omnibus GitLab) +PostgreSQL instances, the Omnibus roles cannot perform all necessary +configuration steps. In this case, use the [Geo with external PostgreSQL instances](external_database.md) +process instead. -This document describes the minimal steps you have to take to replicate your -**primary** GitLab database to a **secondary** node's database. You may have to -change some values, based on attributes including your database's setup and -size. +The stages of the setup process must be completed in the documented order. +Before you attempt the steps in this stage, [complete all prior stages](../setup/index.md#using-omnibus-gitlab). -You are encouraged to first read through all the steps before executing them -in your testing/production environment. +Be sure to read and review all of these steps before you execute them in your +testing or production environments. ## Single instance database replication @@ -214,7 +212,7 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o ## - Prevents automatic upgrade of Postgres since it requires downtime of ## streaming replication to Geo secondary sites ## - Enables standard single-node GitLab services like NGINX, Puma, Redis, - ## Sidekiq, etc. If you are segregating services, then you will need to + ## or Sidekiq. If you are segregating services, then you will need to ## explicitly disable unwanted services. ## roles(['geo_primary_role']) @@ -690,8 +688,8 @@ could do it with [HAProxy](https://www.haproxy.org/). The following IPs and names are used as an example: - `10.6.0.21`: Patroni 1 (`patroni1.internal`) -- `10.6.0.21`: Patroni 2 (`patroni2.internal`) -- `10.6.0.22`: Patroni 3 (`patroni3.internal`) +- `10.6.0.22`: Patroni 2 (`patroni2.internal`) +- `10.6.0.23`: Patroni 3 (`patroni3.internal`) ```plaintext global @@ -716,7 +714,7 @@ backend postgresql server patroni1.internal 10.6.0.21:5432 maxconn 100 check port 8008 server patroni2.internal 10.6.0.22:5432 maxconn 100 check port 8008 - server patroni3.internal 10.6.0.23.195:5432 maxconn 100 check port 8008 + server patroni3.internal 10.6.0.23:5432 maxconn 100 check port 8008 ``` Refer to your preferred Load Balancer's documentation for further guidance. diff --git a/doc/administration/geo/setup/external_database.md b/doc/administration/geo/setup/external_database.md index 13dbf9d1434..756e73f022f 100644 --- a/doc/administration/geo/setup/external_database.md +++ b/doc/administration/geo/setup/external_database.md @@ -8,7 +8,7 @@ type: howto # Geo with external PostgreSQL instances **(PREMIUM SELF)** This document is relevant if you are using a PostgreSQL instance that is *not -managed by Omnibus*. This includes cloud-managed instances like AWS RDS, or +managed by Omnibus*. This includes cloud-managed instances like Amazon RDS, or manually installed and configured PostgreSQL instances. NOTE: @@ -58,7 +58,7 @@ developed and tested. We aim to be compatible with most external To set up an external database, you can either: -- Set up [streaming replication](https://www.postgresql.org/docs/12/warm-standby.html#STREAMING-REPLICATION-SLOTS) yourself (for example AWS RDS, bare metal not managed by Omnibus, and so on). +- Set up [streaming replication](https://www.postgresql.org/docs/12/warm-standby.html#STREAMING-REPLICATION-SLOTS) yourself (for example Amazon RDS, bare metal not managed by Omnibus, and so on). - Perform the Omnibus configuration manually as follows. #### Leverage your cloud provider's tools to replicate the primary database @@ -200,7 +200,7 @@ This is for the installation of extensions during installation and upgrades. As To setup an external tracking database, follow the instructions below: NOTE: -If you want to use AWS RDS as a tracking database, make sure it has access to +If you want to use Amazon RDS as a tracking database, make sure it has access to the secondary database. Unfortunately, just assigning the same security group is not enough as outbound rules do not apply to RDS PostgreSQL databases. Therefore, you need to explicitly add an inbound rule to the read-replica's security group allowing any TCP traffic from diff --git a/doc/administration/get_started.md b/doc/administration/get_started.md index 2455aecafea..bff2aee1d2d 100644 --- a/doc/administration/get_started.md +++ b/doc/administration/get_started.md @@ -44,7 +44,7 @@ Get started: **More resources** - Learn more about [running multiple Agile teams](https://www.youtube.com/watch?v=VR2r1TJCDew). -- Sync group memberships [by using LDAP](../administration/auth/ldap/index.md#group-sync). +- Sync group memberships [by using LDAP](../administration/auth/ldap/ldap_synchronization.md#group-sync). - Manage user access with inherited permissions. Use up to 20 levels of subgroups to organize both teams and projects. - Learn more about [inherited permissions](../user/project/members/index.md#inherited-membership). - View [nested category examples](../user/group/subgroups/index.md#overview). @@ -231,7 +231,7 @@ Rate limits also improve the security of your application. ### Configure rate limits for self-managed GitLab -You can make changes to your default rate limits from the Admin Area. For more information about configuration, see the [Admin Area page](../security/rate_limits.md#admin-area-settings). +You can make changes to your default rate limits from the Admin Area. For more information about configuration, see the [Admin Area page](../security/rate_limits.md#configurable-limits). - Define [issues rate limits](../user/admin_area/settings/rate_limit_on_issues_creation.md) to set a maximum number of issue creation requests per minute, per user. - Enforce [user and IP rate limits](../user/admin_area/settings/user_and_ip_rate_limits.md) for unauthenticated web requests. @@ -249,7 +249,7 @@ Rate limits also improve the security of your application. ### Configure rate limits for GitLab SaaS -You can make changes to your default rate limits from the Admin Area. For more information about configuration, see the [Admin Area page](../security/rate_limits.md#admin-area-settings). +You can make changes to your default rate limits from the Admin Area. For more information about configuration, see the [Admin Area page](../security/rate_limits.md#configurable-limits). - Review the rate limit page. - Read our [API page](../api/index.md) for more information about API and rate limiting. diff --git a/doc/administration/gitaly/praefect.md b/doc/administration/gitaly/praefect.md index d3ea71bc8d2..da456131a52 100644 --- a/doc/administration/gitaly/praefect.md +++ b/doc/administration/gitaly/praefect.md @@ -851,6 +851,10 @@ of choice already. Some examples include [HAProxy](https://www.haproxy.org/) Big-IP LTM, and Citrix Net Scaler. This documentation outlines what ports and protocols you need configure. +WARNING: +Long-running background jobs can maintain an idle connection with Praefect for up 6 hours. Set your load balancer timeout to be at least +6 hours long. + | LB Port | Backend Port | Protocol | |:--------|:-------------|:---------| | 2305 | 2305 | TCP | diff --git a/doc/administration/gitaly/troubleshooting.md b/doc/administration/gitaly/troubleshooting.md index 1f90ebb7565..d6d93b8af94 100644 --- a/doc/administration/gitaly/troubleshooting.md +++ b/doc/administration/gitaly/troubleshooting.md @@ -157,7 +157,7 @@ Confirm the following are all true: successfully creates the project but doesn't create the README. - When [tailing the logs](https://docs.gitlab.com/omnibus/settings/logs.html#tail-logs-in-a-console-on-the-server) on a Gitaly client and reproducing the error, you get `401` errors - when reaching the [`/api/v4/internal/allowed`](../../development/internal_api.md) endpoint: + when reaching the [`/api/v4/internal/allowed`](../../development/internal_api/index.md) endpoint: ```shell # api_json.log diff --git a/doc/administration/img/audit_events_v14_5.png b/doc/administration/img/audit_events_v14_5.png Binary files differnew file mode 100644 index 00000000000..57190463d05 --- /dev/null +++ b/doc/administration/img/audit_events_v14_5.png diff --git a/doc/administration/img/audit_log_v13_6.png b/doc/administration/img/audit_log_v13_6.png Binary files differdeleted file mode 100644 index 82ff3e9c87b..00000000000 --- a/doc/administration/img/audit_log_v13_6.png +++ /dev/null diff --git a/doc/administration/incoming_email.md b/doc/administration/incoming_email.md index 5b72583bc95..6b390cfc77a 100644 --- a/doc/administration/incoming_email.md +++ b/doc/administration/incoming_email.md @@ -155,6 +155,50 @@ Reply by email should now be working. 1. Find the `incoming_email` section in `config/gitlab.yml`, enable the feature and fill in the details for your specific IMAP server and email account (see [examples](#configuration-examples) below). +If you use systemd units to manage GitLab: + +1. Add `gitlab-mailroom.service` as a dependency to `gitlab.target`: + + ```shell + sudo systemctl edit gitlab.target + ``` + + In the editor that opens, add the following and save the file: + + ```plaintext + [Unit] + Wants=gitlab-mailroom.service + ``` + +1. If you run Redis and PostgreSQL on the same machine, you should add a + dependency on Redis. Run: + + ```shell + sudo systemctl edit gitlab-mailroom.service + ``` + + In the editor that opens, add the following and save the file: + + ```plaintext + [Unit] + Wants=redis-server.service + After=redis-server.service + ``` + +1. Start `gitlab-mailroom.service`: + + ```shell + sudo systemctl start gitlab-mailroom.service + ``` + +1. Verify that everything is configured correctly: + + ```shell + sudo -u git -H bundle exec rake gitlab:incoming_email:check RAILS_ENV=production + ``` + +If you use the SysV init script to manage GitLab: + 1. Enable `mail_room` in the init script at `/etc/default/gitlab`: ```shell diff --git a/doc/administration/index.md b/doc/administration/index.md index ee17edc35fc..53a3c305aab 100644 --- a/doc/administration/index.md +++ b/doc/administration/index.md @@ -114,7 +114,7 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Mattermost](../integration/mattermost/index.md): Integrate with [Mattermost](https://mattermost.com), an open source, private cloud workplace for web messaging. - [PlantUML](integration/plantuml.md): Create diagrams in AsciiDoc and Markdown documents created in snippets, wikis, and repositories. -- [Web terminals](integration/terminal.md): Provide terminal access to your applications deployed to Kubernetes from GitLab CI/CD [environments](../ci/environments/index.md#web-terminals). +- [Web terminals](integration/terminal.md): Provide terminal access to your applications deployed to Kubernetes from GitLab CI/CD [environments](../ci/environments/index.md#web-terminals-deprecated). ## User settings and permissions diff --git a/doc/administration/instance_limits.md b/doc/administration/instance_limits.md index a2729e60545..0b470146b14 100644 --- a/doc/administration/instance_limits.md +++ b/doc/administration/instance_limits.md @@ -15,9 +15,7 @@ performance, data, or could even exhaust the allocated resources for the applica Rate limits can be used to improve the security and durability of GitLab. -For example, one script can make thousands of web requests per second. Whether malicious, apathetic, or just a bug, your application and infrastructure may not be able to cope with the load. Rate limits can help to mitigate these types of attacks. - -Read more about [configuring rate limits](../security/rate_limits.md) in the Security documentation. +Read more about [configuring rate limits](../security/rate_limits.md). ### Issue creation @@ -128,16 +126,6 @@ This setting limits the import/export actions for groups and projects. Read more about [import/export rate limits](../user/admin_area/settings/import_export_rate_limits.md). -### Rack attack - -This method of rate limiting is cumbersome, but has some advantages. It allows -throttling of specific paths, and is also integrated into Git and container -registry requests. - -Read more about the [Rack Attack initializer](../security/rack_attack.md) method of setting rate limits. - -- **Default rate limit**: Disabled. - ### Member Invitations Limit the maximum daily member invitations allowed per group hierarchy. @@ -155,6 +143,9 @@ This only applies to project and group webhooks. Calls over the rate limit are logged into `auth.log`. +To set this limit for a self-managed installation, run the following in the +[GitLab Rails console](operations/rails_console.md#starting-a-rails-console-session): + ```ruby # If limits don't exist for the default plan, you can create one with: # Plan.default.create_limits! @@ -222,10 +213,12 @@ When the number exceeds the limit the page displays an alert and links to a pagi > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/51401) in GitLab 11.10. The number of pipelines that can be created in a single push is 4. -This is to prevent the accidental creation of pipelines when `git push --all` +This limit prevents the accidental creation of pipelines when `git push --all` or `git push --mirror` is used. -Read more in the [CI documentation](../ci/yaml/index.md#processing-git-pushes). +This limit does not affect any of the updated merge request pipelines. +All updated merge requests have a pipeline created when using +[pipelines for merge requests](../ci/pipelines/merge_request_pipelines.md). ## Retention of activity history @@ -397,6 +390,27 @@ Plan.default.actual_limits.update!(ci_project_subscriptions: 500) Set the limit to `0` to disable it. +### Limit the number of pipeline triggers + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/33696) in GitLab 14.6. + +You can set a limit on the maximum number of pipeline triggers per project. This +limit is checked every time a new trigger is created. + +If a new trigger would cause the total number of pipeline triggers to exceed the +limit, the trigger is considered invalid. + +Set the limit to `0` to disable it. Defaults to `0` on self-managed instances. + +To set this limit to `100` on a self-managed installation, run the following in the +[GitLab Rails console](operations/rails_console.md#starting-a-rails-console-session): + +```ruby +Plan.default.actual_limits.update!(pipeline_triggers: 100) +``` + +This limit is [enabled on GitLab.com](../user/gitlab_com/index.md#gitlab-cicd). + ### Number of pipeline schedules > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/29566) in GitLab 12.10. @@ -543,6 +557,7 @@ Plan.default.actual_limits.update!(pages_file_entries: 100) The total number of registered runners is limited at the group and project levels. Each time a new runner is registered, GitLab checks these limits against runners that have been active in the last 3 months. A runner's registration fails if it exceeds the limit for the scope determined by the runner registration token. +If the limit value is set to zero, the limit is disabled. - GitLab SaaS subscribers have different limits defined per plan, affecting all projects using that plan. - Self-managed GitLab Premium and Ultimate limits are defined by a default plan that affects all projects: @@ -590,25 +605,64 @@ Plan.default.actual_limits.update!(dast_profile_schedules: 50) ### Maximum size and depth of CI/CD configuration YAML files -The default maximum size of a CI/CD configuration YAML file is 1 megabyte and the default depth is 100. +The default maximum size of a CI/CD configuration YAML file is 1 megabyte and the +default depth is 100. + +You can change these limits in the [GitLab Rails console](operations/rails_console.md#starting-a-rails-console-session): + +- To update the maximum YAML size, update `max_yaml_size_bytes` with the new value in megabytes: + + ```ruby + ApplicationSetting.update!(max_yaml_size_bytes: 2.megabytes) + ``` -You can change these limits in the [GitLab Rails console](operations/rails_console.md#starting-a-rails-console-session). -Update `max_yaml_size_bytes` with the new value in megabytes: + The `max_yaml_size_bytes` value is not directly tied to the size of the YAML file, + but rather the memory allocated for the relevant objects. + +- To update the maximum YAML depth, update `max_yaml_depth` with the new value in megabytes: + + ```ruby + ApplicationSetting.update!(max_yaml_depth: 125) + ``` + +To disable this limitation entirely, disable the feature flag in the console: ```ruby -ApplicationSetting.update!(max_yaml_size_bytes: 2.megabytes) +Feature.disable(:ci_yaml_limit_size) ``` -Update `max_yaml_depth` with the new value in megabytes: +### Limit dotenv variables + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/321552) in GitLab 14.5. + +You can set a limit on the maximum number of variables inside of a dotenv artifact. +This limit is checked every time a dotenv file is exported as an artifact. + +Set the limit to `0` to disable it. Defaults to `0` on self-managed instances. + +To set this limit to `100` on a self-managed instance, run the following command in the +[GitLab Rails console](operations/rails_console.md#starting-a-rails-console-session): ```ruby -ApplicationSetting.update!(max_yaml_depth: 125) +Plan.default.actual_limits.update!(dotenv_variable_limit: 100) ``` -To disable this limitation entirely, disable the feature flag in the console: +This limit is [enabled on GitLab.com](../user/gitlab_com/index.md#gitlab-cicd). + +### Limit dotenv file size + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/321552) in GitLab 14.5. + +You can set a limit on the maximum size of a dotenv artifact. This limit is checked +every time a dotenv file is exported as an artifact. + +Set the limit to `0` to disable it. Defaults to 5KB. + +To set this limit to 5KB on a self-managed installation, run the following in the +[GitLab Rails console](operations/rails_console.md#starting-a-rails-console-session): ```ruby -Feature.disable(:ci_yaml_limit_size) +Plan.default.actual_limits.update!(dotenv_size_limit: 5.kilobytes) ``` ## Instance monitoring and metrics @@ -837,6 +891,17 @@ Set the limit to `0` to allow any file size. When asking for versions of a given NuGet package name, the GitLab Package Registry returns a maximum of 300 versions. +## Dependency Proxy Limits + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6396) in GitLab 14.5. + +The maximum file size for an image cached in the +[Dependency Proxy](../user/packages/dependency_proxy/index.md) +varies by file type: + +- Image blob: 5 GB +- Image manifest: 10 MB + ## Branch retargeting on merge > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/320902) in GitLab 13.9. diff --git a/doc/administration/integration/terminal.md b/doc/administration/integration/terminal.md index 45b94781adc..07b9ba87d8e 100644 --- a/doc/administration/integration/terminal.md +++ b/doc/administration/integration/terminal.md @@ -4,12 +4,17 @@ group: Integrations info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Web terminals **(FREE)** +# Web terminals (DEPRECATED) **(FREE)** + +> [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. + +WARNING: +This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. With the introduction of the [Kubernetes integration](../../user/infrastructure/clusters/index.md), GitLab can store and use credentials for a Kubernetes cluster. GitLab uses these credentials to provide access to -[web terminals](../../ci/environments/index.md#web-terminals) for environments. +[web terminals](../../ci/environments/index.md#web-terminals-deprecated) for environments. NOTE: Only project maintainers and owners can access web terminals. diff --git a/doc/administration/load_balancer.md b/doc/administration/load_balancer.md index ca66791166e..60293e56202 100644 --- a/doc/administration/load_balancer.md +++ b/doc/administration/load_balancer.md @@ -7,7 +7,7 @@ type: reference # Load Balancer for multi-node GitLab **(FREE SELF)** -In an multi-node GitLab configuration, you need a load balancer to route +In a multi-node GitLab configuration, you need a load balancer to route traffic to the application servers. The specifics on which load balancer to use or the exact configuration is beyond the scope of GitLab documentation. We hope that if you're managing HA systems like GitLab you have a load balancer of @@ -69,7 +69,7 @@ for details on managing SSL certificates and configuring NGINX. | 443 | 443 | TCP or HTTPS (*1*) (*2*) | | 22 | 22 | TCP | -- (*1*): [Web terminal](../ci/environments/index.md#web-terminals) support requires +- (*1*): [Web terminal](../ci/environments/index.md#web-terminals-deprecated) support requires your load balancer to correctly handle WebSocket connections. When using HTTP or HTTPS proxying, this means your load balancer must be configured to pass through the `Connection` and `Upgrade` hop-by-hop headers. See the diff --git a/doc/administration/logs.md b/doc/administration/logs.md index a9fd698a525..bf74a96a627 100644 --- a/doc/administration/logs.md +++ b/doc/administration/logs.md @@ -752,7 +752,6 @@ Depending on your installation method, this file is located at: This log records: -- Information whenever [Rack Attack](../security/rack_attack.md) registers an abusive request. - Requests over the [Rate Limit](../user/admin_area/settings/rate_limits_on_raw_endpoints.md) on raw endpoints. - [Protected paths](../user/admin_area/settings/protected_paths.md) abusive requests. - In GitLab versions [12.3](https://gitlab.com/gitlab-org/gitlab/-/issues/29239) and later, diff --git a/doc/administration/maintenance_mode/index.md b/doc/administration/maintenance_mode/index.md index d5bcd132665..2d17062e955 100644 --- a/doc/administration/maintenance_mode/index.md +++ b/doc/administration/maintenance_mode/index.md @@ -119,7 +119,7 @@ For most JSON requests, POST, PUT, PATCH, and DELETE are blocked, and the API re | POST | `/admin/session`, `/admin/session/destroy` | To allow [Administrator mode for GitLab administrators](https://gitlab.com/groups/gitlab-org/-/epics/2158) | | POST | Paths ending with `/compare`| Git revision routes. | | POST | `.git/git-upload-pack` | To allow Git pull/clone. | -| POST | `/api/v4/internal` | [internal API routes](../../development/internal_api.md) | +| POST | `/api/v4/internal` | [internal API routes](../../development/internal_api/index.md) | | POST | `/admin/sidekiq` | To allow management of background jobs in the Admin UI | | POST | `/admin/geo` | To allow updating Geo Nodes in the administrator UI | | POST | `/api/v4/geo_replication`| To allow certain Geo-specific administrator UI actions on secondary sites | diff --git a/doc/administration/monitoring/performance/performance_bar.md b/doc/administration/monitoring/performance/performance_bar.md index 0befd9eac5b..d798feb71a9 100644 --- a/doc/administration/monitoring/performance/performance_bar.md +++ b/doc/administration/monitoring/performance/performance_bar.md @@ -114,7 +114,7 @@ for a given group: 1. On the left sidebar, select **Settings > Metrics and profiling** (`admin/application_settings/metrics_and_profiling`), and expand **Profiling - Performance bar**. -1. Click **Allow non-administrators to access to the performance bar**. +1. Click **Allow non-administrators access to the performance bar**. 1. In the **Allow access to members of the following group** field, provide the full path of the group allowed to access the performance. 1. Click **Save changes**. diff --git a/doc/administration/monitoring/prometheus/gitlab_metrics.md b/doc/administration/monitoring/prometheus/gitlab_metrics.md index 1d275010556..2f9c1e3bc9c 100644 --- a/doc/administration/monitoring/prometheus/gitlab_metrics.md +++ b/doc/administration/monitoring/prometheus/gitlab_metrics.md @@ -41,6 +41,7 @@ The following metrics are available: | `gitlab_cache_misses_total` | Counter | 10.2 | Cache read miss | `controller`, `action` | | `gitlab_cache_operation_duration_seconds` | Histogram | 10.2 | Cache access time | | | `gitlab_cache_operations_total` | Counter | 12.2 | Cache operations by controller or action | `controller`, `action`, `operation` | +| `gitlab_ci_pipeline_builder_scoped_variables_duration` | Histogram | 14.5 | Time in seconds it takes to create the scoped variables for a CI/CD job | `gitlab_ci_pipeline_creation_duration_seconds` | Histogram | 13.0 | Time in seconds it takes to create a CI/CD pipeline | | | `gitlab_ci_pipeline_size_builds` | Histogram | 13.1 | Total number of builds within a pipeline grouped by a pipeline source | `source` | | `job_waiter_started_total` | Counter | 12.9 | Number of batches of jobs started where a web request is waiting for the jobs to complete | `worker` | @@ -143,7 +144,7 @@ The following metrics are available: | `gitlab_snowplow_events_total` | Counter | 14.1 | Total number of GitLab Snowplow product intelligence events emitted | | | `gitlab_snowplow_failed_events_total` | Counter | 14.1 | Total number of GitLab Snowplow product intelligence events emission failures | | | `gitlab_snowplow_successful_events_total` | Counter | 14.1 | Total number of GitLab Snowplow product intelligence events emission successes | | -| `gitlab_ci_build_trace_errors_total` | Counter | 14.4 | Total amount of different error types on a build trace | `type` | +| `gitlab_ci_build_trace_errors_total` | Counter | 14.4 | Total amount of different error types on a build trace | `error_reason` | ## Metrics controlled by a feature flag @@ -189,9 +190,6 @@ configuration option in `gitlab.yml`. These metrics are served from the | `geo_lfs_objects` | Gauge | 10.2 | Total number of LFS objects available on primary | `url` | | `geo_lfs_objects_synced` | Gauge | 10.2 | Number of LFS objects synced on secondary | `url` | | `geo_lfs_objects_failed` | Gauge | 10.2 | Number of LFS objects failed to sync on secondary | `url` | -| `geo_attachments` | Gauge | 10.2 | Total number of file attachments available on primary | `url` | -| `geo_attachments_synced` | Gauge | 10.2 | Number of attachments synced on secondary | `url` | -| `geo_attachments_failed` | Gauge | 10.2 | Number of attachments failed to sync on secondary | `url` | | `geo_last_event_id` | Gauge | 10.2 | Database ID of the latest event log entry on the primary | `url` | | `geo_last_event_timestamp` | Gauge | 10.2 | UNIX timestamp of the latest event log entry on the primary | `url` | | `geo_cursor_last_event_id` | Gauge | 10.2 | Last database ID of the event log processed by the secondary | `url` | @@ -200,7 +198,6 @@ configuration option in `gitlab.yml`. These metrics are served from the | `geo_last_successful_status_check_timestamp` | Gauge | 10.2 | Last timestamp when the status was successfully updated | `url` | | `geo_lfs_objects_synced_missing_on_primary` | Gauge | 10.7 | Number of LFS objects marked as synced due to the file missing on the primary | `url` | | `geo_job_artifacts_synced_missing_on_primary` | Gauge | 10.7 | Number of job artifacts marked as synced due to the file missing on the primary | `url` | -| `geo_attachments_synced_missing_on_primary` | Gauge | 10.7 | Number of attachments marked as synced due to the file missing on the primary | `url` | | `geo_repositories_checksummed` | Gauge | 10.7 | Number of repositories checksummed on primary | `url` | | `geo_repositories_checksum_failed` | Gauge | 10.7 | Number of repositories failed to calculate the checksum on primary | `url` | | `geo_wikis_checksummed` | Gauge | 10.7 | Number of wikis checksummed on primary | `url` | @@ -275,6 +272,8 @@ configuration option in `gitlab.yml`. These metrics are served from the | `geo_uploads_synced` | Gauge | 14.1 | Number of uploads synced on secondary | `url` | | `geo_uploads_failed` | Gauge | 14.1 | Number of syncable uploads failed to sync on secondary | `url` | | `geo_uploads_registry` | Gauge | 14.1 | Number of uploads in the registry | `url` | +| `gitlab_sli:rails_request_apdex:total` | Counter | 14.4 | The number of request-apdex measurements, [more information the development documentation](../../../development/application_slis/rails_request_apdex.md) | `endpoint_id`, `feature_category`, `request_urgency` | +| `gitlab_sli:rails_request_apdex:success_total` | Counter | 14.4 | The number of succesful requests that met the target duration for their urgency. Devide by `gitlab_sli:rails_requests_apdex:total` to get a success ratio | `endpoint_id`, `feature_category`, `request_urgency` | ## Database load balancing metrics **(PREMIUM SELF)** diff --git a/doc/administration/monitoring/prometheus/index.md b/doc/administration/monitoring/prometheus/index.md index e04aad9c6b8..e86ca596955 100644 --- a/doc/administration/monitoring/prometheus/index.md +++ b/doc/administration/monitoring/prometheus/index.md @@ -345,6 +345,12 @@ memory, disk, and CPU utilization. [Read more about the node exporter](node_exporter.md). +### Puma exporter + +The Puma exporter allows you to measure various Puma metrics. + +[Read more about the Puma exporter](puma_exporter.md). + ### Redis exporter The Redis exporter allows you to measure various Redis metrics. diff --git a/doc/administration/monitoring/prometheus/puma_exporter.md b/doc/administration/monitoring/prometheus/puma_exporter.md index c348e74afaf..804c4243cfa 100644 --- a/doc/administration/monitoring/prometheus/puma_exporter.md +++ b/doc/administration/monitoring/prometheus/puma_exporter.md @@ -25,3 +25,5 @@ To enable the Puma exporter: for the changes to take effect. Prometheus begins collecting performance data from the Puma exporter exposed at `localhost:8083`. + +For more information on using Puma with GitLab, see [Puma](../../operations/puma.md). diff --git a/doc/administration/nfs.md b/doc/administration/nfs.md index f18c39af24a..2a2e9f05312 100644 --- a/doc/administration/nfs.md +++ b/doc/administration/nfs.md @@ -93,6 +93,27 @@ is moved to NFS. We are investigating the use of [fast lookup as the default](https://gitlab.com/groups/gitlab-org/-/epics/3104). +## Improving NFS performance with GitLab + +NFS performance with GitLab can in some cases be improved with +[direct Git access](gitaly/index.md#direct-access-to-git-in-gitlab) using +[Rugged](https://github.com/libgit2/rugged). + +From GitLab 12.1, GitLab automatically detects if Rugged can and should be used per storage. +If you previously enabled Rugged using the feature flag and you want to use automatic detection instead, +you must unset the feature flag: + +```shell +sudo gitlab-rake gitlab:features:unset_rugged +``` + +If the Rugged feature flag is explicitly set to either `true` or `false`, GitLab uses the value explicitly set. + +From GitLab 12.7, Rugged is only automatically enabled for use with Puma +if the [Puma thread count is set to `1`](../install/requirements.md#puma-settings). + +To use Rugged with a Puma thread count of more than `1`, enable Rugged using the [feature flag](../development/gitaly.md#legacy-rugged-code). + ## NFS server Installing the `nfs-kernel-server` package allows you to share directories with @@ -165,32 +186,6 @@ You may not need to disable NFS server delegation if you know you are using a ve the Linux kernel that has been fixed. That said, GitLab still encourages instance administrators to keep NFS server delegation disabled. -### Improving NFS performance with GitLab - -NFS performance with GitLab can in some cases be improved with -[direct Git access](gitaly/index.md#direct-access-to-git-in-gitlab) using -[Rugged](https://github.com/libgit2/rugged). - -NOTE: -From GitLab 12.1, it automatically detects if Rugged can and should be used per storage. - -If you previously enabled Rugged using the feature flag, you need to unset the feature flag by using: - -```shell -sudo gitlab-rake gitlab:features:unset_rugged -``` - -If the Rugged feature flag is explicitly set to either `true` or `false`, GitLab uses the value explicitly set. - -#### Improving NFS performance with Puma - -NOTE: -From GitLab 12.7, Rugged is not automatically enabled if Puma thread count is greater than `1`. - -If you want to use Rugged with Puma, [set Puma thread count to `1`](../install/requirements.md#puma-settings). - -If you want to use Rugged with Puma thread count more than `1`, Rugged can be enabled using the [feature flag](../development/gitaly.md#legacy-rugged-code). - ## NFS client The `nfs-common` provides NFS functionality without installing server components which @@ -232,7 +227,7 @@ Note there are several options that you should consider using: It's recommended that you use `hard` in your mount options, unless you have a specific reason to use `soft`. -On GitLab.com, we use `soft` because there were times when we had NFS servers +When GitLab.com used NFS, we used `soft` because there were times when we had NFS servers reboot and `soft` improved availability, but everyone's infrastructure is different. If your NFS is provided by on-premise storage arrays with redundant controllers, for example, you shouldn't need to worry about NFS server availability. diff --git a/doc/administration/object_storage.md b/doc/administration/object_storage.md index 6c77576cb27..8576b429213 100644 --- a/doc/administration/object_storage.md +++ b/doc/administration/object_storage.md @@ -2,7 +2,6 @@ stage: Enablement group: Distribution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference --- # Object storage **(FREE SELF)** @@ -77,7 +76,13 @@ Mattermost. See the [full table for a complete list](#storage-specific-configura However, backups can be configured with [server side encryption](../raketasks/backup_restore.md#s3-encrypted-buckets) separately. Enabling consolidated object storage enables object storage for all object -types. If you want to use local storage for specific object types, you can +types. If not all buckets are specified, `sudo gitlab-ctl reconfigure` may fail with the error like: + +```plaintext +Object storage for <object type> must have a bucket specified +``` + +If you want to use local storage for specific object types, you can [selectively disable object storages](#selectively-disabling-object-storage). Most types of objects, such as CI artifacts, LFS files, upload @@ -123,8 +128,8 @@ See the section on [ETag mismatch errors](#etag-mismatch) for more details. gitlab_rails['object_store']['objects']['pages']['bucket'] = '<pages>' ``` - For GitLab 9.4 or later, if you're using AWS IAM profiles, be sure to omit the - AWS access key and secret access key/value pairs. For example: + If you're using AWS IAM profiles, omit the AWS access key and secret access + key/value pairs. For example: ```ruby gitlab_rails['object_store']['connection'] = { @@ -558,7 +563,7 @@ supported by consolidated configuration form, refer to the following guides: | [Mattermost](https://docs.mattermost.com/administration/config-settings.html#file-storage)| **{dotted-circle}** No | | [Packages](packages/index.md#using-object-storage) (optional feature) | **{check-circle}** Yes | | [Dependency Proxy](packages/dependency_proxy.md#using-object-storage) (optional feature) **(PREMIUM SELF)** | **{check-circle}** Yes | -| [Pseudonymizer](pseudonymizer.md#configuration) (optional feature) **(ULTIMATE SELF)** | **{dotted-circle}** No | +| [Pseudonymizer](pseudonymizer.md) (optional feature) | **{dotted-circle}** No | | [Autoscale runner caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) (optional for improved performance) | **{dotted-circle}** No | | [Terraform state files](terraform_state.md#using-object-storage) | **{check-circle}** Yes | | [GitLab Pages content](pages/index.md#using-object-storage) | **{check-circle}** Yes | diff --git a/doc/administration/operations/extra_sidekiq_processes.md b/doc/administration/operations/extra_sidekiq_processes.md index 31cbd6c457b..02cb7ad0bca 100644 --- a/doc/administration/operations/extra_sidekiq_processes.md +++ b/doc/administration/operations/extra_sidekiq_processes.md @@ -30,6 +30,11 @@ can be started. > - [Sidekiq cluster moved](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/181) to GitLab Free in 12.10. > - [Sidekiq cluster became default](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/4140) in GitLab 13.0. +When starting multiple processes, the number of processes should +equal (and **not** exceed) the number of CPU cores you want to +dedicate to Sidekiq. Each Sidekiq process can use only 1 CPU +core, subject to the available workload and concurrency settings. + To start multiple processes: 1. Using the `sidekiq['queue_groups']` array setting, specify how many processes to @@ -166,23 +171,41 @@ one only processes imports and the other processes all other queues: ## Number of threads -Each process defined under `sidekiq` starts with a +By default each process defined under `sidekiq` starts with a number of threads that equals the number of queues, plus one spare thread. For example, a process that handles the `process_commit` and `post_receive` queues uses three threads in total. -## Manage concurrency +These thread run inside a single Ruby process, and each process +can only use a single CPU core. The usefulness of threading depends +on the work having some external dependencies to wait on, like database queries or +HTTP requests. Most Sidekiq deployments benefit from this threading, and when +running fewer queues in a process, increasing the thread count might be +even more desirable to make the most effective use of CPU resources. + +### Manage thread counts explicitly + +The correct maximum thread count (also called concurrency) depends on the workload. +Typical values range from `1` for highly CPU-bound tasks to `15` or higher for mixed +low-priority work. A reasonable starting range is `15` to `25` for a non-specialized +deployment. + +You can find example values used by GitLab.com by searching for `concurrency:` in +[the Helm charts](https://gitlab.com/gitlab-com/gl-infra/k8s-workloads/gitlab-com/-/blob/master/releases/gitlab/values/gprd.yaml.gotmpl). +The values vary according to the work each specific deployment of Sidekiq does. +Any other specialized deployments with processes dedicated to specific queues should +have the concurrency tuned according to: +have the concurrency tuned according to: -When setting the maximum concurrency, keep in mind this normally should -not exceed the number of CPU cores available. The values in the examples -below are arbitrary and not particular recommendations. +- The CPU usage of each type of process. +- The throughput achieved. Each thread requires a Redis connection, so adding threads may increase Redis latency and potentially cause client timeouts. See the [Sidekiq documentation about Redis](https://github.com/mperham/sidekiq/wiki/Using-Redis) for more details. -### When running Sidekiq cluster (default) +#### When running Sidekiq cluster (default) Running Sidekiq cluster is the default in GitLab 13.0 and later. @@ -203,7 +226,7 @@ Running Sidekiq cluster is the default in GitLab 13.0 and later. the other. Setting `min_concurrency` to `0` disables the limit. For each queue group, let `N` be one more than the number of queues. The -concurrency factor are set to: +concurrency is set to: 1. `N`, if it's between `min_concurrency` and `max_concurrency`. 1. `max_concurrency`, if `N` exceeds this value. @@ -215,7 +238,7 @@ regardless of the number of queues. When `min_concurrency` is greater than `max_concurrency`, it is treated as being equal to `max_concurrency`. -### When running a single Sidekiq process +#### When running a single Sidekiq process Running a single Sidekiq process is the default in GitLab 12.10 and earlier. diff --git a/doc/administration/operations/fast_ssh_key_lookup.md b/doc/administration/operations/fast_ssh_key_lookup.md index 8aa5af4c2bf..c99f94589d7 100644 --- a/doc/administration/operations/fast_ssh_key_lookup.md +++ b/doc/administration/operations/fast_ssh_key_lookup.md @@ -26,11 +26,8 @@ indexed lookup in the GitLab database. This page describes how to enable the fas lookup of authorized SSH keys. WARNING: -OpenSSH version 6.9+ is required because -`AuthorizedKeysCommand` must be able to accept a fingerprint. These -instructions break installations that use older versions of OpenSSH, such as -those included with CentOS 6 as of September 2017. If you want to use this -feature for CentOS 6, follow [the instructions on how to build and install a custom OpenSSH package](#compiling-a-custom-version-of-openssh-for-centos-6) before continuing. +OpenSSH version 6.9+ is required because `AuthorizedKeysCommand` must be +able to accept a fingerprint. Check the version of OpenSSH on your server. ## Fast lookup is required for Geo **(PREMIUM)** @@ -132,100 +129,43 @@ This is a brief overview. Please refer to the above instructions for more contex 1. Remove the `AuthorizedKeysCommand` lines from `/etc/ssh/sshd_config` or from `/assets/sshd_config` if you are using Omnibus Docker. 1. Reload `sshd`: `sudo service sshd reload`. -## Compiling a custom version of OpenSSH for CentOS 6 +## Use `gitlab-sshd` instead of OpenSSH -Building a custom version of OpenSSH is not necessary for Ubuntu 16.04 users, -since Ubuntu 16.04 ships with OpenSSH 7.2. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/299109) in GitLab 14.5. -It is also unnecessary for CentOS 7.4 users, as that version ships with -OpenSSH 7.4. If you are using CentOS 7.0 - 7.3, we strongly recommend that you -upgrade to CentOS 7.4 instead of following this procedure. This should be as -simple as running `yum update`. - -CentOS 6 users must build their own OpenSSH package to enable SSH lookups via -the database. The following instructions can be used to build OpenSSH 7.5: - -1. First, download the package and install the required packages: - - ```shell - sudo su - - cd /tmp - curl --remote-name "https://cdn.openbsd.org/pub/OpenBSD/OpenSSH/portable/openssh-7.5p1.tar.gz" - tar xzvf openssh-7.5p1.tar.gz - yum install rpm-build gcc make wget openssl-devel krb5-devel pam-devel libX11-devel xmkmf libXt-devel - ``` - -1. Prepare the build by copying files to the right place: - - ```shell - mkdir -p /root/rpmbuild/{SOURCES,SPECS} - cp ./openssh-7.5p1/contrib/redhat/openssh.spec /root/rpmbuild/SPECS/ - cp openssh-7.5p1.tar.gz /root/rpmbuild/SOURCES/ - cd /root/rpmbuild/SPECS - ``` - -1. Next, set the spec settings properly: - - ```shell - sed -i -e "s/%define no_gnome_askpass 0/%define no_gnome_askpass 1/g" openssh.spec - sed -i -e "s/%define no_x11_askpass 0/%define no_x11_askpass 1/g" openssh.spec - sed -i -e "s/BuildPreReq/BuildRequires/g" openssh.spec - ``` - -1. Build the RPMs: - - ```shell - rpmbuild -bb openssh.spec - ``` - -1. Ensure the RPMs were built: - - ```shell - ls -al /root/rpmbuild/RPMS/x86_64/ - ``` - - You should see something as the following: - - ```plaintext - total 1324 - drwxr-xr-x. 2 root root 4096 Jun 20 19:37 . - drwxr-xr-x. 3 root root 19 Jun 20 19:37 .. - -rw-r--r--. 1 root root 470828 Jun 20 19:37 openssh-7.5p1-1.x86_64.rpm - -rw-r--r--. 1 root root 490716 Jun 20 19:37 openssh-clients-7.5p1-1.x86_64.rpm - -rw-r--r--. 1 root root 17020 Jun 20 19:37 openssh-debuginfo-7.5p1-1.x86_64.rpm - -rw-r--r--. 1 root root 367516 Jun 20 19:37 openssh-server-7.5p1-1.x86_64.rpm - ``` - -1. Install the packages. OpenSSH packages replace `/etc/pam.d/sshd` - with their own versions, which may prevent users from logging in, so be sure - that the file is backed up and restored after installation: - - ```shell - timestamp=$(date +%s) - cp /etc/pam.d/sshd pam-ssh-conf-$timestamp - rpm -Uvh /root/rpmbuild/RPMS/x86_64/*.rpm - yes | cp pam-ssh-conf-$timestamp /etc/pam.d/sshd - ``` - -1. Verify the installed version. In another window, attempt to sign in to the - server: - - ```shell - ssh -v <your-centos-machine> +WARNING: +`gitlab-sshd` is in [**Alpha**](https://about.gitlab.com/handbook/product/gitlab-the-product/#alpha-beta-ga). +It is not ready for production use. + +`gitlab-sshd` is [a standalone SSH server](https://gitlab.com/gitlab-org/gitlab-shell/-/tree/main/internal/sshd) + written in Go. It is provided as a part of `gitlab-shell` package. It has a lower memory + use as a OpenSSH alternative and supports + [group access restriction by IP address](../../user/group/index.md) for applications + running behind the proxy. + +If you are considering switching from OpenSSH to `gitlab-sshd`, consider these concerns: + +- The `gitlab-sshd` component is only available for + [Cloud Native Helm Charts](https://docs.gitlab.com/charts/) deployments. +- `gitlab-sshd` supports the PROXY protocol. It can run behind proxy servers that rely + on it, such as HAProxy. +- `gitlab-sshd` does not share a SSH port with the system administrator's OpenSSH, + and requires a bind to port 22. +- `gitlab-sshd` **does not** support SSH certificates. + +To switch from OpenSSH to `gitlab-sshd`: + +1. Set the `gitlab-shell` charts `sshDaemon` option to + [`gitlab-sshd`](https://docs.gitlab.com/charts/charts/gitlab/gitlab-shell/index.html#installation-command-line-options). + For example: + + ```yaml + gitlab: + gitlab-shell: + sshDaemon: gitlab-sshd ``` - You should see a line that reads: "debug1: Remote protocol version 2.0, remote software version OpenSSH_7.5" - - If not, you may need to restart `sshd` (for example, `systemctl restart sshd.service`). - -1. *IMPORTANT!* Open a new SSH session to your server before exiting to make - sure everything is working! If you need to downgrade, simple install the - older package: - - ```shell - # Only run this if you run into a problem logging in - yum downgrade openssh-server openssh openssh-clients - ``` +1. Perform a Helm upgrade. ## SELinux support and limitations diff --git a/doc/administration/operations/moving_repositories.md b/doc/administration/operations/moving_repositories.md index 8aeaadc17e9..9cf7ac18c81 100644 --- a/doc/administration/operations/moving_repositories.md +++ b/doc/administration/operations/moving_repositories.md @@ -49,8 +49,8 @@ until the move has completed. To move repositories: -1. Ensure all storages are accessible to the GitLab instance. In this example, these are - `<original_storage_name>` and `<cluster_storage_name>`. +1. Ensure all [local and cluster storages](../gitaly/configure_gitaly.md#mixed-configuration) are accessible to the GitLab instance. In + this example, these are `<original_storage_name>` and `<cluster_storage_name>`. 1. [Configure repository storage weights](../repository_storage_paths.md#configure-where-new-repositories-are-stored) so that the new storages receives all new projects. This stops new projects from being created on existing storages while the migration is in progress. diff --git a/doc/administration/operations/puma.md b/doc/administration/operations/puma.md index 775761d655d..f1f02b606f5 100644 --- a/doc/administration/operations/puma.md +++ b/doc/administration/operations/puma.md @@ -224,3 +224,8 @@ in Puma when using the Linux package, and which ones have no corresponding count | `unicorn['exporter_enabled']` | `puma['exporter_enabled']` | | `unicorn['exporter_address']` | `puma['exporter_address']` | | `unicorn['exporter_port']` | `puma['exporter_port']` | + +## Puma exporter + +You can use the Puma exporter to measure various Puma metrics. For more information, see +[Puma exporter](../monitoring/prometheus/puma_exporter.md). diff --git a/doc/administration/operations/ssh_certificates.md b/doc/administration/operations/ssh_certificates.md index 77dc4eb180b..4fcc5d7c916 100644 --- a/doc/administration/operations/ssh_certificates.md +++ b/doc/administration/operations/ssh_certificates.md @@ -19,10 +19,8 @@ In such setups some external automated process is needed to constantly upload the new keys to GitLab. WARNING: -OpenSSH version 6.9+ is required because that version -introduced the `AuthorizedPrincipalsCommand` configuration option. If -using CentOS 6, you can [follow these instructions](fast_ssh_key_lookup.md#compiling-a-custom-version-of-openssh-for-centos-6) -to compile an up-to-date version. +OpenSSH version 6.9+ is required because `AuthorizedKeysCommand` must be +able to accept a fingerprint. Check the version of OpenSSH on your server. ## Why use OpenSSH certificates? diff --git a/doc/administration/package_information/deprecated_os.md b/doc/administration/package_information/deprecated_os.md index 35b333241b2..7234d68e4b2 100644 --- a/doc/administration/package_information/deprecated_os.md +++ b/doc/administration/package_information/deprecated_os.md @@ -64,6 +64,7 @@ The following lists the currently supported OSs and their possible EOL dates. | Debian 9 | GitLab CE / GitLab EE 9.3.0 | amd64 | 2022 | <https://wiki.debian.org/DebianReleases#Production_Releases> | | Debian 10 | GitLab CE / GitLab EE 12.2.0 | amd64, arm64 | TBD | <https://wiki.debian.org/DebianReleases#Production_Releases> | | OpenSUSE 15.2 | GitLab CE / GitLab EE 13.11.0 | x86_64, aarch64 | Dec 2021 | <https://en.opensuse.org/Lifetime> | +| OpenSUSE 15.3 | GitLab CE / GitLab EE 14.5.0 | x86_64, aarch64 | Nov 2022 | <https://en.opensuse.org/Lifetime> | | SLES 12 | GitLab EE 9.0.0 | x86_64 | Oct 2027 | <https://www.suse.com/lifecycle/> | | Ubuntu 18.04 | GitLab CE / GitLab EE 10.7.0 | amd64 | April 2023 | <https://wiki.ubuntu.com/Releases> | | Ubuntu 20.04 | GitLab CE / GitLab EE 13.2.0 | amd64, arm64 | April 2025 | <https://wiki.ubuntu.com/Releases> | diff --git a/doc/administration/package_information/deprecation_policy.md b/doc/administration/package_information/deprecation_policy.md index 80ce72d54f5..d45c2ea3127 100644 --- a/doc/administration/package_information/deprecation_policy.md +++ b/doc/administration/package_information/deprecation_policy.md @@ -1,7 +1,7 @@ --- stage: Enablement group: Distribution -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Deprecation policy **(FREE SELF)** @@ -35,9 +35,11 @@ This section lists steps necessary for deprecating and removing configuration. We can differentiate two different types of configuration: -- Sensitive: Configuration that can cause major service outage ( Data integrity, - installation integrity, preventing users from reaching the installation, etc.) -- Regular: Configuration that can make a feature unavailable but still makes the installation useable ( Change in default project/group settings, miscommunication with other components and similar ) +- Sensitive: Configuration that can cause major service outage (like data integrity, + installation integrity, or preventing users from reaching the installation) +- Regular: Configuration that can make a feature unavailable but still makes the + installation useable (like a change in default project/group settings, or + miscommunication with other components) We also need to differentiate deprecation and removal procedure. diff --git a/doc/administration/pages/index.md b/doc/administration/pages/index.md index 8a0d3f552bf..163eb5388b6 100644 --- a/doc/administration/pages/index.md +++ b/doc/administration/pages/index.md @@ -237,6 +237,7 @@ control over how the Pages daemon runs and serves content in your environment. | `log_verbose` | Verbose logging, true/false. | | `propagate_correlation_id` | Set to true (false by default) to re-use existing Correlation ID from the incoming request header `X-Request-ID` if present. If a reverse proxy sets this header, the value is propagated in the request chain. | | `max_connections` | Limit on the number of concurrent connections to the HTTP, HTTPS or proxy listeners. | +| `max_uri_length` | The maximum length of URIs accepted by GitLab Pages. Set to 0 for unlimited length. [Introduced](https://gitlab.com/gitlab-org/gitlab-pages/-/issues/659) in GitLab 14.5. | `metrics_address` | The address to listen on for metrics requests. | | `redirect_http` | Redirect pages from HTTP to HTTPS, true/false. | | `sentry_dsn` | The address for sending Sentry crash reporting to. | @@ -258,8 +259,8 @@ control over how the Pages daemon runs and serves content in your environment. | `FF_ENABLE_REDIRECTS` | Feature flag to enable/disable redirects (enabled by default). Read the [redirects documentation](../../user/project/pages/redirects.md#feature-flag-for-redirects) for more information. | | `FF_ENABLE_PLACEHOLDERS` | Feature flag to enable/disable rewrites (disabled by default). Read the [redirects documentation](../../user/project/pages/redirects.md#feature-flag-for-rewrites) for more information. | | `use_legacy_storage` | Temporarily-introduced parameter allowing to use legacy domain configuration source and storage. [Removed in 14.3](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6166). | - ---- +| `rate_limit_source_ip` | Rate limit per source IP in number of requests per second. Set to `0` to disable this feature. | +| `rate_limit_source_ip_burst` | Rate limit per source IP maximum burst allowed per second. | ## Advanced configuration @@ -647,7 +648,7 @@ To override the global maximum pages size for a specific group: ## Running GitLab Pages on a separate server You can run the GitLab Pages daemon on a separate server to decrease the load on -your main application server. +your main application server. This configuration does not support mutual TLS (mTLS). See the [corresponding feature proposal](https://gitlab.com/gitlab-org/gitlab-pages/-/issues/548) for more information. To configure GitLab Pages on a separate server: @@ -1031,6 +1032,38 @@ GitLab Pages are part of the [regular backup](../../raketasks/backup_restore.md) You should strongly consider running GitLab Pages under a different hostname than GitLab to prevent XSS attacks. +### Rate limits + +> [Introduced](https://gitlab.com/gitlab-org/gitlab-pages/-/issues/631) in GitLab 14.5. + +You can enforce source-IP rate limits to help minimize the risk of a Denial of Service (DoS) attack. GitLab Pages +uses a [token bucket algorithm](https://en.wikipedia.org/wiki/Token_bucket) to enforce rate limiting. By default, +requests that exceed the specified limits are reported but not rejected. + +Source-IP rate limits are enforced using the following: + +- `rate_limit_source_ip`: Set the maximum threshold in number of requests per second. Set to 0 to disable this feature. +- `rate_limit_source_ip_burst`: Sets the maximum threshold of number of requests allowed in an initial outburst of requests. + For example, when you load a web page that loads a number of resources at the same time. + +#### Enable source-IP rate limits + +1. Set rate limits in `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_pages['rate_limit_source_ip'] = 20.0 + gitlab_pages['rate_limit_source_ip_burst'] = 600 + ``` + +1. To reject requests that exceed the specified limits, enable the `FF_ENABLE_RATE_LIMITER` feature flag in + `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_pages['env'] = {'FF_ENABLE_RATE_LIMITER' => 'true'} + ``` + +1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/administration/pages/source.md b/doc/administration/pages/source.md index 278d792052a..3a277204d21 100644 --- a/doc/administration/pages/source.md +++ b/doc/administration/pages/source.md @@ -132,11 +132,11 @@ The Pages daemon doesn't listen to the outside world. https: false artifacts_server: false external_http: ["127.0.0.1:8090"] - secret_file: /home/git/gitlab/gitlab-pages/gitlab-pages-secret + secret_file: /home/git/gitlab/gitlab-pages-secret ``` 1. Add the following configuration file to - `/home/git/gitlab/gitlab-pages/gitlab-pages.conf`, and be sure to change + `/home/git/gitlab-pages/gitlab-pages.conf`, and be sure to change `example.io` to the FQDN from which you want to serve GitLab Pages and `gitlab.example.com` to the URL of your GitLab instance: @@ -159,12 +159,27 @@ The Pages daemon doesn't listen to the outside world. sudo -u git -H openssl rand -base64 32 > /home/git/gitlab/gitlab-pages-secret ``` -1. Edit `/etc/default/gitlab` and set `gitlab_pages_enabled` to `true` in - order to enable the pages daemon: +1. To enable the pages daemon: - ```ini - gitlab_pages_enabled=true - ``` + - If your system uses systemd as init, run: + + ```shell + sudo systemctl edit gitlab.target + ``` + + In the editor that opens, add the following and save the file: + + ```plaintext + [Unit] + Wants=gitlab-pages.service + ``` + + - If your system uses SysV init instead, edit `/etc/default/gitlab` and set + `gitlab_pages_enabled` to `true`: + + ```ini + gitlab_pages_enabled=true + ``` 1. Copy the `gitlab-pages` NGINX configuration file: diff --git a/doc/administration/postgresql/external.md b/doc/administration/postgresql/external.md index 8f0fe0ace87..67c5448a8a0 100644 --- a/doc/administration/postgresql/external.md +++ b/doc/administration/postgresql/external.md @@ -4,7 +4,7 @@ group: Database info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Configure GitLab using an external PostgreSQL service +# Configure GitLab using an external PostgreSQL service **(FREE SELF)** If you're hosting GitLab on a cloud provider, you can optionally use a managed service for PostgreSQL. For example, AWS offers a managed Relational diff --git a/doc/administration/postgresql/index.md b/doc/administration/postgresql/index.md index bce78bbccff..15425a6d9f2 100644 --- a/doc/administration/postgresql/index.md +++ b/doc/administration/postgresql/index.md @@ -2,26 +2,15 @@ stage: Enablement group: Database info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference --- -# Configuring PostgreSQL for scaling +# Configuring PostgreSQL for scaling **(FREE SELF)** In this section, you'll be guided through configuring a PostgreSQL database to be used with GitLab in one of our [reference architectures](../reference_architectures/index.md). There are essentially three setups to choose from. -## PostgreSQL replication and failover with Omnibus GitLab **(PREMIUM SELF)** - -This setup is for when you have installed GitLab using the -[Omnibus GitLab **Enterprise Edition** (EE) package](https://about.gitlab.com/install/?version=ee). - -All the tools that are needed like PostgreSQL, PgBouncer, and Patroni are bundled in -the package, so you can it to set up the whole PostgreSQL infrastructure (primary, replica). - -[> Read how to set up PostgreSQL replication and failover using Omnibus GitLab](replication_and_failover.md) - -## Standalone PostgreSQL using Omnibus GitLab **(FREE SELF)** +## Standalone PostgreSQL using Omnibus GitLab This setup is for when you have installed the [Omnibus GitLab packages](https://about.gitlab.com/install/) (CE or EE), @@ -29,7 +18,7 @@ to use the bundled PostgreSQL having only its service enabled. [> Read how to set up a standalone PostgreSQL instance using Omnibus GitLab](standalone.md) -## Provide your own PostgreSQL instance **(FREE SELF)** +## Provide your own PostgreSQL instance This setup is for when you have installed GitLab using the [Omnibus GitLab packages](https://about.gitlab.com/install/) (CE or EE), @@ -37,3 +26,13 @@ or installed it [from source](../../install/installation.md), but you want to us your own external PostgreSQL server. [> Read how to set up an external PostgreSQL instance](external.md) + +## PostgreSQL replication and failover with Omnibus GitLab **(PREMIUM SELF)** + +This setup is for when you have installed GitLab using the +[Omnibus GitLab **Enterprise Edition** (EE) package](https://about.gitlab.com/install/?version=ee). + +All the tools that are needed like PostgreSQL, PgBouncer, and Patroni are bundled in +the package, so you can use it to set up the whole PostgreSQL infrastructure (primary, replica). + +[> Read how to set up PostgreSQL replication and failover using Omnibus GitLab](replication_and_failover.md) diff --git a/doc/administration/postgresql/pgbouncer.md b/doc/administration/postgresql/pgbouncer.md index e215622bbc7..e5fef61540a 100644 --- a/doc/administration/postgresql/pgbouncer.md +++ b/doc/administration/postgresql/pgbouncer.md @@ -84,7 +84,7 @@ This content has been moved to a [new location](replication_and_failover.md#conf Do not backup or restore GitLab through a PgBouncer connection: it causes a GitLab outage. -[Read more about this and how to reconfigure backups](../../raketasks/backup_restore.md#backup-and-restore-for-installations-using-pgbouncer). +[Read more about this and how to reconfigure backups](../../raketasks/backup_restore.md#back-up-and-restore-for-installations-using-pgbouncer). ## Enable Monitoring @@ -172,7 +172,7 @@ ote_pid | tls Some database changes have to be done directly, and not through PgBouncer. -Read more about the affected tasks: [database restores](../../raketasks/backup_restore.md#backup-and-restore-for-installations-using-pgbouncer) +Read more about the affected tasks: [database restores](../../raketasks/backup_restore.md#back-up-and-restore-for-installations-using-pgbouncer) and [GitLab upgrades](../../update/zero_downtime.md#use-postgresql-ha). 1. To find the primary node, run the following on a database node: diff --git a/doc/administration/postgresql/replication_and_failover.md b/doc/administration/postgresql/replication_and_failover.md index dc569a81abf..01fe4bf64ba 100644 --- a/doc/administration/postgresql/replication_and_failover.md +++ b/doc/administration/postgresql/replication_and_failover.md @@ -501,7 +501,7 @@ in the Troubleshooting section before proceeding. Do not backup or restore GitLab through a PgBouncer connection: this causes a GitLab outage. -[Read more about this and how to reconfigure backups](../../raketasks/backup_restore.md#backup-and-restore-for-installations-using-pgbouncer). +[Read more about this and how to reconfigure backups](../../raketasks/backup_restore.md#back-up-and-restore-for-installations-using-pgbouncer). ### Ensure GitLab is running diff --git a/doc/administration/pseudonymizer.md b/doc/administration/pseudonymizer.md index da3a2e4b34c..bd6982bea12 100644 --- a/doc/administration/pseudonymizer.md +++ b/doc/administration/pseudonymizer.md @@ -6,33 +6,38 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Pseudonymizer **(ULTIMATE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/5532) in GitLab 11.1. +Your GitLab database contains sensitive information. To protect sensitive information +when you run analytics on your database, you can use the Pseudonymizer service, which: -As the GitLab database hosts sensitive information, using it unfiltered for analytics -implies high security requirements. To help alleviate this constraint, the Pseudonymizer -service is used to export GitLab data in a pseudonymized way. +1. Uses `HMAC(SHA256)` to mutate fields containing sensitive information. +1. Preserves references (referential integrity) between fields. +1. Exports your GitLab data, scrubbed of sensitive material. WARNING: -This process is not impervious. If the source data is available, it's possible for -a user to correlate data to the pseudonymized version. +If the source data is available, users can compare and correlate the scrubbed data +with the original. -The Pseudonymizer currently uses `HMAC(SHA256)` to mutate fields that shouldn't -be textually exported. This ensures that: +To generate a pseudonymized data set: -- the end-user of the data source cannot infer/revert the pseudonymized fields -- the referential integrity is maintained +1. [Configure Pseudonymizer](#configure-pseudonymizer) fields and output location. +1. [Enable Pseudonymizer data collection](#enable-pseudonymizer-data-collection). +1. Optional. [Generate a data set manually](#generate-data-set-manually). -## Configuration +## Configure Pseudonymizer -To configure the Pseudonymizer, you need to: +To use the Pseudonymizer, configure both the fields you want to anonymize, and the location to +store the scrubbed data: -- Provide a manifest file that describes which fields should be included or - pseudonymized ([example `manifest.yml` file](https://gitlab.com/gitlab-org/gitlab/-/tree/master/config/pseudonymizer.yml)). - A default manifest is provided with the GitLab installation, using a relative file path that resolves from the Rails root. - Alternatively, you can use an absolute file path. -- Use an object storage and specify the connection parameters in the `pseudonymizer.upload.connection` configuration option. - -[Read more about using object storage with GitLab](object_storage.md). +1. **Create a manifest file**: This file describes the fields to include or pseudonymize. + - **Default manifest** - GitLab provides a default manifest in your GitLab installation + ([example `manifest.yml` file](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/pseudonymizer.yml)). + To use the example manifest file, use the `config/pseudonymizer.yml` relative path + when you configure connection parameters. + - **Custom manifest** - To use a custom manifest file, use the absolute path to + the file when you configure the connection parameters. +1. **Configure connection parameters**: In the configuration method appropriate for + your version of GitLab, specify the [object storage](object_storage.md) + connection parameters (`pseudonymizer.upload.connection`). **For Omnibus installations:** @@ -50,7 +55,7 @@ To configure the Pseudonymizer, you need to: } ``` - If you are using AWS IAM profiles, be sure to omit the AWS access key and secret access key/value pairs. + If you are using AWS IAM profiles, omit the AWS access key and secret access key/value pairs. ```ruby gitlab_rails['pseudonymizer_upload_connection'] = { @@ -85,24 +90,34 @@ To configure the Pseudonymizer, you need to: 1. Save the file and [restart GitLab](restart_gitlab.md#installations-from-source) for the changes to take effect. -## Usage +## Enable Pseudonymizer data collection + +To enable data collection: + +1. On the top bar, select **Menu > Admin**. +1. On the left sidebar, select **Settings > Metrics and Profiling**, then expand + **Pseudonymizer data collection**. +1. Select **Enable Pseudonymizer data collection**. +1. Select **Save changes**. -You can optionally run the Pseudonymizer using the following environment variables: +## Generate data set manually -- `PSEUDONYMIZER_OUTPUT_DIR` - where to store the output CSV files (defaults to `/tmp`) -- `PSEUDONYMIZER_BATCH` - the batch size when querying the DB (defaults to `100000`) +You can also run the Pseudonymizer manually: -```shell -## Omnibus -sudo gitlab-rake gitlab:db:pseudonymizer +1. Set these environment variables: + - `PSEUDONYMIZER_OUTPUT_DIR` - Where to store the output CSV files. Defaults to `/tmp`. + These commands produce CSV files that can be quite large. Make sure the directory + can store a file at least 10% of the size of your database. + - `PSEUDONYMIZER_BATCH` - The batch size when querying the database. Defaults to `100000`. +1. Run the command appropriate for your application: + - **Omnibus GitLab**: + `sudo gitlab-rake gitlab:db:pseudonymizer` + - **Installations from source**: + `sudo -u git -H bundle exec rake gitlab:db:pseudonymizer RAILS_ENV=production` -## Source -sudo -u git -H bundle exec rake gitlab:db:pseudonymizer RAILS_ENV=production -``` +After you run the command, upload the output CSV files to your configured object +storage. After the upload completes, delete the output file from the local disk. -This produces some CSV files that might be very large, so make sure the -`PSEUDONYMIZER_OUTPUT_DIR` has sufficient space. As a rule of thumb, at least -10% of the database size is recommended. +## Related topics -After the pseudonymizer has run, the output CSV files should be uploaded to the -configured object storage and deleted from the local disk. +- [Using object storage with GitLab](object_storage.md). diff --git a/doc/administration/raketasks/github_import.md b/doc/administration/raketasks/github_import.md index f29e2a6c7f6..0cdfd1c28ff 100644 --- a/doc/administration/raketasks/github_import.md +++ b/doc/administration/raketasks/github_import.md @@ -6,8 +6,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w # GitHub import **(FREE SELF)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10308) in GitLab 9.1. - To retrieve and import GitHub repositories, you need a [GitHub personal access token](https://github.com/settings/tokens). A username should be passed as the second argument to the Rake task, which becomes the owner of the project. You can resume an import diff --git a/doc/administration/raketasks/ldap.md b/doc/administration/raketasks/ldap.md index 585d254e41d..62ddd9be2b3 100644 --- a/doc/administration/raketasks/ldap.md +++ b/doc/administration/raketasks/ldap.md @@ -38,13 +38,13 @@ rake gitlab:ldap:check[50] > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/14735) in GitLab 12.2. -The following task runs a [group sync](../auth/ldap/index.md#group-sync) immediately. This is valuable -when you'd like to update all configured group memberships against LDAP without +The following task runs a [group sync](../auth/ldap/ldap_synchronization.md#group-sync) immediately. +This is valuable when you'd like to update all configured group memberships against LDAP without waiting for the next scheduled group sync to be run. NOTE: If you'd like to change the frequency at which a group sync is performed, -[adjust the cron schedule](../auth/ldap/index.md#adjust-ldap-group-sync-schedule) +[adjust the cron schedule](../auth/ldap/ldap_synchronization.md#adjust-ldap-group-sync-schedule) instead. **Omnibus Installation** diff --git a/doc/administration/raketasks/project_import_export.md b/doc/administration/raketasks/project_import_export.md index e0ca7bfdeaf..13f8dfdccc2 100644 --- a/doc/administration/raketasks/project_import_export.md +++ b/doc/administration/raketasks/project_import_export.md @@ -6,9 +6,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Project import/export administration **(FREE SELF)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/3050) in GitLab 8.9. -> - From GitLab 11.3, import/export can use object storage automatically. - GitLab provides Rake tasks relating to project import and export. For more information, see: - [Project import/export documentation](../../user/project/settings/import_export.md). diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index e731085b0ce..9c3c33e1fa8 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -12,9 +12,10 @@ full list of reference architectures, see > - **Supported users (approximate):** 10,000 > - **High Availability:** Yes ([Praefect](#configure-praefect-postgresql) needs a third-party PostgreSQL solution for HA) -> - **Cloud Native Hybrid:** [Yes](#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) -> - **Test requests per second (RPS) rates:** API: 200 RPS, Web: 20 RPS, Git (Pull): 20 RPS, Git (Push): 4 RPS -> - **[Latest 10k weekly performance testing results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k)** +> - **Cloud Native Hybrid Alternative:** [Yes](#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) +> - **Performance tested daily with the [GitLab Performance Tool](https://gitlab.com/gitlab-org/quality/performance)**: +> - **Test requests per second (RPS) rates:** API: 200 RPS, Web: 20 RPS, Git (Pull): 20 RPS, Git (Push): 4 RPS +> - **[Latest Results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/10k)** | Service | Nodes | Configuration | GCP | AWS | Azure | |-----------------------------------------------------|-------------|-------------------------|------------------|--------------|-----------| @@ -36,7 +37,7 @@ full list of reference architectures, see <!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix --> <!-- markdownlint-disable MD029 --> -1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. Google Cloud SQL and AWS RDS are known to work, however Azure Database for PostgreSQL is [not recommended](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61) due to performance issues. Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However it is also used optionally by Prometheus for Omnibus auto host discovery. +1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. Google Cloud SQL and Amazon RDS are known to work, however Azure Database for PostgreSQL is [not recommended](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61) due to performance issues. Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However it is also used optionally by Prometheus for Omnibus auto host discovery. 2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS Elasticache are known to work. 3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work. 4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work. @@ -279,7 +280,7 @@ The basic ports to be used are shown in the table below. | 443 | 443 | TCP or HTTPS (*1*) (*2*) | | 22 | 22 | TCP | -- (*1*): [Web terminal](../../ci/environments/index.md#web-terminals) support requires +- (*1*): [Web terminal](../../ci/environments/index.md#web-terminals-deprecated) support requires your load balancer to correctly handle WebSocket connections. When using HTTP or HTTPS proxying, this means your load balancer must be configured to pass through the `Connection` and `Upgrade` hop-by-hop headers. See the @@ -2085,7 +2086,7 @@ on what features you intend to use: | [Mattermost](https://docs.mattermost.com/administration/config-settings.html#file-storage)| No | | [Packages](../packages/index.md#using-object-storage) (optional feature) | Yes | | [Dependency Proxy](../packages/dependency_proxy.md#using-object-storage) (optional feature) | Yes | -| [Pseudonymizer](../pseudonymizer.md#configuration) (optional feature) **(ULTIMATE SELF)** | No | +| [Pseudonymizer](../pseudonymizer.md) (optional feature) | No | | [Autoscale runner caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) (optional for improved performance) | No | | [Terraform state files](../terraform_state.md#using-object-storage) | Yes | @@ -2199,7 +2200,7 @@ services where applicable): <!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix --> <!-- markdownlint-disable MD029 --> -1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. Google Cloud SQL and AWS RDS are known to work, however Azure Database for PostgreSQL is [not recommended](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61) due to performance issues. Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However it is also used optionally by Prometheus for Omnibus auto host discovery. +1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. Google Cloud SQL and Amazon RDS are known to work, however Azure Database for PostgreSQL is [not recommended](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61) due to performance issues. Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However it is also used optionally by Prometheus for Omnibus auto host discovery. 2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS Elasticache are known to work. 3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work. 4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work. diff --git a/doc/administration/reference_architectures/1k_users.md b/doc/administration/reference_architectures/1k_users.md index ae832c2226f..5488d8d33a6 100644 --- a/doc/administration/reference_architectures/1k_users.md +++ b/doc/administration/reference_architectures/1k_users.md @@ -20,8 +20,9 @@ many organizations. > follow a modified [3K reference architecture](3k_users.md#supported-modifications-for-lower-user-counts-ha). > - **Cloud Native Hybrid:** No. For a cloud native hybrid environment, you > can follow a [modified hybrid reference architecture](#cloud-native-hybrid-reference-architecture-with-helm-charts). -> - **Test requests per second (RPS) rates:** API: 20 RPS, Web: 2 RPS, Git (Pull): 2 RPS, Git (Push): 1 RPS -> - **[Latest 1k weekly performance testing results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/1k)** +> - **Performance tested daily with the [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance)**: +> - **Test requests per second (RPS) rates:** API: 20 RPS, Web: 2 RPS, Git (Pull): 2 RPS, Git (Push): 1 RPS +> - **[Latest Results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/1k)** | Users | Configuration | GCP | AWS | Azure | |--------------|-------------------------|----------------|--------------|----------| diff --git a/doc/administration/reference_architectures/25k_users.md b/doc/administration/reference_architectures/25k_users.md index 267f81efd91..25cafbe667b 100644 --- a/doc/administration/reference_architectures/25k_users.md +++ b/doc/administration/reference_architectures/25k_users.md @@ -12,9 +12,10 @@ full list of reference architectures, see > - **Supported users (approximate):** 25,000 > - **High Availability:** Yes ([Praefect](#configure-praefect-postgresql) needs a third-party PostgreSQL solution for HA) -> - **Cloud Native Hybrid:** [Yes](#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) -> - **Test requests per second (RPS) rates:** API: 500 RPS, Web: 50 RPS, Git (Pull): 50 RPS, Git (Push): 10 RPS -> - **[Latest 25k weekly performance testing results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/25k)** +> - **Cloud Native Hybrid Alternative:** [Yes](#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) +> - **Performance tested weekly with the [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance)**: +> - **Test requests per second (RPS) rates:** API: 500 RPS, Web: 50 RPS, Git (Pull): 50 RPS, Git (Push): 10 RPS +> - **[Latest Results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/25k)** | Service | Nodes | Configuration | GCP | AWS | Azure | |---------------------------------------------------|-------------|-------------------------|------------------|--------------|-----------| @@ -36,7 +37,7 @@ full list of reference architectures, see <!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix --> <!-- markdownlint-disable MD029 --> -1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. Google Cloud SQL and AWS RDS are known to work, however Azure Database for PostgreSQL is [not recommended](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61) due to performance issues. Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However it is also used optionally by Prometheus for Omnibus auto host discovery. +1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. Google Cloud SQL and Amazon RDS are known to work, however Azure Database for PostgreSQL is [not recommended](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61) due to performance issues. Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However it is also used optionally by Prometheus for Omnibus auto host discovery. 2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS Elasticache are known to work. 3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work. 4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work. @@ -282,7 +283,7 @@ The basic ports to be used are shown in the table below. | 443 | 443 | TCP or HTTPS (*1*) (*2*) | | 22 | 22 | TCP | -- (*1*): [Web terminal](../../ci/environments/index.md#web-terminals) support requires +- (*1*): [Web terminal](../../ci/environments/index.md#web-terminals-deprecated) support requires your load balancer to correctly handle WebSocket connections. When using HTTP or HTTPS proxying, this means your load balancer must be configured to pass through the `Connection` and `Upgrade` hop-by-hop headers. See the @@ -2091,7 +2092,7 @@ on what features you intend to use: | [Mattermost](https://docs.mattermost.com/administration/config-settings.html#file-storage)| No | | [Packages](../packages/index.md#using-object-storage) (optional feature) | Yes | | [Dependency Proxy](../packages/dependency_proxy.md#using-object-storage) (optional feature) | Yes | -| [Pseudonymizer](../pseudonymizer.md#configuration) (optional feature) **(ULTIMATE SELF)** | No | +| [Pseudonymizer](../pseudonymizer.md) (optional feature) | No | | [Autoscale runner caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) (optional for improved performance) | No | | [Terraform state files](../terraform_state.md#using-object-storage) | Yes | @@ -2199,7 +2200,7 @@ services where applicable): <!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix --> <!-- markdownlint-disable MD029 --> -1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. Google Cloud SQL and AWS RDS are known to work, however Azure Database for PostgreSQL is [not recommended](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61) due to performance issues. Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However it is also used optionally by Prometheus for Omnibus auto host discovery. +1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. Google Cloud SQL and Amazon RDS are known to work, however Azure Database for PostgreSQL is [not recommended](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61) due to performance issues. Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However it is also used optionally by Prometheus for Omnibus auto host discovery. 2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS Elasticache are known to work. 3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work. 4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work. diff --git a/doc/administration/reference_architectures/2k_users.md b/doc/administration/reference_architectures/2k_users.md index 23b15b563f7..e619294704f 100644 --- a/doc/administration/reference_architectures/2k_users.md +++ b/doc/administration/reference_architectures/2k_users.md @@ -14,8 +14,9 @@ For a full list of reference architectures, see > - **High Availability:** No. For a highly-available environment, you can > follow a modified [3K reference architecture](3k_users.md#supported-modifications-for-lower-user-counts-ha). > - **Cloud Native Hybrid:** [Yes](#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) -> - **Test requests per second (RPS) rates:** API: 40 RPS, Web: 4 RPS, Git (Pull): 4 RPS, Git (Push): 1 RPS -> - **[Latest 2k weekly performance testing results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/2k)** +> - **Performance tested daily with the [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance)**: +> - **Test requests per second (RPS) rates:** API: 40 RPS, Web: 4 RPS, Git (Pull): 4 RPS, Git (Push): 1 RPS +> - **[Latest Results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/2k)** | Service | Nodes | Configuration | GCP | AWS | Azure | |------------------------------------------|--------|-------------------------|-----------------|--------------|----------| @@ -29,7 +30,7 @@ For a full list of reference architectures, see | NFS server (optional, not recommended) | 1 | 4 vCPU, 3.6 GB memory | `n1-highcpu-4` | `c5.xlarge` | `F4s v2` | <!-- markdownlint-disable MD029 --> -1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. Google Cloud SQL and AWS RDS are known to work, however Azure Database for PostgreSQL is [not recommended](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61) due to performance issues. Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However it is also used optionally by Prometheus for Omnibus auto host discovery. +1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. Google Cloud SQL and Amazon RDS are known to work, however Azure Database for PostgreSQL is [not recommended](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61) due to performance issues. Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However it is also used optionally by Prometheus for Omnibus auto host discovery. 2. Can be optionally run as reputable third-party external PaaS Redis solutions. Google Memorystore and AWS Elasticache are known to work. 3. Can be optionally run as reputable third-party load balancing services (LB PaaS). AWS ELB is known to work. 4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work. @@ -176,7 +177,7 @@ table: | 443 | 443 | TCP or HTTPS (*1*) (*2*) | | 22 | 22 | TCP | -- (*1*): [Web terminal](../../ci/environments/index.md#web-terminals) support +- (*1*): [Web terminal](../../ci/environments/index.md#web-terminals-deprecated) support requires your load balancer to correctly handle WebSocket connections. When using HTTP or HTTPS proxying, your load balancer must be configured to pass through the `Connection` and `Upgrade` hop-by-hop headers. For @@ -352,7 +353,7 @@ Omnibus: ```ruby ## Enable Redis redis['enable'] = true - + # Avoid running unnecessary services on the Redis server gitaly['enable'] = false postgresql['enable'] = false @@ -922,7 +923,7 @@ on what features you intend to use: | [Mattermost](https://docs.mattermost.com/administration/config-settings.html#file-storage)| No | | [Packages](../packages/index.md#using-object-storage) (optional feature) | Yes | | [Dependency Proxy](../packages/dependency_proxy.md#using-object-storage) (optional feature) | Yes | -| [Pseudonymizer](../pseudonymizer.md#configuration) (optional feature) **(ULTIMATE SELF)** | No | +| [Pseudonymizer](../pseudonymizer.md) (optional feature) | No | | [Autoscale runner caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) (optional for improved performance) | No | | [Terraform state files](../terraform_state.md#using-object-storage) | Yes | @@ -1027,7 +1028,7 @@ services where applicable): <!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix --> <!-- markdownlint-disable MD029 --> -1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. Google Cloud SQL and AWS RDS are known to work, however Azure Database for PostgreSQL is [not recommended](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61) due to performance issues. Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However it is also used optionally by Prometheus for Omnibus auto host discovery. +1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. Google Cloud SQL and Amazon RDS are known to work, however Azure Database for PostgreSQL is [not recommended](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61) due to performance issues. Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However it is also used optionally by Prometheus for Omnibus auto host discovery. 2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS Elasticache are known to work. 3. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work. <!-- markdownlint-enable MD029 --> diff --git a/doc/administration/reference_architectures/3k_users.md b/doc/administration/reference_architectures/3k_users.md index 575dd22b729..9332ae8d271 100644 --- a/doc/administration/reference_architectures/3k_users.md +++ b/doc/administration/reference_architectures/3k_users.md @@ -22,9 +22,10 @@ For a full list of reference architectures, see > - **Supported users (approximate):** 3,000 > - **High Availability:** Yes, although [Praefect](#configure-praefect-postgresql) needs a third-party PostgreSQL solution -> - **Cloud Native Hybrid:** [Yes](#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) -> - **Test requests per second (RPS) rates:** API: 60 RPS, Web: 6 RPS, Git (Pull): 6 RPS, Git (Push): 1 RPS -> - **[Latest 3k weekly performance testing results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/3k)** +> - **Cloud Native Hybrid Alternative:** [Yes](#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) +> - **Performance tested weekly with the [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance)**: +> - **Test requests per second (RPS) rates:** API: 60 RPS, Web: 6 RPS, Git (Pull): 6 RPS, Git (Push): 1 RPS +> - **[Latest Results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/3k)** | Service | Nodes | Configuration | GCP | AWS | Azure | |--------------------------------------------|-------------|-----------------------|-----------------|--------------|----------| @@ -45,7 +46,7 @@ For a full list of reference architectures, see <!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix --> <!-- markdownlint-disable MD029 --> -1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. Google Cloud SQL and AWS RDS are known to work, however Azure Database for PostgreSQL is [not recommended](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61) due to performance issues. Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However it is also used optionally by Prometheus for Omnibus auto host discovery. +1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. Google Cloud SQL and Amazon RDS are known to work, however Azure Database for PostgreSQL is [not recommended](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61) due to performance issues. Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However it is also used optionally by Prometheus for Omnibus auto host discovery. 2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS Elasticache are known to work. 3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work. 4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work. @@ -296,7 +297,7 @@ The basic ports to be used are shown in the table below. | 443 | 443 | TCP or HTTPS (*1*) (*2*) | | 22 | 22 | TCP | -- (*1*): [Web terminal](../../ci/environments/index.md#web-terminals) support requires +- (*1*): [Web terminal](../../ci/environments/index.md#web-terminals-deprecated) support requires your load balancer to correctly handle WebSocket connections. When using HTTP or HTTPS proxying, this means your load balancer must be configured to pass through the `Connection` and `Upgrade` hop-by-hop headers. See the @@ -2039,7 +2040,7 @@ on what features you intend to use: | [Mattermost](https://docs.mattermost.com/administration/config-settings.html#file-storage)| No | | [Packages](../packages/index.md#using-object-storage) (optional feature) | Yes | | [Dependency Proxy](../packages/dependency_proxy.md#using-object-storage) (optional feature) | Yes | -| [Pseudonymizer](../pseudonymizer.md#configuration) (optional feature) **(ULTIMATE SELF)** | No | +| [Pseudonymizer](../pseudonymizer.md) (optional feature) | No | | [Autoscale runner caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) (optional for improved performance) | No | | [Terraform state files](../terraform_state.md#using-object-storage) | Yes | @@ -2106,7 +2107,7 @@ but with smaller performance requirements, several modifications can be consider - GitLab Rails and Sidekiq: Stateless services don't have a minimum node count. Two are enough for redundancy. - Gitaly and Praefect: A quorum is not strictly necessary. Two Gitaly nodes and two Praefect nodes are enough for redundancy. - Running select components in reputable Cloud PaaS solutions: Select components of the GitLab setup can instead be run on Cloud Provider PaaS solutions. By doing this, additional dependent components can also be removed: - - PostgreSQL: Can be run on reputable Cloud PaaS solutions such as Google Cloud SQL or AWS RDS. In this setup, the PgBouncer and Consul nodes are no longer required: + - PostgreSQL: Can be run on reputable Cloud PaaS solutions such as Google Cloud SQL or Amazon RDS. In this setup, the PgBouncer and Consul nodes are no longer required: - Consul may still be desired if [Prometheus](../monitoring/prometheus/index.md) auto discovery is a requirement, otherwise you would need to [manually add scrape configurations](../monitoring/prometheus/index.md#adding-custom-scrape-configurations) for all nodes. - As Redis Sentinel runs on the same box as Consul in this architecture, it may need to be run on a separate box if Redis is still being run via Omnibus. - Redis: Can be run on reputable Cloud PaaS solutions such as Google Memorystore and AWS ElastiCache. In this setup, the Redis Sentinel is no longer required. @@ -2170,7 +2171,7 @@ services where applicable): <!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix --> <!-- markdownlint-disable MD029 --> -1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. Google Cloud SQL and AWS RDS are known to work, however Azure Database for PostgreSQL is [not recommended](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61) due to performance issues. Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However it is also used optionally by Prometheus for Omnibus auto host discovery. +1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. Google Cloud SQL and Amazon RDS are known to work, however Azure Database for PostgreSQL is [not recommended](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61) due to performance issues. Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However it is also used optionally by Prometheus for Omnibus auto host discovery. 2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS Elasticache are known to work. 3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work. 4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work. diff --git a/doc/administration/reference_architectures/50k_users.md b/doc/administration/reference_architectures/50k_users.md index be44f464e7e..bbdf798d9ad 100644 --- a/doc/administration/reference_architectures/50k_users.md +++ b/doc/administration/reference_architectures/50k_users.md @@ -12,9 +12,10 @@ full list of reference architectures, see > - **Supported users (approximate):** 50,000 > - **High Availability:** Yes ([Praefect](#configure-praefect-postgresql) needs a third-party PostgreSQL solution for HA) -> - **Cloud Native Hybrid:** [Yes](#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) -> - **Test requests per second (RPS) rates:** API: 1000 RPS, Web: 100 RPS, Git (Pull): 100 RPS, Git (Push): 20 RPS -> - **[Latest 50k weekly performance testing results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/50k)** +> - **Cloud Native Hybrid Alternative:** [Yes](#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) +> - **Performance tested weekly with the [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance)**: +> - **Test requests per second (RPS) rates:** API: 1000 RPS, Web: 100 RPS, Git (Pull): 100 RPS, Git (Push): 20 RPS +> - **[Latest Results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/50k)** | Service | Nodes | Configuration | GCP | AWS | Azure | |---------------------------------------------------|-------------|-------------------------|------------------|---------------|-----------| @@ -36,7 +37,7 @@ full list of reference architectures, see <!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix --> <!-- markdownlint-disable MD029 --> -1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. Google Cloud SQL and AWS RDS are known to work, however Azure Database for PostgreSQL is [not recommended](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61) due to performance issues. Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However it is also used optionally by Prometheus for Omnibus auto host discovery. +1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. Google Cloud SQL and Amazon RDS are known to work, however Azure Database for PostgreSQL is [not recommended](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61) due to performance issues. Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However it is also used optionally by Prometheus for Omnibus auto host discovery. 2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS Elasticache are known to work. 3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work. 4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work. @@ -288,7 +289,7 @@ The basic ports to be used are shown in the table below. | 443 | 443 | TCP or HTTPS (*1*) (*2*) | | 22 | 22 | TCP | -- (*1*): [Web terminal](../../ci/environments/index.md#web-terminals) support requires +- (*1*): [Web terminal](../../ci/environments/index.md#web-terminals-deprecated) support requires your load balancer to correctly handle WebSocket connections. When using HTTP or HTTPS proxying, this means your load balancer must be configured to pass through the `Connection` and `Upgrade` hop-by-hop headers. See the @@ -2105,7 +2106,7 @@ on what features you intend to use: | [Mattermost](https://docs.mattermost.com/administration/config-settings.html#file-storage)| No | | [Packages](../packages/index.md#using-object-storage) (optional feature) | Yes | | [Dependency Proxy](../packages/dependency_proxy.md#using-object-storage) (optional feature) | Yes | -| [Pseudonymizer](../pseudonymizer.md#configuration) (optional feature) **(ULTIMATE SELF)** | No | +| [Pseudonymizer](../pseudonymizer.md) (optional feature) | No | | [Autoscale runner caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) (optional for improved performance) | No | | [Terraform state files](../terraform_state.md#using-object-storage) | Yes | @@ -2213,7 +2214,7 @@ services where applicable): <!-- Disable ordered list rule https://github.com/DavidAnson/markdownlint/blob/main/doc/Rules.md#md029---ordered-list-item-prefix --> <!-- markdownlint-disable MD029 --> -1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. Google Cloud SQL and AWS RDS are known to work, however Azure Database for PostgreSQL is [not recommended](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61) due to performance issues. Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However it is also used optionally by Prometheus for Omnibus auto host discovery. +1. Can be optionally run on reputable third-party external PaaS PostgreSQL solutions. Google Cloud SQL and Amazon RDS are known to work, however Azure Database for PostgreSQL is [not recommended](https://gitlab.com/gitlab-org/quality/reference-architectures/-/issues/61) due to performance issues. Consul is primarily used for PostgreSQL high availability so can be ignored when using a PostgreSQL PaaS setup. However it is also used optionally by Prometheus for Omnibus auto host discovery. 2. Can be optionally run on reputable third-party external PaaS Redis solutions. Google Memorystore and AWS Elasticache are known to work. 3. Can be optionally run on reputable third-party load balancing services (LB PaaS). AWS ELB is known to work. 4. Should be run on reputable third-party object storage (storage PaaS) for cloud implementations. Google Cloud Storage and AWS S3 are known to work. diff --git a/doc/administration/reference_architectures/5k_users.md b/doc/administration/reference_architectures/5k_users.md index a5526986be1..a1921f50e4e 100644 --- a/doc/administration/reference_architectures/5k_users.md +++ b/doc/administration/reference_architectures/5k_users.md @@ -19,9 +19,10 @@ costly-to-operate environment by using the > - **Supported users (approximate):** 5,000 > - **High Availability:** Yes ([Praefect](#configure-praefect-postgresql) needs a third-party PostgreSQL solution for HA) -> - **Cloud Native Hybrid:** [Yes](#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) -> - **Test requests per second (RPS) rates:** API: 100 RPS, Web: 10 RPS, Git (Pull): 10 RPS, Git (Push): 2 RPS -> - **[Latest 5k weekly performance testing results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/5k)** +> - **Cloud Native Hybrid Alternative:** [Yes](#cloud-native-hybrid-reference-architecture-with-helm-charts-alternative) +> - **Performance tested weekly with the [GitLab Performance Tool (GPT)](https://gitlab.com/gitlab-org/quality/performance)**: +> - **Test requests per second (RPS) rates:** API: 100 RPS, Web: 10 RPS, Git (Pull): 10 RPS, Git (Push): 2 RPS +> - **[Latest Results](https://gitlab.com/gitlab-org/quality/performance/-/wikis/Benchmarks/Latest/5k)** | Service | Nodes | Configuration | GCP | AWS | Azure | |--------------------------------------------|-------------|-------------------------|-----------------|--------------|----------| @@ -288,7 +289,7 @@ The basic ports to be used are shown in the table below. | 443 | 443 | TCP or HTTPS (*1*) (*2*) | | 22 | 22 | TCP | -- (*1*): [Web terminal](../../ci/environments/index.md#web-terminals) support requires +- (*1*): [Web terminal](../../ci/environments/index.md#web-terminals-deprecated) support requires your load balancer to correctly handle WebSocket connections. When using HTTP or HTTPS proxying, this means your load balancer must be configured to pass through the `Connection` and `Upgrade` hop-by-hop headers. See the @@ -2033,7 +2034,7 @@ on what features you intend to use: | [Mattermost](https://docs.mattermost.com/administration/config-settings.html#file-storage)| No | | [Packages](../packages/index.md#using-object-storage) (optional feature) | Yes | | [Dependency Proxy](../packages/dependency_proxy.md#using-object-storage) (optional feature) | Yes | -| [Pseudonymizer](../pseudonymizer.md#configuration) (optional feature) **(ULTIMATE SELF)** | No | +| [Pseudonymizer](../pseudonymizer.md) (optional feature) | No | | [Autoscale runner caching](https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching) (optional for improved performance) | No | | [Terraform state files](../terraform_state.md#using-object-storage) | Yes | diff --git a/doc/administration/repository_storage_paths.md b/doc/administration/repository_storage_paths.md index ed50d0e7263..00aa8c214c5 100644 --- a/doc/administration/repository_storage_paths.md +++ b/doc/administration/repository_storage_paths.md @@ -14,7 +14,7 @@ storage is either: - A `path`, which points directly to the directory where the repositories are stored. GitLab directly accessing a directory containing repositories [is deprecated](https://gitlab.com/gitlab-org/gitaly/-/issues/1690). - GitLab should be configured to access GitLab repositories though a `gitaly_address`. + GitLab should be configured to access GitLab repositories through a `gitaly_address`. GitLab allows you to define multiple repository storages to distribute the storage load between several mount points. For example: diff --git a/doc/administration/repository_storage_types.md b/doc/administration/repository_storage_types.md index d2bab9a1e04..a85f678fe95 100644 --- a/doc/administration/repository_storage_types.md +++ b/doc/administration/repository_storage_types.md @@ -101,8 +101,12 @@ To look up a project's hash path using a Rails console: #### From hashed path to project name -Administrators can look up a project's name from its hashed storage path using a Rails console. To -look up a project's name from its hashed storage path: +Administrators can look up a project's name from its hashed storage path using: + +- A Rails console. +- The `config` file in the `*.git` directory. + +To look up a project's name using the Rails console: 1. Start a [Rails console](operations/rails_console.md#starting-a-rails-console-session). 1. Run a command similar to this example: @@ -121,6 +125,14 @@ The output includes the project ID and the project name. For example: => #<Project id:16 it/supportteam/ticketsystem> ``` +To look up a project's name using the `config` file in the `*.git` directory: + +1. Navigate to the to the `*.git` directory. This directory is located in `/var/opt/gitlab/git-data/repositories/@hashed/`, where the first four + characters of the hash are the first two directories in the path under `@hashed/`. For example, on a default Omnibus GitLab installation the + `*.git` directory of the hash `b17eb17ef6d19c7a5b1ee83b907c595526dcb1eb06db8227d650d5dda0a9f4ce8cd9` would be + `/var/opt/gitlab/git-data/repositories/@hashed/b1/7e/b17ef6d19c7a5b1ee83b907c595526dcb1eb06db8227d650d5dda0a9f4ce8cd9.git`. +1. Open the `config` file and locate the `fullpath=` key under `[gitlab]`. + ### Hashed object pools > [Introduced](https://gitlab.com/gitlab-org/gitaly/-/issues/1606) in GitLab 12.1. diff --git a/doc/administration/restart_gitlab.md b/doc/administration/restart_gitlab.md index b8f09b00773..9579d413bf8 100644 --- a/doc/administration/restart_gitlab.md +++ b/doc/administration/restart_gitlab.md @@ -103,39 +103,15 @@ If you have followed the official installation guide to [install GitLab from source](../install/installation.md), run the following command to restart GitLab: ```shell -sudo service gitlab restart -``` +# For systems running systemd +sudo systemctl restart gitlab.target -The output should be similar to this: - -```plaintext -Shutting down GitLab Puma -Shutting down GitLab Sidekiq -Shutting down GitLab Workhorse -Shutting down GitLab MailRoom -... -GitLab is not running. -Starting GitLab Puma -Starting GitLab Sidekiq -Starting GitLab Workhorse -Starting GitLab MailRoom -... -The GitLab Puma web server with pid 28059 is running. -The GitLab Sidekiq job dispatcher with pid 28176 is running. -The GitLab Workhorse with pid 28122 is running. -The GitLab MailRoom email processor with pid 28114 is running. -GitLab and all its components are up and running. +# For systems running SysV init +sudo service gitlab restart ``` This should restart Puma, Sidekiq, GitLab Workhorse, and [Mailroom](reply_by_email.md) -(if enabled). The init service file that does all the magic can be found on -your server in `/etc/init.d/gitlab`. - ---- - -If you are using other init systems, like `systemd`, you can check the -[GitLab Recipes](https://gitlab.com/gitlab-org/gitlab-recipes/tree/master/init) repository for some unofficial services. These are -**not** officially supported so use them at your own risk. +(if enabled). ## Helm chart installations diff --git a/doc/administration/sidekiq.md b/doc/administration/sidekiq.md index 4aee88ed9cb..8f1119f6868 100644 --- a/doc/administration/sidekiq.md +++ b/doc/administration/sidekiq.md @@ -74,6 +74,20 @@ you want using steps 1 and 2 from the GitLab downloads page. postgresql['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 10.10.1.30/32 10.10.1.31/32 10.10.1.32/32 10.10.1.33/32 10.10.1.38/32) ``` +1. If you run multiple Sidekiq nodes with a shared file storage, such as NFS, you must + specify the UIDs and GIDs to ensure they match between servers. Specifying the UIDs + and GIDs prevents permissions issues in the file system. This advice is similar to the + [advice for Geo setups](geo/replication/multiple_servers.md#step-4-configure-the-frontend-application-nodes-on-the-geo-secondary-site): + + ```ruby + user['uid'] = 9000 + user['gid'] = 9000 + web_server['uid'] = 9001 + web_server['gid'] = 9001 + registry['uid'] = 9002 + registry['gid'] = 9002 + ``` + 1. Disable other services: ```ruby diff --git a/doc/administration/troubleshooting/debug.md b/doc/administration/troubleshooting/debug.md index e00243aca0a..744e12d4da1 100644 --- a/doc/administration/troubleshooting/debug.md +++ b/doc/administration/troubleshooting/debug.md @@ -258,7 +258,7 @@ separate Rails process to debug the issue: ### GitLab: API is not accessible This often occurs when GitLab Shell attempts to request authorization via the -[internal API](../../development/internal_api.md) (for example, `http://localhost:8080/api/v4/internal/allowed`), and +[internal API](../../development/internal_api/index.md) (for example, `http://localhost:8080/api/v4/internal/allowed`), and something in the check fails. There are many reasons why this may happen: 1. Timeout connecting to a database (for example, PostgreSQL or Redis) @@ -275,7 +275,7 @@ strace -ttTfyyy -s 1024 -p <PID of puma worker> -o /tmp/puma.txt If you cannot isolate which Unicorn worker is the issue, try to run `strace` on all the Unicorn workers to see where the -[`/internal/allowed`](../../development/internal_api.md) endpoint gets stuck: +[`/internal/allowed`](../../development/internal_api/index.md) endpoint gets stuck: ```shell ps auwx | grep puma | awk '{ print " -p " $2}' | xargs strace -ttTfyyy -s 1024 -o /tmp/puma.txt diff --git a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md index 109f451be5a..87f514a2fdd 100644 --- a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md +++ b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md @@ -965,6 +965,22 @@ license.save License.current # check to make sure it applied ``` +This is needed for example in a known edge-case with +[expired license and multiple LDAP servers](../auth/ldap/ldap-troubleshooting.md#expired-license-causes-errors-with-multiple-ldap-servers). + +### Remove licenses + +To clean up the [License History table](../../user/admin_area/license.md#license-history): + +```ruby +TYPE = :trial? +# or :expired? + +License.select(&TYPE).each(&:destroy!) + +# or even License.all.each(&:destroy!) +``` + ## Registry ### Registry Disk Space Usage by Project diff --git a/doc/administration/troubleshooting/kubernetes_cheat_sheet.md b/doc/administration/troubleshooting/kubernetes_cheat_sheet.md index abeba514e4b..64a6979c016 100644 --- a/doc/administration/troubleshooting/kubernetes_cheat_sheet.md +++ b/doc/administration/troubleshooting/kubernetes_cheat_sheet.md @@ -165,8 +165,16 @@ and they will assist you with any issues you are having. - How to connect to a GitLab PostgreSQL database: + In GitLab 14.2 (chart 5.2) and later: + + ```shell + kubectl exec -it <task-runner-pod-name> -- /srv/gitlab/bin/rails dbconsole --include-password --database main + ``` + + In GitLab 14.1 (chart 5.1) and earlier: + ```shell - kubectl exec -it <task-runner-pod-name> -- /srv/gitlab/bin/rails dbconsole -p + kubectl exec -it <task-runner-pod-name> -- /srv/gitlab/bin/rails dbconsole --include-password ``` - How to get information about Helm installation status: diff --git a/doc/administration/troubleshooting/postgresql.md b/doc/administration/troubleshooting/postgresql.md index 3df957bacf9..f8cb45dd00d 100644 --- a/doc/administration/troubleshooting/postgresql.md +++ b/doc/administration/troubleshooting/postgresql.md @@ -2,10 +2,9 @@ stage: Enablement group: Database info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference --- -# PostgreSQL +# PostgreSQL **(FREE SELF)** This page contains information about PostgreSQL the GitLab Support team uses when troubleshooting. GitLab makes this information public, so that anyone can |
