diff options
Diffstat (limited to 'doc')
591 files changed, 13548 insertions, 7175 deletions
diff --git a/doc/.vale/gitlab/CurrentStatus.yml b/doc/.vale/gitlab/CurrentStatus.yml index 16a6995843a..040fd3d8a8f 100644 --- a/doc/.vale/gitlab/CurrentStatus.yml +++ b/doc/.vale/gitlab/CurrentStatus.yml @@ -5,9 +5,10 @@ # # For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles extends: existence -message: 'Avoid words like "%s" that promise future changes, because documentation is about the current state of the product.' +message: 'Avoid words like "%s" when you write about future features. Our documentation is about the current state of the product.' level: suggestion ignorecase: true -link: https://docs.gitlab.com/ee/development/documentation/styleguide/index.html#usage-list +link: https://docs.gitlab.com/ee/development/documentation/styleguide/#promising-features-in-future-versions tokens: - currently + - yet diff --git a/doc/.vale/gitlab/HeadingContent.yml b/doc/.vale/gitlab/HeadingContent.yml new file mode 100644 index 00000000000..a8dc596f2a2 --- /dev/null +++ b/doc/.vale/gitlab/HeadingContent.yml @@ -0,0 +1,18 @@ +--- +# Error: gitlab.HeadingContent +# +# Checks for generic, unhelpful subheadings. +# +# For a list of all options, see https://errata-ai.gitbook.io/vale/getting-started/styles +extends: existence +message: 'Rename the subheading "%s", or re-purpose the content elsewhere.' +level: warning +scope: heading +link: https://docs.gitlab.com/ee/development/documentation/styleguide/#headings-1 +ignorecase: false +tokens: + - How it works + - Limitations + - Overview + - Use cases? + - Important notes? diff --git a/doc/.vale/gitlab/SubstitutionWarning.yml b/doc/.vale/gitlab/SubstitutionWarning.yml index 7ee32e41ade..84d1bab3cc6 100644 --- a/doc/.vale/gitlab/SubstitutionWarning.yml +++ b/doc/.vale/gitlab/SubstitutionWarning.yml @@ -14,11 +14,12 @@ swap: click: select code base: codebase config: configuration + deselect: clear + deselected: cleared distro: distribution file name: filename filesystem: file system info: information - need to: must repo: repository timezone: time zone utilize: use diff --git a/doc/.vale/gitlab/Substitutions.yml b/doc/.vale/gitlab/Substitutions.yml index 1cc5a60ac91..a0670d75f18 100644 --- a/doc/.vale/gitlab/Substitutions.yml +++ b/doc/.vale/gitlab/Substitutions.yml @@ -13,6 +13,7 @@ ignorecase: true swap: codequality: code quality Customer [Pp]ortal: Customers Portal + disallow: prevent frontmatter: front matter GitLabber: GitLab team member GitLabbers: GitLab team members diff --git a/doc/.vale/gitlab/Uppercase.yml b/doc/.vale/gitlab/Uppercase.yml index c9021dc862e..f1b06e10fe6 100644 --- a/doc/.vale/gitlab/Uppercase.yml +++ b/doc/.vale/gitlab/Uppercase.yml @@ -51,6 +51,7 @@ exceptions: - EKS - ELB - EOL + - EWM - EXIF - FAQ - FIDO @@ -144,6 +145,7 @@ exceptions: - RSA - RDS - RSS + - RTC - RVM - SAAS - SAML @@ -171,6 +173,7 @@ exceptions: - SSH - SSL - SSO + - STI - SVG - SVN - TCP diff --git a/doc/.vale/gitlab/Wordy.yml b/doc/.vale/gitlab/Wordy.yml index 7818ecbb74a..716fdc6c38d 100644 --- a/doc/.vale/gitlab/Wordy.yml +++ b/doc/.vale/gitlab/Wordy.yml @@ -5,9 +5,12 @@ # # For a list of all options, see https://docs.errata.ai/vale/styles extends: substitution -message: 'Be concise: "%s" is less wordy than "%s".' +message: '%s "%s".' link: https://docs.gitlab.com/ee/development/documentation/styleguide/word_list.html level: suggestion ignorecase: true swap: - in order to: to + in order to: "Be concise: use 'to' rather than" + needs? to: "Rewrite the sentence, or use 'must', instead of" + note that: "Be concise: rewrite the sentence to not use" + please: "Remove this word from the sentence: " diff --git a/doc/.vale/gitlab/spelling-exceptions.txt b/doc/.vale/gitlab/spelling-exceptions.txt index 98254c2259b..6dc0704963b 100644 --- a/doc/.vale/gitlab/spelling-exceptions.txt +++ b/doc/.vale/gitlab/spelling-exceptions.txt @@ -229,6 +229,7 @@ Gemfile Gemnasium Gemojione Getter +getters Getters gettext Git @@ -576,6 +577,7 @@ sharded sharding shfmt Shibboleth +Shimo Shopify Sidekiq Silverlight diff --git a/doc/administration/audit_event_streaming.md b/doc/administration/audit_event_streaming.md index 3bdc67e5a69..ca59dff7ef4 100644 --- a/doc/administration/audit_event_streaming.md +++ b/doc/administration/audit_event_streaming.md @@ -25,6 +25,24 @@ GitLab can stream a single event more than once to the same destination. Use the WARNING: Event streaming destinations will receive **all** audit event data, which could include sensitive information. Make sure you trust the destination endpoint. +### Add event streaming destination using GitLab UI + +Users with at least the Owner role of a group can add event streaming destinations for it: + +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Security & Compliance > Audit events** +1. On the main area, select **Streams** tab. + - When the destination list is empty, select **Add stream** activate edit mode and add a new destination. + - When the destination list is not empty, select **{plus}** under the **Streams** tab to activate edit mode. +1. Enter the endpoint you wish to add and select **Add**. + +Event streaming is enabled if: + +- No warning is shown. +- The added endpoint is displayed in the UI. + +### Add event streaming destination using the API + To enable event streaming, a group owner must add a new event streaming destination using the `externalAuditEventDestinationCreate` mutation in the GraphQL API. @@ -34,8 +52,8 @@ mutation { errors externalAuditEventDestination { destinationUrl - group { verificationToken + group { name } } @@ -50,7 +68,17 @@ Event streaming is enabled if: ## List currently enabled streaming destinations -Group owners can view a list of event streaming destinations at any time using the `externalAuditEventDesinations` query type. +### List currently enabled streaming destination using GitLab UI + +Users with at least the Owner role of a group can list event streaming destinations: + +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Security & Compliance > Audit events** +1. On the main area, select **Streams** tab. + +### List currently enabled streaming destinations using the API + +Group owners can view a list of event streaming destinations at any time using the `externalAuditEventDestinations` query type. ```graphql query { @@ -69,6 +97,45 @@ query { If the resulting list is empty, then audit event streaming is not enabled for that group. +## Delete currently enabled streaming destination + +Group Owners can delete event streaming destinations at any time using the `deleteAuditEventDestinations` mutation type. + +### Delete currently enabled streaming using GitLab UI + +Uses with at least the Owner role of a group can delete event streaming destination. + +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Security & Compliance > Audit events** +1. On the main area, select **Streams** tab. +1. Select **{remove}** at the right side of each item. + +The external streaming destination is delete when: + +- No warning is shown. +- The deleted endpoint is not displayed in the UI. + +### Delete currently enabled streaming using the API + +You can delete an event streaming destination by specifying an ID. Get the required ID by [listing the details](audit_event_streaming.md#list-currently-enabled-streaming-destinations-using-the-api) of event streaming destinations. + +```graphql + +mutation{ + externalAuditEventDestinationDestroy(input: { id: destination }) { + errors + } +} + +``` + +Destination is deleted if: + +- The returned `errors` object is empty. +- The API responds with `200 OK`. + +When the last destination is successfully deleted, event streaming is disabled for the group. + ## Verify event authenticity > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345424) in GitLab 14.8. @@ -78,3 +145,228 @@ token is generated when the event destination is created and cannot be changed. Each streamed event contains a random alphanumeric identifier for the `X-Gitlab-Event-Streaming-Token` HTTP header that can be verified against the destination's value when [listing streaming destinations](#list-currently-enabled-streaming-destinations). + +## Audit event streaming on Git operations + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/332747) in GitLab 14.9 [with a flag](../administration/feature_flags.md) named `audit_event_streaming_git_operations`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](feature_flags.md) named `audit_event_streaming_git_operations`. On GitLab.com, this feature is not available. + +Streaming audit events can be sent when signed-in users push or pull a project's remote Git repositories: + +- [Using SSH](../ssh/index.md). +- Using HTTP or HTTPS. +- Using the **Download** button (**{download}**) in GitLab UI. + +Audit events are not captured for users that are not signed in. For example, when downloading a public project. + +To configure streaming audit events for Git operations, see [Add a new event streaming destination](#add-a-new-event-streaming-destination). + +### Request headers + +Request headers are formatted as follows: + +```plaintext +POST /logs HTTP/1.1 +Host: <DESTINATION_HOST> +Content-Type: application/x-www-form-urlencoded +X-Gitlab-Event-Streaming-Token: <DESTINATION_TOKEN> +``` + +### Example responses for SSH events + +Fetch: + +```json +{ + "id": 1, + "author_id": 1, + "entity_id": 29, + "entity_type": "Project", + "details": { + "author_name": "Administrator", + "target_id": 29, + "target_type": "Project", + "target_details": "example-project", + "custom_message": { + "protocol": "ssh", + "action": "git-upload-pack" + }, + "ip_address": "127.0.0.1", + "entity_path": "example-group/example-project" + }, + "ip_address": "127.0.0.1", + "author_name": "Administrator", + "entity_path": "example-group/example-project", + "target_details": "example-project", + "created_at": "2022-02-23T06:21:05.283Z", + "target_type": "Project", + "target_id": 29 +} +``` + +Push: + +```json +{ + "id": 1, + "author_id": 1, + "entity_id": 29, + "entity_type": "Project", + "details": { + "author_name": "Administrator", + "target_id": 29, + "target_type": "Project", + "target_details": "example-project", + "custom_message": { + "protocol": "ssh", + "action": "git-receive-pack" + }, + "ip_address": "127.0.0.1", + "entity_path": "example-group/example-project" + }, + "ip_address": "127.0.0.1", + "author_name": "Administrator", + "entity_path": "example-group/example-project", + "target_details": "example-project", + "created_at": "2022-02-23T06:23:08.746Z", + "target_type": "Project", + "target_id": 29 +} +``` + +### Example responses for HTTP and HTTPS events + +Fetch: + +```json +{ + "id": 1, + "author_id": 1, + "entity_id": 29, + "entity_type": "Project", + "details": { + "author_name": "Administrator", + "target_id": 29, + "target_type": "Project", + "target_details": "example-project", + "custom_message": { + "protocol": "http", + "action": "git-upload-pack" + }, + "ip_address": "127.0.0.1", + "entity_path": "example-group/example-project" + }, + "ip_address": "127.0.0.1", + "author_name": "Administrator", + "entity_path": "example-group/example-project", + "target_details": "example-project", + "created_at": "2022-02-23T06:25:43.938Z", + "target_type": "Project", + "target_id": 29 +} +``` + +Push: + +```json +{ + "id": 1, + "author_id": 1, + "entity_id": 29, + "entity_type": "Project", + "details": { + "author_name": "Administrator", + "target_id": 29, + "target_type": "Project", + "target_details": "example-project", + "custom_message": { + "protocol": "http", + "action": "git-receive-pack" + }, + "ip_address": "127.0.0.1", + "entity_path": "example-group/example-project" + }, + "ip_address": "127.0.0.1", + "author_name": "Administrator", + "entity_path": "example-group/example-project", + "target_details": "example-project", + "created_at": "2022-02-23T06:26:29.294Z", + "target_type": "Project", + "target_id": 29 +} +``` + +### Example responses for events from GitLab UI download button + +Fetch: + +```json +{ + "id": 1, + "author_id": 99, + "entity_id": 29, + "entity_type": "Project", + "details": { + "custom_message": "Repository Download Started", + "author_name": "example_username", + "target_id": 29, + "target_type": "Project", + "target_details": "example-group/example-project", + "ip_address": "127.0.0.1", + "entity_path": "example-group/example-project" + }, + "ip_address": "127.0.0.1", + "author_name": "example_username", + "entity_path": "example-group/example-project", + "target_details": "example-group/example-project", + "created_at": "2022-02-23T06:27:17.873Z", + "target_type": "Project", + "target_id": 29 +} +``` + +## Audit event streaming on merge request approval actions + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/271162) in GitLab 14.9. + +Stream audit events that relate to merge approval actions performed within a project. + +### Request headers + +Request headers are formatted as follows: + +```plaintext +POST /logs HTTP/1.1 +Host: <DESTINATION_HOST> +Content-Type: application/x-www-form-urlencoded +X-Gitlab-Event-Streaming-Token: <DESTINATION_TOKEN> +``` + +### Example request body + +```json +{ + "id": 1, + "author_id": 1, + "entity_id": 6, + "entity_type": "Project", + "details": { + "author_name": "example_username", + "target_id": 20, + "target_type": "MergeRequest", + "target_details": "merge request title", + "custom_message": "Approved merge request", + "ip_address": "127.0.0.1", + "entity_path": "example-group/example-project" + }, + "ip_address": "127.0.0.1", + "author_name": "example_username", + "entity_path": "example-group/example-project", + "target_details": "merge request title", + "created_at": "2022-03-09T06:53:11.181Z", + "target_type": "MergeRequest", + "target_id": 20 +} +``` diff --git a/doc/administration/audit_events.md b/doc/administration/audit_events.md index d4902a18cac..955f7a6a830 100644 --- a/doc/administration/audit_events.md +++ b/doc/administration/audit_events.md @@ -44,17 +44,22 @@ There are two kinds of events logged: - Instance events scoped to the whole GitLab instance, used by your Compliance team to perform formal audits. +NOTE: +Some events are recorded and available only as [streaming audit events](audit_event_streaming.md). + ### Impersonation data > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/536) in GitLab 13.0. When a user is being [impersonated](../user/admin_area/index.md#user-impersonation), their actions are logged as audit events as usual, with two additional details: -1. Usual audit events include information about the impersonating administrator. These are visible in their respective Audit Event pages depending on their type (Group/Project/User). -1. Extra audit events are recorded for the start and stop of the administrator's impersonation session. These are visible in - the: +1. Usual audit events include information about the impersonating administrator. These audit events are visible in their + respective audit event pages depending on their type (group, project, or user). +1. Extra audit events are recorded for the start and stop of the administrator's impersonation session. These audit events + are visible in the: - Instance audit events. - - Group audit events for all groups the user belongs to (GitLab 14.8 and later). This is limited to 20 groups for performance reasons. + - Group audit events for all groups the user belongs to (GitLab 14.8 and later). For performance reasons, group audit + events are limited to the oldest 20 groups to which you belong.  @@ -107,6 +112,8 @@ From there, you can see the following actions: - Compliance framework created, updated, or deleted. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340649) in GitLab 14.5. - Event streaming destination created, updated, or deleted. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/344664) in GitLab 14.6. - Instance administrator started or stopped impersonation of a group member. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/300961) in GitLab 14.8. +- Group deploy token was successfully created, revoked, or deleted. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/353452) in GitLab 14.9. +- Failed attempt to create a group deploy token. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/353452) in GitLab 14.9. Group events can also be accessed via the [Group Audit Events API](../api/audit_events.md#group-audit-events) @@ -158,6 +165,8 @@ From there, you can see the following actions: - Allowing force push to protected branch changed ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/338873) in GitLab 14.3) - Code owner approval requirement on merge requests targeting protected branch changed ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/338873) in GitLab 14.3) - Users and groups allowed to merge and push to protected branch added or removed ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/338873) in GitLab 14.3) +- Project deploy token was successfully created, revoked or deleted ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/353451) in GitLab 14.9) +- Failed attempt to create a project deploy token ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/353451) in GitLab 14.9) Project events can also be accessed via the [Project Audit Events API](../api/audit_events.md#project-audit-events). diff --git a/doc/administration/auditor_users.md b/doc/administration/auditor_users.md index 12e222290e7..7c89f34ce61 100644 --- a/doc/administration/auditor_users.md +++ b/doc/administration/auditor_users.md @@ -1,6 +1,6 @@ --- -stage: Enablement -group: Distribution +stage: Manage +group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- diff --git a/doc/administration/auth/atlassian.md b/doc/administration/auth/atlassian.md index 7c6aee2c2da..6be53922a5a 100644 --- a/doc/administration/auth/atlassian.md +++ b/doc/administration/auth/atlassian.md @@ -13,14 +13,14 @@ To enable the Atlassian OmniAuth provider for passwordless authentication you mu 1. Go to <https://developer.atlassian.com/console/myapps/> and sign-in with the Atlassian account to administer the application. -1. Click **Create a new app**. -1. Choose an App Name, such as 'GitLab', and click **Create**. +1. Select **Create a new app**. +1. Choose an App Name, such as 'GitLab', and select **Create**. 1. Note the `Client ID` and `Secret` for the [GitLab configuration](#gitlab-configuration) steps. -1. On the left sidebar under **APIS AND FEATURES**, click **OAuth 2.0 (3LO)**. -1. Enter the GitLab callback URL using the format `https://gitlab.example.com/users/auth/atlassian_oauth2/callback` and click **Save changes**. -1. Click **+ Add** in the left sidebar under **APIS AND FEATURES**. -1. Click **Add** for **Jira platform REST API** and then **Configure**. -1. Click **Add** next to the following scopes: +1. On the left sidebar under **APIS AND FEATURES**, select **OAuth 2.0 (3LO)**. +1. Enter the GitLab callback URL using the format `https://gitlab.example.com/users/auth/atlassian_oauth2/callback` and select **Save changes**. +1. Select **+ Add** in the left sidebar under **APIS AND FEATURES**. +1. Select **Add** for **Jira platform REST API** and then **Configure**. +1. Select **Add** next to the following scopes: - **View Jira issue data** - **View user profiles** - **Create and manage issues** @@ -73,6 +73,6 @@ To enable the Atlassian OmniAuth provider for passwordless authentication you mu 1. Save the configuration file. 1. [Reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) or [restart GitLab](../restart_gitlab.md#installations-from-source) for the changes to take effect if you installed GitLab via Omnibus or from source respectively. -On the sign-in page there should now be an Atlassian icon below the regular sign in form. Click the icon to begin the authentication process. +On the sign-in page there should now be an Atlassian icon below the regular sign in form. Select the icon to begin the authentication process. If everything goes right, the user is signed in to GitLab using their Atlassian credentials. diff --git a/doc/administration/auth/ldap/ldap-troubleshooting.md b/doc/administration/auth/ldap/ldap-troubleshooting.md index 06fe579e101..b8391bec72f 100644 --- a/doc/administration/auth/ldap/ldap-troubleshooting.md +++ b/doc/administration/auth/ldap/ldap-troubleshooting.md @@ -89,14 +89,14 @@ established but GitLab doesn't show you LDAP users in the output, one of the following is most likely true: - The `bind_dn` user doesn't have enough permissions to traverse the user tree. -- The user(s) don't fall under the [configured `base`](index.md#configure-ldap). -- The [configured `user_filter`](index.md#set-up-ldap-user-filter) blocks access to the user(s). +- The users don't fall under the [configured `base`](index.md#configure-ldap). +- The [configured `user_filter`](index.md#set-up-ldap-user-filter) blocks access to the users. In this case, you con confirm which of the above is true using [ldapsearch](#ldapsearch) with the existing LDAP configuration in your `/etc/gitlab/gitlab.rb`. -#### User(s) cannot sign-in +#### Users cannot sign-in A user can have trouble signing in for any number of reasons. To get started, here are some questions to ask yourself: @@ -284,7 +284,7 @@ If you don't find a particular user's GitLab email in the output, then that user hasn't signed in with LDAP yet. Next, GitLab searches its `identities` table for the existing -link between this user and the configured LDAP provider(s): +link between this user and the configured LDAP providers: ```sql Identity Load (0.9ms) SELECT "identities".* FROM "identities" WHERE "identities"."user_id" = 20 AND (provider LIKE 'ldap%') LIMIT 1 @@ -334,7 +334,7 @@ Gitlab::Auth::Ldap::Person.find_by_uid('<uid>', adapter) ### Group memberships **(PREMIUM SELF)** -#### Membership(s) not granted +#### Memberships not granted Sometimes you may think a particular user should be added to a GitLab group via LDAP group sync, but for some reason it's not happening. You can check several @@ -348,7 +348,7 @@ things to debug the situation. 1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Overview > Users**. 1. Search for the user. - 1. Open the user by clicking their name. Do not click **Edit**. + 1. Open the user by selecting their name. Do not select **Edit**. 1. Select the **Identities** tab. There should be an LDAP identity with an LDAP DN as the 'Identifier'. If not, this user hasn't signed in with LDAP yet and must do so first. @@ -558,7 +558,7 @@ aren't blocked or unable to access their accounts. NOTE: The following script requires that any new accounts with the new -email address are removed first. This is because emails have to be unique in GitLab. +email address are removed first. Email addresses must be unique in GitLab. Go to the [rails console](#rails-console) and then run: diff --git a/doc/administration/cicd.md b/doc/administration/cicd.md index a7bd07d5d38..5e061320c5b 100644 --- a/doc/administration/cicd.md +++ b/doc/administration/cicd.md @@ -71,6 +71,27 @@ Plan.default.actual_limits.update!(ci_needs_size_limit: 100) To disable directed acyclic graphs (DAG), set the limit to `0`. +## Change maximum scheduled pipeline frequency + +[Scheduled pipelines](../ci/pipelines/schedules.md) can be configured with any [cron value](../topics/cron/index.md), +but they do not always run exactly when scheduled. An internal process, called the +_pipeline schedule worker_, queues all the scheduled pipelines, but does not +run continuously. The worker runs on its own schedule, and scheduled pipelines that +are ready to start are only queued the next time the worker runs. Scheduled pipelines +can't run more frequently than the worker. + +The default frequency of the pipeline schedule worker is `3-59/10 * * * *` (every ten minutes, +starting with `0:03`, `0:13`, `0:23`, and so on). The default frequency for GitLab.com +is listed in the [GitLab.com settings](../user/gitlab_com/index.md#gitlab-cicd). + +To change the frequency of the pipeline schedule worker: + +1. Edit the `gitlab_rails['pipeline_schedule_worker_cron']` value in your instance's `gitlab.rb` file. +1. [Reconfigure GitLab](restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. + +For example, to set the maximum frequency of pipelines to twice a day, set `pipeline_schedule_worker_cron` +to a cron value of `0 */12 * * *` (`00:00` and `12:00` every day). + <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/administration/clusters/kas.md b/doc/administration/clusters/kas.md index 192b636e246..e5c371b9d40 100644 --- a/doc/administration/clusters/kas.md +++ b/doc/administration/clusters/kas.md @@ -4,34 +4,33 @@ group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Install the GitLab Agent Server for Kubernetes (KAS) **(FREE SELF)** +# Install the GitLab agent server for Kubernetes (KAS) **(FREE SELF)** -> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3834) in GitLab 13.10, the GitLab Agent Server (KAS) became available on GitLab.com under `wss://kas.gitlab.com`. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3834) in GitLab 13.10, the GitLab agent server (KAS) became available on GitLab.com at `wss://kas.gitlab.com`. > - [Moved](https://gitlab.com/groups/gitlab-org/-/epics/6290) from GitLab Premium to GitLab Free in 14.5. -The GitLab Agent Server for Kubernetes is a GitLab backend service dedicated to -managing the [GitLab Agent for Kubernetes](../../user/clusters/agent/index.md). +The agent server is a component you install together with GitLab. It is required to +manage the [GitLab agent for Kubernetes](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent). -The KAS acronym refers to the former name, Kubernetes Agent Server. +The KAS acronym refers to the former name, `Kubernetes agent server`. -The KAS is already installed and available in GitLab.com under `wss://kas.gitlab.com`. -This document describes how to install a KAS for GitLab self-managed instances. +The agent server for Kubernetes is installed and available on GitLab.com at `wss://kas.gitlab.com`. +If you use self-managed GitLab, you must install an agent server or specify an external installation. ## Installation options -As a GitLab administrator of self-managed instances, you can install KAS according to your GitLab -installation method: +As a GitLab administrator, you can install the agent server: -- For [Omnibus installations](#install-kas-with-omnibus). -- For [GitLab Helm Chart installations](#install-kas-with-the-gitlab-helm-chart). +- For [Omnibus installations](#for-omnibus). +- For [GitLab Helm Chart installations](#for-gitlab-helm-chart). -You can also opt to use an [external KAS](#use-an-external-kas-installation). +Or, you can [use an external agent server](#use-an-external-installation). -### Install KAS with Omnibus +### For Omnibus For [Omnibus](https://docs.gitlab.com/omnibus/) package installations: -1. Edit `/etc/gitlab/gitlab.rb` to enable the Agent Server: +1. To enable the agent server, edit `/etc/gitlab/gitlab.rb`: ```ruby gitlab_kas['enable'] = true @@ -39,49 +38,49 @@ For [Omnibus](https://docs.gitlab.com/omnibus/) package installations: 1. [Reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure). -To configure any additional options related to your KAS, -refer to the **Enable GitLab KAS** section of the +For additional configuration options, see the **Enable GitLab KAS** section of the [`gitlab.rb.template`](https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/master/files/gitlab-config-template/gitlab.rb.template). -### Install KAS with the GitLab Helm Chart +### For GitLab Helm Chart -For GitLab [Helm Chart](https://docs.gitlab.com/charts/) -installations, you must set `global.kas.enabled` to `true`. -For example, in a shell with `helm` and `kubectl` -installed, run: +For GitLab [Helm Chart](https://docs.gitlab.com/charts/) installations: -```shell -helm repo add gitlab https://charts.gitlab.io/ -helm repo update -helm upgrade --install gitlab gitlab/gitlab \ - --timeout 600s \ - --set global.hosts.domain=<YOUR_DOMAIN> \ - --set global.hosts.externalIP=<YOUR_IP> \ - --set certmanager-issuer.email=<YOUR_EMAIL> \ - --set global.kas.enabled=true # <-- without this, KAS will not be installed -``` +1. Set `global.kas.enabled` to `true`. For example, in a shell with `helm` and `kubectl` + installed, run: -To configure KAS, use a `gitlab.kas` sub-section in your `values.yaml` file: + ```shell + helm repo add gitlab https://charts.gitlab.io/ + helm repo update + helm upgrade --install gitlab gitlab/gitlab \ + --timeout 600s \ + --set global.hosts.domain=<YOUR_DOMAIN> \ + --set global.hosts.externalIP=<YOUR_IP> \ + --set certmanager-issuer.email=<YOUR_EMAIL> \ + --set global.kas.enabled=true # <-- without this setting, the agent server will not be installed + ``` -```yaml -gitlab: - kas: - # put your KAS custom options here -``` +1. To configure the agent server, use a `gitlab.kas` sub-section in your `values.yaml` file: + + ```yaml + gitlab: + kas: + # put your custom options here + ``` For details, see [how to use the GitLab-KAS chart](https://docs.gitlab.com/charts/charts/gitlab/kas/). -### Use an external KAS installation +### Use an external installation > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/299850) in GitLab 13.10. -Besides installing KAS with GitLab, you can opt to configure GitLab to use an external KAS. +Instead of installing the agent server, you can configure GitLab to use an external agent server. -For GitLab instances installed through the GitLab Helm Chart, see [how to configure your external KAS](https://docs.gitlab.com/charts/charts/globals.html#external-kas). +If you used the GitLab Helm Chart to install GitLab, see +[how to configure your external agent server](https://docs.gitlab.com/charts/charts/globals.html#external-kas). -For GitLab instances installed through Omnibus packages: +If you used the Omnibus packages: -1. Edit `/etc/gitlab/gitlab.rb` adding the paths to your external KAS: +1. Edit `/etc/gitlab/gitlab.rb` and add the paths to your external agent server: ```ruby gitlab_kas['enable'] = false @@ -96,7 +95,7 @@ For GitLab instances installed through Omnibus packages: ## Troubleshooting -If you have issues while using the GitLab Agent Server for Kubernetes, view the +If you have issues while using the agent server for Kubernetes, view the service logs by running the following command: ```shell @@ -105,9 +104,9 @@ kubectl logs -f -l=app=kas -n <YOUR-GITLAB-NAMESPACE> In Omnibus GitLab, find the logs in `/var/log/gitlab/gitlab-kas/`. -You can also [troubleshoot issues with individual Agents](../../user/clusters/agent/troubleshooting.md). +You can also [troubleshoot issues with individual agents](../../user/clusters/agent/troubleshooting.md). -### KAS logs - GitOps: failed to get project information +### GitOps: failed to get project information If you get the following error message: @@ -115,11 +114,11 @@ If you get the following error message: {"level":"warn","time":"2020-10-30T08:37:26.123Z","msg":"GitOps: failed to get project info","agent_id":4,"project_id":"root/kas-manifest001","error":"error kind: 0; status: 404"} ``` -It means that the specified manifest project `root/kas-manifest001` -doesn't exist or the manifest project is private. To fix it, make sure the project path is correct -and its visibility is [set to public](../../public_access/public_access.md). +The project specified by the manifest (`root/kas-manifest001`) +doesn't exist or the project where the manifest is kept is private. To fix this issue, +ensure the project path is correct and that the project's visibility is [set to public](../../public_access/public_access.md). -### KAS logs - Configuration file not found +### Configuration file not found If you get the following error message: @@ -127,29 +126,29 @@ If you get the following error message: time="2020-10-29T04:44:14Z" level=warning msg="Config: failed to fetch" agent_id=2 error="configuration file not found: \".gitlab/agents/test-agent/config.yaml\ ``` -It means that the path to the configuration project is incorrect, -or the path to `config.yaml` inside the project is not valid. +The path is incorrect for either: + +- The repository where the agent was registered. +- The agent configuration file. -To fix this, ensure that the paths to the configuration repository and to the `config.yaml` file -are correct. +To fix this issue, ensure that the paths are correct. -### KAS logs - `dial tcp <GITLAB_INTERNAL_IP>:443: connect: connection refused` +### `dial tcp <GITLAB_INTERNAL_IP>:443: connect: connection refused` -If you are running a self-managed GitLab instance and: +If you are running self-managed GitLab and: - The instance isn't running behind an SSL-terminating proxy. - The instance doesn't have HTTPS configured on the GitLab instance itself. - The instance's hostname resolves locally to its internal IP address. -You may see the following error when the KAS tries to connect to the GitLab API: +When the agent server tries to connect to the GitLab API, the following error might occur: ```json {"level":"error","time":"2021-08-16T14:56:47.289Z","msg":"GetAgentInfo()","correlation_id":"01FD7QE35RXXXX8R47WZFBAXTN","grpc_service":"gitlab.agent.reverse_tunnel.rpc.ReverseTunnel","grpc_method":"Connect","error":"Get \"https://gitlab.example.com/api/v4/internal/kubernetes/agent_info\": dial tcp 172.17.0.4:443: connect: connection refused"} ``` -To fix this for [Omnibus](https://docs.gitlab.com/omnibus/) package installations, -set the following parameter in `/etc/gitlab/gitlab.rb` -(replacing `gitlab.example.com` with your GitLab instance's hostname): +To fix this issue for [Omnibus](https://docs.gitlab.com/omnibus/) package installations, +set the following parameter in `/etc/gitlab/gitlab.rb`. Replace `gitlab.example.com` with your GitLab instance's hostname: ```ruby gitlab_kas['gitlab_address'] = 'http://gitlab.example.com' diff --git a/doc/administration/database_load_balancing.md b/doc/administration/database_load_balancing.md deleted file mode 100644 index 92b8342f251..00000000000 --- a/doc/administration/database_load_balancing.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: 'postgresql/database_load_balancing.md' -remove_date: '2022-02-19' ---- - -This file was moved to [another location](postgresql/database_load_balancing.md). - -<!-- This redirect file can be deleted after <2022-02-19>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/docs_self_host.md b/doc/administration/docs_self_host.md index 3a4511e5aa4..8464c35c3bb 100644 --- a/doc/administration/docs_self_host.md +++ b/doc/administration/docs_self_host.md @@ -18,7 +18,7 @@ To host the GitLab product documentation, you can use: - Your own web server After you create a website by using one of these methods, you redirect the UI links -in the product to point to your website. +in the product to point to your website. NOTE: The website you create must be hosted under a subdirectory that matches diff --git a/doc/administration/encrypted_configuration.md b/doc/administration/encrypted_configuration.md index 9224def4a5a..baa6e967507 100644 --- a/doc/administration/encrypted_configuration.md +++ b/doc/administration/encrypted_configuration.md @@ -1,7 +1,7 @@ --- stage: Enablement group: Distribution -info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers" +info: "To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments" type: reference --- diff --git a/doc/administration/feature_flags.md b/doc/administration/feature_flags.md index 85a7304b5e8..aebd6eb1d4c 100644 --- a/doc/administration/feature_flags.md +++ b/doc/administration/feature_flags.md @@ -26,7 +26,7 @@ Features behind flags can be gradually rolled out, typically: 1. The feature becomes enabled by default. 1. The feature flag is removed. -These features can be enabled and disabled to allow or disallow users to use +These features can be enabled and disabled to allow or prevent users from using them. It can be done by GitLab administrators with access to GitLab Rails console. diff --git a/doc/administration/geo/disaster_recovery/index.md b/doc/administration/geo/disaster_recovery/index.md index f7367ef863b..1725c283396 100644 --- a/doc/administration/geo/disaster_recovery/index.md +++ b/doc/administration/geo/disaster_recovery/index.md @@ -10,7 +10,7 @@ Geo replicates your database, your Git repositories, and few other assets, but there are some [limitations](../index.md#limitations). WARNING: -Disaster recovery for multi-secondary configurations is in **Alpha**. +Disaster recovery for multi-secondary configurations is in [**Alpha**](../../../policy/alpha-beta-support.md#alpha-features). For the latest updates, check the [Disaster Recovery epic for complete maturity](https://gitlab.com/groups/gitlab-org/-/epics/3574). Multi-secondary configurations require the complete re-synchronization and re-configuration of all non-promoted secondaries and causes downtime. diff --git a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md index b207be47aa1..9b3e0ecb427 100644 --- a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md +++ b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_multi_node.md @@ -6,7 +6,7 @@ type: howto --- WARNING: -This runbook is in **alpha**. For complete, production-ready documentation, see the +This runbook is in [**Alpha**](../../../../policy/alpha-beta-support.md#alpha-features). For complete, production-ready documentation, see the [disaster recovery documentation](../index.md). # Disaster Recovery (Geo) promotion runbooks **(PREMIUM SELF)** diff --git a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md index 5a6f9eb8be7..fa30b11fe32 100644 --- a/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md +++ b/doc/administration/geo/disaster_recovery/runbooks/planned_failover_single_node.md @@ -6,7 +6,7 @@ type: howto --- WARNING: -This runbook is in **alpha**. For complete, production-ready documentation, see the +This runbook is in [**Alpha**](../../../../policy/alpha-beta-support.md#alpha-features). For complete, production-ready documentation, see the [disaster recovery documentation](../index.md). # Disaster Recovery (Geo) promotion runbooks **(PREMIUM SELF)** diff --git a/doc/administration/geo/index.md b/doc/administration/geo/index.md index c4164284e97..1b80e91c9c4 100644 --- a/doc/administration/geo/index.md +++ b/doc/administration/geo/index.md @@ -23,7 +23,7 @@ to clone and fetch large repositories, speeding up development. For a video introduction to Geo, see [Introduction to GitLab Geo - GitLab Features](https://www.youtube.com/watch?v=-HDLxSjEh6w). -To make sure you're using the right version of the documentation, navigate to [the Geo page on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/administration/geo/index.md) and choose the appropriate release from the **Switch branch/tag** dropdown. For example, [`v13.7.6-ee`](https://gitlab.com/gitlab-org/gitlab/-/blob/v13.7.6-ee/doc/administration/geo/index.md). +To make sure you're using the right version of the documentation, go to [the Geo page on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/administration/geo/index.md) and choose the appropriate release from the **Switch branch/tag** dropdown list. For example, [`v13.7.6-ee`](https://gitlab.com/gitlab-org/gitlab/-/blob/v13.7.6-ee/doc/administration/geo/index.md). Geo uses a set of defined terms that are described in the [Geo Glossary](glossary.md). Be sure to familiarize yourself with those terms. @@ -148,14 +148,14 @@ NOTE: When using HTTP or HTTPS proxying, your load balancer must be configured to pass through the `Connection` and `Upgrade` hop-by-hop headers. See the [web terminal](../integration/terminal.md) integration guide for more details. NOTE: -When using HTTPS protocol for port 443, you need to add an SSL certificate to the load balancers. +When using HTTPS protocol for port 443, you must add an SSL certificate to the load balancers. If you wish to terminate SSL at the GitLab application server instead, use TCP protocol. #### Internal URL HTTP requests from any Geo secondary site to the primary Geo site use the Internal URL of the primary Geo site. If this is not explicitly defined in the primary Geo site settings in the Admin Area, the -public URL of the primary site will be used. +public URL of the primary site is used. To update the internal URL of the primary Geo site: @@ -187,7 +187,7 @@ Because the replicated database instance is read-only, we need this additional d This daemon: - Reads a log of events replicated by the **primary** site to the **secondary** database instance. -- Updates the Geo Tracking Database instance with changes that need to be executed. +- Updates the Geo Tracking Database instance with changes that must be executed. When something is marked to be updated in the tracking database instance, asynchronous jobs running on the **secondary** site execute the required operations and update the state. @@ -198,9 +198,9 @@ This new architecture allows GitLab to be resilient to connectivity issues betwe WARNING: This list of limitations only reflects the latest version of GitLab. If you are using an older version, extra limitations may be in place. -- Pushing directly to a **secondary** site redirects (for HTTP) or proxies (for SSH) the request to the **primary** site instead of [handling it directly](https://gitlab.com/gitlab-org/gitlab/-/issues/1381), except when using Git over HTTP with credentials embedded within the URI. For example, `https://user:password@secondary.tld`. +- Pushing directly to a **secondary** site redirects (for HTTP) or proxies (for SSH) the request to the **primary** site instead of [handling it directly](https://gitlab.com/gitlab-org/gitlab/-/issues/1381), except when using Git over HTTP with credentials embedded in the URI. For example, `https://user:password@secondary.tld`. - The **primary** site has to be online for OAuth login to happen. Existing sessions and Git are not affected. Support for the **secondary** site to use an OAuth provider independent from the primary is [being planned](https://gitlab.com/gitlab-org/gitlab/-/issues/208465). -- The installation takes multiple manual steps that together can take about an hour depending on circumstances. We are working on improving this experience. See [Omnibus GitLab issue #2978](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/2978) for details. +- The installation takes multiple manual steps that together can take about an hour depending on circumstances. Consider using [the GitLab Environment Toolkit](https://gitlab.com/gitlab-org/gitlab-environment-toolkit) to deploy and operate production GitLab instances based on our [Reference Architectures](../reference_architectures/index.md), including automation of common daily tasks. We are planning to [improve Geo's installation even further](https://gitlab.com/groups/gitlab-org/-/epics/1465). - Real-time updates of issues/merge requests (for example, via long polling) doesn't work on the **secondary** site. - GitLab Runners cannot register with a **secondary** site. Support for this is [planned for the future](https://gitlab.com/gitlab-org/gitlab/-/issues/3294). - [Selective synchronization](replication/configuration.md#selective-synchronization) only limits what repositories and files are replicated. The entire PostgreSQL data is still replicated. Selective synchronization is not built to accommodate compliance / export control use cases. @@ -210,13 +210,25 @@ This list of limitations only reflects the latest version of GitLab. If you are There is a complete list of all GitLab [data types](replication/datatypes.md) and [existing support for replication and verification](replication/datatypes.md#limitations-on-replicationverification). +### View replication data on the primary site + +If you try to view replication data on the primary site, you receive a warning that this may be inconsistent: + +> Viewing projects and designs data from a primary site is not possible when using a unified URL. Visit the secondary site directly. + +The only way to view projects replication data for a particular secondary site is to visit that secondary site directly. For example, `https://<IP of your secondary site>/admin/geo/replication/projects`. +An [epic exists](https://gitlab.com/groups/gitlab-org/-/epics/4623) to fix this limitation. + +The only way to view designs replication data for a particular secondary site is to visit that secondary site directly. For example, `https://<IP of your secondary site>/admin/geo/replication/designs`. +An [epic exists](https://gitlab.com/groups/gitlab-org/-/epics/4624) to fix this limitation. + ## Setup instructions For setup instructions, see [Setting up Geo](setup/index.md). ## Post-installation documentation -After installing GitLab on the **secondary** site(s) and performing the initial configuration, see the following documentation for post-installation information. +After installing GitLab on the **secondary** sites and performing the initial configuration, see the following documentation for post-installation information. ### Configuring Geo @@ -224,7 +236,7 @@ For information on configuring Geo, see [Geo configuration](replication/configur ### Updating Geo -For information on how to update your Geo site(s) to the latest GitLab version, see [Updating the Geo sites](replication/updating_the_geo_sites.md). +For information on how to update your Geo sites to the latest GitLab version, see [Updating the Geo sites](replication/updating_the_geo_sites.md). ### Pausing and resuming replication @@ -237,8 +249,8 @@ secondary. If the site is paused, be sure to resume before promoting. This issue has been fixed in GitLab 13.4 and later. WARNING: -Pausing and resuming of replication is currently only supported for Geo installations using an -Omnibus GitLab-managed database. External databases are currently not supported. +Pausing and resuming of replication is only supported for Geo installations using an +Omnibus GitLab-managed database. External databases are not supported. In some circumstances, like during [upgrades](replication/updating_the_geo_sites.md) or a [planned failover](disaster_recovery/planned_failover.md), it is desirable to pause replication between the primary and secondary. diff --git a/doc/administration/geo/replication/object_storage.md b/doc/administration/geo/replication/object_storage.md index 3a10d3bad58..06f25fa15af 100644 --- a/doc/administration/geo/replication/object_storage.md +++ b/doc/administration/geo/replication/object_storage.md @@ -35,7 +35,7 @@ To have: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10586) in GitLab 12.4. WARNING: -This is a [**beta** feature](https://about.gitlab.com/handbook/product/#beta) and is not ready yet for production use at any scale. The main limitations are a lack of testing at scale and no verification of any replicated data. +This is a [**Beta** feature](../../../policy/alpha-beta-support.md#beta-features) and is not ready yet for production use at any scale. The main limitations are a lack of testing at scale and no verification of any replicated data. **Secondary** sites can replicate files stored on the **primary** site regardless of whether they are stored on the local file system or in object storage. diff --git a/doc/administration/geo/replication/troubleshooting.md b/doc/administration/geo/replication/troubleshooting.md index 1371e5d84c8..8f55ce99787 100644 --- a/doc/administration/geo/replication/troubleshooting.md +++ b/doc/administration/geo/replication/troubleshooting.md @@ -268,7 +268,7 @@ sudo gitlab-rake gitlab:geo:check GitLab Geo is available ... no Try fixing it: - Upload a new license that includes the GitLab Geo feature + Add a new license that includes the GitLab Geo feature For more information see: https://about.gitlab.com/features/gitlab-geo/ GitLab Geo is enabled ... Exception: PG::UndefinedTable: ERROR: relation "geo_nodes" does not exist @@ -1043,6 +1043,25 @@ To resolve this issue: the **primary** node using IPv4 in the `/etc/hosts` file. Alternatively, you should [enable IPv6 on the **primary** node](https://docs.gitlab.com/omnibus/settings/nginx.html#setting-the-nginx-listen-address-or-addresses). +### Secondary site returns 502 errors with Geo proxying + +When [Geo proxying for secondary sites](../secondary_proxy/index.md) is enabled, and the secondary site user interface returns +502 errors, it is possible that the response header proxied from the primary site is too large. + +Check the NGINX logs for errors similar to this example: + +```plaintext +2022/01/26 00:02:13 [error] 26641#0: *829148 upstream sent too big header while reading response header from upstream, client: 1.2.3.4, server: geo.staging.gitlab.com, request: "POST /users/sign_in HTTP/2.0", upstream: "http://unix:/var/opt/gitlab/gitlab-workhorse/sockets/socket:/users/sign_in", host: "geo.staging.gitlab.com", referrer: "https://geo.staging.gitlab.com/users/sign_in" +``` + +To resolve this issue: + +1. Set `nginx['proxy_custom_buffer_size'] = '8k'` in `/etc/gitlab.rb` on all web nodes on the secondary site. +1. Reconfigure the **secondary** using `sudo gitlab-ctl reconfigure`. + +If you still get this error, you can further increase the buffer size by repeating the steps above +and changing the `8k` size, for example by doubling it to `16k`. + ### Geo Admin Area shows 'Unknown' for health status and 'Request failed with status code 401' If using a load balancer, ensure that the load balancer's URL is set as the `external_url` in the diff --git a/doc/administration/geo/replication/using_a_geo_server.md b/doc/administration/geo/replication/using_a_geo_server.md deleted file mode 100644 index 62562a1149d..00000000000 --- a/doc/administration/geo/replication/using_a_geo_server.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: '../../geo/replication/usage.md' -remove_date: '2022-02-01' ---- - -This document was moved to [another location](../../geo/replication/usage.md). - -<!-- This redirect file can be deleted after 2022-02-01 --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/geo/secondary_proxy/index.md b/doc/administration/geo/secondary_proxy/index.md index c8dcbb81312..768adab9101 100644 --- a/doc/administration/geo/secondary_proxy/index.md +++ b/doc/administration/geo/secondary_proxy/index.md @@ -140,6 +140,8 @@ for details. To use TLS certificates with Let's Encrypt, you can manually point the domain to one of the Geo sites, generate the certificate, then copy it to all other sites. +- [Viewing projects and designs data from a primary site is not possible when using a unified URL](../index.md#view-replication-data-on-the-primary-site). + ## Behavior of secondary sites when the primary Geo site is down Considering that web traffic is proxied to the primary, the behavior of the secondary sites differs when the primary diff --git a/doc/administration/geo/setup/database.md b/doc/administration/geo/setup/database.md index a8bebef8f44..9c917be123e 100644 --- a/doc/administration/geo/setup/database.md +++ b/doc/administration/geo/setup/database.md @@ -192,7 +192,7 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o for more details. NOTE: - If you need to use `0.0.0.0` or `*` as the listen_address, you also need to add + If you need to use `0.0.0.0` or `*` as the listen_address, you also must add `127.0.0.1/32` to the `postgresql['md5_auth_cidr_addresses']` setting, to allow Rails to connect through `127.0.0.1`. For more information, see [omnibus-5258](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/5258). @@ -378,7 +378,7 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o 1. Configure PostgreSQL: This step is similar to how we configured the **primary** instance. - We need to enable this, even if using a single node. + We must enable this, even if using a single node. Edit `/etc/gitlab/gitlab.rb` and add the following, replacing the IP addresses with addresses appropriate to your network configuration: @@ -407,7 +407,7 @@ There is an [issue where support is being discussed](https://gitlab.com/gitlab-o ``` For external PostgreSQL instances, see [additional instructions](external_database.md). - If you bring a former **primary** node back online to serve as a **secondary** node, then you also need to remove `roles(['geo_primary_role'])` or `geo_primary_role['enable'] = true`. + If you bring a former **primary** node back online to serve as a **secondary** node, then you also must remove `roles(['geo_primary_role'])` or `geo_primary_role['enable'] = true`. 1. Reconfigure GitLab for the changes to take effect: @@ -472,7 +472,7 @@ data before running `pg_basebackup`. initial replication to take under an hour. - Pass `--sslmode=disable` to skip PostgreSQL TLS authentication altogether (for example, you know the network path is secure, or you are using a site-to-site - VPN). This is **not** safe over the public Internet! + VPN). It is **not** safe over the public Internet! - You can read more details about each `sslmode` in the [PostgreSQL documentation](https://www.postgresql.org/docs/12/libpq-ssl.html#LIBPQ-SSL-PROTECTION); the instructions above are carefully written to ensure protection against @@ -480,7 +480,7 @@ data before running `pg_basebackup`. - Change the `--slot-name` to the name of the replication slot to be used on the **primary** database. The script attempts to create the replication slot automatically if it does not exist. - - If you're repurposing an old server into a Geo **secondary** node, you need to + - If you're repurposing an old server into a Geo **secondary** node, you must add `--force` to the command line. - When not in a production machine you can disable backup step if you really sure this is what you want by adding `--skip-backup` @@ -547,7 +547,7 @@ FATAL: could not connect to the primary server: FATAL: password authentication On all GitLab Geo **secondary** servers: -1. The first step isn't necessary from a configuration perspective, since the hashed `'sql_replication_password'` +1. The first step isn't necessary from a configuration perspective, because the hashed `'sql_replication_password'` is not used on the GitLab Geo **secondary**. However in the event that **secondary** needs to be promoted to the GitLab Geo **primary**, make sure to match the `'sql_replication_password'` in the secondary server configuration. @@ -612,7 +612,7 @@ and other database best practices. ##### Step 1. Configure Patroni permanent replication slot on the primary site -To set up database replication with Patroni on a secondary node, we need to +To set up database replication with Patroni on a secondary node, we must configure a _permanent replication slot_ on the primary node's Patroni cluster, and ensure password authentication is used. @@ -678,7 +678,7 @@ Leader instance**: ##### Step 2. Configure the internal load balancer on the primary site To avoid reconfiguring the Standby Leader on the secondary site whenever a new -Leader is elected on the primary site, we need to set up a TCP internal load +Leader is elected on the primary site, we must set up a TCP internal load balancer which gives a single endpoint for connecting to the Patroni cluster's Leader. @@ -860,7 +860,7 @@ For each Patroni instance on the secondary site: ### Migrating from repmgr to Patroni -1. Before migrating, it is recommended that there is no replication lag between the primary and secondary sites and that replication is paused. In GitLab 13.2 and later, you can pause and resume replication with `gitlab-ctl geo-replication-pause` and `gitlab-ctl geo-replication-resume` on a Geo secondary database node. +1. Before migrating, we recommend that there is no replication lag between the primary and secondary sites and that replication is paused. In GitLab 13.2 and later, you can pause and resume replication with `gitlab-ctl geo-replication-pause` and `gitlab-ctl geo-replication-resume` on a Geo secondary database node. 1. Follow the [instructions to migrate repmgr to Patroni](../../postgresql/replication_and_failover.md#switching-from-repmgr-to-patroni). When configuring Patroni on each primary site database node, add `patroni['replication_slots'] = { '<slot_name>' => 'physical' }` to `gitlab.rb` where `<slot_name>` is the name of the replication slot for your Geo secondary. This ensures that Patroni recognizes the replication slot as permanent and not drop it upon restarting. 1. If database replication to the secondary was paused before migration, resume replication once Patroni is confirmed working on the primary. @@ -869,7 +869,7 @@ to `gitlab.rb` where `<slot_name>` is the name of the replication slot for your Before the introduction of Patroni, Geo had no Omnibus support for HA setups on the secondary node. -With Patroni it's now possible to support that. In order to migrate the existing PostgreSQL to Patroni: +With Patroni it's now possible to support that. To migrate the existing PostgreSQL to Patroni: 1. Make sure you have a Consul cluster setup on the secondary (similar to how you set it up on the primary). 1. [Configure a permanent replication slot](#step-1-configure-patroni-permanent-replication-slot-on-the-primary-site). @@ -894,7 +894,7 @@ A production-ready and secure setup requires at least three Consul nodes, two Patroni nodes and one PgBouncer node on the secondary site. Because of [omnibus-6587](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6587), Consul can't track multiple -services, so these need to be different than the nodes used for the Standby Cluster database. +services, so these must be different than the nodes used for the Standby Cluster database. Be sure to use [password credentials](../../postgresql/replication_and_failover.md#database-authorization-for-patroni) and other database best practices. @@ -1037,7 +1037,7 @@ For each node running the `gitlab-rails`, `sidekiq`, and `geo-logcursor` service sudo -i ``` -1. Edit `/etc/gitlab/gitlab.rb` and add the following attributes. You may have other attributes set, but the following need to be set. +1. Edit `/etc/gitlab/gitlab.rb` and add the following attributes. You may have other attributes set, but the following must be set. ```ruby # Tracking database settings diff --git a/doc/administration/geo/setup/index.md b/doc/administration/geo/setup/index.md index 7d365f73101..04a341aa822 100644 --- a/doc/administration/geo/setup/index.md +++ b/doc/administration/geo/setup/index.md @@ -10,7 +10,7 @@ type: howto These instructions assume you have a working instance of GitLab. They guide you through: 1. Making your existing instance the **primary** site. -1. Adding **secondary** site(s). +1. Adding **secondary** sites. WARNING: The steps below should be followed in the order they appear. **Make sure the GitLab version is the same on all sites.** @@ -19,15 +19,15 @@ The steps below should be followed in the order they appear. **Make sure the Git If you installed GitLab using the Omnibus packages (highly recommended): -1. [Install GitLab Enterprise Edition](https://about.gitlab.com/install/) on the node(s) that will serve as the **secondary** site. Do not create an account or log in to the new **secondary** site. -1. [Upload the GitLab License](../../../user/admin_area/license.md) on the **primary** site to unlock Geo. The license must be for [GitLab Premium](https://about.gitlab.com/pricing/) or higher. +1. [Install GitLab Enterprise Edition](https://about.gitlab.com/install/) on the nodes that will serve as the **secondary** site. Do not create an account or log in to the new **secondary** site. +1. [Add the GitLab License](../../../user/admin_area/license.md) on the **primary** site to unlock Geo. The license must be for [GitLab Premium](https://about.gitlab.com/pricing/) or higher. 1. [Set up the database replication](database.md) (`primary (read-write) <-> secondary (read-only)` topology). -1. [Configure fast lookup of authorized SSH keys in the database](../../operations/fast_ssh_key_lookup.md). This step is required and needs to be done on **both** the **primary** and **secondary** site(s). -1. [Configure GitLab](../replication/configuration.md) to set the **primary** and **secondary** site(s). -1. Optional: [Configure a secondary LDAP server](../../auth/ldap/index.md) for the **secondary** site(s). See [notes on LDAP](../index.md#ldap). +1. [Configure fast lookup of authorized SSH keys in the database](../../operations/fast_ssh_key_lookup.md). This step is required and needs to be done on **both** the **primary** and **secondary** sites. +1. [Configure GitLab](../replication/configuration.md) to set the **primary** and **secondary** sites. +1. Optional: [Configure a secondary LDAP server](../../auth/ldap/index.md) for the **secondary** sites. See [notes on LDAP](../index.md#ldap). 1. Follow the [Using a Geo Site](../replication/usage.md) guide. 1. [Configure Geo secondary proxying](../secondary_proxy/index.md) to use a single, unified URL for all Geo sites. This step is recommended to accelerate most read requests while transparently proxying writes to the primary Geo site. ## Post-installation documentation -After installing GitLab on the **secondary** site(s) and performing the initial configuration, see the [following documentation for post-installation information](../index.md#post-installation-documentation). +After installing GitLab on the **secondary** sites and performing the initial configuration, see the [following documentation for post-installation information](../index.md#post-installation-documentation). diff --git a/doc/administration/get_started.md b/doc/administration/get_started.md index f3b4400c8c3..e6a2e833af3 100644 --- a/doc/administration/get_started.md +++ b/doc/administration/get_started.md @@ -39,8 +39,8 @@ Get started: - Create a [project](../user/project/working_with_projects.md#create-a-project). - Create a [group](../user/group/index.md#create-a-group). - [Add members](../user/group/index.md#add-users-to-a-group) to the group. -- Create a [subgroup](../user/group/subgroups/index.md#creating-a-subgroup). -- [Add members](../user/group/subgroups/index.md#membership) to the subgroup. +- Create a [subgroup](../user/group/subgroups/index.md#create-a-subgroup). +- [Add members](../user/group/subgroups/index.md#subgroup-membership) to the subgroup. - Enable [external authorization control](../user/admin_area/settings/external_authorization.md#configuration). **More resources** @@ -49,7 +49,7 @@ Get started: - Sync group memberships [by using LDAP](../administration/auth/ldap/ldap_synchronization.md#group-sync). - Manage user access with inherited permissions. Use up to 20 levels of subgroups to organize both teams and projects. - Learn more about [inherited permissions](../user/project/members/index.md#inherited-membership). - - View [nested category examples](../user/group/subgroups/index.md#overview). + - View an [example](../user/group/subgroups/index.md). ## Import projects diff --git a/doc/administration/gitaly/index.md b/doc/administration/gitaly/index.md index ea24e901943..f88a1aa2b4d 100644 --- a/doc/administration/gitaly/index.md +++ b/doc/administration/gitaly/index.md @@ -13,7 +13,7 @@ It is used by GitLab to read and write Git data. Gitaly is present in every GitLab installation and coordinates Git repository storage and retrieval. Gitaly can be: -- A simple background service operating on a single instance Omnibus GitLab (all of +- A background service operating on a single instance Omnibus GitLab (all of GitLab on one machine). - Separated onto its own instance and configured in a full cluster configuration, depending on scaling and availability requirements. @@ -71,7 +71,7 @@ the current status of these issues, please refer to the referenced issues and ep | Gitaly Cluster + Geo - Issues retrying failed syncs | If Gitaly Cluster is used on a Geo secondary site, repositories that have failed to sync could continue to fail when Geo tries to resync them. Recovering from this state requires assistance from support to run manual steps. Work is in-progress to update Gitaly Cluster to [identify repositories with a unique and persistent identifier](https://gitlab.com/gitlab-org/gitaly/-/issues/3485), which is expected to resolve the issue. | No known solution at this time. | | Database inconsistencies due to repository access outside of Gitaly Cluster's control | Operations that write to the repository storage that do not go through normal Gitaly Cluster methods can cause database inconsistencies. These can include (but are not limited to) snapshot restoration for cluster node disks, node upgrades which modify files under Git control, or any other disk operation that may touch repository storage external to GitLab. The Gitaly team is actively working to provide manual commands to [reconcile the Praefect database with the repository storage](https://gitlab.com/groups/gitlab-org/-/epics/6723). | Don't directly change repositories on any Gitaly Cluster node at this time. | | Praefect unable to insert data into the database due to migrations not being applied after an upgrade | If the database is not kept up to date with completed migrations, then the Praefect node is unable to perform normal operation. | Make sure the Praefect database is up and running with all migrations completed (For example: `/opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml sql-migrate-status` should show a list of all applied migrations). Consider [requesting live upgrade assistance](https://about.gitlab.com/support/scheduling-live-upgrade-assistance.html) so your upgrade plan can be reviewed by support. | -| Restoring a Gitaly Cluster node from a snapshot in a running cluster | Because the Gitaly Cluster runs with consistent state, introducing a single node that is behind will result in the cluster not being able to reconcile the nodes data and other nodes data | Don't restore a single Gitaly Cluster node from a backup snapshot. If you need to restore from backup, it's best to snapshot all Gitaly Cluster nodes at the same time and take a database dump of the Praefect database. | +| Restoring a Gitaly Cluster node from a snapshot in a running cluster | Because the Gitaly Cluster runs with consistent state, introducing a single node that is behind will result in the cluster not being able to reconcile the nodes data and other nodes data | Don't restore a single Gitaly Cluster node from a backup snapshot. If you must restore from backup, it's best to snapshot all Gitaly Cluster nodes at the same time and take a database dump of the Praefect database. | ### Snapshot backup and recovery limitations @@ -179,7 +179,7 @@ In this example: - Repositories are stored on a virtual storage called `storage-1`. - Three Gitaly nodes provide `storage-1` access: `gitaly-1`, `gitaly-2`, and `gitaly-3`. - The three Gitaly nodes share data in three separate hashed storage locations. -- The [replication factor](#replication-factor) is `3`. There are three copies maintained +- The [replication factor](#replication-factor) is `3`. Three copies are maintained of each repository. The availability objectives for Gitaly clusters assuming a single node failure are: @@ -237,7 +237,7 @@ Engineering support for NFS for Git repositories is deprecated. Technical suppor is not well suited to Git workloads which are CPU and IOPS sensitive. Specifically: -- Git is sensitive to file system latency. Even simple operations require many +- Git is sensitive to file system latency. Some operations require many read operations. Operations that are fast on block storage can become an order of magnitude slower. This significantly impacts GitLab application performance. - NFS performance optimizations that prevent the performance gap between @@ -386,8 +386,7 @@ them back to direct Gitaly storage: 1. Create and configure a new [Gitaly server](configure_gitaly.md#run-gitaly-on-its-own-server). 1. [Move the repositories](../operations/moving_repositories.md#move-repositories) - to the newly created storage. There are different possibilities to move them - by shard or by group, this gives you the opportunity to spread them over + to the newly created storage. You can move them by shard or by group, which gives you the opportunity to spread them over multiple Gitaly servers. ## Monitor Gitaly and Gitaly Cluster @@ -509,7 +508,7 @@ The following metrics are available from the `/metrics` endpoint: They reflect configuration defined for this instance of Praefect. - `gitaly_praefect_replication_latency_bucket`, a histogram measuring the amount of time it takes - for replication to complete once the replication job starts. Available in GitLab 12.10 and later. + for replication to complete after the replication job starts. Available in GitLab 12.10 and later. - `gitaly_praefect_replication_delay_bucket`, a histogram measuring how much time passes between when the replication job is created and when it starts. Available in GitLab 12.10 and later. - `gitaly_praefect_node_latency_bucket`, a histogram measuring the latency in Gitaly returning @@ -542,7 +541,7 @@ You can also monitor the [Praefect logs](../logs.md#praefect-logs). The following metrics are available from the `/db_metrics` endpoint: - `gitaly_praefect_unavailable_repositories`, the number of repositories that have no healthy, up to date replicas. -- `gitaly_praefect_read_only_repositories`, the number of repositories in read-only mode within a virtual storage. +- `gitaly_praefect_read_only_repositories`, the number of repositories in read-only mode in a virtual storage. This metric is available for backwards compatibility reasons. `gitaly_praefect_unavailable_repositories` is more accurate. - `gitaly_praefect_replication_queue_depth`, the number of jobs in the replication queue. diff --git a/doc/administration/gitaly/praefect.md b/doc/administration/gitaly/praefect.md index 8f17835b8a3..d7a06994b6c 100644 --- a/doc/administration/gitaly/praefect.md +++ b/doc/administration/gitaly/praefect.md @@ -616,7 +616,7 @@ Note the following: - IP address, you must add it as a Subject Alternative Name to the certificate. - When running Praefect sub-commands such as `dial-nodes` and `list-untracked-repositories` from the command line with [Gitaly TLS enabled](configure_gitaly.md#enable-tls-support), you must set the `SSL_CERT_DIR` or `SSL_CERT_FILE` - environment variable so that the Gitaly certificate is trusted. For example: + environment variable so that the Gitaly certificate is trusted. For example: ```shell sudo SSL_CERT_DIR=/etc/gitlab/trusted_certs /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml dial-nodes @@ -1196,6 +1196,9 @@ You can configure: current assignments: gitaly-1, gitaly-2 ``` +If `default_replication_factor` is unset, the repositories are always replicated on every node defined in `virtual_storages`. If a new +node is introduced to the virtual storage, both new and existing repositories are replicated to the node automatically. + ## Automatic failover and primary election strategies Praefect regularly checks the health of each Gitaly node. This is used to automatically fail over diff --git a/doc/administration/gitaly/recovery.md b/doc/administration/gitaly/recovery.md index 9210c8f08d3..a7166f7e62e 100644 --- a/doc/administration/gitaly/recovery.md +++ b/doc/administration/gitaly/recovery.md @@ -76,7 +76,7 @@ The following parameters are available: some assigned copies that are not available. NOTE: -`dataloss` is still in beta and the output format is subject to change. +`dataloss` is still in [Beta](../../policy/alpha-beta-support.md#beta-features) and the output format is subject to change. To check for repositories with outdated primaries or for unavailable repositories, run: @@ -377,7 +377,7 @@ sudo /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.t The `track-repository` Praefect sub-command adds repositories on disk to the Praefect database to be tracked. ```shell -sudo /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml track-repository -virtual-storage <virtual-storage> -repository <repository> -replicate-immediately +sudo /opt/gitlab/embedded/bin/praefect -config /var/opt/gitlab/praefect/config.toml track-repository -virtual-storage <virtual-storage> -authoritative-storage <storage-name> -repository <repository> -replicate-immediately ``` - `-virtual-storage` is the virtual storage the repository is located in. Virtual storages are configured in `/etc/gitlab/gitlab.rb` under `praefect['virtual_storages]` and looks like the following: diff --git a/doc/administration/gitaly/troubleshooting.md b/doc/administration/gitaly/troubleshooting.md index ff136b44340..516af4ca469 100644 --- a/doc/administration/gitaly/troubleshooting.md +++ b/doc/administration/gitaly/troubleshooting.md @@ -554,7 +554,7 @@ Is [some cases](index.md#known-issues) the Praefect database can get out of sync a given repository is fully synced on all nodes, run the [`gitlab:praefect:replicas` Rake task](../raketasks/praefect.md#replica-checksums) that checksums the repository on all Gitaly nodes. -The [Praefect dataloss](recovery.md#check-for-data-loss) command only checks the state of the repo in the Praefect database, and cannot +The [Praefect dataloss](recovery.md#check-for-data-loss) command only checks the state of the repository in the Praefect database, and cannot be relied to detect sync problems in this scenario. ### Relation does not exist errors @@ -610,3 +610,45 @@ Possible solutions: - Provision larger VMs to gain access to larger network traffic allowances. - Use your cloud service's monitoring and logging to check that the Praefect nodes are not exhausting their traffic allowances. + +## Profiling Gitaly + +Gitaly exposes several of Golang's built-in performance profiling tools on the Prometheus listen port. For example, if Prometheus is listening +on port `9236` of the GitLab server: + +- Get a list of running `goroutines` and their backtraces: + + ```shell + curl --output goroutines.txt "http://<gitaly_server>:9236/debug/pprof/goroutine?debug=2" + ``` + +- Run a CPU profile for 30 seconds: + + ```shell + curl --output cpu.bin "http://<gitaly_server>:9236/debug/pprof/profile" + ``` + +- Profile heap memory usage: + + ```shell + curl --output heap.bin "http://<gitaly_server>:9236/debug/pprof/heap" + ``` + +- Record a 5 second execution trace. This will impact Gitaly's performance while running: + + ```shell + curl --output trace.bin "http://<gitaly_server>:9236/debug/pprof/trace?seconds=5" + ``` + +On a host with `go` installed, the CPU profile and heap profile can be viewed in a browser: + +```shell +go tool pprof -http=:8001 cpu.bin +go tool pprof -http=:8001 heap.bin +``` + +Execution traces can be viewed by running: + +```shell +go tool trace heap.bin +``` diff --git a/doc/administration/incoming_email.md b/doc/administration/incoming_email.md index cd1f39b1295..caade32b7c2 100644 --- a/doc/administration/incoming_email.md +++ b/doc/administration/incoming_email.md @@ -68,11 +68,16 @@ this method only supports replies, and not the other features of [incoming email ## Accepted headers -Email is processed correctly when a configured email address is present in one of the following headers: +> Accepting `Received` headers [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/81489) in GitLab 14.9 [with a flag](feature_flags.md) named `use_received_header_for_incoming_emails`. Enabled by default. + +Email is processed correctly when a configured email address is present in one of the following headers +(sorted in the order they are checked): - `To` +- `References` - `Delivered-To` - `Envelope-To` or `X-Envelope-To` +- `Received` In GitLab 14.6 and later, [Service Desk](../user/project/service_desk.md) also checks accepted headers. @@ -84,6 +89,9 @@ However, it might not include the configured GitLab email address if: - The address was included when using "Reply all". - The email was forwarded. +The `Received` header can contain multiple email addresses. These are checked in the order that they appear. +The first match is used. + ## Rejected headers To prevent unwanted issue creation from automatic email systems, GitLab ignores all incoming email diff --git a/doc/administration/index.md b/doc/administration/index.md index bd6549fca80..50dc2bc905a 100644 --- a/doc/administration/index.md +++ b/doc/administration/index.md @@ -35,7 +35,7 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Installing GitLab on Amazon Web Services (AWS)](../install/aws/index.md): Set up GitLab on Amazon AWS. - [Geo](geo/index.md): Replicate your GitLab instance to other geographic locations as a read-only fully operational version. - [Disaster Recovery](geo/disaster_recovery/index.md): Quickly fail-over to a different site with minimal effort in a disaster situation. -- [Add License](../user/admin_area/license.md): Upload a license at install time to unlock features that are in paid tiers of GitLab. +- [Add License](../user/admin_area/license.md): Add a license at install time to unlock features that are in paid tiers of GitLab. ### Configuring GitLab @@ -68,7 +68,7 @@ Learn how to install, configure, update, and maintain your GitLab instance. - [Elasticsearch](../integration/elasticsearch.md): Enable Elasticsearch to empower Advanced Search. Use when you deal with a huge amount of data. - [External Classification Policy Authorization](../user/admin_area/settings/external_authorization.md) -- [Upload a license](../user/admin_area/license.md): Upload a license to unlock +- [Add a license](../user/admin_area/license.md): Add a license to unlock features that are in paid tiers of GitLab. - [Admin Area](../user/admin_area/index.md): for self-managed instance-wide configuration and maintenance. diff --git a/doc/administration/instance_limits.md b/doc/administration/instance_limits.md index 614ab5278c5..11a34f5b5f8 100644 --- a/doc/administration/instance_limits.md +++ b/doc/administration/instance_limits.md @@ -153,6 +153,17 @@ Set the limit to `0` to disable it. - **Default rate limit**: Disabled (unlimited). +### Search rate limit + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/80631) in GitLab 14.9 + +This setting limits global search requests. + +| Limit | Default (requests per minute) | +|-------------------------|-------------------------------| +| Authenticated user | 30 | +| Unauthenticated user | 10 | + ## Gitaly concurrency limit Clone traffic can put a large strain on your Gitaly service. To prevent such workloads from overwhelming your Gitaly server, you can set concurrency limits in Gitaly's configuration file. @@ -208,13 +219,18 @@ When the number exceeds the limit the page displays an alert and links to a pagi > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/51401) in GitLab 11.10. -The number of pipelines that can be created in a single push is 4. -This limit prevents the accidental creation of pipelines when `git push --all` -or `git push --mirror` is used. +When pushing multiple changes with a single Git push, like multiple tags or branches, +only four tag or branch pipelines can be triggered. This limit prevents the accidental +creation of a large number of pipelines when using `git push --all` or `git push --mirror`. + +[Merge request pipelines](../ci/pipelines/merge_request_pipelines.md) are not limited. +If the Git push updates multiple merge requests at the same time, a merge request pipeline +can trigger for every updated merge request. -This limit does not affect any of the updated merge request pipelines. -All updated merge requests have a pipeline created when using -[merge request pipelines](../ci/pipelines/merge_request_pipelines.md). +To remove the limit so that any number of pipelines can trigger for a single Git push event, +administrators can enable the `git_push_create_all_pipelines` [feature flag](feature_flags.md). +Enabling this feature flag is not recommended, as it can cause excessive load on the GitLab +instance if too many changes are pushed at once and a flood of pipelines are created accidentally. ## Retention of activity history @@ -558,7 +574,7 @@ Plan.default.actual_limits.update!(ci_max_artifact_size_junit: 10) ### Number of files per GitLab Pages web-site -The total number of file entries (including directories and symlinks) is limited to `100000` per +The total number of file entries (including directories and symlinks) is limited to `200,000` per GitLab Pages website. This is the default limit for all [GitLab self-managed and SaaS plans](https://about.gitlab.com/pricing/). @@ -800,7 +816,7 @@ Reports that go over the 20 MB limit aren't loaded. Affected reports: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/8638) in GitLab 13.3. You can set a limit on the content of repository files that are indexed in -Elasticsearch. Any files larger than this limit only index the file name. +Elasticsearch. Any files larger than this limit only index the filename. The file content is neither indexed nor searchable. Setting a limit helps reduce the memory usage of the indexing processes and @@ -942,3 +958,7 @@ varies by file type: If a branch is merged while open merge requests still point to it, GitLab can retarget merge requests pointing to the now-merged branch. To learn more, read [Branch retargeting on merge](../user/project/merge_requests/getting_started.md#branch-retargeting-on-merge). + +## CDN-based limits on GitLab.com + +In addition to application-based limits, GitLab.com is configured to use Cloudflare's standard DDoS protection and Spectrum to protect Git over SSH. Cloudflare terminates client TLS connections but is not application aware and cannot be used for limits tied to users or groups. Cloudflare page rules and rate limits are configured with Terraform. These configurations are [not public](https://about.gitlab.com/handbook/communication/#not-public) because they include security and abuse implementations that detect malicious activities and making them public would undermine those operations. diff --git a/doc/administration/integration/plantuml.md b/doc/administration/integration/plantuml.md index 94fef89d966..5a9699b3a39 100644 --- a/doc/administration/integration/plantuml.md +++ b/doc/administration/integration/plantuml.md @@ -21,7 +21,7 @@ blocks to an HTML image tag, with the source pointing to the PlantUML instance. diagram delimiters `@startuml`/`@enduml` aren't required, as these are replaced by the `plantuml` block: -- **Markdown** +- **Markdown** files with the extension `.md`: ````markdown ```plantuml @@ -30,7 +30,10 @@ by the `plantuml` block: ``` ```` -- **AsciiDoc** + For additional acceptable extensions, review the + [`languages.yaml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/vendor/languages.yml#L3174) file. + +- **AsciiDoc** files with the extension `.asciidoc`, `.adoc`, or `.asc`: ```plaintext [plantuml, format="png", id="myDiagram", width="200px"] @@ -224,6 +227,16 @@ these steps: gitlab_rails['env'] = { 'PLANTUML_ENCODING' => 'deflate' } ``` + In GitLab Helm chart, you can set it by adding a variable to the + [global.extraEnv](https://gitlab.com/gitlab-org/charts/gitlab/blob/master/doc/charts/globals.md#extraenv) + section, like this: + + ```yaml + global: + extraEnv: + PLANTUML_ENCODING: deflate + ``` + - For GitLab versions 13.1 and later, PlantUML integration now [requires a header prefix in the URL](https://github.com/plantuml/plantuml/issues/117#issuecomment-6235450160) to distinguish different encoding types. diff --git a/doc/administration/integration/terminal.md b/doc/administration/integration/terminal.md index f570c9b559f..792afc6c3d7 100644 --- a/doc/administration/integration/terminal.md +++ b/doc/administration/integration/terminal.md @@ -11,6 +11,11 @@ info: To determine the technical writer assigned to the Stage/Group associated w WARNING: This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. +- Read more about the non-deprecated [Web Terminals accessible through the Web IDE](../../user/project/web_ide/index.md). +- Read more about the non-deprecated [Web Terminals accessible from a running CI job](../../ci/interactive_web_terminal/index.md). + +--- + With the introduction of the [Kubernetes integration](../../user/infrastructure/clusters/index.md), GitLab can store and use credentials for a Kubernetes cluster. GitLab uses these credentials to provide access to diff --git a/doc/administration/job_artifacts.md b/doc/administration/job_artifacts.md index 8f824fcefb3..544a5973052 100644 --- a/doc/administration/job_artifacts.md +++ b/doc/administration/job_artifacts.md @@ -307,6 +307,33 @@ To migrate back to local storage: ## Expiring artifacts +> [In GitLab 14.6](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/76504), we improved the performance of removing expired artifacts, introduced [with a flag](feature_flags.md) named `ci_destroy_all_expired_service`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to +[enable the feature flag](feature_flags.md) named `ci_destroy_all_expired_service`. The feature is not ready for +production use. +On GitLab.com, this feature is not available. + +### Removing expired job artifacts on GitLab self-managed instances + +In the process of migrating old artifacts for our SaaS customers, we are working to resolve any potential unrecoverable data loss for self-managed customers for artifacts that they may not want deleted yet. Before we can use the more performant way of cleaning up expired artifacts, we need to do some remediation to make sure customers don't lose their data, which is part of our effort in [the relevant epic](https://gitlab.com/groups/gitlab-org/-/epics/7097). + +Two options are available: + +- If you don't need any artifacts created before 2020-06-23, an Administrator can enable the worker for removing expired CI/CD artifacts: + + ```ruby + Feature.enable(:ci_destroy_all_expired_service) + ``` + +- If you want to keep any artifacts (including job logs) before 2020-06-23, follow the [progress of the migration effort](https://gitlab.com/groups/gitlab-org/-/epics/7097) where we work on a resolution to have this flag fully enabled in a future release. + +Alternatively, Administrators can also run commands in the Rails console to +[delete artifacts from completed jobs prior to a specific date](#delete-job-artifacts-from-jobs-completed-before-a-specific-date). + +### Usage details + If [`artifacts:expire_in`](../ci/yaml/index.md#artifactsexpire_in) is used to set an expiry for the artifacts, they are marked for deletion right after that date passes. Otherwise, they expire per the [default artifacts expiration setting](../user/admin_area/settings/continuous_integration.md). @@ -387,6 +414,37 @@ space, and in some cases, manually delete job artifacts to reclaim disk space. One possible first step is to [clean up _orphaned_ artifact files](../raketasks/cleanup.md#remove-orphan-artifact-files). +#### List projects and builds with artifacts with a specific expiration (or no expiration) + +Using a [Rails console](operations/rails_console.md), you can find projects that have job artifacts with either: + +- No expiration date. +- An expiration date more than 7 days in the future. + +Similar to [deleting artifacts](#delete-job-artifacts-from-jobs-completed-before-a-specific-date), use the following example time frames +and alter them as needed: + +- `7.days.from_now` +- `10.days.from_now` +- `2.weeks.from_now` +- `3.months.from_now` + +Each of the following scripts also limits the search to 50 results with `.limit(50)`, but this number can also be changed as needed: + +```ruby +# Find builds & projects with artifacts that never expire +builds_with_artifacts_that_never_expire = Ci::Build.with_downloadable_artifacts.where(artifacts_expire_at: nil).limit(50) +builds_with_artifacts_that_never_expire.find_each do |build| + puts "Build with id #{build.id} has artifacts that don't expire and belongs to project #{build.project.full_path}" +end + +# Find builds & projects with artifacts that expire after 7 days from today +builds_with_artifacts_that_expire_in_a_week = Ci::Build.with_downloadable_artifacts.where('artifacts_expire_at > ?', 7.days.from_now).limit(50) +builds_with_artifacts_that_expire_in_a_week.find_each do |build| + puts "Build with id #{build.id} has artifacts that expire at #{build.artifacts_expire_at} and belongs to project #{build.project.full_path}" +end +``` + #### List projects by total size of job artifacts stored List the top 20 projects, sorted by the total size of job artifacts stored, by @@ -435,7 +493,7 @@ list = arts.order(size: :desc).limit(50).each do |art| end ``` -To change the number of projects listed, change the number in `limit(50)`. +To change the number of job artifacts listed, change the number in `limit(50)`. #### Delete job artifacts from jobs completed before a specific date @@ -572,3 +630,15 @@ review: ``` In both cases, you might need to add `region` to the job artifact [object storage configuration](#connection-settings). + +### Job artifact upload fails with `500 Internal Server Error (Missing file)` + +Bucket names that include folder paths are not supported with [consolidated object storage](object_storage.md#consolidated-object-storage-configuration). +For example, `bucket/path`. If a bucket name has a path in it, you might receive an error similar to: + +```plaintext +WARNING: Uploading artifacts as "archive" to coordinator... POST https://gitlab.example.com/api/v4/jobs/job_id/artifacts?artifact_format=zip&artifact_type=archive&expire_in=1+day: 500 Internal Server Error (Missing file) +FATAL: invalid argument +``` + +If a job artifact fails to upload with the above error when using consolidated object storage, make sure you are [using separate buckets](object_storage.md#use-separate-buckets) for each data type. diff --git a/doc/administration/logs.md b/doc/administration/logs.md index c4e9642d048..22e13270179 100644 --- a/doc/administration/logs.md +++ b/doc/administration/logs.md @@ -1049,7 +1049,10 @@ For example: ## Mattermost Logs -For Omnibus GitLab installations, Mattermost logs are in `/var/log/gitlab/mattermost/mattermost.log`. +For Omnibus GitLab installations, Mattermost logs are in these locations: + +- `/var/log/gitlab/mattermost/mattermost.log` +- `/var/log/gitlab/mattermost/current` ## Workhorse Logs @@ -1095,9 +1098,9 @@ For Omnibus GitLab installations, GitLab Monitor logs are in `/var/log/gitlab/gi For Omnibus GitLab installations, GitLab Exporter logs are in `/var/log/gitlab/gitlab-exporter/`. -## GitLab Agent Server +## GitLab agent server -For Omnibus GitLab installations, GitLab Agent Server logs are +For Omnibus GitLab installations, GitLab agent server logs are in `/var/log/gitlab/gitlab-kas/`. ## Praefect Logs diff --git a/doc/administration/monitoring/gitlab_self_monitoring_project/index.md b/doc/administration/monitoring/gitlab_self_monitoring_project/index.md index 911d76a89e3..09cef51d14c 100644 --- a/doc/administration/monitoring/gitlab_self_monitoring_project/index.md +++ b/doc/administration/monitoring/gitlab_self_monitoring_project/index.md @@ -8,10 +8,11 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/32351) in GitLab 12.7 [with a flag](../../feature_flags.md) named `self_monitoring_project`. Disabled by default. > - Generally available in GitLab 12.8. [Feature flag `self_monitoring_project`](https://gitlab.com/gitlab-org/gitlab/-/issues/198511) removed. -> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/348909) in GitLab 14.8. +> - [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/348909) in GitLab 14.9. Planned for removal in GitLab 15.0. WARNING: -This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/348909) in GitLab 14.8. +This feature is in its end-of-life process. It is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/348909) +for use in GitLab 14.9, and is planned for removal in GitLab 15.0. GitLab provides administrators insights into the health of their GitLab instance. diff --git a/doc/administration/monitoring/index.md b/doc/administration/monitoring/index.md index df655053723..381c807dbc5 100644 --- a/doc/administration/monitoring/index.md +++ b/doc/administration/monitoring/index.md @@ -24,6 +24,3 @@ Explore our features to monitor your GitLab instance: provide health check information when probed. - [`nginx_status`](https://docs.gitlab.com/omnibus/settings/nginx.html#enablingdisabling-nginx_status): Monitor your NGINX server status. -- [Auto Monitoring](../../topics/autodevops/stages.md#auto-monitoring): Automated - monitoring for your application's server and response metrics, provided by - [Auto DevOps](../../topics/autodevops/index.md). diff --git a/doc/administration/monitoring/performance/img/performance_bar_request_selector_warning.png b/doc/administration/monitoring/performance/img/performance_bar_request_selector_warning.png Binary files differdeleted file mode 100644 index 3872c8b41b2..00000000000 --- a/doc/administration/monitoring/performance/img/performance_bar_request_selector_warning.png +++ /dev/null diff --git a/doc/administration/monitoring/performance/performance_bar.md b/doc/administration/monitoring/performance/performance_bar.md index e93dbd3a2d6..bc311f73dfe 100644 --- a/doc/administration/monitoring/performance/performance_bar.md +++ b/doc/administration/monitoring/performance/performance_bar.md @@ -30,8 +30,7 @@ From left to right, the performance bar displays: is enabled. It shows which server role was used for the query. "Primary" means that the query was sent to the read/write primary server. "Replica" means it was sent to a read-only replica. - - **Config name**: shows up only when the - `GITLAB_MULTIPLE_DATABASE_METRICS` environment variable is set. This is + - **Configuration name**: this is used to distinguish between different databases configured for different GitLab features. The name shown is the same name used to configure database connections in GitLab. @@ -48,12 +47,13 @@ From left to right, the performance bar displays: - **External HTTP calls**: the time taken (in milliseconds) and the total number of external calls to other systems. Click to display a modal window with more details. -- **Load timings** of the page: if your browser supports load timings (Chromium - and Chrome) several values in milliseconds, separated by slashes. +- **Load timings** of the page: if your browser supports load timings, several + values in milliseconds, separated by slashes. Click to display a modal window with more details. The values, from left to right: - **Backend**: time needed for the base page to load. - [**First Contentful Paint**](https://web.dev/first-contentful-paint/): - Time until something was visible to the user. + Time until something was visible to the user. Displays `NaN` if your browser does not + support this feature. - [**DomContentLoaded**](https://developers.google.com/web/fundamentals/performance/critical-rendering-path/measure-crp) Event. - **Total number of requests** the page loaded. - **Memory**: the amount of memory consumed and objects allocated during the selected request. @@ -64,6 +64,9 @@ From left to right, the performance bar displays: can be added by its full URL (authenticated as the current user), or by the value of its `X-Request-Id` header. - **Download**: a link to download the raw JSON used to generate the Performance Bar reports. +- **Memory Report**: a link that generates a + [memory profiling](../../../development/performance.md#using-memory-profiler) + report of the current URL. - **Flamegraph** with mode: a link to generate a [flamegraph](../../../development/profiling.md#speedscope-flamegraphs) of the current URL with the selected [Stackprof mode](https://github.com/tmm1/stackprof#sampling): - The **Wall** mode samples every *interval* of the time on a clock on a wall. The interval is set to `10100` microseconds. @@ -91,18 +94,14 @@ For non-administrators to display the performance bar, it must be ## Request warnings +> [Warning icon in the request selector removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/82187) in GitLab 14.9. + Requests that exceed predefined limits display a warning **{warning}** icon and explanation next to the metric. In this example, the Gitaly call duration exceeded the threshold.  -If any requests on the current page generated warnings, the warning icon displays -next to the **Requests** selector menu. In this selector menu, an exclamation `(!)` -appears next to requests with warnings. - - - ## Enable the performance bar for non-administrators The performance bar is disabled by default for non-administrators. To enable it diff --git a/doc/administration/monitoring/prometheus/index.md b/doc/administration/monitoring/prometheus/index.md index 9d90c4f3cc7..22f7419be9a 100644 --- a/doc/administration/monitoring/prometheus/index.md +++ b/doc/administration/monitoring/prometheus/index.md @@ -301,7 +301,7 @@ Prometheus has several custom flags to configure local storage: - `storage.tsdb.retention.time`: when to remove old data. Defaults to `15d`. Overrides `storage.tsdb.retention` if this flag is set to anything other than the default. -- `storage.tsdb.retention.size`: [EXPERIMENTAL] the maximum number of bytes of storage blocks to +- `storage.tsdb.retention.size`: (experimental) the maximum number of bytes of storage blocks to retain. The oldest data is removed first. Defaults to `0` (disabled). This flag is experimental and may change in future releases. Units supported: `B`, `KB`, `MB`, `GB`, `TB`, `PB`, `EB`. For example, `512MB`. diff --git a/doc/administration/object_storage.md b/doc/administration/object_storage.md index 420f2191ef2..8ea4ad9019e 100644 --- a/doc/administration/object_storage.md +++ b/doc/administration/object_storage.md @@ -62,7 +62,7 @@ Using the consolidated object storage configuration has a number of advantages: - It enables the use of [encrypted S3 buckets](#encrypted-s3-buckets). - It [uploads files to S3 with proper `Content-MD5` headers](https://gitlab.com/gitlab-org/gitlab-workhorse/-/issues/222). -Because [direct upload mode](../development/uploads.md#direct-upload) +Because [direct upload mode](../development/uploads/implementation.md#direct-upload) must be enabled, only the following providers can be used: - [Amazon S3-compatible providers](#s3-compatible-connection-settings) @@ -598,6 +598,7 @@ See the following additional guides: 1. Configure [database lookup of SSH keys](operations/fast_ssh_key_lookup.md) to eliminate the need for a shared `authorized_keys` file. 1. [Prevent local disk usage for job logs](job_logs.md#prevent-local-disk-usage). +1. [Disable Pages local storage](pages/index.md#disable-pages-local-storage). ## Warnings, limitations, and known issues diff --git a/doc/administration/operations/fast_ssh_key_lookup.md b/doc/administration/operations/fast_ssh_key_lookup.md index dca99879cc3..8877b1266e0 100644 --- a/doc/administration/operations/fast_ssh_key_lookup.md +++ b/doc/administration/operations/fast_ssh_key_lookup.md @@ -182,7 +182,7 @@ file for the environment, since it isn't generated dynamically. ## Troubleshooting If your SSH traffic is [slow](https://github.com/linux-pam/linux-pam/issues/270) -or causing high CPU load, be sure to check the size of `/var/log/btmp`, and ensure it is rotated on a regular basis. +or causing high CPU load, be sure to check the size of `/var/log/btmp`, and ensure it is rotated on a regular basis or after reaching a certain size. If this file is very large, GitLab SSH fast lookup can cause the bottleneck to be hit more frequently, thus decreasing performance even further. If you are able to, you may consider disabling [`UsePAM` in your `sshd_config`](https://linux.die.net/man/5/sshd_config) to avoid reading `/var/log/btmp` altogether. diff --git a/doc/administration/operations/puma.md b/doc/administration/operations/puma.md index c7df8249ae4..9e828b39f46 100644 --- a/doc/administration/operations/puma.md +++ b/doc/administration/operations/puma.md @@ -4,45 +4,19 @@ group: Distribution info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Puma **(FREE SELF)** +# Configure the bundled Puma instance of the GitLab package **(FREE SELF)** -Puma is a simple, fast, multi-threaded, and highly concurrent HTTP 1.1 server for -Ruby applications. It's the default GitLab web server since GitLab 13.0 -and has replaced Unicorn. From GitLab 14.0, Unicorn is no longer supported. +Puma is a fast, multi-threaded, and highly concurrent HTTP 1.1 server for +Ruby applications. It runs the core Rails application that provides the user-facing +features of GitLab. -NOTE: -Starting with GitLab 13.0, Puma is the default web server and Unicorn has been disabled. -In GitLab 14.0, Unicorn was removed from the Linux package and only Puma is available. - -## Configure Puma - -To configure Puma: - -1. Determine suitable Puma worker and thread [settings](../../install/requirements.md#puma-settings). -1. If you're switching from Unicorn, [convert any custom settings to Puma](#convert-unicorn-settings-to-puma). -1. For multi-node deployments, configure the load balancer to use the - [readiness check](../load_balancer.md#readiness-check). -1. Reconfigure GitLab so the above changes take effect: - - ```shell - sudo gitlab-ctl reconfigure - ``` - -For Helm-based deployments, see the -[`webservice` chart documentation](https://docs.gitlab.com/charts/charts/gitlab/webservice/index.html). - -For more details about the Puma configuration, see the -[Puma documentation](https://github.com/puma/puma#configuration). +## Reducing memory use -## Puma Worker Killer +To reduce memory use, Puma forks worker processes. Each time a worker is created, +it shares memory with the primary process. The worker uses additional memory only +when it changes or adds to its memory pages. -Puma forks worker processes as part of a strategy to reduce memory use. - -Each time a worker is created, it shares memory with the primary process and -only uses additional memory when it makes changes or additions to its memory pages. - -Memory use by workers therefore increases over time, and Puma Worker Killer is the -mechanism that recovers this memory. +Memory use increases over time, but you can use Puma Worker Killer to recover memory. By default: @@ -50,6 +24,8 @@ By default: exceeds a [memory limit](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/cluster/puma_worker_killer_initializer.rb). - Rolling restarts of Puma workers are performed every 12 hours. +### Change the memory limit setting + To change the memory limit setting: 1. Edit `/etc/gitlab/gitlab.rb`: @@ -58,26 +34,28 @@ To change the memory limit setting: puma['per_worker_max_memory_mb'] = 1024 ``` -1. Reconfigure GitLab for the changes to take effect: +1. Reconfigure GitLab: ```shell sudo gitlab-ctl reconfigure ``` -There are costs associated with killing and replacing workers including -reduced capacity to run GitLab, and CPU that is consumed -restarting the workers. `per_worker_max_memory_mb` should be set to a -higher value if the worker killer is replacing workers too often. +When workers are killed and replaced, capacity to run GitLab is reduced, +and CPU is consumed. Set `per_worker_max_memory_mb` to a higher value if the worker killer +is replacing workers too often. -Worker count is calculated based on CPU cores, so a small GitLab deployment +Worker count is calculated based on CPU cores. A small GitLab deployment with 4-8 workers may experience performance issues if workers are being restarted -frequently, once or more per minute. This is too often. +too often (once or more per minute). A higher value of `1200` or more would be beneficial if the server has free memory. -The worker killer checks every 20 seconds, and can be monitored using -[the Puma log](../logs.md#puma_stdoutlog) `/var/log/gitlab/puma/puma_stdout.log`. -For example, for GitLab 13.5: +### Monitor worker memory + +The worker killer checks memory every 20 seconds. + +To monitor the worker killer, use [the Puma log](../logs.md#puma_stdoutlog) `/var/log/gitlab/puma/puma_stdout.log`. +For example: ```plaintext PumaWorkerKiller: Out of memory. 4 workers consuming total: 4871.23828125 MB @@ -88,9 +66,9 @@ From this output: - The formula that calculates the maximum memory value results in workers being killed before they reach the `per_worker_max_memory_mb` value. -- The default values for the formula before GitLab 13.5 were 550MB for the primary - and `per_worker_max_memory_mb` specified 850MB for each worker. -- As of GitLab 13.5 the values are primary: 800MB, worker: 1024MB. +- In GitLab 13.4 and earlier, the default values for the formula were 550MB for the primary + and 850MB for each worker. +- In GitLab 13.5 and later, the values are primary: 800MB, worker: 1024MB. - The threshold for workers to be killed is set at 98% of the limit: ```plaintext @@ -102,16 +80,15 @@ From this output: Increasing the maximum to `1200`, for example, would set a `max: 5488 MB` value. -Workers use additional memory on top of the shared memory, how much +Workers use additional memory on top of the shared memory. The amount of memory depends on a site's use of GitLab. -## Worker timeout +## Change the worker timeout -A [timeout of 60 seconds](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/initializers/rack_timeout.rb) -is used when Puma is enabled. +The default Puma [timeout is 60 seconds](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/initializers/rack_timeout.rb). NOTE: -Unlike Unicorn, the `puma['worker_timeout']` setting does not set the maximum request duration. +The `puma['worker_timeout']` setting does not set the maximum request duration. To change the worker timeout to 600 seconds: @@ -123,26 +100,38 @@ To change the worker timeout to 600 seconds: } ``` -1. Reconfigure GitLab for the changes to take effect: +1. Reconfigure GitLab: ```shell sudo gitlab-ctl reconfigure ``` -## Memory-constrained environments +## Disable Puma clustered mode in memory-constrained environments In a memory-constrained environment with less than 4GB of RAM available, consider disabling Puma -[Clustered mode](https://github.com/puma/puma#clustered-mode). +[clustered mode](https://github.com/puma/puma#clustered-mode). -Configuring Puma by setting the amount of `workers` to `0` could reduce memory usage by hundreds of MB. -For details on Puma worker and thread settings, see the [Puma requirements](../../install/requirements.md#puma-settings). +Set the number of `workers` to `0` to reduce memory usage by hundreds of MB: + +1. Edit `/etc/gitlab/gitlab.rb`: -Unlike in a Clustered mode, which is set up by default, only a single Puma process would serve the application. + ```ruby + puma['worker_processes'] = 0 + ``` -The downside of running Puma with such configuration is the reduced throughput, which could be -considered as a fair tradeoff in a memory-constraint environment. +1. Reconfigure GitLab: -When running Puma in Single mode, some features are not supported: + ```shell + sudo gitlab-ctl reconfigure + ``` + +Unlike in a clustered mode, which is set up by default, only a single Puma process would serve the application. +For details on Puma worker and thread settings, see the [Puma requirements](../../install/requirements.md#puma-settings). + +The downside of running Puma in this configuration is the reduced throughput, which can be +considered a fair tradeoff in a memory-constrained environment. + +When running Puma in single mode, some features are not supported: - [Phased restart](https://gitlab.com/gitlab-org/gitlab/-/issues/300665) - [Puma Worker Killer](https://gitlab.com/gitlab-org/gitlab/-/issues/300664) @@ -151,22 +140,23 @@ To learn more, visit [epic 5303](https://gitlab.com/groups/gitlab-org/-/epics/53 ## Performance caveat when using Puma with Rugged -For deployments where NFS is used to store Git repository, we allow GitLab to use -[direct Git access](../gitaly/index.md#direct-access-to-git-in-gitlab) to improve performance using +For deployments where NFS is used to store Git repositories, GitLab uses +[direct Git access](../gitaly/index.md#direct-access-to-git-in-gitlab) to improve performance by using [Rugged](https://github.com/libgit2/rugged). Rugged usage is automatically enabled if direct Git access [is available](../gitaly/index.md#how-it-works) -and Puma is running single threaded, unless it is disabled by -[feature flags](../../development/gitaly.md#legacy-rugged-code). +and Puma is running single threaded, unless it is disabled by a +[feature flag](../../development/gitaly.md#legacy-rugged-code). -MRI Ruby uses a GVL. This allows MRI Ruby to be multi-threaded, but running at -most on a single core. Since Rugged can use a thread for long periods of -time (due to intensive I/O operations of Git access), this can starve other threads -that might be processing requests. This is not a case for Unicorn or Puma running -in a single thread mode, as concurrently at most one request is being processed. +MRI Ruby uses a Global VM Lock (GVL). GVL allows MRI Ruby to be multi-threaded, but running at +most on a single core. -We are actively working on removing Rugged usage. Even though performance without Rugged +Git includes intensive I/O operations. When Rugged uses a thread for a long period of time, +other threads that might be processing requests can starve. Puma running in single thread mode +does not have this issue, because concurrently at most one request is being processed. + +GitLab is working to remove Rugged usage. Even though performance without Rugged is acceptable today, in some cases it might be still beneficial to run with it. Given the caveat of running Rugged with multi-threaded Puma, and acceptable @@ -177,55 +167,70 @@ This default behavior may not be the optimal configuration in some situations. I plays an important role in your deployment, we suggest you benchmark to find the optimal configuration: -- The safest option is to start with single-threaded Puma. When working with - Rugged, single-threaded Puma works the same as Unicorn. -- To force Rugged to be used with multi-threaded Puma, you can use - [feature flags](../../development/gitaly.md#legacy-rugged-code). +- The safest option is to start with single-threaded Puma. +- To force Rugged to be used with multi-threaded Puma, you can use a + [feature flag](../../development/gitaly.md#legacy-rugged-code). -## Convert Unicorn settings to Puma +## Switch from Unicorn to Puma NOTE: -Starting with GitLab 13.0, Puma is the default web server and Unicorn has been -disabled by default. In GitLab 14.0, Unicorn was removed from the Linux package -and only Puma is available. +For Helm-based deployments, see the +[`webservice` chart documentation](https://docs.gitlab.com/charts/charts/gitlab/webservice/index.html). + +Starting with GitLab 13.0, Puma is the default web server and Unicorn has been disabled. +In GitLab 14.0, [Unicorn was removed](../../update/removals.md#unicorn-in-gitlab-self-managed) +from the Linux package and is no longer supported. -Puma has a multi-thread architecture which uses less memory than a multi-process +Puma has a multi-thread architecture that uses less memory than a multi-process application server like Unicorn. On GitLab.com, we saw a 40% reduction in memory -consumption. Most Rails applications requests normally include a proportion of I/O wait time. +consumption. Most Rails application requests normally include a proportion of I/O wait time. -During I/O wait time MRI Ruby releases the GVL (Global VM Lock) to other threads. +During I/O wait time, MRI Ruby releases the GVL to other threads. Multi-threaded Puma can therefore still serve more requests than a single process. When switching to Puma, any Unicorn server configuration will _not_ carry over automatically, due to differences between the two application servers. -The table below summarizes which Unicorn configuration keys correspond to those -in Puma when using the Linux package, and which ones have no corresponding counterpart. - -| Unicorn | Puma | -| ------------------------------------ | ---------------------------------- | -| `unicorn['enable']` | `puma['enable']` | -| `unicorn['worker_timeout']` | `puma['worker_timeout']` | -| `unicorn['worker_processes']` | `puma['worker_processes']` | -| n/a | `puma['ha']` | -| n/a | `puma['min_threads']` | -| n/a | `puma['max_threads']` | -| `unicorn['listen']` | `puma['listen']` | -| `unicorn['port']` | `puma['port']` | -| `unicorn['socket']` | `puma['socket']` | -| `unicorn['pidfile']` | `puma['pidfile']` | -| `unicorn['tcp_nopush']` | n/a | -| `unicorn['backlog_socket']` | n/a | -| `unicorn['somaxconn']` | `puma['somaxconn']` | -| n/a | `puma['state_path']` | -| `unicorn['log_directory']` | `puma['log_directory']` | -| `unicorn['worker_memory_limit_min']` | n/a | -| `unicorn['worker_memory_limit_max']` | `puma['per_worker_max_memory_mb']` | -| `unicorn['exporter_enabled']` | `puma['exporter_enabled']` | -| `unicorn['exporter_address']` | `puma['exporter_address']` | -| `unicorn['exporter_port']` | `puma['exporter_port']` | - -## Puma exporter - -You can use the Puma exporter to measure various Puma metrics. For more information, see -[Puma exporter](../monitoring/prometheus/puma_exporter.md). +To switch from Unicorn to Puma: + +1. Determine suitable Puma [worker and thread settings](../../install/requirements.md#puma-settings). +1. Convert any custom Unicorn settings to Puma. + + The table below summarizes which Unicorn configuration keys correspond to those + in Puma when using the Linux package, and which ones have no corresponding counterpart. + + | Unicorn | Puma | + | ------------------------------------ | ---------------------------------- | + | `unicorn['enable']` | `puma['enable']` | + | `unicorn['worker_timeout']` | `puma['worker_timeout']` | + | `unicorn['worker_processes']` | `puma['worker_processes']` | + | n/a | `puma['ha']` | + | n/a | `puma['min_threads']` | + | n/a | `puma['max_threads']` | + | `unicorn['listen']` | `puma['listen']` | + | `unicorn['port']` | `puma['port']` | + | `unicorn['socket']` | `puma['socket']` | + | `unicorn['pidfile']` | `puma['pidfile']` | + | `unicorn['tcp_nopush']` | n/a | + | `unicorn['backlog_socket']` | n/a | + | `unicorn['somaxconn']` | `puma['somaxconn']` | + | n/a | `puma['state_path']` | + | `unicorn['log_directory']` | `puma['log_directory']` | + | `unicorn['worker_memory_limit_min']` | n/a | + | `unicorn['worker_memory_limit_max']` | `puma['per_worker_max_memory_mb']` | + | `unicorn['exporter_enabled']` | `puma['exporter_enabled']` | + | `unicorn['exporter_address']` | `puma['exporter_address']` | + | `unicorn['exporter_port']` | `puma['exporter_port']` | + +1. Reconfigure GitLab: + + ```shell + sudo gitlab-ctl reconfigure + ``` + +1. Optional. For multi-node deployments, configure the load balancer to use the + [readiness check](../load_balancer.md#readiness-check). + +## Related topics + +- [Use the Puma exporter to measure various Puma metrics](../monitoring/prometheus/puma_exporter.md) diff --git a/doc/administration/package_information/deprecated_os.md b/doc/administration/package_information/deprecated_os.md deleted file mode 100644 index 1f6fe0fce5d..00000000000 --- a/doc/administration/package_information/deprecated_os.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: 'supported_os.md' -remove_date: '2022-02-18' ---- - -This document was moved to [another location](supported_os.md). - -<!-- This redirect file can be deleted after <2022-02-18>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/administration/package_information/index.md b/doc/administration/package_information/index.md index 2781f789409..c908f526569 100644 --- a/doc/administration/package_information/index.md +++ b/doc/administration/package_information/index.md @@ -1,7 +1,7 @@ --- stage: Enablement group: Distribution -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Package information **(FREE SELF)** diff --git a/doc/administration/package_information/licensing.md b/doc/administration/package_information/licensing.md index 02358c66993..a7bf5c52d7b 100644 --- a/doc/administration/package_information/licensing.md +++ b/doc/administration/package_information/licensing.md @@ -1,7 +1,7 @@ --- stage: Enablement group: Distribution -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Package Licensing **(FREE SELF)** diff --git a/doc/administration/package_information/omnibus_packages.md b/doc/administration/package_information/omnibus_packages.md index 115d6c394ad..21e3f5711ba 100644 --- a/doc/administration/package_information/omnibus_packages.md +++ b/doc/administration/package_information/omnibus_packages.md @@ -1,7 +1,7 @@ --- stage: Enablement group: Distribution -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Omnibus based packages and images **(FREE SELF)** diff --git a/doc/administration/package_information/postgresql_versions.md b/doc/administration/package_information/postgresql_versions.md index 97a35fb29ed..e707cb70187 100644 --- a/doc/administration/package_information/postgresql_versions.md +++ b/doc/administration/package_information/postgresql_versions.md @@ -1,7 +1,7 @@ --- stage: Enablement group: Distribution -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # PostgreSQL versions shipped with Omnibus GitLab **(FREE SELF)** diff --git a/doc/administration/package_information/signed_packages.md b/doc/administration/package_information/signed_packages.md index fb605f8d5be..d7bcfa113ff 100644 --- a/doc/administration/package_information/signed_packages.md +++ b/doc/administration/package_information/signed_packages.md @@ -1,7 +1,7 @@ --- stage: Enablement group: Distribution -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Package Signatures **(FREE SELF)** diff --git a/doc/administration/package_information/supported_os.md b/doc/administration/package_information/supported_os.md index 71ebc4d3647..547b8bf8658 100644 --- a/doc/administration/package_information/supported_os.md +++ b/doc/administration/package_information/supported_os.md @@ -27,6 +27,7 @@ The following lists the currently supported OSs and their possible EOL dates. | SLES 12 | GitLab EE 9.0.0 | x86_64 | Oct 2027 | <https://www.suse.com/lifecycle/> | | Ubuntu 18.04 | GitLab CE / GitLab EE 10.7.0 | amd64 | April 2023 | <https://wiki.ubuntu.com/Releases> | | Ubuntu 20.04 | GitLab CE / GitLab EE 13.2.0 | amd64, arm64 | April 2025 | <https://wiki.ubuntu.com/Releases> | +| Amazon Linux 2 | GitLab CE / GitLab EE 14.9.0 | amd64, arm64 | June 2023 | <https://aws.amazon.com/amazon-linux-2/faqs> | | Raspbian Buster | GitLab CE 12.2.0 | armhf | 2022 | <https://wiki.debian.org/DebianReleases#Production_Releases> | NOTE: diff --git a/doc/administration/packages/container_registry.md b/doc/administration/packages/container_registry.md index 33a5311709f..7bf7362c68b 100644 --- a/doc/administration/packages/container_registry.md +++ b/doc/administration/packages/container_registry.md @@ -457,6 +457,22 @@ To configure the `s3` storage driver in Omnibus: - `pathstyle` should be set to true to use `host/bucket_name/object` style paths instead of `bucket_name.host/object`. [Set to false for AWS S3](https://aws.amazon.com/blogs/aws/amazon-s3-path-deprecation-plan-the-rest-of-the-story/). + You can set a rate limit on connections to S3 to avoid 503 errors from the S3 API. To do this, + set `maxrequestspersecond` to a number within the [S3 request rate threshold](https://aws.amazon.com/premiumsupport/knowledge-center/s3-503-within-request-rate-prefix/): + + ```ruby + registry['storage'] = { + 's3' => { + 'accesskey' => 's3-access-key', + 'secretkey' => 's3-secret-key-for-access-key', + 'bucket' => 'your-s3-bucket', + 'region' => 'your-s3-region', + 'regionendpoint' => 'your-s3-regionendpoint', + 'maxrequestspersecond' => 100 + } + } + ``` + 1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. **Installations from source** @@ -1704,3 +1720,38 @@ What does this mean? This strongly suggests that the S3 user does not have the r [permissions to perform a HEAD request](https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadObject.html). The solution: check the [IAM permissions again](https://docs.docker.com/registry/storage-drivers/s3/). Once the right permissions were set, the error goes away. + +### Missing `gitlab-registry.key` prevents container repository deletion + +If you disable your GitLab instance's Container Registry and try to remove a project that has +container repositories, the following error occurs: + +```plaintext +Errno::ENOENT: No such file or directory @ rb_sysopen - /var/opt/gitlab/gitlab-rails/etc/gitlab-registry.key +``` + +In this case, follow these steps: + +1. Temporarily enable the instance-wide setting for the Container Registry in your `gitlab.rb`: + + ```ruby + gitlab_rails['registry_enabled'] = true + ``` + +1. Save the file and [reconfigure GitLab](../restart_gitlab.md#omnibus-gitlab-reconfigure) + for the changes to take effect. +1. Try the removal again. + +If you still can't remove the repository using the common methods, you can use the +[GitLab Rails console](../troubleshooting/navigating_gitlab_via_rails_console.md) +to remove the project by force: + +```ruby +# Path to the project you'd like to remove +prj = Project.find_by_full_path(<project_path>) + +# The following will delete the project's container registry, so be sure to double-check the path beforehand! +if prj.has_container_registry_tags? + prj.container_repositories.each { |p| p.destroy } +end +``` diff --git a/doc/administration/pages/index.md b/doc/administration/pages/index.md index f000824711f..56ccf70d169 100644 --- a/doc/administration/pages/index.md +++ b/doc/administration/pages/index.md @@ -547,6 +547,7 @@ archive. You can modify the cache behavior by changing the following configurati | `zip_cache_cleanup` | The interval at which archives are cleaned from memory if they have already expired. Default is 30s. | | `zip_cache_refresh` | The time interval in which an archive is extended in memory if accessed before `zip_cache_expiration`. This works together with `zip_cache_expiration` to determine if an archive is extended in memory. See the [example below](#zip-cache-refresh-example) for important details. Default is 30s. | | `zip_open_timeout` | The maximum time allowed to open a ZIP archive. Increase this time for big archives or slow network connections, as doing so may affect the latency of serving Pages. Default is 30s. | +| `zip_http_client_timeout` | The maximum time for the ZIP HTTP client. Default is 30m. | #### ZIP cache refresh example @@ -1387,15 +1388,20 @@ This can happen if your `gitlab-secrets.json` file is out of date between GitLab Pages. Follow steps 8-10 of [Running GitLab Pages on a separate server](#running-gitlab-pages-on-a-separate-server), in all of your GitLab Pages instances. -### Intermittent 502 errors when using an AWS Network Load Balancer and GitLab Pages is running on multiple application servers +### Intermittent 502 errors when using an AWS Network Load Balancer and GitLab Pages Connections will time out when using a Network Load Balancer with client IP preservation enabled and [the request is looped back to the source server](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/load-balancer-troubleshooting.html#loopback-timeout). This can happen to GitLab instances with multiple servers -running both the core GitLab application and GitLab Pages. +running both the core GitLab application and GitLab Pages. This can also happen when a single +container is running both the core GitLab application and GitLab Pages. AWS [recommends using an IP target type](https://aws.amazon.com/premiumsupport/knowledge-center/target-connection-fails-load-balancer/) to resolve this issue. +Turning off [client IP preservation](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/load-balancer-target-groups.html#client-ip-preservation) +may resolve this issue when the core GitLab application and GitLab Pages run on the same host or +container. + ### 500 error with `securecookie: failed to generate random iv` and `Failed to save the session` This problem most likely results from an [out-dated operating system](../package_information/supported_os.md#os-versions-that-are-no-longer-supported). diff --git a/doc/administration/postgresql/external.md b/doc/administration/postgresql/external.md index 67c5448a8a0..f4e4c7f8bef 100644 --- a/doc/administration/postgresql/external.md +++ b/doc/administration/postgresql/external.md @@ -21,7 +21,7 @@ If you use a cloud-managed service, or provide your own PostgreSQL instance: 1. If you are using a cloud-managed service, you may need to grant additional roles to your `gitlab` user: - Amazon RDS requires the [`rds_superuser`](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Appendix.PostgreSQL.CommonDBATasks.html#Appendix.PostgreSQL.CommonDBATasks.Roles) role. - - Azure Database for PostgreSQL requires the [`azure_pg_admin`](https://docs.microsoft.com/en-us/azure/postgresql/howto-create-users#how-to-create-additional-admin-users-in-azure-database-for-postgresql) role. + - Azure Database for PostgreSQL requires the [`azure_pg_admin`](https://docs.microsoft.com/en-us/azure/postgresql/howto-create-users#how-to-create-additional-admin-users-in-azure-database-for-postgresql) role. Azure Database for PostgreSQL - Flexible Server requires [allow-listing extensions before they can be installed](https://docs.microsoft.com/en-us/azure/postgresql/flexible-server/concepts-extensions#how-to-use-postgresql-extensions). - Google Cloud SQL requires the [`cloudsqlsuperuser`](https://cloud.google.com/sql/docs/postgres/users#default-users) role. This is for the installation of extensions during installation and upgrades. As an alternative, diff --git a/doc/administration/raketasks/doctor.md b/doc/administration/raketasks/doctor.md index bed3cdcbcfe..457077462a6 100644 --- a/doc/administration/raketasks/doctor.md +++ b/doc/administration/raketasks/doctor.md @@ -6,4 +6,6 @@ remove_date: '2022-03-04' This document was moved to [another location](check.md#verify-database-values-can-be-decrypted-using-the-current-secrets). <!-- This redirect file can be deleted after 2022-03-04. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/administration/reference_architectures/10k_users.md b/doc/administration/reference_architectures/10k_users.md index eb1127b5e99..fcce44f62b2 100644 --- a/doc/administration/reference_architectures/10k_users.md +++ b/doc/administration/reference_architectures/10k_users.md @@ -1363,6 +1363,7 @@ To configure the Praefect nodes, on each one: alertmanager['enable'] = false grafana['enable'] = false gitlab_exporter['enable'] = false + gitlab_kas['enable'] = false nginx['enable'] = false # Praefect Configuration @@ -1503,6 +1504,7 @@ On each node: alertmanager['enable'] = false grafana['enable'] = false gitlab_exporter['enable'] = false + gitlab_kas['enable'] = false nginx['enable'] = false # Prevent database migrations from running on upgrade automatically @@ -1524,6 +1526,11 @@ On each node: # Gitaly Auth Token # Should be the same as praefect_internal_token gitaly['auth_token'] = '<praefect_internal_token>' + + # Gitaly Pack-objects cache + # Recommended to be enabled for improved performance but can notably increase disk I/O + # Refer to https://docs.gitlab.com/ee/administration/gitaly/configure_gitaly.html#pack-objects-cache for more info + gitaly['pack_objects_cache_enabled'] = true ``` 1. Append the following to `/etc/gitlab/gitlab.rb` for each respective server: @@ -1631,7 +1638,7 @@ To configure Praefect with TLS: ```ruby git_data_dirs({ "default" => { - "gitaly_address" => 'tls://LOAD_BALANCER_SERVER_ADDRESS:2305', + "gitaly_address" => 'tls://LOAD_BALANCER_SERVER_ADDRESS:3305', "gitaly_token" => 'PRAEFECT_EXTERNAL_TOKEN' } }) @@ -1675,6 +1682,7 @@ To configure the Sidekiq nodes, on each one: alertmanager['enable'] = false grafana['enable'] = false gitlab_exporter['enable'] = false + gitlab_kas['enable'] = false nginx['enable'] = false # External URL diff --git a/doc/administration/reference_architectures/25k_users.md b/doc/administration/reference_architectures/25k_users.md index 86819024eeb..c08fe985b40 100644 --- a/doc/administration/reference_architectures/25k_users.md +++ b/doc/administration/reference_architectures/25k_users.md @@ -1367,6 +1367,7 @@ To configure the Praefect nodes, on each one: alertmanager['enable'] = false grafana['enable'] = false gitlab_exporter['enable'] = false + gitlab_kas['enable'] = false nginx['enable'] = false # Praefect Configuration @@ -1507,6 +1508,7 @@ On each node: alertmanager['enable'] = false grafana['enable'] = false gitlab_exporter['enable'] = false + gitlab_kas['enable'] = false nginx['enable'] = false # Prevent database migrations from running on upgrade automatically @@ -1528,6 +1530,11 @@ On each node: # Gitaly Auth Token # Should be the same as praefect_internal_token gitaly['auth_token'] = '<praefect_internal_token>' + + # Gitaly Pack-objects cache + # Recommended to be enabled for improved performance but can notably increase disk I/O + # Refer to https://docs.gitlab.com/ee/administration/gitaly/configure_gitaly.html#pack-objects-cache for more info + gitaly['pack_objects_cache_enabled'] = true ``` 1. Append the following to `/etc/gitlab/gitlab.rb` for each respective server: @@ -1635,7 +1642,7 @@ To configure Praefect with TLS: ```ruby git_data_dirs({ "default" => { - "gitaly_address" => 'tls://LOAD_BALANCER_SERVER_ADDRESS:2305', + "gitaly_address" => 'tls://LOAD_BALANCER_SERVER_ADDRESS:3305', "gitaly_token" => 'PRAEFECT_EXTERNAL_TOKEN' } }) @@ -1679,6 +1686,7 @@ To configure the Sidekiq nodes, on each one: alertmanager['enable'] = false grafana['enable'] = false gitlab_exporter['enable'] = false + gitlab_kas['enable'] = false nginx['enable'] = false # External URL diff --git a/doc/administration/reference_architectures/2k_users.md b/doc/administration/reference_architectures/2k_users.md index f6c484b08b1..6f6c02c309a 100644 --- a/doc/administration/reference_architectures/2k_users.md +++ b/doc/administration/reference_architectures/2k_users.md @@ -286,11 +286,6 @@ further configuration steps. ```ruby # Disable all components except PostgreSQL related ones roles(['postgres_role']) - prometheus['enable'] = false - alertmanager['enable'] = false - pgbouncer_exporter['enable'] = false - redis_exporter['enable'] = false - gitlab_exporter['enable'] = false # Set the network addresses that the exporters used for monitoring will listen on node_exporter['listen_address'] = '0.0.0.0:9100' @@ -365,19 +360,7 @@ Omnibus: ```ruby ## Enable Redis - redis['enable'] = true - - # Avoid running unnecessary services on the Redis server - gitaly['enable'] = false - postgresql['enable'] = false - puma['enable'] = false - sidekiq['enable'] = false - gitlab_workhorse['enable'] = false - prometheus['enable'] = false - alertmanager['enable'] = false - grafana['enable'] = false - gitlab_exporter['enable'] = false - nginx['enable'] = false + roles(["redis_master_role"]) redis['bind'] = '0.0.0.0' redis['port'] = 6379 @@ -481,6 +464,7 @@ To configure the Gitaly server, on the server node you want to use for Gitaly: alertmanager['enable'] = false grafana['enable'] = false gitlab_exporter['enable'] = false + gitlab_kas['enable'] = false nginx['enable'] = false # Prevent database migrations from running on upgrade automatically diff --git a/doc/administration/reference_architectures/3k_users.md b/doc/administration/reference_architectures/3k_users.md index 587303a1f8f..76f81e65580 100644 --- a/doc/administration/reference_architectures/3k_users.md +++ b/doc/administration/reference_architectures/3k_users.md @@ -1307,6 +1307,7 @@ To configure the Praefect nodes, on each one: alertmanager['enable'] = false grafana['enable'] = false gitlab_exporter['enable'] = false + gitlab_kas['enable'] = false nginx['enable'] = false # Praefect Configuration @@ -1447,6 +1448,7 @@ On each node: alertmanager['enable'] = false grafana['enable'] = false gitlab_exporter['enable'] = false + gitlab_kas['enable'] = false nginx['enable'] = false # Prevent database migrations from running on upgrade automatically @@ -1468,6 +1470,11 @@ On each node: # Gitaly Auth Token # Should be the same as praefect_internal_token gitaly['auth_token'] = '<praefect_internal_token>' + + # Gitaly Pack-objects cache + # Recommended to be enabled for improved performance but can notably increase disk I/O + # Refer to https://docs.gitlab.com/ee/administration/gitaly/configure_gitaly.html#pack-objects-cache for more info + gitaly['pack_objects_cache_enabled'] = true ``` 1. Append the following to `/etc/gitlab/gitlab.rb` for each respective server: @@ -1575,7 +1582,7 @@ To configure Praefect with TLS: ```ruby git_data_dirs({ "default" => { - "gitaly_address" => 'tls://LOAD_BALANCER_SERVER_ADDRESS:2305', + "gitaly_address" => 'tls://LOAD_BALANCER_SERVER_ADDRESS:3305', "gitaly_token" => 'PRAEFECT_EXTERNAL_TOKEN' } }) @@ -1621,6 +1628,7 @@ To configure the Sidekiq nodes, one each one: alertmanager['enable'] = false grafana['enable'] = false gitlab_exporter['enable'] = false + gitlab_kas['enable'] = false nginx['enable'] = false # External URL diff --git a/doc/administration/reference_architectures/50k_users.md b/doc/administration/reference_architectures/50k_users.md index f4bf232d548..dfa963d1ad0 100644 --- a/doc/administration/reference_architectures/50k_users.md +++ b/doc/administration/reference_architectures/50k_users.md @@ -1376,6 +1376,7 @@ To configure the Praefect nodes, on each one: alertmanager['enable'] = false grafana['enable'] = false gitlab_exporter['enable'] = false + gitlab_kas['enable'] = false nginx['enable'] = false # Praefect Configuration @@ -1516,6 +1517,7 @@ On each node: alertmanager['enable'] = false grafana['enable'] = false gitlab_exporter['enable'] = false + gitlab_kas['enable'] = false nginx['enable'] = false # Prevent database migrations from running on upgrade automatically @@ -1537,6 +1539,11 @@ On each node: # Gitaly Auth Token # Should be the same as praefect_internal_token gitaly['auth_token'] = '<praefect_internal_token>' + + # Gitaly Pack-objects cache + # Recommended to be enabled for improved performance but can notably increase disk I/O + # Refer to https://docs.gitlab.com/ee/administration/gitaly/configure_gitaly.html#pack-objects-cache for more info + gitaly['pack_objects_cache_enabled'] = true ``` 1. Append the following to `/etc/gitlab/gitlab.rb` for each respective server: @@ -1644,7 +1651,7 @@ To configure Praefect with TLS: ```ruby git_data_dirs({ "default" => { - "gitaly_address" => 'tls://LOAD_BALANCER_SERVER_ADDRESS:2305', + "gitaly_address" => 'tls://LOAD_BALANCER_SERVER_ADDRESS:3305', "gitaly_token" => 'PRAEFECT_EXTERNAL_TOKEN' } }) @@ -1688,6 +1695,7 @@ To configure the Sidekiq nodes, on each one: alertmanager['enable'] = false grafana['enable'] = false gitlab_exporter['enable'] = false + gitlab_kas['enable'] = false nginx['enable'] = false # External URL diff --git a/doc/administration/reference_architectures/5k_users.md b/doc/administration/reference_architectures/5k_users.md index 4014ec04904..f2463afbf3b 100644 --- a/doc/administration/reference_architectures/5k_users.md +++ b/doc/administration/reference_architectures/5k_users.md @@ -1305,6 +1305,7 @@ To configure the Praefect nodes, on each one: alertmanager['enable'] = false grafana['enable'] = false gitlab_exporter['enable'] = false + gitlab_kas['enable'] = false nginx['enable'] = false # Praefect Configuration @@ -1445,6 +1446,7 @@ On each node: alertmanager['enable'] = false grafana['enable'] = false gitlab_exporter['enable'] = false + gitlab_kas['enable'] = false nginx['enable'] = false # Prevent database migrations from running on upgrade automatically @@ -1466,6 +1468,11 @@ On each node: # Gitaly Auth Token # Should be the same as praefect_internal_token gitaly['auth_token'] = '<praefect_internal_token>' + + # Gitaly Pack-objects cache + # Recommended to be enabled for improved performance but can notably increase disk I/O + # Refer to https://docs.gitlab.com/ee/administration/gitaly/configure_gitaly.html#pack-objects-cache for more info + gitaly['pack_objects_cache_enabled'] = true ``` 1. Append the following to `/etc/gitlab/gitlab.rb` for each respective server: @@ -1573,7 +1580,7 @@ To configure Praefect with TLS: ```ruby git_data_dirs({ "default" => { - "gitaly_address" => 'tls://LOAD_BALANCER_SERVER_ADDRESS:2305', + "gitaly_address" => 'tls://LOAD_BALANCER_SERVER_ADDRESS:3305', "gitaly_token" => 'PRAEFECT_EXTERNAL_TOKEN' } }) @@ -1617,6 +1624,7 @@ To configure the Sidekiq nodes, one each one: alertmanager['enable'] = false grafana['enable'] = false gitlab_exporter['enable'] = false + gitlab_kas['enable'] = false nginx['enable'] = false # External URL diff --git a/doc/administration/reference_architectures/index.md b/doc/administration/reference_architectures/index.md index 815155866e8..3fcd6d7ae4e 100644 --- a/doc/administration/reference_architectures/index.md +++ b/doc/administration/reference_architectures/index.md @@ -103,12 +103,14 @@ The following table details the testing done against the reference architectures <style> table.test-coverage td { + border-top: 1px solid #dbdbdb; border-left: 1px solid #dbdbdb; border-right: 1px solid #dbdbdb; border-bottom: 1px solid #dbdbdb; } table.test-coverage th { + border-top: 1px solid #dbdbdb; border-left: 1px solid #dbdbdb; border-right: 1px solid #dbdbdb; border-bottom: 1px solid #dbdbdb; @@ -131,7 +133,6 @@ table.test-coverage th { <th scope="col">Omnibus</th> <th scope="col">Cloud Native Hybrid</th> <th scope="col">Omnibus</th> - <th scope="col">Cloud Native Hybrid</th> </tr> <tr> <th scope="row">1k</th> @@ -140,7 +141,6 @@ table.test-coverage th { <td></td> <td></td> <td></td> - <td></td> </tr> <tr> <th scope="row">2k</th> @@ -149,7 +149,6 @@ table.test-coverage th { <td></td> <td></td> <td></td> - <td></td> </tr> <tr> <th scope="row">3k</th> @@ -158,7 +157,6 @@ table.test-coverage th { <td></td> <td></td> <td></td> - <td></td> </tr> <tr> <th scope="row">5k</th> @@ -167,7 +165,6 @@ table.test-coverage th { <td></td> <td></td> <td></td> - <td></td> </tr> <tr> <th scope="row">10k</th> @@ -176,7 +173,6 @@ table.test-coverage th { <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k">Ad-Hoc (inc Cloud Services)</a></td> <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k-Cloud-Native-Hybrid">Ad-Hoc</a></td> <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/10k">Ad-Hoc</a></td> - <td></td> </tr> <tr> <th scope="row">25k</th> @@ -185,7 +181,6 @@ table.test-coverage th { <td></td> <td></td> <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/25k">Ad-Hoc</a></td> - <td></td> </tr> <tr> <th scope="row">50k</th> @@ -194,7 +189,6 @@ table.test-coverage th { <td><a href="https://gitlab.com/gitlab-org/quality/performance/-/wikis/Past-Results/50k">Ad-Hoc (inc Cloud Services)</a></td> <td></td> <td></td> - <td></td> </tr> </table> @@ -218,7 +212,6 @@ The Standard Reference Architectures are designed to be platform agnostic, with <th scope="col">Omnibus</th> <th scope="col">Cloud Native Hybrid</th> <th scope="col">Omnibus</th> - <th scope="col">Cloud Native Hybrid</th> </tr> <tr> <th scope="row">1k</th> @@ -227,7 +220,6 @@ The Standard Reference Architectures are designed to be platform agnostic, with <td></td> <td></td> <td></td> - <td></td> </tr> <tr> <th scope="row">2k</th> @@ -236,7 +228,6 @@ The Standard Reference Architectures are designed to be platform agnostic, with <td></td> <td></td> <td></td> - <td></td> </tr> <tr> <th scope="row">3k</th> @@ -245,7 +236,6 @@ The Standard Reference Architectures are designed to be platform agnostic, with <td></td> <td></td> <td></td> - <td></td> </tr> <tr> <th scope="row">5k</th> @@ -254,7 +244,6 @@ The Standard Reference Architectures are designed to be platform agnostic, with <td></td> <td></td> <td></td> - <td></td> </tr> <tr> <th scope="row">10k</th> @@ -263,7 +252,6 @@ The Standard Reference Architectures are designed to be platform agnostic, with <td></td> <td></td> <td></td> - <td></td> </tr> <tr> <th scope="row">25k</th> @@ -272,7 +260,6 @@ The Standard Reference Architectures are designed to be platform agnostic, with <td></td> <td></td> <td></td> - <td></td> </tr> <tr> <th scope="row">50k</th> @@ -281,10 +268,92 @@ The Standard Reference Architectures are designed to be platform agnostic, with <td></td> <td></td> <td></td> + </tr> +</table> + +### Recommended cloud providers and services + +NOTE: +The following lists are non exhaustive. Generally, other cloud providers not listed +here will likely work with the same specs, but this hasn't been validated. +Additionally, when it comes to other cloud provider services not listed here, +it's advised to be cautious as each implementation can be notably different +and should be tested thoroughly before production use. + +Through testing and real life usage, the Reference Architectures are validated and supported on the following cloud providers: + +<table> +<thead> + <tr> + <th>Reference Architecture</th> + <th>GCP</th> + <th>AWS</th> + <th>Azure</th> + <th>Bare Metal</th> + </tr> +</thead> +<tbody> + <tr> + <td>Omnibus</td> + <td>✅</td> + <td>✅</td> + <td>✅</td> + <td>✅</td> + </tr> + <tr> + <td>Cloud Native Hybrid</td> + <td>✅</td> + <td>✅</td> + <td></td> + <td></td> + </tr> +</tbody> +</table> + +Additionally, the following cloud provider services are validated and supported for use as part of the Reference Architectures: + +<table> +<thead> + <tr> + <th>Cloud Service</th> + <th>GCP</th> + <th>AWS</th> + <th>Bare Metal</th> + </tr> +</thead> +<tbody> + <tr> + <td>Object Storage</td> + <td>✅ <a href="https://cloud.google.com/storage" target="_blank">Cloud Storage</a></td> + <td>✅ <a href="https://aws.amazon.com/s3/" target="_blank">S3</a></td> + <td>✅ <a href="https://min.io/" target="_blank">MinIO</a></td> + </tr> + <tr> + <td>Database</td> + <td>✅ <a href="https://cloud.google.com/sql" target="_blank" rel="noopener noreferrer">Cloud SQL</a></td> + <td>✅ <a href="https://aws.amazon.com/rds/" target="_blank" rel="noopener noreferrer">RDS</a></td> + <td></td> + </tr> + <tr> + <td>Redis</td> + <td></td> + <td>✅ <a href="https://aws.amazon.com/elasticache/" target="_blank" rel="noopener noreferrer">Elasticache</a></td> <td></td> </tr> +</tbody> </table> +The following specific cloud provider services have been found to have issues in terms of either functionality or performance. As such, they either have caveats that should be considered or are not recommended: + +- [Azure Blob Storage](https://azure.microsoft.com/en-gb/services/storage/blobs/) has been found to have performance limits that can impact production use at certain times. For larger Reference Architectures the service may not be sufficient for production use and an alternative is recommended for use instead. +- [Azure Database for PostgreSQL Server](https://azure.microsoft.com/en-gb/services/postgresql/#overview) (Single / Flexible) is not recommended for use due to notable performance issues or missing functionality. +- [AWS Aurora Database](https://aws.amazon.com/rds/aurora) is not recommended due to compatibility issues. + +NOTE: +As a general rule we unfortunately don't recommend Azure Services at this time. +If required, we advise thorough testing is done at your intended scale +over a sustained period to validate if the service is suitable. + ## Availability Components GitLab comes with the following components for your use, listed from least to diff --git a/doc/administration/reply_by_email.md b/doc/administration/reply_by_email.md index 055f40ead64..13fa8450996 100644 --- a/doc/administration/reply_by_email.md +++ b/doc/administration/reply_by_email.md @@ -47,6 +47,8 @@ following headers, in this order: 1. `References` header 1. `Delivered-To` header 1. `Envelope-To` header +1. `X-Envelope-To` header +1. `Received` header If it finds a reply key, it leaves your reply as a comment on the entity the notification was about (issue, merge request, commit...). diff --git a/doc/administration/repository_checks.md b/doc/administration/repository_checks.md index 0591f5b3c40..6dd8faac65b 100644 --- a/doc/administration/repository_checks.md +++ b/doc/administration/repository_checks.md @@ -32,6 +32,12 @@ Instead of checking repositories manually, GitLab can be configured to run the c When enabled, GitLab periodically runs a repository check on all project repositories and wiki repositories to detect possible data corruption. A project is checked no more than once per month. +Administrators can configure the frequency of repository checks. To edit the frequency: + +- For Omnibus GitLab installations, edit `gitlab_rails['repository_check_worker_cron']` in + `/etc/gitlab/gitlab.rb`. +- For source-based installations, edit `[gitlab.cron_jobs.repository_check_worker]` in + `/home/git/gitlab/config/gitlab.yml`. If any projects fail their repository checks, all GitLab administrators receive an email notification of the situation. By default, this notification is sent out once a week at midnight at diff --git a/doc/administration/restart_gitlab.md b/doc/administration/restart_gitlab.md index 88909b2b9d8..44bfa153123 100644 --- a/doc/administration/restart_gitlab.md +++ b/doc/administration/restart_gitlab.md @@ -9,6 +9,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w Depending on how you installed GitLab, there are different methods to restart its services. +NOTE: +A short downtime is expected for all methods. + ## Omnibus installations If you have used the [Omnibus packages](https://about.gitlab.com/install/) to install GitLab, then @@ -90,8 +93,8 @@ application that powers Omnibus GitLab, makes sure that all things like director permissions, and services are in place and in the same shape that they were initially shipped. -It also restarts GitLab components where needed, if any of their -configuration files have changed. +It also [restarts GitLab components](#how-to-restart-gitlab) +where needed, if any of their configuration files have changed. If you manually edit any files in `/var/opt/gitlab` that are managed by Chef, running reconfigure reverts the changes AND restarts the services that diff --git a/doc/administration/sidekiq.md b/doc/administration/sidekiq.md index fea6ac9e554..b6c1c6a704e 100644 --- a/doc/administration/sidekiq.md +++ b/doc/administration/sidekiq.md @@ -217,6 +217,73 @@ To enable health checks for Sidekiq: sudo gitlab-ctl reconfigure ``` +## Configure LDAP and user or group synchronization + +If you use LDAP for user and group management, you must add the LDAP configuration to your Sidekiq node as well as the LDAP +synchronization worker. If the LDAP configuration and LDAP synchronization worker are not applied to your Sidekiq node, +users and groups are not automatically synchronized. + +For more information about configuring LDAP for GitLab, see: + +- [GitLab LDAP configuration documentation](auth/ldap/index.md#configure-ldap) +- [LDAP synchronization documentation](auth/ldap/ldap_synchronization.md#adjust-ldap-user-sync-schedule) + +To enable LDAP with the synchronization worker for Sidekiq: + +1. Edit `/etc/gitlab/gitlab.rb`: + + ```ruby + gitlab_rails['ldap_enabled'] = true + gitlab_rails['prevent_ldap_sign_in'] = false + gitlab_rails['ldap_servers'] = { + 'main' => { + 'label' => 'LDAP', + 'host' => 'ldap.mydomain.com', + 'port' => 389, + 'uid' => 'sAMAccountName', + 'encryption' => 'simple_tls', + 'verify_certificates' => true, + 'bind_dn' => '_the_full_dn_of_the_user_you_will_bind_with', + 'password' => '_the_password_of_the_bind_user', + 'tls_options' => { + 'ca_file' => '', + 'ssl_version' => '', + 'ciphers' => '', + 'cert' => '', + 'key' => '' + }, + 'timeout' => 10, + 'active_directory' => true, + 'allow_username_or_email_login' => false, + 'block_auto_created_users' => false, + 'base' => 'dc=example,dc=com', + 'user_filter' => '', + 'attributes' => { + 'username' => ['uid', 'userid', 'sAMAccountName'], + 'email' => ['mail', 'email', 'userPrincipalName'], + 'name' => 'cn', + 'first_name' => 'givenName', + 'last_name' => 'sn' + }, + 'lowercase_usernames' => false, + + # Enterprise Edition only + # https://docs.gitlab.com/ee/administration/auth/ldap/ldap_synchronization.html + 'group_base' => '', + 'admin_group' => '', + 'external_groups' => [], + 'sync_ssh_keys' => false + } + } + gitlab_rails['ldap_sync_worker_cron'] = "0 */12 * * *" + ``` + +1. Reconfigure GitLab: + + ```shell + sudo gitlab-ctl reconfigure + ``` + ## Related topics - [Extra Sidekiq processes](operations/extra_sidekiq_processes.md) diff --git a/doc/administration/terraform_state.md b/doc/administration/terraform_state.md index d1113125c4e..5e8f7cb18bb 100644 --- a/doc/administration/terraform_state.md +++ b/doc/administration/terraform_state.md @@ -141,10 +141,10 @@ You can optionally track progress and verify that all packages migrated successf - `sudo gitlab-rails dbconsole` for Omnibus GitLab instances. - `sudo -u git -H psql -d gitlabhq_production` for source-installed instances. -Verify `objectstg` below (where `store=2`) has count of all states: +Verify `objectstg` below (where `file_store=2`) has count of all states: ```shell -gitlabhq_production=# SELECT count(*) AS total, sum(case when store = '1' then 1 else 0 end) AS filesystem, sum(case when store = '2' then 1 else 0 end) AS objectstg FROM terraform_states; +gitlabhq_production=# SELECT count(*) AS total, sum(case when file_store = '1' then 1 else 0 end) AS filesystem, sum(case when file_store = '2' then 1 else 0 end) AS objectstg FROM terraform_states; total | filesystem | objectstg ------+------------+----------- diff --git a/doc/administration/troubleshooting/debug.md b/doc/administration/troubleshooting/debug.md index 744e12d4da1..0520ce470cb 100644 --- a/doc/administration/troubleshooting/debug.md +++ b/doc/administration/troubleshooting/debug.md @@ -225,7 +225,7 @@ gitlab_rails['env'] = { ``` For source installations, set the environment variable. -Refer to [Puma Worker timeout](../operations/puma.md#worker-timeout). +Refer to [Puma Worker timeout](../operations/puma.md#change-the-worker-timeout). [Reconfigure](../restart_gitlab.md#omnibus-gitlab-reconfigure) GitLab for the changes to take effect. diff --git a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md index 265c5278fd6..1c948771f5b 100644 --- a/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md +++ b/doc/administration/troubleshooting/gitlab_rails_cheat_sheet.md @@ -771,6 +771,26 @@ group.require_two_factor_authentication=false group.save ``` +### Check and toggle a feature for all projects in a group + +```ruby +projects = Group.find_by_name('_group_name').projects +projects.each do |p| + state = p.<feature-name>? + + if state + puts "#{p.name} has <feature-name> already enabled. Skipping..." + else + puts "#{p.name} didn't have <feature-name> enabled. Enabling..." + p.project_feature.update!(builds_access_level: ProjectFeature::PRIVATE) + end +end +``` + +To find features that can be toggled, run `pp p.project_feature`. +Available permission levels are listed in +[concerns/featurable.rb](https://gitlab.com/gitlab-org/gitlab/blob/master/app/models/concerns/featurable.rb). + ## Authentication ### Re-enable standard web sign-in form @@ -1023,7 +1043,7 @@ This is needed for example in a known edge-case with ### Remove licenses -To clean up the [License History table](../../user/admin_area/license.md#view-license-details-and-history): +To clean up the [License History table](../../user/admin_area/license_file.md#view-license-details-and-history): ```ruby TYPE = :trial? diff --git a/doc/administration/troubleshooting/img/AzureAD-scim_provisioning.png b/doc/administration/troubleshooting/img/AzureAD-scim_provisioning.png Binary files differindex b8edcfa31c2..b2c385151a6 100644 --- a/doc/administration/troubleshooting/img/AzureAD-scim_provisioning.png +++ b/doc/administration/troubleshooting/img/AzureAD-scim_provisioning.png diff --git a/doc/administration/troubleshooting/linux_cheat_sheet.md b/doc/administration/troubleshooting/linux_cheat_sheet.md index b66e88a8d60..913437c2d77 100644 --- a/doc/administration/troubleshooting/linux_cheat_sheet.md +++ b/doc/administration/troubleshooting/linux_cheat_sheet.md @@ -14,7 +14,7 @@ having an issue with GitLab, you may want to check your [support options](https: first, before attempting to use this information. WARNING: -If you administer GitLab you are expected to know these commands for your distribution +It is [beyond the scope of GitLab Support to assist in systems administration](https://about.gitlab.com/support/statement-of-support.html#training). GitLab administrators are expected to know these commands for their distribution of choice. If you are a GitLab Support Engineer, consider this a cross-reference to translate `yum` -> `apt-get` and the like. diff --git a/doc/administration/troubleshooting/postgresql.md b/doc/administration/troubleshooting/postgresql.md index 47fd424b1fd..d0ed3c5c12a 100644 --- a/doc/administration/troubleshooting/postgresql.md +++ b/doc/administration/troubleshooting/postgresql.md @@ -107,6 +107,10 @@ This section is for links to information elsewhere in the GitLab documentation. To fix this, see [Backup and restore a non-packaged PostgreSQL database](https://docs.gitlab.com/omnibus/settings/database.html#backup-and-restore-a-non-packaged-postgresql-database). +- Deploying PostgreSQL on Azure Database for PostgreSQL - Flexible Server may result in an error stating `extension "btree_gist" is not allow-listed for "azure_pg_admin" users in Azure Database for PostgreSQL` + + To resolve the above error, [allow-list the extension](https://docs.microsoft.com/en-us/azure/postgresql/flexible-server/concepts-extensions#how-to-use-postgresql-extensions) prior to install. + ## Support topics ### Database deadlocks diff --git a/doc/administration/uploads.md b/doc/administration/uploads.md index 55c3f85bfb9..aba7b5fc318 100644 --- a/doc/administration/uploads.md +++ b/doc/administration/uploads.md @@ -67,7 +67,7 @@ For source installations the following settings are nested under `uploads:` and |---------|-------------|---------| | `enabled` | Enable/disable object storage | `false` | | `remote_directory` | The bucket name where Uploads will be stored| | -| `direct_upload` | Set to `true` to remove Puma from the Upload path. Workhorse handles the actual Artifact Upload to Object Storage while Puma does minimal processing to keep track of the upload. There is no need for local shared storage. The option may be removed if support for a single storage type for all files is introduced. Read more on [direct upload](../development/uploads.md#direct-upload). | `false` | +| `direct_upload` | Set to `true` to remove Puma from the Upload path. Workhorse handles the actual Artifact Upload to Object Storage while Puma does minimal processing to keep track of the upload. There is no need for local shared storage. The option may be removed if support for a single storage type for all files is introduced. Read more on [direct upload](../development/uploads/implementation.md#direct-upload). | `false` | | `background_upload` | Set to `false` to disable automatic upload. Option may be removed once upload is direct to S3 (if `direct_upload` is set to `true` it will override `background_upload`) | `true` | | `proxy_download` | Set to `true` to enable proxying all files served. Option allows to reduce egress traffic as this allows clients to download directly from remote storage instead of proxying all data | `false` | | `connection` | Various connection options described below | | diff --git a/doc/api/alert_management_alerts.md b/doc/api/alert_management_alerts.md new file mode 100644 index 00000000000..2323cecfd6a --- /dev/null +++ b/doc/api/alert_management_alerts.md @@ -0,0 +1,129 @@ +--- +stage: Monitor +group: Respond +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Alert Management alerts API **(FREE)** + +The Alert Management alerts API is limited to metric images. For more API endpoints, see the +[GraphQL API](graphql/reference/index.md#alertmanagementalert). + +## Upload metric image + +```plaintext +POST /projects/:id/alert_management_alerts/:alert_iid/metric_images +``` + +| Attribute | Type | Required | Description | +|-------------|---------|----------|--------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `alert_iid` | integer | yes | The internal ID of a project's alert. | + +```shell +curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" --form 'file=@/path/to/file.png' \ +--form 'url=http://example.com' --form 'url_text=Example website' "https://gitlab.example.com/api/v4/projects/5/alert_management_alerts/93/metric_images" +``` + +Example response: + +```json +{ + "id": 17, + "created_at": "2020-11-12T20:07:58.156Z", + "filename": "sample_2054", + "file_path": "/uploads/-/system/alert_metric_image/file/17/sample_2054.png", + "url": "example.com/metric", + "url_text": "An example metric" +} +``` + +## List metric images + +```plaintext +GET /projects/:id/alert_management_alerts/:alert_iid/metric_images +``` + +| Attribute | Type | Required | Description | +|-------------|---------|----------|--------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `alert_iid` | integer | yes | The internal ID of a project's alert. | + +```shell +curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/alert_management_alerts/93/metric_images" +``` + +Example response: + +```json +[ + { + "id": 17, + "created_at": "2020-11-12T20:07:58.156Z", + "filename": "sample_2054", + "file_path": "/uploads/-/system/alert_metric_image/file/17/sample_2054.png", + "url": "example.com/metric", + "url_text": "An example metric" + }, + { + "id": 18, + "created_at": "2020-11-12T20:14:26.441Z", + "filename": "sample_2054", + "file_path": "/uploads/-/system/alert_metric_image/file/18/sample_2054.png", + "url": "example.com/metric", + "url_text": "An example metric" + } +] +``` + +## Update metric image + +```plaintext +PUT /projects/:id/alert_management_alerts/:alert_iid/metric_image/:image_id +``` + +| Attribute | Type | Required | Description | +|-------------|---------|----------|--------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `alert_iid` | integer | yes | The internal ID of a project's alert. | +| `image_id` | integer | yes | The ID of the image. | +| `url` | string | no | The URL to view more metrics information. | +| `url_text` | string | no | A description of the image or URL. | + +```shell +curl --header "PRIVATE-TOKEN: <your_access_token>" --request PUT --form 'url=http://example.com' --form 'url_text=Example website' "https://gitlab.example.com/api/v4/projects/5/alert_management_alerts/93/metric_images/1" +``` + +Example response: + +```json +{ + "id": 23, + "created_at": "2020-11-13T00:06:18.084Z", + "filename": "file.png", + "file_path": "/uploads/-/system/alert_metric_image/file/23/file.png", + "url": "http://example.com", + "url_text": "Example website" +} +``` + +## Delete metric image + +```plaintext +DELETE /projects/:id/alert_management_alerts/:alert_iid/metric_images/:image_id +``` + +| Attribute | Type | Required | Description | +|-------------|---------|----------|--------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `alert_iid` | integer | yes | The internal ID of a project's alert. | +| `image_id` | integer | yes | The ID of the image. | + +```shell +curl --header "PRIVATE-TOKEN: <your_access_token>" --request DELETE "https://gitlab.example.com/api/v4/projects/5/alert_management_alerts/93/metric_images/1" +``` + +Can return the following status codes: + +- `204 No Content`: if the image was deleted successfully. +- `422 Unprocessable`: if the image could not be deleted. diff --git a/doc/api/api_resources.md b/doc/api/api_resources.md index 3d54402ea0b..eabaa4217b5 100644 --- a/doc/api/api_resources.md +++ b/doc/api/api_resources.md @@ -118,6 +118,7 @@ The following API resources are available in the group context: | [Invitations](invitations.md) | `/groups/:id/invitations` (also available for projects) | | [Issues](issues.md) | `/groups/:id/issues` (also available for projects and standalone) | | [Issues Statistics](issues_statistics.md) | `/groups/:id/issues_statistics` (also available for projects and standalone) | +| [Linked epics](linked_epics.md) | `/groups/:id/epics/.../related_epics` | | [Members](members.md) | `/groups/:id/members` (also available for projects) | | [Merge requests](merge_requests.md) | `/groups/:id/merge_requests` (also available for projects and standalone) | | [Notes](notes.md) (comments) | `/groups/:id/epics/.../notes` (also available for projects) | diff --git a/doc/api/broadcast_messages.md b/doc/api/broadcast_messages.md index 5aca0667f31..a14de8dadd3 100644 --- a/doc/api/broadcast_messages.md +++ b/doc/api/broadcast_messages.md @@ -6,6 +6,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Broadcast Messages API **(FREE SELF)** +> 'target_access_levels' [introduced](https://gitlab.com/gitlab-org/growth/team-tasks/-/issues/461) in GitLab 14.8 [with a flag](../administration/feature_flags.md) named `role_targeted_broadcast_messages`. Disabled by default. + Broadcast messages API operates on [broadcast messages](../user/admin_area/broadcast_messages.md). As of GitLab 12.8, GET requests do not require authentication. All other broadcast message API endpoints are accessible only to administrators. Non-GET requests by: @@ -39,6 +41,7 @@ Example response: "font":"#FFFFFF", "id":1, "active": false, + "target_access_levels": [10,30], "target_path": "*/welcome", "broadcast_type": "banner", "dismissable": false @@ -77,6 +80,7 @@ Example response: "font":"#FFFFFF", "id":1, "active":false, + "target_access_levels": [10,30], "target_path": "*/welcome", "broadcast_type": "banner", "dismissable": false @@ -93,21 +97,31 @@ POST /broadcast_messages Parameters: -| Attribute | Type | Required | Description | -|:----------------|:---------|:---------|:------------------------------------------------------| -| `message` | string | yes | Message to display. | -| `starts_at` | datetime | no | Starting time (defaults to current time). Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) | -| `ends_at` | datetime | no | Ending time (defaults to one hour from current time). Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) | -| `color` | string | no | Background color hex code. | -| `font` | string | no | Foreground color hex code. | -| `target_path` | string | no | Target path of the broadcast message. | -| `broadcast_type`| string | no | Appearance type (defaults to banner) | -| `dismissable` | boolean | no | Can the user dismiss the message? | +| Attribute | Type | Required | Description | +|:-----------------------|:------------------|:---------|:------------------------------------------------------| +| `message` | string | yes | Message to display. | +| `starts_at` | datetime | no | Starting time (defaults to current time). Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) | +| `ends_at` | datetime | no | Ending time (defaults to one hour from current time). Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) | +| `color` | string | no | Background color hex code. | +| `font` | string | no | Foreground color hex code. | +| `target_access_levels` | array of integers | no | Target access levels (roles) of the broadcast message.| +| `target_path` | string | no | Target path of the broadcast message. | +| `broadcast_type` | string | no | Appearance type (defaults to banner) | +| `dismissable` | boolean | no | Can the user dismiss the message? | + +The `target_access_levels` are defined in the `Gitlab::Access` module. The +following levels are valid: + +- Guest (`10`) +- Reporter (`20`) +- Developer (`30`) +- Maintainer (`40`) +- Owner (`50`) Example request: ```shell -curl --data "message=Deploy in progress&color=#cecece" \ +curl --data "message=Deploy in progress&color=#cecece&target_access_levels[]=10&target_access_levels[]=30" \ --header "PRIVATE-TOKEN: <your_access_token>" \ "https://gitlab.example.com/api/v4/broadcast_messages" ``` @@ -123,6 +137,7 @@ Example response: "font":"#FFFFFF", "id":1, "active": true, + "target_access_levels": [10,30], "target_path": "*/welcome", "broadcast_type": "notification", "dismissable": false @@ -139,17 +154,27 @@ PUT /broadcast_messages/:id Parameters: -| Attribute | Type | Required | Description | -|:----------------|:---------|:---------|:--------------------------------------| -| `id` | integer | yes | ID of broadcast message to update. | -| `message` | string | no | Message to display. | -| `starts_at` | datetime | no | Starting time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) | -| `ends_at` | datetime | no | Ending time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) | -| `color` | string | no | Background color hex code. | -| `font` | string | no | Foreground color hex code. | -| `target_path` | string | no | Target path of the broadcast message. | -| `broadcast_type`| string | no | Appearance type (defaults to banner) | -| `dismissable` | boolean | no | Can the user dismiss the message? | +| Attribute | Type | Required | Description | +|:-----------------------|:------------------|:---------|:------------------------------------------------------| +| `id` | integer | yes | ID of broadcast message to update. | +| `message` | string | no | Message to display. | +| `starts_at` | datetime | no | Starting time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) | +| `ends_at` | datetime | no | Ending time. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) | +| `color` | string | no | Background color hex code. | +| `font` | string | no | Foreground color hex code. | +| `target_access_levels` | array of integers | no | Target access levels (roles) of the broadcast message.| +| `target_path` | string | no | Target path of the broadcast message. | +| `broadcast_type` | string | no | Appearance type (defaults to banner) | +| `dismissable` | boolean | no | Can the user dismiss the message? | + +The `target_access_levels` are defined in the `Gitlab::Access` module. The +following levels are valid: + +- Guest (`10`) +- Reporter (`20`) +- Developer (`30`) +- Maintainer (`40`) +- Owner (`50`) Example request: @@ -169,6 +194,7 @@ Example response: "font":"#FFFFFF", "id":1, "active": true, + "target_access_levels": [10,30], "target_path": "*/welcome", "broadcast_type": "notification", "dismissable": false diff --git a/doc/api/container_registry.md b/doc/api/container_registry.md index b61ec34b882..eb5a124c40c 100644 --- a/doc/api/container_registry.md +++ b/doc/api/container_registry.md @@ -210,10 +210,11 @@ GET /registry/repositories/:id | `id` | integer/string | yes | The ID of the registry repository accessible by the authenticated user. | | `tags` | boolean | no | If the parameter is included as `true`, the response includes an array of `"tags"`. | | `tags_count` | boolean | no | If the parameter is included as `true`, the response includes `"tags_count"`. | +| `size` | boolean | no | If the parameter is included as `true`, the response includes `"size"`. This is the deduplicated size of all images within the repository. Deduplication eliminates extra copies of identical data. For example, if you upload the same image twice, the Container Registry stores only one copy. This field is only available on GitLab.com for repositories created after `2021-11-04`. | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" \ - "https://gitlab.example.com/api/v4/registry/repositories/2?tags=true&tags_count=true" + "https://gitlab.example.com/api/v4/registry/repositories/2?tags=true&tags_count=true&size=true" ``` Example response: @@ -234,7 +235,8 @@ Example response: "path": "group/project:0.0.1", "location": "gitlab.example.com:5000/group/project:0.0.1" } - ] + ], + "size": 2818413 } ``` diff --git a/doc/api/dependencies.md b/doc/api/dependencies.md index 6fb99d11f4f..85ad6085fb2 100644 --- a/doc/api/dependencies.md +++ b/doc/api/dependencies.md @@ -7,7 +7,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Dependencies API **(ULTIMATE)** WARNING: -This API is in an alpha stage and considered unstable. +This API is in an [Alpha](../policy/alpha-beta-support.md#alpha-features) stage and considered unstable. The response payload may be subject to change or breakage across GitLab releases. diff --git a/doc/api/deploy_tokens.md b/doc/api/deploy_tokens.md index 77c2fe5e162..ace7e55f882 100644 --- a/doc/api/deploy_tokens.md +++ b/doc/api/deploy_tokens.md @@ -94,6 +94,46 @@ Example response: ] ``` +### Get a project deploy token + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/82467) in GitLab 14.9. + +Get a single project's deploy token by ID. + +```plaintext +GET /projects/:id/deploy_tokens/:token_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| ---------- | -------------- | ---------------------- | ----------- | +| `id` | integer/string | **{check-circle}** Yes | ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `token_id` | integer | **{check-circle}** Yes | ID of the deploy token | + +Example request: + +```shell +curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/deploy_tokens/1" +``` + +Example response: + +```json +{ + "id": 1, + "name": "MyToken", + "username": "gitlab+deploy-token-1", + "expires_at": "2020-02-14T00:00:00.000Z", + "revoked": false, + "expired": false, + "scopes": [ + "read_repository", + "read_registry" + ] +} +``` + ### Create a project deploy token > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/21811) in GitLab 12.9. @@ -108,7 +148,7 @@ Parameters: | Attribute | Type | Required | Description | | ------------ | ---------------- | ---------------------- | ----------- | -| `id` | integer/string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | **{check-circle}** Yes | ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `name` | string | **{check-circle}** Yes | New deploy token's name | | `expires_at` | datetime | **{dotted-circle}** No | Expiration date for the deploy token. Does not expire if no value is provided. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) | | `username` | string | **{dotted-circle}** No | Username for deploy token. Default is `gitlab+deploy-token-{n}` | @@ -153,8 +193,8 @@ Parameters: | Attribute | Type | Required | Description | | ---------- | -------------- | ---------------------- | ----------- | -| `id` | integer/string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | -| `token_id` | integer | **{check-circle}** Yes | The ID of the deploy token | +| `id` | integer/string | **{check-circle}** Yes | ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | +| `token_id` | integer | **{check-circle}** Yes | ID of the deploy token | Example request: @@ -210,6 +250,46 @@ Example response: ] ``` +### Get a group deploy token + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/82467) in GitLab 14.9. + +Get a single group's deploy token by ID. + +```plaintext +GET /groups/:id/deploy_tokens/:token_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +| ----------- | -------------- | ---------------------- | ----------- | +| `id` | integer/string | **{check-circle}** Yes | ID or [URL-encoded path of the group](index.md#namespaced-path-encoding) owned by the authenticated user | +| `token_id` | integer | **{check-circle}** Yes | ID of the deploy token | + +Example request: + +```shell +curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/1/deploy_tokens/1" +``` + +Example response: + +```json +{ + "id": 1, + "name": "MyToken", + "username": "gitlab+deploy-token-1", + "expires_at": "2020-02-14T00:00:00.000Z", + "revoked": false, + "expired": false, + "scopes": [ + "read_repository", + "read_registry" + ] +} +``` + ### Create a group deploy token > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/21811) in GitLab 12.9. @@ -224,7 +304,7 @@ Parameters: | Attribute | Type | Required | Description | | ------------ | ---- | --------- | ----------- | -| `id` | integer/string | **{check-circle}** Yes | The ID or [URL-encoded path of the group](index.md#namespaced-path-encoding) owned by the authenticated user | +| `id` | integer/string | **{check-circle}** Yes | ID or [URL-encoded path of the group](index.md#namespaced-path-encoding) owned by the authenticated user | | `name` | string | **{check-circle}** Yes | New deploy token's name | | `expires_at` | datetime | **{dotted-circle}** No | Expiration date for the deploy token. Does not expire if no value is provided. Expected in ISO 8601 format (`2019-03-15T08:00:00Z`) | | `username` | string | **{dotted-circle}** No | Username for deploy token. Default is `gitlab+deploy-token-{n}` | @@ -269,8 +349,8 @@ Parameters: | Attribute | Type | Required | Description | | ----------- | -------------- | ---------------------- | ----------- | -| `id` | integer/string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | -| `token_id` | integer | **{check-circle}** Yes | The ID of the deploy token | +| `id` | integer/string | **{check-circle}** Yes | ID or [URL-encoded path of the group](index.md#namespaced-path-encoding) owned by the authenticated user | +| `token_id` | integer | **{check-circle}** Yes | ID of the deploy token | Example request: diff --git a/doc/api/deployments.md b/doc/api/deployments.md index 29f9bd3e2ee..4a09f9a6605 100644 --- a/doc/api/deployments.md +++ b/doc/api/deployments.md @@ -281,7 +281,8 @@ Deployments created by users on GitLab Premium or higher include the `approvals` "avatar_url": "https://www.gravatar.com/avatar/e83ac685f68ea07553ad3054c738c709?s=80&d=identicon", "web_url": "http://localhost:3000/project_6_bot" }, - "status": "approved" + "status": "approved", + "created_at": "2022-02-24T20:22:30.097Z" } ], ... @@ -351,7 +352,8 @@ Deployments created by users on GitLab Premium or higher include the `approvals` "avatar_url": "https://www.gravatar.com/avatar/e83ac685f68ea07553ad3054c738c709?s=80&d=identicon", "web_url": "http://localhost:3000/project_6_bot" }, - "status": "approved" + "status": "approved", + "created_at": "2022-02-24T20:22:30.097Z" } ], ... @@ -417,7 +419,8 @@ Deployments created by users on GitLab Premium or higher include the `approvals` "avatar_url": "https://www.gravatar.com/avatar/e83ac685f68ea07553ad3054c738c709?s=80&d=identicon", "web_url": "http://localhost:3000/project_6_bot" }, - "status": "approved" + "status": "approved", + "created_at": "2022-02-24T20:22:30.097Z" } ], ... @@ -462,9 +465,10 @@ POST /projects/:id/deployments/:deployment_id/approval | `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | | `deployment_id` | integer | yes | The ID of the deployment. | | `status` | string | yes | The status of the approval (either `approved` or `rejected`). | +| `comment` | string | no | A comment to go with the approval | ```shell -curl --data "status=approved" \ +curl --data "status=approved&comment=Looks good to me" \ --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/deployments/1/approval" ``` @@ -480,6 +484,8 @@ Example response: "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", "web_url": "http://localhost:3000/root" }, - "status": "approved" + "status": "approved", + "created_at": "2022-02-24T20:22:30.097Z", + "comment":"Looks good to me" } ``` diff --git a/doc/api/dora/metrics.md b/doc/api/dora/metrics.md index 3f078945b0f..afc29f03598 100644 --- a/doc/api/dora/metrics.md +++ b/doc/api/dora/metrics.md @@ -9,6 +9,7 @@ type: reference, api > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/279039) in GitLab 13.10. > - The legacy key/value pair `{ "<date>" => "<value>" }` was removed from the payload in GitLab 14.0. +> `time_to_restore_service` metric was introduced in GitLab 14.9. All methods require at least the Reporter role. @@ -20,14 +21,14 @@ Get project-level DORA metrics. GET /projects/:id/dora/metrics ``` -| Attribute | Type | Required | Description | -|-------------- |-------- |----------|----------------------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](../index.md#namespaced-path-encoding) can be accessed by the authenticated user. | -| `metric` | string | yes | The [metric name](../../user/analytics/ci_cd_analytics.md#supported-metrics-in-gitlab). One of `deployment_frequency` or `lead_time_for_changes`. | -| `start_date` | string | no | Date range to start from. ISO 8601 Date format, for example `2021-03-01`. Default is 3 months ago. | -| `end_date` | string | no | Date range to end at. ISO 8601 Date format, for example `2021-03-01`. Default is the current date. | -| `interval` | string | no | The bucketing interval. One of `all`, `monthly` or `daily`. Default is `daily`. | -| `environment_tier` | string | no | The [tier of the environment](../../ci/environments/index.md#deployment-tier-of-environments). Default is `production`. | +| Attribute | Type | Required | Description | +|-------------- |-------- |----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](../index.md#namespaced-path-encoding) can be accessed by the authenticated user. | +| `metric` | string | yes | The [metric name](../../user/analytics/ci_cd_analytics.md#supported-metrics-in-gitlab). One of `deployment_frequency`, `lead_time_for_changes` or `time_to_restore_service`.| +| `start_date` | string | no | Date range to start from. ISO 8601 Date format, for example `2021-03-01`. Default is 3 months ago. | +| `end_date` | string | no | Date range to end at. ISO 8601 Date format, for example `2021-03-01`. Default is the current date. | +| `interval` | string | no | The bucketing interval. One of `all`, `monthly` or `daily`. Default is `daily`. | +| `environment_tier` | string | no | The [tier of the environment](../../ci/environments/index.md#deployment-tier-of-environments). Default is `production`. | Example request: @@ -63,7 +64,7 @@ GET /groups/:id/dora/metrics | Attribute | Type | Required | Description | |-------------- |-------- |----------|----------------------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](../index.md#namespaced-path-encoding) can be accessed by the authenticated user. | -| `metric` | string | yes | The [metric name](../../user/analytics/ci_cd_analytics.md#supported-metrics-in-gitlab). One of `deployment_frequency` or `lead_time_for_changes`. | +| `metric` | string | yes | The [metric name](../../user/analytics/ci_cd_analytics.md#supported-metrics-in-gitlab). One of `deployment_frequency`, `lead_time_for_changes` or `time_to_restore_service`. | | `start_date` | string | no | Date range to start from. ISO 8601 Date format, for example `2021-03-01`. Default is 3 months ago. | | `end_date` | string | no | Date range to end at. ISO 8601 Date format, for example `2021-03-01`. Default is the current date. | | `interval` | string | no | The bucketing interval. One of `all`, `monthly` or `daily`. Default is `daily`. | @@ -97,6 +98,7 @@ API response has a different meaning depending on the provided `metric` query parameter: | `metric` query parameter | Description of `value` in response | -| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| ------------------------ |--------------------------------------------------------------------------------------------------------------------------------------------------------------| | `deployment_frequency` | The number of successful deployments during the time period. | | `lead_time_for_changes` | The median number of seconds between the merge of the merge request (MR) and the deployment of the MR's commits for all MRs deployed during the time period. | +| `time_to_restore_service` | The median number of seconds an incident was open during the time period. Available only for production environment | diff --git a/doc/api/environments.md b/doc/api/environments.md index 8188e0e7b85..40d161485ff 100644 --- a/doc/api/environments.md +++ b/doc/api/environments.md @@ -35,7 +35,90 @@ Example response: "name": "review/fix-foo", "slug": "review-fix-foo-dfjre3", "external_url": "https://review-fix-foo-dfjre3.gitlab.example.com", - "state": "available" + "state": "available", + "created_at": "2019-05-25T18:55:13.252Z", + "updated_at": "2019-05-27T18:55:13.252Z", + "enable_advanced_logs_querying": false, + "logs_api_path": "/project/-/logs/k8s.json?environment_name=review%2Ffix-foo", + "last_deployment": { + "id": 100, + "iid": 34, + "ref": "fdroid", + "sha": "416d8ea11849050d3d1f5104cf8cf51053e790ab", + "created_at": "2019-03-25T18:55:13.252Z", + "status": "success", + "user": { + "id": 1, + "name": "Administrator", + "state": "active", + "username": "root", + "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", + "web_url": "http://localhost:3000/root" + }, + "deployable": { + "id": 710, + "status": "success", + "stage": "deploy", + "name": "staging", + "ref": "fdroid", + "tag": false, + "coverage": null, + "created_at": "2019-03-25T18:55:13.215Z", + "started_at": "2019-03-25T12:54:50.082Z", + "finished_at": "2019-03-25T18:55:13.216Z", + "duration": 21623.13423, + "user": { + "id": 1, + "name": "Administrator", + "username": "root", + "state": "active", + "avatar_url": "http://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", + "web_url": "http://gitlab.dev/root", + "created_at": "2015-12-21T13:14:24.077Z", + "bio": null, + "location": null, + "public_email": "", + "skype": "", + "linkedin": "", + "twitter": "", + "website_url": "", + "organization": null + }, + "commit": { + "id": "416d8ea11849050d3d1f5104cf8cf51053e790ab", + "short_id": "416d8ea1", + "created_at": "2016-01-02T15:39:18.000Z", + "parent_ids": [ + "e9a4449c95c64358840902508fc827f1a2eab7df" + ], + "title": "Removed fabric to fix #40", + "message": "Removed fabric to fix #40\n", + "author_name": "Administrator", + "author_email": "admin@example.com", + "authored_date": "2016-01-02T15:39:18.000Z", + "committer_name": "Administrator", + "committer_email": "admin@example.com", + "committed_date": "2016-01-02T15:39:18.000Z" + }, + "pipeline": { + "id": 34, + "sha": "416d8ea11849050d3d1f5104cf8cf51053e790ab", + "ref": "fdroid", + "status": "success", + "web_url": "http://localhost:3000/Commit451/lab-coat/pipelines/34" + }, + "web_url": "http://localhost:3000/Commit451/lab-coat/-/jobs/710", + "artifacts": [ + { + "file_type": "trace", + "size": 1305, + "filename": "job.log", + "file_format": null + } + ], + "runner": null, + "artifacts_expire_at": null + } } ] ``` @@ -66,6 +149,8 @@ Example of response "state": "available", "created_at": "2019-05-25T18:55:13.252Z", "updated_at": "2019-05-27T18:55:13.252Z", + "enable_advanced_logs_querying": false, + "logs_api_path": "/project/-/logs/k8s.json?environment_name=review%2Ffix-foo", "last_deployment": { "id": 100, "iid": 34, diff --git a/doc/api/epics.md b/doc/api/epics.md index deb74cf21e9..de20af08915 100644 --- a/doc/api/epics.md +++ b/doc/api/epics.md @@ -131,6 +131,7 @@ Example response: "labels": [], "upvotes": 4, "downvotes": 0, + "color": "#1068bf", "_links":{ "self": "http://gitlab.example.com/api/v4/groups/7/epics/4", "epic_issues": "http://gitlab.example.com/api/v4/groups/7/epics/4/issues", @@ -179,6 +180,7 @@ Example response: "labels": [], "upvotes": 4, "downvotes": 0, + "color": "#1068bf", "_links":{ "self": "http://gitlab.example.com/api/v4/groups/17/epics/35", "epic_issues": "http://gitlab.example.com/api/v4/groups/17/epics/35/issues", @@ -252,6 +254,7 @@ Example response: "labels": [], "upvotes": 4, "downvotes": 0, + "color": "#1068bf", "subscribed": true, "_links":{ "self": "http://gitlab.example.com/api/v4/groups/7/epics/5", @@ -283,6 +286,7 @@ POST /groups/:id/epics | `title` | string | yes | The title of the epic | | `labels` | string | no | The comma-separated list of labels | | `description` | string | no | The description of the epic. Limited to 1,048,576 characters. | +| `color` | string | no | The color of the epic. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7641) in GitLab 14.8, behind a feature flag named `epic_highlight_color` (disabled by default) | | `confidential` | boolean | no | Whether the epic should be confidential | | `created_at` | string | no | When the epic was created. Date time string, ISO 8601 formatted, for example `2016-03-11T03:45:40Z` . Requires administrator or project/group owner privileges ([available](https://gitlab.com/gitlab-org/gitlab/-/issues/255309) in GitLab 13.5 and later) | | `start_date_is_fixed` | boolean | no | Whether start date should be sourced from `start_date_fixed` or from milestones (in GitLab 11.3 and later) | @@ -340,6 +344,7 @@ Example response: "labels": [], "upvotes": 4, "downvotes": 0, + "color": "#1068bf", "_links":{ "self": "http://gitlab.example.com/api/v4/groups/7/epics/6", "epic_issues": "http://gitlab.example.com/api/v4/groups/7/epics/6/issues", @@ -381,6 +386,7 @@ PUT /groups/:id/epics/:epic_iid | `state_event` | string | no | State event for an epic. Set `close` to close the epic and `reopen` to reopen it (in GitLab 11.4 and later) | | `title` | string | no | The title of an epic | | `updated_at` | string | no | When the epic was updated. Date time string, ISO 8601 formatted, for example `2016-03-11T03:45:40Z` . Requires administrator or project/group owner privileges ([available](https://gitlab.com/gitlab-org/gitlab/-/issues/255309) in GitLab 13.5 and later) | +| `color` | string | no | The color of the epic. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7641) in GitLab 14.8, behind a feature flag named `epic_highlight_color` (disabled by default) | ```shell curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/1/epics/5?title=New%20Title&parent_id=29" @@ -430,7 +436,8 @@ Example response: "closed_at": "2018-08-18T12:22:05.239Z", "labels": [], "upvotes": 4, - "downvotes": 0 + "downvotes": 0, + "color": "#1068bf" } ``` diff --git a/doc/api/features.md b/doc/api/features.md index 30efacb1d00..593e4adedd7 100644 --- a/doc/api/features.md +++ b/doc/api/features.md @@ -95,7 +95,7 @@ Example response: "milestone": "11.8", "type": "ops", "group": "group::ecosystem", - "default_enabled": false + "default_enabled": true }, { "name": "marginalia", diff --git a/doc/api/geo_nodes.md b/doc/api/geo_nodes.md index a152c443902..d2cca1a5856 100644 --- a/doc/api/geo_nodes.md +++ b/doc/api/geo_nodes.md @@ -63,7 +63,8 @@ Example response: "sync_object_storage": false, "clone_protocol": "http", "web_edit_url": "https://primary.example.com/admin/geo/sites/3/edit", - "web_geo_projects_url": "http://secondary.example.com/admin/geo/projects", + "web_geo_projects_url": "https://secondary.example.com/admin/geo/projects", + "web_geo_replication_details_url": "https://secondary.example.com/admin/geo/sites/3/replication/lfs_objects", "_links": { "self": "https://primary.example.com/api/v4/geo_nodes/3", "status": "https://primary.example.com/api/v4/geo_nodes/3/status", @@ -72,6 +73,10 @@ Example response: } ``` +WARNING: +The `web_geo_projects_url` attribute is in its end-of-life process. It is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/80106) +for use in GitLab 14.9. + ## Retrieve configuration about all Geo nodes ```plaintext @@ -130,6 +135,7 @@ Example response: "clone_protocol": "http", "web_edit_url": "https://primary.example.com/admin/geo/sites/2/edit", "web_geo_projects_url": "https://secondary.example.com/admin/geo/projects", + "web_geo_replication_details_url": "https://secondary.example.com/admin/geo/sites/2/replication/lfs_objects", "_links": { "self":"https://primary.example.com/api/v4/geo_nodes/2", "status":"https://primary.example.com/api/v4/geo_nodes/2/status", @@ -139,6 +145,10 @@ Example response: ] ``` +WARNING: +The `web_geo_projects_url` attribute is in its end-of-life process. It is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/80106) +for use in GitLab 14.9. + ## Retrieve configuration about a specific Geo node ```plaintext @@ -228,6 +238,7 @@ Example response: "clone_protocol": "http", "web_edit_url": "https://primary.example.com/admin/geo/sites/2/edit", "web_geo_projects_url": "https://secondary.example.com/admin/geo/projects", + "web_geo_replication_details_url": "https://secondary.example.com/admin/geo/sites/2/replication/lfs_objects", "_links": { "self":"https://primary.example.com/api/v4/geo_nodes/2", "status":"https://primary.example.com/api/v4/geo_nodes/2/status", @@ -236,6 +247,10 @@ Example response: } ``` +WARNING: +The `web_geo_projects_url` attribute is in its end-of-life process. It is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/80106) +for use in GitLab 14.9. + ## Delete a Geo node Removes the Geo node. diff --git a/doc/api/graphql/index.md b/doc/api/graphql/index.md index d0e65b8c4eb..457b083c217 100644 --- a/doc/api/graphql/index.md +++ b/doc/api/graphql/index.md @@ -187,55 +187,74 @@ NOTE: The complexity limits may be revised in future, and additionally, the complexity of a query may be altered. -## Spam - -GraphQL mutations can be detected as spam. If this happens, a -[GraphQL top-level error](https://spec.graphql.org/June2018/#sec-Errors) is raised. For example: - -```json -{ - "errors": [ - { - "message": "Request denied. Spam detected", - "locations": [ { "line": 6, "column": 7 } ], - "path": [ "updateSnippet" ], - "extensions": { - "spam": true +## Resolve mutations detected as spam + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327360) in GitLab 13.11. + +GraphQL mutations can be detected as spam. If a mutation is detected as spam and: + +- A CAPTCHA service is not configured, a + [GraphQL top-level error](https://spec.graphql.org/June2018/#sec-Errors) is raised. For example: + + ```json + { + "errors": [ + { + "message": "Request denied. Spam detected", + "locations": [ { "line": 6, "column": 7 } ], + "path": [ "updateSnippet" ], + "extensions": { + "spam": true + } + } + ], + "data": { + "updateSnippet": { + "snippet": null } } - ], - "data": { - "updateSnippet": { - "snippet": null + } + ``` + +- A CAPTCHA service is configured, you receive a response with: + - `needsCaptchaResponse` set to `true`. + - The `spamLogId` and `captchaSiteKey` fields set. + + For example: + + ```json + { + "errors": [ + { + "message": "Request denied. Solve CAPTCHA challenge and retry", + "locations": [ { "line": 6, "column": 7 } ], + "path": [ "updateSnippet" ], + "extensions": { + "needsCaptchaResponse": true, + "captchaSiteKey": "6LeIxAcTAAAAAJcZVRqyHh71UMIEGNQ_MXjiZKhI", + "spamLogId": 67 + } + } + ], + "data": { + "updateSnippet": { + "snippet": null, + } } } -} -``` - -If a mutation is detected as potential spam and a CAPTCHA service is configured: + ``` - Use the `captchaSiteKey` to obtain a CAPTCHA response value using the appropriate CAPTCHA API. Only [Google reCAPTCHA v2](https://developers.google.com/recaptcha/docs/display) is supported. - Resubmit the request with the `X-GitLab-Captcha-Response` and `X-GitLab-Spam-Log-Id` headers set. - -```json -{ - "errors": [ - { - "message": "Request denied. Solve CAPTCHA challenge and retry", - "locations": [ { "line": 6, "column": 7 } ], - "path": [ "updateSnippet" ], - "extensions": { - "needsCaptchaResponse": true, - "captchaSiteKey": "6LeIxAcTAAAAAJcZVRqyHh71UMIEGNQ_MXjiZKhI", - "spamLogId": 67 - } - } - ], - "data": { - "updateSnippet": { - "snippet": null, - } - } -} + +NOTE: +The GitLab GraphiQL implementation doesn't permit passing of headers, so we must write +this as a cURL query. `--data-binary` is used to properly handle escaped double quotes +in the JSON-embedded query. + +```shell +export CAPTCHA_RESPONSE="<CAPTCHA response obtained from CAPTCHA service>" +export SPAM_LOG_ID="<spam_log_id obtained from initial REST response>" +curl --header "Authorization: Bearer $PRIVATE_TOKEN" --header "Content-Type: application/json" --header "X-GitLab-Captcha-Response: $CAPTCHA_RESPONSE" --header "X-GitLab-Spam-Log-Id: $SPAM_LOG_ID" --request POST --data-binary '{"query": "mutation {createSnippet(input: {title: \"Title\" visibilityLevel: public blobActions: [ { action: create filePath: \"BlobPath\" content: \"BlobContent\" } ] }) { snippet { id title } errors }}"}' "https://gitlab.example.com/api/graphql" ``` diff --git a/doc/api/graphql/reference/index.md b/doc/api/graphql/reference/index.md index 49c0361123e..f6a6504df70 100644 --- a/doc/api/graphql/reference/index.md +++ b/doc/api/graphql/reference/index.md @@ -1,7 +1,7 @@ --- stage: Ecosystem group: Integrations -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- <!--- @@ -157,6 +157,12 @@ Returns [`GeoNode`](#geonode). | ---- | ---- | ----------- | | <a id="querygeonodename"></a>`name` | [`String`](#string) | Name of the Geo node. Defaults to the current Geo node name. | +### `Query.gitpodEnabled` + +Whether Gitpod is enabled in application settings. + +Returns [`Boolean`](#boolean). + ### `Query.group` Find a group. @@ -560,6 +566,18 @@ Returns [`Vulnerability`](#vulnerability). | ---- | ---- | ----------- | | <a id="queryvulnerabilityid"></a>`id` | [`VulnerabilityID!`](#vulnerabilityid) | Global ID of the Vulnerability. | +### `Query.workItem` + +Find a work item. Returns `null` if `work_items` feature flag is disabled. The feature is experimental and is subject to change without notice. + +Returns [`WorkItem`](#workitem). + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="queryworkitemid"></a>`id` | [`WorkItemID!`](#workitemid) | Global ID of the work item. | + ## `Mutation` type The `Mutation` type contains all the mutations you can execute. @@ -1143,8 +1161,6 @@ Input type: `ConfigureSecretDetectionInput` ### `Mutation.corpusCreate` -Available only when feature flag `corpus_management` is enabled. This flag is enabled by default. - Input type: `CorpusCreateInput` #### Arguments @@ -1354,6 +1370,7 @@ Input type: `CreateEpicInput` | ---- | ---- | ----------- | | <a id="mutationcreateepicaddlabelids"></a>`addLabelIds` | [`[ID!]`](#id) | IDs of labels to be added to the epic. | | <a id="mutationcreateepicclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationcreateepiccolor"></a>`color` | [`String`](#string) | Color of the epic. Available only when feature flag `epic_color_highlight` is enabled. This flag is disabled by default, because the feature is experimental and is subject to change without notice. | | <a id="mutationcreateepicconfidential"></a>`confidential` | [`Boolean`](#boolean) | Indicates if the epic is confidential. | | <a id="mutationcreateepicdescription"></a>`description` | [`String`](#string) | Description of the epic. | | <a id="mutationcreateepicduedatefixed"></a>`dueDateFixed` | [`String`](#string) | End date of the epic. | @@ -1466,6 +1483,11 @@ Input type: `CreateIterationInput` ### `Mutation.createNote` +Creates a Note. +If the body of the Note contains only quick actions, +the Note will be destroyed during an update, and no Note will be +returned. + Input type: `CreateNoteInput` #### Arguments @@ -1476,6 +1498,7 @@ Input type: `CreateNoteInput` | <a id="mutationcreatenoteclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | | <a id="mutationcreatenoteconfidential"></a>`confidential` | [`Boolean`](#boolean) | Confidentiality flag of a note. Default is false. | | <a id="mutationcreatenotediscussionid"></a>`discussionId` | [`DiscussionID`](#discussionid) | Global ID of the discussion this note is in reply to. | +| <a id="mutationcreatenotemergerequestdiffheadsha"></a>`mergeRequestDiffHeadSha` | [`String`](#string) | SHA of the head commit which is used to ensure that the merge request has not been updated since the request was sent. | | <a id="mutationcreatenotenoteableid"></a>`noteableId` | [`NoteableID!`](#noteableid) | Global ID of the resource to add a note to. | #### Fields @@ -1845,6 +1868,7 @@ Input type: `DastSiteProfileCreateInput` | <a id="mutationdastsiteprofilecreatefullpath"></a>`fullPath` | [`ID!`](#id) | Project the site profile belongs to. | | <a id="mutationdastsiteprofilecreateprofilename"></a>`profileName` | [`String!`](#string) | Name of the site profile. | | <a id="mutationdastsiteprofilecreaterequestheaders"></a>`requestHeaders` | [`String`](#string) | Comma-separated list of request header names and values to be added to every request made by DAST. | +| <a id="mutationdastsiteprofilecreatescanmethod"></a>`scanMethod` | [`DastScanMethodType`](#dastscanmethodtype) | Scan method by the scanner. Is not saved or updated if `dast_api_scanner` feature flag is disabled. | | <a id="mutationdastsiteprofilecreatetargettype"></a>`targetType` | [`DastTargetTypeEnum`](#dasttargettypeenum) | Type of target to be scanned. | | <a id="mutationdastsiteprofilecreatetargeturl"></a>`targetUrl` | [`String`](#string) | URL of the target to be scanned. | @@ -1890,6 +1914,7 @@ Input type: `DastSiteProfileUpdateInput` | <a id="mutationdastsiteprofileupdateid"></a>`id` | [`DastSiteProfileID!`](#dastsiteprofileid) | ID of the site profile to be updated. | | <a id="mutationdastsiteprofileupdateprofilename"></a>`profileName` | [`String!`](#string) | Name of the site profile. | | <a id="mutationdastsiteprofileupdaterequestheaders"></a>`requestHeaders` | [`String`](#string) | Comma-separated list of request header names and values to be added to every request made by DAST. | +| <a id="mutationdastsiteprofileupdatescanmethod"></a>`scanMethod` | [`DastScanMethodType`](#dastscanmethodtype) | Scan method by the scanner. Is not saved or updated if `dast_api_scanner` feature flag is disabled. | | <a id="mutationdastsiteprofileupdatetargettype"></a>`targetType` | [`DastTargetTypeEnum`](#dasttargettypeenum) | Type of target to be scanned. | | <a id="mutationdastsiteprofileupdatetargeturl"></a>`targetUrl` | [`String`](#string) | URL of the target to be scanned. | @@ -4218,6 +4243,47 @@ Input type: `RunnersRegistrationTokenResetInput` | <a id="mutationrunnersregistrationtokenreseterrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | | <a id="mutationrunnersregistrationtokenresettoken"></a>`token` | [`String`](#string) | Runner token after mutation. | +### `Mutation.savedReplyCreate` + +Input type: `SavedReplyCreateInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationsavedreplycreateclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationsavedreplycreatecontent"></a>`content` | [`String!`](#string) | Content of the saved reply. | +| <a id="mutationsavedreplycreatename"></a>`name` | [`String!`](#string) | Name of the saved reply. | + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationsavedreplycreateclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationsavedreplycreateerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | +| <a id="mutationsavedreplycreatesavedreply"></a>`savedReply` | [`SavedReply`](#savedreply) | Updated saved reply. | + +### `Mutation.savedReplyUpdate` + +Input type: `SavedReplyUpdateInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationsavedreplyupdateclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationsavedreplyupdatecontent"></a>`content` | [`String!`](#string) | Content of the saved reply. | +| <a id="mutationsavedreplyupdateid"></a>`id` | [`UsersSavedReplyID!`](#userssavedreplyid) | Global ID of the saved reply. | +| <a id="mutationsavedreplyupdatename"></a>`name` | [`String!`](#string) | Name of the saved reply. | + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationsavedreplyupdateclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationsavedreplyupdateerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | +| <a id="mutationsavedreplyupdatesavedreply"></a>`savedReply` | [`SavedReply`](#savedreply) | Updated saved reply. | + ### `Mutation.scanExecutionPolicyCommit` Commits the `policy_yaml` content to the assigned security policy project for the given project(`project_path`). @@ -4420,6 +4486,25 @@ Input type: `TimelineEventDestroyInput` | <a id="mutationtimelineeventdestroyerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | | <a id="mutationtimelineeventdestroytimelineevent"></a>`timelineEvent` | [`TimelineEventType`](#timelineeventtype) | Timeline event. | +### `Mutation.timelineEventPromoteFromNote` + +Input type: `TimelineEventPromoteFromNoteInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationtimelineeventpromotefromnoteclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationtimelineeventpromotefromnotenoteid"></a>`noteId` | [`NoteID!`](#noteid) | Note ID from which the timeline event promoted. | + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationtimelineeventpromotefromnoteclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationtimelineeventpromotefromnoteerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | +| <a id="mutationtimelineeventpromotefromnotetimelineevent"></a>`timelineEvent` | [`TimelineEventType`](#timelineeventtype) | Timeline event. | + ### `Mutation.timelineEventUpdate` Input type: `TimelineEventUpdateInput` @@ -4725,6 +4810,7 @@ Input type: `UpdateEpicInput` | ---- | ---- | ----------- | | <a id="mutationupdateepicaddlabelids"></a>`addLabelIds` | [`[ID!]`](#id) | IDs of labels to be added to the epic. | | <a id="mutationupdateepicclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationupdateepiccolor"></a>`color` | [`String`](#string) | Color of the epic. Available only when feature flag `epic_color_highlight` is enabled. This flag is disabled by default, because the feature is experimental and is subject to change without notice. | | <a id="mutationupdateepicconfidential"></a>`confidential` | [`Boolean`](#boolean) | Indicates if the epic is confidential. | | <a id="mutationupdateepicdescription"></a>`description` | [`String`](#string) | Description of the epic. | | <a id="mutationupdateepicduedatefixed"></a>`dueDateFixed` | [`String`](#string) | End date of the epic. | @@ -4770,7 +4856,7 @@ Input type: `UpdateEpicBoardListInput` Updates a DiffNote on an image (a `Note` where the `position.positionType` is `"image"`). If the body of the Note contains only quick actions, -the Note will be destroyed during the update, and no Note will be +the Note will be destroyed during an update, and no Note will be returned. Input type: `UpdateImageDiffNoteInput` @@ -4877,7 +4963,7 @@ Input type: `UpdateNamespacePackageSettingsInput` Updates a Note. If the body of the Note contains only quick actions, -the Note will be destroyed during the update, and no Note will be +the Note will be destroyed during an update, and no Note will be returned. Input type: `UpdateNoteInput` @@ -5178,6 +5264,29 @@ Input type: `WorkItemCreateInput` | <a id="mutationworkitemcreateerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | | <a id="mutationworkitemcreateworkitem"></a>`workItem` | [`WorkItem`](#workitem) | Created work item. | +### `Mutation.workItemCreateFromTask` + +Creates a work item from a task in another work item's description. Available only when feature flag `work_items` is enabled. This feature is experimental and is subject to change without notice. + +Input type: `WorkItemCreateFromTaskInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationworkitemcreatefromtaskclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationworkitemcreatefromtaskid"></a>`id` | [`WorkItemID!`](#workitemid) | Global ID of the work item. | +| <a id="mutationworkitemcreatefromtaskworkitemdata"></a>`workItemData` | [`WorkItemConvertTaskInput!`](#workitemconverttaskinput) | Arguments necessary to convert a task into a work item. | + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mutationworkitemcreatefromtaskclientmutationid"></a>`clientMutationId` | [`String`](#string) | A unique identifier for the client performing the mutation. | +| <a id="mutationworkitemcreatefromtaskerrors"></a>`errors` | [`[String!]!`](#string) | Errors encountered during execution of the mutation. | +| <a id="mutationworkitemcreatefromtasknewworkitem"></a>`newWorkItem` | [`WorkItem`](#workitem) | New work item created from task. | +| <a id="mutationworkitemcreatefromtaskworkitem"></a>`workItem` | [`WorkItem`](#workitem) | Updated work item. | + ### `Mutation.workItemDelete` Deletes a work item. Available only when feature flag `work_items` is enabled. The feature is experimental and is subject to change without notice. @@ -5721,6 +5830,7 @@ The edge type for [`CiRunner`](#cirunner). | Name | Type | Description | | ---- | ---- | ----------- | | <a id="cirunneredgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | +| <a id="cirunneredgeediturl"></a>`editUrl` | [`String`](#string) | Web URL of the runner edit page. The value depends on where you put this field in the query. You can use it for projects or groups. | | <a id="cirunneredgenode"></a>`node` | [`CiRunner`](#cirunner) | The item at the end of the edge. | | <a id="cirunneredgeweburl"></a>`webUrl` | [`String`](#string) | Web URL of the runner. The value depends on where you put this field in the query. You can use it for projects or groups. | @@ -5912,6 +6022,29 @@ The edge type for [`ComplianceFramework`](#complianceframework). | <a id="complianceframeworkedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | | <a id="complianceframeworkedgenode"></a>`node` | [`ComplianceFramework`](#complianceframework) | The item at the end of the edge. | +#### `ComplianceViolationConnection` + +The connection type for [`ComplianceViolation`](#complianceviolation). + +##### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="complianceviolationconnectionedges"></a>`edges` | [`[ComplianceViolationEdge]`](#complianceviolationedge) | A list of edges. | +| <a id="complianceviolationconnectionnodes"></a>`nodes` | [`[ComplianceViolation]`](#complianceviolation) | A list of nodes. | +| <a id="complianceviolationconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. | + +#### `ComplianceViolationEdge` + +The edge type for [`ComplianceViolation`](#complianceviolation). + +##### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="complianceviolationedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | +| <a id="complianceviolationedgenode"></a>`node` | [`ComplianceViolation`](#complianceviolation) | The item at the end of the edge. | + #### `ConnectedAgentConnection` The connection type for [`ConnectedAgent`](#connectedagent). @@ -7001,6 +7134,29 @@ The edge type for [`MergeRequest`](#mergerequest). | <a id="mergerequestedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | | <a id="mergerequestedgenode"></a>`node` | [`MergeRequest`](#mergerequest) | The item at the end of the edge. | +#### `MergeRequestParticipantConnection` + +The connection type for [`MergeRequestParticipant`](#mergerequestparticipant). + +##### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergerequestparticipantconnectionedges"></a>`edges` | [`[MergeRequestParticipantEdge]`](#mergerequestparticipantedge) | A list of edges. | +| <a id="mergerequestparticipantconnectionnodes"></a>`nodes` | [`[MergeRequestParticipant]`](#mergerequestparticipant) | A list of nodes. | +| <a id="mergerequestparticipantconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. | + +#### `MergeRequestParticipantEdge` + +The edge type for [`MergeRequestParticipant`](#mergerequestparticipant). + +##### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergerequestparticipantedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | +| <a id="mergerequestparticipantedgenode"></a>`node` | [`MergeRequestParticipant`](#mergerequestparticipant) | The item at the end of the edge. | + #### `MergeRequestReviewerConnection` The connection type for [`MergeRequestReviewer`](#mergerequestreviewer). @@ -7694,6 +7850,29 @@ The edge type for [`SastCiConfigurationOptionsEntity`](#sastciconfigurationoptio | <a id="sastciconfigurationoptionsentityedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | | <a id="sastciconfigurationoptionsentityedgenode"></a>`node` | [`SastCiConfigurationOptionsEntity`](#sastciconfigurationoptionsentity) | The item at the end of the edge. | +#### `SavedReplyConnection` + +The connection type for [`SavedReply`](#savedreply). + +##### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="savedreplyconnectionedges"></a>`edges` | [`[SavedReplyEdge]`](#savedreplyedge) | A list of edges. | +| <a id="savedreplyconnectionnodes"></a>`nodes` | [`[SavedReply]`](#savedreply) | A list of nodes. | +| <a id="savedreplyconnectionpageinfo"></a>`pageInfo` | [`PageInfo!`](#pageinfo) | Information to aid in pagination. | + +#### `SavedReplyEdge` + +The edge type for [`SavedReply`](#savedreply). + +##### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="savedreplyedgecursor"></a>`cursor` | [`String!`](#string) | A cursor for use in pagination. | +| <a id="savedreplyedgenode"></a>`node` | [`SavedReply`](#savedreply) | The item at the end of the edge. | + #### `ScanConnection` The connection type for [`Scan`](#scan). @@ -8513,6 +8692,7 @@ Describes an alert from the project's Alert Management. | <a id="alertmanagementalertstatus"></a>`status` | [`AlertManagementStatus`](#alertmanagementstatus) | Status of the alert. | | <a id="alertmanagementalerttitle"></a>`title` | [`String`](#string) | Title of the alert. | | <a id="alertmanagementalertupdatedat"></a>`updatedAt` | [`Time`](#time) | Timestamp the alert was last updated. | +| <a id="alertmanagementalertweburl"></a>`webUrl` | [`String!`](#string) | URL of the alert. | #### Fields with arguments @@ -8691,6 +8871,7 @@ An emoji awarded by a user. | Name | Type | Description | | ---- | ---- | ----------- | | <a id="baseserviceactive"></a>`active` | [`Boolean`](#boolean) | Indicates if the service is active. | +| <a id="baseserviceservicetype"></a>`serviceType` | [`ServiceType`](#servicetype) | Type of the service. | | <a id="baseservicetype"></a>`type` | [`String`](#string) | Class name of the service. | ### `Blob` @@ -8795,6 +8976,7 @@ Represents an epic on an issue board. | <a id="boardepicauthor"></a>`author` | [`UserCore!`](#usercore) | Author of the epic. | | <a id="boardepicawardemoji"></a>`awardEmoji` | [`AwardEmojiConnection`](#awardemojiconnection) | List of award emojis associated with the epic. (see [Connections](#connections)) | | <a id="boardepicclosedat"></a>`closedAt` | [`Time`](#time) | Timestamp of when the epic was closed. | +| <a id="boardepiccolor"></a>`color` | [`String!`](#string) | Color of the epic. Available only when feature flag `epic_color_highlight` is enabled. This flag is disabled by default, because the feature is experimental and is subject to change without notice. | | <a id="boardepicconfidential"></a>`confidential` | [`Boolean`](#boolean) | Indicates if the epic is confidential. | | <a id="boardepiccreatedat"></a>`createdAt` | [`Time`](#time) | Timestamp of when the epic was created. | | <a id="boardepicdescendantcounts"></a>`descendantCounts` | [`EpicDescendantCount`](#epicdescendantcount) | Number of open and closed descendant epics and issues. | @@ -8830,6 +9012,7 @@ Represents an epic on an issue board. | <a id="boardepicstartdateisfixed"></a>`startDateIsFixed` | [`Boolean`](#boolean) | Indicates if the start date has been manually set. | | <a id="boardepicstate"></a>`state` | [`EpicState!`](#epicstate) | State of the epic. | | <a id="boardepicsubscribed"></a>`subscribed` | [`Boolean!`](#boolean) | Indicates the currently logged in user is subscribed to the epic. | +| <a id="boardepictextcolor"></a>`textColor` | [`String!`](#string) | Text color generated for the epic. Available only when feature flag `epic_color_highlight` is enabled. This flag is disabled by default, because the feature is experimental and is subject to change without notice. | | <a id="boardepictitle"></a>`title` | [`String`](#string) | Title of the epic. | | <a id="boardepictitlehtml"></a>`titleHtml` | [`String`](#string) | The GitLab Flavored Markdown rendering of `title`. | | <a id="boardepicupdatedat"></a>`updatedAt` | [`Time`](#time) | Timestamp of when the epic was updated. | @@ -9429,6 +9612,9 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="commitpipelinessha"></a>`sha` | [`String`](#string) | Filter pipelines by the sha of the commit they are run for. | | <a id="commitpipelinessource"></a>`source` | [`String`](#string) | Filter pipelines by their source. | | <a id="commitpipelinesstatus"></a>`status` | [`PipelineStatusEnum`](#pipelinestatusenum) | Filter pipelines by their status. | +| <a id="commitpipelinesupdatedafter"></a>`updatedAfter` | [`Time`](#time) | Pipelines updated after this date. | +| <a id="commitpipelinesupdatedbefore"></a>`updatedBefore` | [`Time`](#time) | Pipelines updated before this date. | +| <a id="commitpipelinesusername"></a>`username` | [`String`](#string) | Filter pipelines by the user that triggered the pipeline. | ### `ComplianceFramework` @@ -9444,6 +9630,20 @@ Represents a ComplianceFramework associated with a Project. | <a id="complianceframeworkname"></a>`name` | [`String!`](#string) | Name of the compliance framework. | | <a id="complianceframeworkpipelineconfigurationfullpath"></a>`pipelineConfigurationFullPath` | [`String`](#string) | Full path of the compliance pipeline configuration stored in a project repository, such as `.gitlab/.compliance-gitlab-ci.yml@compliance/hipaa` **(ULTIMATE)**. | +### `ComplianceViolation` + +Compliance violation associated with a merged merge request. + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="complianceviolationid"></a>`id` | [`ID!`](#id) | Compliance violation ID. | +| <a id="complianceviolationmergerequest"></a>`mergeRequest` | [`MergeRequest!`](#mergerequest) | Merge request the compliance violation occurred in. | +| <a id="complianceviolationreason"></a>`reason` | [`ComplianceViolationReason!`](#complianceviolationreason) | Reason the compliance violation occurred. | +| <a id="complianceviolationseveritylevel"></a>`severityLevel` | [`ComplianceViolationSeverity!`](#complianceviolationseverity) | Severity of the compliance violation. | +| <a id="complianceviolationviolatinguser"></a>`violatingUser` | [`UserCore!`](#usercore) | User suspected of causing the compliance violation. | + ### `ComposerMetadata` Composer metadata. @@ -9555,6 +9755,7 @@ Details of a container repository. | <a id="containerrepositorydetailsname"></a>`name` | [`String!`](#string) | Name of the container repository. | | <a id="containerrepositorydetailspath"></a>`path` | [`String!`](#string) | Path of the container repository. | | <a id="containerrepositorydetailsproject"></a>`project` | [`Project!`](#project) | Project of the container registry. | +| <a id="containerrepositorydetailssize"></a>`size` | [`Float`](#float) | Deduplicated size of the image repository in bytes. This is only available on GitLab.com for repositories created after `2021-11-04`. | | <a id="containerrepositorydetailsstatus"></a>`status` | [`ContainerRepositoryStatus`](#containerrepositorystatus) | Status of the container repository. | | <a id="containerrepositorydetailstagscount"></a>`tagsCount` | [`Int!`](#int) | Number of tags associated with this image. | | <a id="containerrepositorydetailsupdatedat"></a>`updatedAt` | [`Time!`](#time) | Timestamp when the container repository was updated. | @@ -9622,7 +9823,7 @@ Represents the current license. | <a id="currentlicensecreatedat"></a>`createdAt` | [`Date`](#date) | Date when the license was added. | | <a id="currentlicenseemail"></a>`email` | [`String`](#string) | Email of the licensee. | | <a id="currentlicenseexpiresat"></a>`expiresAt` | [`Date`](#date) | Date when the license expires. | -| <a id="currentlicenseid"></a>`id` | [`ID!`](#id) | ID of the license. | +| <a id="currentlicenseid"></a>`id` | [`ID!`](#id) | ID of the license extracted from the license data. | | <a id="currentlicenselastsync"></a>`lastSync` | [`Time`](#time) | Date when the license was last synced. | | <a id="currentlicensemaximumusercount"></a>`maximumUserCount` | [`Int`](#int) | Highest number of billable users on the system during the term of the current license. | | <a id="currentlicensename"></a>`name` | [`String`](#string) | Name of the licensee. | @@ -9763,6 +9964,7 @@ Represents a DAST Site Profile. | <a id="dastsiteprofileprofilename"></a>`profileName` | [`String`](#string) | Name of the site profile. | | <a id="dastsiteprofilereferencedinsecuritypolicies"></a>`referencedInSecurityPolicies` | [`[String!]`](#string) | List of security policy names that are referencing given project. | | <a id="dastsiteprofilerequestheaders"></a>`requestHeaders` | [`String`](#string) | Comma-separated list of request header names and values to be added to every request made by DAST. | +| <a id="dastsiteprofilescanmethod"></a>`scanMethod` | [`DastScanMethodType`](#dastscanmethodtype) | Scan method used by the scanner. Always returns `null` if `dast_api_scanner` feature flag is disabled. | | <a id="dastsiteprofiletargettype"></a>`targetType` | [`DastTargetTypeEnum`](#dasttargettypeenum) | Type of target to be scanned. | | <a id="dastsiteprofiletargeturl"></a>`targetUrl` | [`String`](#string) | URL of the target to be scanned. | | <a id="dastsiteprofileuserpermissions"></a>`userPermissions` | [`DastSiteProfilePermissions!`](#dastsiteprofilepermissions) | Permissions for the current user on the resource. | @@ -9889,6 +10091,7 @@ A single design. | <a id="designnotes"></a>`notes` | [`NoteConnection!`](#noteconnection) | All notes on this noteable. (see [Connections](#connections)) | | <a id="designnotescount"></a>`notesCount` | [`Int!`](#int) | Total count of user-created notes for this design. | | <a id="designproject"></a>`project` | [`Project!`](#project) | Project the design belongs to. | +| <a id="designweburl"></a>`webUrl` | [`String!`](#string) | URL of the design. | #### Fields with arguments @@ -10322,6 +10525,7 @@ Represents an epic. | <a id="epicauthor"></a>`author` | [`UserCore!`](#usercore) | Author of the epic. | | <a id="epicawardemoji"></a>`awardEmoji` | [`AwardEmojiConnection`](#awardemojiconnection) | List of award emojis associated with the epic. (see [Connections](#connections)) | | <a id="epicclosedat"></a>`closedAt` | [`Time`](#time) | Timestamp of when the epic was closed. | +| <a id="epiccolor"></a>`color` | [`String!`](#string) | Color of the epic. Available only when feature flag `epic_color_highlight` is enabled. This flag is disabled by default, because the feature is experimental and is subject to change without notice. | | <a id="epicconfidential"></a>`confidential` | [`Boolean`](#boolean) | Indicates if the epic is confidential. | | <a id="epiccreatedat"></a>`createdAt` | [`Time`](#time) | Timestamp of when the epic was created. | | <a id="epicdescendantcounts"></a>`descendantCounts` | [`EpicDescendantCount`](#epicdescendantcount) | Number of open and closed descendant epics and issues. | @@ -10357,6 +10561,7 @@ Represents an epic. | <a id="epicstartdateisfixed"></a>`startDateIsFixed` | [`Boolean`](#boolean) | Indicates if the start date has been manually set. | | <a id="epicstate"></a>`state` | [`EpicState!`](#epicstate) | State of the epic. | | <a id="epicsubscribed"></a>`subscribed` | [`Boolean!`](#boolean) | Indicates the currently logged in user is subscribed to the epic. | +| <a id="epictextcolor"></a>`textColor` | [`String!`](#string) | Text color generated for the epic. Available only when feature flag `epic_color_highlight` is enabled. This flag is disabled by default, because the feature is experimental and is subject to change without notice. | | <a id="epictitle"></a>`title` | [`String`](#string) | Title of the epic. | | <a id="epictitlehtml"></a>`titleHtml` | [`String`](#string) | The GitLab Flavored Markdown rendering of `title`. | | <a id="epicupdatedat"></a>`updatedAt` | [`Time`](#time) | Timestamp of when the epic was updated. | @@ -10653,10 +10858,11 @@ Represents an epic board list. | Name | Type | Description | | ---- | ---- | ----------- | | <a id="epiclistcollapsed"></a>`collapsed` | [`Boolean`](#boolean) | Indicates if this list is collapsed for this user. | -| <a id="epiclistepicscount"></a>`epicsCount` | [`Int`](#int) | Count of epics in the list. | +| <a id="epiclistepicscount"></a>`epicsCount` **{warning-solid}** | [`Int`](#int) | **Deprecated** in 14.9. This was renamed. Use: `metadata`. | | <a id="epiclistid"></a>`id` | [`BoardsEpicListID!`](#boardsepiclistid) | Global ID of the board list. | | <a id="epiclistlabel"></a>`label` | [`Label`](#label) | Label of the list. | | <a id="epiclistlisttype"></a>`listType` | [`String!`](#string) | Type of the list. | +| <a id="epiclistmetadata"></a>`metadata` | [`EpicListMetadata`](#epiclistmetadata) | Epic list metatada. | | <a id="epiclistposition"></a>`position` | [`Int`](#int) | Position of the list within the board. | | <a id="epiclisttitle"></a>`title` | [`String!`](#string) | Title of the list. | @@ -10678,6 +10884,17 @@ four standard [pagination arguments](#connection-pagination-arguments): | ---- | ---- | ----------- | | <a id="epiclistepicsfilters"></a>`filters` | [`EpicFilters`](#epicfilters) | Filters applied when selecting epics in the board list. | +### `EpicListMetadata` + +Represents epic board list metadata. + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="epiclistmetadataepicscount"></a>`epicsCount` | [`Int`](#int) | Count of epics in the list. | +| <a id="epiclistmetadatatotalweight"></a>`totalWeight` | [`Int`](#int) | Total weight of all issues in the list. Available only when feature flag `epic_board_total_weight` is enabled. This flag is disabled by default, because the feature is experimental and is subject to change without notice. | + ### `EpicPermissions` Check permissions for the current user on an epic. @@ -10807,7 +11024,7 @@ four standard [pagination arguments](#connection-pagination-arguments): ##### `GeoNode.jobArtifactRegistries` -Find Job Artifact registries on this Geo node Available only when feature flag `geo_job_artifact_replication` is enabled. This flag is disabled by default, because the feature is experimental and is subject to change without notice. +Find Job Artifact registries on this Geo node. Returns [`JobArtifactRegistryConnection`](#jobartifactregistryconnection). @@ -10971,10 +11188,10 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="groupadditionalpurchasedstoragesize"></a>`additionalPurchasedStorageSize` | [`Float`](#float) | Additional storage purchased for the root namespace in bytes. | | <a id="groupautodevopsenabled"></a>`autoDevopsEnabled` | [`Boolean`](#boolean) | Indicates whether Auto DevOps is enabled for all projects within this group. | | <a id="groupavatarurl"></a>`avatarUrl` | [`String`](#string) | Avatar URL of the group. | -| <a id="groupbillablememberscount"></a>`billableMembersCount` | [`Int`](#int) | Number of billable users in the group. | | <a id="groupcontacts"></a>`contacts` | [`CustomerRelationsContactConnection`](#customerrelationscontactconnection) | Find contacts of this group. (see [Connections](#connections)) | | <a id="groupcontainerrepositoriescount"></a>`containerRepositoriesCount` | [`Int!`](#int) | Number of container repositories in the group. | | <a id="groupcontainslockedprojects"></a>`containsLockedProjects` | [`Boolean!`](#boolean) | Includes at least one project where the repository size exceeds the limit. | +| <a id="groupcrossprojectpipelineavailable"></a>`crossProjectPipelineAvailable` | [`Boolean!`](#boolean) | Indicates if the cross_project_pipeline feature is available for the namespace. | | <a id="groupcustomemoji"></a>`customEmoji` | [`CustomEmojiConnection`](#customemojiconnection) | Custom emoji within this namespace. Available only when feature flag `custom_emoji` is enabled. This flag is disabled by default, because the feature is experimental and is subject to change without notice. (see [Connections](#connections)) | | <a id="groupdependencyproxyblobcount"></a>`dependencyProxyBlobCount` | [`Int!`](#int) | Number of dependency proxy blobs cached in the group. | | <a id="groupdependencyproxyblobs"></a>`dependencyProxyBlobs` | [`DependencyProxyBlobConnection`](#dependencyproxyblobconnection) | Dependency Proxy blobs. (see [Connections](#connections)) | @@ -11021,10 +11238,21 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="groupvisibility"></a>`visibility` | [`String`](#string) | Visibility of the namespace. | | <a id="groupvulnerabilityscanners"></a>`vulnerabilityScanners` | [`VulnerabilityScannerConnection`](#vulnerabilityscannerconnection) | Vulnerability scanners reported on the project vulnerabilities of the group and its subgroups. (see [Connections](#connections)) | | <a id="groupweburl"></a>`webUrl` | [`String!`](#string) | Web URL of the group. | -| <a id="groupworkitemtypes"></a>`workItemTypes` | [`WorkItemTypeConnection`](#workitemtypeconnection) | Work item types available to the group. Available only when feature flag `work_items` is enabled. This flag is disabled by default, because the feature is experimental and is subject to change without notice. (see [Connections](#connections)) | #### Fields with arguments +##### `Group.billableMembersCount` + +Number of billable users in the group. + +Returns [`Int`](#int). + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="groupbillablememberscountrequestedhostedplan"></a>`requestedHostedPlan` | [`String`](#string) | Plan from which to get billable members. | + ##### `Group.board` A single board of the group. @@ -11245,6 +11473,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="groupissuesepicid"></a>`epicId` | [`String`](#string) | ID of an epic associated with the issues, "none" and "any" values are supported. | | <a id="groupissuesiid"></a>`iid` | [`String`](#string) | IID of the issue. For example, "1". | | <a id="groupissuesiids"></a>`iids` | [`[String!]`](#string) | List of IIDs of issues. For example, `["1", "2"]`. | +| <a id="groupissuesincludearchived"></a>`includeArchived` | [`Boolean`](#boolean) | Return issues from archived projects. | | <a id="groupissuesincludesubepics"></a>`includeSubepics` | [`Boolean`](#boolean) | Whether to include subepics when filtering issues by epicId. | | <a id="groupissuesincludesubgroups"></a>`includeSubgroups` | [`Boolean`](#boolean) | Include issues belonging to subgroups. | | <a id="groupissuesiterationid"></a>`iterationId` | [`[ID]`](#id) | List of iteration Global IDs applied to the issue. | @@ -11341,6 +11570,23 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="grouplabelsonlygrouplabels"></a>`onlyGroupLabels` | [`Boolean`](#boolean) | Include only group level labels. | | <a id="grouplabelssearchterm"></a>`searchTerm` | [`String`](#string) | Search term to find labels with. | +##### `Group.mergeRequestViolations` + +Compliance violations reported on merge requests merged within the group. + +Returns [`ComplianceViolationConnection`](#complianceviolationconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="groupmergerequestviolationsfilters"></a>`filters` | [`ComplianceViolationInput`](#complianceviolationinput) | Filters applied when retrieving compliance violations. | +| <a id="groupmergerequestviolationssort"></a>`sort` | [`ComplianceViolationSort`](#complianceviolationsort) | List compliance violations by sort order. | + ##### `Group.mergeRequests` Merge requests for projects in this group. @@ -11361,6 +11607,7 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="groupmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | | <a id="groupmergerequestsdraft"></a>`draft` | [`Boolean`](#boolean) | Limit result to draft merge requests. | | <a id="groupmergerequestsiids"></a>`iids` | [`[String!]`](#string) | Array of IIDs of merge requests, for example `[1, 2]`. | +| <a id="groupmergerequestsincludearchived"></a>`includeArchived` | [`Boolean`](#boolean) | Return merge requests from archived projects. | | <a id="groupmergerequestsincludesubgroups"></a>`includeSubgroups` | [`Boolean`](#boolean) | Include merge requests belonging to subgroups. | | <a id="groupmergerequestslabels"></a>`labels` | [`[String!]`](#string) | Array of label names. All resolved merge requests will have all of these labels. | | <a id="groupmergerequestsmergedafter"></a>`mergedAfter` | [`Time`](#time) | Merge requests merged after this date. | @@ -11561,6 +11808,22 @@ Returns [`VulnerabilitySeveritiesCount`](#vulnerabilityseveritiescount). | <a id="groupvulnerabilityseveritiescountseverity"></a>`severity` | [`[VulnerabilitySeverity!]`](#vulnerabilityseverity) | Filter vulnerabilities by severity. | | <a id="groupvulnerabilityseveritiescountstate"></a>`state` | [`[VulnerabilityState!]`](#vulnerabilitystate) | Filter vulnerabilities by state. | +##### `Group.workItemTypes` + +Work item types available to the group. Returns `null` if `work_items` feature flag is disabled. This flag is disabled by default, because the feature is experimental and is subject to change without notice. + +Returns [`WorkItemTypeConnection`](#workitemtypeconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="groupworkitemtypestaskable"></a>`taskable` | [`Boolean`](#boolean) | If `true`, only taskable work item types will be returned. Argument is experimental and can be removed in the future without notice. | + ### `GroupMember` Represents a Group Membership. @@ -11575,6 +11838,7 @@ Represents a Group Membership. | <a id="groupmemberexpiresat"></a>`expiresAt` | [`Time`](#time) | Date and time the membership expires. | | <a id="groupmembergroup"></a>`group` | [`Group`](#group) | Group that a User is a member of. | | <a id="groupmemberid"></a>`id` | [`ID!`](#id) | ID of the member. | +| <a id="groupmembernotificationemail"></a>`notificationEmail` | [`String`](#string) | Group notification email for User. Only availble for admins. | | <a id="groupmemberupdatedat"></a>`updatedAt` | [`Time`](#time) | Date and time the membership was last updated. | | <a id="groupmemberuser"></a>`user` | [`UserCore`](#usercore) | User that is associated with the member object. | | <a id="groupmemberuserpermissions"></a>`userPermissions` | [`GroupPermissions!`](#grouppermissions) | Permissions for the current user on the resource. | @@ -11918,17 +12182,30 @@ Represents an iteration object. | <a id="iterationid"></a>`id` | [`ID!`](#id) | ID of the iteration. | | <a id="iterationiid"></a>`iid` | [`ID!`](#id) | Internal ID of the iteration. | | <a id="iterationiterationcadence"></a>`iterationCadence` | [`IterationCadence!`](#iterationcadence) | Cadence of the iteration. | -| <a id="iterationreport"></a>`report` | [`TimeboxReport`](#timeboxreport) | Historically accurate report about the timebox. | | <a id="iterationscopedpath"></a>`scopedPath` | [`String`](#string) | Web path of the iteration, scoped to the query parent. Only valid for Project parents. Returns null in other contexts. | | <a id="iterationscopedurl"></a>`scopedUrl` | [`String`](#string) | Web URL of the iteration, scoped to the query parent. Only valid for Project parents. Returns null in other contexts. | | <a id="iterationsequence"></a>`sequence` | [`Int!`](#int) | Sequence number for the iteration when you sort the containing cadence's iterations by the start and end date. The earliest starting and ending iteration is assigned 1. | | <a id="iterationstartdate"></a>`startDate` | [`Time`](#time) | Timestamp of the iteration start date. | | <a id="iterationstate"></a>`state` | [`IterationState!`](#iterationstate) | State of the iteration. | -| <a id="iterationtitle"></a>`title` | [`String!`](#string) | Title of the iteration. | +| <a id="iterationtitle"></a>`title` | [`String`](#string) | Title of the iteration. Title must be specified unless iteration_cadences feature flag is enabled. | | <a id="iterationupdatedat"></a>`updatedAt` | [`Time!`](#time) | Timestamp of last iteration update. | | <a id="iterationwebpath"></a>`webPath` | [`String!`](#string) | Web path of the iteration. | | <a id="iterationweburl"></a>`webUrl` | [`String!`](#string) | Web URL of the iteration. | +#### Fields with arguments + +##### `Iteration.report` + +Historically accurate report about the timebox. + +Returns [`TimeboxReport`](#timeboxreport). + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="iterationreportfullpath"></a>`fullPath` | [`String`](#string) | Full path of the project or group used as a scope for report. For example, `gitlab-org` or `gitlab-org/gitlab`. | + ### `IterationCadence` Represents an iteration cadence. @@ -11978,6 +12255,7 @@ Represents an iteration cadence. | Name | Type | Description | | ---- | ---- | ----------- | | <a id="jiraserviceactive"></a>`active` | [`Boolean`](#boolean) | Indicates if the service is active. | +| <a id="jiraserviceservicetype"></a>`serviceType` | [`ServiceType`](#servicetype) | Type of the service. | | <a id="jiraservicetype"></a>`type` | [`String`](#string) | Class name of the service. | #### Fields with arguments @@ -12094,7 +12372,7 @@ Represents an entry from the Cloud License history. | <a id="licensehistoryentrycreatedat"></a>`createdAt` | [`Date`](#date) | Date when the license was added. | | <a id="licensehistoryentryemail"></a>`email` | [`String`](#string) | Email of the licensee. | | <a id="licensehistoryentryexpiresat"></a>`expiresAt` | [`Date`](#date) | Date when the license expires. | -| <a id="licensehistoryentryid"></a>`id` | [`ID!`](#id) | ID of the license. | +| <a id="licensehistoryentryid"></a>`id` | [`ID!`](#id) | ID of the license extracted from the license data. | | <a id="licensehistoryentryname"></a>`name` | [`String`](#string) | Name of the licensee. | | <a id="licensehistoryentryplan"></a>`plan` | [`String!`](#string) | Name of the subscription plan. | | <a id="licensehistoryentrystartsat"></a>`startsAt` | [`Date`](#date) | Date when the license started. | @@ -12130,13 +12408,14 @@ Maven metadata. | <a id="mergerequestapproved"></a>`approved` | [`Boolean!`](#boolean) | Indicates if the merge request has all the required approvals. Returns true if no required approvals are configured. | | <a id="mergerequestapprovedby"></a>`approvedBy` | [`UserCoreConnection`](#usercoreconnection) | Users who approved the merge request. (see [Connections](#connections)) | | <a id="mergerequestassignees"></a>`assignees` | [`MergeRequestAssigneeConnection`](#mergerequestassigneeconnection) | Assignees of the merge request. (see [Connections](#connections)) | -| <a id="mergerequestauthor"></a>`author` | [`UserCore`](#usercore) | User who created this merge request. | +| <a id="mergerequestauthor"></a>`author` | [`MergeRequestAuthor`](#mergerequestauthor) | User who created this merge request. | | <a id="mergerequestautomergeenabled"></a>`autoMergeEnabled` | [`Boolean!`](#boolean) | Indicates if auto merge is enabled for the merge request. | | <a id="mergerequestautomergestrategy"></a>`autoMergeStrategy` | [`String`](#string) | Selected auto merge strategy. | | <a id="mergerequestavailableautomergestrategies"></a>`availableAutoMergeStrategies` | [`[String!]`](#string) | Array of available auto merge strategies. | | <a id="mergerequestcommitcount"></a>`commitCount` | [`Int`](#int) | Number of commits in the merge request. | | <a id="mergerequestcommits"></a>`commits` | [`CommitConnection`](#commitconnection) | Merge request commits. (see [Connections](#connections)) | | <a id="mergerequestcommitswithoutmergecommits"></a>`commitsWithoutMergeCommits` | [`CommitConnection`](#commitconnection) | Merge request commits excluding merge commits. (see [Connections](#connections)) | +| <a id="mergerequestcommitters"></a>`committers` | [`UserCoreConnection`](#usercoreconnection) | Users who have added commits to the merge request. (see [Connections](#connections)) | | <a id="mergerequestconflicts"></a>`conflicts` | [`Boolean!`](#boolean) | Indicates if the merge request has conflicts. | | <a id="mergerequestcreatedat"></a>`createdAt` | [`Time!`](#time) | Timestamp of when the merge request was created. | | <a id="mergerequestdefaultmergecommitmessage"></a>`defaultMergeCommitMessage` | [`String`](#string) | Default merge commit message of the merge request. | @@ -12175,7 +12454,7 @@ Maven metadata. | <a id="mergerequestmergedat"></a>`mergedAt` | [`Time`](#time) | Timestamp of when the merge request was merged, null if not merged. | | <a id="mergerequestmilestone"></a>`milestone` | [`Milestone`](#milestone) | Milestone of the merge request. | | <a id="mergerequestnotes"></a>`notes` | [`NoteConnection!`](#noteconnection) | All notes on this noteable. (see [Connections](#connections)) | -| <a id="mergerequestparticipants"></a>`participants` | [`UserCoreConnection`](#usercoreconnection) | Participants in the merge request. This includes the author, assignees, reviewers, and users mentioned in notes. (see [Connections](#connections)) | +| <a id="mergerequestparticipants"></a>`participants` | [`MergeRequestParticipantConnection`](#mergerequestparticipantconnection) | Participants in the merge request. This includes the author, assignees, reviewers, and users mentioned in notes. (see [Connections](#connections)) | | <a id="mergerequestproject"></a>`project` | [`Project!`](#project) | Alias for target_project. | | <a id="mergerequestprojectid"></a>`projectId` | [`Int!`](#int) | ID of the merge request project. | | <a id="mergerequestrebasecommitsha"></a>`rebaseCommitSha` | [`String`](#string) | Rebase commit SHA of the merge request. | @@ -12260,6 +12539,9 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="mergerequestpipelinessha"></a>`sha` | [`String`](#string) | Filter pipelines by the sha of the commit they are run for. | | <a id="mergerequestpipelinessource"></a>`source` | [`String`](#string) | Filter pipelines by their source. | | <a id="mergerequestpipelinesstatus"></a>`status` | [`PipelineStatusEnum`](#pipelinestatusenum) | Filter pipelines by their status. | +| <a id="mergerequestpipelinesupdatedafter"></a>`updatedAfter` | [`Time`](#time) | Pipelines updated after this date. | +| <a id="mergerequestpipelinesupdatedbefore"></a>`updatedBefore` | [`Time`](#time) | Pipelines updated before this date. | +| <a id="mergerequestpipelinesusername"></a>`username` | [`String`](#string) | Filter pipelines by the user that triggered the pipeline. | ##### `MergeRequest.reference` @@ -12296,6 +12578,7 @@ A user assigned to a merge request. | <a id="mergerequestassigneebot"></a>`bot` | [`Boolean!`](#boolean) | Indicates if the user is a bot. | | <a id="mergerequestassigneecallouts"></a>`callouts` | [`UserCalloutConnection`](#usercalloutconnection) | User callouts that belong to the user. (see [Connections](#connections)) | | <a id="mergerequestassigneeemail"></a>`email` **{warning-solid}** | [`String`](#string) | **Deprecated** in 13.7. This was renamed. Use: [`User.publicEmail`](#userpublicemail). | +| <a id="mergerequestassigneegitpodenabled"></a>`gitpodEnabled` | [`Boolean`](#boolean) | Whether Gitpod is enabled at the user level. | | <a id="mergerequestassigneegroupcount"></a>`groupCount` | [`Int`](#int) | Group count for the user. | | <a id="mergerequestassigneegroupmemberships"></a>`groupMemberships` | [`GroupMemberConnection`](#groupmemberconnection) | Group memberships of the user. (see [Connections](#connections)) | | <a id="mergerequestassigneeid"></a>`id` | [`ID!`](#id) | ID of the user. | @@ -12303,8 +12586,11 @@ A user assigned to a merge request. | <a id="mergerequestassigneemergerequestinteraction"></a>`mergeRequestInteraction` | [`UserMergeRequestInteraction`](#usermergerequestinteraction) | Details of this user's interactions with the merge request. | | <a id="mergerequestassigneename"></a>`name` | [`String!`](#string) | Human-readable name of the user. Returns `****` if the user is a project bot and the requester does not have permission to view the project. | | <a id="mergerequestassigneenamespace"></a>`namespace` | [`Namespace`](#namespace) | Personal namespace of the user. | +| <a id="mergerequestassigneepreferencesgitpodpath"></a>`preferencesGitpodPath` | [`String`](#string) | Web path to the Gitpod section within user preferences. | +| <a id="mergerequestassigneeprofileenablegitpodpath"></a>`profileEnableGitpodPath` | [`String`](#string) | Web path to enable Gitpod for the user. | | <a id="mergerequestassigneeprojectmemberships"></a>`projectMemberships` | [`ProjectMemberConnection`](#projectmemberconnection) | Project memberships of the user. (see [Connections](#connections)) | | <a id="mergerequestassigneepublicemail"></a>`publicEmail` | [`String`](#string) | User's public email. | +| <a id="mergerequestassigneesavedreplies"></a>`savedReplies` | [`SavedReplyConnection`](#savedreplyconnection) | Saved replies authored by the user. (see [Connections](#connections)) | | <a id="mergerequestassigneestate"></a>`state` | [`UserState!`](#userstate) | State of the user. | | <a id="mergerequestassigneestatus"></a>`status` | [`UserStatus`](#userstatus) | User status. | | <a id="mergerequestassigneeuserpermissions"></a>`userPermissions` | [`UserPermissions!`](#userpermissions) | Permissions for the current user on the resource. | @@ -12510,6 +12796,236 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="mergerequestassigneetodosstate"></a>`state` | [`[TodoStateEnum!]`](#todostateenum) | State of the todo. | | <a id="mergerequestassigneetodostype"></a>`type` | [`[TodoTargetEnum!]`](#todotargetenum) | Type of the todo. | +### `MergeRequestAuthor` + +The author of the merge request. + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergerequestauthoravatarurl"></a>`avatarUrl` | [`String`](#string) | URL of the user's avatar. | +| <a id="mergerequestauthorbot"></a>`bot` | [`Boolean!`](#boolean) | Indicates if the user is a bot. | +| <a id="mergerequestauthorcallouts"></a>`callouts` | [`UserCalloutConnection`](#usercalloutconnection) | User callouts that belong to the user. (see [Connections](#connections)) | +| <a id="mergerequestauthoremail"></a>`email` **{warning-solid}** | [`String`](#string) | **Deprecated** in 13.7. This was renamed. Use: [`User.publicEmail`](#userpublicemail). | +| <a id="mergerequestauthorgitpodenabled"></a>`gitpodEnabled` | [`Boolean`](#boolean) | Whether Gitpod is enabled at the user level. | +| <a id="mergerequestauthorgroupcount"></a>`groupCount` | [`Int`](#int) | Group count for the user. | +| <a id="mergerequestauthorgroupmemberships"></a>`groupMemberships` | [`GroupMemberConnection`](#groupmemberconnection) | Group memberships of the user. (see [Connections](#connections)) | +| <a id="mergerequestauthorid"></a>`id` | [`ID!`](#id) | ID of the user. | +| <a id="mergerequestauthorlocation"></a>`location` | [`String`](#string) | Location of the user. | +| <a id="mergerequestauthormergerequestinteraction"></a>`mergeRequestInteraction` | [`UserMergeRequestInteraction`](#usermergerequestinteraction) | Details of this user's interactions with the merge request. | +| <a id="mergerequestauthorname"></a>`name` | [`String!`](#string) | Human-readable name of the user. Returns `****` if the user is a project bot and the requester does not have permission to view the project. | +| <a id="mergerequestauthornamespace"></a>`namespace` | [`Namespace`](#namespace) | Personal namespace of the user. | +| <a id="mergerequestauthorpreferencesgitpodpath"></a>`preferencesGitpodPath` | [`String`](#string) | Web path to the Gitpod section within user preferences. | +| <a id="mergerequestauthorprofileenablegitpodpath"></a>`profileEnableGitpodPath` | [`String`](#string) | Web path to enable Gitpod for the user. | +| <a id="mergerequestauthorprojectmemberships"></a>`projectMemberships` | [`ProjectMemberConnection`](#projectmemberconnection) | Project memberships of the user. (see [Connections](#connections)) | +| <a id="mergerequestauthorpublicemail"></a>`publicEmail` | [`String`](#string) | User's public email. | +| <a id="mergerequestauthorsavedreplies"></a>`savedReplies` | [`SavedReplyConnection`](#savedreplyconnection) | Saved replies authored by the user. (see [Connections](#connections)) | +| <a id="mergerequestauthorstate"></a>`state` | [`UserState!`](#userstate) | State of the user. | +| <a id="mergerequestauthorstatus"></a>`status` | [`UserStatus`](#userstatus) | User status. | +| <a id="mergerequestauthoruserpermissions"></a>`userPermissions` | [`UserPermissions!`](#userpermissions) | Permissions for the current user on the resource. | +| <a id="mergerequestauthorusername"></a>`username` | [`String!`](#string) | Username of the user. Unique within this instance of GitLab. | +| <a id="mergerequestauthorwebpath"></a>`webPath` | [`String!`](#string) | Web path of the user. | +| <a id="mergerequestauthorweburl"></a>`webUrl` | [`String!`](#string) | Web URL of the user. | + +#### Fields with arguments + +##### `MergeRequestAuthor.assignedMergeRequests` + +Merge requests assigned to the user. + +Returns [`MergeRequestConnection`](#mergerequestconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergerequestauthorassignedmergerequestsauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author. | +| <a id="mergerequestauthorassignedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | +| <a id="mergerequestauthorassignedmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | +| <a id="mergerequestauthorassignedmergerequestsdraft"></a>`draft` | [`Boolean`](#boolean) | Limit result to draft merge requests. | +| <a id="mergerequestauthorassignedmergerequestsiids"></a>`iids` | [`[String!]`](#string) | Array of IIDs of merge requests, for example `[1, 2]`. | +| <a id="mergerequestauthorassignedmergerequestslabels"></a>`labels` | [`[String!]`](#string) | Array of label names. All resolved merge requests will have all of these labels. | +| <a id="mergerequestauthorassignedmergerequestsmergedafter"></a>`mergedAfter` | [`Time`](#time) | Merge requests merged after this date. | +| <a id="mergerequestauthorassignedmergerequestsmergedbefore"></a>`mergedBefore` | [`Time`](#time) | Merge requests merged before this date. | +| <a id="mergerequestauthorassignedmergerequestsmilestonetitle"></a>`milestoneTitle` | [`String`](#string) | Title of the milestone. | +| <a id="mergerequestauthorassignedmergerequestsnot"></a>`not` | [`MergeRequestsResolverNegatedParams`](#mergerequestsresolvernegatedparams) | List of negated arguments. Warning: this argument is experimental and a subject to change in future. | +| <a id="mergerequestauthorassignedmergerequestsprojectid"></a>`projectId` | [`ProjectID`](#projectid) | The global ID of the project the authored merge requests should be in. Incompatible with projectPath. | +| <a id="mergerequestauthorassignedmergerequestsprojectpath"></a>`projectPath` | [`String`](#string) | The full-path of the project the authored merge requests should be in. Incompatible with projectId. | +| <a id="mergerequestauthorassignedmergerequestsreviewerusername"></a>`reviewerUsername` | [`String`](#string) | Username of the reviewer. | +| <a id="mergerequestauthorassignedmergerequestssort"></a>`sort` | [`MergeRequestSort`](#mergerequestsort) | Sort merge requests by this criteria. | +| <a id="mergerequestauthorassignedmergerequestssourcebranches"></a>`sourceBranches` | [`[String!]`](#string) | Array of source branch names. All resolved merge requests will have one of these branches as their source. | +| <a id="mergerequestauthorassignedmergerequestsstate"></a>`state` | [`MergeRequestState`](#mergerequeststate) | Merge request state. If provided, all resolved merge requests will have this state. | +| <a id="mergerequestauthorassignedmergerequeststargetbranches"></a>`targetBranches` | [`[String!]`](#string) | Array of target branch names. All resolved merge requests will have one of these branches as their target. | +| <a id="mergerequestauthorassignedmergerequestsupdatedafter"></a>`updatedAfter` | [`Time`](#time) | Merge requests updated after this timestamp. | +| <a id="mergerequestauthorassignedmergerequestsupdatedbefore"></a>`updatedBefore` | [`Time`](#time) | Merge requests updated before this timestamp. | + +##### `MergeRequestAuthor.authoredMergeRequests` + +Merge requests authored by the user. + +Returns [`MergeRequestConnection`](#mergerequestconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergerequestauthorauthoredmergerequestsassigneeusername"></a>`assigneeUsername` | [`String`](#string) | Username of the assignee. | +| <a id="mergerequestauthorauthoredmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | +| <a id="mergerequestauthorauthoredmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | +| <a id="mergerequestauthorauthoredmergerequestsdraft"></a>`draft` | [`Boolean`](#boolean) | Limit result to draft merge requests. | +| <a id="mergerequestauthorauthoredmergerequestsiids"></a>`iids` | [`[String!]`](#string) | Array of IIDs of merge requests, for example `[1, 2]`. | +| <a id="mergerequestauthorauthoredmergerequestslabels"></a>`labels` | [`[String!]`](#string) | Array of label names. All resolved merge requests will have all of these labels. | +| <a id="mergerequestauthorauthoredmergerequestsmergedafter"></a>`mergedAfter` | [`Time`](#time) | Merge requests merged after this date. | +| <a id="mergerequestauthorauthoredmergerequestsmergedbefore"></a>`mergedBefore` | [`Time`](#time) | Merge requests merged before this date. | +| <a id="mergerequestauthorauthoredmergerequestsmilestonetitle"></a>`milestoneTitle` | [`String`](#string) | Title of the milestone. | +| <a id="mergerequestauthorauthoredmergerequestsnot"></a>`not` | [`MergeRequestsResolverNegatedParams`](#mergerequestsresolvernegatedparams) | List of negated arguments. Warning: this argument is experimental and a subject to change in future. | +| <a id="mergerequestauthorauthoredmergerequestsprojectid"></a>`projectId` | [`ProjectID`](#projectid) | The global ID of the project the authored merge requests should be in. Incompatible with projectPath. | +| <a id="mergerequestauthorauthoredmergerequestsprojectpath"></a>`projectPath` | [`String`](#string) | The full-path of the project the authored merge requests should be in. Incompatible with projectId. | +| <a id="mergerequestauthorauthoredmergerequestsreviewerusername"></a>`reviewerUsername` | [`String`](#string) | Username of the reviewer. | +| <a id="mergerequestauthorauthoredmergerequestssort"></a>`sort` | [`MergeRequestSort`](#mergerequestsort) | Sort merge requests by this criteria. | +| <a id="mergerequestauthorauthoredmergerequestssourcebranches"></a>`sourceBranches` | [`[String!]`](#string) | Array of source branch names. All resolved merge requests will have one of these branches as their source. | +| <a id="mergerequestauthorauthoredmergerequestsstate"></a>`state` | [`MergeRequestState`](#mergerequeststate) | Merge request state. If provided, all resolved merge requests will have this state. | +| <a id="mergerequestauthorauthoredmergerequeststargetbranches"></a>`targetBranches` | [`[String!]`](#string) | Array of target branch names. All resolved merge requests will have one of these branches as their target. | +| <a id="mergerequestauthorauthoredmergerequestsupdatedafter"></a>`updatedAfter` | [`Time`](#time) | Merge requests updated after this timestamp. | +| <a id="mergerequestauthorauthoredmergerequestsupdatedbefore"></a>`updatedBefore` | [`Time`](#time) | Merge requests updated before this timestamp. | + +##### `MergeRequestAuthor.groups` + +Groups where the user has access. + +Returns [`GroupConnection`](#groupconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergerequestauthorgroupspermissionscope"></a>`permissionScope` | [`GroupPermission`](#grouppermission) | Filter by permissions the user has on groups. | +| <a id="mergerequestauthorgroupssearch"></a>`search` | [`String`](#string) | Search by group name or path. | + +##### `MergeRequestAuthor.reviewRequestedMergeRequests` + +Merge requests assigned to the user for review. + +Returns [`MergeRequestConnection`](#mergerequestconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergerequestauthorreviewrequestedmergerequestsassigneeusername"></a>`assigneeUsername` | [`String`](#string) | Username of the assignee. | +| <a id="mergerequestauthorreviewrequestedmergerequestsauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author. | +| <a id="mergerequestauthorreviewrequestedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | +| <a id="mergerequestauthorreviewrequestedmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | +| <a id="mergerequestauthorreviewrequestedmergerequestsdraft"></a>`draft` | [`Boolean`](#boolean) | Limit result to draft merge requests. | +| <a id="mergerequestauthorreviewrequestedmergerequestsiids"></a>`iids` | [`[String!]`](#string) | Array of IIDs of merge requests, for example `[1, 2]`. | +| <a id="mergerequestauthorreviewrequestedmergerequestslabels"></a>`labels` | [`[String!]`](#string) | Array of label names. All resolved merge requests will have all of these labels. | +| <a id="mergerequestauthorreviewrequestedmergerequestsmergedafter"></a>`mergedAfter` | [`Time`](#time) | Merge requests merged after this date. | +| <a id="mergerequestauthorreviewrequestedmergerequestsmergedbefore"></a>`mergedBefore` | [`Time`](#time) | Merge requests merged before this date. | +| <a id="mergerequestauthorreviewrequestedmergerequestsmilestonetitle"></a>`milestoneTitle` | [`String`](#string) | Title of the milestone. | +| <a id="mergerequestauthorreviewrequestedmergerequestsnot"></a>`not` | [`MergeRequestsResolverNegatedParams`](#mergerequestsresolvernegatedparams) | List of negated arguments. Warning: this argument is experimental and a subject to change in future. | +| <a id="mergerequestauthorreviewrequestedmergerequestsprojectid"></a>`projectId` | [`ProjectID`](#projectid) | The global ID of the project the authored merge requests should be in. Incompatible with projectPath. | +| <a id="mergerequestauthorreviewrequestedmergerequestsprojectpath"></a>`projectPath` | [`String`](#string) | The full-path of the project the authored merge requests should be in. Incompatible with projectId. | +| <a id="mergerequestauthorreviewrequestedmergerequestssort"></a>`sort` | [`MergeRequestSort`](#mergerequestsort) | Sort merge requests by this criteria. | +| <a id="mergerequestauthorreviewrequestedmergerequestssourcebranches"></a>`sourceBranches` | [`[String!]`](#string) | Array of source branch names. All resolved merge requests will have one of these branches as their source. | +| <a id="mergerequestauthorreviewrequestedmergerequestsstate"></a>`state` | [`MergeRequestState`](#mergerequeststate) | Merge request state. If provided, all resolved merge requests will have this state. | +| <a id="mergerequestauthorreviewrequestedmergerequeststargetbranches"></a>`targetBranches` | [`[String!]`](#string) | Array of target branch names. All resolved merge requests will have one of these branches as their target. | +| <a id="mergerequestauthorreviewrequestedmergerequestsupdatedafter"></a>`updatedAfter` | [`Time`](#time) | Merge requests updated after this timestamp. | +| <a id="mergerequestauthorreviewrequestedmergerequestsupdatedbefore"></a>`updatedBefore` | [`Time`](#time) | Merge requests updated before this timestamp. | + +##### `MergeRequestAuthor.snippets` + +Snippets authored by the user. + +Returns [`SnippetConnection`](#snippetconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergerequestauthorsnippetsids"></a>`ids` | [`[SnippetID!]`](#snippetid) | Array of global snippet IDs. For example, `gid://gitlab/ProjectSnippet/1`. | +| <a id="mergerequestauthorsnippetstype"></a>`type` | [`TypeEnum`](#typeenum) | Type of snippet. | +| <a id="mergerequestauthorsnippetsvisibility"></a>`visibility` | [`VisibilityScopesEnum`](#visibilityscopesenum) | Visibility of the snippet. | + +##### `MergeRequestAuthor.starredProjects` + +Projects starred by the user. + +Returns [`ProjectConnection`](#projectconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergerequestauthorstarredprojectssearch"></a>`search` | [`String`](#string) | Search query. | + +##### `MergeRequestAuthor.timelogs` + +Time logged by the user. + +Returns [`TimelogConnection`](#timelogconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergerequestauthortimelogsenddate"></a>`endDate` | [`Time`](#time) | List timelogs within a date range where the logged date is equal to or before endDate. | +| <a id="mergerequestauthortimelogsendtime"></a>`endTime` | [`Time`](#time) | List timelogs within a time range where the logged time is equal to or before endTime. | +| <a id="mergerequestauthortimelogsgroupid"></a>`groupId` | [`GroupID`](#groupid) | List timelogs for a group. | +| <a id="mergerequestauthortimelogsprojectid"></a>`projectId` | [`ProjectID`](#projectid) | List timelogs for a project. | +| <a id="mergerequestauthortimelogsstartdate"></a>`startDate` | [`Time`](#time) | List timelogs within a date range where the logged date is equal to or after startDate. | +| <a id="mergerequestauthortimelogsstarttime"></a>`startTime` | [`Time`](#time) | List timelogs within a time range where the logged time is equal to or after startTime. | +| <a id="mergerequestauthortimelogsusername"></a>`username` | [`String`](#string) | List timelogs for a user. | + +##### `MergeRequestAuthor.todos` + +To-do items of the user. + +Returns [`TodoConnection`](#todoconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergerequestauthortodosaction"></a>`action` | [`[TodoActionEnum!]`](#todoactionenum) | Action to be filtered. | +| <a id="mergerequestauthortodosauthorid"></a>`authorId` | [`[ID!]`](#id) | ID of an author. | +| <a id="mergerequestauthortodosgroupid"></a>`groupId` | [`[ID!]`](#id) | ID of a group. | +| <a id="mergerequestauthortodosprojectid"></a>`projectId` | [`[ID!]`](#id) | ID of a project. | +| <a id="mergerequestauthortodosstate"></a>`state` | [`[TodoStateEnum!]`](#todostateenum) | State of the todo. | +| <a id="mergerequestauthortodostype"></a>`type` | [`[TodoTargetEnum!]`](#todotargetenum) | Type of the todo. | + ### `MergeRequestDiffRegistry` Represents the Geo sync and verification state of a Merge Request diff. @@ -12527,6 +13043,236 @@ Represents the Geo sync and verification state of a Merge Request diff. | <a id="mergerequestdiffregistryretrycount"></a>`retryCount` | [`Int`](#int) | Number of consecutive failed sync attempts of the MergeRequestDiffRegistry. | | <a id="mergerequestdiffregistrystate"></a>`state` | [`RegistryState`](#registrystate) | Sync state of the MergeRequestDiffRegistry. | +### `MergeRequestParticipant` + +A user participating in a merge request. + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergerequestparticipantavatarurl"></a>`avatarUrl` | [`String`](#string) | URL of the user's avatar. | +| <a id="mergerequestparticipantbot"></a>`bot` | [`Boolean!`](#boolean) | Indicates if the user is a bot. | +| <a id="mergerequestparticipantcallouts"></a>`callouts` | [`UserCalloutConnection`](#usercalloutconnection) | User callouts that belong to the user. (see [Connections](#connections)) | +| <a id="mergerequestparticipantemail"></a>`email` **{warning-solid}** | [`String`](#string) | **Deprecated** in 13.7. This was renamed. Use: [`User.publicEmail`](#userpublicemail). | +| <a id="mergerequestparticipantgitpodenabled"></a>`gitpodEnabled` | [`Boolean`](#boolean) | Whether Gitpod is enabled at the user level. | +| <a id="mergerequestparticipantgroupcount"></a>`groupCount` | [`Int`](#int) | Group count for the user. | +| <a id="mergerequestparticipantgroupmemberships"></a>`groupMemberships` | [`GroupMemberConnection`](#groupmemberconnection) | Group memberships of the user. (see [Connections](#connections)) | +| <a id="mergerequestparticipantid"></a>`id` | [`ID!`](#id) | ID of the user. | +| <a id="mergerequestparticipantlocation"></a>`location` | [`String`](#string) | Location of the user. | +| <a id="mergerequestparticipantmergerequestinteraction"></a>`mergeRequestInteraction` | [`UserMergeRequestInteraction`](#usermergerequestinteraction) | Details of this user's interactions with the merge request. | +| <a id="mergerequestparticipantname"></a>`name` | [`String!`](#string) | Human-readable name of the user. Returns `****` if the user is a project bot and the requester does not have permission to view the project. | +| <a id="mergerequestparticipantnamespace"></a>`namespace` | [`Namespace`](#namespace) | Personal namespace of the user. | +| <a id="mergerequestparticipantpreferencesgitpodpath"></a>`preferencesGitpodPath` | [`String`](#string) | Web path to the Gitpod section within user preferences. | +| <a id="mergerequestparticipantprofileenablegitpodpath"></a>`profileEnableGitpodPath` | [`String`](#string) | Web path to enable Gitpod for the user. | +| <a id="mergerequestparticipantprojectmemberships"></a>`projectMemberships` | [`ProjectMemberConnection`](#projectmemberconnection) | Project memberships of the user. (see [Connections](#connections)) | +| <a id="mergerequestparticipantpublicemail"></a>`publicEmail` | [`String`](#string) | User's public email. | +| <a id="mergerequestparticipantsavedreplies"></a>`savedReplies` | [`SavedReplyConnection`](#savedreplyconnection) | Saved replies authored by the user. (see [Connections](#connections)) | +| <a id="mergerequestparticipantstate"></a>`state` | [`UserState!`](#userstate) | State of the user. | +| <a id="mergerequestparticipantstatus"></a>`status` | [`UserStatus`](#userstatus) | User status. | +| <a id="mergerequestparticipantuserpermissions"></a>`userPermissions` | [`UserPermissions!`](#userpermissions) | Permissions for the current user on the resource. | +| <a id="mergerequestparticipantusername"></a>`username` | [`String!`](#string) | Username of the user. Unique within this instance of GitLab. | +| <a id="mergerequestparticipantwebpath"></a>`webPath` | [`String!`](#string) | Web path of the user. | +| <a id="mergerequestparticipantweburl"></a>`webUrl` | [`String!`](#string) | Web URL of the user. | + +#### Fields with arguments + +##### `MergeRequestParticipant.assignedMergeRequests` + +Merge requests assigned to the user. + +Returns [`MergeRequestConnection`](#mergerequestconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergerequestparticipantassignedmergerequestsauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author. | +| <a id="mergerequestparticipantassignedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | +| <a id="mergerequestparticipantassignedmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | +| <a id="mergerequestparticipantassignedmergerequestsdraft"></a>`draft` | [`Boolean`](#boolean) | Limit result to draft merge requests. | +| <a id="mergerequestparticipantassignedmergerequestsiids"></a>`iids` | [`[String!]`](#string) | Array of IIDs of merge requests, for example `[1, 2]`. | +| <a id="mergerequestparticipantassignedmergerequestslabels"></a>`labels` | [`[String!]`](#string) | Array of label names. All resolved merge requests will have all of these labels. | +| <a id="mergerequestparticipantassignedmergerequestsmergedafter"></a>`mergedAfter` | [`Time`](#time) | Merge requests merged after this date. | +| <a id="mergerequestparticipantassignedmergerequestsmergedbefore"></a>`mergedBefore` | [`Time`](#time) | Merge requests merged before this date. | +| <a id="mergerequestparticipantassignedmergerequestsmilestonetitle"></a>`milestoneTitle` | [`String`](#string) | Title of the milestone. | +| <a id="mergerequestparticipantassignedmergerequestsnot"></a>`not` | [`MergeRequestsResolverNegatedParams`](#mergerequestsresolvernegatedparams) | List of negated arguments. Warning: this argument is experimental and a subject to change in future. | +| <a id="mergerequestparticipantassignedmergerequestsprojectid"></a>`projectId` | [`ProjectID`](#projectid) | The global ID of the project the authored merge requests should be in. Incompatible with projectPath. | +| <a id="mergerequestparticipantassignedmergerequestsprojectpath"></a>`projectPath` | [`String`](#string) | The full-path of the project the authored merge requests should be in. Incompatible with projectId. | +| <a id="mergerequestparticipantassignedmergerequestsreviewerusername"></a>`reviewerUsername` | [`String`](#string) | Username of the reviewer. | +| <a id="mergerequestparticipantassignedmergerequestssort"></a>`sort` | [`MergeRequestSort`](#mergerequestsort) | Sort merge requests by this criteria. | +| <a id="mergerequestparticipantassignedmergerequestssourcebranches"></a>`sourceBranches` | [`[String!]`](#string) | Array of source branch names. All resolved merge requests will have one of these branches as their source. | +| <a id="mergerequestparticipantassignedmergerequestsstate"></a>`state` | [`MergeRequestState`](#mergerequeststate) | Merge request state. If provided, all resolved merge requests will have this state. | +| <a id="mergerequestparticipantassignedmergerequeststargetbranches"></a>`targetBranches` | [`[String!]`](#string) | Array of target branch names. All resolved merge requests will have one of these branches as their target. | +| <a id="mergerequestparticipantassignedmergerequestsupdatedafter"></a>`updatedAfter` | [`Time`](#time) | Merge requests updated after this timestamp. | +| <a id="mergerequestparticipantassignedmergerequestsupdatedbefore"></a>`updatedBefore` | [`Time`](#time) | Merge requests updated before this timestamp. | + +##### `MergeRequestParticipant.authoredMergeRequests` + +Merge requests authored by the user. + +Returns [`MergeRequestConnection`](#mergerequestconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergerequestparticipantauthoredmergerequestsassigneeusername"></a>`assigneeUsername` | [`String`](#string) | Username of the assignee. | +| <a id="mergerequestparticipantauthoredmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | +| <a id="mergerequestparticipantauthoredmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | +| <a id="mergerequestparticipantauthoredmergerequestsdraft"></a>`draft` | [`Boolean`](#boolean) | Limit result to draft merge requests. | +| <a id="mergerequestparticipantauthoredmergerequestsiids"></a>`iids` | [`[String!]`](#string) | Array of IIDs of merge requests, for example `[1, 2]`. | +| <a id="mergerequestparticipantauthoredmergerequestslabels"></a>`labels` | [`[String!]`](#string) | Array of label names. All resolved merge requests will have all of these labels. | +| <a id="mergerequestparticipantauthoredmergerequestsmergedafter"></a>`mergedAfter` | [`Time`](#time) | Merge requests merged after this date. | +| <a id="mergerequestparticipantauthoredmergerequestsmergedbefore"></a>`mergedBefore` | [`Time`](#time) | Merge requests merged before this date. | +| <a id="mergerequestparticipantauthoredmergerequestsmilestonetitle"></a>`milestoneTitle` | [`String`](#string) | Title of the milestone. | +| <a id="mergerequestparticipantauthoredmergerequestsnot"></a>`not` | [`MergeRequestsResolverNegatedParams`](#mergerequestsresolvernegatedparams) | List of negated arguments. Warning: this argument is experimental and a subject to change in future. | +| <a id="mergerequestparticipantauthoredmergerequestsprojectid"></a>`projectId` | [`ProjectID`](#projectid) | The global ID of the project the authored merge requests should be in. Incompatible with projectPath. | +| <a id="mergerequestparticipantauthoredmergerequestsprojectpath"></a>`projectPath` | [`String`](#string) | The full-path of the project the authored merge requests should be in. Incompatible with projectId. | +| <a id="mergerequestparticipantauthoredmergerequestsreviewerusername"></a>`reviewerUsername` | [`String`](#string) | Username of the reviewer. | +| <a id="mergerequestparticipantauthoredmergerequestssort"></a>`sort` | [`MergeRequestSort`](#mergerequestsort) | Sort merge requests by this criteria. | +| <a id="mergerequestparticipantauthoredmergerequestssourcebranches"></a>`sourceBranches` | [`[String!]`](#string) | Array of source branch names. All resolved merge requests will have one of these branches as their source. | +| <a id="mergerequestparticipantauthoredmergerequestsstate"></a>`state` | [`MergeRequestState`](#mergerequeststate) | Merge request state. If provided, all resolved merge requests will have this state. | +| <a id="mergerequestparticipantauthoredmergerequeststargetbranches"></a>`targetBranches` | [`[String!]`](#string) | Array of target branch names. All resolved merge requests will have one of these branches as their target. | +| <a id="mergerequestparticipantauthoredmergerequestsupdatedafter"></a>`updatedAfter` | [`Time`](#time) | Merge requests updated after this timestamp. | +| <a id="mergerequestparticipantauthoredmergerequestsupdatedbefore"></a>`updatedBefore` | [`Time`](#time) | Merge requests updated before this timestamp. | + +##### `MergeRequestParticipant.groups` + +Groups where the user has access. + +Returns [`GroupConnection`](#groupconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergerequestparticipantgroupspermissionscope"></a>`permissionScope` | [`GroupPermission`](#grouppermission) | Filter by permissions the user has on groups. | +| <a id="mergerequestparticipantgroupssearch"></a>`search` | [`String`](#string) | Search by group name or path. | + +##### `MergeRequestParticipant.reviewRequestedMergeRequests` + +Merge requests assigned to the user for review. + +Returns [`MergeRequestConnection`](#mergerequestconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergerequestparticipantreviewrequestedmergerequestsassigneeusername"></a>`assigneeUsername` | [`String`](#string) | Username of the assignee. | +| <a id="mergerequestparticipantreviewrequestedmergerequestsauthorusername"></a>`authorUsername` | [`String`](#string) | Username of the author. | +| <a id="mergerequestparticipantreviewrequestedmergerequestscreatedafter"></a>`createdAfter` | [`Time`](#time) | Merge requests created after this timestamp. | +| <a id="mergerequestparticipantreviewrequestedmergerequestscreatedbefore"></a>`createdBefore` | [`Time`](#time) | Merge requests created before this timestamp. | +| <a id="mergerequestparticipantreviewrequestedmergerequestsdraft"></a>`draft` | [`Boolean`](#boolean) | Limit result to draft merge requests. | +| <a id="mergerequestparticipantreviewrequestedmergerequestsiids"></a>`iids` | [`[String!]`](#string) | Array of IIDs of merge requests, for example `[1, 2]`. | +| <a id="mergerequestparticipantreviewrequestedmergerequestslabels"></a>`labels` | [`[String!]`](#string) | Array of label names. All resolved merge requests will have all of these labels. | +| <a id="mergerequestparticipantreviewrequestedmergerequestsmergedafter"></a>`mergedAfter` | [`Time`](#time) | Merge requests merged after this date. | +| <a id="mergerequestparticipantreviewrequestedmergerequestsmergedbefore"></a>`mergedBefore` | [`Time`](#time) | Merge requests merged before this date. | +| <a id="mergerequestparticipantreviewrequestedmergerequestsmilestonetitle"></a>`milestoneTitle` | [`String`](#string) | Title of the milestone. | +| <a id="mergerequestparticipantreviewrequestedmergerequestsnot"></a>`not` | [`MergeRequestsResolverNegatedParams`](#mergerequestsresolvernegatedparams) | List of negated arguments. Warning: this argument is experimental and a subject to change in future. | +| <a id="mergerequestparticipantreviewrequestedmergerequestsprojectid"></a>`projectId` | [`ProjectID`](#projectid) | The global ID of the project the authored merge requests should be in. Incompatible with projectPath. | +| <a id="mergerequestparticipantreviewrequestedmergerequestsprojectpath"></a>`projectPath` | [`String`](#string) | The full-path of the project the authored merge requests should be in. Incompatible with projectId. | +| <a id="mergerequestparticipantreviewrequestedmergerequestssort"></a>`sort` | [`MergeRequestSort`](#mergerequestsort) | Sort merge requests by this criteria. | +| <a id="mergerequestparticipantreviewrequestedmergerequestssourcebranches"></a>`sourceBranches` | [`[String!]`](#string) | Array of source branch names. All resolved merge requests will have one of these branches as their source. | +| <a id="mergerequestparticipantreviewrequestedmergerequestsstate"></a>`state` | [`MergeRequestState`](#mergerequeststate) | Merge request state. If provided, all resolved merge requests will have this state. | +| <a id="mergerequestparticipantreviewrequestedmergerequeststargetbranches"></a>`targetBranches` | [`[String!]`](#string) | Array of target branch names. All resolved merge requests will have one of these branches as their target. | +| <a id="mergerequestparticipantreviewrequestedmergerequestsupdatedafter"></a>`updatedAfter` | [`Time`](#time) | Merge requests updated after this timestamp. | +| <a id="mergerequestparticipantreviewrequestedmergerequestsupdatedbefore"></a>`updatedBefore` | [`Time`](#time) | Merge requests updated before this timestamp. | + +##### `MergeRequestParticipant.snippets` + +Snippets authored by the user. + +Returns [`SnippetConnection`](#snippetconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergerequestparticipantsnippetsids"></a>`ids` | [`[SnippetID!]`](#snippetid) | Array of global snippet IDs. For example, `gid://gitlab/ProjectSnippet/1`. | +| <a id="mergerequestparticipantsnippetstype"></a>`type` | [`TypeEnum`](#typeenum) | Type of snippet. | +| <a id="mergerequestparticipantsnippetsvisibility"></a>`visibility` | [`VisibilityScopesEnum`](#visibilityscopesenum) | Visibility of the snippet. | + +##### `MergeRequestParticipant.starredProjects` + +Projects starred by the user. + +Returns [`ProjectConnection`](#projectconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergerequestparticipantstarredprojectssearch"></a>`search` | [`String`](#string) | Search query. | + +##### `MergeRequestParticipant.timelogs` + +Time logged by the user. + +Returns [`TimelogConnection`](#timelogconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergerequestparticipanttimelogsenddate"></a>`endDate` | [`Time`](#time) | List timelogs within a date range where the logged date is equal to or before endDate. | +| <a id="mergerequestparticipanttimelogsendtime"></a>`endTime` | [`Time`](#time) | List timelogs within a time range where the logged time is equal to or before endTime. | +| <a id="mergerequestparticipanttimelogsgroupid"></a>`groupId` | [`GroupID`](#groupid) | List timelogs for a group. | +| <a id="mergerequestparticipanttimelogsprojectid"></a>`projectId` | [`ProjectID`](#projectid) | List timelogs for a project. | +| <a id="mergerequestparticipanttimelogsstartdate"></a>`startDate` | [`Time`](#time) | List timelogs within a date range where the logged date is equal to or after startDate. | +| <a id="mergerequestparticipanttimelogsstarttime"></a>`startTime` | [`Time`](#time) | List timelogs within a time range where the logged time is equal to or after startTime. | +| <a id="mergerequestparticipanttimelogsusername"></a>`username` | [`String`](#string) | List timelogs for a user. | + +##### `MergeRequestParticipant.todos` + +To-do items of the user. + +Returns [`TodoConnection`](#todoconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="mergerequestparticipanttodosaction"></a>`action` | [`[TodoActionEnum!]`](#todoactionenum) | Action to be filtered. | +| <a id="mergerequestparticipanttodosauthorid"></a>`authorId` | [`[ID!]`](#id) | ID of an author. | +| <a id="mergerequestparticipanttodosgroupid"></a>`groupId` | [`[ID!]`](#id) | ID of a group. | +| <a id="mergerequestparticipanttodosprojectid"></a>`projectId` | [`[ID!]`](#id) | ID of a project. | +| <a id="mergerequestparticipanttodosstate"></a>`state` | [`[TodoStateEnum!]`](#todostateenum) | State of the todo. | +| <a id="mergerequestparticipanttodostype"></a>`type` | [`[TodoTargetEnum!]`](#todotargetenum) | Type of the todo. | + ### `MergeRequestPermissions` Check permissions for the current user on a merge request. @@ -12557,6 +13303,7 @@ A user assigned to a merge request as a reviewer. | <a id="mergerequestreviewerbot"></a>`bot` | [`Boolean!`](#boolean) | Indicates if the user is a bot. | | <a id="mergerequestreviewercallouts"></a>`callouts` | [`UserCalloutConnection`](#usercalloutconnection) | User callouts that belong to the user. (see [Connections](#connections)) | | <a id="mergerequestrevieweremail"></a>`email` **{warning-solid}** | [`String`](#string) | **Deprecated** in 13.7. This was renamed. Use: [`User.publicEmail`](#userpublicemail). | +| <a id="mergerequestreviewergitpodenabled"></a>`gitpodEnabled` | [`Boolean`](#boolean) | Whether Gitpod is enabled at the user level. | | <a id="mergerequestreviewergroupcount"></a>`groupCount` | [`Int`](#int) | Group count for the user. | | <a id="mergerequestreviewergroupmemberships"></a>`groupMemberships` | [`GroupMemberConnection`](#groupmemberconnection) | Group memberships of the user. (see [Connections](#connections)) | | <a id="mergerequestreviewerid"></a>`id` | [`ID!`](#id) | ID of the user. | @@ -12564,8 +13311,11 @@ A user assigned to a merge request as a reviewer. | <a id="mergerequestreviewermergerequestinteraction"></a>`mergeRequestInteraction` | [`UserMergeRequestInteraction`](#usermergerequestinteraction) | Details of this user's interactions with the merge request. | | <a id="mergerequestreviewername"></a>`name` | [`String!`](#string) | Human-readable name of the user. Returns `****` if the user is a project bot and the requester does not have permission to view the project. | | <a id="mergerequestreviewernamespace"></a>`namespace` | [`Namespace`](#namespace) | Personal namespace of the user. | +| <a id="mergerequestreviewerpreferencesgitpodpath"></a>`preferencesGitpodPath` | [`String`](#string) | Web path to the Gitpod section within user preferences. | +| <a id="mergerequestreviewerprofileenablegitpodpath"></a>`profileEnableGitpodPath` | [`String`](#string) | Web path to enable Gitpod for the user. | | <a id="mergerequestreviewerprojectmemberships"></a>`projectMemberships` | [`ProjectMemberConnection`](#projectmemberconnection) | Project memberships of the user. (see [Connections](#connections)) | | <a id="mergerequestreviewerpublicemail"></a>`publicEmail` | [`String`](#string) | User's public email. | +| <a id="mergerequestreviewersavedreplies"></a>`savedReplies` | [`SavedReplyConnection`](#savedreplyconnection) | Saved replies authored by the user. (see [Connections](#connections)) | | <a id="mergerequestreviewerstate"></a>`state` | [`UserState!`](#userstate) | State of the user. | | <a id="mergerequestreviewerstatus"></a>`status` | [`UserStatus`](#userstatus) | User status. | | <a id="mergerequestrevieweruserpermissions"></a>`userPermissions` | [`UserPermissions!`](#userpermissions) | Permissions for the current user on the resource. | @@ -12851,7 +13601,6 @@ Represents a milestone. | <a id="milestoneid"></a>`id` | [`ID!`](#id) | ID of the milestone. | | <a id="milestoneiid"></a>`iid` | [`ID!`](#id) | Internal ID of the milestone. | | <a id="milestoneprojectmilestone"></a>`projectMilestone` | [`Boolean!`](#boolean) | Indicates if milestone is at project level. | -| <a id="milestonereport"></a>`report` | [`TimeboxReport`](#timeboxreport) | Historically accurate report about the timebox. | | <a id="milestonestartdate"></a>`startDate` | [`Time`](#time) | Timestamp of the milestone start date. | | <a id="milestonestate"></a>`state` | [`MilestoneStateEnum!`](#milestonestateenum) | State of the milestone. | | <a id="milestonestats"></a>`stats` | [`MilestoneStats`](#milestonestats) | Milestone statistics. | @@ -12860,6 +13609,20 @@ Represents a milestone. | <a id="milestoneupdatedat"></a>`updatedAt` | [`Time!`](#time) | Timestamp of last milestone update. | | <a id="milestonewebpath"></a>`webPath` | [`String!`](#string) | Web path of the milestone. | +#### Fields with arguments + +##### `Milestone.report` + +Historically accurate report about the timebox. + +Returns [`TimeboxReport`](#timeboxreport). + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="milestonereportfullpath"></a>`fullPath` | [`String`](#string) | Full path of the project or group used as a scope for report. For example, `gitlab-org` or `gitlab-org/gitlab`. | + ### `MilestoneStats` Contains statistics about a milestone. @@ -12880,6 +13643,7 @@ Contains statistics about a milestone. | <a id="namespaceactualrepositorysizelimit"></a>`actualRepositorySizeLimit` | [`Float`](#float) | Size limit for repositories in the namespace in bytes. | | <a id="namespaceadditionalpurchasedstoragesize"></a>`additionalPurchasedStorageSize` | [`Float`](#float) | Additional storage purchased for the root namespace in bytes. | | <a id="namespacecontainslockedprojects"></a>`containsLockedProjects` | [`Boolean!`](#boolean) | Includes at least one project where the repository size exceeds the limit. | +| <a id="namespacecrossprojectpipelineavailable"></a>`crossProjectPipelineAvailable` | [`Boolean!`](#boolean) | Indicates if the cross_project_pipeline feature is available for the namespace. | | <a id="namespacedescription"></a>`description` | [`String`](#string) | Description of the namespace. | | <a id="namespacedescriptionhtml"></a>`descriptionHtml` | [`String`](#string) | The GitLab Flavored Markdown rendering of `description`. | | <a id="namespacefullname"></a>`fullName` | [`String!`](#string) | Full name of the namespace. | @@ -13529,7 +14293,7 @@ Represents vulnerability finding of a security report on the pipeline. | <a id="projectcontainerexpirationpolicy"></a>`containerExpirationPolicy` | [`ContainerExpirationPolicy`](#containerexpirationpolicy) | Container expiration policy of the project. | | <a id="projectcontainerregistryenabled"></a>`containerRegistryEnabled` | [`Boolean`](#boolean) | Indicates if Container Registry is enabled for the current user. | | <a id="projectcontainerrepositoriescount"></a>`containerRepositoriesCount` | [`Int!`](#int) | Number of container repositories in the project. | -| <a id="projectcorpuses"></a>`corpuses` | [`CoverageFuzzingCorpusConnection`](#coveragefuzzingcorpusconnection) | Find corpuses of the project. Available only when feature flag `corpus_management` is enabled. This flag is enabled by default. (see [Connections](#connections)) | +| <a id="projectcorpuses"></a>`corpuses` | [`CoverageFuzzingCorpusConnection`](#coveragefuzzingcorpusconnection) | Find corpuses of the project. (see [Connections](#connections)) | | <a id="projectcreatedat"></a>`createdAt` | [`Time`](#time) | Timestamp of the project creation. | | <a id="projectdastscannerprofiles"></a>`dastScannerProfiles` | [`DastScannerProfileConnection`](#dastscannerprofileconnection) | DAST scanner profiles associated with the project. (see [Connections](#connections)) | | <a id="projectdastsiteprofiles"></a>`dastSiteProfiles` | [`DastSiteProfileConnection`](#dastsiteprofileconnection) | DAST Site Profiles associated with the project. (see [Connections](#connections)) | @@ -13593,7 +14357,6 @@ Represents vulnerability finding of a security report on the pipeline. | <a id="projectvulnerabilityscanners"></a>`vulnerabilityScanners` | [`VulnerabilityScannerConnection`](#vulnerabilityscannerconnection) | Vulnerability scanners reported on the project vulnerabilities. (see [Connections](#connections)) | | <a id="projectweburl"></a>`webUrl` | [`String`](#string) | Web URL of the project. | | <a id="projectwikienabled"></a>`wikiEnabled` | [`Boolean`](#boolean) | Indicates if Wikis are enabled for the current user. | -| <a id="projectworkitemtypes"></a>`workItemTypes` | [`WorkItemTypeConnection`](#workitemtypeconnection) | Work item types available to the project. Available only when feature flag `work_items` is enabled. This flag is disabled by default, because the feature is experimental and is subject to change without notice. (see [Connections](#connections)) | #### Fields with arguments @@ -14209,6 +14972,10 @@ four standard [pagination arguments](#connection-pagination-arguments): Network Policies of the project. +WARNING: +**Deprecated** in 14.8. +Network policies are deprecated and will be removed in GitLab 15.0. + Returns [`NetworkPolicyConnection`](#networkpolicyconnection). This field returns a [connection](#connections). It accepts the @@ -14287,6 +15054,9 @@ four standard [pagination arguments](#connection-pagination-arguments): | <a id="projectpipelinessha"></a>`sha` | [`String`](#string) | Filter pipelines by the sha of the commit they are run for. | | <a id="projectpipelinessource"></a>`source` | [`String`](#string) | Filter pipelines by their source. | | <a id="projectpipelinesstatus"></a>`status` | [`PipelineStatusEnum`](#pipelinestatusenum) | Filter pipelines by their status. | +| <a id="projectpipelinesupdatedafter"></a>`updatedAfter` | [`Time`](#time) | Pipelines updated after this date. | +| <a id="projectpipelinesupdatedbefore"></a>`updatedBefore` | [`Time`](#time) | Pipelines updated before this date. | +| <a id="projectpipelinesusername"></a>`username` | [`String`](#string) | Filter pipelines by the user that triggered the pipeline. | ##### `Project.projectMembers` @@ -14401,6 +15171,18 @@ Returns [`[ProjectSecurityTraining!]`](#projectsecuritytraining). | ---- | ---- | ----------- | | <a id="projectsecuritytrainingprovidersonlyenabled"></a>`onlyEnabled` | [`Boolean`](#boolean) | Filter the list by only enabled security trainings. | +##### `Project.securityTrainingUrls` + +Security training URLs for the enabled training providers of the project. + +Returns [`[SecurityTrainingUrl!]`](#securitytrainingurl). + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="projectsecuritytrainingurlsidentifierexternalids"></a>`identifierExternalIds` | [`[String!]!`](#string) | List of external IDs of vulnerability identifiers. | + ##### `Project.sentryDetailedError` Detailed version of a Sentry error on the project. @@ -14544,6 +15326,22 @@ Returns [`VulnerabilitySeveritiesCount`](#vulnerabilityseveritiescount). | <a id="projectvulnerabilityseveritiescountseverity"></a>`severity` | [`[VulnerabilitySeverity!]`](#vulnerabilityseverity) | Filter vulnerabilities by severity. | | <a id="projectvulnerabilityseveritiescountstate"></a>`state` | [`[VulnerabilityState!]`](#vulnerabilitystate) | Filter vulnerabilities by state. | +##### `Project.workItemTypes` + +Work item types available to the project. Returns `null` if `work_items` feature flag is disabled. This flag is disabled by default, because the feature is experimental and is subject to change without notice. + +Returns [`WorkItemTypeConnection`](#workitemtypeconnection). + +This field returns a [connection](#connections). It accepts the +four standard [pagination arguments](#connection-pagination-arguments): +`before: String`, `after: String`, `first: Int`, `last: Int`. + +###### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="projectworkitemtypestaskable"></a>`taskable` | [`Boolean`](#boolean) | If `true`, only taskable work item types will be returned. Argument is experimental and can be removed in the future without notice. | + ### `ProjectCiCdSetting` #### Fields @@ -14896,6 +15694,7 @@ Returns [`Tree`](#tree). | <a id="repositoryblobblamepath"></a>`blamePath` | [`String`](#string) | Web path to blob blame page. | | <a id="repositoryblobcancurrentuserpushtobranch"></a>`canCurrentUserPushToBranch` | [`Boolean`](#boolean) | Whether the current user can push to the branch. | | <a id="repositoryblobcanmodifyblob"></a>`canModifyBlob` | [`Boolean`](#boolean) | Whether the current user can modify the blob. | +| <a id="repositoryblobcodenavigationpath"></a>`codeNavigationPath` | [`String`](#string) | Web path for code navigation. | | <a id="repositoryblobcodeowners"></a>`codeOwners` | [`[UserCore!]`](#usercore) | List of code owners for the blob. | | <a id="repositoryblobeditblobpath"></a>`editBlobPath` | [`String`](#string) | Web path to edit the blob in the old-style editor. | | <a id="repositoryblobenvironmentexternalurlforroutemap"></a>`environmentExternalUrlForRouteMap` | [`String`](#string) | Web path to blob on an environment. | @@ -14905,6 +15704,8 @@ Returns [`Tree`](#tree). | <a id="repositoryblobfiletype"></a>`fileType` | [`String`](#string) | Expected format of the blob based on the extension. | | <a id="repositoryblobfindfilepath"></a>`findFilePath` | [`String`](#string) | Web path to find file. | | <a id="repositoryblobforkandeditpath"></a>`forkAndEditPath` | [`String`](#string) | Web path to edit this blob using a forked project. | +| <a id="repositoryblobforkandviewpath"></a>`forkAndViewPath` | [`String`](#string) | Web path to view this blob using a forked project. | +| <a id="repositoryblobgitpodbloburl"></a>`gitpodBlobUrl` | [`String`](#string) | URL to the blob within Gitpod. | | <a id="repositoryblobhistorypath"></a>`historyPath` | [`String`](#string) | Web path to blob history page. | | <a id="repositoryblobid"></a>`id` | [`ID!`](#id) | ID of the blob. | | <a id="repositoryblobideeditpath"></a>`ideEditPath` | [`String`](#string) | Web path to edit this blob in the Web IDE. | @@ -14918,6 +15719,7 @@ Returns [`Tree`](#tree). | <a id="repositoryblobpermalinkpath"></a>`permalinkPath` | [`String`](#string) | Web path to blob permalink. | | <a id="repositoryblobpipelineeditorpath"></a>`pipelineEditorPath` | [`String`](#string) | Web path to edit .gitlab-ci.yml file. | | <a id="repositoryblobplaindata"></a>`plainData` | [`String`](#string) | Blob plain highlighted data. | +| <a id="repositoryblobprojectblobpathroot"></a>`projectBlobPathRoot` | [`String`](#string) | Web path for the root of the blob. | | <a id="repositoryblobrawblob"></a>`rawBlob` | [`String`](#string) | Raw content of the blob. | | <a id="repositoryblobrawpath"></a>`rawPath` | [`String`](#string) | Web path to download the raw blob. | | <a id="repositoryblobrawsize"></a>`rawSize` | [`Int`](#int) | Size (in bytes) of the blob, or the blob target if stored externally. | @@ -15104,6 +15906,16 @@ Represents an entity for options in SAST CI configuration. | <a id="sastciconfigurationoptionsentitylabel"></a>`label` | [`String`](#string) | Label of option entity. | | <a id="sastciconfigurationoptionsentityvalue"></a>`value` | [`String`](#string) | Value of option entity. | +### `SavedReply` + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="savedreplycontent"></a>`content` | [`String!`](#string) | Content of the saved reply. | +| <a id="savedreplyid"></a>`id` | [`UsersSavedReplyID!`](#userssavedreplyid) | Global ID of the saved reply. | +| <a id="savedreplyname"></a>`name` | [`String!`](#string) | Name of the saved reply. | + ### `Scan` Represents the security scan information. @@ -15114,6 +15926,8 @@ Represents the security scan information. | ---- | ---- | ----------- | | <a id="scanerrors"></a>`errors` | [`[String!]!`](#string) | List of errors. | | <a id="scanname"></a>`name` | [`String!`](#string) | Name of the scan. | +| <a id="scanstatus"></a>`status` | [`ScanStatus!`](#scanstatus) | Indicates the status of the scan. | +| <a id="scanwarnings"></a>`warnings` | [`[String!]!`](#string) | List of warnings. | ### `ScanExecutionPolicy` @@ -15198,6 +16012,18 @@ Represents a list of security scanners. | <a id="securityscannersenabled"></a>`enabled` | [`[SecurityScannerType!]`](#securityscannertype) | List of analyzers which are enabled for the project. | | <a id="securityscannerspipelinerun"></a>`pipelineRun` | [`[SecurityScannerType!]`](#securityscannertype) | List of analyzers which ran successfully in the latest pipeline. | +### `SecurityTrainingUrl` + +Represents a URL related to a security training. + +#### Fields + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="securitytrainingurlname"></a>`name` | [`String`](#string) | Name of the training provider. | +| <a id="securitytrainingurlstatus"></a>`status` | [`TrainingUrlRequestStatus`](#trainingurlrequeststatus) | Status of the request to training provider. | +| <a id="securitytrainingurlurl"></a>`url` | [`String`](#string) | URL of the link for security training content. | + ### `SentryDetailedError` A Sentry error. @@ -15764,6 +16590,7 @@ Representing a to-do entry. | <a id="todoid"></a>`id` | [`ID!`](#id) | ID of the to-do item. | | <a id="todoproject"></a>`project` | [`Project`](#project) | Project this to-do item is associated with. | | <a id="todostate"></a>`state` | [`TodoStateEnum!`](#todostateenum) | State of the to-do item. | +| <a id="todotarget"></a>`target` | [`Todoable!`](#todoable) | Target of the to-do item. | | <a id="todotargettype"></a>`targetType` | [`TodoTargetEnum!`](#todotargetenum) | Target type of the to-do item. | ### `Topic` @@ -15856,14 +16683,18 @@ Core represention of a GitLab user. | <a id="usercorebot"></a>`bot` | [`Boolean!`](#boolean) | Indicates if the user is a bot. | | <a id="usercorecallouts"></a>`callouts` | [`UserCalloutConnection`](#usercalloutconnection) | User callouts that belong to the user. (see [Connections](#connections)) | | <a id="usercoreemail"></a>`email` **{warning-solid}** | [`String`](#string) | **Deprecated** in 13.7. This was renamed. Use: [`User.publicEmail`](#userpublicemail). | +| <a id="usercoregitpodenabled"></a>`gitpodEnabled` | [`Boolean`](#boolean) | Whether Gitpod is enabled at the user level. | | <a id="usercoregroupcount"></a>`groupCount` | [`Int`](#int) | Group count for the user. | | <a id="usercoregroupmemberships"></a>`groupMemberships` | [`GroupMemberConnection`](#groupmemberconnection) | Group memberships of the user. (see [Connections](#connections)) | | <a id="usercoreid"></a>`id` | [`ID!`](#id) | ID of the user. | | <a id="usercorelocation"></a>`location` | [`String`](#string) | Location of the user. | | <a id="usercorename"></a>`name` | [`String!`](#string) | Human-readable name of the user. Returns `****` if the user is a project bot and the requester does not have permission to view the project. | | <a id="usercorenamespace"></a>`namespace` | [`Namespace`](#namespace) | Personal namespace of the user. | +| <a id="usercorepreferencesgitpodpath"></a>`preferencesGitpodPath` | [`String`](#string) | Web path to the Gitpod section within user preferences. | +| <a id="usercoreprofileenablegitpodpath"></a>`profileEnableGitpodPath` | [`String`](#string) | Web path to enable Gitpod for the user. | | <a id="usercoreprojectmemberships"></a>`projectMemberships` | [`ProjectMemberConnection`](#projectmemberconnection) | Project memberships of the user. (see [Connections](#connections)) | | <a id="usercorepublicemail"></a>`publicEmail` | [`String`](#string) | User's public email. | +| <a id="usercoresavedreplies"></a>`savedReplies` | [`SavedReplyConnection`](#savedreplyconnection) | Saved replies authored by the user. (see [Connections](#connections)) | | <a id="usercorestate"></a>`state` | [`UserState!`](#userstate) | State of the user. | | <a id="usercorestatus"></a>`status` | [`UserStatus`](#userstatus) | User status. | | <a id="usercoreuserpermissions"></a>`userPermissions` | [`UserPermissions!`](#userpermissions) | Permissions for the current user on the resource. | @@ -16701,6 +17532,7 @@ Represents vulnerability letter grades with associated projects. | <a id="workitemdescriptionhtml"></a>`descriptionHtml` | [`String`](#string) | The GitLab Flavored Markdown rendering of `description`. | | <a id="workitemid"></a>`id` | [`WorkItemID!`](#workitemid) | Global ID of the work item. | | <a id="workitemiid"></a>`iid` | [`ID!`](#id) | Internal ID of the work item. | +| <a id="workitemlockversion"></a>`lockVersion` | [`Int!`](#int) | Lock version of the work item. Incremented each time the work item is updated. | | <a id="workitemstate"></a>`state` | [`WorkItemState!`](#workitemstate) | State of the work item. | | <a id="workitemtitle"></a>`title` | [`String!`](#string) | Title of the work item. | | <a id="workitemtitlehtml"></a>`titleHtml` | [`String`](#string) | The GitLab Flavored Markdown rendering of `title`. | @@ -16990,6 +17822,43 @@ Mode of a commit action. | <a id="commitencodingbase64"></a>`BASE64` | Base64 encoding. | | <a id="commitencodingtext"></a>`TEXT` | Text encoding. | +### `ComplianceViolationReason` + +Reason for the compliance violation. + +| Value | Description | +| ----- | ----------- | +| <a id="complianceviolationreasonapproved_by_committer"></a>`APPROVED_BY_COMMITTER` | Approved by committer. | +| <a id="complianceviolationreasonapproved_by_insufficient_users"></a>`APPROVED_BY_INSUFFICIENT_USERS` | Approved by insufficient users. | +| <a id="complianceviolationreasonapproved_by_merge_request_author"></a>`APPROVED_BY_MERGE_REQUEST_AUTHOR` | Approved by merge request author. | + +### `ComplianceViolationSeverity` + +Severity of the compliance violation. + +| Value | Description | +| ----- | ----------- | +| <a id="complianceviolationseveritycritical"></a>`CRITICAL` | Critical severity. | +| <a id="complianceviolationseverityhigh"></a>`HIGH` | High severity. | +| <a id="complianceviolationseverityinfo"></a>`INFO` | Info severity. | +| <a id="complianceviolationseveritylow"></a>`LOW` | Low severity. | +| <a id="complianceviolationseveritymedium"></a>`MEDIUM` | Medium severity. | + +### `ComplianceViolationSort` + +Compliance violation sort values. + +| Value | Description | +| ----- | ----------- | +| <a id="complianceviolationsortmerged_at_asc"></a>`MERGED_AT_ASC` | Date merged in ascending order, further sorted by ID in ascending order. | +| <a id="complianceviolationsortmerged_at_desc"></a>`MERGED_AT_DESC` | Date merged in descending order, further sorted by ID in descending order. | +| <a id="complianceviolationsortmerge_request_title_asc"></a>`MERGE_REQUEST_TITLE_ASC` | Merge request title in ascending order, further sorted by ID in ascending order. | +| <a id="complianceviolationsortmerge_request_title_desc"></a>`MERGE_REQUEST_TITLE_DESC` | Merge request title in descending order, further sorted by ID in descending order. | +| <a id="complianceviolationsortseverity_level_asc"></a>`SEVERITY_LEVEL_ASC` | Severity in ascending order, further sorted by ID in ascending order. | +| <a id="complianceviolationsortseverity_level_desc"></a>`SEVERITY_LEVEL_DESC` | Severity in descending order, further sorted by ID in descending order. | +| <a id="complianceviolationsortviolation_reason_asc"></a>`VIOLATION_REASON_ASC` | Violation reason in ascending order, further sorted by ID in ascending order. | +| <a id="complianceviolationsortviolation_reason_desc"></a>`VIOLATION_REASON_DESC` | Violation reason in descending order, further sorted by ID in descending order. | + ### `ConanMetadatumFileTypeEnum` Conan file types. @@ -17087,6 +17956,17 @@ Unit for the duration of Dast Profile Cadence. | <a id="dastprofilecadenceunitweek"></a>`WEEK` | DAST Profile Cadence duration in weeks. | | <a id="dastprofilecadenceunityear"></a>`YEAR` | DAST Profile Cadence duration in years. | +### `DastScanMethodType` + +Scan method to be used by the scanner. + +| Value | Description | +| ----- | ----------- | +| <a id="dastscanmethodtypehar"></a>`HAR` | HAR scan method. | +| <a id="dastscanmethodtypeopenapi"></a>`OPENAPI` | OpenAPI scan method. | +| <a id="dastscanmethodtypepostman_collection"></a>`POSTMAN_COLLECTION` | Postman scan method. | +| <a id="dastscanmethodtypewebsite"></a>`WEBSITE` | Website scan method. | + ### `DastScanTypeEnum` | Value | Description | @@ -17218,6 +18098,7 @@ All supported DORA metric types. | ----- | ----------- | | <a id="dorametrictypedeployment_frequency"></a>`DEPLOYMENT_FREQUENCY` | Deployment frequency. | | <a id="dorametrictypelead_time_for_changes"></a>`LEAD_TIME_FOR_CHANGES` | Lead time for changes. | +| <a id="dorametrictypetime_to_restore_service"></a>`TIME_TO_RESTORE_SERVICE` | Time to restore service. | ### `EntryType` @@ -17934,6 +18815,20 @@ Size of UI component in SAST configuration page. | <a id="sastuicomponentsizemedium"></a>`MEDIUM` | Size of UI component in SAST configuration page is medium. | | <a id="sastuicomponentsizesmall"></a>`SMALL` | Size of UI component in SAST configuration page is small. | +### `ScanStatus` + +The status of the security scan. + +| Value | Description | +| ----- | ----------- | +| <a id="scanstatuscreated"></a>`CREATED` | The scan has been created. | +| <a id="scanstatusjob_failed"></a>`JOB_FAILED` | The related CI build failed. | +| <a id="scanstatuspreparation_failed"></a>`PREPARATION_FAILED` | Report couldn't be prepared. | +| <a id="scanstatuspreparing"></a>`PREPARING` | Preparing the report for the scan. | +| <a id="scanstatuspurged"></a>`PURGED` | Report for the scan has been removed from the database. | +| <a id="scanstatusreport_error"></a>`REPORT_ERROR` | The report artifact provided by the CI build couldn't be parsed. | +| <a id="scanstatussucceeded"></a>`SUCCEEDED` | The report has been successfully prepared. | + ### `SecurityReportTypeEnum` | Value | Description | @@ -17995,7 +18890,9 @@ State of a Sentry error. | <a id="servicetypeexternal_wiki_service"></a>`EXTERNAL_WIKI_SERVICE` | ExternalWikiService type. | | <a id="servicetypeflowdock_service"></a>`FLOWDOCK_SERVICE` | FlowdockService type. | | <a id="servicetypegithub_service"></a>`GITHUB_SERVICE` | GithubService type. | +| <a id="servicetypegitlab_slack_application_service"></a>`GITLAB_SLACK_APPLICATION_SERVICE` | GitlabSlackApplicationService type (Gitlab.com only). | | <a id="servicetypehangouts_chat_service"></a>`HANGOUTS_CHAT_SERVICE` | HangoutsChatService type. | +| <a id="servicetypeharbor_service"></a>`HARBOR_SERVICE` | HarborService type. | | <a id="servicetypeirker_service"></a>`IRKER_SERVICE` | IrkerService type. | | <a id="servicetypejenkins_service"></a>`JENKINS_SERVICE` | JenkinsService type. | | <a id="servicetypejira_service"></a>`JIRA_SERVICE` | JiraService type. | @@ -18110,6 +19007,15 @@ State of a test report. | <a id="todotargetenumissue"></a>`ISSUE` | Issue. | | <a id="todotargetenummergerequest"></a>`MERGEREQUEST` | Merge request. | +### `TrainingUrlRequestStatus` + +Status of the request to the training provider. The URL of a TrainingUrl is calculated asynchronously. When PENDING, the URL of the TrainingUrl will be null. When COMPLETED, the URL of the TrainingUrl will be available. + +| Value | Description | +| ----- | ----------- | +| <a id="trainingurlrequeststatuscompleted"></a>`COMPLETED` | Completed request. | +| <a id="trainingurlrequeststatuspending"></a>`PENDING` | Pending request. | + ### `TypeEnum` | Value | Description | @@ -18124,6 +19030,8 @@ Name of the feature that the callout is for. | Value | Description | | ----- | ----------- | | <a id="usercalloutfeaturenameenumactive_user_count_threshold"></a>`ACTIVE_USER_COUNT_THRESHOLD` | Callout feature name for active_user_count_threshold. | +| <a id="usercalloutfeaturenameenumattention_requests_side_nav"></a>`ATTENTION_REQUESTS_SIDE_NAV` | Callout feature name for attention_requests_side_nav. | +| <a id="usercalloutfeaturenameenumattention_requests_top_nav"></a>`ATTENTION_REQUESTS_TOP_NAV` | Callout feature name for attention_requests_top_nav. | | <a id="usercalloutfeaturenameenumbuy_pipeline_minutes_notification_dot"></a>`BUY_PIPELINE_MINUTES_NOTIFICATION_DOT` | Callout feature name for buy_pipeline_minutes_notification_dot. | | <a id="usercalloutfeaturenameenumcanary_deployment"></a>`CANARY_DEPLOYMENT` | Callout feature name for canary_deployment. | | <a id="usercalloutfeaturenameenumci_deprecation_warning_for_types_keyword"></a>`CI_DEPRECATION_WARNING_FOR_TYPES_KEYWORD` | Callout feature name for ci_deprecation_warning_for_types_keyword. | @@ -18146,6 +19054,10 @@ Name of the feature that the callout is for. | <a id="usercalloutfeaturenameenumsecurity_configuration_upgrade_banner"></a>`SECURITY_CONFIGURATION_UPGRADE_BANNER` | Callout feature name for security_configuration_upgrade_banner. | | <a id="usercalloutfeaturenameenumsecurity_newsletter_callout"></a>`SECURITY_NEWSLETTER_CALLOUT` | Callout feature name for security_newsletter_callout. | | <a id="usercalloutfeaturenameenumsecurity_training_feature_promotion"></a>`SECURITY_TRAINING_FEATURE_PROMOTION` | Callout feature name for security_training_feature_promotion. | +| <a id="usercalloutfeaturenameenumstorage_enforcement_banner_first_enforcement_threshold"></a>`STORAGE_ENFORCEMENT_BANNER_FIRST_ENFORCEMENT_THRESHOLD` | Callout feature name for storage_enforcement_banner_first_enforcement_threshold. | +| <a id="usercalloutfeaturenameenumstorage_enforcement_banner_fourth_enforcement_threshold"></a>`STORAGE_ENFORCEMENT_BANNER_FOURTH_ENFORCEMENT_THRESHOLD` | Callout feature name for storage_enforcement_banner_fourth_enforcement_threshold. | +| <a id="usercalloutfeaturenameenumstorage_enforcement_banner_second_enforcement_threshold"></a>`STORAGE_ENFORCEMENT_BANNER_SECOND_ENFORCEMENT_THRESHOLD` | Callout feature name for storage_enforcement_banner_second_enforcement_threshold. | +| <a id="usercalloutfeaturenameenumstorage_enforcement_banner_third_enforcement_threshold"></a>`STORAGE_ENFORCEMENT_BANNER_THIRD_ENFORCEMENT_THRESHOLD` | Callout feature name for storage_enforcement_banner_third_enforcement_threshold. | | <a id="usercalloutfeaturenameenumsuggest_pipeline"></a>`SUGGEST_PIPELINE` | Callout feature name for suggest_pipeline. | | <a id="usercalloutfeaturenameenumsuggest_popover_dismissed"></a>`SUGGEST_POPOVER_DISMISSED` | Callout feature name for suggest_popover_dismissed. | | <a id="usercalloutfeaturenameenumtabs_position_highlight"></a>`TABS_POSITION_HIGHLIGHT` | Callout feature name for tabs_position_highlight. | @@ -18863,6 +19775,12 @@ A `UserID` is a global ID. It is encoded as a string. An example `UserID` is: `"gid://gitlab/User/1"`. +### `UsersSavedReplyID` + +A `UsersSavedReplyID` is a global ID. It is encoded as a string. + +An example `UsersSavedReplyID` is: `"gid://gitlab/Users::SavedReply/1"`. + ### `VulnerabilitiesExternalIssueLinkID` A `VulnerabilitiesExternalIssueLinkID` is a global ID. It is encoded as a string. @@ -18893,6 +19811,9 @@ A `WorkItemID` is a global ID. It is encoded as a string. An example `WorkItemID` is: `"gid://gitlab/WorkItem/1"`. +While we transition from Issues into Work Items this type will temporarily support +`IssueID` like: `"gid://gitlab/Issue/1"`. This behavior will be removed without notice in the future. + ### `WorkItemsTypeID` A `WorkItemsTypeID` is a global ID. It is encoded as a string. @@ -19218,6 +20139,7 @@ Implementations: | Name | Type | Description | | ---- | ---- | ----------- | | <a id="serviceactive"></a>`active` | [`Boolean`](#boolean) | Indicates if the service is active. | +| <a id="serviceservicetype"></a>`serviceType` | [`ServiceType`](#servicetype) | Type of the service. | | <a id="servicetype"></a>`type` | [`String`](#string) | Class name of the service. | #### `TimeboxReportInterface` @@ -19227,11 +20149,38 @@ Implementations: - [`Iteration`](#iteration) - [`Milestone`](#milestone) +##### Fields with arguments + +###### `TimeboxReportInterface.report` + +Historically accurate report about the timebox. + +Returns [`TimeboxReport`](#timeboxreport). + +####### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="timeboxreportinterfacereportfullpath"></a>`fullPath` | [`String`](#string) | Full path of the project or group used as a scope for report. For example, `gitlab-org` or `gitlab-org/gitlab`. | + +#### `Todoable` + +Implementations: + +- [`AlertManagementAlert`](#alertmanagementalert) +- [`BoardEpic`](#boardepic) +- [`Commit`](#commit) +- [`Design`](#design) +- [`Epic`](#epic) +- [`EpicIssue`](#epicissue) +- [`Issue`](#issue) +- [`MergeRequest`](#mergerequest) + ##### Fields | Name | Type | Description | | ---- | ---- | ----------- | -| <a id="timeboxreportinterfacereport"></a>`report` | [`TimeboxReport`](#timeboxreport) | Historically accurate report about the timebox. | +| <a id="todoableweburl"></a>`webUrl` | [`String`](#string) | URL of this object. | #### `User` @@ -19240,6 +20189,8 @@ Representation of a GitLab user. Implementations: - [`MergeRequestAssignee`](#mergerequestassignee) +- [`MergeRequestAuthor`](#mergerequestauthor) +- [`MergeRequestParticipant`](#mergerequestparticipant) - [`MergeRequestReviewer`](#mergerequestreviewer) - [`UserCore`](#usercore) @@ -19251,14 +20202,18 @@ Implementations: | <a id="userbot"></a>`bot` | [`Boolean!`](#boolean) | Indicates if the user is a bot. | | <a id="usercallouts"></a>`callouts` | [`UserCalloutConnection`](#usercalloutconnection) | User callouts that belong to the user. (see [Connections](#connections)) | | <a id="useremail"></a>`email` **{warning-solid}** | [`String`](#string) | **Deprecated** in 13.7. This was renamed. Use: [`User.publicEmail`](#userpublicemail). | +| <a id="usergitpodenabled"></a>`gitpodEnabled` | [`Boolean`](#boolean) | Whether Gitpod is enabled at the user level. | | <a id="usergroupcount"></a>`groupCount` | [`Int`](#int) | Group count for the user. | | <a id="usergroupmemberships"></a>`groupMemberships` | [`GroupMemberConnection`](#groupmemberconnection) | Group memberships of the user. (see [Connections](#connections)) | | <a id="userid"></a>`id` | [`ID!`](#id) | ID of the user. | | <a id="userlocation"></a>`location` | [`String`](#string) | Location of the user. | | <a id="username"></a>`name` | [`String!`](#string) | Human-readable name of the user. Returns `****` if the user is a project bot and the requester does not have permission to view the project. | | <a id="usernamespace"></a>`namespace` | [`Namespace`](#namespace) | Personal namespace of the user. | +| <a id="userpreferencesgitpodpath"></a>`preferencesGitpodPath` | [`String`](#string) | Web path to the Gitpod section within user preferences. | +| <a id="userprofileenablegitpodpath"></a>`profileEnableGitpodPath` | [`String`](#string) | Web path to enable Gitpod for the user. | | <a id="userprojectmemberships"></a>`projectMemberships` | [`ProjectMemberConnection`](#projectmemberconnection) | Project memberships of the user. (see [Connections](#connections)) | | <a id="userpublicemail"></a>`publicEmail` | [`String`](#string) | User's public email. | +| <a id="usersavedreplies"></a>`savedReplies` | [`SavedReplyConnection`](#savedreplyconnection) | Saved replies authored by the user. (see [Connections](#connections)) | | <a id="userstate"></a>`state` | [`UserState!`](#userstate) | State of the user. | | <a id="userstatus"></a>`status` | [`UserStatus`](#userstatus) | User status. | | <a id="useruserpermissions"></a>`userPermissions` | [`UserPermissions!`](#userpermissions) | Permissions for the current user on the resource. | @@ -19538,6 +20493,16 @@ Field that are available while modifying the custom mapping attributes for an HT | <a id="complianceframeworkinputname"></a>`name` | [`String`](#string) | New name for the compliance framework. | | <a id="complianceframeworkinputpipelineconfigurationfullpath"></a>`pipelineConfigurationFullPath` | [`String`](#string) | Full path of the compliance pipeline configuration stored in a project repository, such as `.gitlab/.compliance-gitlab-ci.yml@compliance/hipaa` **(ULTIMATE)**. | +### `ComplianceViolationInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="complianceviolationinputmergedafter"></a>`mergedAfter` | [`Date`](#date) | Merged date of merge requests merged after a compliance violation was created. | +| <a id="complianceviolationinputmergedbefore"></a>`mergedBefore` | [`Date`](#date) | Merged date of merge requests merged before a compliance violation was created. | +| <a id="complianceviolationinputprojectids"></a>`projectIds` | [`[ProjectID!]`](#projectid) | Filter compliance violations by project. | + ### `DastProfileCadenceInput` Represents DAST Profile Cadence. @@ -19609,8 +20574,8 @@ Input type for DastSiteProfile authentication. | ---- | ---- | ----------- | | <a id="diffpositioninputbasesha"></a>`baseSha` | [`String`](#string) | Merge base of the branch the comment was made on. | | <a id="diffpositioninputheadsha"></a>`headSha` | [`String!`](#string) | SHA of the HEAD at the time the comment was made. | -| <a id="diffpositioninputnewline"></a>`newLine` | [`Int`](#int) | Line on HEAD SHA that was changed. | -| <a id="diffpositioninputoldline"></a>`oldLine` | [`Int`](#int) | Line on start SHA that was changed. | +| <a id="diffpositioninputnewline"></a>`newLine` | [`Int`](#int) | Line on HEAD SHA that was changed. Please see the [REST API Documentation](https://docs.gitlab.com/ee/api/discussions.html#create-a-new-thread-in-the-merge-request-diff) for more information on how to use this field. | +| <a id="diffpositioninputoldline"></a>`oldLine` | [`Int`](#int) | Line on start SHA that was changed. Please see the [REST API Documentation](https://docs.gitlab.com/ee/api/discussions.html#create-a-new-thread-in-the-merge-request-diff) for more information on how to use this field. | | <a id="diffpositioninputpaths"></a>`paths` | [`DiffPathsInput!`](#diffpathsinput) | The paths of the file that was changed. Both of the properties of this input are optional, but at least one of them is required. | | <a id="diffpositioninputstartsha"></a>`startSha` | [`String!`](#string) | SHA of the branch being compared against. | @@ -19902,3 +20867,15 @@ A time-frame defined as a closed inclusive range of two dates. | Name | Type | Description | | ---- | ---- | ----------- | | <a id="vulnerabilityscannervendorinputname"></a>`name` | [`String!`](#string) | Name of the vendor/maintainer. | + +### `WorkItemConvertTaskInput` + +#### Arguments + +| Name | Type | Description | +| ---- | ---- | ----------- | +| <a id="workitemconverttaskinputlinenumberend"></a>`lineNumberEnd` | [`Int!`](#int) | Last line in the Markdown source that defines the list item task. | +| <a id="workitemconverttaskinputlinenumberstart"></a>`lineNumberStart` | [`Int!`](#int) | First line in the Markdown source that defines the list item task. | +| <a id="workitemconverttaskinputlockversion"></a>`lockVersion` | [`Int!`](#int) | Current lock version of the work item containing the task in the description. | +| <a id="workitemconverttaskinputtitle"></a>`title` | [`String!`](#string) | Full string of the task to be replaced. New title for the created work item. | +| <a id="workitemconverttaskinputworkitemtypeid"></a>`workItemTypeId` | [`WorkItemsTypeID!`](#workitemstypeid) | Global ID of the work item type used to create the new work item. | diff --git a/doc/api/group_labels.md b/doc/api/group_labels.md index 96b8a162e34..e0f1b451231 100644 --- a/doc/api/group_labels.md +++ b/doc/api/group_labels.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/21368) in GitLab 11.8. -This API supports managing [group labels](../user/project/labels.md#project-labels-and-group-labels). +This API supports managing [group labels](../user/project/labels.md#types-of-labels). It allows users to list, create, update, and delete group labels. Furthermore, users can subscribe to and unsubscribe from group labels. diff --git a/doc/api/group_wikis.md b/doc/api/group_wikis.md index 4af907bd387..a78f989a755 100644 --- a/doc/api/group_wikis.md +++ b/doc/api/group_wikis.md @@ -7,7 +7,10 @@ type: reference, api # Group wikis API **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/212199) in GitLab 13.5. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/212199) in GitLab 13.5. +> - The `encoding` field was [added](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/81150) in GitLab 14.9. +> - The `render_html` attribute was [added](https://gitlab.com/gitlab-org/gitlab/-/issues/336792) in GitLab 14.9. +> - The `version` attribute was [added](https://gitlab.com/gitlab-org/gitlab/-/issues/336792) in GitLab 14.9. The [group wikis](../user/project/wiki/group.md) API is available only in APIv4. An API for [project wikis](wikis.md) is also available. @@ -37,18 +40,21 @@ Example response: "content" : "Here is an instruction how to deploy this project.", "format" : "markdown", "slug" : "deploy", - "title" : "deploy" + "title" : "deploy", + "encoding": "UTF-8" }, { "content" : "Our development process is described here.", "format" : "markdown", "slug" : "development", - "title" : "development" + "title" : "development", + "encoding": "UTF-8" },{ "content" : "* [Deploy](deploy)\n* [Development](development)", "format" : "markdown", "slug" : "home", - "title" : "home" + "title" : "home", + "encoding": "UTF-8" } ] ``` @@ -65,6 +71,8 @@ GET /groups/:id/wikis/:slug | --------- | ------- | -------- | --------------------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the group](index.md#namespaced-path-encoding) | | `slug` | string | yes | URL-encoded slug (a unique string) of the wiki page, such as `dir%2Fpage_name` | +| `render_html` | boolean | no | Return the rendered HTML of the wiki page | +| `version` | string | no | Wiki page version sha | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/1/wikis/home" @@ -77,7 +85,8 @@ Example response: "content" : "home page", "format" : "markdown", "slug" : "home", - "title" : "home" + "title" : "home", + "encoding": "UTF-8" } ``` @@ -109,7 +118,8 @@ Example response: "content" : "Hello world", "format" : "markdown", "slug" : "Hello", - "title" : "Hello" + "title" : "Hello", + "encoding": "UTF-8" } ``` @@ -142,7 +152,8 @@ Example response: "content" : "documentation", "format" : "markdown", "slug" : "Docs", - "title" : "Docs" + "title" : "Docs", + "encoding": "UTF-8" } ``` diff --git a/doc/api/groups.md b/doc/api/groups.md index 2e0f9526e16..120090c18a2 100644 --- a/doc/api/groups.md +++ b/doc/api/groups.md @@ -443,6 +443,7 @@ Example response: "builds_access_level":"enabled", "snippets_access_level":"enabled", "pages_access_level":"enabled", + "security_and_compliance_access_level":"enabled", "emails_disabled":null, "shared_runners_enabled":true, "lfs_enabled":true, @@ -483,7 +484,9 @@ Example response: ## Details of a group Get all details of a group. This endpoint can be accessed without authentication -if the group is publicly accessible. In case the user that requests is administrator of the group, it returns the `runners_token` for the group too. +if the group is publicly accessible. In case the user that requests is an administrator +if the group is publicly accessible. With authentication, it returns the `runners_token` +for the group too, if the user is an administrator or group owner. ```plaintext GET /groups/:id @@ -505,6 +508,11 @@ To get the details of all projects within a group, use either the [list a group' curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/4" ``` +NOTE: +There is [a known issue](https://gitlab.com/gitlab-org/gitlab/-/issues/345200) that can +prevent `runners_token` from being returned when the call has the `with_projects=false` +parameter. + This endpoint returns: - All projects and shared projects in GitLab 12.5 and earlier. @@ -795,7 +803,7 @@ Parameters: | `two_factor_grace_period` | integer | no | Time before Two-factor authentication is enforced (in hours). | | `project_creation_level` | string | no | Determine if developers can create projects in the group. Can be `noone` (No one), `maintainer` (users with the Maintainer role), or `developer` (users with the Developer or Maintainer role). | | `auto_devops_enabled` | boolean | no | Default to Auto DevOps pipeline for all projects within this group. | -| `subgroup_creation_level` | string | no | Allowed to [create subgroups](../user/group/subgroups/index.md#creating-a-subgroup). Can be `owner` (Owners), or `maintainer` (users with the Maintainer role). | +| `subgroup_creation_level` | string | no | Allowed to [create subgroups](../user/group/subgroups/index.md#create-a-subgroup). Can be `owner` (Owners), or `maintainer` (users with the Maintainer role). | | `emails_disabled` | boolean | no | Disable email notifications | | `avatar` | mixed | no | Image file for avatar of the group. [Introduced in GitLab 12.9](https://gitlab.com/gitlab-org/gitlab/-/issues/36681) | | `mentions_disabled` | boolean | no | Disable the capability of a group from getting mentioned | @@ -858,7 +866,7 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ Transfer a group to a new parent group or turn a subgroup to a top-level group. Available to administrators and users: - With the Owner role for the group to transfer. -- With permission to [create a subgroup](../user/group/subgroups/index.md#creating-a-subgroup) in the new parent group if transferring a group. +- With permission to [create a subgroup](../user/group/subgroups/index.md#create-a-subgroup) in the new parent group if transferring a group. - With [permission to create a top-level group](../administration/user_settings.md#prevent-users-from-creating-top-level-groups) if turning a subgroup into a top-level group. ```plaintext @@ -898,7 +906,7 @@ PUT /groups/:id | `two_factor_grace_period` | integer | no | Time before Two-factor authentication is enforced (in hours). | | `project_creation_level` | string | no | Determine if developers can create projects in the group. Can be `noone` (No one), `maintainer` (users with the Maintainer role), or `developer` (users with the Developer or Maintainer role). | | `auto_devops_enabled` | boolean | no | Default to Auto DevOps pipeline for all projects within this group. | -| `subgroup_creation_level` | string | no | Allowed to [create subgroups](../user/group/subgroups/index.md#creating-a-subgroup). Can be `owner` (Owners), or `maintainer` (users with the Maintainer role). | +| `subgroup_creation_level` | string | no | Allowed to [create subgroups](../user/group/subgroups/index.md#create-a-subgroup). Can be `owner` (Owners), or `maintainer` (users with the Maintainer role). | | `emails_disabled` | boolean | no | Disable email notifications | | `avatar` | mixed | no | Image file for avatar of the group. [Introduced in GitLab 12.9](https://gitlab.com/gitlab-org/gitlab/-/issues/36681) | | `mentions_disabled` | boolean | no | Disable the capability of a group from getting mentioned | diff --git a/doc/api/index.md b/doc/api/index.md index 589bc0416a1..178c2f05a6d 100644 --- a/doc/api/index.md +++ b/doc/api/index.md @@ -28,7 +28,7 @@ For an introduction and basic steps, see ## SCIM API **(PREMIUM SAAS)** -GitLab provides an [SCIM API](scim.md) that both implements +GitLab provides a [SCIM API](scim.md) that both implements [the RFC7644 protocol](https://tools.ietf.org/html/rfc7644) and provides the `/Users` endpoint. The base URL is `/api/scim/v2/groups/:group_path/Users/`. @@ -767,3 +767,35 @@ some API endpoints also support `text/plain`. In [GitLab 13.10 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/250342), API endpoints do not support `text/plain` by default, unless it's explicitly documented. + +## Resolve requests detected as spam + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352913) in GitLab 14.9. + +REST API requests can be detected as spam. If a request is detected as spam and: + +- A CAPTCHA service is not configured, an error response is returned. For example: + + ```json + {"message":{"error":"Your snippet has been recognized as spam and has been discarded."}} + ``` + +- A CAPTCHA service is configured, you receive a response with: + - `needs_captcha_response` set to `true`. + - The `spam_log_id` and `captcha_site_key` fields set. + + For example: + + ```json + {"needs_captcha_response":true,"spam_log_id":42,"captcha_site_key":"6LeIxAcTAAAAAJcZVRqyHh71UMIEGNQ_MXjiZKhI","message":{"error":"Your snippet has been recognized as spam. Please, change the content or solve the reCAPTCHA to proceed."}} + ``` + +- Use the `captcha_site_key` to obtain a CAPTCHA response value using the appropriate CAPTCHA API. + Only [Google reCAPTCHA v2](https://developers.google.com/recaptcha/docs/display) is supported. +- Resubmit the request with the `X-GitLab-Captcha-Response` and `X-GitLab-Spam-Log-Id` headers set. + +```shell +export CAPTCHA_RESPONSE="<CAPTCHA response obtained from CAPTCHA service>" +export SPAM_LOG_ID="<spam_log_id obtained from initial REST response>" +curl --request POST --header "PRIVATE-TOKEN: $PRIVATE_TOKEN" --header "X-GitLab-Captcha-Response: $CAPTCHA_RESPONSE" --header "X-GitLab-Spam-Log-Id: $SPAM_LOG_ID" "https://gitlab.example.com/api/v4/snippets?title=Title&file_name=FileName&content=Content&visibility=public" +``` diff --git a/doc/api/issues.md b/doc/api/issues.md index 5801f072062..ef0727e1c13 100644 --- a/doc/api/issues.md +++ b/doc/api/issues.md @@ -210,6 +210,28 @@ Issues created by users on GitLab Premium or higher include the `epic` property: } ``` +Issues created by users on GitLab Premium or higher include the `iteration` property: + +```json +{ + "iteration": { + "id":90, + "iid":4, + "sequence":2, + "group_id":162, + "title":null, + "description":null, + "state":2, + "created_at":"2022-03-14T05:21:11.929Z", + "updated_at":"2022-03-14T05:21:11.929Z", + "start_date":"2022-03-08", + "due_date":"2022-03-14", + "web_url":"http://gitlab.com/groups/my-group/-/iterations/90" + } + ... +} +``` + Issues created by users on GitLab Ultimate include the `health_status` property: ```json diff --git a/doc/api/jobs.md b/doc/api/jobs.md index 89018548f5f..85cdf7d892a 100644 --- a/doc/api/jobs.md +++ b/doc/api/jobs.md @@ -474,7 +474,7 @@ Example of response } ``` -## Get GitLab Agent by `CI_JOB_TOKEN` **(PREMIUM)** +## Get GitLab agent by `CI_JOB_TOKEN` **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/324269) in GitLab 13.11. @@ -814,7 +814,7 @@ NOTE: You can't delete archived jobs with the API, but you can [delete job artifacts and logs from jobs completed before a specific date](../administration/job_artifacts.md#delete-job-artifacts-and-logs-from-jobs-completed-before-a-specific-date) -## Play a job +## Run a job Triggers a manual action to start a job. @@ -822,16 +822,38 @@ Triggers a manual action to start a job. POST /projects/:id/jobs/:job_id/play ``` -| Attribute | Type | Required | Description | -|-----------|----------------|------------------------|-------------| -| `id` | integer/string | **{check-circle}** Yes | ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `job_id` | integer | **{check-circle}** Yes | ID of a job. | +| Attribute | Type | Required | Description | +|----------------------------|-----------------|------------------------|-------------| +| `id` | integer/string | **{check-circle}** Yes | ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `job_id` | integer | **{check-circle}** Yes | ID of a job. | +| `job_variables_attributes` | array of hashes | **{dotted-circle}** No | An array containing the custom variables available to the job. [Introduced in](https://gitlab.com/gitlab-org/gitlab/-/issues/37267) GitLab 14.9. | + +Example request: ```shell -curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/jobs/1/play" +curl --request POST "https://gitlab.example.com/api/v4/projects/1/jobs/1/play + --header "PRIVATE-TOKEN: <your_access_token>" + --data @variables.json ``` -Example of response +`@variables.json` is structured like: + +```json +{ + "job_variables_attributes": [ + { + "key": "TEST_VAR_1", + "value": "test1" + }, + { + "key": "TEST_VAR_2", + "value": "test2" + } + ] +} +``` + +Example response: ```json { diff --git a/doc/api/linked_epics.md b/doc/api/linked_epics.md new file mode 100644 index 00000000000..89168c344f3 --- /dev/null +++ b/doc/api/linked_epics.md @@ -0,0 +1,90 @@ +--- +stage: Plan +group: Product Planning +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Linked epics API **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352493) in GitLab 14.9 [with a flag](../administration/feature_flags.md) named `related_epics_widget`. Enabled by default. + +FLAG: +On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../administration/feature_flags.md) named `related_epics_widget`. On GitLab.com, this feature is available. + +If the Related Epics feature is not available in your GitLab plan, a `403` status code is returned. + +## List linked epics + +Get a list of a given epic's linked epics filtered according to the user authorizations. + +```plaintext +GET /groups/:id/epics/:epic_iid/related_epics +``` + +Supported attributes: + +| Attribute | Type | Required | Description | +| ---------- | -------------- | ---------------------- | ------------------------------------------------------------------------- | +| `epic_iid` | integer | **{check-circle}** Yes | Internal ID of a group's epic | +| `id` | integer/string | **{check-circle}** Yes | ID or [URL-encoded path of the group](index.md#namespaced-path-encoding). | + +Example request: + +```shell +curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/groups/:id/epics/:epic_iid/related_epics" +``` + +Example response: + +```json +[ + { + "id":2, + "iid":2, + "color":"#1068bf", + "text_color":"#FFFFFF", + "group_id":2, + "parent_id":null, + "parent_iid":null, + "title":"My title 2", + "description":null, + "confidential":false, + "author":{ + "id":3, + "username":"user3", + "name":"Sidney Jones4", + "state":"active", + "avatar_url":"https://www.gravatar.com/avatar/82797019f038ab535a84c6591e7bc936?s=80u0026d=identicon", + "web_url":"http://localhost/user3" + }, + "start_date":null, + "end_date":null, + "due_date":null, + "state":"opened", + "web_url":"http://localhost/groups/group1/-/epics/2", + "references":{ + "short":"u00262", + "relative":"u00262", + "full":"group1u00262" + }, + "created_at":"2022-03-10T18:35:24.479Z", + "updated_at":"2022-03-10T18:35:24.479Z", + "closed_at":null, + "labels":[ + + ], + "upvotes":0, + "downvotes":0, + "_links":{ + "self":"http://localhost/api/v4/groups/2/epics/2", + "epic_issues":"http://localhost/api/v4/groups/2/epics/2/issues", + "group":"http://localhost/api/v4/groups/2", + "parent":null + }, + "related_epic_link_id":1, + "link_type":"relates_to", + "link_created_at":"2022-03-10T18:35:24.496+00:00", + "link_updated_at":"2022-03-10T18:35:24.496+00:00" + } +] +``` diff --git a/doc/api/merge_request_approvals.md b/doc/api/merge_request_approvals.md index 6a0b66ac5dc..e569abd323e 100644 --- a/doc/api/merge_request_approvals.md +++ b/doc/api/merge_request_approvals.md @@ -58,9 +58,9 @@ POST /projects/:id/approvals | `id` | integer or string | yes | The ID or [URL-encoded path of a project](index.md#namespaced-path-encoding) | | `approvals_before_merge` | integer | no | How many approvals are required before an MR can be merged. Deprecated in 12.0 in favor of Approval Rules API. | | `reset_approvals_on_push` | boolean | no | Reset approvals on a new push | -| `disable_overriding_approvers_per_merge_request` | boolean | no | Allow/Disallow overriding approvers per MR | -| `merge_requests_author_approval` | boolean | no | Allow/Disallow authors from self approving merge requests; `true` means authors can self approve | -| `merge_requests_disable_committers_approval` | boolean | no | Allow/Disallow committers from self approving merge requests | +| `disable_overriding_approvers_per_merge_request` | boolean | no | Allow or prevent overriding approvers per MR | +| `merge_requests_author_approval` | boolean | no | Allow or prevent authors from self approving merge requests; `true` means authors can self approve | +| `merge_requests_disable_committers_approval` | boolean | no | Allow or prevent committers from self approving merge requests | | `require_password_to_approve` | boolean | no | Require approver to enter a password to authenticate before adding the approval | ```json diff --git a/doc/api/merge_requests.md b/doc/api/merge_requests.md index a54ea33aca8..0c065c0f2f5 100644 --- a/doc/api/merge_requests.md +++ b/doc/api/merge_requests.md @@ -609,13 +609,13 @@ GET /projects/:id/merge_requests/:merge_request_iid Parameters: -| Attribute | Type | Required | Description | -|----------------------------------|----------------|----------|------------------------------------------------------------------------------------------------------------------| +| Attribute | Type | Required | Description | +|----------------------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| | `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | -| `render_html` | boolean | no | If `true` response includes rendered HTML for title and description. | -| `include_diverged_commits_count` | boolean | no | If `true` response includes the commits behind the target branch. | -| `include_rebase_in_progress` | boolean | no | If `true` response includes whether a rebase operation is in progress. | +| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| `render_html` | boolean | no | If `true` response includes rendered HTML for title and description. | +| `include_diverged_commits_count` | boolean | no | If `true` response includes the commits behind the target branch. | +| `include_rebase_in_progress` | boolean | no | If `true` response includes whether a rebase operation is in progress. | ```json { @@ -775,8 +775,8 @@ the `approvals_before_merge` parameter: ### Single merge request response notes - The `merge_status` field may hold one of the following values: - - `unchecked`: We have not checked this yet. - - `checking`: We are currently checking if the merge request can be merged. + - `unchecked`: This merge request has not yet been checked. + - `checking`: This merge request is currently being checked to see if it can be merged. - `can_be_merged`: This merge request can be merged without conflict. - `cannot_be_merged`: There are merge conflicts between the source and target branches. - `cannot_be_merged_recheck`: Currently unchecked. Before the current changes, there were conflicts. @@ -803,10 +803,10 @@ GET /projects/:id/merge_requests/:merge_request_iid/participants Parameters: -| Attribute | Type | Required | Description | -|----------------------------------|----------------|----------|------------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | yes | The internal ID of the merge request. | ```json [ @@ -839,10 +839,10 @@ GET /projects/:id/merge_requests/:merge_request_iid/commits Parameters: -| Attribute | Type | Required | Description | -|----------------------------------|----------------|----------|------------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | yes | The internal ID of the merge request. | ```json [ @@ -886,11 +886,11 @@ GET /projects/:id/merge_requests/:merge_request_iid/changes Parameters: -| Attribute | Type | Required | Description | -|----------------------------------|----------------|----------|------------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | -| `access_raw_diffs` | boolean | no | Retrieve change diffs via Gitaly. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| `access_raw_diffs` | boolean | no | Retrieve change diffs via Gitaly. | ```json { @@ -1008,10 +1008,10 @@ GET /projects/:id/merge_requests/:merge_request_iid/pipelines Parameters: -| Attribute | Type | Required | Description | -|----------------------------------|----------------|----------|------------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | yes | The internal ID of the merge request. | ```json [ @@ -1044,10 +1044,10 @@ POST /projects/:id/merge_requests/:merge_request_iid/pipelines Parameters: -| Attribute | Type | Required | Description | -|----------------------------------|----------------|----------|------------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | yes | The internal ID of the merge request. | ```json { @@ -1445,10 +1445,10 @@ Only for administrators and project owners. Deletes the merge request in questio DELETE /projects/:id/merge_requests/:merge_request_iid ``` -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | yes | The internal ID of the merge request. | ```shell curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/4/merge_requests/85" @@ -1464,16 +1464,16 @@ PUT /projects/:id/merge_requests/:merge_request_iid/merge Parameters: -| Attribute | Type | Required | Description | -|--------------------------------|----------------|----------|------------------------------------------------------------------------------------------------------------------| +| Attribute | Type | Required | Description | +|--------------------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| | `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | -| `merge_commit_message` | string | no | Custom merge commit message. | -| `squash_commit_message` | string | no | Custom squash commit message. | -| `squash` | boolean | no | If `true` the commits are squashed into a single commit on merge. | -| `should_remove_source_branch` | boolean | no | If `true` removes the source branch. | -| `merge_when_pipeline_succeeds` | boolean | no | If `true` the MR is merged when the pipeline succeeds. | -| `sha` | string | no | If present, then this SHA must match the HEAD of the source branch, otherwise the merge fails. | +| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| `merge_commit_message` | string | no | Custom merge commit message. | +| `squash_commit_message` | string | no | Custom squash commit message. | +| `squash` | boolean | no | If `true` the commits are squashed into a single commit on merge. | +| `should_remove_source_branch` | boolean | no | If `true` removes the source branch. | +| `merge_when_pipeline_succeeds` | boolean | no | If `true` the MR is merged when the pipeline succeeds. | +| `sha` | string | no | If present, then this SHA must match the HEAD of the source branch, otherwise the merge fails. | ```json { @@ -1625,12 +1625,12 @@ the `approvals_before_merge` parameter: This API returns specific HTTP status codes on failure: -| HTTP Status | Message | Reason | -| :---: | ------- | ------ | -| `401` | `Unauthorized` | This user does not have permission to accept this merge request. | -| `405` | `Method Not Allowed` | The merge request cannot be accepted because it is `Draft`, `Closed`, `Pipeline Pending Completion`, or `Failed`. `Success` is required. | -| `406` | `Branch cannot be merged` | The branch has conflicts and cannot be merged. | -| `409` | `SHA does not match HEAD of source branch` | The provided `sha` parameter does not match the HEAD of the source. | +| HTTP Status | Message | Reason | +|:------------|--------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------| +| `401` | `Unauthorized` | This user does not have permission to accept this merge request. | +| `405` | `Method Not Allowed` | The merge request cannot be accepted because it is `Draft`, `Closed`, `Pipeline Pending Completion`, or `Failed`. `Success` is required. | +| `406` | `Branch cannot be merged` | The branch has conflicts and cannot be merged. | +| `409` | `SHA does not match HEAD of source branch` | The provided `sha` parameter does not match the HEAD of the source. | For additional important notes on response data, read [Single merge request response notes](#single-merge-request-response-notes). @@ -1655,10 +1655,10 @@ GET /projects/:id/merge_requests/:merge_request_iid/merge_ref Parameters: -| Attribute | Type | Required | Description | -|--------------------------------|----------------|----------|------------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | yes | The internal ID of the merge request. | ```json { @@ -1668,9 +1668,13 @@ Parameters: ## Cancel Merge When Pipeline Succeeds -- If you don't have permissions to accept this merge request - you receive a `HTTP 401 Unauthorized`. -- If the merge request is already merged or closed - you receive a `HTTP 405 Method Not Allowed` and the error message 'Method Not Allowed'. -- In case the merge request is not set to be merged when the pipeline succeeds, you also receive a `HTTP 406 Not Acceptable` error. +This API returns specific HTTP status codes on failure: + +| HTTP Status | Message | Reason | +|-------------|----------------------|-----------------------------------------------------------------------| +| `401` | `Unauthorized` | This user does not have permission to cancel this merge request. | +| `405` | `Method Not Allowed` | The merge request is already merged or closed. | +| `406` | `Not Acceptable` | The merge request is not set to be merged when the pipeline succeeds. | ```plaintext POST /projects/:id/merge_requests/:merge_request_iid/cancel_merge_when_pipeline_succeeds @@ -1678,10 +1682,10 @@ POST /projects/:id/merge_requests/:merge_request_iid/cancel_merge_when_pipeline_ Parameters: -| Attribute | Type | Required | Description | -|--------------------------------|----------------|----------|------------------------------------------------------------------------------------------------------------------| -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | yes | The internal ID of the merge request. | ```json { @@ -1845,11 +1849,11 @@ you receive a `403 Forbidden` response. PUT /projects/:id/merge_requests/:merge_request_iid/rebase ``` -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | -| `skip_ci` | boolean | no | Set to `true` to skip creating a CI pipeline. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| `skip_ci` | boolean | no | Set to `true` to skip creating a CI pipeline. | ```shell curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/76/merge_requests/1/rebase" @@ -1908,10 +1912,10 @@ Get all the issues that would be closed by merging the provided merge request. GET /projects/:id/merge_requests/:merge_request_iid/closes_issues ``` -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | yes | The internal ID of the merge request. | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/76/merge_requests/1/closes_issues" @@ -1984,10 +1988,10 @@ status code `HTTP 304 Not Modified` is returned. POST /projects/:id/merge_requests/:merge_request_iid/subscribe ``` -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | yes | The internal ID of the merge request. | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/merge_requests/17/subscribe" @@ -2154,10 +2158,10 @@ not subscribed to the merge request, the status code `HTTP 304 Not Modified` is POST /projects/:id/merge_requests/:merge_request_iid/unsubscribe ``` -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | yes | The internal ID of the merge request. | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/merge_requests/17/unsubscribe" @@ -2324,10 +2328,10 @@ status code `HTTP 304 Not Modified` is returned. POST /projects/:id/merge_requests/:merge_request_iid/todo ``` -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | yes | The internal ID of the merge request. | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/merge_requests/27/todo" @@ -2451,9 +2455,9 @@ read [SHAs in the API response](#shas-in-the-api-response). GET /projects/:id/merge_requests/:merge_request_iid/versions ``` -| Attribute | Type | Required | Description | -| --------- | ------- | -------- | --------------------- | -| `id` | String | yes | The ID of the project. | +| Attribute | Type | Required | Description | +|---------------------|---------|----------|---------------------------------------| +| `id` | String | yes | The ID of the project. | | `merge_request_iid` | integer | yes | The internal ID of the merge request. | ```shell @@ -2486,9 +2490,11 @@ Example response: ### SHAs in the API response -- `head_commit_sha`: The HEAD commit of the source branch. -- `base_commit_sha`: The merge-base commit SHA between the source branch and the target branches. -- `start_commit_sha`: The HEAD commit SHA of the target branch when this version of the diff was created. +| SHA field | Purpose | +|--------------------|-------------------------------------------------------------------------------------| +| `head_commit_sha` | The HEAD commit of the source branch. | +| `base_commit_sha` | The merge-base commit SHA between the source branch and the target branches. | +| `start_commit_sha` | The HEAD commit SHA of the target branch when this version of the diff was created. | ## Get a single MR diff version @@ -2499,8 +2505,8 @@ read [SHAs in the API response](#shas-in-the-api-response). GET /projects/:id/merge_requests/:merge_request_iid/versions/:version_id ``` -| Attribute | Type | Required | Description | -| --------- | ------- | -------- | --------------------- | +| Attribute | Type | Required | Description | +|---------------------|---------|----------|-------------------------------------------| | `id` | String | yes | The ID of the project. | | `merge_request_iid` | integer | yes | The internal ID of the merge request. | | `version_id` | integer | yes | The ID of the merge request diff version. | @@ -2567,11 +2573,11 @@ Sets an estimated time of work for this merge request. POST /projects/:id/merge_requests/:merge_request_iid/time_estimate ``` -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | -| `duration` | string | yes | The duration in human format, such as `3h30m`. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| `duration` | string | yes | The duration in human format, such as `3h30m`. | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/merge_requests/93/time_estimate?duration=3h30m" @@ -2596,10 +2602,10 @@ Resets the estimated time for this merge request to 0 seconds. POST /projects/:id/merge_requests/:merge_request_iid/reset_time_estimate ``` -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of a project's merge_request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | yes | The internal ID of a project's merge_request. | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/merge_requests/93/reset_time_estimate" @@ -2624,12 +2630,12 @@ Adds spent time for this merge request. POST /projects/:id/merge_requests/:merge_request_iid/add_spent_time ``` -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | -| `duration` | string | yes | The duration in human format, such as `3h30m` | -| `summary` | string | no | A summary of how the time was spent. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| `duration` | string | yes | The duration in human format, such as `3h30m` | +| `summary` | string | no | A summary of how the time was spent. | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/merge_requests/93/add_spent_time?duration=1h" @@ -2654,10 +2660,10 @@ Resets the total spent time for this merge request to 0 seconds. POST /projects/:id/merge_requests/:merge_request_iid/reset_spent_time ``` -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of a project's merge_request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | yes | The internal ID of a project's merge_request. | ```shell curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/merge_requests/93/reset_spent_time" @@ -2680,10 +2686,10 @@ Example response: GET /projects/:id/merge_requests/:merge_request_iid/time_stats ``` -| Attribute | Type | Required | Description | -| --------- | ---- | -------- | ----------- | -| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | -| `merge_request_iid` | integer | yes | The internal ID of the merge request. | +| Attribute | Type | Required | Description | +|---------------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------| +| `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `merge_request_iid` | integer | yes | The internal ID of the merge request. | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/5/merge_requests/93/time_stats" diff --git a/doc/api/notes.md b/doc/api/notes.md index 445940e02fc..83631c70f8a 100644 --- a/doc/api/notes.md +++ b/doc/api/notes.md @@ -399,10 +399,11 @@ Parameters: | Attribute | Type | Required | Description | |---------------------|----------------|----------|------------------------------------------------------------------------------------------------------------------------------| -| `id` | integer or string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) | -| `merge_request_iid` | integer | yes | The IID of a project merge request | -| `body` | string | yes | The content of a note. Limited to 1,000,000 characters. | -| `created_at` | string | no | Date time string, ISO 8601 formatted. Example: `2016-03-11T03:45:40Z` (requires administrator or project/group owner rights) | +| `id` | integer or string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) | +| `merge_request_iid` | integer | yes | The IID of a project merge request | +| `body` | string | yes | The content of a note. Limited to 1,000,000 characters. | +| `created_at` | string | no | Date time string, ISO 8601 formatted. Example: `2016-03-11T03:45:40Z` (requires administrator or project/group owner rights) | +| `merge_request_diff_sha`| string | no | The SHA of the head commit which is used to ensure that the merge request hasn't been updated since the API request was sent. This is required for the /merge quick action | ### Modify existing merge request note diff --git a/doc/api/oauth2.md b/doc/api/oauth2.md index 59a929e30f4..7b38ac39b96 100644 --- a/doc/api/oauth2.md +++ b/doc/api/oauth2.md @@ -2,7 +2,7 @@ type: reference, howto stage: Manage group: Authentication and Authorization -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # OAuth 2.0 identity provider API **(FREE)** @@ -79,7 +79,7 @@ The Authorization code with PKCE flow, PKCE for short, makes it possible to secu the OAuth exchange of client credentials for access tokens on public clients without requiring access to the _Client Secret_ at all. This makes the PKCE flow advantageous for single page JavaScript applications or other client side apps where keeping secrets -from the user is a technical impossibility. +from the user is a technical impossibility. Before starting the flow, generate the `STATE`, the `CODE_VERIFIER` and the `CODE_CHALLENGE`. diff --git a/doc/api/packages/pypi.md b/doc/api/packages/pypi.md index 592b976da59..5ed162a3d05 100644 --- a/doc/api/packages/pypi.md +++ b/doc/api/packages/pypi.md @@ -175,6 +175,7 @@ PUT projects/:id/packages/pypi | Attribute | Type | Required | Description | | --------- | ---- | -------- | ----------- | | `id` | string | yes | The ID or full path of the project. | +| `requires_python` | string | no | The PyPI required version. | ```shell curl --request PUT \ diff --git a/doc/api/project_import_export.md b/doc/api/project_import_export.md index 218036b1ee0..46fa05d6cbc 100644 --- a/doc/api/project_import_export.md +++ b/doc/api/project_import_export.md @@ -246,6 +246,61 @@ curl --request POST \ The `Content-Length` header must return a valid number. The maximum file size is 10 gigabytes. The `Content-Type` header must be `application/gzip`. +## Import a file from AWS S3 + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/348874) in GitLab 14.9 in [Beta](https://about.gitlab.com/handbook/product/gitlab-the-product/#beta), [with a flag](../administration/feature_flags.md) named `import_project_from_remote_file_s3`. Disabled by default. + +FLAG: +On self-managed GitLab and GitLab.com, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `import_project_from_remote_file_s3`. This feature is not ready for production use. + +```plaintext +POST /projects/remote-import-s3 +``` + +| Attribute | Type | Required | Description | +| ------------------- | -------------- | -------- | ---------------------------------------- | +| `namespace` | integer/string | no | The ID or path of the namespace to import the project to. Defaults to the current user's namespace. | +| `name` | string | no | The name of the project to import. If not provided, defaults to the path of the project. | +| `region` | string | yes | [AWS S3 region name where the file is stored.](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html#Regions) | +| `bucket_name` | string | yes | [AWS S3 bucket name where the file is stored.](https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucketnamingrules.html) | +| `file_key` | string | yes | [AWS S3 file key to identify the file.](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingObjects.html) | +| `access_key_id` | string | yes | [AWS S3 access key ID.](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys). | +| `secret_access_key` | string | yes | [AWS S3 secret access key.](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys) | + +The passed override parameters take precedence over all values defined in the export file. + +```shell +curl --request POST \ + --url "http://localhost:3000/api/v4/projects/remote-import-s3" \ + --header "PRIVATE-TOKEN: <your gitlab access key>" \ + --header 'Content-Type: application/json' \ + --data '{ + "name": "Sample Project", + "path": "sample-project", + "region": "<Your S3 region name>", + "bucket_name": "<Your S3 bucket name>", + "file_key": "<Your S3 file key>", + "access_key_id": "<Your AWS access key id>", + "secret_access_key": "<Your AWS secret access key>" +}' +``` + +```json +{ + "id": 1, + "description": null, + "name": "Sample project", + "name_with_namespace": "Administrator / sample-project", + "path": "sample-project", + "path_with_namespace": "root/sample-project", + "created_at": "2018-02-13T09:05:58.023Z", + "import_status": "scheduled", + "correlation_id": "mezklWso3Za", + "failed_relations": [], + "import_error": null +} +``` + ## Import status Get the status of an import. diff --git a/doc/api/project_snippets.md b/doc/api/project_snippets.md index efbd8bf9431..90bd2cd6f83 100644 --- a/doc/api/project_snippets.md +++ b/doc/api/project_snippets.md @@ -15,7 +15,7 @@ Constants for snippet visibility levels are: | visibility | Description | | ---------- | ----------- | -| `private` | The snippet is visible only to the snippet creator | +| `private` | The snippet is visible only to project members | | `internal` | The snippet is visible for any logged in user except [external users](../user/permissions.md#external-users) | | `public` | The snippet can be accessed without any authentication | diff --git a/doc/api/projects.md b/doc/api/projects.md index db8d2361439..ee649377745 100644 --- a/doc/api/projects.md +++ b/doc/api/projects.md @@ -151,6 +151,7 @@ When the user is authenticated and `simple` is not set this returns something li "resolve_outdated_diff_discussions": false, "container_registry_enabled": false, // deprecated, use container_registry_access_level instead "container_registry_access_level": "disabled", + "security_and_compliance_access_level": "disabled", "created_at": "2013-09-30T13:46:02Z", "last_activity_at": "2013-09-30T13:46:02Z", "creator_id": 3, @@ -245,6 +246,7 @@ When the user is authenticated and `simple` is not set this returns something li "resolve_outdated_diff_discussions": false, "container_registry_enabled": false, // deprecated, use container_registry_access_level instead "container_registry_access_level": "disabled", + "security_and_compliance_access_level": "disabled", "created_at": "2013-09-30T13:46:02Z", "last_activity_at": "2013-09-30T13:46:02Z", "creator_id": 3, @@ -440,6 +442,7 @@ GET /users/:user_id/projects "resolve_outdated_diff_discussions": false, "container_registry_enabled": false, // deprecated, use container_registry_access_level instead "container_registry_access_level": "disabled", + "security_and_compliance_access_level": "disabled", "created_at": "2013-09-30T13:46:02Z", "last_activity_at": "2013-09-30T13:46:02Z", "creator_id": 3, @@ -534,6 +537,7 @@ GET /users/:user_id/projects "resolve_outdated_diff_discussions": false, "container_registry_enabled": false, // deprecated, use container_registry_access_level instead "container_registry_access_level": "disabled", + "security_and_compliance_access_level": "disabled", "created_at": "2013-09-30T13:46:02Z", "last_activity_at": "2013-09-30T13:46:02Z", "creator_id": 3, @@ -688,6 +692,7 @@ Example response: "resolve_outdated_diff_discussions": false, "container_registry_enabled": false, // deprecated, use container_registry_access_level instead "container_registry_access_level": "disabled", + "security_and_compliance_access_level": "disabled", "created_at": "2013-09-30T13:46:02Z", "last_activity_at": "2013-09-30T13:46:02Z", "creator_id": 3, @@ -777,6 +782,7 @@ Example response: "resolve_outdated_diff_discussions": false, "container_registry_enabled": false, // deprecated, use container_registry_access_level instead "container_registry_access_level": "disabled", + "security_and_compliance_access_level": "disabled", "created_at": "2013-09-30T13:46:02Z", "last_activity_at": "2013-09-30T13:46:02Z", "creator_id": 3, @@ -910,6 +916,7 @@ GET /projects/:id "resolve_outdated_diff_discussions": false, "container_registry_enabled": false, // deprecated, use container_registry_access_level instead "container_registry_access_level": "disabled", + "security_and_compliance_access_level": "disabled", "container_expiration_policy": { "cadence": "7d", "enabled": false, @@ -1207,7 +1214,7 @@ POST /projects | Attribute | Type | Required | Description | |-------------------------------------------------------------|---------|------------------------|-------------| | `name` | string | **{check-circle}** Yes (if path isn't provided) | The name of the new project. Equals path if not provided. | -| `path` | string | **{check-circle}** Yes (if name isn't provided) | Repository name for new project. Generated based on name if not provided (generated as lowercase with dashes). | +| `path` | string | **{check-circle}** Yes (if name isn't provided) | Repository name for new project. Generated based on name if not provided (generated as lowercase with dashes). Starting with GitLab 14.9, path must not start or end with a special character and must not contain consecutive special characters. | | `allow_merge_on_skipped_pipeline` | boolean | **{dotted-circle}** No | Set whether or not merge requests can be merged with skipped jobs. | | `analytics_access_level` | string | **{dotted-circle}** No | One of `disabled`, `private` or `enabled` | | `approvals_before_merge` **(PREMIUM)** | integer | **{dotted-circle}** No | How many approvers should approve merge requests by default. To configure approval rules, see [Merge request approvals API](merge_request_approvals.md). | @@ -1257,6 +1264,7 @@ POST /projects | `request_access_enabled` | boolean | **{dotted-circle}** No | Allow users to request member access. | | `requirements_access_level` | string | **{dotted-circle}** No | One of `disabled`, `private` or `enabled` | | `resolve_outdated_diff_discussions` | boolean | **{dotted-circle}** No | Automatically resolve merge request diffs discussions on lines changed with a push. | +| `security_and_compliance_access_level` | string | **{dotted-circle}** No | (GitLab 14.9 and later) Security and compliance access level. One of `disabled`, `private`, or `enabled`. | | `shared_runners_enabled` | boolean | **{dotted-circle}** No | Enable shared runners for this project. | | `snippets_access_level` | string | **{dotted-circle}** No | One of `disabled`, `private`, or `enabled`. | | `snippets_enabled` | boolean | **{dotted-circle}** No | _(Deprecated)_ Enable snippets for this project. Use `snippets_access_level` instead. | @@ -1334,6 +1342,7 @@ POST /projects/user/:user_id | `request_access_enabled` | boolean | **{dotted-circle}** No | Allow users to request member access. | | `requirements_access_level` | string | **{dotted-circle}** No | One of `disabled`, `private`, `enabled` or `public` | | `resolve_outdated_diff_discussions` | boolean | **{dotted-circle}** No | Automatically resolve merge request diffs discussions on lines changed with a push. | +| `security_and_compliance_access_level` | string | **{dotted-circle}** No | (GitLab 14.9 and later) Security and compliance access level. One of `disabled`, `private`, or `enabled`. | | `shared_runners_enabled` | boolean | **{dotted-circle}** No | Enable shared runners for this project. | | `snippets_access_level` | string | **{dotted-circle}** No | One of `disabled`, `private`, or `enabled`. | | `snippets_enabled` | boolean | **{dotted-circle}** No | _(Deprecated)_ Enable snippets for this project. Use `snippets_access_level` instead. | @@ -1433,6 +1442,7 @@ Supported attributes: | `requirements_access_level` | string | **{dotted-circle}** No | One of `disabled`, `private`, `enabled` or `public` | | `resolve_outdated_diff_discussions` | boolean | **{dotted-circle}** No | Automatically resolve merge request diffs discussions on lines changed with a push. | | `restrict_user_defined_variables` | boolean | **{dotted-circle}** No | Allow only users with the Maintainer role to pass user-defined variables when triggering a pipeline. For example when the pipeline is triggered in the UI, with the API, or by a trigger token. | +| `security_and_compliance_access_level` | string | **{dotted-circle}** No | (GitLab 14.9 and later) Security and compliance access level. One of `disabled`, `private`, or `enabled`. | | `service_desk_enabled` | boolean | **{dotted-circle}** No | Enable or disable Service Desk feature. | | `shared_runners_enabled` | boolean | **{dotted-circle}** No | Enable shared runners for this project. | | `snippets_access_level` | string | **{dotted-circle}** No | One of `disabled`, `private`, or `enabled`. | @@ -1536,6 +1546,7 @@ Example responses: "resolve_outdated_diff_discussions": false, "container_registry_enabled": false, // deprecated, use container_registry_access_level instead "container_registry_access_level": "disabled", + "security_and_compliance_access_level": "disabled", "created_at": "2013-09-30T13:46:02Z", "last_activity_at": "2013-09-30T13:46:02Z", "creator_id": 3, @@ -1630,6 +1641,7 @@ Example response: "resolve_outdated_diff_discussions": false, "container_registry_enabled": false, // deprecated, use container_registry_access_level instead "container_registry_access_level": "disabled", + "security_and_compliance_access_level": "disabled", "created_at": "2013-09-30T13:46:02Z", "last_activity_at": "2013-09-30T13:46:02Z", "creator_id": 3, @@ -1730,6 +1742,7 @@ Example response: "resolve_outdated_diff_discussions": false, "container_registry_enabled": false, // deprecated, use container_registry_access_level instead "container_registry_access_level": "disabled", + "security_and_compliance_access_level": "disabled", "created_at": "2013-09-30T13:46:02Z", "last_activity_at": "2013-09-30T13:46:02Z", "creator_id": 3, @@ -1910,6 +1923,7 @@ Example response: "resolve_outdated_diff_discussions": false, "container_registry_enabled": false, // deprecated, use container_registry_access_level instead "container_registry_access_level": "disabled", + "security_and_compliance_access_level": "disabled", "created_at": "2013-09-30T13:46:02Z", "last_activity_at": "2013-09-30T13:46:02Z", "creator_id": 3, @@ -2031,6 +2045,7 @@ Example response: "resolve_outdated_diff_discussions": false, "container_registry_enabled": false, // deprecated, use container_registry_access_level instead "container_registry_access_level": "disabled", + "security_and_compliance_access_level": "disabled", "created_at": "2013-09-30T13:46:02Z", "last_activity_at": "2013-09-30T13:46:02Z", "creator_id": 3, @@ -2691,6 +2706,7 @@ Example response: "builds_access_level": "enabled", "snippets_access_level": "enabled", "pages_access_level": "enabled", + "security_and_compliance_access_level": "enabled", "emails_disabled": null, "shared_runners_enabled": true, "lfs_enabled": true, @@ -2758,6 +2774,7 @@ with the API scope enabled. |--------------|---------|------------------------|-------------| | `import_url` | string | **{check-circle}** Yes | URL of remote repository being mirrored (with `user:token` if needed). | | `mirror` | boolean | **{check-circle}** Yes | Enables pull mirroring on project when set to `true`. | +| `mirror_trigger_builds`| boolean | **{dotted-circle}** No | Trigger pipelines for mirror updates when set to `true`. | | `only_mirror_protected_branches`| boolean | **{dotted-circle}** No | Limits mirroring to only protected branches when set to `true`. | ## Start the pull mirroring process for a Project **(PREMIUM)** diff --git a/doc/api/resource_access_tokens.md b/doc/api/resource_access_tokens.md index c77a8f5d0d6..a9c43625193 100644 --- a/doc/api/resource_access_tokens.md +++ b/doc/api/resource_access_tokens.md @@ -6,4 +6,6 @@ remove_date: '2022-04-06' This document was moved to [another location](project_access_tokens.md). <!-- This redirect file can be deleted after <2022-04-06>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/api/runners.md b/doc/api/runners.md index 3423a02c078..7437860239e 100644 --- a/doc/api/runners.md +++ b/doc/api/runners.md @@ -15,7 +15,7 @@ There are two tokens to take into account when connecting a runner with GitLab. | Token | Description | | ----- | ----------- | | Registration token | Token used to [register the runner](https://docs.gitlab.com/runner/register/). It can be [obtained through GitLab](../ci/runners/index.md). | -| Authentication token | Token used to authenticate the runner with the GitLab instance. It is obtained automatically when you [register a runner](https://docs.gitlab.com/runner/register/) or by the Runner API when you manually [register a runner](#register-a-new-runner) or [reset the authentication token](#reset-runners-authentication-token). | +| Authentication token | Token used to authenticate the runner with the GitLab instance. It is obtained automatically when you [register a runner](https://docs.gitlab.com/runner/register/) or by the Runner API when you manually [register a runner](#register-a-new-runner) or [reset the authentication token](#reset-runners-authentication-token-by-using-the-runner-id). | Here's an example of how the two tokens are used in runner registration: @@ -48,7 +48,7 @@ GET /runners?tag_list=tag1,tag2 |------------|--------------|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `scope` | string | no | Deprecated: Use `type` or `status` instead. The scope of specific runners to show, one of: `active`, `paused`, `online` and `offline`; showing all runners if none provided | | `type` | string | no | The type of runners to show, one of: `instance_type`, `group_type`, `project_type` | -| `status` | string | no | The status of runners to show, one of: `online` and `offline`. `active` and `paused` are also possible values which were deprecated in GitLab 14.8 and will be removed in GitLab 15.0 | +| `status` | string | no | The status of runners to show, one of: `online` and `offline`. `active` and `paused` are also possible values which were deprecated in GitLab 14.8 and will be removed in GitLab 16.0 | | `paused` | boolean | no | Whether to include only runners that are accepting or ignoring new jobs | | `tag_list` | string array | no | List of the runner's tags | @@ -58,11 +58,11 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a NOTE: The `active` and `paused` values in the `status` query parameter were deprecated [in GitLab 14.8](https://gitlab.com/gitlab-org/gitlab/-/issues/347211). -and will be removed in [GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/351109). They are replaced by the `paused` query parameter. +and will be removed in [GitLab 16.0](https://gitlab.com/gitlab-org/gitlab/-/issues/351109). They are replaced by the `paused` query parameter. NOTE: The `active` attribute in the response was deprecated [in GitLab 14.8](https://gitlab.com/gitlab-org/gitlab/-/issues/347211). -and will be removed in [GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/351109). It is replaced by the `paused` attribute. +and will be removed in [GitLab 16.0](https://gitlab.com/gitlab-org/gitlab/-/issues/351109). It is replaced by the `paused` attribute. Example response: @@ -113,7 +113,7 @@ GET /runners/all?tag_list=tag1,tag2 |------------|--------------|----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `scope` | string | no | Deprecated: Use `type` or `status` instead. The scope of runners to show, one of: `specific`, `shared`, `active`, `paused`, `online` and `offline`; showing all runners if none provided | | `type` | string | no | The type of runners to show, one of: `instance_type`, `group_type`, `project_type` | -| `status` | string | no | The status of runners to show, one of: `online` and `offline`. `active` and `paused` are also possible values which were deprecated in GitLab 14.8 and will be removed in GitLab 15.0 | +| `status` | string | no | The status of runners to show, one of: `online` and `offline`. `active` and `paused` are also possible values which were deprecated in GitLab 14.8 and will be removed in GitLab 16.0 | | `paused` | boolean | no | Whether to include only runners that are accepting or ignoring new jobs | | `tag_list` | string array | no | List of the runner's tags | @@ -123,11 +123,11 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a NOTE: The `active` and `paused` values in the `status` query parameter were deprecated [in GitLab 14.8](https://gitlab.com/gitlab-org/gitlab/-/issues/347211). -and will be removed in [GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/351109). They are replaced by the `paused` query parameter. +and will be removed in [GitLab 16.0](https://gitlab.com/gitlab-org/gitlab/-/issues/351109). They are replaced by the `paused` query parameter. NOTE: The `active` attribute in the response was deprecated [in GitLab 14.8](https://gitlab.com/gitlab-org/gitlab/-/issues/347211). -and will be removed in [GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/351109). It is replaced by the `paused` attribute. +and will be removed in [GitLab 16.0](https://gitlab.com/gitlab-org/gitlab/-/issues/351109). It is replaced by the `paused` attribute. Example response: @@ -213,7 +213,7 @@ and removed in [GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/issues/21432 NOTE: The `active` attribute in the response was deprecated [in GitLab 14.8](https://gitlab.com/gitlab-org/gitlab/-/issues/347211). -and will be removed in [GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/351109). It is replaced by the `paused` attribute. +and will be removed in [GitLab 16.0](https://gitlab.com/gitlab-org/gitlab/-/issues/351109). It is replaced by the `paused` attribute. Example response: @@ -283,7 +283,7 @@ and [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/214322) in GitLab 13 NOTE: The `active` query parameter was deprecated [in GitLab 14.8](https://gitlab.com/gitlab-org/gitlab/-/issues/347211). -and will be removed in [GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/351109). It is replaced by the `paused` attribute. +and will be removed in [GitLab 16.0](https://gitlab.com/gitlab-org/gitlab/-/issues/351109). It is replaced by the `paused` attribute. Example response: @@ -353,7 +353,7 @@ curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>" \ NOTE: The `active` form attribute was deprecated [in GitLab 14.8](https://gitlab.com/gitlab-org/gitlab/-/issues/347211). -and will be removed in [GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/351109). It is replaced by the `paused` attribute. +and will be removed in [GitLab 16.0](https://gitlab.com/gitlab-org/gitlab/-/issues/351109). It is replaced by the `paused` attribute. ## List runner's jobs @@ -464,7 +464,7 @@ GET /projects/:id/runners?tag_list=tag1,tag2 | `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user | | `scope` | string | no | Deprecated: Use `type` or `status` instead. The scope of specific runners to show, one of: `active`, `paused`, `online` and `offline`; showing all runners if none provided | | `type` | string | no | The type of runners to show, one of: `instance_type`, `group_type`, `project_type` | -| `status` | string | no | The status of runners to show, one of: `online` and `offline`. `active` and `paused` are also possible values which were deprecated in GitLab 14.8 and will be removed in GitLab 15.0 | +| `status` | string | no | The status of runners to show, one of: `online` and `offline`. `active` and `paused` are also possible values which were deprecated in GitLab 14.8 and will be removed in GitLab 16.0 | | `paused` | boolean | no | Whether to include only runners that are accepting or ignoring new jobs | | `tag_list` | string array | no | List of the runner's tags | @@ -474,11 +474,11 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a NOTE: The `active` and `paused` values in the `status` query parameter were deprecated [in GitLab 14.8](https://gitlab.com/gitlab-org/gitlab/-/issues/347211). -and will be removed in [GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/351109). They are replaced by the `paused` query parameter. +and will be removed in [GitLab 16.0](https://gitlab.com/gitlab-org/gitlab/-/issues/351109). They are replaced by the `paused` query parameter. NOTE: The `active` attribute in the response was deprecated [in GitLab 14.8](https://gitlab.com/gitlab-org/gitlab/-/issues/347211). -and will be removed in [GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/351109). It is replaced by the `paused` attribute. +and will be removed in [GitLab 16.0](https://gitlab.com/gitlab-org/gitlab/-/issues/351109). It is replaced by the `paused` attribute. Example response: @@ -580,7 +580,7 @@ GET /groups/:id/runners?tag_list=tag1,tag2 |------------|----------------|----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `id` | integer | yes | The ID of the group owned by the authenticated user | | `type` | string | no | The type of runners to show, one of: `instance_type`, `group_type`, `project_type`. The `project_type` value is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/351466) and will be removed in GitLab 15.0 | -| `status` | string | no | The status of runners to show, one of: `online` and `offline`. `active` and `paused` are also possible values which were deprecated in GitLab 14.8 and will be removed in GitLab 15.0 | +| `status` | string | no | The status of runners to show, one of: `online` and `offline`. `active` and `paused` are also possible values which were deprecated in GitLab 14.8 and will be removed in GitLab 16.0 | | `paused` | boolean | no | Whether to include only runners that are accepting or ignoring new jobs | | `tag_list` | string array | no | List of the runner's tags | @@ -590,11 +590,11 @@ curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/a NOTE: The `active` and `paused` values in the `status` query parameter were deprecated [in GitLab 14.8](https://gitlab.com/gitlab-org/gitlab/-/issues/347211). -and will be removed in [GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/351109). They are replaced by the `paused` query parameter. +and will be removed in [GitLab 16.0](https://gitlab.com/gitlab-org/gitlab/-/issues/351109). They are replaced by the `paused` query parameter. NOTE: The `active` attribute in the response was deprecated [in GitLab 14.8](https://gitlab.com/gitlab-org/gitlab/-/issues/347211). -and will be removed in [GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/351109). It is replaced by the `paused` attribute. +and will be removed in [GitLab 16.0](https://gitlab.com/gitlab-org/gitlab/-/issues/351109). It is replaced by the `paused` attribute. Example response: @@ -660,7 +660,7 @@ POST /runners | `access_level` | string | no | The access_level of the runner; `not_protected` or `ref_protected` | | `maximum_timeout` | integer | no | Maximum timeout set when this runner handles the job | | `maintainer_note` | string | no | [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/350730), see `maintenance_note` | -| `maintenance_note` | string | no | Free-form maintenance notes for the runner (255 characters) | +| `maintenance_note` | string | no | Free-form maintenance notes for the runner (1024 characters) | ```shell curl --request POST "https://gitlab.example.com/api/v4/runners" \ @@ -799,9 +799,9 @@ curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ "https://gitlab.example.com/api/v4/groups/9/runners/reset_registration_token" ``` -## Reset runner's authentication token +## Reset runner's authentication token by using the runner ID -Resets the runner's authentication token. +Resets the runner's authentication token by using its runner ID. ```plaintext POST /runners/:id/reset_authentication_token @@ -824,3 +824,29 @@ Example response: "token_expires_at": "2021-09-27T21:05:03.203Z" } ``` + +## Reset runner's authentication token by using the current token + +Resets the runner's authentication token by using the current token's value as an input. + +```plaintext +POST /runners/reset_authentication_token +``` + +| Attribute | Type | Required | Description | +|-----------|---------|----------|---------------------------------| +| `token` | string | yes | The current token of the runner | + +```shell +curl --request POST --form "token=<current token>" \ + "https://gitlab.example.com/api/v4/runners/reset_authentication_token" +``` + +Example response: + +```json +{ + "token": "6337ff461c94fd3fa32ba3b1ff4125", + "token_expires_at": "2021-09-27T21:05:03.203Z" +} +``` diff --git a/doc/api/search.md b/doc/api/search.md index 2969cf439dd..8c681904e98 100644 --- a/doc/api/search.md +++ b/doc/api/search.md @@ -27,7 +27,7 @@ GET /search | `order_by` | string | no | Allowed values are `created_at` only. If this is not set, the results are either sorted by `created_at` in descending order for basic search, or by the most relevant documents when using advanced search.| | `sort` | string | no | Allowed values are `asc` or `desc` only. If this is not set, the results are either sorted by `created_at` in descending order for basic search, or by the most relevant documents when using advanced search.| -Search the expression within the specified scope. Currently these scopes are supported: projects, issues, merge_requests, milestones, snippet_titles, users. +Search the expression in the specified scope. These scopes are supported: projects, issues, merge_requests, milestones, snippet_titles, users. If Elasticsearch is enabled additional scopes available are blobs, wiki_blobs, notes, and commits. Find more about [the feature](../integration/elasticsearch.md). **(PREMIUM)** @@ -344,7 +344,7 @@ Filters are available for this scope: - path - extension -to use a filter simply include it in your query like so: `a query filename:some_name*`. +To use a filter, include it in your query. For example: `a query filename:some_name*`. You may use wildcards (`*`) to use glob matching. @@ -432,7 +432,7 @@ Example response: ## Group Search API -Search within the specified group. +Search in the specified group. If a user is not a member of a group and the group is private, a `GET` request on that group results in a `404` status code. @@ -450,7 +450,7 @@ GET /groups/:id/search | `order_by` | string | no | Allowed values are `created_at` only. If this is not set, the results are either sorted by `created_at` in descending order for basic search, or by the most relevant documents when using advanced search.| | `sort` | string | no | Allowed values are `asc` or `desc` only. If this is not set, the results are either sorted by `created_at` in descending order for basic search, or by the most relevant documents when using advanced search.| -Search the expression within the specified scope. Currently these scopes are supported: projects, issues, merge_requests, milestones, users. +Search the expression in the specified scope. These scopes are supported: projects, issues, merge_requests, milestones, users. If Elasticsearch is enabled additional scopes available are blobs, wiki_blobs, notes, and commits. Find more about [the feature](../integration/elasticsearch.md). **(PREMIUM)** @@ -736,7 +736,7 @@ Filters are available for this scope: - path - extension -to use a filter simply include it in your query like so: `a query filename:some_name*`. +To use a filter, include it in your query. For example: `a query filename:some_name*`. You may use wildcards (`*`) to use glob matching. @@ -824,7 +824,7 @@ Example response: ## Project Search API -Search within the specified project. +Search in the specified project. If a user is not a member of a project and the project is private, a `GET` request on that project results in a `404` status code. @@ -843,7 +843,7 @@ GET /projects/:id/search | `order_by` | string | no | Allowed values are `created_at` only. If this is not set, the results are either sorted by `created_at` in descending order for basic search, or by the most relevant documents when using advanced search.| | `sort` | string | no | Allowed values are `asc` or `desc` only. If this is not set, the results are either sorted by `created_at` in descending order for basic search, or by the most relevant documents when using advanced search.| -Search the expression within the specified scope. Currently these scopes are supported: issues, merge_requests, milestones, notes, wiki_blobs, commits, blobs, users. +Search the expression in the specified scope. These scopes are supported: issues, merge_requests, milestones, notes, wiki_blobs, commits, blobs, users. The response depends on the requested scope. @@ -1065,7 +1065,7 @@ Filters are available for this scope: - path - extension -To use a filter simply include it in your query like: `a query filename:some_name*`. +To use a filter, include it in your query. For example: `a query filename:some_name*`. You may use wildcards (`*`) to use glob matching. Wiki blobs searches are performed on both filenames and contents. Search @@ -1148,7 +1148,7 @@ Filters are available for this scope: - path - extension -To use a filter simply include it in your query like: `a query filename:some_name*`. +To use a filter, include it in your query. For example: `a query filename:some_name*`. You may use wildcards (`*`) to use glob matching. Blobs searches are performed on both filenames and contents. Search results: diff --git a/doc/api/secure_files.md b/doc/api/secure_files.md new file mode 100644 index 00000000000..96190920a33 --- /dev/null +++ b/doc/api/secure_files.md @@ -0,0 +1,171 @@ +--- +stage: Verify +group: Pipeline Authoring +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +type: reference, api +--- + +# Project-level Secure Files API **(FREE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/78227) in GitLab 14.8. [Deployed behind the `ci_secure_files` flag](../administration/feature_flags.md), disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, +ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `ci_secure_files`. +The feature is not ready for production use. + +## List project secure files + +Get list of secure files in a project. + +```plaintext +GET /projects/:project_id/secure_files +``` + +Supported attributes: + +| Attribute | Type | Required | Description | +|--------------|----------------|------------------------|-------------| +| `project_id` | integer/string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | + +Example request: + +```shell +curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/secure_files" +``` + +Example response: + +```json +[ + { + "id": 1, + "name": "myfile.jks", + "checksum": "16630b189ab34b2e3504f4758e1054d2e478deda510b2b08cc0ef38d12e80aac", + "checksum_algorithm": "sha256", + "permissions": "read_only", + "created_at": "2022-02-22T22:22:22.222Z" + }, + { + "id": 2, + "name": "myotherfile.jks", + "checksum": "16630b189ab34b2e3504f4758e1054d2e478deda510b2b08cc0ef38d12e80aa2", + "checksum_algorithm": "sha256", + "permissions": "execute", + "created_at": "2022-02-22T22:22:22.222Z" + } +] +``` + +## Show secure file details + +Get the details of a specific secure file in a project. + +```plaintext +GET /projects/:project_id/secure_files/:id +``` + +Supported attributes: + +| Attribute | Type | Required | Description | +|--------------|----------------|------------------------|-------------| +| `project_id` | integer/string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `id` | integer | **{check-circle}** Yes | The `id` of a secure file. | + +Example request: + +```shell +curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/secure_files/1" +``` + +Example response: + +```json +{ + "id": 1, + "name": "myfile.jks", + "checksum": "16630b189ab34b2e3504f4758e1054d2e478deda510b2b08cc0ef38d12e80aac", + "checksum_algorithm": "sha256", + "permissions": "read_only", + "created_at": "2022-02-22T22:22:22.222Z" +} +``` + +## Create secure file + +Create a new secure file. + +```plaintext +POST /projects/:project_id/secure_files +``` + +Supported attributes: + +| Attribute | Type | Required | Description | +|---------------|----------------|------------------------|-------------| +| `project_id` | integer/string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `name` | string | **{check-circle}** Yes | The `name` of the file being uploaded. | +| `file` | file | **{check-circle}** Yes | The `file` being uploaded. | +| `permissions` | string | **{dotted-circle}** No | The file is created with the specified permissions when created in the CI/CD job. Available types are: `read_only` (default), `read_write`, and `execute`. | + +Example request: + +```shell +curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \ + "https://gitlab.example.com/api/v4/projects/1/secure_files" --form "name=myfile.jks" --form "file=@/path/to/file/myfile.jks" +``` + +Example response: + +```json +{ + "id": 1, + "name": "myfile.jks", + "checksum": "16630b189ab34b2e3504f4758e1054d2e478deda510b2b08cc0ef38d12e80aac", + "checksum_algorithm": "sha256", + "permissions": "read_only", + "created_at": "2022-02-22T22:22:22.222Z" +} +``` + +## Download secure file + +Download the contents of a project's secure file. + +```plaintext +GET /projects/:project_id/download/:id +``` + +Supported attributes: + +| Attribute | Type | Required | Description | +|--------------|----------------|------------------------|-------------| +| `project_id` | integer/string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `id` | integer | **{check-circle}** Yes | The `id` of a secure file. | + +Example request: + +```shell +curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" https://gitlab.example.com/api/v4/projects/1/secure_files/1/download --output myfile.jks +``` + +## Remove secure file + +Remove a project's secure file. + +```plaintext +DELETE /projects/:project_id/secure_files/:id +``` + +Supported attributes: + +| Attribute | Type | Required | Description | +|--------------|----------------|------------------------|-------------| +| `project_id` | integer/string | **{check-circle}** Yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) owned by the authenticated user. | +| `id` | integer | **{check-circle}** Yes | The `id` of a secure file. | + +Example request: + +```shell +curl --request DELETE --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/secure_files/1" +``` diff --git a/doc/api/settings.md b/doc/api/settings.md index 4246c7a46f1..b9c52ff01fd 100644 --- a/doc/api/settings.md +++ b/doc/api/settings.md @@ -53,6 +53,10 @@ Example response: "gravatar_enabled" : true, "sign_in_text" : null, "container_expiration_policies_enable_historic_entries": true, + "container_registry_cleanup_tags_service_max_list_size": 200, + "container_registry_delete_tags_service_timeout": 250, + "container_registry_expiration_policies_caching": true, + "container_registry_expiration_policies_worker_capacity": 4, "container_registry_token_expire_delay": 5, "repository_storages_weighted": {"default": 100}, "plantuml_enabled": false, @@ -158,6 +162,11 @@ Example response: "external_authorization_service_timeout": 0.5, "user_oauth_applications": true, "after_sign_out_path": "", + "container_expiration_policies_enable_historic_entries": true, + "container_registry_cleanup_tags_service_max_list_size": 200, + "container_registry_delete_tags_service_timeout": 250, + "container_registry_expiration_policies_caching": true, + "container_registry_expiration_policies_worker_capacity": 4, "container_registry_token_expire_delay": 5, "repository_storages": ["default"], "plantuml_enabled": false, @@ -248,6 +257,11 @@ listed in the descriptions of the relevant settings. | `automatic_purchased_storage_allocation` | boolean | no | Enabling this permits automatic allocation of purchased storage in a namespace. | | `check_namespace_plan` **(PREMIUM)** | boolean | no | Enabling this makes only licensed EE features available to projects if the project namespace's plan includes the feature or if the project is public. | | `commit_email_hostname` | string | no | Custom hostname (for private commit emails). | +| `container_expiration_policies_enable_historic_entries` | boolean | no | Enable [cleanup policies](../user/packages/container_registry/reduce_container_registry_storage.md#enable-the-cleanup-policy) for all projects. | +| `container_registry_cleanup_tags_service_max_list_size` | integer | no | The maximum number of tags that can be deleted in a single execution of [cleanup policies](../user/packages/container_registry/reduce_container_registry_storage.md#set-cleanup-limits-to-conserve-resources). | +| `container_registry_delete_tags_service_timeout` | integer | no | The maximum time, in seconds, that the cleanup process can take to delete a batch of tags for [cleanup policies](../user/packages/container_registry/reduce_container_registry_storage.md#set-cleanup-limits-to-conserve-resources). | +| `container_registry_expiration_policies_caching` | boolean | no | Caching during the execution of [cleanup policies](../user/packages/container_registry/reduce_container_registry_storage.md#set-cleanup-limits-to-conserve-resources). | +| `container_registry_expiration_policies_worker_capacity` | integer | no | Number of workers for [cleanup policies](../user/packages/container_registry/reduce_container_registry_storage.md#set-cleanup-limits-to-conserve-resources). | | `container_registry_token_expire_delay` | integer | no | Container Registry token duration in minutes. | | `deactivate_dormant_users` | boolean | no | Enable [automatic deactivation of dormant users](../user/admin_area/moderate_users.md#automatically-deactivate-dormant-users). | | `default_artifacts_expire_in` | string | no | Set the default expiration time for each job's artifacts. | @@ -376,7 +390,9 @@ listed in the descriptions of the relevant settings. | `push_event_hooks_limit` | integer | no | Number of changes (branches or tags) in a single push to determine whether webhooks and services fire or not. Webhooks and services aren't submitted if it surpasses that value. | | `rate_limiting_response_text` | string | no | When rate limiting is enabled via the `throttle_*` settings, send this plain text response when a rate limit is exceeded. 'Retry later' is sent if this is blank. | | `raw_blob_request_limit` | integer | no | Max number of requests per minute for each raw path. Default: 300. To disable throttling set to 0.| -| `user_email_lookup_limit` | integer | no | Max number of requests per minute for email lookup. Default: 60. To disable throttling set to 0.| +| `user_email_lookup_limit` | integer | no | **{warning}** **[Removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/80631/)** in GitLab 14.9. Replaced by `search_rate_limit`. Max number of requests per minute for email lookup. Default: 60. To disable throttling set to 0.| +| `search_rate_limit` | integer | no | Max number of requests per minute for performing a search while authenticated. Default: 30. To disable throttling set to 0.| +| `search_rate_limit_unauthenticated` | integer | no | Max number of requests per minute for performing a search while unauthenticated. Default: 10. To disable throttling set to 0.| | `recaptcha_enabled` | boolean | no | (**If enabled, requires:** `recaptcha_private_key` and `recaptcha_site_key`) Enable reCAPTCHA. | | `recaptcha_private_key` | string | required by: `recaptcha_enabled` | Private key for reCAPTCHA. | | `recaptcha_site_key` | string | required by: `recaptcha_enabled` | Site key for reCAPTCHA. | diff --git a/doc/api/status_checks.md b/doc/api/status_checks.md index a3a342854dd..3671ddbd138 100644 --- a/doc/api/status_checks.md +++ b/doc/api/status_checks.md @@ -44,8 +44,17 @@ GET /projects/:id/merge_requests/:merge_request_iid/status_checks ## Set status of an external status check +> - Introduced in GitLab 14.9, `passed` status to pass external status checks. Introduced [with a flag](../administration/feature_flags.md) named `status_checks_add_status_field`. Disabled by default. +> - Introduced in GitLab 14.9, `failed` status to fail external status checks. Introduced [with a flag](../administration/feature_flags.md) named `status_checks_add_status_field`. Disabled by default. +> - `pass` status to pass checks is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/339039) in GitLab 14.9. Replaced with `passed`. + +FLAG: +On self-managed GitLab, by default setting `passed` instead of `pass` is unavailable. Also, setting `failed` is unavailable by default. To support +setting `passed` and `failed` instead of only `pass`, ask an administrator to [enable the feature flag](../administration/feature_flags.md) named +`status_checks_add_status_field`. On GitLab.com, this feature is not available. + For a single merge request, use the API to inform GitLab that a merge request has passed a check by an external service. -To set the status of an external check, the personal access token used must belong to a user with at least the developer role on the target project of the merge request. +To set the status of an external check, the personal access token used must belong to a user with at least the Developer role on the target project of the merge request. Execute this API call as any user with rights to approve the merge request itself. @@ -55,13 +64,14 @@ POST /projects/:id/merge_requests/:merge_request_iid/status_check_responses **Parameters:** -| Attribute | Type | Required | Description | -| -------------------------- | ------- | -------- | ------------------------------------- | -| `id` | integer | yes | ID of a project | -| `merge_request_iid` | integer | yes | IID of a merge request | -| `sha` | string | yes | SHA at `HEAD` of the source branch | -| `external_status_check_id` | integer | yes | ID of an external status check | -| `status` | string | no | Set to `pass` to pass the check | +| Attribute | Type | Required | Description | +| -------------------------- | ------- | -------- | ---------------------------------------------------------------------------- | +| `id` | integer | yes | ID of a project | +| `merge_request_iid` | integer | yes | IID of a merge request | +| `sha` | string | yes | SHA at `HEAD` of the source branch | +| `external_status_check_id` | integer | yes | ID of an external status check | +| `status` | string | no | Set to `passed` to pass the check or `failed` to fail it (GitLab 14.9 and later with feature flag `status_checks_add_status_field` enabled) | +| `status` | string | no | Set to `pass` to pass the check (GitLab 14.0 to GitLab 14.8, and GitLab 14.9 and later with feature flag `status_checks_add_status_field` disabled) | NOTE: `sha` must be the SHA at the `HEAD` of the merge request's source branch. diff --git a/doc/api/system_hooks.md b/doc/api/system_hooks.md index 3587016ac6b..f14f1322184 100644 --- a/doc/api/system_hooks.md +++ b/doc/api/system_hooks.md @@ -46,6 +46,43 @@ Example response: ] ``` +## Get system hook + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/81595) in GitLab 14.9. + +Get a system hook by its ID. + +```plaintext +GET /hooks/:id +``` + +| Attribute | Type | Required | Description | +| --------- | ---- | -------- | ----------- | +| `id` | integer | yes | The ID of the hook | + +Example request: + +```shell +curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/hooks/1" +``` + +Example response: + +```json +[ + { + "id": 1, + "url": "https://gitlab.example.com/hook", + "created_at": "2016-10-31T12:32:15.192Z", + "push_events": true, + "tag_push_events": false, + "merge_requests_events": true, + "repository_update_events": true, + "enable_ssl_verification": true + } +] +``` + ## Add new system hook Add a new system hook. diff --git a/doc/api/topics.md b/doc/api/topics.md index 538b9af9374..1246e36f2cb 100644 --- a/doc/api/topics.md +++ b/doc/api/topics.md @@ -203,3 +203,28 @@ curl --request PUT \ --header "PRIVATE-TOKEN: <your_access_token>" \ "https://gitlab.example.com/api/v4/topics/1" ``` + +## Delete a project topic + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/80725) in GitLab 14.9. + +You must be an administrator to delete a project. +When you delete a project topic, you also delete the topic assignment for projects. + +```plaintext +DELETE /topics/:id +``` + +Supported attributes: + +| Attribute | Type | Required | Description | +| ------------- | ------- | ---------------------- | ------------------- | +| `id` | integer | **{check-circle}** Yes | ID of project topic | + +Example request: + +```shell +curl --request DELETE \ + --header "PRIVATE-TOKEN: <your_access_token>" \ + "https://gitlab.example.com/api/v4/topics/1" +``` diff --git a/doc/api/users.md b/doc/api/users.md index 925563aeb1f..de9af59de93 100644 --- a/doc/api/users.md +++ b/doc/api/users.md @@ -603,7 +603,7 @@ Parameters: "avatar_url": "http://localhost:3000/uploads/user/avatar/1/index.jpg", "web_url": "http://localhost:3000/john_smith", "created_at": "2012-05-23T08:00:58Z", - "is_admin": false, + "is_admin": true, "bio": "", "location": null, "public_email": "john@example.com", @@ -958,6 +958,32 @@ Parameters: } ``` +## Single SSH key for given user + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/81790) in GitLab 14.9. + +Get a single key for a given user. + +```plaintext +GET /users/:id/keys/:key_id +``` + +Parameters: + +| Attribute | Type | Required | Description | +|-----------|---------|----------|----------------------| +| `id` | integer | yes | ID of specified user | +| `key_id` | integer | yes | SSH key ID | + +```json +{ + "id": 1, + "title": "Public key", + "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIEAiPWx6WM4lhHNedGfBpPJNPpZ7yKu+dnn1SJejgt4596k6YjzGGphH2TUxwKzxcKDKKezwkpfnxPkSMkuEspGRt/aZZ9wa++Oi7Qkr8prgHc4soW6NUlfDzpvZK2H5E7eQaSeP3SAwGmQKUFHCddNaP0L+hM7zhFNzjFvpaMgJw0=", + "created_at": "2014-08-01T14:47:39.080Z" +} +``` + ## Add SSH key Creates a new key owned by the currently authenticated user. diff --git a/doc/api/v3_to_v4.md b/doc/api/v3_to_v4.md index 9c2170e0709..875f4528c0e 100644 --- a/doc/api/v3_to_v4.md +++ b/doc/api/v3_to_v4.md @@ -6,4 +6,6 @@ remove_date: '2023-01-31' This document was moved to [another location](https://gitlab.com/gitlab-org/gitlab-foss/-/blob/11-0-stable/doc/api/v3_to_v4.md). <!-- This redirect file can be deleted after <2023-01-31>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/api/vulnerability_exports.md b/doc/api/vulnerability_exports.md index 9395a4ee5de..6f6a661dbd5 100644 --- a/doc/api/vulnerability_exports.md +++ b/doc/api/vulnerability_exports.md @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/197494) in GitLab 12.10. [Updated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/30397) in GitLab 13.0. WARNING: -This API is in an alpha stage and considered unstable. +This API is in an [Alpha](../policy/alpha-beta-support.md#alpha-features) stage and considered unstable. The response payload may be subject to change or breakage across GitLab releases. diff --git a/doc/api/wikis.md b/doc/api/wikis.md index fb49e9465bc..b7c9d0d6008 100644 --- a/doc/api/wikis.md +++ b/doc/api/wikis.md @@ -6,6 +6,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Project wikis API **(FREE)** +> - The `encoding` field was [added](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/81150) in GitLab 14.9. +> - The `render_html` attribute was [added](https://gitlab.com/gitlab-org/gitlab/-/issues/336792) in GitLab 14.9. +> - The `version` attribute was [added](https://gitlab.com/gitlab-org/gitlab/-/issues/336792) in GitLab 14.9. + The project [wikis](../user/project/wiki/index.md) API is available only in APIv4. An API for [group wikis](group_wikis.md) is also available. @@ -34,18 +38,21 @@ Example response: "content" : "Here is an instruction how to deploy this project.", "format" : "markdown", "slug" : "deploy", - "title" : "deploy" + "title" : "deploy", + "encoding": "UTF-8" }, { "content" : "Our development process is described here.", "format" : "markdown", "slug" : "development", - "title" : "development" + "title" : "development", + "encoding": "UTF-8" },{ "content" : "* [Deploy](deploy)\n* [Development](development)", "format" : "markdown", "slug" : "home", - "title" : "home" + "title" : "home", + "encoding": "UTF-8" } ] ``` @@ -62,6 +69,8 @@ GET /projects/:id/wikis/:slug | --------- | ------- | -------- | --------------------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](index.md#namespaced-path-encoding) | | `slug` | string | yes | URLencoded slug (a unique string) of the wiki page, such as `dir%2Fpage_name` | +| `render_html` | boolean | no | Return the rendered HTML of the wiki page | +| `version` | string | no | Wiki page version sha | ```shell curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/wikis/home" @@ -74,7 +83,8 @@ Example response: "content" : "home page", "format" : "markdown", "slug" : "home", - "title" : "home" + "title" : "home", + "encoding": "UTF-8" } ``` @@ -105,7 +115,8 @@ Example response: "content" : "Hello world", "format" : "markdown", "slug" : "Hello", - "title" : "Hello" + "title" : "Hello", + "encoding": "UTF-8" } ``` @@ -137,7 +148,8 @@ Example response: "content" : "documentation", "format" : "markdown", "slug" : "Docs", - "title" : "Docs" + "title" : "Docs", + "encoding": "UTF-8" } ``` diff --git a/doc/architecture/blueprints/ci_scale/index.md b/doc/architecture/blueprints/ci_scale/index.md index 092f8a7119a..be21f824392 100644 --- a/doc/architecture/blueprints/ci_scale/index.md +++ b/doc/architecture/blueprints/ci_scale/index.md @@ -174,15 +174,15 @@ Work required to achieve our next CI/CD scaling target is tracked in the 1. ✓ Migrate primary keys to big integers on GitLab.com. 1. ✓ Implement the new architecture of builds queuing on GitLab.com. -1. Make the new builds queuing architecture generally available. -1. Partition CI/CD data using time-decay pattern. +1. [Make the new builds queuing architecture generally available](https://gitlab.com/groups/gitlab-org/-/epics/6954). +1. [Partition CI/CD data using time-decay pattern](../ci_data_decay/). ## Status |-------------|--------------| | Created at | 21.01.2021 | | Approved at | 26.04.2021 | -| Updated at | 06.12.2021 | +| Updated at | 28.02.2022 | Status: In progress. diff --git a/doc/architecture/blueprints/composable_codebase_using_rails_engines/index.md b/doc/architecture/blueprints/composable_codebase_using_rails_engines/index.md index 82529a4c199..5f0f0a7aa63 100644 --- a/doc/architecture/blueprints/composable_codebase_using_rails_engines/index.md +++ b/doc/architecture/blueprints/composable_codebase_using_rails_engines/index.md @@ -383,7 +383,7 @@ What was done? ```ruby # config/engines.rb # Load only in case we are running web_server or rails console - if Gitlab::Runtime.web_server? || Gitlab::Runtime.console? + if Gitlab::Runtime.puma? || Gitlab::Runtime.console? require 'web_engine' end ``` diff --git a/doc/architecture/blueprints/consolidating_groups_and_projects/index.md b/doc/architecture/blueprints/consolidating_groups_and_projects/index.md index 6040ac1e50f..52e9b48419c 100644 --- a/doc/architecture/blueprints/consolidating_groups_and_projects/index.md +++ b/doc/architecture/blueprints/consolidating_groups_and_projects/index.md @@ -173,4 +173,4 @@ DRIs: ## Related topics -- [Workspaces developer documentation](../../../development/workspaces/index.md) +- [Workspace developer documentation](../../../development/workspace/index.md) diff --git a/doc/architecture/blueprints/container_registry_metadata_database/index.md b/doc/architecture/blueprints/container_registry_metadata_database/index.md index c1aac235085..ba0bee04f35 100644 --- a/doc/architecture/blueprints/container_registry_metadata_database/index.md +++ b/doc/architecture/blueprints/container_registry_metadata_database/index.md @@ -1,7 +1,7 @@ --- stage: Package group: Package -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments comments: false description: 'Container Registry metadata database' --- diff --git a/doc/architecture/blueprints/gitlab_to_kubernetes_communication/index.md b/doc/architecture/blueprints/gitlab_to_kubernetes_communication/index.md index 754988487de..b22636ac1d9 100644 --- a/doc/architecture/blueprints/gitlab_to_kubernetes_communication/index.md +++ b/doc/architecture/blueprints/gitlab_to_kubernetes_communication/index.md @@ -1,7 +1,7 @@ --- stage: Configure group: Configure -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments comments: false description: 'GitLab to Kubernetes communication' --- @@ -9,7 +9,7 @@ description: 'GitLab to Kubernetes communication' # GitLab to Kubernetes communication **(FREE)** The goal of this document is to define how GitLab can communicate with Kubernetes -and in-cluster services through the GitLab Agent. +and in-cluster services through the GitLab agent. ## Challenges @@ -48,7 +48,7 @@ are stored on the GitLab side and this is yet another security concern for our c For more discussion on these issues, read [issue #212810](https://gitlab.com/gitlab-org/gitlab/-/issues/212810). -## GitLab Agent epic +## GitLab agent epic To address these challenges and provide some new features, the Configure group is building an active in-cluster component that inverts the @@ -62,12 +62,12 @@ The customer does not need to provide any credentials to GitLab, and is in full control of what permissions the agent has. For more information, visit the -[GitLab Agent repository](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent) or +[GitLab agent repository](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent) or [the epic](https://gitlab.com/groups/gitlab-org/-/epics/3329). ### Request routing -Agents connect to the server-side component called GitLab Agent Server +Agents connect to the server-side component called GitLab agent server (`gitlab-kas`) and keep an open connection that waits for commands. The difficulty with the approach is in routing requests from GitLab to the correct agent. Each cluster may contain multiple logical agents, and each may be running as multiple diff --git a/doc/architecture/blueprints/object_storage/index.md b/doc/architecture/blueprints/object_storage/index.md index 7864c951eca..3420caa325f 100644 --- a/doc/architecture/blueprints/object_storage/index.md +++ b/doc/architecture/blueprints/object_storage/index.md @@ -196,7 +196,7 @@ require one bucket. ## Additional reading materials -- [Uploads development documentation: The problem description](../../../development/uploads.md#the-problem-description). +- [Uploads development guide](../../../development/uploads/index.md). - [Speed up the monolith, building a smart reverse proxy in Go](https://archive.fosdem.org/2020/schedule/event/speedupmonolith/): a presentation explaining a bit of workhorse history and the challenge we faced in releasing the first cloud-native installation. - [Object Storage improvements epic](https://gitlab.com/groups/gitlab-org/-/epics/483). - We are moving to GraphQL API, but [we do not support direct upload](https://gitlab.com/gitlab-org/gitlab/-/issues/280819). diff --git a/doc/architecture/blueprints/runner_scaling/gitlab-autoscaling-overview.png b/doc/architecture/blueprints/runner_scaling/gitlab-autoscaling-overview.png Binary files differindex c3ba615784f..2c8c753cddd 100644 --- a/doc/architecture/blueprints/runner_scaling/gitlab-autoscaling-overview.png +++ b/doc/architecture/blueprints/runner_scaling/gitlab-autoscaling-overview.png diff --git a/doc/ci/cloud_deployment/ecs/quick_start_guide.md b/doc/ci/cloud_deployment/ecs/quick_start_guide.md index 9cf9d2a1f06..5571c51ef27 100644 --- a/doc/ci/cloud_deployment/ecs/quick_start_guide.md +++ b/doc/ci/cloud_deployment/ecs/quick_start_guide.md @@ -40,10 +40,10 @@ For the first step here, you create a demo application from a project template. Use a GitLab project template to get started. As the name suggests, these projects provide a bare-bones application built on some well-known frameworks. -1. In GitLab, click the plus icon (**{plus-square}**) at the top of the navigation bar, and select +1. In GitLab, select the plus icon (**{plus-square}**) at the top of the navigation bar, and select **New project**. -1. Click the **Create from template** button, where you can choose from a Ruby on Rails, Spring, or +1. Select **Create from template**, where you can choose from a Ruby on Rails, Spring, or NodeJS Express project. For this guide, use the Ruby on Rails template.  @@ -52,7 +52,7 @@ bare-bones application built on some well-known frameworks. take advantage of the features available in the [GitLab Ultimate plan](https://about.gitlab.com/pricing/). -1. Click **Create project**. +1. Select **Create project**. Now that you created a demo project, you must containerize the application and push it to the container registry. @@ -65,7 +65,7 @@ GitLab [Auto Build](../../../topics/autodevops/stages.md#auto-build) and [Container Registry](../../../user/packages/container_registry/index.md). 1. Go to **ecs-demo** project on GitLab. -1. Click **Setup up CI/CD**. It brings you to a `.gitlab-ci.yml` +1. Select **Setup up CI/CD**. It brings you to a `.gitlab-ci.yml` creation form. 1. Copy and paste the following content into the empty `.gitlab-ci.yml`. This defines [a pipeline for continuous deployment to ECS](../index.md#deploy-your-application-to-the-aws-elastic-container-service-ecs). @@ -75,7 +75,7 @@ and [Container Registry](../../../user/packages/container_registry/index.md). - template: AWS/Deploy-ECS.gitlab-ci.yml ``` -1. Click **Commit Changes**. It automatically triggers a new pipeline. In this pipeline, the `build` +1. Select **Commit Changes**. It automatically triggers a new pipeline. In this pipeline, the `build` job containerizes the application and pushes the image to [GitLab Container Registry](../../../user/packages/container_registry/index.md).  @@ -97,14 +97,14 @@ later. is a specification about how the application image is started by an [ECS service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html). 1. Go to **ECS > Task Definitions** on [AWS console](https://aws.amazon.com/). -1. Click **Create new Task Definition**. +1. Select **Create new Task Definition**.  -1. Choose **EC2** as the launch type. Click **Next Step**. +1. Choose **EC2** as the launch type. Select **Next Step**. 1. Set `ecs_demo` to **Task Definition Name**. 1. Set `512` to **Task Size > Task memory** and **Task CPU**. -1. Click **Container Definitions > Add container**. This opens a container registration form. +1. Select **Container Definitions > Add container**. This opens a container registration form. 1. Set `web` to **Container name**. 1. Set `registry.gitlab.com/<your-namespace>/ecs-demo/master:latest` to **Image**. Alternatively, you can copy and paste the image path from the [GitLab Container Registry page](#push-a-containerized-application-image-to-gitlab-container-registry). @@ -115,7 +115,7 @@ is a specification about how the application image is started by an [ECS service  -1. Click **Create**. +1. Select **Create**. Now you have the initial task definition. Next, you create an actual infrastructure to run the application image. @@ -127,13 +127,13 @@ is a virtual group of [ECS services](https://docs.aws.amazon.com/AmazonECS/lates It's also associated with EC2 or Fargate as the computation resource. 1. Go to **ECS > Clusters** on [AWS console](https://aws.amazon.com/). -1. Click **Create Cluster**. -1. Select **EC2 Linux + Networking** as the cluster template. Click **Next Step**. +1. Select **Create Cluster**. +1. Select **EC2 Linux + Networking** as the cluster template. Select **Next Step**. 1. Set `ecs-demo` to **Cluster Name**. 1. Choose the default [VPC](https://aws.amazon.com/vpc/?vpc-blogs.sort-by=item.additionalFields.createdDate&vpc-blogs.sort-order=desc) in **Networking**. If there are no existing VPCs, you can leave it as-is to create a new one. 1. Set all available subnets of the VPC to **Subnets**. -1. Click **Create**. +1. Select **Create**. 1. Make sure that the ECS cluster has been successfully created.  @@ -154,7 +154,7 @@ Note the following: is a daemon to create an application container based on the [ECS task definition](#create-an-ecs-task-definition). 1. Go to **ECS > Clusters > ecs-demo > Services** on the [AWS console](https://aws.amazon.com/) -1. Click **Deploy**. This opens a service creation form. +1. Select **Deploy**. This opens a service creation form. 1. Select `EC2` in **Launch Type**. 1. Set `ecs_demo` to **Task definition**. This corresponds to [the task definition you created above](#create-an-ecs-task-definition). 1. Set `ecs_demo` to **Service name**. @@ -162,7 +162,7 @@ is a daemon to create an application container based on the [ECS task definition  -1. Click **Deploy**. +1. Select **Deploy**. 1. Make sure that the created service is active.  @@ -176,7 +176,7 @@ Now, the demo application is accessible from the internet. 1. Go to **EC2 > Instances** on the [AWS console](https://aws.amazon.com/) 1. Search by `ECS Instance` to find the corresponding EC2 instance that [the ECS cluster created](#create-an-ecs-cluster). -1. Click the ID of the EC2 instance. This brings you to the instance detail page. +1. Select the ID of the EC2 instance. This brings you to the instance detail page. 1. Copy **Public IPv4 address** and paste it in the browser. Now you can see the demo application running. @@ -195,15 +195,15 @@ For GitLab to access the ECS cluster, service, and task definition that you crea create a deployer user on AWS: 1. Go to **IAM > Users** on [AWS console](https://aws.amazon.com/). -1. Click **Add user**. +1. Select **Add user**. 1. Set `ecs_demo` to **User name**. -1. Enable **Programmatic access** checkbox. Click **Next: Permissions**. +1. Enable **Programmatic access** checkbox. Select **Next: Permissions**. 1. Select `Attach existing policies directly` in **Set permissions**. -1. Select `AmazonECS_FullAccess` from the policy list. Click **Next: Tags** and **Next: Review**. +1. Select `AmazonECS_FullAccess` from the policy list. Select **Next: Tags** and **Next: Review**.  -1. Click **Create user**. +1. Select **Create user**. 1. Take note of the **Access key ID** and **Secret access key** of the created user. NOTE: @@ -216,7 +216,7 @@ These variables are injected into the pipeline jobs and can access the ECS API. 1. Go to **ecs-demo** project on GitLab. 1. Go to **Settings > CI/CD > Variables**. -1. Click **Add Variable** and set the following key-value pairs. +1. Select **Add Variable** and set the following key-value pairs. |Key|Value|Note| |---|---|---| @@ -233,9 +233,9 @@ Change a file in the project and see if it's reflected in the demo application o 1. Go to **ecs-demo** project on GitLab. 1. Open the file at **app > views > welcome > `index.html.erb`**. -1. Click **Edit**. +1. Select **Edit**. 1. Change the text to `You're on ECS!`. -1. Click **Commit Changes**. This automatically triggers a new pipeline. Wait until it finishes. +1. Select **Commit Changes**. This automatically triggers a new pipeline. Wait until it finishes. 1. [Access the running application on the ECS cluster](#view-the-demo-application). You should see this: diff --git a/doc/ci/cloud_services/index.md b/doc/ci/cloud_services/index.md index a80231a04c2..1493a930099 100644 --- a/doc/ci/cloud_services/index.md +++ b/doc/ci/cloud_services/index.md @@ -18,6 +18,13 @@ GitLab CI/CD supports [OpenID Connect (OIDC)](https://openid.net/connect/faq/) t The original implementation of `CI_JOB_JWT` supports [HashiCorp Vault integration](../examples/authenticating-with-hashicorp-vault/). The updated implementation of `CI_JOB_JWT_V2` supports additional cloud providers with OIDC including AWS, GCP, and Vault. +NOTE: +Configuring OIDC enables JWT token access to the target environments for all pipelines. +When you configure OIDC for a pipeline, you should complete a software supply chain security +review for the pipeline, focusing on the additional access. You can use the [software supply chain security awareness assessment](https://about.gitlab.com/quiz/software-supply-chain-security/) +as a starting point, and for more information about supply chain attacks, see +[How a DevOps Platform helps protect against supply chain attacks](https://about.gitlab.com/blog/2021/04/28/devops-platform-supply-chain-attacks/). + WARNING: The `CI_JOB_JWT_V2` variable is under development [(alpha)](../../policy/alpha-beta-support.md#alpha-features) and is not yet suitable for production use. diff --git a/doc/ci/directed_acyclic_graph/index.md b/doc/ci/directed_acyclic_graph/index.md index abf43390834..5d0aaf7cc45 100644 --- a/doc/ci/directed_acyclic_graph/index.md +++ b/doc/ci/directed_acyclic_graph/index.md @@ -81,7 +81,7 @@ are certain use cases that you may need to work around. For more information, ch ## Needs Visualization -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/215517) in GitLab 13.1 as a [Beta feature](https://about.gitlab.com/handbook/product/#beta). +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/215517) in GitLab 13.1 as a [Beta feature](../../policy/alpha-beta-support.md#beta-features). > - It became a [standard feature](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/38517) in 13.3. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/52208) in GitLab 13.9. diff --git a/doc/ci/environments/deployment_approvals.md b/doc/ci/environments/deployment_approvals.md index e320118c191..45af78aa036 100644 --- a/doc/ci/environments/deployment_approvals.md +++ b/doc/ci/environments/deployment_approvals.md @@ -11,7 +11,7 @@ description: Require approvals prior to deploying to a Protected Environment > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/347342) in GitLab 14.8. WARNING: -This feature is in an alpha stage and subject to change without prior notice. +This feature is in a [Beta](../../policy/alpha-beta-support.md#beta-features) stage and subject to change without prior notice. It may be useful to require additional approvals before deploying to certain protected environments (for example, production). This pre-deployment approval requirement is useful to accommodate testing, security, or compliance processes that must happen before each deployment. @@ -79,30 +79,53 @@ Maintainer role. ## Approve or reject a deployment -NOTE: -This functionality is currently only available through the API. UI is planned for the near future. See [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/342180/). +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/342180/) in GitLab 14.9 + +Using either the GitLab UI or the API, you can: + +- Approve a deployment to allow it to proceed. +- Reject a deployment to prevent it. + +### Approve or reject a deployment using the UI + +Prerequisites: + +- Permission to deploy to the protected environment. + +To approve or reject a deployment to a protected environment using the UI: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Deployments > Environments**. +1. In the deployment's row, select **Approval options** (**{thumb-up}**). +1. Select **Approve** or **Reject**. + +### Approve or reject a deployment using the API + +Prerequisites: -A blocked deployment is enqueued as soon as it receives the required number of approvals. A single rejection causes the deployment to fail. The creator of a deployment cannot approve it, even if they have permission to deploy. +- Permission to deploy to the protected environment. -Using the [Deployments API](../../api/deployments.md#approve-or-reject-a-blocked-deployment), users who are allowed to deploy to the protected environment can approve or reject a blocked deployment. +To approve or reject a deployment to a protected environment using the API, pass the +required attributes. For more details, see +[Approve or reject a blocked deployment](../../api/deployments.md#approve-or-reject-a-blocked-deployment). Example: ```shell -curl --data "status=approved" \ +curl --data "status=approved&comment=Looks good to me" \ --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/deployments/1/approval" ``` -### How to see blocked deployments +## How to see blocked deployments -#### Using the UI +### Using the UI 1. On the top bar, select **Menu > Projects** and find your project. 1. On the left sidebar, select **Deployments > Environments**. 1. Select the environment being deployed to. 1. Look for the `blocked` label. -#### Using the API +### Using the API Use the [Deployments API](../../api/deployments.md) to see deployments. diff --git a/doc/ci/environments/img/environments_list_v14_3.png b/doc/ci/environments/img/environments_list_v14_3.png Binary files differdeleted file mode 100644 index 8fdb85338e7..00000000000 --- a/doc/ci/environments/img/environments_list_v14_3.png +++ /dev/null diff --git a/doc/ci/environments/img/environments_list_v14_8.png b/doc/ci/environments/img/environments_list_v14_8.png Binary files differnew file mode 100644 index 00000000000..df439fb96dc --- /dev/null +++ b/doc/ci/environments/img/environments_list_v14_8.png diff --git a/doc/ci/environments/index.md b/doc/ci/environments/index.md index 63bdd279927..c2612b6ff16 100644 --- a/doc/ci/environments/index.md +++ b/doc/ci/environments/index.md @@ -35,7 +35,7 @@ To view a list of environments and deployments: 1. On the left sidebar, select **Deployments > Environments**. The environments are displayed. -  +  1. To view a list of deployments for an environment, select the environment name, for example, `staging`. @@ -440,7 +440,7 @@ stop_review: when: manual ``` -Both jobs must have the same [`rules`](../yaml/index.md#only--except) +Both jobs must have the same [`rules`](../yaml/index.md#rules) or [`only/except`](../yaml/index.md#only--except) configuration. Otherwise, the `stop_review` job might not be included in all pipelines that include the `deploy_review` job, and you cannot trigger `action: stop` to stop the environment automatically. diff --git a/doc/ci/environments/protected_environments.md b/doc/ci/environments/protected_environments.md index c7d1653aace..adc215c7aa1 100644 --- a/doc/ci/environments/protected_environments.md +++ b/doc/ci/environments/protected_environments.md @@ -163,9 +163,8 @@ For more information, see [Deployment safety](deployment_safety.md). Typically, large enterprise organizations have an explicit permission boundary between [developers and operators](https://about.gitlab.com/topics/devops/). Developers build and test their code, and operators deploy and monitor the -application. With group-level protected environments, the permission of each -group is carefully configured in order to prevent unauthorized access and -maintain proper separation of duty. Group-level protected environments +application. With group-level protected environments, operators can +restrict access to critical environments from developers. Group-level protected environments extend the [project-level protected environments](#protecting-environments) to the group-level. @@ -194,12 +193,6 @@ and are protected at the same time. ### Configure group-level memberships -In an enterprise organization, with thousands of projects under a single group, -ensuring that all of the [project-level protected environments](#protecting-environments) -are properly configured is not a scalable solution. For example, a developer -might gain privileged access to a higher environment when they are given the Maintainer role -for a new project. Group-level protected environments can be a solution in this situation. - To maximize the effectiveness of group-level protected environments, [group-level memberships](../../user/group/index.md) must be correctly configured: @@ -237,7 +230,7 @@ Having this configuration in place: - If a user is about to run a deployment job in a project but disallowed to deploy to the environment, the deployment job fails with an error message. -### Protect a group-level environment +### Protect critical environments under a group To protect a group-level environment: diff --git a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md index edc58684057..5fca3513ff7 100644 --- a/doc/ci/examples/authenticating-with-hashicorp-vault/index.md +++ b/doc/ci/examples/authenticating-with-hashicorp-vault/index.md @@ -121,8 +121,8 @@ Then create policies that allow you to read these secrets (one for each secret): $ vault policy write myproject-staging - <<EOF # Policy name: myproject-staging # -# Read-only permission on 'secret/data/myproject/staging/*' path -path "secret/data/myproject/staging/*" { +# Read-only permission on 'secret/myproject/staging/*' path +path "secret/myproject/staging/*" { capabilities = [ "read" ] } EOF @@ -131,8 +131,8 @@ Success! Uploaded policy: myproject-staging $ vault policy write myproject-production - <<EOF # Policy name: myproject-production # -# Read-only permission on 'secret/data/myproject/production/*' path -path "secret/data/myproject/production/*" { +# Read-only permission on 'secret/myproject/production/*' path +path "secret/myproject/production/*" { capabilities = [ "read" ] } EOF diff --git a/doc/ci/jobs/ci_job_token.md b/doc/ci/jobs/ci_job_token.md index a62f670cb12..b0002f559ba 100644 --- a/doc/ci/jobs/ci_job_token.md +++ b/doc/ci/jobs/ci_job_token.md @@ -23,6 +23,12 @@ You can use a GitLab CI/CD job token to authenticate with specific API endpoints - [Releases](../../api/releases/index.md). - [Terraform plan](../../user/infrastructure/index.md). +NOTE: +There's an open issue, +[GitLab-#333444](https://gitlab.com/gitlab-org/gitlab/-/issues/333444), +which prevents you from using a job token with internal projects. This bug only impacts self-managed +GitLab instances. + The token has the same permissions to access the API as the user that caused the job to run. A user can cause a job to run by pushing a commit, triggering a manual job, being the owner of a scheduled pipeline, and so on. Therefore, this user must be assigned to diff --git a/doc/ci/jobs/index.md b/doc/ci/jobs/index.md index 39e14d0d20a..4fa251eb3cf 100644 --- a/doc/ci/jobs/index.md +++ b/doc/ci/jobs/index.md @@ -103,6 +103,9 @@ Job names must be 255 characters or less. [Introduced](https://gitlab.com/gitlab in GitLab 14.5, [with a feature flag](../../administration/feature_flags.md) named `ci_validate_job_length`. Enabled by default. To disable it, ask an administrator to [disable the feature flag](../../administration/feature_flags.md). +Use unique names for your jobs. If multiple jobs have the same name, +only one is added to the pipeline, and it's difficult to predict which one is chosen. + ## Group jobs in a pipeline If you have many similar jobs, your [pipeline graph](../pipelines/index.md#visualize-pipelines) becomes long and hard diff --git a/doc/ci/jobs/job_control.md b/doc/ci/jobs/job_control.md index 6523de0ed1e..3a302cf352b 100644 --- a/doc/ci/jobs/job_control.md +++ b/doc/ci/jobs/job_control.md @@ -85,6 +85,28 @@ be triggered by the same event (a push to the source branch for an open merge re See how to [prevent duplicate pipelines](#avoid-duplicate-pipelines) for more details. +#### Run jobs for scheduled pipelines + +To configure a job to be executed only when the pipeline has been +scheduled, use the [`rules`](../yaml/index.md#rules) keyword. + +In this example, `make world` runs in scheduled pipelines, and `make build` +runs in branch and tag pipelines: + +```yaml +job:on-schedule: + rules: + - if: $CI_PIPELINE_SOURCE == "schedule" + script: + - make world + +job: + rules: + - if: $CI_PIPELINE_SOURCE == "push" + script: + - make build +``` + ### Complex rules You can use all `rules` keywords, like `if`, `changes`, and `exists`, in the same @@ -115,7 +137,7 @@ job1: script: - echo This rule uses parentheses. rules: - if: ($CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH || $CI_COMMIT_BRANCH == "develop") && $MY_VARIABLE + - if: ($CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH || $CI_COMMIT_BRANCH == "develop") && $MY_VARIABLE ``` WARNING: @@ -727,6 +749,38 @@ deploystacks: - ${PROVIDER}-${STACK} ``` +#### Fetch artifacts from a `parallel:matrix` job + +You can fetch artifacts from a job created with [`parallel:matrix`](../yaml/index.md#parallelmatrix) +by using the [`dependencies`](../yaml/index.md#dependencies) keyword. Use the job name +as the value for `dependencies` as a string in the form: + +```plaintext +<job_name> [<matrix argument 1>, <matrix argument 2>, ... <matrix argument N>] +``` + +For example, to fetch the artifacts from the job with a `RUBY_VERSION` of `2.7` and +a `PROVIDER` of `aws`: + +```yaml +ruby: + image: ruby:${RUBY_VERSION} + parallel: + matrix: + - RUBY_VERSION: ["2.5", "2.6", "2.7", "3.0", "3.1"] + PROVIDER: [aws, gcp] + script: bundle install + +deploy: + image: ruby:2.7 + stage: deploy + dependencies: + - "ruby: [2.7, aws]" + script: echo hello +``` + +Quotes around the `dependencies` entry are required. + ## Use predefined CI/CD variables to run jobs only in specific pipeline types You can use [predefined CI/CD variables](../variables/predefined_variables.md) to choose @@ -812,13 +866,9 @@ due to computational complexity, and some features, like negative lookaheads, be Only a subset of features provided by [Ruby Regexp](https://ruby-doc.org/core/Regexp.html) are now supported. -From GitLab 11.9.7 to GitLab 12.0, GitLab provided a feature flag to -let you use unsafe regexp syntax. After migrating to safe syntax, you should disable -this feature flag again: - -```ruby -Feature.enable(:allow_unsafe_ruby_regexp) -``` +From GitLab 11.9.7 to GitLab 14.9, GitLab provided a feature flag to let you +use unsafe regexp syntax. We've fully migrated to RE2 now, and that feature +flag is no longer available. ## CI/CD variable expressions diff --git a/doc/ci/lint.md b/doc/ci/lint.md index c0df0b2a439..256e669a66e 100644 --- a/doc/ci/lint.md +++ b/doc/ci/lint.md @@ -25,9 +25,8 @@ configuration added with the [`includes` keyword](yaml/index.md#include). To check CI/CD configuration with the CI lint tool: 1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select one of: +1. On the left sidebar, select: - **CI/CD > Pipelines** - - **CI/CD > Jobs** 1. In the top right, select **CI lint**. 1. Paste a copy of the CI/CD configuration you want to check into the text box. 1. Select **Validate**. @@ -48,9 +47,8 @@ Prerequisites: To simulate a pipeline: 1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select one of: +1. On the left sidebar, select: - **CI/CD > Pipelines** - - **CI/CD > Jobs** 1. In the top right, select **CI lint**. 1. Paste a copy of the CI/CD configuration you want to check into the text box. 1. Select **Simulate pipeline creation for the default branch**. diff --git a/doc/ci/metrics_reports.md b/doc/ci/metrics_reports.md index 5b472eec7b5..542d55665c7 100644 --- a/doc/ci/metrics_reports.md +++ b/doc/ci/metrics_reports.md @@ -55,3 +55,14 @@ An advanced example of an OpenMetrics text file (from the [Prometheus documentat renders in the merge request widget as:  + +## Troubleshooting + +### Metrics reports did not change + +You can see `Metrics reports did not change` when trying to view metrics reports in merge requests. Reasons for this are: + +- The target branch for the merge request doesn't have a baseline metrics report for comparison. +- You don't have GitLab license at the Premium tier or above. + +There is [an issue open](https://gitlab.com/gitlab-org/gitlab/-/issues/343065) to improve this message. diff --git a/doc/ci/migration/circleci.md b/doc/ci/migration/circleci.md index efaae873588..d95199a70d0 100644 --- a/doc/ci/migration/circleci.md +++ b/doc/ci/migration/circleci.md @@ -169,7 +169,7 @@ job1: - if: '$CI_PIPELINE_SOURCE == "schedule" && $CI_COMMIT_REF_NAME == "try-schedule-workflow"' ``` -After the pipeline configuration is saved, you configure the cron schedule in the [GitLab UI](../pipelines/schedules.md#configuring-pipeline-schedules), and can enable or disable schedules in the UI as well. +After the pipeline configuration is saved, you configure the cron schedule in the [GitLab UI](../pipelines/schedules.md#add-a-pipeline-schedule), and can enable or disable schedules in the UI as well. #### Manual run diff --git a/doc/ci/pipelines/cicd_minutes.md b/doc/ci/pipelines/cicd_minutes.md index 8b10a74bd78..f9d9a4f738b 100644 --- a/doc/ci/pipelines/cicd_minutes.md +++ b/doc/ci/pipelines/cicd_minutes.md @@ -223,3 +223,25 @@ On GitLab SaaS an email notification is sent to the namespace owners when: - The available CI/CD minutes are below 30% of the quota. - The available CI/CD minutes are below 5% of the quota. - All CI/CD minutes have been used. + +## Reduce consumption of CI/CD minutes + +If your project consumes too many CI/CD minutes, there are some strategies you can +use to reduce your CI/CD minutes usage: + +- If you are using project mirrors, ensure that [pipelines for mirror updates](../../user/project/repository/mirror/pull.md#trigger-pipelines-for-mirror-updates) + is disabled. +- Reduce the frequency of [scheduled pipelines](schedules.md). +- [Skip pipelines](index.md#skip-a-pipeline) when not needed. +- Use [interruptible](../yaml/index.md#interruptible) jobs which can be auto-canceled + if a new pipeline starts. +- If a job doesn't have to run in every pipeline, use [`rules`](../jobs/job_control.md) + to make it only run when it's needed. +- [Use private runners](../runners/runners_scope.md#group-runners) for some jobs. +- If you are working from a fork and you submit a merge request to the parent project, + you can ask a maintainer to run a pipeline [in the parent project](merge_request_pipelines.md#run-pipelines-in-the-parent-project). + +If you manage an open source project, these improvements can also reduce CI/CD minutes +consumption for contributor fork projects, enabling more contributions. + +See our [pipeline efficiency guide](pipeline_efficiency.md) for more details. diff --git a/doc/ci/pipelines/img/pipeline_schedule_play.png b/doc/ci/pipelines/img/pipeline_schedule_play.png Binary files differdeleted file mode 100644 index ec6eb0d156b..00000000000 --- a/doc/ci/pipelines/img/pipeline_schedule_play.png +++ /dev/null diff --git a/doc/ci/pipelines/img/pipeline_schedule_variables.png b/doc/ci/pipelines/img/pipeline_schedule_variables.png Binary files differdeleted file mode 100644 index ce3c3dc6af1..00000000000 --- a/doc/ci/pipelines/img/pipeline_schedule_variables.png +++ /dev/null diff --git a/doc/ci/pipelines/img/pipeline_schedules_list.png b/doc/ci/pipelines/img/pipeline_schedules_list.png Binary files differdeleted file mode 100644 index 541fe4f9b1d..00000000000 --- a/doc/ci/pipelines/img/pipeline_schedules_list.png +++ /dev/null diff --git a/doc/ci/pipelines/img/pipeline_schedules_new_form.png b/doc/ci/pipelines/img/pipeline_schedules_new_form.png Binary files differdeleted file mode 100644 index 993fbf8ca00..00000000000 --- a/doc/ci/pipelines/img/pipeline_schedules_new_form.png +++ /dev/null diff --git a/doc/ci/pipelines/img/pipeline_schedules_ownership.png b/doc/ci/pipelines/img/pipeline_schedules_ownership.png Binary files differdeleted file mode 100644 index 8fc5c5fbc82..00000000000 --- a/doc/ci/pipelines/img/pipeline_schedules_ownership.png +++ /dev/null diff --git a/doc/ci/pipelines/index.md b/doc/ci/pipelines/index.md index 5a5fd2b5540..4d1af735b13 100644 --- a/doc/ci/pipelines/index.md +++ b/doc/ci/pipelines/index.md @@ -140,7 +140,7 @@ to its **Pipelines** tab.  -Click a pipeline to open the **Pipeline Details** page and show +Select a pipeline to open the **Pipeline Details** page and show the jobs that were run for that pipeline. From here you can cancel a running pipeline, retry jobs on a failed pipeline, or [delete a pipeline](#delete-a-pipeline). @@ -246,7 +246,7 @@ For each `var` or `file_var`, a key and value are required. [Manual jobs](../jobs/job_control.md#create-a-job-that-must-be-run-manually), allow you to require manual interaction before moving forward in the pipeline. -You can do this straight from the pipeline graph. Just click the play button +You can do this straight from the pipeline graph. Just select the play button to execute that particular job. For example, your pipeline can start automatically, but require a manual action to @@ -259,8 +259,8 @@ In the example below, the `production` stage has a job with a manual action: > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/27188) in GitLab 11.11. -Multiple manual actions in a single stage can be started at the same time using the "Play all manual" button. -After you click this button, each individual manual action is triggered and refreshed +Multiple manual actions in a single stage can be started at the same time using the "Play all manual" +After you select this action, each individual manual action is triggered and refreshed to an updated status. This functionality is only available: @@ -283,9 +283,9 @@ pipelines. Users with the Owner role for a project can delete a pipeline by clicking on the pipeline in the **CI/CD > Pipelines** to get to the **Pipeline Details** -page, then using the **Delete** button. +page, then selecting **Delete**. - + WARNING: Deleting a pipeline expires all pipeline caches, and deletes all related objects, @@ -314,7 +314,7 @@ sensitive information like deployment credentials and tokens. **Runners** marked as **protected** can run jobs only on protected branches, preventing untrusted code from executing on the protected runner and preserving deployment keys and other credentials from being unintentionally -accessed. In order to ensure that jobs intended to be executed on protected +accessed. To ensure that jobs intended to be executed on protected runners do not use regular runners, they must be tagged accordingly. ### How pipeline duration is calculated @@ -385,6 +385,12 @@ You can group the jobs by: [Multi-project pipeline graphs](multi_project_pipelines.md#multi-project-pipeline-visualization) help you visualize the entire pipeline, including all cross-project inter-dependencies. **(PREMIUM)** +If a stage contains more than 100 jobs, only the first 100 jobs are listed in the +pipeline graph. The remaining jobs still run as normal. To see the jobs: + +- Select the pipeline, and the jobs are listed on the right side of the pipeline details page. +- On the left sidebar, select **CI/CD > Jobs**. + ### View job dependencies in the pipeline graph > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/298973) in GitLab 13.12. @@ -428,7 +434,7 @@ fix it. Pipeline mini graphs only display jobs by stage. -Stages in pipeline mini graphs are collapsible. Hover your mouse over them and click to expand their jobs. +Stages in pipeline mini graphs are expandable. Hover your mouse over each stage to see the name and status, and select a stage to expand its jobs list. | Mini graph | Mini graph expanded | |:-------------------------------------------------------------|:---------------------------------------------------------------| diff --git a/doc/ci/pipelines/merge_request_pipelines.md b/doc/ci/pipelines/merge_request_pipelines.md index dcc3e7e6919..d80b745e6bc 100644 --- a/doc/ci/pipelines/merge_request_pipelines.md +++ b/doc/ci/pipelines/merge_request_pipelines.md @@ -41,8 +41,10 @@ Both of these types of pipelines can appear on the **Pipelines** tab of a merge The three types of merge request pipelines are: - Merge request pipelines, which run on the changes in the merge request's - source branch. These pipelines display a `detached` label to indicate that the + source branch. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/352939) + in GitLab 14.9, these pipelines display a `merge request` label to indicate that the pipeline ran only on the contents of the source branch, ignoring the target branch. + In GitLab 14.8 and earlier, the label is `detached`. - [Merged results pipelines](merged_results_pipelines.md), which run on the result of combining the source branch's changes with the target branch. - [Merge trains](merge_trains.md), which run when merging multiple merge requests diff --git a/doc/ci/pipelines/merge_trains.md b/doc/ci/pipelines/merge_trains.md index ffcce06cfbd..12ea3a70536 100644 --- a/doc/ci/pipelines/merge_trains.md +++ b/doc/ci/pipelines/merge_trains.md @@ -201,6 +201,10 @@ the merged result is out of date and the pipeline can't be retried. Instead, you should [add the merge request to the train](#add-a-merge-request-to-a-merge-train) again, which triggers a new pipeline. +If a job only fails intermittently, you can try using the [`retry`](../yaml/index.md#retry) +keyword in the `.gitlab-ci.yml` file to have the job retried before the pipeline completes. +If it succeeds after a retry, the merge request is not removed from the merge train. + ### Unable to add to merge train with message "The pipeline for this merge request failed." Sometimes the **Start/Add to merge train** button is not available and the merge request says, diff --git a/doc/ci/pipelines/merged_results_pipelines.md b/doc/ci/pipelines/merged_results_pipelines.md index 4794107cc87..7df9ea3f72f 100644 --- a/doc/ci/pipelines/merged_results_pipelines.md +++ b/doc/ci/pipelines/merged_results_pipelines.md @@ -24,7 +24,7 @@ Merged results pipelines can't run when: - The merge request is a [**Draft** merge request](../../user/project/merge_requests/drafts.md). In these cases, the pipeline runs as a [merge request pipeline](merge_request_pipelines.md) -and is labeled as `detached`. +and [is labeled as `merge request`](merge_request_pipelines.md#types-of-merge-request-pipelines). ## Prerequisites diff --git a/doc/ci/pipelines/pipelines_for_merged_results.md b/doc/ci/pipelines/pipelines_for_merged_results.md index 457f5ce9c30..0c3100a51f1 100644 --- a/doc/ci/pipelines/pipelines_for_merged_results.md +++ b/doc/ci/pipelines/pipelines_for_merged_results.md @@ -6,4 +6,6 @@ remove_date: '2022-04-27' This document was moved to [another location](merged_results_pipelines.md). <!-- This redirect file can be deleted after 2022-04-27. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/ci/pipelines/schedules.md b/doc/ci/pipelines/schedules.md index fe9db3306cd..8813f3e1d59 100644 --- a/doc/ci/pipelines/schedules.md +++ b/doc/ci/pipelines/schedules.md @@ -6,142 +6,69 @@ disqus_identifier: 'https://docs.gitlab.com/ee/user/project/pipelines/schedules. type: reference, howto --- -# Pipeline schedules **(FREE)** +# Scheduled pipelines **(FREE)** -Pipelines are normally run based on certain conditions being met. For example, when a branch is pushed to repository. - -Pipeline schedules can be used to also run [pipelines](index.md) at specific intervals. For example: - -- Every month on the 22nd (cron example: `0 0 22 * *`) for a certain branch. -- Every month on the 2nd Monday (cron example: `0 0 * * 1#2`). -- Every other Sunday at 0900 hours (cron example: `0 9 * * sun%2`). -- Once every day (cron example: `0 0 * * *`). - -Schedule timing is configured with [cron notation](../../topics/cron/index.md). -You can use any cron value, but scheduled pipelines cannot run more frequently -than the instance's [maximum frequency for scheduled pipelines](#advanced-configuration). - -In addition to using the GitLab UI, pipeline schedules can be maintained using the -[Pipeline schedules API](../../api/pipeline_schedules.md). +Use scheduled pipelines to run GitLab CI/CD [pipelines](index.md) at regular intervals. ## Prerequisites -In order for a scheduled pipeline to be created successfully: - -- The schedule owner must have [permissions](../../user/permissions.md) to merge into the target branch. -- The pipeline configuration must be valid. - -Otherwise the pipeline is not created. - -## Configuring pipeline schedules - -To schedule a pipeline for project: - -1. Navigate to the project's **CI/CD > Schedules** page. -1. Click the **New schedule** button. -1. Fill in the **Schedule a new pipeline** form. -1. Click the **Save pipeline schedule** button. - - - -NOTE: -Pipelines execution [timing is dependent](#advanced-configuration) on Sidekiq's own schedule. - -In the **Schedules** index page you can see a list of the pipelines that are -scheduled to run. The next run is automatically calculated by the server GitLab -is installed on. - - - -### Using variables - -You can pass any number of arbitrary variables. They are available in -GitLab CI/CD so that they can be used in your [`.gitlab-ci.yml` file](../../ci/yaml/index.md). - - - -### Using `rules` - -To configure a job to be executed only when the pipeline has been -scheduled, use the [`rules`](../yaml/index.md#rules) keyword. - -In this example, `make world` runs in scheduled pipelines, and `make build` -runs in branch and tag pipelines: - -```yaml -job:on-schedule: - rules: - - if: $CI_PIPELINE_SOURCE == "schedule" - script: - - make world - -job: - rules: - - if: $CI_PIPELINE_SOURCE == "push" - script: - - make build -``` - -### Advanced configuration **(FREE SELF)** - -Scheduled pipelines can be configured with any [cron value](../../topics/cron/index.md), -but they do not always run exactly when scheduled. An internal process, called the -_pipeline schedule worker_, queues all the scheduled pipelines, but does not -run continuously. The worker runs on its own schedule, and scheduled pipelines that -are ready to start are only queued the next time the worker runs. Scheduled pipelines -can't run more frequently than the worker. - -The default frequency of the pipeline schedule worker is `3-59/10 * * * *` (every ten minutes, -starting with `0:03`, `0:13`, `0:23`, and so on). The default frequency for GitLab.com -is listed in the [GitLab.com settings](../../user/gitlab_com/index.md#gitlab-cicd). - -To change the frequency of the pipeline schedule worker: - -1. Edit the `gitlab_rails['pipeline_schedule_worker_cron']` value in your instance's `gitlab.rb` file. -1. [Reconfigure GitLab](../../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) for the changes to take effect. - -For example, to set the maximum frequency of pipelines to twice a day, set `pipeline_schedule_worker_cron` -to a cron value of `0 */12 * * *` (`00:00` and `12:00` every day). +For a scheduled pipeline to run: -## Working with scheduled pipelines +- The schedule owner must have the Developer role. For pipelines on protected branches, + the schedule owner must be [allowed to merge](../../user/project/protected_branches.md#configure-a-protected-branch) + to the branch. +- The [CI/CD configuration](../yaml/index.md) must be valid. -After configuration, GitLab supports many functions for working with scheduled pipelines. +Otherwise, the pipeline is not created. No error message is displayed. -### Running manually +## Add a pipeline schedule -> [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/15700) in GitLab 10.4. +> Scheduled pipelines for tags [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/23292) in GitLab 14.9. -To trigger a pipeline schedule manually, click the "Play" button: +To add a pipeline schedule: - +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **CI/CD > Schedules**. +1. Select **New schedule** and fill in the form. + - **Interval Pattern**: Select one of the preconfigured intervals, or enter a custom + interval in [cron notation](../../topics/cron/index.md). You can use any cron value, + but scheduled pipelines cannot run more frequently than the instance's + [maximum scheduled pipeline frequency](../../administration/cicd.md#change-maximum-scheduled-pipeline-frequency). + - **Target branch or tag**: Select the branch or tag for the pipeline. + - **Variables**: Add any number of [CI/CD variables](../variables/index.md) to the schedule. + These variables are available only when the scheduled pipeline runs, + and not in any other pipeline run. -This schedules a background job to run the pipeline schedule. A flash -message provides a link to the CI/CD Pipeline index page. +## Run manually -NOTE: -To help avoid abuse, users are rate limited to triggering a pipeline once per -minute. +To trigger a pipeline schedule manually, so that it runs immediately instead of +the next scheduled time: -### Taking ownership +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **CI/CD > Schedules**. +1. On the right of the list, for + the pipeline you want to run, select **Play** (**{play}**). -Pipelines are executed as a user, who owns a schedule. This influences what projects and other resources the pipeline has access to. +You can manually run scheduled pipelines once per minute. -If a user does not own a pipeline, you can take ownership by clicking the **Take ownership** button. -The next time a pipeline is scheduled, your credentials are used. +## Take ownership - +Scheduled pipelines execute with the permissions of the user +who owns the schedule. The pipeline has access to the same resources as the pipeline owner, +including [protected environments](../environments/protected_environments.md) and the +[CI/CD job token](../jobs/ci_job_token.md). -If the owner of a pipeline schedule cannot create -pipelines on the target branch, the schedule stops creating new -pipelines. +To take ownership of a pipeline created by a different user: -This can happen if, for example: +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **CI/CD > Schedules**. +1. On the right of the list, for + the pipeline you want to become owner of, select **Take ownership**. -- The owner is blocked or removed from the project. -- The target branch or tag is protected. +## Related topics -In this case, someone with sufficient privileges must take ownership of the -schedule. +- Pipeline schedules can be maintained by using the [Pipeline schedules API](../../api/pipeline_schedules.md). +- You can [control which jobs are added to scheduled pipelines](../jobs/job_control.md#run-jobs-for-scheduled-pipelines). <!-- ## Troubleshooting diff --git a/doc/ci/pipelines/settings.md b/doc/ci/pipelines/settings.md index e22746dbfa0..7960d0afa85 100644 --- a/doc/ci/pipelines/settings.md +++ b/doc/ci/pipelines/settings.md @@ -190,18 +190,13 @@ You can define how long a job can run before it times out. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General pipelines**. 1. In the **Timeout** field, enter the number of minutes, or a human-readable value like `2 hours`. + Must be 10 minutes or more, and less than one month. Default is 60 minutes. Jobs that exceed the timeout are marked as failed. You can override this value [for individual runners](../runners/configure_runners.md#set-maximum-job-timeout-for-a-runner). -## Add test coverage results to a merge request (DEPRECATED) - -> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) in GitLab 14.9. Replaced by [`coverage` keyword](../yaml/index.md#coverage). - -WARNING: -This feature is in its end-of-life process. It is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) -for use in GitLab 14.9, and is planned for [removal](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) in GitLab 15.0. +## Merge request test coverage results If you use test coverage in your code, you can use a regular expression to find coverage results in the job log. You can then include these results @@ -215,27 +210,38 @@ averaged.  -To define a coverage-parsing regular expression: +### Add test coverage results using `coverage` keyword + +To add test coverage results to a merge request using the project's `.gitlab-ci.yml` file, provide a regular expression +using the [`coverage`](../yaml/index.md#coverage) keyword. + +Setting the regular expression this way takes precedence over project settings. + +### Add test coverage results using project settings (DEPRECATED) + +> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) in GitLab 14.9. Replaced by [`coverage` keyword](../yaml/index.md#coverage). + +WARNING: +This feature is in its end-of-life process. It is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) +for use in GitLab 14.9, and is planned for [removal](https://gitlab.com/gitlab-org/gitlab/-/issues/17633) in GitLab 15.0. -- Using the project's `.gitlab-ci.yml`, provide a regular expression using the [`coverage`](../yaml/index.md#coverage) - keyword. Setting the regular expression this way takes precedence over the project's CI/CD settings. +You can add test coverage results to merge requests using the Project's CI/CD settings: -- Using the Project's CI/CD settings: - - Set using the GitLab UI: +- Set using the GitLab UI: - 1. On the top bar, select **Menu > Projects** and find your project. - 1. On the left sidebar, select **Settings > CI/CD**. - 1. Expand **General pipelines**. - 1. In the **Test coverage parsing** field, enter a regular expression. Leave blank to disable this feature. + 1. On the top bar, select **Menu > Projects** and find your project. + 1. On the left sidebar, select **Settings > CI/CD**. + 1. Expand **General pipelines**. + 1. In the **Test coverage parsing** field, enter a regular expression. Leave blank to disable this feature. - - Set when [editing a project](../../api/projects.md#edit-project) or [creating a project](../../api/projects.md#create-project) - using the GitLab API with the `build_coverage_regex` attribute: +- Set when [editing a project](../../api/projects.md#edit-project) or [creating a project](../../api/projects.md#create-project) + using the GitLab API with the `build_coverage_regex` attribute: - ```shell - curl --request PUT --header "PRIVATE-TOKEN: <your-token>" \ - --url 'https://gitlab.com/api/v4/projects/<your-project-ID>' \ - --data "build_coverage_regex=<your-regular-expression>" - ``` + ```shell + curl --request PUT --header "PRIVATE-TOKEN: <your-token>" \ + --url 'https://gitlab.com/api/v4/projects/<your-project-ID>' \ + --data "build_coverage_regex=<your-regular-expression>" + ``` You can use <https://rubular.com> to test your regular expression. The regular expression returns the **last** match found in the output. @@ -324,7 +330,15 @@ lein cloverage | perl -pe 's/\e\[?.*?[\@-~]//g' Pipeline badges indicate the pipeline status and a test coverage value for your project. These badges are determined by the latest successful pipeline. -### View the code for the pipeline status and coverage reports badges +## Latest release badge + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/33368) in GitLab 14.8. + +A latest release badge indicates the latest release tag name for your project. +By default, the badge fetches the release sorted using the [`released_at`](../../api/releases/index.md#create-a-release) time. +Support for [`semver`](https://semver.org/) sorting is tracked [in this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352945). + +### View the code for the pipeline status, coverage reports, and latest release badges You can view the exact link for your badges. Then you can embed the badge in your HTML or Markdown pages. @@ -332,7 +346,7 @@ or Markdown pages. 1. On the top bar, select **Menu > Projects** and find your project. 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **General pipelines**. -1. In the **Pipeline status** or **Coverage report** sections, view the URLs for the images. +1. In the **Pipeline status**, **Coverage report**, or **Latest release** sections, view the URLs for the images.  @@ -364,9 +378,8 @@ https://gitlab.example.com/<namespace>/<project>/badges/<branch>/pipeline.svg?ig ### Test coverage report badge -You can define the regular expression for the [coverage report](#add-test-coverage-results-to-a-merge-request-deprecated) -that each job log is matched against. This means that each job in the -pipeline can have the test coverage percentage value defined. +You can define the regular expression for the [coverage report](#merge-request-test-coverage-results) that each job log +is matched against. This means that each job in the pipeline can have the test coverage percentage value defined. To access the test coverage badge, use the following link: @@ -406,6 +419,25 @@ If an invalid boundary is set, GitLab automatically adjusts it to be valid. For if `min_good` is set `80`, and `min_acceptable` is set to `85` (too high), GitLab automatically sets `min_acceptable` to `79` (`min_good` - `1`). +### Latest release badge + +When a release exists in your project, it shows the latest release tag name. If there is no release, +it shows `none`. + +You can access a latest release badge image by using the following link: + +```plaintext +https://gitlab.example.com/<namespace>/<project>/-/badges/release.svg +``` + +#### Sorting preferences + +By default, the latest release badge fetches the release using `release_at` time. The use of the query parameter `?order_by=release_at` is optional, and support for `?order_by=semver` is tracked [in this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352945): + +```plaintext +https://gitlab.example.com/<namespace>/<project>/-/badges/release.svg?order_by=release_at +``` + ### Badge styles Pipeline badges can be rendered in different styles by adding the `style=style_name` parameter to the URL. Two styles are available: diff --git a/doc/ci/runners/build_cloud/linux_build_cloud.md b/doc/ci/runners/build_cloud/linux_build_cloud.md deleted file mode 100644 index 2892a30cd2e..00000000000 --- a/doc/ci/runners/build_cloud/linux_build_cloud.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: '../saas/linux_saas_runner.md' -remove_date: '2022-02-05' ---- - -This document was moved to [another location](../saas/linux_saas_runner.md). - -<!-- This redirect file can be deleted after 2022-02-05. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/runners/build_cloud/macos/environment.md b/doc/ci/runners/build_cloud/macos/environment.md deleted file mode 100644 index a534e87cc34..00000000000 --- a/doc/ci/runners/build_cloud/macos/environment.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: '../../saas/macos/environment.md' -remove_date: '2022-02-05' ---- - -This document was moved to [another location](../../saas/macos/environment.md). - -<!-- This redirect file can be deleted after 2022-02-05. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/runners/build_cloud/macos_build_cloud.md b/doc/ci/runners/build_cloud/macos_build_cloud.md deleted file mode 100644 index 50b7e0cfb79..00000000000 --- a/doc/ci/runners/build_cloud/macos_build_cloud.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: '../saas/macos_saas_runner.md' -remove_date: '2022-02-05' ---- - -This document was moved to [another location](../saas/macos_saas_runner.md). - -<!-- This redirect file can be deleted after 2022-02-05. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/runners/build_cloud/windows_build_cloud.md b/doc/ci/runners/build_cloud/windows_build_cloud.md deleted file mode 100644 index fb64938eb9f..00000000000 --- a/doc/ci/runners/build_cloud/windows_build_cloud.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: '../saas/windows_saas_runner.md' -remove_date: '2022-02-05' ---- - -This document was moved to [another location](../saas/windows_saas_runner.md). - -<!-- This redirect file can be deleted after 2022-02-05. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/runners/configure_runners.md b/doc/ci/runners/configure_runners.md index 60e21653a45..6b14cea51d6 100644 --- a/doc/ci/runners/configure_runners.md +++ b/doc/ci/runners/configure_runners.md @@ -23,15 +23,15 @@ if smaller than the [project defined timeout](../pipelines/settings.md#set-a-lim This feature can be used to prevent your shared runner from being overwhelmed by a project that has jobs with a long timeout (for example, one week). -When not configured, runners do not override the project timeout. - On GitLab.com, you cannot override the job timeout for shared runners and must use the [project defined timeout](../pipelines/settings.md#set-a-limit-for-how-long-jobs-can-run). To set the maximum job timeout: 1. In a project, go to **Settings > CI/CD > Runners**. 1. Select your specific runner to edit the settings. -1. Enter a value under **Maximum job timeout**. +1. Enter a value under **Maximum job timeout**. Must be 10 minutes or more. If not + defined, the [project's job timeout setting](../pipelines/settings.md#set-a-limit-for-how-long-jobs-can-run) + is used. 1. Select **Save changes**. How this feature works: @@ -59,7 +59,7 @@ How this feature works: ## Be careful with sensitive information -With some [runner executors](https://docs.gitlab.com/runner/executors/README.html), +With some [runner executors](https://docs.gitlab.com/runner/executors/), if you can run a job on the runner, you can get full access to the file system, and thus any code it runs as well as the token of the runner. With shared runners, this means that anyone that runs jobs on the runner, can access anyone else's code that runs on the @@ -70,7 +70,7 @@ to create a clone of a runner and submit false jobs, for example. The above is easily avoided by restricting the usage of shared runners on large public GitLab instances, controlling access to your GitLab instance, -and using more secure [runner executors](https://docs.gitlab.com/runner/executors/README.html). +and using more secure [runner executors](https://docs.gitlab.com/runner/executors/). ### Prevent runners from revealing sensitive information diff --git a/doc/ci/runners/index.md b/doc/ci/runners/index.md index 064ad64b79f..257dc02805b 100644 --- a/doc/ci/runners/index.md +++ b/doc/ci/runners/index.md @@ -13,9 +13,9 @@ If you are using self-managed GitLab or you use GitLab.com but want to use your If you are using GitLab SaaS (GitLab.com), your CI jobs automatically run on runners provided by GitLab. No configuration is required. Your jobs can run on: -- [Linux runners](build_cloud/linux_build_cloud.md). -- [Windows runners](build_cloud/windows_build_cloud.md) (beta). -- [macOS runners](build_cloud/macos_build_cloud.md) (beta). +- [Linux runners](saas/linux_saas_runner.md). +- [Windows runners](saas/windows_saas_runner.md) ([Beta](../../policy/alpha-beta-support.md#beta-features)). +- [macOS runners](saas/macos_saas_runner.md) ([Beta](../../policy/alpha-beta-support.md#beta-features)). The number of minutes you can use on these runners depends on the [maximum number of CI/CD minutes](../pipelines/cicd_minutes.md) diff --git a/doc/ci/runners/runner_cloud/linux_runner_cloud.md b/doc/ci/runners/runner_cloud/linux_runner_cloud.md deleted file mode 100644 index 2892a30cd2e..00000000000 --- a/doc/ci/runners/runner_cloud/linux_runner_cloud.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: '../saas/linux_saas_runner.md' -remove_date: '2022-02-05' ---- - -This document was moved to [another location](../saas/linux_saas_runner.md). - -<!-- This redirect file can be deleted after 2022-02-05. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/runners/runner_cloud/macos/environment.md b/doc/ci/runners/runner_cloud/macos/environment.md deleted file mode 100644 index 37ad21c28fc..00000000000 --- a/doc/ci/runners/runner_cloud/macos/environment.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: '../../saas/macos/environment.md' -remove_date: '2022-02-05' ---- - -This document was moved to [another location](../../saas/macos/environment.md). - -<!-- This redirect file can be deleted after 2022-02-05. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page -->
\ No newline at end of file diff --git a/doc/ci/runners/runner_cloud/macos_runner_cloud.md b/doc/ci/runners/runner_cloud/macos_runner_cloud.md deleted file mode 100644 index 50b7e0cfb79..00000000000 --- a/doc/ci/runners/runner_cloud/macos_runner_cloud.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: '../saas/macos_saas_runner.md' -remove_date: '2022-02-05' ---- - -This document was moved to [another location](../saas/macos_saas_runner.md). - -<!-- This redirect file can be deleted after 2022-02-05. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/runners/runner_cloud/windows_runner_cloud.md b/doc/ci/runners/runner_cloud/windows_runner_cloud.md deleted file mode 100644 index fb64938eb9f..00000000000 --- a/doc/ci/runners/runner_cloud/windows_runner_cloud.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: '../saas/windows_saas_runner.md' -remove_date: '2022-02-05' ---- - -This document was moved to [another location](../saas/windows_saas_runner.md). - -<!-- This redirect file can be deleted after 2022-02-05. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/ci/runners/runners_scope.md b/doc/ci/runners/runners_scope.md index 7cfd8e50f6c..aa7e268e800 100644 --- a/doc/ci/runners/runners_scope.md +++ b/doc/ci/runners/runners_scope.md @@ -236,6 +236,10 @@ To enable or disable a specific runner for a project: 1. Go to the project's **Settings > CI/CD** and expand the **Runners** section. 1. Click **Enable for this project** or **Disable for this project**. +You can edit a specific runner from any of the projects it's enabled for. +The modifications, which include unlocking, editing tags and the description, +affect all projects that use the runner. + ### Prevent a specific runner from being enabled for other projects You can configure a specific runner so it is "locked" and cannot be enabled for other projects. diff --git a/doc/ci/runners/saas/linux_saas_runner.md b/doc/ci/runners/saas/linux_saas_runner.md index 4d1e628b8e7..fd3fedbc3f5 100644 --- a/doc/ci/runners/saas/linux_saas_runner.md +++ b/doc/ci/runners/saas/linux_saas_runner.md @@ -26,7 +26,7 @@ Jobs handled by shared runners on GitLab.com (`shared-runners-manager-X.gitlab.c **time out after 3 hours**, regardless of the timeout configured in a project. Check issue [#4010](https://gitlab.com/gitlab-com/infrastructure/-/issues/4010) and [#4070](https://gitlab.com/gitlab-com/infrastructure/-/issues/4070) for the reference. -Jobs handled by shared runners on Windows and macOS on GitLab.com **time out after 1 hour** while this service is in the Beta stage. +Jobs handled by shared runners on Windows and macOS on GitLab.com **time out after 1 hour** while this service is in the [Beta](../../../policy/alpha-beta-support.md#beta-features) stage. Below are the runners' settings. diff --git a/doc/ci/runners/saas/macos/environment.md b/doc/ci/runners/saas/macos/environment.md index 3332eab9b44..d1943a487a7 100644 --- a/doc/ci/runners/saas/macos/environment.md +++ b/doc/ci/runners/saas/macos/environment.md @@ -14,7 +14,7 @@ When you use SaaS runners on macOS: ## VM types The virtual machine where your job runs has `sudo` access with no password. -For the Beta, there is only one available machine type, `gbc-macos-large`. +For the [Beta](../../../../policy/alpha-beta-support.md#beta-features), there is only one available machine type, `gbc-macos-large`. | Instance type | vCPUS | Memory (GB) | | --------- | --- | ------- | diff --git a/doc/ci/runners/saas/macos_saas_runner.md b/doc/ci/runners/saas/macos_saas_runner.md index bfe408f4485..885a76a7b46 100644 --- a/doc/ci/runners/saas/macos_saas_runner.md +++ b/doc/ci/runners/saas/macos_saas_runner.md @@ -4,7 +4,10 @@ group: Runner info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# SaaS runners on macOS (Beta) **(FREE SAAS)** +# SaaS runners on macOS (beta) **(FREE SAAS)** + +SaaS runners on macOS are in [Beta](../../../policy/alpha-beta-support.md#beta-features) +and shouldn't be relied upon for mission-critical production jobs. SaaS runners on macOS provide an on-demand macOS build environment integrated with GitLab SaaS [CI/CD](../../../ci/index.md). @@ -12,9 +15,6 @@ Use these runners to build, test, and deploy apps for the Apple ecosystem (macOS of all the capabilities of the GitLab single DevOps platform and not have to manage or operate a build environment. -SaaS runners on macOS are in [Beta](../../../policy/alpha-beta-support.md#beta-features) -and shouldn't be relied upon for mission-critical production jobs. - ## Quickstart To start using SaaS runners on macOS, you must submit an access request [issue](https://gitlab.com/gitlab-com/macos-buildcloud-runners-beta/-/issues/new?issuable_template=beta_access_request). After your diff --git a/doc/ci/secrets/index.md b/doc/ci/secrets/index.md index ea0c0d9cc84..5595559b5f0 100644 --- a/doc/ci/secrets/index.md +++ b/doc/ci/secrets/index.md @@ -9,6 +9,7 @@ type: concepts, howto > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218746) in GitLab 13.4 and GitLab Runner 13.4. > - `file` setting [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/250695) in GitLab 14.1 and GitLab Runner 14.1. +> - `VAULT_NAMESPACE` setting [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/255619) in GitLab 14.9 and GitLab Runner 14.9. Secrets represent sensitive information your CI job needs to complete work. This sensitive information can be items like API tokens, database credentials, or private keys. @@ -90,6 +91,9 @@ To configure your Vault server: If no role is specified, Vault uses the [default role](https://www.vaultproject.io/api/auth/jwt#default_role) specified when the authentication method was configured. - `VAULT_AUTH_PATH` - Optional. The path where the authentication method is mounted, default is `jwt`. + - `VAULT_NAMESPACE` - Optional. The [Vault Enterprise namespace](https://www.vaultproject.io/docs/enterprise/namespaces) to use for reading secrets and authentication. + If no namespace is specified, Vault uses the `root` ("`/`") namespace. + The setting is ignored by Vault Open Source. NOTE: Support for providing these values in the user interface [is tracked in this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/218677). diff --git a/doc/ci/services/index.md b/doc/ci/services/index.md index d14027cb1ef..e6406818b4c 100644 --- a/doc/ci/services/index.md +++ b/doc/ci/services/index.md @@ -438,3 +438,7 @@ docker rm -f -v build service-mysql service-postgres This forcefully (`-f`) removes the `build` container, the two service containers, and all volumes (`-v`) that were created with the container creation. + +## Security when using services containers + +Docker privileged mode applies to services. This means that the service image container can access the host system. You should use container images from trusted sources only. diff --git a/doc/ci/unit_test_reports.md b/doc/ci/unit_test_reports.md index e2de47c6c62..14350ac884f 100644 --- a/doc/ci/unit_test_reports.md +++ b/doc/ci/unit_test_reports.md @@ -231,7 +231,7 @@ The [JunitXML.TestLogger](https://www.nuget.org/packages/JunitXml.TestLogger/) N package can generate test reports for .Net Framework and .Net Core applications. The following example expects a solution in the root folder of the repository, with one or more project files in sub-folders. One result file is produced per test project, and each file -is placed in a new artifacts folder. This example includes optional formatting arguments, which +is placed in the artifacts folder. This example includes optional formatting arguments, which improve the readability of test data in the test widget. A full .Net Core [example is available](https://gitlab.com/Siphonophora/dot-net-cicd-test-logging-demo). @@ -353,7 +353,7 @@ displays a list of test suites and cases reported from the XML file.  -You can view all the known test suites and click on each of these to see further +You can view all the known test suites and select each of these to see further details, including the cases that make up the suite. You can also retrieve the reports via the [GitLab API](../api/pipelines.md#get-a-pipelines-test-report). @@ -366,8 +366,7 @@ If parsing JUnit report XML results in an error, an indicator is shown next to t  -NOTE: -GitLab.com has a 500,000 [test case parsing limit](../user/gitlab_com/#gitlab-cicd). Self-managed administrators can manage this setting on their instance. +For test case parsing limits, see [Max test cases per unit test report](../user/gitlab_com/#gitlab-cicd). GitLab does not parse very [large nodes](https://nokogiri.org/tutorials/parsing_an_html_xml_document.html#parse-options) of JUnit reports. There is [an issue](https://gitlab.com/gitlab-org/gitlab/-/issues/268035) open to make this optional. diff --git a/doc/ci/variables/index.md b/doc/ci/variables/index.md index ba8451110eb..e4e562bb005 100644 --- a/doc/ci/variables/index.md +++ b/doc/ci/variables/index.md @@ -653,7 +653,7 @@ The order of precedence for variables is (from highest to lowest): 1. These all have the same (highest) precedence: - [Trigger variables](../triggers/index.md#pass-cicd-variables-in-the-api-call). - - [Scheduled pipeline variables](../pipelines/schedules.md#using-variables). + - [Scheduled pipeline variables](../pipelines/schedules.md#add-a-pipeline-schedule). - [Manual pipeline run variables](#override-a-variable-when-running-a-pipeline-manually). - Variables added when [creating a pipeline with the API](../../api/pipelines.md#create-a-new-pipeline). 1. Project [variables](#custom-cicd-variables). diff --git a/doc/ci/yaml/gitlab_ci_yaml.md b/doc/ci/yaml/gitlab_ci_yaml.md index 9bab4a7fc77..f3773fbf143 100644 --- a/doc/ci/yaml/gitlab_ci_yaml.md +++ b/doc/ci/yaml/gitlab_ci_yaml.md @@ -1,7 +1,7 @@ --- stage: Verify group: Pipeline Authoring -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: reference --- diff --git a/doc/ci/yaml/includes.md b/doc/ci/yaml/includes.md index 34db6c61d0b..e5a543e7130 100644 --- a/doc/ci/yaml/includes.md +++ b/doc/ci/yaml/includes.md @@ -302,7 +302,7 @@ In `include` sections in your `.gitlab-ci.yml` file, you can use: In GitLab 14.5 and later, you can also use: - [Trigger variables](../triggers/index.md#pass-cicd-variables-in-the-api-call). -- [Scheduled pipeline variables](../pipelines/schedules.md#using-variables). +- [Scheduled pipeline variables](../pipelines/schedules.md#add-a-pipeline-schedule). - [Manual pipeline run variables](../variables/index.md#override-a-variable-when-running-a-pipeline-manually). - Pipeline [predefined variables](../variables/predefined_variables.md). @@ -370,6 +370,11 @@ You can only use the following rules with `include` (and only with [certain vari `rules` keyword `changes` is not supported. +You cannot use [`needs:`](index.md#needs) to create a job dependency that points to +a job added with `include:local:rules`. When the configuration is checked for validity, +GitLab returns `undefined need: <job-name>`. An [issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/345377) +to improve this behavior. + ## Use `include:local` with wildcard file paths > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/25921) in GitLab 13.11. diff --git a/doc/ci/yaml/index.md b/doc/ci/yaml/index.md index d9ea5498b63..a3b91da8f08 100644 --- a/doc/ci/yaml/index.md +++ b/doc/ci/yaml/index.md @@ -135,7 +135,9 @@ The `include` files are: - Always evaluated first and then merged with the content of the `.gitlab-ci.yml` file, regardless of the position of the `include` keyword. -You can [nest](includes.md#use-nested-includes) up to 100 includes, but you can't have duplicate includes. +You can [nest](includes.md#use-nested-includes) up to 100 includes. In [GitLab 14.9 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/28987), +the same file can be included multiple times in nested includes, but duplicates are ignored. + In [GitLab 12.4 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/28212), the time limit to resolve all files is 30 seconds. @@ -1349,7 +1351,7 @@ In this example: **Additional details**: - Coverage regular expressions set in `gitlab-ci.yml` take precedence over coverage regular expression set in the - [GitLab UI](../pipelines/settings.md#add-test-coverage-results-to-a-merge-request-deprecated). + [GitLab UI](../pipelines/settings.md#add-test-coverage-results-using-project-settings-deprecated). - If there is more than one matched line in the job output, the last line is used (the first result of reverse search). - If there are multiple matches in a single line, the last match is searched @@ -1863,7 +1865,7 @@ image: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207484) in GitLab 12.9. -Use `inherit` to [control inheritance of globally-defined defaults and variables](../jobs/index.md#control-the-inheritance-of-default-keywords-and-global-variables). +Use `inherit` to [control inheritance of default keywords and variables](../jobs/index.md#control-the-inheritance-of-default-keywords-and-global-variables). #### `inherit:default` @@ -3690,6 +3692,61 @@ trigger_job: In this example, jobs from subsequent stages wait for the triggered pipeline to successfully complete before starting. +#### `trigger:forward` + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213729) in GitLab 14.9 [with a flag](../../administration/feature_flags.md) named `ci_trigger_forward_variables`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, +ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `ci_trigger_forward_variables`. +The feature is not ready for production use. + +Use `trigger:forward` to specify what to forward to the downstream pipeline. You can control +what is forwarded to both [parent-child pipelines](../pipelines/parent_child_pipelines.md) +and [multi-project pipelines](../pipelines/multi_project_pipelines.md). + +**Possible inputs**: + +- `yaml_variables`: `true` (default), or `false`. When `true`, variables defined + in the trigger job are passed to downstream pipelines. +- `pipeline_variables`: `true` or `false` (default). When `true`, [manual pipeline variables](../variables/index.md#override-a-defined-cicd-variable) + are passed to downstream pipelines. + +**Example of `trigger:forward`**: + +[Run this pipeline manually](../pipelines/index.md#run-a-pipeline-manually), with +the CI/CD variable `MYVAR = my value`: + +```yaml +variables: # default variables for each job + VAR: value + +# Default behavior: +# - VAR is passed to the child +# - MYVAR is not passed to the child +child1: + trigger: + include: .child-pipeline.yml + +# Forward pipeline variables: +# - VAR is passed to the child +# - MYVAR is passed to the child +child2: + trigger: + include: .child-pipeline.yml + forward: + pipeline_variables: true + +# Do not forward YAML variables: +# - VAR is not passed to the child +# - MYVAR is not passed to the child +child3: + trigger: + include: .child-pipeline.yml + forward: + yaml_variables: false +``` + ### `variables` [CI/CD variables](../variables/index.md) are configurable values that are passed to jobs. diff --git a/doc/development/adding_database_indexes.md b/doc/development/adding_database_indexes.md index 0e8e8289464..d263d9b5eb5 100644 --- a/doc/development/adding_database_indexes.md +++ b/doc/development/adding_database_indexes.md @@ -275,11 +275,11 @@ You can verify if the MR was deployed to GitLab.com by executing `/chatops run auto_deploy status <merge_sha>`. To verify existence of the index, you can: -- Use a meta-command in #database-lab, such as: `\d <index_name>` - - Ensure that the index is not [`invalid`](https://www.postgresql.org/docs/12/sql-createindex.html#:~:text=The%20psql%20%5Cd%20command%20will%20report%20such%20an%20index%20as%20INVALID) -- Ask someone in #database to check if the index exists +- Use a meta-command in #database-lab, such as: `\d <index_name>`. + - Ensure that the index is not [`invalid`](https://www.postgresql.org/docs/12/sql-createindex.html#:~:text=The%20psql%20%5Cd%20command%20will%20report%20such%20an%20index%20as%20INVALID). +- Ask someone in #database to check if the index exists. - With proper access, you can also verify directly on production or in a -production clone +production clone. ### Add a migration to create the index synchronously diff --git a/doc/development/agent/gitops.md b/doc/development/agent/gitops.md deleted file mode 100644 index 7c741408ae6..00000000000 --- a/doc/development/agent/gitops.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: 'https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/gitops.md' -remove_date: '2022-02-01' ---- - -This file was moved to [another location](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/gitops.md). - -<!-- This redirect file can be deleted after <2022-02-01>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/development/agent/identity.md b/doc/development/agent/identity.md deleted file mode 100644 index 6caf108a32a..00000000000 --- a/doc/development/agent/identity.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: 'https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/identity_and_auth.md' -remove_date: '2022-02-01' ---- - -This file was moved to [another location](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/identity_and_auth.md). - -<!-- This redirect file can be deleted after <2022-02-01>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/development/agent/index.md b/doc/development/agent/index.md deleted file mode 100644 index 474f8a02933..00000000000 --- a/doc/development/agent/index.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: 'https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/architecture.md' -remove_date: '2022-02-01' ---- - -This file was moved to [another location](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/architecture.md). - -<!-- This redirect file can be deleted after <2022-02-01>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/development/agent/local.md b/doc/development/agent/local.md deleted file mode 100644 index a4b29bea838..00000000000 --- a/doc/development/agent/local.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: 'https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/local.md' -remove_date: '2022-02-01' ---- - -This file was moved to [another location](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/local.md). - -<!-- This redirect file can be deleted after <2022-02-01>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/development/agent/repository_overview.md b/doc/development/agent/repository_overview.md deleted file mode 100644 index 8ea9dceb32a..00000000000 --- a/doc/development/agent/repository_overview.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: 'https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/repository_overview.md' -remove_date: '2022-02-01' ---- - -This file was moved to [another location](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/repository_overview.md). - -<!-- This redirect file can be deleted after <2022-02-01>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/development/agent/routing.md b/doc/development/agent/routing.md deleted file mode 100644 index 364267a45fe..00000000000 --- a/doc/development/agent/routing.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: 'https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/kas_request_routing.md' -remove_date: '2022-02-01' ---- - -This file was moved to [another location](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/kas_request_routing.md). - -<!-- This redirect file can be deleted after <2022-02-01>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/development/agent/user_stories.md b/doc/development/agent/user_stories.md deleted file mode 100644 index 2ed4bbdc9f6..00000000000 --- a/doc/development/agent/user_stories.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: 'https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/user_stories.md' -remove_date: '2022-02-01' ---- - -This file was moved to [another location](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/user_stories.md). - -<!-- This redirect file can be deleted after <2022-02-01>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/development/api_graphql_styleguide.md b/doc/development/api_graphql_styleguide.md index afd745533c9..417ccba26a0 100644 --- a/doc/development/api_graphql_styleguide.md +++ b/doc/development/api_graphql_styleguide.md @@ -442,10 +442,10 @@ booleans: ```ruby class MergeRequestPermissionsType < BasePermissionType - present_using MergeRequestPresenter - graphql_name 'MergeRequestPermissions' + present_using MergeRequestPresenter + abilities :admin_merge_request, :update_merge_request, :create_note ability_field :resolve_note, @@ -1329,6 +1329,10 @@ class UserUpdateMutation < BaseMutation end ``` +Due to changes in the `1.13` version of the `graphql-ruby` gem, `graphql_name` should be the first +line of the class to ensure that type names are generated correctly. The `Graphql::GraphqlNamePosition` cop enforces this. +See [issue #27536](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/27536#note_840245581) for further context. + Our GraphQL mutation names are historically inconsistent, but new mutation names should follow the convention `'{Resource}{Action}'` or `'{Resource}{Action}{Attribute}'`. @@ -1511,9 +1515,9 @@ GraphQL-name of the mutation: ```ruby module Types class MutationType < BaseObject - include Gitlab::Graphql::MountMutation + graphql_name 'Mutation' - graphql_name "Mutation" + include Gitlab::Graphql::MountMutation mount_mutation Mutations::MergeRequests::SetDraft end diff --git a/doc/development/architecture.md b/doc/development/architecture.md index e8d04d68565..dd432dd5e37 100644 --- a/doc/development/architecture.md +++ b/doc/development/architecture.md @@ -101,11 +101,11 @@ understand the GitLab architecture. A complete architecture diagram is available in our [component diagram](#component-diagram) below. - + <!-- -To update this diagram, GitLab team members can edit this source file: -https://docs.google.com/drawings/d/1fBzAyklyveF-i-2q-OHUIqDkYfjjxC4mq5shwKSZHLs/edit. +To update this diagram, use and update this source file: +https://miro.com/app/board/uXjVOH3lzXo=/ --> ### Component diagram @@ -151,7 +151,7 @@ graph LR NGINX -- TCP 8150 --> GitLabKas NGINX --> Registry %% inbound from GitLabShell - GitLabShell --TCP 8080 -->Puma + GitLabShell --> GitLabWorkhorse %% services Puma["Puma (GitLab Rails)"] @@ -349,7 +349,7 @@ Component statuses are linked to configuration documentation for each component. | [GitLab Exporter](#gitlab-exporter) | Generates a variety of GitLab metrics | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | CE & EE | | [GitLab Geo Node](#gitlab-geo) | Geographically distributed GitLab nodes | ⚙ | ⚙ | ❌ | ❌ | ✅ | ❌ | ⚙ | EE Only | | [GitLab Pages](#gitlab-pages) | Hosts static websites | ⚙ | ⚙ | ❌ | ❌ | ✅ | ⚙ | ⚙ | CE & EE | -| [GitLab Agent](#gitlab-agent) | Integrate Kubernetes clusters in a cloud-native way | ⚙ | ⚙ | ⚙ | ❌ | ❌ | ⤓ | ⚙ | EE Only | +| [GitLab agent](#gitlab-agent) | Integrate Kubernetes clusters in a cloud-native way | ⚙ | ⚙ | ⚙ | ❌ | ❌ | ⤓ | ⚙ | EE Only | | [GitLab self-monitoring: Alertmanager](#alertmanager) | Deduplicates, groups, and routes alerts from Prometheus | ⚙ | ⚙ | ✅ | ⚙ | ✅ | ❌ | ❌ | CE & EE | | [GitLab self-monitoring: Grafana](#grafana) | Metrics dashboard | ✅ | ✅ | ⚙ | ⤓ | ✅ | ❌ | ⚙ | CE & EE | | [GitLab self-monitoring: Jaeger](#jaeger) | View traces generated by the GitLab instance | ❌ | ⚙ | ⚙ | ❌ | ❌ | ⤓ | ⚙ | CE & EE | @@ -499,14 +499,14 @@ Geo is a premium feature built to help speed up the development of distributed t GitLab Exporter is a process designed in house that allows us to export metrics about GitLab application internals to Prometheus. You can read more [in the project's README](https://gitlab.com/gitlab-org/gitlab-exporter). -#### GitLab Agent +#### GitLab agent - [Project page](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent) - Configuration: - [Omnibus](https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/files/gitlab-config-template/gitlab.rb.template) - [Charts](https://docs.gitlab.com/charts/charts/gitlab/kas/index.html) -The [GitLab Agent](../user/clusters/agent/index.md) is an active in-cluster +The [GitLab agent](../user/clusters/agent/index.md) is an active in-cluster component for solving GitLab and Kubernetes integration tasks in a secure and cloud-native way. diff --git a/doc/development/backend/create_source_code_be/index.md b/doc/development/backend/create_source_code_be/index.md new file mode 100644 index 00000000000..6421ca3754a --- /dev/null +++ b/doc/development/backend/create_source_code_be/index.md @@ -0,0 +1,143 @@ +--- +stage: Create +group: Source Code +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Create: Source Code Backend + +The Create:Source Code BE team focuses on the GitLab suite of Source Code Management +(SCM) tools. It is responsible for all backend aspects of the product categories +that fall under the [Source Code group](https://about.gitlab.com/handbook/product/categories/#source-code-group) +of the [Create stage](https://about.gitlab.com/handbook/product/categories/#create-stage) +of the [DevOps lifecycle](https://about.gitlab.com/handbook/product/categories/#devops-stages). + +We interface with the Gitaly and Code Review teams, and work closely with the +[Create:Source Code Frontend team](https://about.gitlab.com/handbook/engineering/development/dev/create-source-code-fe). The features +we work with are listed on the +[Features by Group Page](https://about.gitlab.com/handbook/product/categories/features/#createsource-code-group). + +The team works across three codebases: Workhorse, GitLab Shell and GitLab Rails. + +## Workhorse + +GitLab Workhorse is a smart reverse proxy for GitLab. It handles "large" HTTP +requests such as file downloads, file uploads, `git push`, `git pull` and `git` archive downloads. + +Workhorse itself is not a feature, but there are several features in GitLab +that would not work efficiently without Workhorse. + +Workhorse documentation is available in the [Workhorse repository](https://gitlab.com/gitlab-org/gitlab/tree/master/workhorse). + +## GitLab Shell + +GitLab Shell handles Git SSH sessions for GitLab and modifies the list of authorized keys. +For more information, [refer to the README](https://gitlab.com/gitlab-org/gitlab-shell/-/blob/main/README.md). +for GitLab Shell. + +## GitLab Rails + +### Source code API endpoints + +| Endpoint | Threshold | Source | +| -----------------------------------------------------------------------------------|---------------------------------------|--------------------------------------------------------------------------------------| +| `DELETE /api/:version/projects/:id/protected_branches/:name` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/protected_branches.rb) | +| `GET /api/:version/internal/authorized_keys` | `:high` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/internal/base.rb) | | | +| `GET /api/:version/internal/lfs` | `:high` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/internal/lfs.rb)| +| `GET /api/:version/projects/:id/approval_rules` | `:low` | | +| `GET /api/:version/projects/:id/approval_settings` | default | | +| `GET /api/:version/projects/:id/approvals` | default | | +| `GET /api/:version/projects/:id/forks` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/projects.rb) | +| `GET /api/:version/projects/:id/groups` | default | [source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/projects.rb) | +| `GET /api/:version/projects/:id/languages` | `:medium` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/projects.rb) | +| `GET /api/:version/projects/:id/merge_request_approval_setting` | `:medium` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/api/merge_request_approval_settings.rb) | +| `GET /api/:version/projects/:id/merge_requests/:merge_request_iid/approval_rules` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/api/merge_request_approval_rules.rb) | +| `GET /api/:version/projects/:id/merge_requests/:merge_request_iid/approval_settings` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/api/project_approval_settings.rb) | +| `GET /api/:version/projects/:id/merge_requests/:merge_request_iid/approval_state` | `:low` | [source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/merge_request_approvals.rb) | +| `GET /api/:version/projects/:id/merge_requests/:merge_request_iid/approvals` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/merge_request_approvals.rb) | +| `GET /api/:version/projects/:id/protected_branches` | default |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/protected_branches.rb) | +| `GET /api/:version/projects/:id/protected_branches/:name` | default |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/protected_branches.rb) | +| `GET /api/:version/projects/:id/protected_tags` | default | | +| `GET /api/:version/projects/:id/protected_tags/:name` | default | | +| `GET /api/:version/projects/:id/push_rule` | default | | +| `GET /api/:version/projects/:id/remote_mirrors` | default | | +| `GET /api/:version/projects/:id/repository/archive` | default | | +| `GET /api/:version/projects/:id/repository/blobs/:sha` | default | | +| `GET /api/:version/projects/:id/repository/blobs/:sha/raw` | default | | +| `GET /api/:version/projects/:id/repository/branches` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/branches.rb) | +| `GET /api/:version/projects/:id/repository/branches/:branch` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/branches.rb) | +| `GET /api/:version/projects/:id/repository/commits` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/commits.rb)| +| `GET /api/:version/projects/:id/repository/commits/:sha` | default | [source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/commits.rb) | +| `GET /api/:version/projects/:id/repository/commits/:sha/comments` | default | [source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/commits.rb) | +| `GET /api/:version/projects/:id/repository/commits/:sha/diff` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/commits.rb) | +| `GET /api/:version/projects/:id/repository/commits/:sha/merge_requests` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/commits.rb)| +| `GET /api/:version/projects/:id/repository/commits/:sha/refs` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/commits.rb) | +| `GET /api/:version/projects/:id/repository/compare` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/repositories.rb) | +| `GET /api/:version/projects/:id/repository/contributors` | default | | +| `GET /api/:version/projects/:id/repository/files/:file_path` | default | | +| `GET /api/:version/projects/:id/repository/files/:file_path/raw` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/files.rb) | +| `GET /api/:version/projects/:id/repository/tags` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/tags.rb) | +| `GET /api/:version/projects/:id/repository/tree` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/repositories.rb) | +| `GET /api/:version/projects/:id/statistics` | default | | +| `GraphqlController#execute` | default | | +| `HEAD /api/:version/projects/:id/repository/files/:file_path` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/files.rb) | +| `HEAD /api/:version/projects/:id/repository/files/:file_path/raw` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/files.rb) | +| `POST /api/:version/internal/allowed` | default | [source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/internal/base.rb) | +| `POST /api/:version/internal/lfs_authenticate` | `:high` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/internal/base.rb) | +| `POST /api/:version/internal/post_receive` | default | [source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/internal/base.rb) | +| `POST /api/:version/internal/pre_receive` | `:high` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/internal/base.rb) | +| `POST /api/:version/projects/:id/approvals` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/lib/api/project_approvals.rb) | +| `POST /api/:version/projects/:id/merge_requests/:merge_request_iid/approvals` | `:low` | [source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/merge_request_approvals.rb) | +| `POST /api/:version/projects/:id/merge_requests/:merge_request_iid/approve` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/merge_request_approvals.rb) | +| `POST /api/:version/projects/:id/merge_requests/:merge_request_iid/unapprove` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/merge_request_approvals.rb)| +| `POST /api/:version/projects/:id/protected_branches` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/protected_branches.rb)| +| `POST /api/:version/projects/:id/repository/commits` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/commits.rb)| +| `POST /api/:version/projects/:id/repository/files/:file_path` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/files.rb) | +| `PUT /api/:version/projects/:id/push_rule` | default | | +| `PUT /api/:version/projects/:id/repository/files/:file_path` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/files.rb) | +| `Projects::BlameController#show` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/blame_controller.rb) | +| `Projects::BlobController#create` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/blob_controller.rb) | +| `Projects::BlobController#diff` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/blob_controller.rb) | +| `Projects::BlobController#edit` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/blob_controller.rb) | +| `Projects::BlobController#show` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/blob_controller.rb) | +| `Projects::BlobController#update` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/blob_controller.rb) | +| `Projects::BranchesController#create` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/branches_controller.rb) | +| `Projects::BranchesController#destroy` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/branches_controller.rb) | +| `Projects::BranchesController#diverging_commit_counts` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/branches_controller.rb) | +| `Projects::BranchesController#index` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/branches_controller.rb) | +| `Projects::BranchesController#new` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/branches_controller.rb) | +| `Projects::CommitController#branches` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/commit_controller.rb) | +| `Projects::CommitController#merge_requests` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/commit_controller.rb) | +| `Projects::CommitController#pipelines` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/commit_controller.rb) | +| `Projects::CommitController#show` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/commit_controller.rb) | +| `Projects::CommitsController#show` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/commits_controller.rb)| +| `Projects::CommitsController#signatures` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/commits_controller.rb) | +| `Projects::CompareController#create` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/commits_controller.rb) | +| `Projects::CompareController#index` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/compare_controller.rb) | +| `Projects::CompareController#show` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/compare_controller.rb) | +| `Projects::CompareController#signatures` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/compare_controller.rb) | +| `Projects::FindFileController#list` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/find_file_controller.rb) | +| `Projects::FindFileController#show` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/find_file_controller.rb) | +| `Projects::ForksController#index` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/forks_controller.rb) | +| `Projects::GraphsController#show` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/graphs_controller.rb) | +| `Projects::NetworkController#show` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/network_controller.rb) | +| `Projects::PathLocksController#index` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/controllers/projects/path_locks_controller.rb) | +| `Projects::RawController#show` | default | | +| `Projects::RefsController#logs_tree` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/refs_controller.rb) | +| `Projects::RefsController#switch` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/refs_controller.rb) | +| `Projects::RepositoriesController#archive` | default | | +| `Projects::Settings::RepositoryController#show` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/settings/repository_controller.rb) | +| `Projects::TagsController#index` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/tags_controller.rb) | +| `Projects::TagsController#new` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/tags_controller.rb) | +| `Projects::TagsController#show` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/tags_controller.rb) | +| `Projects::TemplatesController#names` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/templates_controller.rb) | +| `Projects::TreeController#show` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects/tree_controller.rb) | +| `ProjectsController#refs` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/projects_controller.rb) | +| `Repositories::GitHttpController#git_receive_pack` | default | | +| `Repositories::GitHttpController#git_upload_pack` | default | | +| `Repositories::GitHttpController#info_refs` | default | | +| `Repositories::LfsApiController#batch` | `:medium` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/repositories/lfs_api_controller.rb) | +| `Repositories::LfsLocksApiController#verify` | default | | +| `Repositories::LfsStorageController#download` | `:medium` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/repositories/lfs_storage_controller.rb) | +| `Repositories::LfsStorageController#upload_authorize` | `:medium` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/repositories/lfs_storage_controller.rb) | +| `Repositories::LfsStorageController#upload_finalize` | `:low` |[source](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/controllers/repositories/lfs_storage_controller.rb) | diff --git a/doc/development/background_migrations.md b/doc/development/background_migrations.md index 49835085f96..9fffbd25518 100644 --- a/doc/development/background_migrations.md +++ b/doc/development/background_migrations.md @@ -489,7 +489,8 @@ View the production Sidekiq log and filter for: - `json.class: BackgroundMigrationWorker` - `json.job_status: fail` -- `json.meta.caller_id: <MyBackgroundMigrationClassName>` +- `json.meta.caller_id: <MyBackgroundMigrationSchedulingMigrationClassName>` +- `json.args: <MyBackgroundMigrationClassName>` Looking at the `json.error_class`, `json.error_message` and `json.error_backtrace` values may be helpful in understanding why the jobs failed. diff --git a/doc/development/bulk_import.md b/doc/development/bulk_import.md index ff0c8a19ca1..a2620faed35 100644 --- a/doc/development/bulk_import.md +++ b/doc/development/bulk_import.md @@ -1,7 +1,7 @@ --- stage: Manage group: Import -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # GitLab Group Migration diff --git a/doc/development/caching.md b/doc/development/caching.md index 20847832e37..7c51bd595f7 100644 --- a/doc/development/caching.md +++ b/doc/development/caching.md @@ -265,6 +265,13 @@ All the time! - As the lookup is similar to a cache lookup (in the GitLab implementation), we can use the same key for both. This is how `Gitlab::Cache.fetch_once` works. +#### Possible downsides + +- Adding new attributes to a cached object using `Gitlab::JsonCache` + and `Gitlab::SafeRequestStore`, for example, can lead to stale data issues + where the cache data doesn't have the appropriate value for the new attribute + (see this past [incident](https://gitlab.com/gitlab-com/gl-infra/production/-/issues/6372)). + ### When to use SQL caching Rails uses this automatically for identical queries in a request, so no action is diff --git a/doc/development/changelog.md b/doc/development/changelog.md index b51db69c2f7..b98ed6cb109 100644 --- a/doc/development/changelog.md +++ b/doc/development/changelog.md @@ -101,6 +101,7 @@ EE: true - _Any_ contribution from a community member, no matter how small, **may** have a changelog entry regardless of these guidelines if the contributor wants one. - Any [GLEX experiment](experiment_guide/gitlab_experiment.md) changes **should not** have a changelog entry. +- An MR that includes only documentation changes **should not** have a changelog entry. For more information, see [how to handle changelog entries with feature flags](feature_flags/index.md#changelog). diff --git a/doc/development/cicd/index.md b/doc/development/cicd/index.md index 2779b457fb9..8677d5b08e3 100644 --- a/doc/development/cicd/index.md +++ b/doc/development/cicd/index.md @@ -7,9 +7,10 @@ type: index, concepts, howto # CI/CD development documentation **(FREE)** -Development guides that are specific to CI/CD are listed here. +Development guides that are specific to CI/CD are listed here: -If you are creating new CI/CD templates, please read [the development guide for GitLab CI/CD templates](templates.md). +- If you are creating new CI/CD templates, please read [the development guide for GitLab CI/CD templates](templates.md). +- If you are adding a new keyword or changing the CI schema, check the [CI schema guide](schema.md) See the [CI/CD YAML reference documentation guide](cicd_reference_documentation_guide.md) to learn how to update the [reference page](../../ci/yaml/index.md). @@ -29,7 +30,7 @@ On the left side we have the events that can trigger a pipeline based on various - A user clicking the "Run pipeline" button in the UI. - When a [merge request is created or updated](../../ci/pipelines/merge_request_pipelines.md). - When an MR is added to a [Merge Train](../../ci/pipelines/merge_trains.md#merge-trains). -- A [scheduled pipeline](../../ci/pipelines/schedules.md#pipeline-schedules). +- A [scheduled pipeline](../../ci/pipelines/schedules.md). - When project is [subscribed to an upstream project](../../ci/pipelines/multi_project_pipelines.md#trigger-a-pipeline-when-an-upstream-project-is-rebuilt). - When [Auto DevOps](../../topics/autodevops/index.md) is enabled. - When GitHub integration is used with [external pull requests](../../ci/ci_cd_for_external_repos/index.md#pipelines-for-external-pull-requests). diff --git a/doc/development/cicd/schema.md b/doc/development/cicd/schema.md new file mode 100644 index 00000000000..b63d951b881 --- /dev/null +++ b/doc/development/cicd/schema.md @@ -0,0 +1,146 @@ +--- +stage: Verify +group: Pipeline Authoring +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +type: index, howto +--- + +# Contribute to the CI Schema **(FREE)** + +The [pipeline editor](../../ci/pipeline_editor/index.md) uses a CI schema to enhance +the authoring experience of our CI configuration files. With the CI schema, the editor can: + +- Validate the content of the CI configuration file as it is being written in the editor. +- Provide autocomplete functionality and suggest available keywords. +- Provide definitions of keywords through annotations. + +As the rules and keywords for configuring our CI configuration files change, so too +should our CI schema. + +This feature is behind the [`schema_linting`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/feature_flags/development/schema_linting.yml) +feature flag for self-managed instances, and is enabled for GitLab.com. + +## JSON Schemas + +The CI schema follows the [JSON Schema Draft-07](https://json-schema.org/draft-07/json-schema-release-notes.html) +specification. Although the CI configuration file is written in YAML, it is converted +into JSON by using `monaco-yaml` before it is validated by the CI schema. + +If you're new to JSON schemas, consider checking out +[this guide](https://json-schema.org/learn/getting-started-step-by-step) for +a step-by-step introduction on how to work with JSON schemas. + +## Update Keywords + +The CI schema is at [`app/assets/javascripts/editor/schema/ci.json`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/editor/schema/ci.json). +It contains all the keywords available for authoring CI configuration files. +Check the [keyword reference](../../ci/yaml/index.md) for a comprehensive list of +all available keywords. + +All keywords are defined under `definitions`. We use these definitions as +[references](https://json-schema.org/learn/getting-started-step-by-step#references) +to share common data structures across the schema. + +For example, this defines the `retry` keyword: + +```json +{ + "definitions": { + "retry": { + "description": "Retry a job if it fails. Can be a simple integer or object definition.", + "oneOf": [ + { + "$ref": "#/definitions/retry_max" + }, + { + "type": "object", + "additionalProperties": false, + "properties": { + "max": { + "$ref": "#/definitions/retry_max" + }, + "when": { + "description": "Either a single or array of error types to trigger job retry.", + "oneOf": [ + { + "$ref": "#/definitions/retry_errors" + }, + { + "type": "array", + "items": { + "$ref": "#/definitions/retry_errors" + } + } + ] + } + } + } + ] + } + } +} +``` + +With this definition, the `retry` keyword is both a property of +the `job_template` definition and the `default` global keyword. Global keywords +that configure pipeline behavior (such as `workflow` and `stages`) are defined +under the topmost **properties** key. + +```json +{ + "properties": { + "default": { + "type": "object", + "properties": { + "retry": { + "$ref": "#/definitions/retry" + }, + } + } + }, + "definitions": { + "job_template": { + "properties": { + "retry": { + "$ref": "#/definitions/retry" + } + }, + } + } +} +``` + +## Guidelines for updating the schema + +- Keep definitions atomic when possible, to be flexible with + referencing keywords. For example, `workflow:rules` uses only a subset of + properties in the `rules` definition. The `rules` properties have their + own definitions, so we can reference them individually. +- When adding new keywords, consider adding a `description` with a link to the + keyword definition in the documentation. This information shows up in the annotations + when the user hovers over the keyword. +- For each property, consider if a `minimum`, `maximum`, or + `default` values are required. Some values might be required, and in others we can set + blank. In the blank case, we can add the following to the definition: + +```json +{ + "keyword": { + "oneOf": [ + { + "type": "null" + }, + ... + ] + } +} +``` + +## Test the schema + +For now, the CI schema can only be tested manually. To verify the behavior is correct: + +1. Enable the `schema_linting` feature flag. +1. Go to **CI/CD** > **Editor**. +1. Write your CI/CD configuration in the editor and verify that the schema validates + it correctly. diff --git a/doc/development/cicd/templates.md b/doc/development/cicd/templates.md index d7edad842b8..c6f59a7e452 100644 --- a/doc/development/cicd/templates.md +++ b/doc/development/cicd/templates.md @@ -96,7 +96,7 @@ Additional points to keep in mind when authoring templates: |------------------------------------------------------|--------------------|---------------| | Can use global keywords, including `stages`. | Yes | No | | Can define jobs. | Yes | Yes | -| Can be selected in the new file UI | Yes | Yes | +| Can be selected in the new file UI | Yes | No | | Can include other job templates with `include` | Yes | No | | Can include other pipeline templates with `include`. | No | No | @@ -105,6 +105,16 @@ Additional points to keep in mind when authoring templates: To make templates easier to follow, templates should all use clear syntax styles, with a consistent format. +The `before_script`, `script`, and `after_script` keywords of every job are linted +using [ShellCheck](https://www.shellcheck.net/) and should follow the +[Shell scripting standards and style guidelines](../shell_scripting_guide/index.md) +as much as possible. + +ShellCheck assumes that the script is designed to run using [Bash](https://www.gnu.org/software/bash/). +Templates which use scripts for shells that aren't compatible with the Bash ShellCheck +rules can be excluded from ShellCheck linting. To exclude a script, add it to the +`EXCLUDED_TEMPLATES` list in [`scripts/lint_templates_bash.rb`](https://gitlab.com/gitlab-org/gitlab/-/tree/master/scripts/lint_templates_bash.rb). + #### Do not hardcode the default branch Use [`$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH`](../../ci/variables/predefined_variables.md) @@ -219,6 +229,30 @@ job1: - echo ${ERROR_MESSAGE} ``` +#### Use all-caps naming for non-local variables + +If you are expecting a variable to be provided via the CI/CD settings, or via the +`variables` keyword, that variable must use all-caps naming with underscores (`_`) +separating words. + +```yaml +.with_login: + before_script: + # SECRET_TOKEN should be provided via the project settings + - docker login -u my-user -p "$SECRET_TOKEN my-registry +``` + +Lower-case naming can optionally be used for variables which are defined locally in +one of the `script` keywords: + +```yaml +job1: + script: + - response="$(curl "https://example.com/json")" + - message="$(echo "$response" | jq -r .message)" + - 'echo "Server responded with: $message"' +``` + ### Backward compatibility A template might be dynamically included with the `include:template:` keyword. If diff --git a/doc/development/code_review.md b/doc/development/code_review.md index 3664ca7642a..ec913df8e4a 100644 --- a/doc/development/code_review.md +++ b/doc/development/code_review.md @@ -74,19 +74,27 @@ It picks reviewers and maintainers from the list at the page, with these behaviors: 1. It doesn't pick people whose Slack or [GitLab status](../user/profile/index.md#set-your-current-status): - - contains the string 'OOO', 'PTO', 'Parental Leave', or 'Friends and Family' - - emoji is `:palm_tree:`, `:beach:`, `:beach_umbrella:`, `:beach_with_umbrella:`, `:ferris_wheel:`, `:thermometer:`, `:face_with_thermometer:`, `:red_circle:`, `:bulb:`, `:sun_with_face:`. - - GitLab user busy indicator is set to true + - Contains the string 'OOO', 'PTO', 'Parental Leave', or 'Friends and Family'. + - GitLab user **Busy** indicator is set to `True`. + - Emoji is any of: + - 🌴 `:palm_tree:` + - 🏖️ `:beach:`, `:beach_umbrella:`, or `:beach_with_umbrella:` + - 🎡 `:ferris_wheel:` + - 🌡️ `:thermometer:` + - 🤒 `:face_with_thermometer:` + - 🔴 `:red_circle:` + - 💡 `:bulb:` + - 🌞 `:sun_with_face:` 1. [Trainee maintainers](https://about.gitlab.com/handbook/engineering/workflow/code-review/#trainee-maintainer) are three times as likely to be picked as other reviewers. 1. Team members whose Slack or [GitLab status](../user/profile/index.md#set-your-current-status) emoji is 🔵 `:large_blue_circle:` are more likely to be picked. This applies to both reviewers and trainee maintainers. - - Reviewers with `:large_blue_circle:` are two times as likely to be picked as other reviewers. - - Trainee maintainers with `:large_blue_circle:` are four times as likely to be picked as other reviewers. + - Reviewers with 🔵 `:large_blue_circle:` are two times as likely to be picked as other reviewers. + - Trainee maintainers with 🔵 `:large_blue_circle:` are four times as likely to be picked as other reviewers. 1. People whose [GitLab status](../user/profile/index.md#set-your-current-status) emoji - is 🔶 `:large_orange_diamond:` are half as likely to be picked. This applies to both reviewers and trainee maintainers. + is 🔶 `:large_orange_diamond:` or 🔸 `:small_orange_diamond:` are half as likely to be picked. This applies to both reviewers and trainee maintainers. 1. It always picks the same reviewers and maintainers for the same - branch name (unless their OOO status changes, as in point 1). It + branch name (unless their out-of-office (OOO) status changes, as in point 1). It removes leading `ce-` and `ee-`, and trailing `-ce` and `-ee`, so that it can be stable for backport branches. @@ -624,7 +632,7 @@ Enterprise Edition instance. This has some implications: [added to Omnibus](https://docs.gitlab.com/omnibus/settings/gitlab.yml#adding-a-new-setting-to-gitlabyml). 1. **File system access** is not possible in a [cloud-native architecture](architecture.md#adapting-existing-and-introducing-new-components). Ensure that we support object storage for any file storage we need to perform. For more - information, see the [uploads documentation](uploads.md). + information, see the [uploads documentation](uploads/index.md). ### Review turnaround time diff --git a/doc/development/contributing/design.md b/doc/development/contributing/design.md index efa8d4b0c41..463a7ee0e0b 100644 --- a/doc/development/contributing/design.md +++ b/doc/development/contributing/design.md @@ -35,7 +35,7 @@ Check these aspects both when _designing_ and _reviewing_ UI changes. - Use clear and consistent [terminology](https://design.gitlab.com/content/terminology/). - Check grammar and spelling. - Consider help content and follow its [guidelines](https://design.gitlab.com/usability/helping-users/). -- Request review from the [appropriate Technical Writer](https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers), +- Request review from the [appropriate Technical Writer](https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments), indicating any specific files or lines they should review, and how to preview or understand the location/context of the text from the user's perspective. diff --git a/doc/development/contributing/issue_workflow.md b/doc/development/contributing/issue_workflow.md index ad8403d242c..4db686b9b1e 100644 --- a/doc/development/contributing/issue_workflow.md +++ b/doc/development/contributing/issue_workflow.md @@ -45,7 +45,7 @@ scheduling into milestones. Labeling is a task for everyone. (For some projects, Most issues will have labels for at least one of the following: -- Type. For example: `~"type::feature"`, `~"type::bug"`, or `~"type::tooling"`. +- Type. For example: `~"type::feature"`, `~"type::bug"`, or `~"type::maintenance"`. - Stage. For example: `~"devops::plan"` or `~"devops::create"`. - Group. For example: `~"group::source code"`, `~"group::knowledge"`, or `~"group::editor"`. - Category. For example: `~"Category:Code Analytics"`, `~"Category:DevOps Reports"`, or `~"Category:Templates"`. @@ -72,19 +72,7 @@ labels, you can _always_ add the type, stage, group, and often the category/feat Type labels are very important. They define what kind of issue this is. Every issue should have one and only one. -The current type labels are: - -- `~"type::feature"` - - `~"feature::addition"` - - `~"feature::enhancement"` -- `~"type::maintenance"` -- `~"type::bug"` -- `~"type::tooling"` - - `~"tooling::pipelines"` - - `~"tooling::workflow"` -- `~"support request"` -- `~meta` -- `~documentation` +The current type labels are [available in the handbook](https://about.gitlab.com/handbook/engineering/metrics/#work-type-classification) A number of type labels have a priority assigned to them, which automatically makes them float to the top, depending on their importance. diff --git a/doc/development/contributing/merge_request_workflow.md b/doc/development/contributing/merge_request_workflow.md index 02148f2a717..a9b4d13ab06 100644 --- a/doc/development/contributing/merge_request_workflow.md +++ b/doc/development/contributing/merge_request_workflow.md @@ -81,7 +81,7 @@ request is as follows: 1. If your MR touches code that executes shell commands, reads or opens files, or handles paths to files on disk, make sure it adheres to the [shell command guidelines](../shell_commands.md) -1. If your code needs to handle file storage, see the [uploads documentation](../uploads.md). +1. If your code needs to handle file storage, see the [uploads documentation](../uploads/index.md). 1. If your merge request adds one or more migrations, make sure to execute all migrations on a fresh database before the MR is reviewed. If the review leads to large changes in the MR, execute the migrations again once the review is complete. @@ -264,8 +264,11 @@ requirements. 1. Peer member testing is optional but recommended when the risk of a change is high. This includes when the changes are [far-reaching](https://about.gitlab.com/handbook/engineering/development/#reducing-the-impact-of-far-reaching-work) or are for [components critical for security](../code_review.md#security). 1. Regressions and bugs are covered with tests that reduce the risk of the issue happening again. +1. Code affected by a feature flag is covered by [automated tests with the feature flag enabled and disabled](../feature_flags/index.md#feature-flags-in-tests), or both + states are tested as part of peer member testing or as part of the rollout plan. 1. [Performance guidelines](../merge_request_performance_guidelines.md) have been followed. 1. [Secure coding guidelines](https://gitlab.com/gitlab-com/gl-security/security-guidelines) have been followed. +1. [Application and rate limit guidelines](../merge_request_application_and_rate_limit_guidelines.md) have been followed. 1. [Documented](../documentation/index.md) in the `/doc` directory. 1. [Changelog entry added](../changelog.md), if necessary. 1. Reviewed by relevant reviewers, and all concerns are addressed for Availability, Regressions, and Security. Documentation reviews should take place as soon as possible, but they should not block a merge request. diff --git a/doc/development/contributing/style_guides.md b/doc/development/contributing/style_guides.md index da926005466..7a4ebbdbadf 100644 --- a/doc/development/contributing/style_guides.md +++ b/doc/development/contributing/style_guides.md @@ -159,25 +159,22 @@ When the number of RuboCop exceptions exceed the default [`exclude-limit` of 15] we may want to resolve exceptions over multiple commits. To minimize confusion, we should track our progress through the exception list. -When auto-generating the `.rubocop_todo.yml` exception list for a particular Cop, -and more than 15 files are affected, we should add the exception list to -a different file in the directory `.rubocop_todo/`. For example, the configuration for the cop -`Gitlab/NamespacedClass` is in `.rubocop_todo/gitlab/namespaced_class.yml`. - -This ensures that our list isn't mistakenly removed by another auto generation of -the `.rubocop_todo.yml`. This also allows us greater visibility into the exceptions -which are currently being resolved. - -One way to generate the initial list is to run the Rake task `rubocop:todo:generate`: +The preferred way to [generate the initial list or a list for specific RuboCop rules](../rake_tasks.md#generate-initial-rubocop-todo-list) +is to run the Rake task `rubocop:todo:generate`: ```shell +# Initial list bundle exec rake rubocop:todo:generate + +# List for specific RuboCop rules +bundle exec rake 'rubocop:todo:generate[Gitlab/NamespacedClass,Lint/Syntax]' ``` -You can then move the list from the freshly generated `.rubocop_todo.yml` for the Cop being actively -resolved and place it in the directory `.rubocop_todo/`. In this scenario, do not commit -auto-generated changes to the `.rubocop_todo.yml`, as an `exclude limit` that is higher than 15 -makes the `.rubocop_todo.yml` hard to parse. +This Rake task creates or updates the exception list in `.rubocop_todo/`. For +example, the configuration for the RuboCop rule `Gitlab/NamespacedClass` is +located in `.rubocop_todo/gitlab/namespaced_class.yml`. + +Make sure to commit any changes in `.rubocop_todo/` after running the Rake task. ### Reveal existing RuboCop exceptions diff --git a/doc/development/contributing/verify/index.md b/doc/development/contributing/verify/index.md new file mode 100644 index 00000000000..a2bb0eca733 --- /dev/null +++ b/doc/development/contributing/verify/index.md @@ -0,0 +1,236 @@ +--- +type: reference, dev +stage: none +group: Verify +--- + +# Contribute to Verify stage codebase + +## What are we working on in Verify? + +Verify stage is working on a comprehensive Continuous Integration platform +integrated into the GitLab product. Our goal is to empower our users to make +great technical and business decisions, by delivering a fast, reliable, secure +platform that verifies assumptions that our users make, and check them against +the criteria defined in CI/CD configuration. They could be unit tests, end-to-end +tests, benchmarking, performance validation, code coverage enforcement, and so on. + +Feedback delivered by GitLab CI/CD makes it possible for our users to make well +informed decisions about technological and business choices they need to make +to succeed. Why is Continuous Integration a mission critical product? + +GitLab CI/CD is our platform to deliver feedback to our users and customers. + +They contribute their continuous integration configuration files +`.gitlab-ci.yml` to describe the questions they want to get answers for. Each +time someone pushes a commit or triggers a pipeline we need to find answers for +very important questions that have been asked in CI/CD configuration. + +Failing to answer these questions or, what might be even worse, providing false +answers, might result in a user making a wrong decision. Such wrong decisions +can have very severe consequences. + +## Core principles of our CI/CD platform + +Data produced by the platform should be: + +1. Accurate. +1. Durable. +1. Accessible. + +The platform itself should be: + +1. Reliable. +1. Secure. +1. Deterministic. +1. Trustworthy. +1. Fast. +1. Simple. + +Since the inception of GitLab CI/CD, we have lived by these principles, +and they serve us and our users well. Some examples of these principles are that: + +- The feedback delivered by GitLab CI/CD and data produced by the platform should be accurate. + If a job fails and we notify a user that it was successful, it can have severe negative consequences. +- Feedback needs to be available when a user needs it and data can not disappear unexpectedly when engineers need it. +- It all doesn’t matter if the platform is not secure and we +are leaking credentials or secrets. +- When a user provides a set of preconditions in a form of CI/CD configuration, the result should be deterministic each time a pipeline runs, because otherwise the platform might not be trustworthy. +- If it is fast, simple to use and has a great UX it will serve our users well. + +## Building things in Verify + +### Measure before you optimize, and make data-informed decisions + +It is very difficult to optimize something that you can not measure. How would you +know if you succeeded, or how significant the success was? If you are working on +a performance or reliability improvement, make sure that you measure things before +you optimize them. + +The best way to measure stuff is to add a Prometheus metric. Counters, gauges, and +histograms are great ways to quickly get approximated results. Unfortunately this +is not the best way to measure tail latency. Prometheus metrics, especially histograms, +are usually approximations. + +If you have to measure tail latency, like how slow something could be or how +large a request payload might be, consider adding custom application logs and +always use structured logging. + +It's useful to use profiling and flamegraphs to understand what the code execution +path truly looks like! + +### Strive for simple solutions, avoid clever solutions + +It is sometimes tempting to use a clever solution to deliver something more +quickly. We want to avoid shipping clever code, because it is usually more +difficult to understand and maintain in the long term. Instead, we want to +focus on boring solutions that make it easier to evolve the codebase and keep the +contribution barrier low. We want to find solutions that are as simple as +possible. + +### Do not confuse boring solutions with easy solutions + +Boring solutions are sometimes confused with easy solutions. Very often the +opposite is true. An easy solution might not be simple - for example, a complex +new library can be included to add a very small functionality that otherwise +could be implemented quickly - it is easier to include this library than to +build this thing, but it would bring a lot of complexity into the product. + +On the other hand, it is also possible to over-engineer a solution when a simple, +well tested, and well maintained library is available. In that case using the +library might make sense. We recognize that we are constantly balancing simple +and easy solutions, and that finding the right balance is important. + +### "Simple" is not mutually exclusive with "flexible" + +Building simple things does not mean that more advanced and flexible solutions +will not be available. A good example here is an expanding complexity of +writing `.gitlab-ci.yml` configuration. For example, you can use a simple +method to define an environment name: + +```yaml +deploy: + environment: production + script: cap deploy +``` + +But the `environment` keyword can be also expanded into another level of +configuration that can offer more flexibility. + +```yaml +deploy: + environment: + name: review/$CI_COMMIT_REF_SLUG + url: https://prod.example.com + script: cap deploy +``` + +This kind of approach shields new users from the complexities of the platform, +but still allows them to go deeper if they need to. This approach can be +applied to many other technical implementations. + +### Make things observable + +GitLab is a DevOps platform. We popularize DevOps because it helps companies +be more efficient and achieve better results. One important component of +DevOps culture is to take ownership over features and code that you are +building. It is very difficult to do that when you don’t know how your features +perform and behave in the production environment. + +This is why we want to make our features and code observable. It +should be written in a way that an author can understand how well or how poorly +the feature or code behaves in the production environment. We usually accomplish +that by introducing the proper mix of Prometheus metrics and application +loggers. + +**TODO** document when to use Prometheus metrics, when to use loggers. Write a +few sentences about histograms and counters. Write a few sentences highlighting +importance of metrics when doing incremental rollouts. + +### Protect customer data + +Making data produced by our CI/CD platform durable is important. We recognize that +data generated in the CI/CD by users and customers is +something important and we must protect it. This data is not only important +because it can contain important information, we also do have compliance and +auditing responsibilities. + +Therefore we must take extra care when we are writing migrations +that permanently removes data from our database, or when we are define +new retention policies. + +As a general rule, when you are writing code that is supposed to remove +data from the database, file system, or object storage, you should get an extra pair +of eyes on your changes. When you are defining a new retention policy, you +should double check with PMs and EMs. + +### Get your changes reviewed + +When your merge request is ready for reviews you must assign +reviewers and then maintainers. Depending on the complexity of a change, you +might want to involve the people that know the most about the codebase area you are +changing. We do have many domain experts in Verify and it is absolutely acceptable to +ask them to review your code when you are not certain if a reviewer or +maintainer assigned by the Reviewer Roulette has enough context about the +change. + +The reviewer roulette offers useful suggestions, but as assigning the right +reviewers is important it should not be done automatically every time. It might +not make sense to assign someone who knows nothing about the area you are +updating, because their feedback might be limited to code style and syntax. +Depending on the complexity and impact of a change, assigning the right people +to review your changes might be very important. + +If you don’t know who to assign, consult `git blame` or ask in the `#verify` +Slack channel (GitLab team members only). + +### Incremental rollouts + +After your merge request is merged by a maintainer, it is time to release it to +users and the wider community. We usually do this with feature flags. +While not every merge request needs a feature flag, most merge +requests in Verify should have feature flags. [**TODO** link to docs about what +needs a feature flag and what doesn’t]. + +If you already follow the advice on this page, you probably already have a +few metrics and perhaps a few loggers added that make your new code observable +in the production environment. You can now use these metrics to incrementally +roll out your changes! + +A typical scenario involves enabling a few features in a few internal projects +while observing your metrics or loggers. Be aware that there might be a +small delay involved in ingesting logs in Elastic or Kibana. After you confirm +the feature works well with internal projects you can start an +incremental rollout for other projects. + +Avoid using "percent of time" incremental rollouts. These are error prone, +especially when you are checking feature flags in a few places in the codebase +and you have not memoized the result of a check in a single place. + +### Do not cause our Universe to implode + +During one of the first GitLab Contributes events we had a discussion about the importance +of keeping CI/CD pipeline, stage, and job statuses accurate. We considered a hypothetical +scenario relating to a software being built by one of our [early customers](https://about.gitlab.com/blog/2016/11/23/gitlab-adoption-growing-at-cern/) + +> What happens if software deployed to the [Large Hadron Collider (LHC)](https://en.wikipedia.org/wiki/Large_Hadron_Collider), +> breaks because of a bug in GitLab CI/CD that showed that a pipeline +> passed, but this data was not accurate and the software deployed was actually +> invalid? A problem like this could cause the LHC to malfunction, which +> could generate a new particle that would then cause the universe to implode. + +That would be quite an undesirable outcome of a small bug in GitLab CI/CD status +processing. Please take extra care when you are working on CI/CD statuses, +we don’t want to implode our Universe! + +This is an extreme and unlikely scenario, but presenting data that is not accurate +can potentially cause a myriad of problems through the +[butterfly effect](https://en.wikipedia.org/wiki/Butterfly_effect). +There are much more likely scenarios that +can have disastrous consequences. GitLab CI/CD is being used by companies +building medical, aviation, and automotive software. Continuous Integration is +a mission critical part of software engineering. + +When you are working on a subsystem for pipeline processing and transitioning +CI/CD statuses, request an additional review from a domain expert and hold +others accountable for doing the same. diff --git a/doc/development/dangerbot.md b/doc/development/dangerbot.md index 8da1f5700e5..9bf0fbe1d78 100644 --- a/doc/development/dangerbot.md +++ b/doc/development/dangerbot.md @@ -121,12 +121,13 @@ to revert the change before merging! #### Adding labels via Danger NOTE: -This is currently applicable to the [`gitlab-org/gitlab`](https://gitlab.com/gitlab-org/gitlab) -project only. +This is applicable to all the projects that use the [`gitlab-dangerfiles` gem](https://rubygems.org/gems/gitlab-dangerfiles). Danger is often used to improve MR hygiene by adding labels. Instead of calling the -API directly in your `Dangerfile`, add the labels to the `project_helper.labels_to_add` array. -The main `Dangerfile` will then take care of adding the labels to the MR with a single API call. +API directly in your `Dangerfile`, add the labels to `helper.labels_to_add` array (with `helper.labels_to_add << label` +or `helper.labels_to_add.concat(array_of_labels)`. +`gitlab-dangerfiles` will then take care of adding the labels to the MR with a single API call after all the rules +have had the chance to add to `helper.labels_to_add`. #### Shared rules and plugins @@ -135,11 +136,30 @@ upstreaming them to the [`gitlab-dangerfiles`](https://gitlab.com/gitlab-org/rub #### Enable Danger on a project -To enable the Dangerfile on another existing GitLab project, run the following -extra steps: +To enable the Dangerfile on another existing GitLab project, complete the following steps: -1. Create a [Project access tokens](../user/project/settings/project_access_tokens.md). -1. Add the token as a CI/CD project variable named `DANGER_GITLAB_API_TOKEN`. +1. Add [`gitlab-dangerfiles`](https://rubygems.org/gems/gitlab-dangerfiles) to your `Gemfile`. +1. Create a `Dangerfile` with the following content: + + ```ruby + require_relative "lib/gitlab-dangerfiles" + + Gitlab::Dangerfiles.for_project(self, &:import_defaults) + ``` + +1. Add the following to your CI/CD configuration: + + ```yaml + include: + - project: 'gitlab-org/quality/pipeline-common' + file: + - '/ci/danger-review.yml' + ``` + +1. If your project is in the `gitlab-org` group, you don't need to set up any token as the `DANGER_GITLAB_API_TOKEN` + variable is available at the group level. If not, follow these last steps: + 1. Create a [Project access tokens](../user/project/settings/project_access_tokens.md). + 1. Add the token as a CI/CD project variable named `DANGER_GITLAB_API_TOKEN`. You should add the ~"Danger bot" label to the merge request before sending it for review. diff --git a/doc/development/database/database_reviewer_guidelines.md b/doc/development/database/database_reviewer_guidelines.md index bc18e606f21..9d5e4821c9f 100644 --- a/doc/development/database/database_reviewer_guidelines.md +++ b/doc/development/database/database_reviewer_guidelines.md @@ -26,7 +26,7 @@ For more information on the database review process, check the [database review ## How to apply for becoming a database reviewer -Team members are encouraged to self-identify as database domain experts and add it to their [team profile](https://gitlab.com/gitlab-com/www-gitlab-com/-/blob/master/data/team.yml) +Team members are encouraged to self-identify as database domain experts, and add it to their profile YAML file: ```yaml projects: @@ -34,10 +34,11 @@ projects: - reviewer database ``` -Assign the MR which adds your expertise to the `team.yml` file to a database maintainer -or the [Database Team's Engineering Manager](https://about.gitlab.com/handbook/engineering/development/enablement/database/). +Create the merge request [using the "Database reviewer" template](https://gitlab.com/gitlab-com/www-gitlab-com/-/blob/master/.gitlab/merge_request_templates/Database%20reviewer.md), +adding your expertise your profile YAML file. Assign to a database maintainer or the +[Database Team's Engineering Manager](https://about.gitlab.com/handbook/engineering/development/enablement/database/). -Once the `team.yml` update is merged, the [Reviewer roulette](../code_review.md#reviewer-roulette) +After the `team.yml` update is merged, the [Reviewer roulette](../code_review.md#reviewer-roulette) may recommend you as a database reviewer. ## Resources for database reviewers diff --git a/doc/development/database/loose_foreign_keys.md b/doc/development/database/loose_foreign_keys.md index d08e90683fe..17a825b4812 100644 --- a/doc/development/database/loose_foreign_keys.md +++ b/doc/development/database/loose_foreign_keys.md @@ -50,6 +50,107 @@ we can: NOTE: For this procedure to work, we must register which tables to clean up asynchronously. +## The `scripts/decomposition/generate-loose-foreign-key` + +We built an automation tool to aid migration of foreign keys into loose foreign keys as part of +decomposition effort. It presents existing keys and allows chosen foreign keys to be automatically +converted into loose foreign keys. This ensures consistency between foreign key and loose foreign +key definitions, and ensures that they are properly tested. + +WARNING: +We strongly advise you to use the automation script for swapping any foreign key to a loose foreign key. + +The tool ensures that all aspects of swapping a foreign key are covered. This includes: + +- Creating a migration to remove a foreign key. +- Updating `db/structure.sql` with the new migration. +- Updating `lib/gitlab/database/gitlab_loose_foreign_keys.yml` to add the new loose foreign key. +- Creating or updating a model's specs to ensure that the loose foreign key is properly supported. +- Creating a new branch, commit, push, and creating a merge request on GitLab.com. +- Creating a merge request template with all the necessary details to validate the safety of the foreign key removal. + +The tool is located at `scripts/decomposition/generate-loose-foreign-key`: + +```shell +$ scripts/decomposition/generate-loose-foreign-key -h + +Usage: scripts/decomposition/generate-loose-foreign-key [options] <filters...> + -c, --cross-schema Show only cross-schema foreign keys + -n, --dry-run Do not execute any commands (dry run) + -b, --[no-]branch Create or not a new branch + -r, --[no-]rspec Create or not a rspecs automatically + -m, --milestone MILESTONE Specify custom milestone (current: 14.8) + -h, --help Prints this help +``` + +For the migration of cross-schema foreign keys, we use the `-c` modifier to show the foreign keys +yet to migrate: + +```shell +$ scripts/decomposition/generate-loose-foreign-key -c +Re-creating current test database +Dropped database 'gitlabhq_test_ee' +Dropped database 'gitlabhq_geo_test_ee' +Created database 'gitlabhq_test_ee' +Created database 'gitlabhq_geo_test_ee' + +Showing cross-schema foreign keys (20): + ID | HAS_LFK | FROM | TO | COLUMN | ON_DELETE + 0 | N | ci_builds | projects | project_id | cascade + 1 | N | ci_job_artifacts | projects | project_id | cascade + 2 | N | ci_pipelines | projects | project_id | cascade + 3 | Y | ci_pipelines | merge_requests | merge_request_id | cascade + 4 | N | external_pull_requests | projects | project_id | cascade + 5 | N | ci_sources_pipelines | projects | project_id | cascade + 6 | N | ci_stages | projects | project_id | cascade + 7 | N | ci_pipeline_schedules | projects | project_id | cascade + 8 | N | ci_runner_projects | projects | project_id | cascade + 9 | Y | dast_site_profiles_pipelines | ci_pipelines | ci_pipeline_id | cascade + 10 | Y | vulnerability_feedback | ci_pipelines | pipeline_id | nullify + 11 | N | ci_variables | projects | project_id | cascade + 12 | N | ci_refs | projects | project_id | cascade + 13 | N | ci_builds_metadata | projects | project_id | cascade + 14 | N | ci_subscriptions_projects | projects | downstream_project_id | cascade + 15 | N | ci_subscriptions_projects | projects | upstream_project_id | cascade + 16 | N | ci_sources_projects | projects | source_project_id | cascade + 17 | N | ci_job_token_project_scope_links | projects | source_project_id | cascade + 18 | N | ci_job_token_project_scope_links | projects | target_project_id | cascade + 19 | N | ci_project_monthly_usages | projects | project_id | cascade + +To match FK write one or many filters to match against FROM/TO/COLUMN: +- scripts/decomposition/generate-loose-foreign-key <filter(s)...> +- scripts/decomposition/generate-loose-foreign-key ci_job_artifacts project_id +- scripts/decomposition/generate-loose-foreign-key dast_site_profiles_pipelines +``` + +The command accepts a list of filters to match from, to, or column for the purpose of the foreign key generation. +For example, run this to swap all foreign keys for `ci_job_token_project_scope_links` for the +decomposed database: + +```shell +scripts/decomposition/generate-loose-foreign-key -c ci_job_token_project_scope_links +``` + +To swap only the `source_project_id` of `ci_job_token_project_scope_links` for the decomposed database, run: + +```shell +scripts/decomposition/generate-loose-foreign-key -c ci_job_token_project_scope_links source_project_id +``` + +To swap all the foreign keys (all having `_id` appended), but not create a new branch (only commit +the changes) and not create rspecs, run: + +```shell +scripts/decomposition/generate-loose-foreign-key -c --no-branch --no-rspec _id +``` + +To swap all foreign keys referencing `projects`, but not create a new branch (only commit the +changes), run: + +```shell +scripts/decomposition/generate-loose-foreign-key -c --no-branch projects +``` + ## Example migration and configuration ### Configure the loose foreign key @@ -148,6 +249,67 @@ end At this point, the setup phase is concluded. The deleted `projects` records should be automatically picked up by the scheduled cleanup worker job. +### Remove the loose foreign key + +When the loose foreign key definition is no longer needed (parent table is removed, or FK is restored), +we need to remove the definition from the YAML file and ensure that we don't leave pending deleted +records in the database. + +1. Remove the loose foreign key definition from the config (`config/gitlab_loose_foreign_keys.yml`). +1. Remove the deletion tracking trigger from the parent table (if the parent table is still there). +1. Remove leftover deleted records from the `loose_foreign_keys_deleted_records` table. + +Migration for removing the trigger: + +```ruby +class UnTrackProjectRecordChanges < Gitlab::Database::Migration[1.0] + include Gitlab::Database::MigrationHelpers::LooseForeignKeyHelpers + + enable_lock_retries! + + def up + untrack_record_deletions(:projects) + end + + def down + track_record_deletions(:projects) + end +end +``` + +With the trigger removal, we prevent further records to be inserted in the `loose_foreign_keys_deleted_records` +table however, there is still a chance for having leftover pending records in the table. These records +must be removed with an inline data migration. + +```ruby +class RemoveLeftoverProjectDeletions < Gitlab::Database::Migration[1.0] + disable_ddl_transaction! + + def up + loop do + result = execute <<~SQL + DELETE FROM "loose_foreign_keys_deleted_records" + WHERE + ("loose_foreign_keys_deleted_records"."partition", "loose_foreign_keys_deleted_records"."id") IN ( + SELECT "loose_foreign_keys_deleted_records"."partition", "loose_foreign_keys_deleted_records"."id" + FROM "loose_foreign_keys_deleted_records" + WHERE + "loose_foreign_keys_deleted_records"."fully_qualified_table_name" = 'public.projects' AND + "loose_foreign_keys_deleted_records"."status" = 1 + LIMIT 100 + ) + SQL + + break if result.cmd_tuples == 0 + end + end + + def down + # no-op + end +end +``` + ## Testing The "`it has loose foreign keys`" shared example can be used to test the presence of the `ON DELETE` trigger and the @@ -234,6 +396,12 @@ We considered using these Rails features as an alternative to foreign keys but t 1. These can lead to severe performance degradation as we load all records from PostgreSQL, loop over them in Ruby, and call individual `DELETE` queries. 1. These can miss data as they only cover the case when the `destroy` method is called directly on the model. There are other cases including `delete_all` and cascading deletes from another parent table that could mean these are missed. +For non-trivial objects that need to clean up data outside the +database (for example, object storage) where you might wish to use `dependent: :destroy`, +see alternatives in +[Avoid `dependent: :nullify` and `dependent: :destroy` across +databases](./multiple_databases.md#avoid-dependent-nullify-and-dependent-destroy-across-databases). + ## Risks of loose foreign keys and possible mitigations In general, the loose foreign keys architecture is eventually consistent and diff --git a/doc/development/database/multiple_databases.md b/doc/development/database/multiple_databases.md index 1338e83070f..c9bbf73be55 100644 --- a/doc/development/database/multiple_databases.md +++ b/doc/development/database/multiple_databases.md @@ -32,7 +32,7 @@ If you are using GDK, you can follow the following steps: 1. On the GDK root directory, run: ```shell - gdk config set gitlab.rails.multiple_databases true + gdk config set gitlab.rails.databases.ci.enabled true ``` 1. Open your `gdk.yml`, and confirm that it has the following lines: @@ -40,7 +40,9 @@ If you are using GDK, you can follow the following steps: ```yaml gitlab: rails: - multiple_databases: true + databases: + ci: + enabled: true ``` 1. Reconfigure GDK: @@ -623,10 +625,14 @@ outcomes when we switch to decomposed, because now you have some queries happening outside the transaction and they may be partially applied while the outer transaction fails, which could lead to surprising bugs. -If you need to do some cleanup after a `destroy` you will need to choose -from some of the options above. If all you need to do is cleanup the child -records themselves from PostgreSQL then you could consider using ["loose foreign -keys"](loose_foreign_keys.md). +For non-trivial objects that need to clean up data outside the +database (for example, object storage), we recommend the setting +[`dependent: :restrict_with_error`](https://guides.rubyonrails.org/association_basics.html#options-for-has-one-dependent). +Such objects should be removed explicitly ahead of time. Using `dependent: :restrict_with_error` +ensures that we forbid destroying the parent object if something is not cleaned up. + +If all you need to do is clean up the child records themselves from PostgreSQL, +consider using [loose foreign keys](loose_foreign_keys.md). ## `config/database.yml` diff --git a/doc/development/database/strings_and_the_text_data_type.md b/doc/development/database/strings_and_the_text_data_type.md index a0dda42fdc7..9674deb4603 100644 --- a/doc/development/database/strings_and_the_text_data_type.md +++ b/doc/development/database/strings_and_the_text_data_type.md @@ -253,6 +253,26 @@ class ValidateTextLimitMigration < Gitlab::Database::Migration[1.0] end ``` +## Increasing a text limit constraint on an existing column + +Increasing text limits on existing database columns can be safely achieved by first adding the new limit (with a different name), +and then dropping the previous limit: + +```ruby +class ChangeMaintainerNoteLimitInCiRunner < Gitlab::Database::Migration[1.0] + disable_ddl_transaction! + + def up + add_text_limit :ci_runners, :maintainer_note, 1024, constraint_name: check_constraint_name(:ci_runners, :maintainer_note, 'max_length_1MB') + remove_text_limit :ci_runners, :maintainer_note, constraint_name: check_constraint_name(:ci_runners, :maintainer_note, 'max_length') + end + + def down + # no-op: Danger of failing if there are records with length(maintainer_note) > 255 + end +end +``` + ## Text limit constraints on large tables If you have to clean up a text column for a really [large table](https://gitlab.com/gitlab-org/gitlab/-/blob/master/rubocop/rubocop-migrations.yml#L3) diff --git a/doc/development/database_review.md b/doc/development/database_review.md index 8e217725a17..4b5845992b9 100644 --- a/doc/development/database_review.md +++ b/doc/development/database_review.md @@ -203,6 +203,12 @@ Include in the MR description: - Order columns based on the [Ordering Table Columns](ordering_table_columns.md) guidelines. - Add foreign keys to any columns pointing to data in other tables, including [an index](migration_style_guide.md#adding-foreign-key-constraints). - Add indexes for fields that are used in statements such as `WHERE`, `ORDER BY`, `GROUP BY`, and `JOIN`s. +- New tables and columns are not necessarily risky, but over time some access patterns are inherently + difficult to scale. To identify these risky patterns in advance, we need to document expectations for + access and size. Include in the MR description answers to these questions: + - What is the anticipated growth for the new table over the next 3 months, 6 months, 1 year? What assumptions are these based on? + - How many reads and writes per hour would you expect this table to have in 3 months, 6 months, 1 year? Under what circumstances are rows updated? What assumptions are these based on? + - Based on the anticipated data volume and access patterns, does the new table pose an availability risk to GitLab.com or self-managed instances? Will the proposed design scale to support the needs of GitLab.com and self-managed customers? #### Preparation when removing columns, tables, indexes, or other structures @@ -245,6 +251,10 @@ Include in the MR description: that post migrations are executed post-deployment in production. - Check [timing guidelines for migrations](migration_style_guide.md#how-long-a-migration-should-take) - Check migrations are reversible and implement a `#down` method +- Check new table migrations: + - Are the stated access patterns and volume reasonable? Do the assumptions they're based on seem sound? Do these patterns pose risks to stability? + - Are the columns [ordered to conserve space](ordering_table_columns.md)? + - Are there foreign keys for references to other tables? - Check data migrations: - Establish a time estimate for execution on GitLab.com. - Depending on timing, data migrations can be placed on regular, post-deploy, or background migrations. diff --git a/doc/development/documentation/graphql_styleguide.md b/doc/development/documentation/graphql_styleguide.md index 5acc8bda6a6..ad19a40a3f5 100644 --- a/doc/development/documentation/graphql_styleguide.md +++ b/doc/development/documentation/graphql_styleguide.md @@ -6,7 +6,7 @@ info: "See the Technical Writers assigned to Development Guidelines: https://abo description: "Writing styles, markup, formatting, and other standards for GraphQL API's GitLab Documentation." --- -# GraphQL API +# Creating a GraphQL example page GraphQL APIs are different from [RESTful APIs](restful_api_styleguide.md). Reference information is generated in our [GraphQL reference](../../api/graphql/reference/index.md). diff --git a/doc/development/documentation/index.md b/doc/development/documentation/index.md index 5cf7bb74549..66d6beb821f 100644 --- a/doc/development/documentation/index.md +++ b/doc/development/documentation/index.md @@ -7,7 +7,7 @@ description: Learn how to contribute to GitLab Documentation. # GitLab Documentation guidelines -The GitLab documentation is [intended as the single source of truth (SSOT)](https://about.gitlab.com/handbook/documentation/) for information about how to configure, use, and troubleshoot GitLab. The documentation contains use cases and usage instructions for every GitLab feature, organized by product area and subject. This includes topics and workflows that span multiple GitLab features, and the use of GitLab with other applications. +The GitLab documentation is [intended as the single source of truth (SSOT)](https://about.gitlab.com/handbook/documentation/) for information about how to configure, use, and troubleshoot GitLab. The documentation contains use cases and usage instructions for every GitLab feature, organized by product area and subject. This includes topics and workflows that span multiple GitLab features and the use of GitLab with other applications. In addition to this page, the following resources can help you craft and contribute to documentation: @@ -55,9 +55,9 @@ docs-only merge requests using the following guide: [Contributions to GitLab docs](workflow.md) are welcome from the entire GitLab community. -To ensure that GitLab docs are current, there are special processes and responsibilities for all [feature changes](workflow.md), that is development work that impacts the appearance, usage, or administration of a feature. +To ensure that the GitLab docs are current, there are special processes and responsibilities for all [feature changes](workflow.md), that is development work that impacts the appearance, usage, or administration of a feature. -However, anyone can contribute [documentation improvements](workflow.md) that are not associated with a feature change. For example, adding a new doc on how to accomplish a use case that's already possible with GitLab or with third-party tools and GitLab. +However, anyone can contribute [documentation improvements](workflow.md) that are not associated with a feature change. For example, adding a new document on how to accomplish a use case that's already possible with GitLab or with third-party tools and GitLab. ## Markdown and styles @@ -87,8 +87,8 @@ belongs to, as well as an information block as described below: - `group`: The [Group](https://about.gitlab.com/company/team/structure/#product-groups) to which the majority of the page's content belongs. - `info`: The following line, which provides direction to contributors regarding - how to contact the Technical Writer associated with the page's Stage and - Group: + how to contact the Technical Writer associated with the page's stage and + group: ```plaintext To determine the technical writer assigned to the Stage/Group @@ -116,7 +116,7 @@ The following metadata should be added when a page is moved to another location: location to which visitors should be redirected for a moved page. [Learn more](redirects.md). - `disqus_identifier`: Identifier for Disqus commenting system. Used to keep - comments with a page that's been moved to a new URL. + comments with a page that has been moved to a new URL. [Learn more](redirects.md#redirections-for-pages-with-disqus-comments). ### Comments metadata @@ -192,8 +192,8 @@ For example: 1. The change shows up in the 14.5 self-managed release, due to missing the release cutoff for 14.4. -The exact cutoff date for each release is flexible, and can be earlier or later -than expected due to holidays, weekends, or other events. In general, MRs merged +The exact cutoff date for each release is flexible, and can be sooner or later +than expected due to holidays, weekends or other events. In general, MRs merged by the 17th should be present in the release on the 22nd, though it is not guaranteed. If it is important that a documentation update is present in that month's release, merge it as early as possible. @@ -209,7 +209,7 @@ with the following conventions: - It's relative to the `doc/` directory in the GitLab repository. - It omits the `.md` extension. -- It doesn't end with a slash (`/`). +- It doesn't end with a forward slash (`/`). The help text follows the [Pajamas guidelines](https://design.gitlab.com/usability/helping-users/#formatting-help-content). @@ -316,7 +316,7 @@ process. This is configured in the `Dangerfile` in the GitLab repository under ## Automatic screenshot generator -You can now set up an automatic screenshot generator to take and compress screenshots, with the +You can now set up an automatic screenshot generator to take and compress screenshots with the help of a configuration file known as **screenshot generator**. ### Use the tool diff --git a/doc/development/documentation/redirects.md b/doc/development/documentation/redirects.md index 8f13048f663..4c748924c67 100644 --- a/doc/development/documentation/redirects.md +++ b/doc/development/documentation/redirects.md @@ -27,7 +27,7 @@ Add a redirect to ensure: - Users see the new page and can update or delete their bookmark. - External sites can update their links, especially sites that have automation that - check for redirecting links. + checks for redirected links. - The documentation site global navigation does not link to a missing page. The links in the global navigation are already tested in the `gitlab-docs` project. @@ -38,26 +38,28 @@ Technical Writers can help with any questions and can review your change. There are two types of redirects: -- Redirect added into the documentation files themselves, for users who +- [Redirect added into the documentation files themselves](#add-a-redirect), for users who view the docs in `/help` on self-managed instances. For example, - [`/help` on GitLab.com](https://gitlab.com/help). -- [GitLab Pages redirects](../../user/project/pages/redirects.md), - for users who view the docs on [`docs.gitlab.com`](https://docs.gitlab.com). + [`/help` on GitLab.com](https://gitlab.com/help). These must be added in the same + MR that renames or moves a doc. Redirects to internal pages expire after three months + and redirects to external pages (starting with `https:`) expire after a year. +- [GitLab Pages redirects](../../user/project/pages/redirects.md), which are added + automatically after redirect files expire. They must not be manually added by + contributors and expire after nine months. Redirects pointing to external sites + are not added to the GitLab Pages redirects. - The Technical Writing team manages the [process](https://gitlab.com/gitlab-org/technical-writing/-/blob/main/.gitlab/issue_templates/tw-monthly-tasks.md) - to regularly update and [clean up the redirects](https://gitlab.com/gitlab-org/gitlab-docs/-/blob/main/doc/raketasks.md#clean-up-redirects). - If you're a contributor, you may add a new redirect, but you don't need to delete - the old ones. This process is automatic and handled by the Technical - Writing team. +Expired redirect files are removed from the documentation projects by the +[`clean_redirects` Rake task](https://gitlab.com/gitlab-org/gitlab-docs/-/blob/main/doc/raketasks.md#clean-up-redirects), +as part of the Technical Writing team's [monthly tasks](https://gitlab.com/gitlab-org/technical-writing/-/blob/main/.gitlab/issue_templates/tw-monthly-tasks.md). + +## Add a redirect NOTE: -If the old page you're renaming doesn't exist in a stable branch, skip the -following steps and ask a Technical Writer to add the redirect in -[`redirects.yaml`](https://gitlab.com/gitlab-org/gitlab-docs/-/blob/main/content/_data/redirects.yaml). -For example, if you add a new page on the 3rd of the month and then rename it before it gets -added in the stable branch on the 18th, the old page will never be part of the internal `/help`. -In that case, you can jump straight to the -[Pages redirect](https://gitlab.com/gitlab-org/gitlab-docs/-/blob/main/doc/maintenance.md#pages-redirects). +If the renamed page is new, you can sometimes skip the following steps and ask a +Technical Writer to manually add the redirect to [`redirects.yaml`](https://gitlab.com/gitlab-org/gitlab-docs/-/blob/main/content/_data/redirects.yaml). +For example, if you add a new page and then rename it before it's added to a release +on the 18th. The old page is not in any version's `/help` section, so a technical writer +can jump straight to the [Pages redirect](https://gitlab.com/gitlab-org/gitlab-docs/-/blob/main/doc/maintenance.md#pages-redirects). To add a redirect: @@ -87,20 +89,13 @@ To add a redirect: bundle exec rake "gitlab:docs:redirect[doc/user/search/old_file.md, https://example.com]" ``` - Alternatively, you can omit the arguments and be asked to enter their values: - - ```shell - bundle exec rake gitlab:docs:redirect - ``` - - If you don't want to use the Rake task, you can use the following template. + - Alternatively, you can omit the arguments and be prompted to enter the values: - Replace the value of `redirect_to` with the new file path and `YYYY-MM-DD` - with the date the file should be removed. + ```shell + bundle exec rake gitlab:docs:redirect + ``` - Redirect files that link to docs in internal documentation projects - are removed after three months. Redirect files that link to external sites are - removed after one year: + If you don't want to use the Rake task, you can use the following template: ```markdown --- @@ -111,9 +106,14 @@ To add a redirect: This document was moved to [another location](../path/to/file/index.md). <!-- This redirect file can be deleted after <YYYY-MM-DD>. --> - <!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> + <!-- Redirects that point to other docs in the same project expire in three months. --> + <!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> + <!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> ``` + - Replace both instances of `../newpath/to/file/index.md` with the new file path. + - Replace `YYYY-MM-DD` with the expiry date, as explained in the template. + 1. If the documentation page being moved has any Disqus comments, follow the steps described in [Redirections for pages with Disqus comments](#redirections-for-pages-with-disqus-comments). 1. Open a merge request with your changes. If a documentation page diff --git a/doc/development/documentation/styleguide/index.md b/doc/development/documentation/styleguide/index.md index 3e9c0177d48..91e9d0c703d 100644 --- a/doc/development/documentation/styleguide/index.md +++ b/doc/development/documentation/styleguide/index.md @@ -655,52 +655,39 @@ This is overridden by the [documentation-specific punctuation rules](#punctuatio ## Headings -- Add only one H1 in each document, by adding `#` at the beginning of - it (when using Markdown). The `h1` becomes the document `<title>`. -- Start with an `h2` (`##`), and respect the order `h2` > `h3` > `h4` > `h5` > `h6`. - Never skip the hierarchy level, such as `h2` > `h4` -- Avoid putting numbers in headings. Numbers shift, hence documentation anchor - links shift too, which eventually leads to dead links. If you think it is - compelling to add numbers in headings, make sure to at least discuss it with - someone in the Merge Request. -- [Avoid using symbols and special characters](https://gitlab.com/gitlab-org/gitlab-docs/-/issues/84) - in headers. Whenever possible, they should be plain and short text. -- When possible, avoid including words that might change in the future. Changing +In the Markdown document: + +- Add one H1 (`#`) at the start of the page. The `h1` becomes the document `<title>`. +- After the H1, follow the order `h2` > `h3` > `h4` > `h5` > `h6`. +- Do not skip a level. For example: `h2` > `h4`. +- Leave one blank line before and after the heading. + +For the heading text, **do**: + +- Be clear and direct. Make every word count. +- Use active verbs for tasks. For example, `Configure GDK` instead of `Configuring GDK`. +- Talk about what the product does, realistically but from a positive perspective. Instead of + `Limitations`, move the content near other similar information. If you must, you can + use the title `Known issues`. +- Use articles and prepositions. +- Add the [product badge](#product-tier-badges) that corresponds to the license tier. +- Follow [capitalization](#capitalization) guidelines. + +For the heading text, **do not**: + +- Use generic words like `Overview` or `Use cases`. Instead, incorporate + the information under a concept heading. +- Use `How it works`. Incorporate this information under a concept, or use a + noun followed by `workflow`. For example, `Merge request workflow`. +- Use `Important Notes`. Incorporate this information closer to where it belongs. +- Use numbers to indicate steps. If the numbers change, the anchor links changes, + which eventually leads to dead links. If you think you must add numbers in headings, + at least discuss it with a writer in the merge request. +- Use words that might change in the future. Changing a heading changes its anchor URL, which affects other linked pages. -- When introducing a new document, be careful for the headings to be - grammatically and syntactically correct. Mention an [assigned technical writer (TW)](https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments) - for review, based upon the [product category](https://about.gitlab.com/handbook/product/categories/). - This is to ensure that no document with wrong heading is going live without an - audit, thus preventing dead links and redirection issues when corrected. -- Use the context provided by parent section headings. That is, don't repeat the parent heading's text in each - subsection's heading. -- Use articles and prepositions in headings where it would make sense in regular text. -- Leave exactly one blank line before and after a heading. -- Do not use links in headings. -- Add the corresponding [product badge](#product-tier-badges) according to the tier the - feature belongs. -- Our documentation site search engine prioritizes words used in headings and - subheadings. Make your subheading titles clear, descriptive, and complete to help - users find the right example, as shown in the section on [heading titles](#heading-titles). -- See [Capitalization](#capitalization) for guidelines on capitalizing headings. - -### Heading titles - -Keep heading titles clear and direct. Make every word count. To accommodate -search engine optimization (SEO), use the imperative, where possible. - -| Do | Don't | -|:--------------------------------------|:------------------------------------------------------------| -| Configure GDK | Configuring GDK | -| GitLab Release and Maintenance Policy | This section covers the GitLab Release and Maintenance Policy | -| Backport to older releases | Backporting to older releases | -| GitLab Pages examples | Examples | - -For guidelines on capitalizing headings, see the section on [capitalization](#capitalization). - -NOTE: -If you change an existing title, be careful. In-page [anchor links](#anchor-links), -links in the GitLab application, and links from external sites can break. +- Repeat text from earlier headings. For example, instead of `Troubleshooting merge requests`, + use `Troubleshooting`. +- Use links. ### Anchor links @@ -1130,6 +1117,17 @@ copy of `https://gitlab.com/gitlab-org/gitlab`, run in a terminal: bin/pngquant compress doc/user/img ``` +### Animated images + +Sometimes an image with animation (such as an animated GIF) +can help the reader understand a complicated interaction with the user interface. + +However, you should use them sparingly and avoid them when you can. +Do not use them to replace written descriptions of processes or the product. + +If you include an animated image, follow the same size and naming conventions we use for images. If the animated image loops, add at least a three +second pause to the end of the loop. + ## Videos Adding GitLab YouTube video tutorials to the documentation is highly diff --git a/doc/development/documentation/styleguide/word_list.md b/doc/development/documentation/styleguide/word_list.md index 2c435cdc69d..c38c6586c3a 100644 --- a/doc/development/documentation/styleguide/word_list.md +++ b/doc/development/documentation/styleguide/word_list.md @@ -89,6 +89,14 @@ For example: Do not use title case **GitLab Agent** or **GitLab Agent for Kubernetes**. +## agent access token + +The token generated when you create an agent for Kubernetes. Use **agent access token**, not: + +- registration token +- secret token +- authentication token + ## allow, enable Try to avoid **allow** and **enable**, unless you are talking about security-related features. @@ -174,8 +182,7 @@ See also [contractions](index.md#contractions). Use one word for **checkbox**. Do not use **check box**. -You **select** (not **check** or **enable**) and **clear** (not **deselect** or **disable**) checkboxes. -For example: +You **select** (not **check** or **enable**) and **clear** (not **deselect** or **disable**) checkboxes. For example: - Select the **Protect environment** checkbox. - Clear the **Protect environment** checkbox. @@ -185,6 +192,8 @@ If you must refer to the checkbox, you can say it is selected or cleared. For ex - Ensure the **Protect environment** checkbox is cleared. - Ensure the **Protect environment** checkbox is selected. +(For `deselect`, [Vale](../testing.md#vale) rule: [`SubstitutionWarning.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/SubstitutionWarning.yml)) + ## checkout, check out Use **check out** as a verb. For the Git command, use `checkout`. @@ -201,10 +210,6 @@ CI/CD is always uppercase. No need to spell it out on first use. Use **CI/CD minutes** instead of **CI minutes**, **pipeline minutes**, **pipeline minutes quota**, or **CI pipeline minutes**. This decision was made in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/342813). -## CI/CD tunnel - -Use lowercase for **tunnel** in **CI/CD tunnel**. - ## click Do not use **click**. Instead, use **select** with buttons, links, menu items, and lists. @@ -256,6 +261,11 @@ Do not use **Developer permissions**. A user who is assigned the Developer role See [the Microsoft style guide](https://docs.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/d/disable-disabled) for guidance on **disable**. Use **inactive** or **off** instead. ([Vale](../testing.md#vale) rule: [`InclusionAbleism.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/InclusionAbleism.yml)) + +## disallow + +Use **prevent** instead of **disallow**. ([Vale](../testing.md#vale) rule: [`Substitutions.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/Substitutions.yml)) + ## dropdown list Use **dropdown list** to refer to the UI element. Do not use **dropdown** without **list** after it. @@ -667,6 +677,10 @@ Do not use [**roles**](#roles) and **permissions** interchangeably. Each user is Permissions are not the same as [**access levels**](#access-level). +## personal access token + +Use lowercase for **personal access token**. + ## please Do not use **please**. For details, see the [Microsoft style guide](https://docs.microsoft.com/en-us/style-guide/a-z-word-list-term-collections/p/please). @@ -951,6 +965,17 @@ One exception: You can use **we recommend** instead of **it is recommended** or Do not use **whitelist**. Another option is **allowlist**. ([Vale](../testing.md#vale) rule: [`InclusionCultural.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/InclusionCultural.yml)) +## yet + +Do not use **yet** when talking about the product or its features. The documentation describes the product as it is today. + +Sometimes you might need to use **yet** when writing a task. If you use +**yet**, ensure the surrounding phrases are written +in present tense, active voice. + +[View guidance about how to write about future features](index.md#promising-features-in-future-versions). +([Vale](../testing.md#vale) rule: [`CurrentStatus.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/.vale/gitlab/CurrentStatus.yml)) + ## you, your, yours Use **you**, **your**, and **yours** instead of [**the user** and **the user's**](#user-users). diff --git a/doc/development/documentation/testing.md b/doc/development/documentation/testing.md index 905b44823c3..49fe0aff3c6 100644 --- a/doc/development/documentation/testing.md +++ b/doc/development/documentation/testing.md @@ -128,7 +128,7 @@ and you should make sure your version matches the version used by GitLab. ## Update linter configuration -[Vale configuration](#vale) and [markdownlint configuration](#markdownlint) is under source control in each +[Vale configuration](#vale) and [markdownlint configuration](#markdownlint) is under source control in each project, so updates must be committed to each project individually. We consider the configuration in the `gitlab` project as the source of truth and that's where all updates should @@ -213,19 +213,19 @@ You can use Vale: #### Vale result types -Vale returns three types of results: `suggestion`, `warning`, and `error`: +Vale returns three types of results: -- **Suggestion**-level results are writing tips and aren't displayed in CI - job output. Suggestions don't break CI. See a list of - [suggestion-level rules](https://gitlab.com/search?utf8=✓&snippets=false&scope=&repository_ref=master&search=path%3Adoc%2F.vale%2Fgitlab+Suggestion%3A&group_id=9970&project_id=278964). -- **Warning**-level results are [Style Guide](styleguide/index.md) violations, aren't displayed in CI - job output, and should contain clear explanations of how to resolve the warning. - Warnings may be technical debt, or can be future error-level test items - (after the Technical Writing team completes its cleanup). Warnings don't break CI. See a list of - [warning-level rules](https://gitlab.com/search?utf8=✓&snippets=false&scope=&repository_ref=master&search=path%3Adoc%2F.vale%2Fgitlab+Warning%3A&group_id=9970&project_id=278964). -- **Error**-level results are Style Guide violations, and should contain clear explanations - of how to resolve the error. Errors break CI and are displayed in CI job output. See a list of - [error-level rules](https://gitlab.com/search?utf8=✓&snippets=false&scope=&repository_ref=master&search=path%3Adoc%2F.vale%2Fgitlab+Error%3A&group_id=9970&project_id=278964). +- **Error** - For branding and trademark issues, and words or phrases with ambiguous meanings. +- **Warning** - For Technical Writing team style preferences. +- **Suggestion** - For basic technical writing tenets and best practices. + +The result types have these attributes: + +| Result type | Displayed in CI/CD job output | Causes CI/CD jobs to fail | Vale rule link | +|--------------|-------------------------------|---------------------------|----------------| +| `error` | **{check-circle}** Yes | **{check-circle}** Yes | [Error-level Vale rules](https://gitlab.com/search?utf8=✓&snippets=false&scope=&repository_ref=master&search=path%3Adoc%2F.vale%2Fgitlab+Error%3A&group_id=9970&project_id=278964) | +| `warning` | **{dotted-circle}** No | **{dotted-circle}** No | [Warning-level Vale rules](https://gitlab.com/search?utf8=✓&snippets=false&scope=&repository_ref=master&search=path%3Adoc%2F.vale%2Fgitlab+Warning%3A&group_id=9970&project_id=278964) | +| `suggestion` | **{dotted-circle}** No | **{dotted-circle}** No | [Suggestion-level Vale rules](https://gitlab.com/search?utf8=✓&snippets=false&scope=&repository_ref=master&search=path%3Adoc%2F.vale%2Fgitlab+Suggestion%3A&group_id=9970&project_id=278964) | #### Vale spelling test @@ -270,12 +270,10 @@ build pipelines: [used (see `variables:` section)](https://gitlab.com/gitlab-org/gitlab-docs/-/blob/main/.gitlab-ci.yml) when building the `image:docs-lint-markdown`. -1. Install [`vale`](https://github.com/errata-ai/vale/releases). For example, to install using - `brew` for macOS, run: +1. Install [`vale`](https://github.com/errata-ai/vale/releases). To install for: - ```shell - brew install vale - ``` + - macOS using `brew`, run: `brew install vale`. + - Linux, use your distribution's package manager or a [released binary](https://github.com/errata-ai/vale/releases). These tools can be [integrated with your code editor](#configure-editors). @@ -313,6 +311,9 @@ To configure Vale in your editor, install one of the following as appropriate: - Visual Studio Code [`errata-ai.vale-server` extension](https://marketplace.visualstudio.com/items?itemName=errata-ai.vale-server). You can configure the plugin to [display only a subset of alerts](#show-subset-of-vale-alerts). - Vim [ALE plugin](https://github.com/dense-analysis/ale). +- Jetbrains IDEs - No plugin exists, but + [this issue comment](https://github.com/errata-ai/vale-server/issues/39#issuecomment-751714451) + contains tips for configuring an external tool. - Emacs [Flycheck extension](https://github.com/flycheck/flycheck). This requires some configuration: diff --git a/doc/development/ee_features.md b/doc/development/ee_features.md index 17e35d34ec7..5bd830715f5 100644 --- a/doc/development/ee_features.md +++ b/doc/development/ee_features.md @@ -16,6 +16,13 @@ info: To determine the technical writer assigned to the Stage/Group associated w [EE features list](https://about.gitlab.com/features/). <!-- markdownlint-enable MD044 --> +## Act as SaaS + +When developing locally, there are times when you need your instance to act like the SaaS version of the product. +In those instances, you can simulate SaaS by exporting an environment variable as seen below: + +`export GITLAB_SIMULATE_SAAS=1` + ## Act as CE when unlicensed Since the implementation of @@ -437,6 +444,69 @@ resolve when you add the indentation to the equation. EE-specific views should be placed in `ee/app/views/`, using extra sub-directories if appropriate. +#### Using `render_if_exists` + +Instead of using regular `render`, we should use `render_if_exists`, which +doesn't render anything if it cannot find the specific partial. We use this +so that we could put `render_if_exists` in CE, keeping code the same between +CE and EE. + +The advantages of this: + +- Very clear hints about where we're extending EE views while reading CE code. + +The disadvantage of this: + +- If we have typos in the partial name, it would be silently ignored. + +##### Caveats + +The `render_if_exists` view path argument must be relative to `app/views/` and `ee/app/views`. +Resolving an EE template path that is relative to the CE view path doesn't work. + +```haml +- # app/views/projects/index.html.haml + += render_if_exists 'button' # Will not render `ee/app/views/projects/_button` and will quietly fail += render_if_exists 'projects/button' # Will render `ee/app/views/projects/_button` +``` + +#### Using `render_ce` + +For `render` and `render_if_exists`, they search for the EE partial first, +and then CE partial. They would only render a particular partial, not all +partials with the same name. We could take the advantage of this, so that +the same partial path (for example, `shared/issuable/form/default_templates`) could +be referring to the CE partial in CE (that is, +`app/views/shared/issuable/form/_default_templates.html.haml`), while EE +partial in EE (that is, +`ee/app/views/shared/issuable/form/_default_templates.html.haml`). This way, +we could show different things between CE and EE. + +However sometimes we would also want to reuse the CE partial in EE partial +because we might just want to add something to the existing CE partial. We +could workaround this by adding another partial with a different name, but it +would be tedious to do so. + +In this case, we could as well just use `render_ce` which would ignore any EE +partials. One example would be +`ee/app/views/shared/issuable/form/_default_templates.html.haml`: + +```haml +- if @project.feature_available?(:issuable_default_templates) + = render_ce 'shared/issuable/form/default_templates' +- elsif show_promotions? + = render 'shared/promotions/promote_issue_templates' +``` + +In the above example, we can't use +`render 'shared/issuable/form/default_templates'` because it would find the +same EE partial, causing infinite recursion. Instead, we could use `render_ce` +so it ignores any partials in `ee/` and then it would render the CE partial +(that is, `app/views/shared/issuable/form/_default_templates.html.haml`) +for the same path (that is, `shared/issuable/form/default_templates`). This way +we could easily wrap around the CE partial. + ### Code in `lib/gitlab/background_migration/` When you create EE-only background migrations, you have to plan for users that @@ -518,69 +588,6 @@ module EE end ``` -#### Using `render_if_exists` - -Instead of using regular `render`, we should use `render_if_exists`, which -doesn't render anything if it cannot find the specific partial. We use this -so that we could put `render_if_exists` in CE, keeping code the same between -CE and EE. - -The advantages of this: - -- Very clear hints about where we're extending EE views while reading CE code. - -The disadvantage of this: - -- If we have typos in the partial name, it would be silently ignored. - -##### Caveats - -The `render_if_exists` view path argument must be relative to `app/views/` and `ee/app/views`. -Resolving an EE template path that is relative to the CE view path doesn't work. - -```haml -- # app/views/projects/index.html.haml - -= render_if_exists 'button' # Will not render `ee/app/views/projects/_button` and will quietly fail -= render_if_exists 'projects/button' # Will render `ee/app/views/projects/_button` -``` - -#### Using `render_ce` - -For `render` and `render_if_exists`, they search for the EE partial first, -and then CE partial. They would only render a particular partial, not all -partials with the same name. We could take the advantage of this, so that -the same partial path (for example, `shared/issuable/form/default_templates`) could -be referring to the CE partial in CE (that is, -`app/views/shared/issuable/form/_default_templates.html.haml`), while EE -partial in EE (that is, -`ee/app/views/shared/issuable/form/_default_templates.html.haml`). This way, -we could show different things between CE and EE. - -However sometimes we would also want to reuse the CE partial in EE partial -because we might just want to add something to the existing CE partial. We -could workaround this by adding another partial with a different name, but it -would be tedious to do so. - -In this case, we could as well just use `render_ce` which would ignore any EE -partials. One example would be -`ee/app/views/shared/issuable/form/_default_templates.html.haml`: - -```haml -- if @project.feature_available?(:issuable_default_templates) - = render_ce 'shared/issuable/form/default_templates' -- elsif show_promotions? - = render 'shared/promotions/promote_issue_templates' -``` - -In the above example, we can't use -`render 'shared/issuable/form/default_templates'` because it would find the -same EE partial, causing infinite recursion. Instead, we could use `render_ce` -so it ignores any partials in `ee/` and then it would render the CE partial -(that is, `app/views/shared/issuable/form/_default_templates.html.haml`) -for the same path (that is, `shared/issuable/form/default_templates`). This way -we could easily wrap around the CE partial. - ### Code in `lib/` Place EE-specific logic in the top-level `EE` module namespace. Namespace the diff --git a/doc/development/emails.md b/doc/development/emails.md index 5361282334f..b8e390988bd 100644 --- a/doc/development/emails.md +++ b/doc/development/emails.md @@ -13,6 +13,9 @@ If a mailer argument needs to be added or removed, it is important to ensure both backward and forward compatibility. Adhere to the Sidekiq steps for [changing the arguments for a worker](sidekiq/compatibility_across_updates.md#changing-the-arguments-for-a-worker). +The same applies to a new mailer method, or a new mailer. If you introduce either, +follow the steps for [adding new workers](sidekiq/compatibility_across_updates.md#adding-new-workers). + In the following example from [`NotificationService`](https://gitlab.com/gitlab-org/gitlab/-/blob/33ccb22e4fc271dbaac94b003a7a1a2915a13441/app/services/notification_service.rb#L74) adding or removing an argument in this mailer's definition may cause problems during deployment before all Rails and Sidekiq nodes have the updated code. diff --git a/doc/development/event_store.md b/doc/development/event_store.md index c6d553f41cd..b00a824e2eb 100644 --- a/doc/development/event_store.md +++ b/doc/development/event_store.md @@ -262,7 +262,7 @@ module Gitlab end ``` -A worker that is only defined in the EE codebase can subscribe to an event in the same way by +A worker that is only defined in the EE codebase can subscribe to an event in the same way by declaring the subscription in `ee/lib/ee/gitlab/event_store.rb`. Subscriptions are stored in memory when the Rails app is loaded and they are immediately frozen. diff --git a/doc/development/experiment_guide/experimentation.md b/doc/development/experiment_guide/experimentation.md index bb782af80cc..28100564555 100644 --- a/doc/development/experiment_guide/experimentation.md +++ b/doc/development/experiment_guide/experimentation.md @@ -6,4 +6,6 @@ remove_date: '2022-04-13' This document was moved to [another location](gitlab_experiment.md). <!-- This redirect file can be deleted after <2022-04-13>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/development/experiment_guide/gitlab_experiment.md b/doc/development/experiment_guide/gitlab_experiment.md index 369690ba86c..78e1f84d701 100644 --- a/doc/development/experiment_guide/gitlab_experiment.md +++ b/doc/development/experiment_guide/gitlab_experiment.md @@ -317,7 +317,7 @@ of tracking an event in Ruby would be: experiment(:pill_color, actor: current_user).track(:clicked) ``` -When you run an experiment with any of the examples so far, an `:assigned` event +When you run an experiment with any of the examples so far, an `:assignment` event is tracked automatically by default. All events that are tracked from an experiment have a special [experiment context](https://gitlab.com/gitlab-org/iglu/-/blob/master/public/schemas/com.gitlab/gitlab_experiment/jsonschema/1-0-3) @@ -448,7 +448,7 @@ The first way is simply by running the experiment. Assuming the experiment has b The second way doesn't run the experiment and is intended to be used if the experiment only needs to surface in the client layer. To accomplish this we can simply `.publish` the experiment. This won't run any logic, but does surface the experiment details in the client layer so they can be utilized there. -An example might be to publish an experiment in a `before_action` in a controller. Assuming we've defined the `PillColorExperiment` class, like we have above, we can surface it to the client by publishing it instead of running it: +An example might be to publish an experiment in a `before_action` in a controller. Assuming we've defined the `PillColorExperiment` class, like we have above, we can surface it to the client by publishing it instead of running it: ```ruby before_action -> { experiment(:pill_color).publish }, only: [:show] diff --git a/doc/development/experiment_guide/index.md b/doc/development/experiment_guide/index.md index 254c136ef79..c34e5eb36dc 100644 --- a/doc/development/experiment_guide/index.md +++ b/doc/development/experiment_guide/index.md @@ -64,3 +64,15 @@ We recommend the following workflow: 1. **If the experiment is a success**, designers add the new icon or illustration to the Pajamas UI kit as part of the cleanup process. Engineers can then add it to the [SVG library](https://gitlab-org.gitlab.io/gitlab-svgs/) and modify the implementation based on the [Frontend Development Guidelines](../fe_guide/icons.md#usage-in-hamlrails-2). + +## Turn off all experiments + +When there is a case on GitLab.com (SaaS) that necessitates turning off all experiments, we have this control. + +You can toggle experiments on SaaS on and off using the `gitlab_experiment` [feature flag](../feature_flags). + +This can be done via chatops: + +- [disable](../feature_flags/controls.md#disabling-feature-flags): `/chatops run feature set gitlab_experiment false` +- [enable](../feature_flags/controls.md#process): `/chatops run feature delete gitlab_experiment` + - This allows the `default_enabled` [value of true in the yml](https://gitlab.com/gitlab-org/gitlab/-/blob/016430f6751b0c34abb24f74608c80a1a8268f20/config/feature_flags/ops/gitlab_experiment.yml#L8) to be honored. diff --git a/doc/development/export_csv.md b/doc/development/export_csv.md index ff827023a50..998e5b1fb3b 100644 --- a/doc/development/export_csv.md +++ b/doc/development/export_csv.md @@ -1,7 +1,7 @@ --- stage: Manage group: Import -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Export to CSV diff --git a/doc/development/fe_guide/content_editor.md b/doc/development/fe_guide/content_editor.md index 139825655e9..2e64f52651e 100644 --- a/doc/development/fe_guide/content_editor.md +++ b/doc/development/fe_guide/content_editor.md @@ -47,7 +47,7 @@ The Content Editor requires two properties: - `renderMarkdown` is an asynchronous function that returns the response (String) of invoking the [Markdown API](../../api/markdown.md). -- `uploadsPath` is a URL that points to a [GitLab upload service](../uploads.md#upload-encodings) +- `uploadsPath` is a URL that points to a [GitLab upload service](../uploads/implementation.md#upload-encodings) with `multipart/form-data` support. See the [`WikiForm.vue`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/assets/javascripts/pages/shared/wikis/components/wiki_form.vue#L207) diff --git a/doc/development/fe_guide/icons.md b/doc/development/fe_guide/icons.md index 3f7490b0221..d107af156db 100644 --- a/doc/development/fe_guide/icons.md +++ b/doc/development/fe_guide/icons.md @@ -88,50 +88,36 @@ Please use the following function inside JS to render an icon: ### Usage in HAML/Rails -To insert a loading spinner in HAML or Rails use the `loading_icon` helper: +To insert a loading spinner in HAML or Rails use the `gl_loading_icon` helper: ```haml -= loading_icon += gl_loading_icon ``` -You can include one or more of the following properties with the `loading_icon` helper, as demonstrated +You can include one or more of the following properties with the `gl_loading_icon` helper, as demonstrated by the examples that follow: -- `container` (optional): wraps the loading icon in a container, which centers the loading icon using the `text-center` CSS property. -- `color` (optional): either `orange` (default), `light`, or `dark`. +- `inline` (optional): uses in an inline element if `true`, otherwise, a block element (default), with the spinner centered. +- `color` (optional): either `dark` (default) or `light`. - `size` (optional): either `sm` (default), `md`, `lg`, or `xl`. -- `css_class` (optional): defaults to an empty string, but can be used for utility classes to fine-tune alignment or spacing. +- `css_class` (optional): defaults to nothing, but can be used for utility classes to fine-tune alignment or spacing. **Example 1:** The following HAML expression generates a loading icon's markup and -centers the icon by wrapping it in a `gl-spinner-container` element. +centers the icon. ```haml -= loading_icon(container: true) -``` - -**Output from example 1:** - -```html -<div class="gl-spinner-container"> - <span class="gl-spinner gl-spinner-orange gl-spinner-sm" aria-label="Loading"></span> -</div> += gl_loading_icon ``` **Example 2:** -The following HAML expression generates a loading icon's markup +The following HAML expression generates an inline loading icon's markup with a custom size. It also appends a margin utility class. ```haml -= loading_icon(size: 'lg', css_class: 'gl-mr-2') -``` - -**Output from example 2:** - -```html -<span class="gl-spinner gl-spinner-orange gl-spinner-lg gl-mr-2" aria-label="Loading"></span> += gl_loading_icon(inline: true, size: 'lg', css_class: 'gl-mr-2') ``` ### Usage in Vue diff --git a/doc/development/fe_guide/style/javascript.md b/doc/development/fe_guide/style/javascript.md index 4a0923ebe19..d04d1879476 100644 --- a/doc/development/fe_guide/style/javascript.md +++ b/doc/development/fe_guide/style/javascript.md @@ -136,7 +136,7 @@ the class name with `js-`. ## ES Module Syntax -For most JavaScript files, use ES module syntax to import or export from modules. +For most JavaScript files, use ES module syntax to import or export from modules. Prefer named exports, as they improve name consistency. ```javascript diff --git a/doc/development/fe_guide/vue3_migration.md b/doc/development/fe_guide/vue3_migration.md index 6e994d5e95d..f174408c946 100644 --- a/doc/development/fe_guide/vue3_migration.md +++ b/doc/development/fe_guide/vue3_migration.md @@ -6,12 +6,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Migration to Vue 3 -Preparations for a Vue 3 migration are tracked in epic [&3174](https://gitlab.com/groups/gitlab-org/-/epics/3174) +The migration from Vue 2 to 3 is tracked in epic [&6252](https://gitlab.com/groups/gitlab-org/-/epics/6252). -In order to prepare for the eventual migration to Vue 3.x, we should not use the following deprecated features in the codebase: - -NOTE: -Our linting rules block the use of these deprecated features. +To ease migration to Vue 3.x, we have added [eslint rules](https://gitlab.com/gitlab-org/frontend/eslint-plugin/-/merge_requests/50) +that prevent us from using the following deprecated features in the codebase. ## Vue filters diff --git a/doc/development/feature_flags/controls.md b/doc/development/feature_flags/controls.md index 6bf5be23ace..f8f03773c12 100644 --- a/doc/development/feature_flags/controls.md +++ b/doc/development/feature_flags/controls.md @@ -72,8 +72,8 @@ group. To enable a feature for 25% of the time, run the following in Slack: ```shell -/chatops run feature set new_navigation_bar 25 --dev -/chatops run feature set new_navigation_bar 25 --staging +/chatops run feature set new_navigation_bar 25 --random --dev +/chatops run feature set new_navigation_bar 25 --random --staging ``` ### Enabling a feature for GitLab.com @@ -121,7 +121,7 @@ command you make so people can understand the change if they need to. To enable a feature for 25% of the time, run the following in Slack: ```shell -/chatops run feature set new_navigation_bar 25 +/chatops run feature set new_navigation_bar 25 --random ``` This sets a feature flag to `true` based on the following formula: diff --git a/doc/development/feature_flags/index.md b/doc/development/feature_flags/index.md index af402713f6e..4b417b26381 100644 --- a/doc/development/feature_flags/index.md +++ b/doc/development/feature_flags/index.md @@ -530,8 +530,9 @@ Feature.remove(:feature_flag_name) ## Feature flags in tests Introducing a feature flag into the codebase creates an additional code path that should be tested. -It is strongly advised to test all code affected by a feature flag, both when **enabled** and **disabled** -to ensure the feature works properly. +It is strongly advised to include automated tests for all code affected by a feature flag, both when **enabled** and **disabled** +to ensure the feature works properly. If automated tests are not included for both states, the functionality associated +with the untested code path should be manually tested before deployment to production. When using the testing environment, all feature flags are enabled by default. diff --git a/doc/development/feature_flags/process.md b/doc/development/feature_flags/process.md index 3fbb207a12b..f98366beb6b 100644 --- a/doc/development/feature_flags/process.md +++ b/doc/development/feature_flags/process.md @@ -6,4 +6,6 @@ remove_date: '2022-03-01' This document was moved to [another location](https://about.gitlab.com/handbook/product-development-flow/feature-flag-lifecycle/). <!-- This redirect file can be deleted after 2022-03-01. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/development/features_inside_dot_gitlab.md b/doc/development/features_inside_dot_gitlab.md index 283a0d5d5fb..7b11b541b5a 100644 --- a/doc/development/features_inside_dot_gitlab.md +++ b/doc/development/features_inside_dot_gitlab.md @@ -12,7 +12,7 @@ When implementing new features, please refer to these existing features to avoid - [Custom Dashboards](../operations/metrics/dashboards/index.md#add-a-new-dashboard-to-your-project): `.gitlab/dashboards/`. - [Issue Templates](../user/project/description_templates.md#create-an-issue-template): `.gitlab/issue_templates/`. - [Merge request Templates](../user/project/description_templates.md#create-a-merge-request-template): `.gitlab/merge_request_templates/`. -- [GitLab Agent](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/configuration_repository.md#layout): `.gitlab/agents/`. +- [GitLab agent](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/configuration_repository.md#layout): `.gitlab/agents/`. - [CODEOWNERS](../user/project/code_owners.md#set-up-code-owners): `.gitlab/CODEOWNERS`. - [Route Maps](../ci/review_apps/#route-maps): `.gitlab/route-map.yml`. - [Customize Auto DevOps Helm Values](../topics/autodevops/customize.md#customize-values-for-helm-chart): `.gitlab/auto-deploy-values.yaml`. diff --git a/doc/development/file_storage.md b/doc/development/file_storage.md index d161206f44d..04e2f381c97 100644 --- a/doc/development/file_storage.md +++ b/doc/development/file_storage.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w We use the [CarrierWave](https://github.com/carrierwaveuploader/carrierwave) gem to handle file upload, store and retrieval. -File uploads should be accelerated by workhorse, for details please refer to [uploads development documentation](uploads.md). +File uploads should be accelerated by workhorse, for details please refer to [uploads development documentation](uploads/index.md). There are many places where file uploading is used, according to contexts: diff --git a/doc/development/fips_compliance.md b/doc/development/fips_compliance.md index 0b6d1751668..8fe5af56f9d 100644 --- a/doc/development/fips_compliance.md +++ b/doc/development/fips_compliance.md @@ -69,55 +69,10 @@ installation instructions, including the [advanced instructions for RHEL](https: Note that `asdf` is not used for dependency management because it's essential to use the RedHat-provided Go compiler and other system dependencies. -### Working around broken frontend asset compilation - -A known bug affects asset compilation with FIPS mode enabled: [issue #322883](https://gitlab.com/gitlab-org/gitlab/-/issues/322883). -Until this is resolved, working on frontend issues is not feasible. We can still -work on backend issues by compiling the assets while FIPS is disabled, and -placing GDK into [static asset mode](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/configuration.md#webpack-settings): - -1. Modify your `gdk.yml` to contain the following: - - ```yaml - webpack: - host: 127.0.0.1 - port: 3808 - static: true - ``` - -1. In the GitLab repository, apply this patch to prevent the assets from being - automatically deleted whenever GDK is restarted: - - ```diff - diff --git a/scripts/frontend/webpack_dev_server.js b/scripts/frontend/webpack_dev_server.js - index fbb80c9617d..114720d457c 100755 - --- a/scripts/frontend/webpack_dev_server.js - +++ b/scripts/frontend/webpack_dev_server.js - @@ -15,7 +15,7 @@ const baseConfig = { - // run webpack in compile-once mode and watch for changes - if (STATIC_MODE) { - nodemon({ - - exec: `rm -rf public/assets/webpack ; yarn run webpack && exec ruby -run -e httpd public/ -p ${DEV_SERVER_PORT}`, - + exec: `ruby -run -e httpd public/ -p ${DEV_SERVER_PORT}`, - watch: [ - 'config/webpack.config.js', - 'app/assets/javascripts', - ``` - -1. Run this command in the GitLab repository to generate the asset files - to be served: - - ```shell - bin/rails gitlab:assets:compile - ``` - -Every time you change a frontend asset, you must re-run this command -(with FIPS mode disabled) before seeing the changes. - ### Enable FIPS mode -After the assets are generated, run this command (as root) and restart the -virtual machine: +After GDK and its dependencies are installed, run this command (as +root) and restart the virtual machine: ```shell fips-mode-setup --enable diff --git a/doc/development/foreign_keys.md b/doc/development/foreign_keys.md index a9edbc68a2e..db8367fe5f5 100644 --- a/doc/development/foreign_keys.md +++ b/doc/development/foreign_keys.md @@ -80,6 +80,12 @@ foreign keys to remove the data as this would result in the file system data being left behind. In such a case you should use a service class instead that takes care of removing non database data. +In cases where the relation spans multiple databases you will have even +further problems using `dependent: :destroy` or the above hooks. You can +read more about alternatives at [Avoid `dependent: :nullify` and +`dependent: :destroy` across +databases](database/multiple_databases.md#avoid-dependent-nullify-and-dependent-destroy-across-databases). + ## Alternative primary keys with has_one associations Sometimes a `has_one` association is used to create a one-to-one relationship: diff --git a/doc/development/geo.md b/doc/development/geo.md index 9f5fd674d38..f37901754aa 100644 --- a/doc/development/geo.md +++ b/doc/development/geo.md @@ -438,3 +438,61 @@ old method: If you want to add easy Geo replication of a resource you're working on, check out our [self-service framework](geo/framework.md). + +## Geo development workflow + +### GET:Geo pipeline + +As part of the [package-and-qa](testing_guide/end_to_end/index.md#using-the-package-and-qa-job) pipeline, there is an option to manually trigger a job named `GET:Geo`. This +pipeline uses [GET](https://gitlab.com/gitlab-org/gitlab-environment-toolkit) to spin up a +[1k](../administration/reference_architectures/1k_users.md) Geo installation, +and run the [`gitlab-qa`](https://gitlab.com/gitlab-org/gitlab-qa) Geo scenario against the instance. +When working on Geo features, it is a good idea to ensure the `qa-geo` job passes in a triggered `GET:Geo pipeline`. + +The pipelines that control the provisioning and teardown of the instance are included in The GitLab Environment Toolkit Configs +[Geo subproject](https://gitlab.com/gitlab-org/quality/gitlab-environment-toolkit-configs/Geo). + +When adding new functionality, consider adding new tests to verify the behavior. For steps, +see the [QA documentation](https://gitlab.com/gitlab-org/gitlab/-/tree/master/qa#writing-tests). + +#### Architecture + +The pipeline involves the interaction of multiple different projects: + +- [GitLab](https://gitlab.com/gitlab-org/gitlab) - The [package-and-qa job](testing_guide/end_to_end/index.md#using-the-package-and-qa-job) is launched from merge requests in this project. +- [`omnibus-gitlab`](https://gitlab.com/gitlab-org/omnibus-gitlab) - Builds relevant artifacts containing the changes from the triggering merge request pipeline. +- [GET-Configs/Geo](https://gitlab.com/gitlab-org/quality/gitlab-environment-toolkit-configs/Geo) - Coordinates the lifecycle of a short-lived Geo installation that can be evaluated. +- [GET](https://gitlab.com/gitlab-org/gitlab-environment-toolkit) - Contains the necessary logic for creating and destroying Geo installations. Used by `GET-Configs/Geo`. +- [`gitlab-qa`](https://gitlab.com/gitlab-org/gitlab-qa) - Tool for running automated tests against a GitLab instance. + +```mermaid +flowchart TD; + GET:Geo-->getcg + Provision-->Terraform + Configure-->Ansible + Geo-->Ansible + QA-->gagq + + subgraph "omnibus-gitlab-mirror" + GET:Geo + end + + subgraph getcg [GitLab-environment-toolkit-configs/Geo] + direction LR + Generate-terraform-config-->Provision + Provision-->Generate-ansible-config + Generate-ansible-config-->Configure + Configure-->Geo + Geo-->QA + QA-->Destroy-geo + end + + subgraph get [GitLab Environment Toolkit] + Terraform + Ansible + end + + subgraph GitLab QA + gagq[GitLab QA Geo Scenario] + end +``` diff --git a/doc/development/gitlab_diagram_overview.odg b/doc/development/gitlab_diagram_overview.odg Binary files differdeleted file mode 100644 index 9bfc7313ff4..00000000000 --- a/doc/development/gitlab_diagram_overview.odg +++ /dev/null diff --git a/doc/development/go_guide/go_upgrade.md b/doc/development/go_guide/go_upgrade.md index 889849799bc..a99253b9723 100644 --- a/doc/development/go_guide/go_upgrade.md +++ b/doc/development/go_guide/go_upgrade.md @@ -134,7 +134,7 @@ if you need help finding the correct person or labels: | GitLab Compose Kit | [Issuer Tracker](https://gitlab.com/gitlab-org/gitlab-compose-kit/-/issues) | | GitLab Container Registry | [Issue Tracker](https://gitlab.com/gitlab-org/container-registry) | | GitLab Elasticsearch Indexer | [Issue Tracker](https://gitlab.com/gitlab-org/gitlab-elasticsearch-indexer/-/issues) | -| GitLab Agent Server (KAS) | [Issue Tracker](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/issues) | +| GitLab agent server for Kubernetes (KAS) | [Issue Tracker](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/issues) | | GitLab Pages | [Issue Tracker](https://gitlab.com/gitlab-org/gitlab-pages/-/issues) | | GitLab Quality Images | [Issue Tracker](https://gitlab.com/gitlab-org/gitlab-build-images/-/issues) | | GitLab Shell | [Issue Tracker](https://gitlab.com/gitlab-org/gitlab-shell/-/issues) | diff --git a/doc/development/gotchas.md b/doc/development/gotchas.md index fbb6b0219aa..d89dbbcf904 100644 --- a/doc/development/gotchas.md +++ b/doc/development/gotchas.md @@ -166,6 +166,18 @@ end Since Active Record is not calling the `.new` method on model classes to instantiate the objects, you should use `expect_next_found_instance_of` or `allow_next_found_instance_of` mock helpers to setup mock on objects returned by Active Record query & finder methods._ +It is also possible to set mocks and expectations for multiple instances of the same Active Record model by using the `expect_next_found_(number)_instances_of` and `allow_next_found_(number)_instances_of` helpers, like so; + +```ruby +expect_next_found_2_instances_of(Project) do |project| + expect(project).to receive(:add_import_job) +end + +allow_next_found_2_instances_of(Project) do |project| + allow(project).to receive(:add_import_job) +end +``` + If we also want to initialize the instance with some particular arguments, we could also pass it like: diff --git a/doc/development/i18n/proofreader.md b/doc/development/i18n/proofreader.md index 76ab00eebfb..7c9777527ef 100644 --- a/doc/development/i18n/proofreader.md +++ b/doc/development/i18n/proofreader.md @@ -98,9 +98,11 @@ are very appreciative of the work done by translators and proofreaders! - Paulo George Gomes Bezerra - [GitLab](https://gitlab.com/paulobezerra), [Crowdin](https://crowdin.com/profile/paulogomes.rep) - André Gama - [GitLab](https://gitlab.com/andregamma), [Crowdin](https://crowdin.com/profile/ToeOficial) - Eduardo Addad de Oliveira - [GitLab](https://gitlab.com/eduardoaddad), [Crowdin](https://crowdin.com/profile/eduardoaddad) + - Horberlan Brito - [GitLab](https://gitlab.com/horberlan), [Crowdin](https://crowdin.com/profile/horberlan) - Romanian - Mircea Pop - [GitLab](https://gitlab.com/eeex), [Crowdin](https://crowdin.com/profile/eex) - Rareș Pița - [GitLab](https://gitlab.com/dlphin), [Crowdin](https://crowdin.com/profile/dlphin) + - Nicolae Liviu - [GitLab](https://gitlab.com/nicklcanada), [Crowdin](https://crowdin.com/profile/nicklcanada) - Russian - Nikita Grylov - [GitLab](https://gitlab.com/nixel2007), [Crowdin](https://crowdin.com/profile/nixel2007) - Alexy Lustin - [GitLab](https://gitlab.com/allustin), [Crowdin](https://crowdin.com/profile/lustin) @@ -110,6 +112,8 @@ are very appreciative of the work done by translators and proofreaders! - Iaroslav Postovalov - [GitLab](https://gitlab.com/CMDR_Tvis), [Crowdin](https://crowdin.com/profile/CMDR_Tvis) - Serbian (Latin and Cyrillic) - Proofreaders needed. +- Sinhalese/Sinhala සිංහල + - හෙළබස (HelaBasa) - [GitLab](https://gitlab.com/helabasa), [Crowdin](https://crowdin.com/profile/helabasa) - Slovak - Proofreaders needed. - Spanish diff --git a/doc/development/img/architecture_simplified.png b/doc/development/img/architecture_simplified.png Binary files differdeleted file mode 100644 index bab673feb4a..00000000000 --- a/doc/development/img/architecture_simplified.png +++ /dev/null diff --git a/doc/development/img/architecture_simplified_v14_9.png b/doc/development/img/architecture_simplified_v14_9.png Binary files differnew file mode 100644 index 00000000000..dcdb9d7b6ae --- /dev/null +++ b/doc/development/img/architecture_simplified_v14_9.png diff --git a/doc/development/img/merge_request_reports_v14_7.png b/doc/development/img/merge_request_reports_v14_7.png Binary files differnew file mode 100644 index 00000000000..282d6f96aa6 --- /dev/null +++ b/doc/development/img/merge_request_reports_v14_7.png diff --git a/doc/development/img/merge_widget_v14_7.png b/doc/development/img/merge_widget_v14_7.png Binary files differnew file mode 100644 index 00000000000..d5e8ed8df52 --- /dev/null +++ b/doc/development/img/merge_widget_v14_7.png diff --git a/doc/development/import_project.md b/doc/development/import_project.md index 86e6e04347c..e910983997c 100644 --- a/doc/development/import_project.md +++ b/doc/development/import_project.md @@ -127,11 +127,11 @@ A project with that name already exists. ##### `Exception: Error importing repository into (namespace) - No space left on device` -The disk has insufficient space to complete the import. +The disk has insufficient space to complete the import. During import, the tarball is cached in your configured `shared_path` directory. Verify the disk has enough free space to accommodate both the cached tarball and the unpacked -project files on disk. +project files on disk. ### Importing via the Rails console diff --git a/doc/development/index.md b/doc/development/index.md index 0501f70b818..048112215fc 100644 --- a/doc/development/index.md +++ b/doc/development/index.md @@ -227,7 +227,7 @@ the [reviewer values](https://about.gitlab.com/handbook/engineering/workflow/rev - [Working with merge request diffs](diffs.md) - [Approval Rules](approval_rules.md) - [Repository mirroring](repository_mirroring.md) -- [File uploads](uploads.md) +- [Uploads development guide](uploads/index.md) - [Auto DevOps development guide](auto_devops.md) - [Renaming features](renaming_features.md) - [Code Intelligence](code_intelligence/index.md) @@ -275,6 +275,7 @@ See [database guidelines](database/index.md). ## Integration guides +- [Integrations development guide](integrations/index.md) - [Jira Connect app](integrations/jira_connect.md) - [Security Scanners](integrations/secure.md) - [Secure Partner Integration](integrations/secure_partner_integration.md) @@ -329,6 +330,10 @@ See [database guidelines](database/index.md). - [CI/CD development documentation](cicd/index.md) - [AppSec development documentation](appsec/index.md) +## Technical Reference by Group + +- [Create: Source Code BE](backend/create_source_code_be/index.md) + ## Other Development guides - [Defining relations between files using projections](projections.md) @@ -338,6 +343,7 @@ See [database guidelines](database/index.md). - [Dashboards for stage groups](stage_group_dashboards.md) - [Preventing transient bugs](transient/prevention-patterns.md) - [GitLab Application SLIs](application_slis/index.md) +- [Spam protection and CAPTCHA development guide](spam_protection_and_captcha/index.md) ## Other GitLab Development Kit (GDK) guides diff --git a/doc/development/insert_into_tables_in_batches.md b/doc/development/insert_into_tables_in_batches.md index cfa0c862471..cd659a3d19b 100644 --- a/doc/development/insert_into_tables_in_batches.md +++ b/doc/development/insert_into_tables_in_batches.md @@ -122,7 +122,7 @@ Large parts of ActiveRecord's persistence API are built around the notion of cal of these callbacks fire in response to model life cycle events such as `save` or `create`. These callbacks cannot be used with bulk insertions, since they are meant to be called for every instance that is saved or created. Since these events do not fire when -records are inserted in bulk, we currently disallow their use. +records are inserted in bulk, we currently prevent their use. The specifics around which callbacks are explicitly allowed are defined in [`BulkInsertSafe`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/models/concerns/bulk_insert_safe.rb). diff --git a/doc/development/integrations/index.md b/doc/development/integrations/index.md new file mode 100644 index 00000000000..34ac307c98a --- /dev/null +++ b/doc/development/integrations/index.md @@ -0,0 +1,332 @@ +--- +stage: Ecosystem +group: Integrations +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +description: "GitLab's development guidelines for Integrations" +--- + +# Integrations development guide **(FREE)** + +This page provides development guidelines for implementing [GitLab integrations](../../user/project/integrations/index.md), +which are part of our [main Rails project](https://gitlab.com/gitlab-org/gitlab). + +Also see our [direction page](https://about.gitlab.com/direction/ecosystem/integrations/) for an overview of our strategy around integrations. + +This guide is a work in progress. You're welcome to ping `@gitlab-org/ecosystem-stage/integrations` +if you need clarification or spot any outdated information. + +## Add a new integration + +### Define the integration + +1. Add a new model in `app/models/integrations` extending from `Integration`. + - For example, `Integrations::FooBar` in `app/models/integrations/foo_bar.rb`. + - For certain types of integrations, you can also build on these base classes: + - `Integrations::BaseChatNotification` + - `Integrations::BaseIssueTracker` + - `Integrations::BaseMonitoring` + - `Integrations::BaseSlashCommands` + - For integrations that primarily trigger HTTP calls to external services, you can + also use the `Integrations::HasWebHook` concern. This reuses the [webhook functionality](../../user/project/integrations/webhooks.md) + in GitLab through an associated `ServiceHook` model, and automatically records request logs + which can be viewed in the integration settings. +1. Add the integration's underscored name (`'foo_bar'`) to `Integration::INTEGRATION_NAMES`. +1. Add the integration as an association on `Project`: + + ```ruby + has_one :foo_bar_integration, class_name: 'Integrations::FooBar' + ``` + +1. TEMPORARY: Accommodate the current migration to [rename "services" to "integrations"](#rename-services-to-integrations): + - Add the integration's camel-cased name (`'FooBar'`) to `Gitlab::Integrations::StiType::NAMESPACED_INTEGRATIONS`. + +### Define properties + +Integrations can define arbitrary properties to store their configuration with the class method `Integration.prop_accessor`. +The values are stored as a serialized JSON hash in the `integrations.properties` column. + +For example: + +```ruby +module Integrations + class FooBar < Integration + prop_accessor :url + prop_accessor :tags + end +end +``` + +`Integration.prop_accessor` installs accessor methods on the class. Here we would have `#url`, `#url=` and `#url_changed?`, to manage the `url` field. Fields stored in `Integration#properties` should be accessed by these accessors directly on the model, just like other ActiveRecord attributes. + +You should always access the properties through their getters, and not interact with the `properties` hash directly. +You **must not** write to the `properties` hash, you **must** use the generated setter method instead. Direct writes to this +hash are not persisted. + +You should also define validations for all your properties. + +Also refer to the section [Customize the frontend form](#customize-the-frontend-form) below to see how these properties +are exposed in the frontend form for the integration. + +There is an alternative approach using `Integration.data_field`, which you may see in other integrations. +With data fields the values are stored in a separate table per integration. At the moment we don't recommend using this for new integrations. + +### Define trigger events + +Integrations are triggered by calling their `#execute` method in response to events in GitLab, +which gets passed a payload hash with details about the event. + +The supported events have some overlap with [webhook events](../../user/project/integrations/webhook_events.md), +and receive the same payload. You can specify the events you're interested in by overriding +the class method `Integration.supported_events` in your model. + +The following events are supported for integrations: + +| Event type | Default | Value | Trigger +|:-----------------------------------------------------------------------------------------------|:--------|:---------------------|:-- +| Alert event | | `alert` | A a new, unique alert is recorded. +| Commit event | ✓ | `commit` | A commit is created or updated. +| [Deployment event](../../user/project/integrations/webhook_events.md#deployment-events) | | `deployment` | A deployment starts or finishes. +| [Issue event](../../user/project/integrations/webhook_events.md#issue-events) | ✓ | `issue` | An issue is created, updated, or closed. +| [Confidential issue event](../../user/project/integrations/webhook_events.md#issue-events) | ✓ | `confidential_issue` | A confidential issue is created, updated, or closed. +| [Job event](../../user/project/integrations/webhook_events.md#job-events) | | `job` +| [Merge request event](../../user/project/integrations/webhook_events.md#merge-request-events) | ✓ | `merge_request` | A merge request is created, updated, or merged. +| [Comment event](../../user/project/integrations/webhook_events.md#comment-events) | | `comment` | A new comment is added. +| [Confidential comment event](../../user/project/integrations/webhook_events.md#comment-events) | | `confidential_note` | A new comment on a confidential issue is added. +| [Pipeline event](../../user/project/integrations/webhook_events.md#pipeline-events) | | `pipeline` | A pipeline status changes. +| [Push event](../../user/project/integrations/webhook_events.md#push-events) | ✓ | `push` | A push is made to the repository. +| [Tag push event](../../user/project/integrations/webhook_events.md#tag-events) | ✓ | `tag_push` | New tags are pushed to the repository. +| Vulnerability event **(ULTIMATE)** | | `vulnerability` | A new, unique vulnerability is recorded. +| [Wiki page event](../../user/project/integrations/webhook_events.md#wiki-page-events) | ✓ | `wiki_page` | A wiki page is created or updated. + +#### Event examples + +This example defines an integration that responds to `commit` and `merge_request` events: + +```ruby +module Integrations + class FooBar < Integration + def self.supported_events + %w[commit merge_request] + end + end +end +``` + +An integration can also not respond to events, and implement custom functionality some other way: + +```ruby +module Integrations + class FooBar < Integration + def self.supported_events + [] + end + end +end +``` + +### Customize the frontend form + +The frontend form is generated dynamically based on metadata defined in the model. + +By default, the integration form provides: + +- A checkbox to enable or disable the integration. +- Checkboxes for each of the trigger events returned from `Integration#configurable_events`. + +You can also add help text at the top of the form by either overriding `Integration#help`, +or providing a template in `app/views/projects/services/$INTEGRATION_NAME/_help.html.haml`. + +To add your custom properties to the form, you can define the metadata for them in `Integration#fields`. + +This method should return an array of hashes for each field, where the keys can be: + +| Key | Type | Required | Default | Description +|:---------------|:--------|:---------|:-----------------------------|:-- +| `type:` | string | true | | The type of the form field. Can be `text`, `textarea`, `password`, `checkbox`, or `select`. +| `name:` | string | true | | The property name for the form field. This must match a `prop_accessor` [defined on the class](#define-properties). +| `required:` | boolean | false | `false` | Specify if the form field is required or optional. +| `title:` | string | false | Capitalized value of `name:` | The label for the form field. +| `placeholder:` | string | false | | A placeholder for the form field. +| `help:` | string | false | | A help text that displays below the form field. + +#### Additional keys for `type: 'checkbox'` + +| Key | Type | Required | Default | Description +|:------------------|:-------|:---------|:------------------|:-- +| `checkbox_label:` | string | false | Value of `title:` | A custom label that displays next to the checkbox. + +#### Additional keys for `type: 'select'` + +| Key | Type | Required | Default | Description +|:-----------|:------|:---------|:--------|:-- +| `choices:` | array | true | | A nested array of `[label, value]` tuples. + +#### Additional keys for `type: 'password'` + +| Key | Type | Required | Default | Description +|:----------------------------|:-------|:---------|:------------------|:-- +| `non_empty_password_title:` | string | false | Value of `title:` | An alternative label that displays when a value is already stored. +| `non_empty_password_help:` | string | false | Value of `help:` | An alternative help text that displays when a value is already stored. + +#### Frontend form examples + +This example defines a required `url` field, and optional `username` and `password` fields: + +```ruby +module Integrations + class FooBar < Integration + prop_accessor :url, :username, :password + + def fields + [ + { + type: 'text', + name: 'url', + title: s_('FooBarIntegration|Server URL'), + placeholder: 'https://example.com/', + required: true + }, + { + type: 'text', + name: 'username', + title: s_('FooBarIntegration|Username'), + }, + { + type: 'password', + name: 'password', + title: s_('FoobarIntegration|Password' + non_empty_password_title: s_('FooBarIntegration|Enter new password') + } + ] + end + end +end +``` + +### Expose the integration in the API + +#### REST API + +To expose the integration in the [REST API](../../api/integrations.md): + +1. Add the integration's class (`::Integrations::FooBar`) to `API::Helpers::IntegrationsHelpers.integration_classes`. +1. Add all properties that should be exposed to `API::Helpers::IntegrationsHelpers.integrations`. +1. Update the reference documentation in `doc/api/integrations.md`, add a new section for your integration, and document all properties. + +You can also refer to our [REST API style guide](../api_styleguide.md). + +#### GraphQL API + +Integrations use the `Types::Projects::ServiceType` type by default, +which only exposes the `type` and `active` properties. + +To expose additional properties, you can write a class implementing `ServiceType`: + +```ruby +# in app/graphql/types/project/services/foo_bar_service_type.rb +module Types + module Projects + module Services + class FooBarServiceType < BaseObject + graphql_name 'FooBarService' + implements(Types::Projects::ServiceType) + authorize :read_project + + field :frobinity, + GraphQL::Types::Float, + null: true, + description: 'The level of frobinity.' + + field :foo_label, + GraphQL::Types::String, + null: true, + description: 'The foo label to apply.' + end + end + end +end +``` + +Each property you want to expose should have a field defined for it. You can also expose any public instance method of the integration. + +Contact a member of the Integrations team to discuss the best authorization. + +Reference documentation for GraphQL is automatically generated. + +You can also refer to our [GraphQL API style guide](../api_graphql_styleguide.md). + +## Availability of integrations + +By default, integrations are available on the project, group, and instance level. +Most integrations only act in a project context, but can be still configured +from the group and instance levels. + +For some integrations it can make sense to only make it available on the project level. +To do that, the integration must be removed from `Integration::INTEGRATION_NAMES` and +added to `Integration::PROJECT_SPECIFIC_INTEGRATION_NAMES` instead. + +When developing a new integration, we also recommend you gate the availability behind a +[feature flag](../feature_flags/index.md) in `Integration.available_integration_names`. + +## Documentation + +You can provide help text in the integration form, including links to off-site documentation, +as described above in [Customize the frontend form](#customize-the-frontend-form). Refer to +our [usability guidelines](https://design.gitlab.com/usability/helping-users) for help text. + +For more detailed documentation, provide a page in `doc/user/project/integrations`, +and link it from the [Integrations overview](../../user/project/integrations/overview.md). + +You can also refer to our general [documentation guidelines](../documentation/index.md). + +## Testing + +It is often sufficient to add tests for the integration model in `spec/models/integrations`, +and a factory with example settings in `spec/factories/integrations.rb`. + +Each integration is also tested as part of generalized tests. For example, there are feature specs +that verify that the settings form is rendering correctly for all integrations. + +If your integration implements any custom behavior, especially in the frontend, this should be +covered by additional tests. + +You can also refer to our general [testing guidelines](../testing_guide/index.md). + +## Internationalization + +All UI strings should be prepared for translation by following our [internationalization guidelines](../i18n/externalization.md). + +The strings should use the integration name as [namespace](../i18n/externalization.md#namespaces), for example, `s_('FooBarIntegration|My string')`. + +## Ongoing migrations and refactorings + +The Integrations team is in the process of some larger migrations that developers should be aware of. + +### [Rename "services" to "integrations"](https://gitlab.com/groups/gitlab-org/-/epics/2504) + +The "integrations" in GitLab were historically called "services", which frequently caused +confusion with our "service" classes in `app/services`. We sometimes also called +them "project services" because they were initially only available on projects, which is +not the case anymore. + +We decided to change the naming from "services" and "project services" to "integrations". +This refactoring is an ongoing effort, and there are still references to the old names in some places. + +Developers should be especially aware that we still use the old class names for the STI column +`integrations.type`. For example, a class `Integrations::FooBar` still stores +the old name `FooBarService` in the database. This mapping is handled via `Gitlab::Integrations::StiType` +and should be mostly transparent to the rest of the app. + +### [Consolidate integration settings](https://gitlab.com/groups/gitlab-org/-/epics/3955) + +We want to unify the way integration properties are defined. + +## Integration examples + +You can refer to these issues for examples of adding new integrations: + +- [Datadog](https://gitlab.com/gitlab-org/gitlab/-/issues/270123): Metrics collector, similar to the Prometheus integration. +- [EWM/RTC](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/36662): External issue tracker. +- [Shimo](https://gitlab.com/gitlab-org/gitlab/-/issues/343386): External wiki, similar to the Confluence and External Wiki integrations. +- [Webex Teams](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/31543): Chat notifications. +- [ZenTao](https://gitlab.com/gitlab-org/gitlab/-/issues/338178): External issue tracker with custom issue views, similar to the Jira integration. diff --git a/doc/development/integrations/jira_connect.md b/doc/development/integrations/jira_connect.md index cfa1fdba699..5391b2c119e 100644 --- a/doc/development/integrations/jira_connect.md +++ b/doc/development/integrations/jira_connect.md @@ -65,3 +65,52 @@ If the app install failed, you might need to delete `jira_connect_installations` 1. Open the [database console](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/howto/postgresql.md#access-postgresql). 1. Run `TRUNCATE TABLE jira_connect_installations CASCADE;`. + +#### Not authorized to access the file + +If you use Gitpod and you get an error about Jira not being able to access the descriptor file, you might need to make the GDK's port public by following these steps: + +1. Open your GitLab workspace in Gitpod. +1. When the GDK is running, select **Ports** in the bottom-right corner. +1. On the left sidebar, select the port the GDK is listening to (typically `3000`). +1. If the port is marked as private, select the lock icon to make it public. + +## Test the GitLab OAuth authentication flow + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/81126) in GitLab 14.9 [with a flag](../../administration/feature_flags.md) named `jira_connect_oauth`. Disabled by default. + +GitLab for Jira users can authenticate with GitLab using GitLab OAuth. + +WARNING: +This feature is not ready for production use. The feature flag should only be enabled in development. + +The following steps describe setting up an environment to test the GitLab OAuth flow: + +1. Start a Gitpod session and open the rails console. + + ```shell + bundle exec rails console + ``` + +1. Enable the feature flag. + + ```shell + Feature.enable(:jira_connect_oauth) + ``` + +1. On your GitLab instance, go to **Admin > Applications**. +1. Create a new application with the following settings: + - Name: `Jira Connect` + - Redirect URI: `YOUR_GITPOD_INSTANCE/-/jira_connect/oauth_callbacks` + - Scopes: `api` + - Trusted: **No** + - Confidential: **No** +1. Copy the Application ID. +1. Go to [gitpod.io/variables](https://gitpod.io/variables). +1. Create a new variable named `JIRA_CONNECT_OAUTH_CLIENT_ID`, with a scope of `*/*`, and paste the Application ID as the value. + +If you already have an active Gitpod instance, use the following command in the Gitpod terminal to set the environment variable: + +```shell +eval $(gp env -e JIRA_CONNECT_OAUTH_CLIENT_ID=$YOUR_APPLICATION_ID) +``` diff --git a/doc/development/integrations/secure.md b/doc/development/integrations/secure.md index 27a166aebf9..11fb06bd128 100644 --- a/doc/development/integrations/secure.md +++ b/doc/development/integrations/secure.md @@ -329,18 +329,42 @@ You can find the schemas for these scanners here: ### Enable report validation -In GitLab 14.10 and later, report validation against the schemas is enabled. To enable report validation for versions earlier than 14.10, -set [`VALIDATE_SCHEMA`](../../user/application_security/#enable-security-report-validation) to -`"true"`. +> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/354928) in GitLab 14.9, and planned for removal in GitLab 15.0. +DISCLAIMER: +This page contains information related to upcoming products, features, and functionality. +It is important to note that the information presented is for informational purposes only. +Please do not rely on this information for purchasing or planning purposes. +As with all projects, the items mentioned on this page are subject to change or delay. +The development, release, and timing of any products, features, or functionality remain at the +sole discretion of GitLab Inc. +In GitLab 15.0 and later, report validation is enabled and enforced. Reports that fail validation +are not ingested, and an error message displays on the corresponding pipeline. -Reports that don't pass validation are not ingested by GitLab, and an error message -displays on the corresponding pipeline. +In GitLab 14.10 and later, report validation against the schemas is enabled but not enforced. +Reports that fail validation are ingested but display a warning in the pipeline security tab. -You should ensure that reports generated by the scanner pass validation against the schema version -declared in your reports. GitLab uses the +To enforce report validation for GitLab version 14.10 and earlier, set +[`VALIDATE_SCHEMA`](../../user/application_security/#enable-security-report-validation) to `"true"`. + +### Report validation + +You must ensure that reports generated by the scanner pass validation against the schema version +declared in your reports. Reports that don't pass validation are not ingested by GitLab, and an +error message displays on the corresponding pipeline. + +Reports that use a deprecated version of the secure report schema are ingested but cause a warning +message to display on the corresponding pipeline. If you see this warning, update your +analyzer to use the latest available schemas. + +After the deprecation period for a schema version, the file is removed from GitLab. Reports that +declare removed versions are rejected, and an error message displays on the corresponding pipeline. + +GitLab uses the [`json_schemer`](https://www.rubydoc.info/gems/json_schemer) gem to perform validation. -Ongoing improvements to report validation is tracked [in this epic](https://gitlab.com/groups/gitlab-org/-/epics/6968). +Ongoing improvements to report validation are tracked [in this epic](https://gitlab.com/groups/gitlab-org/-/epics/6968). +In the meantime, you can see which versions are supported in the +[source code](https://gitlab.com/gitlab-org/gitlab/-/blob/08dd756429731a0cca1e27ca9d59eea226398a7d/lib/gitlab/ci/parsers/security/validators/schema_validator.rb#L9-27). ### Report Fields diff --git a/doc/development/internal_api.md b/doc/development/internal_api.md deleted file mode 100644 index 7790a5e23e6..00000000000 --- a/doc/development/internal_api.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: 'internal_api/index.md' -remove_date: '2022-02-09' ---- - -This document was moved to [another location](internal_api/index.md). - -<!-- This redirect file can be deleted after <2022-02-09>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/development/internal_api/index.md b/doc/development/internal_api/index.md index db978253747..ef58d6c2c44 100644 --- a/doc/development/internal_api/index.md +++ b/doc/development/internal_api/index.md @@ -42,7 +42,7 @@ file, and include the token Base64 encoded in a `secret_token` parameter or in the `Gitlab-Shared-Secret` header. NOTE: -The internal API used by GitLab Pages, and GitLab Agent Server (`kas`) uses JSON Web Token (JWT) +The internal API used by GitLab Pages, and GitLab agent server (`kas`) uses JSON Web Token (JWT) authentication, which is different from GitLab Shell. ## Git Authentication @@ -400,25 +400,22 @@ Example response: } ``` -## GitLab Agent endpoints +## GitLab agent endpoints > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/41045) in GitLab 13.4. > - This feature is not deployed on GitLab.com > - It's not recommended for production use. -The following endpoints are used by the GitLab Agent Server (`kas`) +The following endpoints are used by the GitLab agent server (`kas`) for various purposes. These endpoints are all authenticated using JWT. The JWT secret is stored in a file specified in `config/gitlab.yml`. By default, the location is in the root of the GitLab Rails app in a file called `.gitlab_kas_secret`. -WARNING: -The GitLab Agent is under development and is not recommended for production use. +### GitLab agent information -### GitLab Agent information - -Called from GitLab Agent Server (`kas`) to retrieve agent +Called from GitLab agent server (`kas`) to retrieve agent information for the given agent token. This returns the Gitaly connection information for the agent's project in order for `kas` to fetch and update the agent's configuration. @@ -434,9 +431,9 @@ curl --request GET --header "Gitlab-Kas-Api-Request: <JWT token>" \ --header "Authorization: Bearer <agent token>" "http://localhost:3000/api/v4/internal/kubernetes/agent_info" ``` -### GitLab Agent project information +### GitLab agent project information -Called from GitLab Agent Server (`kas`) to retrieve project +Called from GitLab agent server (`kas`) to retrieve project information for the given agent token. This returns the Gitaly connection for the requested project. GitLab `kas` uses this to configure the agent to fetch Kubernetes resources from the project repository to @@ -460,9 +457,9 @@ curl --request GET --header "Gitlab-Kas-Api-Request: <JWT token>" \ --header "Authorization: Bearer <agent token>" "http://localhost:3000/api/v4/internal/kubernetes/project_info?id=7" ``` -### GitLab Agent usage metrics +### GitLab agent usage metrics -Called from GitLab Agent Server (`kas`) to increase the usage +Called from GitLab agent server (`kas`) to increase the usage metric counters. | Attribute | Type | Required | Description | @@ -481,9 +478,9 @@ curl --request POST --header "Gitlab-Kas-Api-Request: <JWT token>" --header "Con --data '{"gitops_sync_count":1}' "http://localhost:3000/api/v4/internal/kubernetes/usage_metrics" ``` -### GitLab Agent alert metrics +### GitLab agent alert metrics -Called from GitLab Agent Server (KAS) to save alerts derived from Cilium on Kubernetes +Called from GitLab agent server (KAS) to save alerts derived from Cilium on Kubernetes Cluster. | Attribute | Type | Required | Description | @@ -505,7 +502,7 @@ curl --request POST --header "Gitlab-Kas-Api-Request: <JWT token>" \ ### Create Starboard vulnerability -Called from the GitLab Agent Server (`kas`) to create a security vulnerability +Called from the GitLab agent server (`kas`) to create a security vulnerability from a Starboard vulnerability report. This request is idempotent. Multiple requests with the same data create a single vulnerability. The response contains the UUID of the created vulnerability finding. @@ -563,7 +560,7 @@ Example response: ### Resolve Starboard vulnerabilities -Called from the GitLab Agent Server (`kas`) to resolve Starboard security vulnerabilities. +Called from the GitLab agent server (`kas`) to resolve Starboard security vulnerabilities. Accepts a list of finding UUIDs and marks all Starboard vulnerabilities not identified by the list as resolved. diff --git a/doc/development/internal_users.md b/doc/development/internal_users.md index 35db2016e1c..95ca593e31e 100644 --- a/doc/development/internal_users.md +++ b/doc/development/internal_users.md @@ -42,3 +42,5 @@ Other examples of internal users: - [Ghost User](../user/profile/account/delete_account.md#associated-records) - [Support Bot](../user/project/service_desk.md#support-bot-user) - Visual Review Bot +- Resource access tokens (including [project access tokens](../user/project/settings/project_access_tokens.md)). + These are implemented as `project_bot` users with a `PersonalAccessToken`. diff --git a/doc/development/kubernetes.md b/doc/development/kubernetes.md index 45c94019c63..a6d9c754838 100644 --- a/doc/development/kubernetes.md +++ b/doc/development/kubernetes.md @@ -136,7 +136,7 @@ Mitigation strategies include: 1. Not allowing redirects to attacker controller resources: [`Kubeclient::KubeClient`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/kubernetes/kube_client.rb#) - can be configured to disallow any redirects by passing in + can be configured to prevent any redirects by passing in `http_max_redirects: 0` as an option. 1. Not exposing error messages: by doing so, we prevent attackers from triggering errors to expose results from diff --git a/doc/development/licensed_feature_availability.md b/doc/development/licensed_feature_availability.md index 629d0027ffe..0de3f94cf70 100644 --- a/doc/development/licensed_feature_availability.md +++ b/doc/development/licensed_feature_availability.md @@ -17,9 +17,8 @@ feature such as [Related issues](../user/project/issues/related_issues.md) or [Service Desk](../user/project/service_desk.md), it should be restricted on namespace scope. -1. Add the feature symbol on `EES_FEATURES`, `EEP_FEATURES`, or `EEU_FEATURES` constants in - `ee/app/models/license.rb`. Note that the prefix `EES` signifies Starter, `EEP` signifies - Premium, and `EEU` signifies Ultimate. +1. Add the feature symbol on `STARTER_FEATURES`, `PREMIUM_FEATURES`, or `ULTIMATE_FEATURES` constants in + `ee/app/models/gitlab_subscriptions/features.rb`. 1. Check using: ```ruby @@ -33,8 +32,8 @@ However, for features such as [Geo](../administration/geo/index.md) and to only a subset of projects or namespaces, the check is made directly in the instance license. -1. Add the feature symbol on `EES_FEATURES`, `EEP_FEATURES` or `EEU_FEATURES` constants in - `ee/app/models/license.rb`. +1. Add the feature symbol to `STARTER_FEATURES`, `PREMIUM_FEATURES` or `ULTIMATE_FEATURES` constants in + `ee/app/models/gitlab_subscriptions/features.rb`. 1. Add the same feature symbol to `GLOBAL_FEATURES`. 1. Check using: diff --git a/doc/development/merge_request_application_and_rate_limit_guidelines.md b/doc/development/merge_request_application_and_rate_limit_guidelines.md new file mode 100644 index 00000000000..94ae126802a --- /dev/null +++ b/doc/development/merge_request_application_and_rate_limit_guidelines.md @@ -0,0 +1,28 @@ +--- +stage: none +group: unassigned +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Application and rate limit guidelines + +GitLab, like most large applications, enforces limits within certain features. +The absences of limits can affect security, performance, data, or could even +exhaust the allocated resources for the application. + +Every new feature should have safe usage limits included in its implementation. +Limits are applicable for: + +- System-level resource pools such as API requests, SSHD connections, database connections, storage, and so on. +- Domain-level objects such as CI minutes, groups, sign-in attempts, and so on. + +## When limits are required + +1. Limits are required if the absence of the limit matches severity 1 - 3 in the severity definitions for [limit-related bugs](https://about.gitlab.com/handbook/engineering/quality/issue-triage/#limit-related-bugs). +1. [GitLab application limits](../administration/instance_limits.md) documentation must be updated anytime limits are added, removed, or updated. + +## Additional reading + +- Existing [GitLab application limits](../administration/instance_limits.md) +- Product processes: [introducing application limits](https://about.gitlab.com/handbook/product/product-processes/#introducing-application-limits) +- Development docs: [guide for adding application limits](application_limits.md) diff --git a/doc/development/merge_request_concepts/index.md b/doc/development/merge_request_concepts/index.md new file mode 100644 index 00000000000..f1dab69543a --- /dev/null +++ b/doc/development/merge_request_concepts/index.md @@ -0,0 +1,62 @@ +--- +type: reference, dev +stage: create +group: code_review +info: "See the Technical Writers assigned to Development Guidelines: https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments-to-development-guidelines" +--- + +# Merge Request Concepts + +**NOTE**: +The documentation below is the single source of truth for the merge request terminology and functionality. + +## Overview + +The merge request is made up of several different key components and ideas that encompass the overall merge request experience. These concepts sometimes have competing and confusing terminology or overlap with other concepts. The concepts this will cover are: + +1. Merge widget +1. Report widgets +1. Merge checks +1. Approval rules + +### Merge widget + +The merge widget is the component of the merge request where the `merge` button exists: + + + +This area of the merge request is where all of the options and commit messages are defined prior to merging. It also contains information about what is in the merge request, what issues may be closed, and other important information to the merging process. + +### Report widgets + +Reports are widgets within the merge request that report information about changes within the merge request. These widgets provide information to better help the author understand the changes and further improvements to the proposed changes. + +[Design Documentation](https://design.gitlab.com/regions/merge-request-reports) + + + +### Merge checks + +Merge checks are statuses that can either pass or fail and conditionally control the availability of the merge button being available within a merge request. The key distinguishing factor in a merge check is that users **do not** interact with the merge checks inside of the merge request, but are able to influence whether or not the check passes or fails. Results from the check are processed as true/false to determine whether or not a merge request can be merged. Examples include: + +1. merge conflicts +1. pipeline success +1. threads resolution +1. [external status checks](../../user/project/merge_requests/status_checks.md) +1. required approvals + +When all of the required merge checks are satisfied a merge request becomes mergeable. + +### Approvals + +Approval rules specify users that are required to or can optionally approve a merge request based on some kind of organizational policy. When approvals are required, they effectively become a required merge check. The key differentiator between merge checks and approval rules is that users **do** interact with approval rules, by deciding to approve the merge request. + +Additionally, approval settings provide configuration options to define how those approval rules are applied in a merge request. They can set limitations, add requirements, or modify approvals. + +Examples of approval rules and settings include: + +1. [merge request approval rules](../../user/project/merge_requests/approvals/rules.md) +1. [code owner approvals](../../user/project/code_owners.md) +1. [security approvals](../../user/application_security/index.md#security-approvals-in-merge-requests) +1. [prevent editing approval rules](../../user/project/merge_requests/approvals/settings.md#prevent-editing-approval-rules-in-merge-requests)] +1. [remove all approvals when commits are added](../../user/project/merge_requests/approvals/settings.md#remove-all-approvals-when-commits-are-added-to-the-source-branch) diff --git a/doc/development/merge_request_performance_guidelines.md b/doc/development/merge_request_performance_guidelines.md index c8e99e8547f..40f02f4fb6f 100644 --- a/doc/development/merge_request_performance_guidelines.md +++ b/doc/development/merge_request_performance_guidelines.md @@ -446,49 +446,6 @@ that accepts an upper limit of counting rows. In some cases it's desired that badge counters are loaded asynchronously. This can speed up the initial page load and give a better user experience overall. -## Application/misuse limits - -Every new feature should have safe usage quotas introduced. -The quota should be optimised to a level that we consider the feature to -be performant and usable for the user, but **not limiting**. - -**We want the features to be fully usable for the users.** -**However, we want to ensure that the feature continues to perform well if used at its limit** -**and it doesn't cause availability issues.** - -Consider that it's always better to start with some kind of limitation, -instead of later introducing a breaking change that would result in some -workflows breaking. - -The intent is to provide a safe usage pattern for the feature, -as our implementation decisions are optimised for the given data set. -Our feature limits should reflect the optimisations that we introduced. - -The intent of quotas could be different: - -1. We want to provide higher quotas for higher tiers of features: - we want to provide on GitLab.com more capabilities for different tiers, -1. We want to prevent misuse of the feature: someone accidentally creates - 10000 deploy tokens, because of a broken API script, -1. We want to prevent abuse of the feature: someone purposely creates - a 10000 pipelines to take advantage of the system. - -Examples: - -1. Pipeline Schedules: It's very unlikely that user wants to create - more than 50 schedules. - In such cases it's rather expected that this is either misuse - or abuse of the feature. Lack of the upper limit can result - in service degradation as the system tries to process all schedules - assigned the project. - -1. GitLab CI/CD includes: We started with the limit of maximum of 50 nested includes. - We understood that performance of the feature was acceptable at that level. - We received a request from the community that the limit is too small. - We had a time to understand the customer requirement, and implement an additional - fail-safe mechanism (time-based one) to increase the limit 100, and if needed increase it - further without negative impact on availability of the feature and GitLab. - ## Usage of feature flags Each feature that has performance critical elements or has a known performance deficiency @@ -569,7 +526,7 @@ end The usage of shared temporary storage is required if your intent is to persistent file for a disk-based storage, and not Object Storage. -[Workhorse direct_upload](uploads.md#direct-upload) when accepting file +[Workhorse direct_upload](uploads/implementation.md#direct-upload) when accepting file can write it to shared storage, and later GitLab Rails can perform a move operation. The move operation on the same destination is instantaneous. The system instead of performing `copy` operation just re-attaches file into a new place. @@ -593,7 +550,7 @@ that implements a seamless support for Shared and Object Storage-based persisten #### Data access Each feature that accepts data uploads or allows to download them needs to use -[Workhorse direct_upload](uploads.md#direct-upload). It means that uploads needs to be +[Workhorse direct_upload](uploads/implementation.md#direct-upload). It means that uploads needs to be saved directly to Object Storage by Workhorse, and all downloads needs to be served by Workhorse. @@ -605,5 +562,5 @@ can time out, which is especially problematic for slow clients. If clients take to upload/download the processing slot might be killed due to request processing timeout (usually between 30s-60s). -For the above reasons it is required that [Workhorse direct_upload](uploads.md#direct-upload) is implemented +For the above reasons it is required that [Workhorse direct_upload](uploads/implementation.md#direct-upload) is implemented for all file uploads and downloads. diff --git a/doc/development/new_fe_guide/modules/widget_extensions.md b/doc/development/new_fe_guide/modules/widget_extensions.md index 37712cb2cec..d3cd839464d 100644 --- a/doc/development/new_fe_guide/modules/widget_extensions.md +++ b/doc/development/new_fe_guide/modules/widget_extensions.md @@ -128,6 +128,7 @@ mentioned below: variant: '', // Optional: GitLab UI badge variant, defaults to info }, actions: [], // Optional: Action button for row + children: [], // Optional: Child content to render, structure matches the same structure } ``` diff --git a/doc/development/packages.md b/doc/development/packages.md index 38c1b941eaf..35a93c77c7f 100644 --- a/doc/development/packages.md +++ b/doc/development/packages.md @@ -151,7 +151,7 @@ During this phase, the idea is to collect as much information as possible about 1. Empty file structure (API file, base service for this package) 1. Authentication system for "logging in" to the package manager 1. Identify metadata and create applicable tables - 1. Workhorse route for [object storage direct upload](uploads.md#direct-upload) + 1. Workhorse route for [object storage direct upload](uploads/implementation.md#direct-upload) 1. Endpoints required for upload/publish 1. Endpoints required for install/download 1. Endpoints required for required actions @@ -210,7 +210,7 @@ File uploads should be handled by GitLab Workhorse using object accelerated uplo the workhorse proxy that checks all incoming requests to GitLab intercept the upload request, upload the file, and forward a request to the main GitLab codebase only containing the metadata and file location rather than the file itself. An overview of this process can be found in the -[development documentation](uploads.md#direct-upload). +[development documentation](uploads/implementation.md#direct-upload). In terms of code, this means a route must be added to the [GitLab Workhorse project](https://gitlab.com/gitlab-org/gitlab-workhorse) for each upload endpoint being added @@ -272,7 +272,7 @@ features must be implemented when the feature flag is removed. - File format guards (only accept valid file formats for the package type) - Name regex with validation - Version regex with validation -- Workhorse route for [accelerated](uploads.md#how-to-add-a-new-upload-route) uploads +- Workhorse route for [accelerated](uploads/working_with_uploads.md) uploads - Background workers for extracting package metadata (if applicable) - Documentation (how to use the feature) - API Documentation (individual endpoints with curl examples) diff --git a/doc/development/performance.md b/doc/development/performance.md index b5294c8359d..1e3e0570206 100644 --- a/doc/development/performance.md +++ b/doc/development/performance.md @@ -7,7 +7,38 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Performance Guidelines This document describes various guidelines to follow to ensure good and -consistent performance of GitLab. +consistent performance of GitLab. Refer to the [Index](#performance-documentation) section below to navigate to Performance-related pages. + +## Performance Documentation + +- General: + - [Solving performance issues](#workflow) + - [Handbook performance page](https://about.gitlab.com/handbook/engineering/performance/) + - [Merge request performance guidelines](../development/merge_request_performance_guidelines.md) +- Backend: + - [Tooling](#tooling) + - Database: + - [Query performance guidelines](../development/query_performance.md) + - [Pagination performance guidelines](../development/database/pagination_performance_guidelines.md) + - [Keyset pagination performance](../development/database/keyset_pagination.md#performance) + - [Troubleshooting import/export performance issues](../development/import_export.md#troubleshooting-performance-issues) + - [Pipelines performance in the `gitlab` project](../development/pipelines.md#performance) +- Frontend: + - [Performance guidelines](../development/fe_guide/performance.md) + - [Performance dashboards and monitoring guidelines](../development/new_fe_guide/development/performance.md) + - [Browser performance testing guidelines](../user/project/merge_requests/browser_performance_testing.md) + - [`gdk measure` and `gdk measure-workflow`](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/gdk_commands.md#measure-performance) +- QA: + - [Load performance testing](../user/project/merge_requests/load_performance_testing.md) + - [GitLab Performance Tool project](https://gitlab.com/gitlab-org/quality/performance) + - [Review apps performance metrics](../development/testing_guide/review_apps.md#performance-metrics) +- Monitoring & Overview: + - [GitLab performance monitoring](../administration/monitoring/performance/index.md) + - [Development department performance indicators](https://about.gitlab.com/handbook/engineering/development/performance-indicators/) + - [Service measurement](../development/service_measurement.md) +- Self-managed administration and customer-focused: + - [File system performance benchmarking](../administration/operations/filesystem_benchmarking.md) + - [Sidekiq performance troubleshooting](../administration/troubleshooting/sidekiq.md) ## Workflow @@ -95,8 +126,13 @@ end This however leads to the question: how many iterations should we run to get meaningful statistics? -The benchmark-ips Gem basically takes care of all this and much more, and as a -result of this should be used instead of the `Benchmark` module. +The [`benchmark-ips`](https://github.com/evanphx/benchmark-ips) +gem takes care of all this and much more. You should therefore use it instead of the `Benchmark` +module. + +The GitLab Gemfile also contains the [`benchmark-memory`](https://github.com/michaelherold/benchmark-memory) +gem, which works similarly to the `benchmark` and `benchmark-ips` gems. However, `benchmark-memory` +instead returns the memory size, objects, and strings allocated and retained during the benchmark. In short: @@ -110,7 +146,7 @@ In short: - If you must write a benchmark use the benchmark-ips Gem instead of Ruby's `Benchmark` module. -## Profiling +## Profiling with Stackprof By collecting snapshots of process state at regular intervals, profiling allows you to see where time is spent in a process. The @@ -124,15 +160,36 @@ frequency (for example, 100hz, that is 100 stacks per second). This type of prof has quite a low (albeit non-zero) overhead and is generally considered to be safe for production. -### Development - A profiler can be a very useful tool during development, even if it does run *in an unrepresentative environment*. In particular, a method is not necessarily troublesome just because it's executed many times, or takes a long time to execute. Profiles are tools you can use to better understand what is happening in an application - using that information wisely is up to you! -Keeping that in mind, to create a profile, identify (or create) a spec that +There are multiple ways to create a profile with Stackprof. + +### Wrapping a code block + +To profile a specific code block, you can wrap that block in a `Stackprof.run` call: + +```ruby +StackProf.run(mode: :wall, out: 'tmp/stackprof-profiling.dump') do + #... +end +``` + +This creates a `.dump` file that you can [read](#reading-a-stackprof-profile). +For all available options, see the [Stackprof documentation](https://github.com/tmm1/stackprof#all-options). + +### Performance bar + +With the [Performance bar](../administration/monitoring/performance/performance_bar.md), +you have the option to profile a request using Stackprof and immediately output the results to a +[Speedscope flamegraph](profiling.md#speedscope-flamegraphs). + +### RSpec profiling with Stackprof + +To create a profile from a spec, identify (or create) a spec that exercises the troublesome code path, then run it using the `bin/rspec-stackprof` helper, for example: @@ -161,89 +218,10 @@ Finished in 18.19 seconds (files took 4.8 seconds to load) 187 (1.1%) 187 (1.1%) block (4 levels) in class_attribute ``` -You can limit the specs that are run by passing any arguments `rspec` would +You can limit the specs that are run by passing any arguments `RSpec` would normally take. -The output is sorted by the `Samples` column by default. This is the number of -samples taken where the method is the one currently being executed. The `Total` -column shows the number of samples taken where the method, or any of the methods -it calls, were being executed. - -To create a graphical view of the call stack: - -```shell -stackprof tmp/project_policy_spec.rb.dump --graphviz > project_policy_spec.dot -dot -Tsvg project_policy_spec.dot > project_policy_spec.svg -``` - -To load the profile in [KCachegrind](https://kcachegrind.github.io/): - -```shell -stackprof tmp/project_policy_spec.rb.dump --callgrind > project_policy_spec.callgrind -kcachegrind project_policy_spec.callgrind # Linux -qcachegrind project_policy_spec.callgrind # Mac -``` - -For flame graphs, enable raw collection first. Note that raw -collection can generate a very large file, so increase the `INTERVAL`, or -run on a smaller number of specs for smaller file size: - -```shell -RAW=true bin/rspec-stackprof spec/policies/group_member_policy_spec.rb -``` - -You can then generate, and view the resultant flame graph. It might take a -while to generate based on the output file size: - -```shell -# Generate -stackprof --flamegraph tmp/group_member_policy_spec.rb.dump > group_member_policy_spec.flame - -# View -stackprof --flamegraph-viewer=group_member_policy_spec.flame -``` - -It may be useful to zoom in on a specific method, for example: - -```shell -$ stackprof tmp/project_policy_spec.rb.dump --method warm_asset_cache - -TestEnv#warm_asset_cache (/Users/lupine/dev/gitlab.com/gitlab-org/gitlab-development-kit/gitlab/spec/support/test_env.rb:164) - samples: 0 self (0.0%) / 6288 total (36.9%) - callers: - 6288 ( 100.0%) block (2 levels) in <top (required)> - callees (6288 total): - 6288 ( 100.0%) Capybara::RackTest::Driver#visit - code: - | 164 | def warm_asset_cache - | 165 | return if warm_asset_cache? - | 166 | return unless defined?(Capybara) - | 167 | - 6288 (36.9%) | 168 | Capybara.current_session.driver.visit '/' - | 169 | end -$ stackprof tmp/project_policy_spec.rb.dump --method BasePolicy#abilities -BasePolicy#abilities (/Users/lupine/dev/gitlab.com/gitlab-org/gitlab-development-kit/gitlab/app/policies/base_policy.rb:79) - samples: 0 self (0.0%) / 50 total (0.3%) - callers: - 25 ( 50.0%) BasePolicy.abilities - 25 ( 50.0%) BasePolicy#collect_rules - callees (50 total): - 25 ( 50.0%) ProjectPolicy#rules - 25 ( 50.0%) BasePolicy#collect_rules - code: - | 79 | def abilities - | 80 | return RuleSet.empty if @user && @user.blocked? - | 81 | return anonymous_abilities if @user.nil? - 50 (0.3%) | 82 | collect_rules { rules } - | 83 | end -``` - -Since the profile includes the work done by the test suite as well as the -application code, these profiles can be used to investigate slow tests as well. -However, for smaller runs (like this example), this means that the cost of -setting up the test suite tends to dominate. - -### Production +### Using Stackprof in production Stackprof can also be used to profile production workloads. @@ -274,8 +252,8 @@ the timeout. Once profiling stops, the profile is written out to disk at `$STACKPROF_FILE_PREFIX/stackprof.$PID.$RAND.profile`. It can then be inspected -further via the `stackprof` command line tool, as described in the previous -section. +further through the `stackprof` command line tool, as described in the +[Reading a Stackprof profile section](#reading-a-stackprof-profile). Currently supported profiling targets are: @@ -295,14 +273,85 @@ For Sidekiq, the signal can be sent to the `sidekiq-cluster` process via `pkill -USR2 bin/sidekiq-cluster`, which forwards the signal to all Sidekiq children. Alternatively, you can also select a specific PID of interest. -Production profiles can be especially noisy. It can be helpful to visualize them -as a [flame graph](https://github.com/brendangregg/FlameGraph). This can be done -via: +### Reading a Stackprof profile + +The output is sorted by the `Samples` column by default. This is the number of samples taken where +the method is the one currently executed. The `Total` column shows the number of samples taken where +the method (or any of the methods it calls) is executed. + +To create a graphical view of the call stack: ```shell -bundle exec stackprof --stackcollapse /tmp/stackprof.55769.c6c3906452.profile | flamegraph.pl > flamegraph.svg +stackprof tmp/project_policy_spec.rb.dump --graphviz > project_policy_spec.dot +dot -Tsvg project_policy_spec.dot > project_policy_spec.svg ``` +To load the profile in [KCachegrind](https://kcachegrind.github.io/): + +```shell +stackprof tmp/project_policy_spec.rb.dump --callgrind > project_policy_spec.callgrind +kcachegrind project_policy_spec.callgrind # Linux +qcachegrind project_policy_spec.callgrind # Mac +``` + +You can also generate and view the resultant flame graph. To view a flame graph that +`bin/rspec-stackprof` creates, you must set the `RAW` environment variable to `true` when running +`bin/rspec-stackprof`. + +It might take a while to generate based on the output file size: + +```shell +# Generate +stackprof --flamegraph tmp/group_member_policy_spec.rb.dump > group_member_policy_spec.flame + +# View +stackprof --flamegraph-viewer=group_member_policy_spec.flame +``` + +To export the flame graph to an SVG file, use [Brendan Gregg's FlameGraph tool](https://github.com/brendangregg/FlameGraph): + +```shell +stackprof --stackcollapse /tmp/group_member_policy_spec.rb.dump | flamegraph.pl > flamegraph.svg +``` + +It's also possible to view flame graphs through [speedscope](https://github.com/jlfwong/speedscope). +You can do this when using the [performance bar](profiling.md#speedscope-flamegraphs) +and when [profiling code blocks](https://github.com/jlfwong/speedscope/wiki/Importing-from-stackprof-(ruby)). +This option isn't supported by `bin/rspec-stackprof`. + +You can profile speciific methods by using `--method method_name`: + +```shell +$ stackprof tmp/project_policy_spec.rb.dump --method access_allowed_to + +ProjectPolicy#access_allowed_to? (/Users/royzwambag/work/gitlab-development-kit/gitlab/app/policies/project_policy.rb:793) + samples: 0 self (0.0%) / 578 total (0.7%) + callers: + 397 ( 68.7%) block (2 levels) in <class:ProjectPolicy> + 95 ( 16.4%) block in <class:ProjectPolicy> + 86 ( 14.9%) block in <class:ProjectPolicy> + callees (578 total): + 399 ( 69.0%) ProjectPolicy#team_access_level + 141 ( 24.4%) Project::GeneratedAssociationMethods#project_feature + 30 ( 5.2%) DeclarativePolicy::Base#can? + 8 ( 1.4%) Featurable#access_level + code: + | 793 | def access_allowed_to?(feature) + 141 (0.2%) | 794 | return false unless project.project_feature + | 795 | + 8 (0.0%) | 796 | case project.project_feature.access_level(feature) + | 797 | when ProjectFeature::DISABLED + | 798 | false + | 799 | when ProjectFeature::PRIVATE + 429 (0.5%) | 800 | can?(:read_all_resources) || team_access_level >= ProjectFeature.required_minimum_access_level(feature) + | 801 | else +``` + +When using Stackprof to profile specs, the profile includes the work done by the test suite and the +application code. You can therefore use these profiles to investigate slow tests as well. However, +for smaller runs (like this example), this means that the cost of setting up the test suite tends to +dominate. + ## RSpec profiling The GitLab development environment also includes the @@ -459,11 +508,14 @@ The `mem_*` values represent different aspects of how objects and memory are all We can use `memory_profiler` for profiling. -The [`memory_profiler`](https://github.com/SamSaffron/memory_profiler) gem is already present in the GitLab `Gemfile`, -you just need to require it: +The [`memory_profiler`](https://github.com/SamSaffron/memory_profiler) +gem is already present in the GitLab `Gemfile`. It's also available in the [performance bar](../administration/monitoring/performance/performance_bar.md) +for the current URL. + +To use the memory profiler directly in your code, use `require` to add it: ```ruby -require 'sidekiq/testing' +require 'memory_profiler' report = MemoryProfiler.report do # Code you want to profile @@ -473,10 +525,17 @@ output = File.open('/tmp/profile.txt','w') report.pretty_print(output) ``` -The report breaks down 2 key concepts: +The report shows the retained and allocated memory grouped by gem, file, location, and class. The +memory profiler also performs a string analysis that shows how often a string is allocated and +retained. -- Retained: long lived memory use and object count retained due to the execution of the code block. -- Allocated: all object allocation and memory allocation during code block. +#### Retained versus allocated + +- Retained memory: long-lived memory use and object count retained due to the execution of the code + block. This has a direct impact on memory and the garbage collector. +- Allocated memory: all object allocation and memory allocation during the code block. This might + have minimal impact on memory, but substantial impact on performance. The more objects you + allocate, the more work is being done and the slower the application is. As a general rule, **retained** is always smaller than or equal to **allocated**. @@ -512,6 +571,32 @@ Fragmented Ruby heap snapshot could look like this: Memory fragmentation could be reduced by tuning GC parameters [as described in this post](https://www.speedshop.co/2017/12/04/malloc-doubles-ruby-memory.html). This should be considered as a tradeoff, as it may affect overall performance of memory allocation and GC cycles. +### Derailed Benchmarks + +`derailed_benchmarks` is a [gem](https://github.com/zombocom/derailed_benchmarks) +described as "A series of things you can use to benchmark a Rails or Ruby app." +We include `derailed_benchmarks` in our `Gemfile`. + +We run `derailed exec perf:mem` in every pipeline with a `test` stage, in a job +called `memory-on-boot`. ([Read an example job.](https://gitlab.com/gitlab-org/gitlab/-/jobs/2144695684).) +You may find the results: + +- On the merge request **Overview** tab, in the merge request reports area, in the + **Metrics Reports** [dropdown list](../ci/metrics_reports.md). +- In the `memory-on-boot` artifacts for a full report and a dependency breakdown. + +`derailed_benchmarks` also provides other methods to investigate memory. To learn more, +refer to the [gem documentation](https://github.com/zombocom/derailed_benchmarks#running-derailed-exec). +Most of the methods (`derailed exec perf:*`) attempt to boot your Rails app in a +`production` environment and run benchmarks against it. +It is possible both in GDK and GCK: + +- For GDK, follow the + [the instructions](https://github.com/zombocom/derailed_benchmarks#running-in-production-locally) + on the gem page. You must do similar for Redis configurations to avoid errors. +- GCK includes `production` configuration sections + [out of the box](https://gitlab.com/gitlab-org/gitlab-compose-kit#running-production-like). + ## Importance of Changes When working on performance improvements, it's important to always ask yourself @@ -612,7 +697,7 @@ end ## String Freezing -In recent Ruby versions calling `freeze` on a String leads to it being allocated +In recent Ruby versions calling `.freeze` on a String leads to it being allocated only once and re-used. For example, on Ruby 2.3 or later this only allocates the "foo" String once: @@ -626,6 +711,12 @@ Depending on the size of the String and how frequently it would be allocated (before the `.freeze` call was added), this _may_ make things faster, but this isn't guaranteed. +Freezing strings saves memory, as every allocated string uses at least one `RVALUE_SIZE` bytes (40 +bytes on x64) of memory. + +You can use the [memory profiler](#using-memory-profiler) +to see which strings are allocated often and could potentially benefit from a `.freeze`. + Strings are frozen by default in Ruby 3.0. To prepare our codebase for this eventuality, we are adding the following header to all Ruby files: diff --git a/doc/development/permissions.md b/doc/development/permissions.md index a5d211a5d2e..47aebc2f4d2 100644 --- a/doc/development/permissions.md +++ b/doc/development/permissions.md @@ -9,6 +9,23 @@ info: To determine the technical writer assigned to the Stage/Group associated w There are multiple types of permissions across GitLab, and when implementing anything that deals with permissions, all of them should be considered. +## Instance + +### User types + +Each user can be one of the following types: + +- Regular. +- External - access to groups and projects only if direct member. +- [Internal users](internal_users.md) - system created. +- [Auditor](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/policies/ee/base_policy.rb#L9): + - No access to projects or groups settings menu. + - No access to Admin Area. + - Read-only access to everything else. +- [Administrator](https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/policies/base_policy.rb#L6) - read-write access. + +See the [permissions page](../user/permissions.md) for details on how each user type is used. + ## Groups and Projects ### General permissions @@ -38,7 +55,7 @@ Additionally, the following project features can have different visibility level - Issues - Repository - - Merge Request + - Merge request - Forks - Pipelines - Analytics @@ -124,9 +141,9 @@ into different features like Merge Requests and CI flow. | View | License information | Dependency list, License Compliance | Can view repository | | View | Dependency information | Dependency list, License Compliance | Can view repository | | View | Vulnerabilities information | Dependency list | Can view security findings | -| View | Black/Whitelisted licenses for the project | License Compliance, Merge request | Can view repository | -| View | Security findings | Merge Request, CI job page, Pipeline security tab | Can read the project and CI jobs | -| View | Vulnerability feedback | Merge Request | Can read security findings | +| View | Black/Whitelisted licenses for the project | License Compliance, merge request | Can view repository | +| View | Security findings | merge request, CI job page, Pipeline security tab | Can read the project and CI jobs | +| View | Vulnerability feedback | merge request | Can read security findings | | View | Dependency List page | Project | Can access Dependency information | | View | License Compliance page | Project | Can access License information| diff --git a/doc/development/pipelines.md b/doc/development/pipelines.md index f3a4f47eb22..2aef0e10314 100644 --- a/doc/development/pipelines.md +++ b/doc/development/pipelines.md @@ -37,7 +37,7 @@ flowchart LR subgraph backend be["Backend code"]--tested with-->rspec end - + be--generates-->fixtures["frontend fixtures"] fixtures--used in-->jest ``` @@ -69,7 +69,7 @@ In addition, there are a few circumstances where we would always run the full RS - when the `pipeline:run-all-rspec` label is set on the merge request - when the merge request is created by an automation (e.g. Gitaly update or MR targeting a stable branch) - when the merge request is created in a security mirror -- when any CI config file is changed (i.e. `.gitlab-ci.yml` or `.gitlab/ci/**/*`) +- when any CI configuration file is changed (i.e. `.gitlab-ci.yml` or `.gitlab/ci/**/*`) ### Jest minimal jobs @@ -85,7 +85,7 @@ In addition, there are a few circumstances where we would always run the full Je - when the `pipeline:run-all-jest` label is set on the merge request - when the merge request is created by an automation (e.g. Gitaly update or MR targeting a stable branch) - when the merge request is created in a security mirror -- when any CI config file is changed (i.e. `.gitlab-ci.yml` or `.gitlab/ci/**/*`) +- when any CI configuration file is changed (i.e. `.gitlab-ci.yml` or `.gitlab/ci/**/*`) - when any frontend "core" file is changed (i.e. `package.json`, `yarn.lock`, `babel.config.js`, `jest.config.*.js`, `config/helpers/**/*.js`) - when any vendored JavaScript file is changed (i.e. `vendor/assets/javascripts/**/*`) - when any backend file is changed ([see the patterns list for details](https://gitlab.com/gitlab-org/gitlab/-/blob/3616946936c1adbd9e754c1bd06f86ba670796d8/.gitlab/ci/rules.gitlab-ci.yml#L205-216)) @@ -194,6 +194,14 @@ We keep track of retried tests in the `$RETRIED_TESTS_REPORT_FILE` file saved as See the [experiment issue](https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/1148). +### Single database testing + +By default, all tests run with [multiple databases](database/multiple_databases.md). + +We also run tests with a single database in nightly scheduled pipelines, and in merge requests that touch database-related files. + +If you want to force tests to run with a single database, you can add the `pipeline:run-single-db` label to the merge request. + ### Monitoring The GitLab test suite is [monitored](performance.md#rspec-profiling) for the `main` branch, and any branch @@ -218,7 +226,7 @@ of `gitlab-org/gitlab-foss`. These jobs are only created in the following cases: - when the `pipeline:run-as-if-foss` label is set on the merge request - when the merge request is created in the `gitlab-org/security/gitlab` project -- when any CI config file is changed (i.e. `.gitlab-ci.yml` or `.gitlab/ci/**/*`) +- when any CI configuration file is changed (i.e. `.gitlab-ci.yml` or `.gitlab/ci/**/*`) The `* as-if-foss` jobs are run in addition to the regular EE-context jobs. They have the `FOSS_ONLY='1'` variable set and get the `ee/` folder removed before the tests start running. @@ -247,7 +255,7 @@ appending `-jh` to the branch name. If a corresponding JH branch is found, rather than from the default branch. NOTE: -For now, CI will try to fetch the branch on the [GitLab JH mirror](https://gitlab.com/gitlab-org/gitlab-jh/gitlab), so it might take some time for the new JH branch to propagate to the mirror. +For now, CI will try to fetch the branch on the [GitLab JH mirror](https://gitlab.com/gitlab-org/gitlab-jh-mirrors/gitlab), so it might take some time for the new JH branch to propagate to the mirror. ## `undercover` RSpec test @@ -263,6 +271,16 @@ In the event of an emergency, or false positive from this job, add the `pipeline:skip-undercoverage` label to the merge request to allow this job to fail. +You can disable the `undercover` code coverage check by wrapping the desired block of code in `# :nocov:` lines: + +```ruby +# :nocov: +def some_method + # code coverage for this method will be skipped +end +# :nocov: +``` + ## PostgreSQL versions testing Our test suite runs against PG12 as GitLab.com runs on PG12 and @@ -291,6 +309,21 @@ We follow the [PostgreSQL versions shipped with Omnibus GitLab](../administratio | PG11 | `nightly` | `nightly` | `nightly` | `nightly` | `nightly` | `nightly` | | PG13 | `nightly` | `nightly` | `nightly` | `nightly` | `nightly` | `nightly` | +## Redis versions testing + +Our test suite runs against Redis 6 as GitLab.com runs on Redis 6 and +[Omnibus defaults to Redis 6 for new installs and upgrades](https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/master/config/software/redis.rb). + +We do run our test suite against Redis 5 on `nightly` scheduled pipelines, specifically when running backward-compatible and forward-compatible PostgreSQL jobs. + +### Current versions testing + +| Where? | Redis version | +| ------ | ------------------ | +| MRs | 6 | +| `default branch` (non-scheduled pipelines) | 6 | +| `nightly` scheduled pipelines | 5 | + ## Pipelines types for merge requests In general, pipelines for an MR fall into one of the following types (from shorter to longer), depending on the changes made in the MR: @@ -531,6 +564,8 @@ that are scoped to a single [configuration keyword](../ci/yaml/index.md#job-keyw | `.use-pg11-ee` | Same as `.use-pg11` but also use an `elasticsearch` service (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific version of the service). | | `.use-pg12` | Allows a job to use the `postgres` 12 and `redis` services (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific versions of the services). | | `.use-pg12-ee` | Same as `.use-pg12` but also use an `elasticsearch` service (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific version of the service). | +| `.use-pg13` | Allows a job to use the `postgres` 13 and `redis` services (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific versions of the services). | +| `.use-pg13-ee` | Same as `.use-pg13` but also use an `elasticsearch` service (see [`.gitlab/ci/global.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/global.gitlab-ci.yml) for the specific version of the service). | | `.use-kaniko` | Allows a job to use the `kaniko` tool to build Docker images. | | `.as-if-foss` | Simulate the FOSS project by setting the `FOSS_ONLY='1'` CI/CD variable. | | `.use-docker-in-docker` | Allows a job to use Docker in Docker. | @@ -614,6 +649,20 @@ otherwise. If you want a running pipeline to finish even if you push new commits to a merge request, be sure to start the `dont-interrupt-me` job before pushing. +### Git fetch caching + +Because GitLab.com uses the [pack-objects cache](../administration/gitaly/configure_gitaly.md#pack-objects-cache), +concurrent Git fetches of the same pipeline ref are deduplicated on +the Gitaly server (always) and served from cache (when available). + +This works well for the following reasons: + +- The pack-objects cache is enabled on all Gitaly servers on GitLab.com. +- The CI/CD [Git strategy setting](../ci/pipelines/settings.md#choose-the-default-git-strategy) for `gitlab-org/gitlab` is **Git clone**, + causing all jobs to fetch the same data, which maximizes the cache hit ratio. +- We use [shallow clone](../ci/pipelines/settings.md#limit-the-number-of-changes-fetched-during-clone) to avoid downloading the full Git + history for every job. + ### Caching strategy 1. All jobs must only pull caches by default. @@ -647,19 +696,31 @@ request, be sure to start the `dont-interrupt-me` job before pushing. We limit the artifacts that are saved and retrieved by jobs to the minimum in order to reduce the upload/download time and costs, as well as the artifacts storage. -### Git fetch caching +### Components caching -Because GitLab.com uses the [pack-objects cache](../administration/gitaly/configure_gitaly.md#pack-objects-cache), -concurrent Git fetches of the same pipeline ref are deduplicated on -the Gitaly server (always) and served from cache (when available). +Some external components (currently only GitLab Workhorse) of GitLab need to be built from source as a preliminary step for running tests. -This works well for the following reasons: +In [this MR](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/79766), we introduced a new `build-components` job that: -- The pack-objects cache is enabled on all Gitaly servers on GitLab.com. -- The CI/CD [Git strategy setting](../ci/pipelines/settings.md#choose-the-default-git-strategy) for `gitlab-org/gitlab` is **Git clone**, - causing all jobs to fetch the same data, which maximizes the cache hit ratio. -- We use [shallow clone](../ci/pipelines/settings.md#limit-the-number-of-changes-fetched-during-clone) to avoid downloading the full Git - history for every job. +- runs automatically for all GitLab.com `gitlab-org/gitlab` scheduled pipelines +- runs automatically for any `master` commit that touches the `workhorse/` folder +- is manual for GitLab.com's `gitlab-org`'s MRs + +This job tries to download a generic package that contains GitLab Workhorse binaries needed in the GitLab test suite (under `tmp/tests/gitlab-workhorse`). + +- If the package URL returns a 404: + 1. It runs `scripts/setup-test-env`, so that the GitLab Workhorse binaries are built. + 1. It then creates an archive which contains the binaries and upload it [as a generic package](https://gitlab.com/gitlab-org/gitlab/-/packages/). +- Otherwise, if the package already exists, it exit the job successfully. + +We also changed the `setup-test-env` job to: + +1. First download the GitLab Workhorse generic package build and uploaded by `build-components`. +1. If the package is retrieved successfully, its content is placed in the right folder (i.e. `tmp/tests/gitlab-workhorse`), preventing the building of the binaries when `scripts/setup-test-env` is run later on. +1. If the package URL returns a 404, the behavior doesn't change compared to the current one: the GitLab Workhorse binaries are built as part of `scripts/setup-test-env`. + +NOTE: +The version of the package is the workhorse tree SHA (i.e. `git rev-parse HEAD:workhorse`). ### Pre-clone step diff --git a/doc/development/product_qualified_lead_guide/index.md b/doc/development/product_qualified_lead_guide/index.md index f9d18bacecd..6943f931d79 100644 --- a/doc/development/product_qualified_lead_guide/index.md +++ b/doc/development/product_qualified_lead_guide/index.md @@ -35,6 +35,7 @@ The hand-raise lead form accepts the following parameters via provide or inject. ```javascript provide: { + small, user: { namespaceId, userName, @@ -43,9 +44,18 @@ The hand-raise lead form accepts the following parameters via provide or inject. companyName, glmContent, }, + ctaTracking: { + action, + label, + property, + value, + experiment, + }, }, ``` +The `ctaTracking` parameters follow [the `data-track` attributes](../snowplow/implementation.md#data-track-attributes) for implementing Snowplow tracking. The provided tracking attributes are attached to the button inside the `HandRaiseLeadButton` component, which triggers the hand-raise lead modal when selected. + ### Monitor the lead location When embedding a new hand raise form, use a unique `glmContent` or `glm_content` field that is different to any existing values. @@ -82,13 +92,80 @@ The flow of a PQL lead is as follows: 1. Marketo does scoring and sends the form to Salesforce. 1. Our Sales team uses Salesforce to connect to the leads. +### Trial lead flow + +#### Trial lead flow on GitLab.com + +```mermaid +sequenceDiagram + Trial Frontend Forms ->>TrialsController#create_lead: GitLab.com frontend sends [lead] to backend + TrialsController#create_lead->>CreateLeadService: [lead] + TrialsController#create_lead->>ApplyTrialService: [lead] Apply the trial + CreateLeadService->>SubscriptionPortalClient#generate_trial(sync_to_gl=false): [lead] Creates customer account on CustomersDot + ApplyTrialService->>SubscriptionPortalClient#generate_trial(sync_to_gl=true): [lead] Asks CustomersDot to apply the trial on namespace + SubscriptionPortalClient#generate_trial(sync_to_gl=false)->>CustomersDot|TrialsController#create(sync_to_gl=false): GitLab.com sends [lead] to CustomersDot + SubscriptionPortalClient#generate_trial(sync_to_gl=true)->>CustomersDot|TrialsController#create(sync_to_gl=true): GitLab.com asks CustomersDot to apply the trial + + +``` + +#### Trial lead flow on CustomersDot (sync_to_gl) + +```mermaid +sequenceDiagram + CustomersDot|TrialsController#create->>HostedPlans|CreateTrialService#execute: Save [lead] to leads table for monitoring purposes + HostedPlans|CreateTrialService#execute->>BaseTrialService#create_account: Creates a customer record in customers table + HostedPlans|CreateTrialService#create_platypus_lead->>PlatypusLogLeadService: Creates a platypus lead + HostedPlans|CreateTrialService#create_platypus_lead->>Platypus|CreateLeadWorker: Async worker to submit [lead] to Platypus + Platypus|CreateLeadWorker->>Platypus|CreateLeadService: [lead] + Platypus|CreateLeadService->>PlatypusApp#post: [lead] + PlatypusApp#post->>Platypus: [lead] is sent to Platypus +``` + +#### Applying the trial to a namespace on CustomersDot + +```mermaid +sequenceDiagram + HostedPlans|CreateTrialService->load_namespace#Gitlab api/namespaces: Load namespace details + HostedPlans|CreateTrialService->create_order#: Creates an order in orders table + HostedPlans|CreateTrialService->create_trial_history#: Creates a record in trial_histories table +``` + +### Hand raise lead flow + +#### Hand raise flow on GitLab.com + +```mermaid +sequenceDiagram + HandRaiseForm Vue Component->>TrialsController#create_hand_raise_lead: GitLab.com frontend sends [lead] to backend + TrialsController#create_hand_raise_lead->>CreateHandRaiseLeadService: [lead] + CreateHandRaiseLeadService->>SubscriptionPortalClient: [lead] + SubscriptionPortalClient->>CustomersDot|TrialsController#create_hand_raise_lead: GitLab.com sends [lead] to CustomersDot +``` + +#### Hand raise flow on CustomersDot + +```mermaid +sequenceDiagram + CustomersDot|TrialsController#create_hand_raise_lead->>PlatypusLogLeadService: Save [lead] to leads table for monitoring purposes + CustomersDot|TrialsController#create_hand_raise_lead->>Platypus|CreateLeadWorker: Async worker to submit [lead] to Platypus + Platypus|CreateLeadWorker->>Platypus|CreateLeadService: [lead] + Platypus|CreateLeadService->>PlatypusApp#post: [lead] + PlatypusApp#post->>Platypus: [lead] is sent to Platypus +``` + +### PQL flow after Platypus for all lead types + +```mermaid +sequenceDiagram + Platypus->>Workato: [lead] + Workato->>Marketo: [lead] + Marketo->>Salesforce(SFDC): [lead] +``` + ## Monitor and manually test leads - Check the application and Sidekiq logs on `gitlab.com` and CustomersDot to monitor leads. - Check the `leads` table in CustomersDot. - Set up staging credentials for Platypus, and track the leads on the [Platypus Dashboard](https://staging.ci.nexus.gitlabenvironment.cloud/admin/queues/queue/new-lead-queue). - Ask for access to the Marketo Sandbox and validate the leads there. - -## Trials - -Trials follow the same flow as the PQL leads. diff --git a/doc/development/profiling.md b/doc/development/profiling.md index deb743569c5..a3142b06e12 100644 --- a/doc/development/profiling.md +++ b/doc/development/profiling.md @@ -91,6 +91,73 @@ printer = RubyProf::CallStackPrinter.new(result) printer.print(File.open('/tmp/profile.html', 'w')) ``` +### Stackprof support + +By default, `Gitlab::Profiler.profile` uses a tracing profiler called [`ruby-prof`](https://ruby-prof.github.io/). However, sampling profilers +[run faster and use less memory](https://jvns.ca/blog/2017/12/17/how-do-ruby---python-profilers-work-/), so they might be preferred. + +You can switch to [Stackprof](https://github.com/tmm1/stackprof) (a sampling profiler) to generate a profile by passing `sampling_mode: true`. +Pass in a `profiler_options` hash to configure the output file (`out`) of the sampling data. For example: + +```ruby +Gitlab::Profiler.profile('/gitlab-org/gitlab-test', user: User.first, sampling_mode: true, profiler_options: { out: 'tmp/profile.dump' }) +``` + +You can get a summary of where time was spent by running Stackprof against the sampling data. For example: + +```shell +stackprof tmp/profile.dump +``` + +Example sampling data: + +```plaintext +================================== + Mode: wall(1000) + Samples: 8745 (6.92% miss rate) + GC: 1399 (16.00%) +================================== + TOTAL (pct) SAMPLES (pct) FRAME + 1022 (11.7%) 1022 (11.7%) Sprockets::PathUtils#stat + 957 (10.9%) 957 (10.9%) (marking) + 493 (5.6%) 493 (5.6%) Sprockets::PathUtils#entries + 576 (6.6%) 471 (5.4%) Mustermann::AST::Translator#decorator_for + 439 (5.0%) 439 (5.0%) (sweeping) + 630 (7.2%) 241 (2.8%) Sprockets::Cache::FileStore#get + 208 (2.4%) 208 (2.4%) ActiveSupport::FileUpdateChecker#watched + 206 (2.4%) 206 (2.4%) Digest::Instance#file + 544 (6.2%) 176 (2.0%) Sprockets::Cache::FileStore#safe_open + 176 (2.0%) 176 (2.0%) ActiveSupport::FileUpdateChecker#max_mtime + 268 (3.1%) 147 (1.7%) ActiveRecord::ConnectionAdapters::PostgreSQLAdapter#exec_no_cache + 140 (1.6%) 140 (1.6%) ActiveSupport::BacktraceCleaner#add_gem_filter + 116 (1.3%) 116 (1.3%) Bootsnap::CompileCache::ISeq.storage_to_output + 160 (1.8%) 113 (1.3%) Gem::Version#<=> + 109 (1.2%) 109 (1.2%) block in <main> + 108 (1.2%) 108 (1.2%) Gem::Version.new + 131 (1.5%) 105 (1.2%) Sprockets::EncodingUtils#unmarshaled_deflated + 1166 (13.3%) 82 (0.9%) Mustermann::RegexpBased#initialize + 82 (0.9%) 78 (0.9%) FileUtils.touch + 72 (0.8%) 72 (0.8%) Sprockets::Manifest.compile_match_filter + 71 (0.8%) 70 (0.8%) Grape::Router#compile! + 91 (1.0%) 65 (0.7%) ActiveRecord::ConnectionAdapters::PostgreSQL::DatabaseStatements#query + 93 (1.1%) 64 (0.7%) ActionDispatch::Journey::Path::Pattern::AnchoredRegexp#accept + 59 (0.7%) 59 (0.7%) Mustermann::AST::Translator.dispatch_table + 62 (0.7%) 59 (0.7%) Rails::BacktraceCleaner#initialize + 2492 (28.5%) 49 (0.6%) Sprockets::PathUtils#stat_directory + 242 (2.8%) 49 (0.6%) Gitlab::Instrumentation::RedisBase.add_call_details + 47 (0.5%) 47 (0.5%) URI::RFC2396_Parser#escape + 46 (0.5%) 46 (0.5%) #<Class:0x00000001090c2e70>#__setobj__ + 44 (0.5%) 44 (0.5%) Sprockets::Base#normalize_logical_path +``` + +You can also generate flamegraphs: + +```shell +stackprof --d3-flamegraph tmp/profile.dump > flamegraph.html +``` + +See [the Stackprof documentation](https://github.com/tmm1/stackprof) for more details. + ## Speedscope flamegraphs You can generate a flamegraph for a particular URL by selecting a flamegraph sampling mode button in the performance bar or by adding the `performance_bar=flamegraph` parameter to the request. diff --git a/doc/development/rake_tasks.md b/doc/development/rake_tasks.md index 5c8e2d5fc55..1e9367ecee4 100644 --- a/doc/development/rake_tasks.md +++ b/doc/development/rake_tasks.md @@ -196,6 +196,16 @@ One way to generate the initial list is to run the Rake task `rubocop:todo:gener bundle exec rake rubocop:todo:generate ``` +To generate TODO list for specific RuboCop rules, pass them comma-seperated as +argument to the Rake task: + +```shell +bundle exec rake 'rubocop:todo:generate[Gitlab/NamespacedClass,Lint/Syntax]' +bundle exec rake rubocop:todo:generate\[Gitlab/NamespacedClass,Lint/Syntax\] +``` + +Some shells require brackets to be escaped or quoted. + See [Resolving RuboCop exceptions](contributing/style_guides.md#resolving-rubocop-exceptions) on how to proceed from here. @@ -219,14 +229,14 @@ To update the Emoji aliases file (used for Emoji autocomplete), run the following: ```shell -bundle exec rake gemojione:aliases +bundle exec rake tanuki_emoji:aliases ``` To update the Emoji digests file (used for Emoji autocomplete), run the following: ```shell -bundle exec rake gemojione:digests +bundle exec rake tanuki_emoji:digests ``` This updates the file `fixtures/emojis/digests.json` based on the currently @@ -235,7 +245,7 @@ available Emoji. To generate a sprite file containing all the Emoji, run: ```shell -bundle exec rake gemojione:sprite +bundle exec rake tanuki_emoji:sprite ``` If new emoji are added, the sprite sheet may change size. To compensate for diff --git a/doc/development/redis/new_redis_instance.md b/doc/development/redis/new_redis_instance.md index dcd79be0e5c..96f860f3890 100644 --- a/doc/development/redis/new_redis_instance.md +++ b/doc/development/redis/new_redis_instance.md @@ -112,7 +112,7 @@ while and there are no issues, we can proceed. ### Proposed solution: Migrate data by using MultiStore with the fallback strategy -We need a way to migrate users to a new Redis store without causing any inconveniences from UX perspective. +We need a way to migrate users to a new Redis store without causing any inconveniences from UX perspective. We also want the ability to fall back to the "old" Redis instance if something goes wrong with the new instance. Migration Requirements: @@ -129,13 +129,13 @@ We need to write data into both Redis instances (old + new). We read from the new instance, but we need to fall back to the old instance when pre-fetching from the new dedicated Redis instance that failed. We need to log any issues or exceptions with a new instance, but still fall back to the old instance. -The proposed migration strategy is to implement and use the [MultiStore](https://gitlab.com/gitlab-org/gitlab/-/blob/fcc42e80ed261a862ee6ca46b182eee293ae60b6/lib/gitlab/redis/multi_store.rb). -We used this approach with [adding new dedicated Redis instance for session keys](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/579). +The proposed migration strategy is to implement and use the [MultiStore](https://gitlab.com/gitlab-org/gitlab/-/blob/fcc42e80ed261a862ee6ca46b182eee293ae60b6/lib/gitlab/redis/multi_store.rb). +We used this approach with [adding new dedicated Redis instance for session keys](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/579). Also MultiStore comes with corresponding [specs](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/lib/gitlab/redis/multi_store_spec.rb). The MultiStore looks like a `redis-rb ::Redis` instance. -In the new Redis instance class you added in [Step 1](#step-1-support-configuring-the-new-instance), +In the new Redis instance class you added in [Step 1](#step-1-support-configuring-the-new-instance), override the [Redis](https://gitlab.com/gitlab-org/gitlab/-/blob/fcc42e80ed261a862ee6ca46b182eee293ae60b6/lib/gitlab/redis/sessions.rb#L20-28) method from the `::Gitlab::Redis::Wrapper` ```ruby @@ -177,7 +177,7 @@ bin/feature-flag use_primary_store_as_default_for_foo ``` By enabling `use_primary_and_secondary_stores_for_foo` feature flag, our `Gitlab::Redis::Foo` will use `MultiStore` to write to both new Redis instance -and the [old (fallback-instance)](#fallback-instance). +and the [old (fallback-instance)](#fallback-instance). If we fail to fetch data from the new instance, we will fallback and read from the old Redis instance. We can monitor logs for `Gitlab::Redis::MultiStore::ReadFromPrimaryError`, and also the Prometheus counter `gitlab_redis_multi_store_read_fallback_total`. @@ -218,7 +218,7 @@ When a command outside of the supported list is used, `method_missing` will pass This ensures that anything unexpected behaves like it would before. NOTE: -By tracking `gitlab_redis_multi_store_method_missing_total` counter and `Gitlab::Redis::MultiStore::MethodMissingError`, +By tracking `gitlab_redis_multi_store_method_missing_total` counter and `Gitlab::Redis::MultiStore::MethodMissingError`, a developer will need to add an implementation for missing Redis commands before proceeding with the migration. ##### Errors diff --git a/doc/development/secure_coding_guidelines.md b/doc/development/secure_coding_guidelines.md index d34b12c6361..10f6c22e54a 100644 --- a/doc/development/secure_coding_guidelines.md +++ b/doc/development/secure_coding_guidelines.md @@ -253,12 +253,27 @@ the mitigations for a new feature. - [More details](https://dev.gitlab.org/gitlab/gitlabhq/-/merge_requests/2530/diffs) -#### Feature-specific mitigations +#### URL blocker & validation libraries + +[`Gitlab::UrlBlocker`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/url_blocker.rb) can be used to validate that a +provided URL meets a set of constraints. Importantly, when `dns_rebind_protection` is `true`, the method returns a known-safe URI where the hostname +has been replaced with an IP address. This prevents DNS rebinding attacks, because the DNS record has been resolved. However, if we ignore this returned +value, we **will not** be protected against DNS rebinding. -For situations in which an allowlist or GitLab:HTTP cannot be used, it will be necessary to implement mitigations directly in the feature. It is best to validate the destination IP addresses themselves, not just domain names, as DNS can be controlled by the attacker. Below are a list of mitigations that should be implemented. +This is the case with validators such as the `AddressableUrlValidator` (called with `validates :url, addressable_url: {opts}` or `public_url: {opts}`). +Validation errors are only raised when validations are called, for example when a record is created or saved. If we ignore the value returned by the validation +when persisting the record, **we need to recheck** its validity before using it. You can learn more about [Time of Check to Time of Use bugs](#time-of-check-to-time-of-use-bugs) in a later section +of these guidelines. + +#### Feature-specific mitigations There are many tricks to bypass common SSRF validations. If feature-specific mitigations are necessary, they should be reviewed by the AppSec team, or a developer who has worked on SSRF mitigations previously. +For situations in which you can't use an allowlist or GitLab:HTTP, you must implement mitigations +directly in the feature. It's best to validate the destination IP addresses themselves, not just +domain names, as the attacker can control DNS. Below is a list of mitigations that you should +implement. + - Block connections to all localhost addresses - `127.0.0.1/8` (IPv4 - note the subnet mask) - `::1` (IPv6) @@ -270,9 +285,36 @@ There are many tricks to bypass common SSRF validations. If feature-specific mit - `169.254.0.0/16` - In particular, for GCP: `metadata.google.internal` -> `169.254.169.254` - For HTTP connections: Disable redirects or validate the redirect destination -- To mitigate DNS rebinding attacks, validate and use the first IP address received +- To mitigate DNS rebinding attacks, validate and use the first IP address received. + +See [`url_blocker_spec.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/lib/gitlab/url_blocker_spec.rb) for examples of SSRF payloads. See [time of check to time of use bugs](#time-of-check-to-time-of-use-bugs) to learn more about DNS rebinding's class of bug. -See [`url_blocker_spec.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/lib/gitlab/url_blocker_spec.rb) for examples of SSRF payloads +Don't rely on methods like `.start_with?` when validating a URL, or make assumptions about which +part of a string maps to which part of a URL. Use the `URI` class to parse the string, and validate +each component (scheme, host, port, path, and so on). Attackers can create valid URLs which look +safe, but lead to malicious locations. + +```ruby +user_supplied_url = "https://my-safe-site.com@my-evil-site.com" # Content before an @ in a URL is usually for basic authentication +user_supplied_url.start_with?("https://my-safe-site.com") # Don't trust with start_with? for URLs! +=> true +URI.parse(user_supplied_url).host +=> "my-evil-site.com" + +user_supplied_url = "https://my-safe-site.com-my-evil-site.com" +user_supplied_url.start_with?("https://my-safe-site.com") # Don't trust with start_with? for URLs! +=> true +URI.parse(user_supplied_url).host +=> "my-safe-site.com-my-evil-site.com" + +# Here's an example where we unsafely attempt to validate a host while allowing for +# subdomains +user_supplied_url = "https://my-evil-site-my-safe-site.com" +user_supplied_host = URI.parse(user_supplied_url).host +=> "my-evil-site-my-safe-site.com" +user_supplied_host.end_with?("my-safe-site.com") # Don't trust with end_with? +=> true +``` ## XSS guidelines @@ -448,8 +490,7 @@ parameter when using `check_allowed_absolute_path!()`. To use a combination of both checks, follow the example below: ```ruby -path = Gitlab::Utils.check_path_traversal!(path) -Gitlab::Utils.check_allowed_absolute_path!(path, path_allowlist) +Gitlab::Utils.check_allowed_absolute_path_and_path_traversal!(path, path_allowlist) ``` In the REST API, we have the [`FilePath`](https://gitlab.com/gitlab-org/security/gitlab/-/blob/master/lib/api/validations/validators/file_path.rb) @@ -1120,3 +1161,52 @@ func printZipContents(src string) error { return nil } ``` + +## Time of check to time of use bugs + +Time of check to time of use, or TOCTOU, is a class of error which occur when the state of something changes unexpectedly partway during a process. +More specifically, it's when the property you checked and validated has changed when you finally get around to using that property. + +These types of bugs are often seen in environments which allow multi-threading and concurrency, like filesystems and distributed web applications; these are a type of race condition. TOCTOU also occurs when state is checked and stored, then after a period of time that state is relied on without re-checking its accuracy and/or validity. + +### Examples + +**Example 1:** you have a model which accepts a URL as input. When the model is created you verify that the URL's host resolves to a public IP address, to prevent attackers making internal network calls. But DNS records can change ([DNS rebinding](#server-side-request-forgery-ssrf)]). An attacker updates the DNS record to `127.0.0.1`, and when your code resolves those URL's host it results in sending a potentially malicious request to a server on the internal network. The property was valid at the "time of check", but invalid and malicious at "time of use". + +GitLab-specific example can be found in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/214401) where, although `Gitlab::UrlBlocker.validate!` was called, the returned value was not used. This made it vulnerable to TOCTOU bug and SSRF protection bypass through [DNS rebinding](#server-side-request-forgery-ssrf). The fix was to [use the validated IP address](https://gitlab.com/gitlab-org/gitlab/-/commit/7af8abd4df9a98f7a1ae7c4ec9840d0a7a8c684d). + +**Example 2:** you have a feature which schedules jobs. When the user schedules the job, they have permission to do so. But imagine if, between the time they schedule the job and the time it is run, their permissions are restricted. Unless you re-check permissions at time of use, you could inadvertently allow unauthorized activity. + +**Example 3:** you need to fetch a remote file, and perform a `HEAD` request to get and validate the content length and content type. When you subsequently make a `GET` request, though, the file delivered is a different size or different file type. (This is stretching the definition of TOCTOU, but things _have_ changed between time of check and time of use). + +**Example 4:** you allow users to upvote a comment if they haven't already. The server is multi-threaded, and you aren't using transactions or an applicable database index. By repeatedly clicking upvote in quick succession a malicious user is able to add multiple upvotes: the requests arrive at the same time, the checks run in parallel and confirm that no upvote exists yet, and so each upvote is written to the database. + +Here's some pseudocode showing an example of a potential TOCTOU bug: + +```ruby +def upvote(comment, user) + # The time between calling .exists? and .create can lead to TOCTOU, + # particularly if .create is a slow method, or runs in a background job + if Upvote.exists?(comment: comment, user: user) + return + else + Upvote.create(comment: comment, user: user) + end +end +``` + +### Prevention & defense + +- Assume values will change between the time you validate them and the time you use them. +- Perform checks as close to execution time as possible. +- Perform checks after your operation completes. +- Use your framework's validations and database features to impose constraints and atomic reads and writes. +- Read about [Server Side Request Forgery (SSRF) and DNS rebinding](#server-side-request-forgery-ssrf) + +An example of well implemented `Gitlab::UrlBlocker.validate!` call that prevents TOCTOU bug: + +1. [Preventing DNS rebinding in Gitea importer](https://gitlab.com/gitlab-org/gitlab/-/commit/7af8abd4df9a98f7a1ae7c4ec9840d0a7a8c684d) + +### Resources + +- [CWE-367: Time-of-check Time-of-use (TOCTOU) Race Condition](https://cwe.mitre.org/data/definitions/367.html) diff --git a/doc/development/service_ping/implement.md b/doc/development/service_ping/implement.md index 3a1e4c6d87b..25e841e113b 100644 --- a/doc/development/service_ping/implement.md +++ b/doc/development/service_ping/implement.md @@ -16,7 +16,7 @@ Service Ping consists of two kinds of data: To implement a new metric in Service Ping, follow these steps: 1. [Implement the required counter](#types-of-counters) -1. [Name and place the metric](#name-and-place-the-metric) +1. [Name and place the metric](metrics_dictionary.md#metric-key_path) 1. [Test counters manually using your Rails console](#test-counters-manually-using-your-rails-console) 1. [Generate the SQL query](#generate-the-sql-query) 1. [Optimize queries with `#database-lab`](#optimize-queries-with-database-lab) @@ -26,13 +26,12 @@ To implement a new metric in Service Ping, follow these steps: 1. [Verify your metric](#verify-your-metric) 1. [Set up and test Service Ping locally](#set-up-and-test-service-ping-locally) -NOTE: -When you add or change a Service Metric, you must migrate metrics to [instrumentation classes](metrics_instrumentation.md). -For information about the progress on migrating Service ping metrics, see this [epic](https://gitlab.com/groups/gitlab-org/-/epics/5547). - ## Instrumentation classes -We recommend you use [instrumentation classes](metrics_instrumentation.md) in `usage_data.rb` where possible. +NOTE: +Implementing metrics directly in `usage_data.rb` is deprecated. +When you add or change a Service Ping Metric, you must migrate metrics to [instrumentation classes](metrics_instrumentation.md). +For information about the progress on migrating Service Ping metrics, see this [epic](https://gitlab.com/groups/gitlab-org/-/epics/5547). For example, we have the following instrumentation class: `lib/gitlab/usage/metrics/instrumentations/count_boards_metric.rb`. @@ -45,7 +44,7 @@ boards: add_metric('CountBoardsMetric', time_frame: 'all'), ## Types of counters -There are several types of counters in `usage_data.rb`: +There are several types of counters for metrics: - **[Batch counters](#batch-counters)**: Used for counts and sums. - **[Redis counters](#redis-counters):** Used for in-memory counts. @@ -72,64 +71,39 @@ you must add a specialized index on the columns involved in a counter. #### Ordinary batch counters -Simple count of a given `ActiveRecord_Relation`, does a non-distinct batch count, smartly reduces `batch_size`, and handles errors. -Handles the `ActiveRecord::StatementInvalid` error. +Create a new [database metrics](metrics_instrumentation.md#database-metrics) instrumentation class with `count` operation for a given `ActiveRecord_Relation` Method: ```ruby -count(relation, column = nil, batch: true, start: nil, finish: nil) +add_metric('CountIssuesMetric', time_frame: 'all') ``` -Arguments: - -- `relation` the ActiveRecord_Relation to perform the count -- `column` the column to perform the count on, by default is the primary key -- `batch`: default `true` to use batch counting -- `start`: custom start of the batch counting to avoid complex min calculations -- `end`: custom end of the batch counting to avoid complex min calculations - Examples: -```ruby -count(User.active) -count(::Clusters::Cluster.aws_installed.enabled, :cluster_id) -count(::Clusters::Cluster.aws_installed.enabled, :cluster_id, start: ::Clusters::Cluster.minimum(:id), finish: ::Clusters::Cluster.maximum(:id)) -``` +Examples using `usage_data.rb` have been [deprecated](usage_data.md). We recommend to use [instrumentation classes](metrics_instrumentation.md). #### Distinct batch counters -Distinct count of a given `ActiveRecord_Relation` on given column, a distinct batch count, smartly reduces `batch_size`, and handles errors. -Handles the `ActiveRecord::StatementInvalid` error. +Create a new [database metrics](metrics_instrumentation.md#database-metrics) instrumentation class with `distinct_count` operation for a given `ActiveRecord_Relation`. Method: ```ruby -distinct_count(relation, column = nil, batch: true, batch_size: nil, start: nil, finish: nil) +add_metric('CountUsersAssociatingMilestonesToReleasesMetric', time_frame: 'all') ``` -Arguments: - -- `relation`: the ActiveRecord_Relation to perform the count -- `column`: the column to perform the distinct count, by default is the primary key -- `batch`: default `true` to use batch counting -- `batch_size`: if none set it uses default value 10000 from `Gitlab::Database::BatchCounter` -- `start`: custom start of the batch counting to avoid complex min calculations -- `end`: custom end of the batch counting to avoid complex min calculations - WARNING: Counting over non-unique columns can lead to performance issues. For more information, see the [iterating tables in batches](../iterating_tables_in_batches.md) guide. Examples: -```ruby -distinct_count(::Project, :creator_id) -distinct_count(::Note.with_suggestions.where(time_period), :author_id, start: ::User.minimum(:id), finish: ::User.maximum(:id)) -distinct_count(::Clusters::Applications::CertManager.where(time_period).available.joins(:cluster), 'clusters.user_id') -``` +Examples using `usage_data.rb` have been [deprecated](usage_data.md). We recommend to use [instrumentation classes](metrics_instrumentation.md). #### Sum batch operation +There is no support for `sum` for database metrics. + Sum the values of a given ActiveRecord_Relation on given column and handles errors. Handles the `ActiveRecord::StatementInvalid` error @@ -686,29 +660,6 @@ We return fallback values in these cases: | Timeouts, general failures | -1 | | Standard errors in counters | -2 | -## Name and place the metric - -Add the metric in one of the top-level keys: - -- `settings`: for settings related metrics. -- `counts_weekly`: for counters that have data for the most recent 7 days. -- `counts_monthly`: for counters that have data for the most recent 28 days. -- `counts`: for counters that have data for all time. - -### How to get a metric name suggestion - -The metric YAML generator can suggest a metric name for you. -To generate a metric name suggestion, first instrument the metric at the provided `key_path`. -Then, generate the metric's YAML definition and -return to the instrumentation and update it. - -1. Add the metric instrumentation to `lib/gitlab/usage_data.rb` inside one - of the [top-level keys](#name-and-place-the-metric), using any name you choose. -1. Run the [metrics YAML generator](metrics_dictionary.md#metrics-definition-and-validation). -1. Use the metric name suggestion to select a suitable metric name. -1. Update the instrumentation you created in the first step and change the metric name to the suggested name. -1. Update the metric's YAML definition with the correct `key_path`. - ## Test counters manually using your Rails console ```ruby diff --git a/doc/development/service_ping/index.md b/doc/development/service_ping/index.md index 86e70cc8bbc..6878fd1bf28 100644 --- a/doc/development/service_ping/index.md +++ b/doc/development/service_ping/index.md @@ -6,7 +6,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Service Ping Guide **(FREE SELF)** -> Introduced in GitLab Ultimate 11.2, more statistics. +> - Introduced in GitLab Ultimate 11.2, more statistics. +> - In GitLab 14.1, [renamed from Usage Ping to Service Ping](https://gitlab.com/groups/gitlab-org/-/epics/5990). In 14.0 and earlier, use the Usage Ping documentation for the Rails commands appropriate to your version. Service Ping is a GitLab process that collects and sends a weekly payload to GitLab. The payload provides important high-level data that helps our product, support, @@ -68,7 +69,10 @@ Because of these limitations we recommend you: > Introduced in GitLab 14.1. -In GitLab versions 14.1 and later, free self-managed users running [GitLab EE](../ee_features.md) can receive paid features by registering with GitLab and sending us activity data through Service Ping. Features introduced here do not remove the feature from its paid tier. Users can continue to access the features in a paid tier without sharing usage data. +In GitLab versions 14.1 and later, GitLab Free customers with a self-managed instance running +[GitLab EE](../ee_features.md) can receive paid features by registering with GitLab and sending us +activity data through Service Ping. Features introduced here do not remove the feature from its paid +tier. Users can continue to access the features in a paid tier without sharing usage data. #### Features available in 14.1 and later @@ -209,17 +213,17 @@ sequenceDiagram - `uuid` - GitLab instance unique identifier - `hostname` - GitLab instance hostname -- `version` - GitLab instance current versions +- `version` - GitLab instance current versions - `elapsed` - Amount of time which passed since Service Ping report process started and moment of error occurrence - `message` - Error message <pre> <code> { - "uuid"=>"02333324-1cd7-4c3b-a45b-a4993f05fb1d", - "hostname"=>"127.0.0.1", - "version"=>"14.7.0-pre", - "elapsed"=>0.006946, + "uuid"=>"02333324-1cd7-4c3b-a45b-a4993f05fb1d", + "hostname"=>"127.0.0.1", + "version"=>"14.7.0-pre", + "elapsed"=>0.006946, "message"=>'PG::UndefinedColumn: ERROR: column \"non_existent_attribute\" does not exist\nLINE 1: SELECT COUNT(non_existent_attribute) FROM \"issues\" /*applica...' } </code> @@ -576,7 +580,7 @@ skip_db_write: ServicePing::SubmitService.new(skip_db_write: true).execute ``` -## Manually upload Service Ping payload +## Manually upload Service Ping payload > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/7388) in GitLab 14.8 with a flag named `admin_application_settings_service_usage_data_center`. Disabled by default. @@ -596,7 +600,7 @@ To upload payload manually: ## Monitoring -Service Ping reporting process state is monitored with [internal SiSense dashboard](https://app.periscopedata.com/app/gitlab/968489/Product-Intelligence---Service-Ping-Health). +Service Ping reporting process state is monitored with [internal SiSense dashboard](https://app.periscopedata.com/app/gitlab/968489/Product-Intelligence---Service-Ping-Health). ## Troubleshooting diff --git a/doc/development/service_ping/metrics_dictionary.md b/doc/development/service_ping/metrics_dictionary.md index 93eec4efabd..6884844da3f 100644 --- a/doc/development/service_ping/metrics_dictionary.md +++ b/doc/development/service_ping/metrics_dictionary.md @@ -47,13 +47,58 @@ Each metric is defined in a separate YAML file consisting of a number of fields: | `distribution` | yes | `array`; may be set to one of `ce, ee` or `ee`. The [distribution](https://about.gitlab.com/handbook/marketing/strategic-marketing/tiers/#definitions) where the tracked feature is available. | | `performance_indicator_type` | no | `array`; may be set to one of [`gmau`, `smau`, `paid_gmau`, or `umau`](https://about.gitlab.com/handbook/business-technology/data-team/data-catalog/xmau-analysis/). | | `tier` | yes | `array`; may contain one or a combination of `free`, `premium` or `ultimate`. The [tier]( https://about.gitlab.com/handbook/marketing/strategic-marketing/tiers/) where the tracked feature is available. This should be verbose and contain all tiers where a metric is available. | -| `milestone` | no | The milestone when the metric is introduced. | +| `milestone` | no | The milestone when the metric is introduced and when it's available to self-managed instances with the official GitLab release. | | `milestone_removed` | no | The milestone when the metric is removed. | -| `introduced_by_url` | no | The URL to the merge request that introduced the metric. | +| `introduced_by_url` | no | The URL to the merge request that introduced the metric to be available for self-managed instances. | | `repair_issue_url` | no | The URL of the issue that was created to repair a metric with a `broken` status. | | `options` | no | `object`: options information needed to calculate the metric value. | | `skip_validation` | no | This should **not** be set. [Used for imported metrics until we review, update and make them valid](https://gitlab.com/groups/gitlab-org/-/epics/5425). | +### Metric key_path + +The `key_path` of the metric is the location in the JSON Service Ping payload. + +The `key_path` could be composed from multiple parts separated by `.` and it must be unique. + +We recommend to add the metric in one of the top-level keys: + +- `settings`: for settings related metrics. +- `counts_weekly`: for counters that have data for the most recent 7 days. +- `counts_monthly`: for counters that have data for the most recent 28 days. +- `counts`: for counters that have data for all time. + +NOTE: +We can't control what the metric's `key_path` is, because some of them are generated dynamically in `usage_data.rb`. +For example, see [Redis HLL metrics](implement.md#redis-hll-counters). + +### Metric name + +To improve metric discoverability by a wider audience, each metric with +instrumentation added at an appointed `key_path` receives a `name` attribute +filled with the name suggestion, corresponding to the metric `data_source` and instrumentation. +Metric name suggestions can contain two types of elements: + +1. **User input prompts**: enclosed by angle brackets (`< >`), these pieces should be replaced or + removed when you create a metrics YAML file. +1. **Fixed suggestion**: plaintext parts generated according to well-defined algorithms. + They are based on underlying instrumentation, and must not be changed. + +For a metric name to be valid, it must not include any prompt, and fixed suggestions +must not be changed. + +#### Generate a metric name suggestion + +The metric YAML generator can suggest a metric name for you. +To generate a metric name suggestion, first instrument the metric at the provided `key_path`. +Then, generate the metric's YAML definition and +return to the instrumentation and update it. + +1. Add the metric instrumentation class to `lib/gitlab/usage/metrics/instrumentations/`. +1. Add the metric logic in the instrumentation class. +1. Run the [metrics YAML generator](metrics_dictionary.md#metrics-definition-and-validation). +1. Use the metric name suggestion to select a suitable metric name. +1. Update the metric's YAML definition with the correct `key_path`. + ### Metric statuses Metric definitions can have one of the following statuses: @@ -81,21 +126,6 @@ which has a related schema in `/config/metrics/objects_schemas/topology_schema.j - `all`: The metric data applies for the whole time the metric has been active (all-time interval). For example, the following metric counts all users that create issues: `/config/metrics/counts_all/20210216181115_issues.yml`. - `none`: The metric collects a type of data that's not tracked over time, such as settings and configuration information. Therefore, a time interval is not applicable. For example, `uuid` has no time interval applicable: `config/metrics/license/20210201124933_uuid.yml`. -### Metric name - -To improve metric discoverability by a wider audience, each metric with -instrumentation added at an appointed `key_path` receives a `name` attribute -filled with the name suggestion, corresponding to the metric `data_source` and instrumentation. -Metric name suggestions can contain two types of elements: - -1. **User input prompts**: Enclosed by `<>`, these pieces should be replaced or - removed when you create a metrics YAML file. -1. **Fixed suggestion**: Plaintext parts generated according to well-defined algorithms. - They are based on underlying instrumentation, and should not be changed. - -For a metric name to be valid, it must not include any prompt, and no fixed suggestions -should be changed. - ### Data category We use the following categories to classify a metric: diff --git a/doc/development/service_ping/metrics_instrumentation.md b/doc/development/service_ping/metrics_instrumentation.md index c98b0df92aa..c684d9d12ef 100644 --- a/doc/development/service_ping/metrics_instrumentation.md +++ b/doc/development/service_ping/metrics_instrumentation.md @@ -57,6 +57,50 @@ module Gitlab end ``` +### Ordinary batch counters Example + +```ruby +module Gitlab + module Usage + module Metrics + module Instrumentations + class CountIssuesMetric < DatabaseMetric + operation :count + + start { Issue.minimum(:id) } + finish { Issue.maximum(:id) } + + relation { Issue } + end + end + end + end +end +``` + +### Distinct batch counters Example + +```ruby +# frozen_string_literal: true + +module Gitlab + module Usage + module Metrics + module Instrumentations + class CountUsersAssociatingMilestonesToReleasesMetric < DatabaseMetric + operation :distinct_count, column: :author_id + + relation { Release.with_milestones } + + start { Release.minimum(:author_id) } + finish { Release.maximum(:author_id) } + end + end + end + end +end +``` + ## Redis metrics [Example of a merge request that adds a `Redis` metric](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/66582). diff --git a/doc/development/service_ping/review_guidelines.md b/doc/development/service_ping/review_guidelines.md index 137e11608cf..ee2d8f4f4a1 100644 --- a/doc/development/service_ping/review_guidelines.md +++ b/doc/development/service_ping/review_guidelines.md @@ -51,12 +51,13 @@ are regular backend changes. #### The Product Intelligence **reviewer** should - Perform a first-pass review on the merge request and suggest improvements to the author. -- Check the [metrics location](implement.md#name-and-place-the-metric) in +- Check the [metrics location](metrics_dictionary.md#metric-key_path) in the Service Ping JSON payload. -- Suggest that the author checks the [naming suggestion](implement.md#how-to-get-a-metric-name-suggestion) while +- Suggest that the author checks the [naming suggestion](metrics_dictionary.md#generate-a-metric-name-suggestion) while generating the metric's YAML definition. - Add the `~database` label and ask for a [database review](../database_review.md) for metrics that are based on Database. +- Add `~Data Warehouse::Impact Check` for any database metric that has a query change. Changes in queries can affect [data operations](https://about.gitlab.com/handbook/business-technology/data-team/how-we-work/triage/#gitlabcom-db-structure-changes). - For tracking using Redis HLL (HyperLogLog): - Check the Redis slot. - Check if a [feature flag is needed](implement.md#recommendations). diff --git a/doc/development/service_ping/usage_data.md b/doc/development/service_ping/usage_data.md new file mode 100644 index 00000000000..a25ad5f62be --- /dev/null +++ b/doc/development/service_ping/usage_data.md @@ -0,0 +1,70 @@ +--- +stage: Growth +group: Product Intelligence +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Usage Data Metrics guide + +This guide describes deprecated usage for metrics in `usage_data.rb`. + +NOTE: +Implementing metrics direct in `usage_data.rb` is deprecated, We recommend you use [instrumentation classes](metrics_instrumentation.md). + +## Ordinary batch counters + +Simple count of a given `ActiveRecord_Relation`, does a non-distinct batch count, smartly reduces `batch_size`, and handles errors. +Handles the `ActiveRecord::StatementInvalid` error. + +Method: + +```ruby +count(relation, column = nil, batch: true, start: nil, finish: nil) +``` + +Arguments: + +- `relation` the ActiveRecord_Relation to perform the count +- `column` the column to perform the count on, by default is the primary key +- `batch`: default `true` to use batch counting +- `start`: custom start of the batch counting to avoid complex min calculations +- `end`: custom end of the batch counting to avoid complex min calculations + +Examples: + +```ruby +count(User.active) +count(::Clusters::Cluster.aws_installed.enabled, :cluster_id) +count(::Clusters::Cluster.aws_installed.enabled, :cluster_id, start: ::Clusters::Cluster.minimum(:id), finish: ::Clusters::Cluster.maximum(:id)) +``` + +## Distinct batch counters + +Distinct count of a given `ActiveRecord_Relation` on given column, a distinct batch count, smartly reduces `batch_size`, and handles errors. +Handles the `ActiveRecord::StatementInvalid` error. + +Method: + +```ruby +distinct_count(relation, column = nil, batch: true, batch_size: nil, start: nil, finish: nil) +``` + +Arguments: + +- `relation`: the ActiveRecord_Relation to perform the count +- `column`: the column to perform the distinct count, by default is the primary key +- `batch`: default `true` to use batch counting +- `batch_size`: if none set it uses default value 10000 from `Gitlab::Database::BatchCounter` +- `start`: custom start of the batch counting to avoid complex min calculations +- `end`: custom end of the batch counting to avoid complex min calculations + +WARNING: +Counting over non-unique columns can lead to performance issues. For more information, see the [iterating tables in batches](../iterating_tables_in_batches.md) guide. + +Examples: + +```ruby +distinct_count(::Project, :creator_id) +distinct_count(::Note.with_suggestions.where(time_period), :author_id, start: ::User.minimum(:id), finish: ::User.maximum(:id)) +distinct_count(::Clusters::Applications::CertManager.where(time_period).available.joins(:cluster), 'clusters.user_id') +``` diff --git a/doc/development/shared_files.md b/doc/development/shared_files.md index e26464a5d80..4f13bb80761 100644 --- a/doc/development/shared_files.md +++ b/doc/development/shared_files.md @@ -11,5 +11,5 @@ servers in `shared/`, using a shared storage solution like NFS. Although this is some GitLab installations, it must not be the only file storage option for a given feature. This is because [cloud-native GitLab installations do not support it](architecture.md#adapting-existing-and-introducing-new-components). -Our [uploads documentation](uploads.md) describes how to handle file storage in +Our [uploads documentation](uploads/index.md) describes how to handle file storage in such a way that it supports both options: direct disk access and object storage. diff --git a/doc/development/sidekiq/idempotent_jobs.md b/doc/development/sidekiq/idempotent_jobs.md index 4b201e22ca9..38db22f8467 100644 --- a/doc/development/sidekiq/idempotent_jobs.md +++ b/doc/development/sidekiq/idempotent_jobs.md @@ -75,7 +75,7 @@ job executed, the first job would do nothing. GitLab supports two deduplication strategies: -- `until_executing` +- `until_executing`, which is the default strategy - `until_executed` More [deduplication strategies have been @@ -190,6 +190,7 @@ that can tolerate some duplication. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/69372) in GitLab 14.3. > - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/338350) in GitLab 14.4. > - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/338350) in GitLab 14.6. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/346598) in GitLab 14.9. [Feature flag preserve_latest_wal_locations_for_idempotent_jobs](https://gitlab.com/gitlab-org/gitlab/-/issues/346598) removed. The deduplication always take into account the latest binary replication pointer, not the first one. This happens because we drop the same job scheduled for the second time and the Write-Ahead Log (WAL) is lost. @@ -199,10 +200,3 @@ To support both deduplication and maintaining data consistency with load balanci we are preserving the latest WAL location for idempotent jobs in Redis. This way we are always comparing the latest binary replication pointer, making sure that we read from the replica that is fully caught up. - -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to -[disable the feature flag](../../administration/feature_flags.md) named `preserve_latest_wal_locations_for_idempotent_jobs`. - -This feature flag is related to GitLab development and is not intended to be used by GitLab administrators, though. -On GitLab.com, this feature is available. diff --git a/doc/development/sidekiq/worker_attributes.md b/doc/development/sidekiq/worker_attributes.md index d681e17a053..3bd6d313e2c 100644 --- a/doc/development/sidekiq/worker_attributes.md +++ b/doc/development/sidekiq/worker_attributes.md @@ -259,7 +259,7 @@ these scenarios, since `:always` should be considered the exception, not the rul To allow for reads to be served from replicas, we added two additional consistency modes: `:sticky` and `:delayed`. When you declare either `:sticky` or `:delayed` consistency, workers become eligible for database -load-balancing. +load-balancing. In both cases, if the replica is not up-to-date and the time from scheduling the job was less than the minimum delay interval, the jobs sleep up to the minimum delay interval (0.8 seconds). This gives the replication process time to finish. diff --git a/doc/development/sidekiq_style_guide.md b/doc/development/sidekiq_style_guide.md index 3756dd6b598..1b5e7addf29 100644 --- a/doc/development/sidekiq_style_guide.md +++ b/doc/development/sidekiq_style_guide.md @@ -6,4 +6,6 @@ remove_date: '2022-04-13' This document was moved to [another location](sidekiq/index.md). <!-- This redirect file can be deleted after <2022-04-13>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/development/snowplow/implementation.md b/doc/development/snowplow/implementation.md index d35413cfd5f..6061a1d4cd2 100644 --- a/doc/development/snowplow/implementation.md +++ b/doc/development/snowplow/implementation.md @@ -47,10 +47,7 @@ To implement tracking for HAML or Vue templates, add a [`data-track` attribute]( The following example shows `data-track-*` attributes assigned to a button: ```haml -%button.btn{ data: { track: { action: "click_button", label: "template_preview", property: "my-template" } } } - -// or -// %button.btn{ data: { track_action: "click_button", track_label: "template_preview", track_property: "my-template" } } +%button.btn{ data: { track_action: "click_button", track_label: "template_preview", track_property: "my-template" } } ``` ```html @@ -69,7 +66,7 @@ The following example shows `data-track-*` attributes assigned to a button: | `data-track-action` | true | Action the user is taking. Clicks must be prepended with `click` and activations must be prepended with `activate`. For example, focusing a form field is `activate_form_input` and clicking a button is `click_button`. Replaces `data-track-event`, which was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/290962) in GitLab 13.11. | | `data-track-label` | false | The specific element or object to act on. This can be: the label of the element, for example, a tab labeled 'Create from template' for `create_from_template`; a unique identifier if no text is available, for example, `groups_dropdown_close` for closing the Groups dropdown in the top bar; or the name or title attribute of a record being created. | | `data-track-property` | false | Any additional property of the element, or object being acted on. | -| `data-track-value` | false | Describes a numeric value (decimal) directly related to the event. This could be the value of an input. For example, `10` when clicking `internal` visibility. If omitted, this is the element's `value` property or `undefined`. For checkboxes, the default value is the element's checked attribute or `0` when unchecked. | +| `data-track-value` | false | Describes a numeric value (decimal) directly related to the event. This could be the value of an input. For example, `10` when clicking `internal` visibility. If omitted, this is the element's `value` property or `undefined`. For checkboxes, the default value is the element's checked attribute or `0` when unchecked. The value is parsed as numeric before sendind the event. | | `data-track-extra` | false | A key-value pair object passed as a valid JSON string. This attribute is added to the `extra` property in our [`gitlab_standard`](schemas.md#gitlab_standard) schema. | | `data-track-context` | false | To append a custom context object, passed as a valid JSON string. | @@ -520,7 +517,7 @@ To install and run Snowplow Micro, complete these steps to modify the ### Troubleshoot To control content security policy warnings when using an external host, modify `config/gitlab.yml` -to allow or disallow them. To allow them, add the relevant host for `connect_src`. For example, for +to allow or prevent them. To allow them, add the relevant host for `connect_src`. For example, for `https://snowplow.trx.gitlab.net`: ```yaml diff --git a/doc/development/spam_protection_and_captcha/exploratory_testing.md b/doc/development/spam_protection_and_captcha/exploratory_testing.md new file mode 100644 index 00000000000..e508265cf83 --- /dev/null +++ b/doc/development/spam_protection_and_captcha/exploratory_testing.md @@ -0,0 +1,360 @@ +--- +stage: Manage +group: Authentication and Authorization +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Exploratory testing of CAPTCHAs + +You can reliably test CAPTCHA on review apps, and in your local development environment (GDK). +You can always: + +- Force a reCAPTCHA to appear where it is supported. +- Force a checkbox to display, instead of street sign images to find and select. + +To set up testing, follow the configuration on this page. + +## Use appropriate test data + +Make sure you are testing a scenario which has spam/CAPTCHA enabled. For example: +make sure you are editing a _public_ snippet, as only public snippets are checked for spam. + +## Enable feature flags + +Enable any relevant feature flag, if the spam/CAPTCHA support is behind a feature flag. + +## Set up Akismet and reCAPTCHA + +1. To set up reCAPTCHA: + 1. Review the [GitLab reCAPTCHA documentation](../../integration/recaptcha.md). + 1. Get Google's official test reCAPTCHA credentials using the instructions from + [Google's reCAPTCHA documentation](https://developers.google.com/recaptcha/docs/faq#id-like-to-run-automated-tests-with-recaptcha.-what-should-i-do). + 1. For **Site key**, use: `6LeIxAcTAAAAAJcZVRqyHh71UMIEGNQ_MXjiZKhI` + 1. For **Secret key**, use: `6LeIxAcTAAAAAGG-vFI1TnRWxMZNFuojJ4WifJWe` + 1. Go to **Admin -> Settings -> Reporting** settings: `http://gdk.test:3000/admin/application_settings/reporting#js-spam-settings` + 1. Select **Enable reCAPTCHA**. Enabling for login is not required unless you are testing that feature. + 1. Enter the **Site key** and **Secret key**. +1. To set up Akismet: + 1. Review the [GitLab documentation on Akismet](../../integration/akismet.md). + 1. Get an Akismet API key. You can sign up for [a testing key from Akismet](https://akismet.com). + You must enter your local host (such as`gdk.test`) and email when signing up. + 1. Go to GitLab Akismet settings page, for example: + `http://gdk.test:3000/admin/application_settings/reporting#js-spam-settings` + 1. Enable Akismet and enter your Akismet **API key**. +1. To force an Akismet false-positive spam check, refer to the + [Akismet API documentation](https://akismet.com/development/api/#comment-check) and + [Akismet Getting Started documentation](https://docs.akismet.com/getting-started/confirm/) for more details: + 1. You can use `akismet-guaranteed-spam@example.com` as the author email to force spam using the following steps: + 1. Go to user email settings: `http://gdk.test:3000/-/profile/emails` + 1. Add `akismet-guaranteed-spam@example.com` as a secondary email for the administrator user. + 1. Confirm it in the Rails console: `bin/rails c` -> `User.find_by_username('root').emails.last.confirm` + 1. Switch this verified email to be your primary email: + 1. Go to **Avatar dropdown list -> Edit Profile -> Main Settings**. + 1. For **Email**, enter `akismet-guaranteed-spam@example.com` to replace `admin@example.com`. + 1. Select **Update Profile Settings** to save your changes. + +## Test in the web UI + +After you have all the above configuration in place, you can test CAPTCHAs. Test +in an area of the application which already has CAPTCHA support, such as: + +- Creating or editing an issue. +- Creating or editing a public snippet. Only **public** snippets are checked for spam. + +## Test in a development environment + +After you force Spam Flagging + CAPTCHA using the steps above, you can test the +behavior with any spam-protected model/controller action. + +### Test with CAPTCHA enabled (CONDITIONAL_ALLOW verdict) + +If CAPTCHA is enabled in these areas, you must solve the CAPTCHA popup modal before you can resubmit the form: + +- **Admin -> Settings -> Reporting -> Spam** +- **Anti-bot Protection -> Enable reCAPTCHA** + +<!-- vale gitlab.Substitutions = NO --> + +### Testing with CAPTCHA disabled ("DISALLOW" verdict) + +<!-- vale gitlab.Substitutions = YES --> + +If CAPTCHA is disabled in **Admin -> Settings -> Reporting -> Spam** and **Anti-bot Protection -> Enable reCAPTCHA**, +no CAPTCHA popup displays. You are prevented from submitting the form at all. + +### HTML page to render reCAPTCHA + +NOTE: +If you use **Google's official test reCAPTCHA credentials** listed in +[Set up Akismet and reCAPTCHA](#set-up-akismet-and-recaptcha), the +CAPTCHA response string does not matter. It can be any string. If you use a +real, valid key pair, you must solve the CAPTCHA to obtain a +valid CAPTCHA response to use. You can do this once only, and only before it expires. + +To directly test the GraphQL API via [GraphQL Explorer](http://gdk.test:3000/-/graphql-explorer), +get a reCAPTCHA response string via this form: `public/recaptcha.html` (`http://gdk.test:3000/recaptcha.html`): + +```html +<html> +<head> + <title>reCAPTCHA demo: Explicit render after an onload callback</title> + <script type="text/javascript"> + var onloadCallback = function() { + grecaptcha.render('html_element', { + 'sitekey' : '6Ld05AsaAAAAAMsm1yTUp4qsdFARN15rQJPPqv6i' + }); + }; + function onSubmit() { + window.document.getElementById('recaptchaResponse').innerHTML = grecaptcha.getResponse(); + return false; + } + </script> +</head> +<body> +<form onsubmit="return onSubmit()"> + <div id="html_element"></div> + <br> + <input type="submit" value="Submit"> +</form> +<div> + <h1>recaptchaResponse:</h1> + <div id="recaptchaResponse"></div> +</div> +<script src="https://www.google.com/recaptcha/api.js?onload=onloadCallback&render=explicit" + async defer> +</script> +</body> +</html> +``` + +## Spam/CAPTCHA API exploratory testing examples + +These sections describe the steps needed to perform manual exploratory testing of +various scenarios of the Spam and CAPTCHA behavior for the REST and GraphQL APIs. + +For the prerequisites, you must: + +1. Perform all the steps listed above to enable Spam and CAPTCHA in the development environment, + and force form submissions to require a CAPTCHA. +1. Ensure you have created an HTML page to render CAPTCHA under the `/public` directory, + with a page that contains a form to manually generate a valid CAPTCHA response string. + If you use **Google's official test reCAPTCHA credentials** listed in + [Set up Akismet and reCAPTCHA](#set-up-akismet-and-recaptcha), the contents of the + CAPTCHA response string don't matter. +1. Go to **Admin -> Settings -> Reporting -> Spam and Anti-bot protection**. +1. Select or clear **Enable reCAPTCHA** and **Enable Akismet** according to your + scenario's needs. + +The following examples use snippet creation as an example. You could also use +snippet updates, issue creation, or issue updates. Issues and snippets are the +only models with full Spam and CAPTCHA support. + +### Initial setup + +1. Create an API token. +1. Export it in your terminal for the REST commands: `export PRIVATE_TOKEN=<your_api_token>` +1. Ensure you are logged into GitLab development environment at `localhost:3000` before using GraphiQL explorer, + because it uses your logged-in user as authorization for running GraphQL queries. +1. For the GraphQL examples, use the GraphiQL explorer at `http://localhost:3000/-/graphql-explorer`. +1. Use the `--include` (`-i`) option to `curl` to print the HTTP response headers, including the status code. + +### Scenario: Akismet and CAPTCHA enabled + +In this example, Akismet and CAPTCHA are enabled: + +1. [Initial request](#initial-request). + +<!-- TODO in future edit + +Some example videos: + +- REST API: + + + +GraphQL API: + + + +--> + +#### Initial request + +This initial request fails because no CAPTCHA response is provided. + +REST request: + +```shell +curl --request POST --header "PRIVATE-TOKEN: $PRIVATE_TOKEN" "http://localhost:3000/api/v4/snippets?title=Title&file_name=FileName&content=Content&visibility=public" +``` + +REST response: + +```shell +{"needs_captcha_response":true,"spam_log_id":42,"captcha_site_key":"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX","message":{"error":"Your snippet has been recognized as spam. Please, change the content or solve the reCAPTCHA to proceed."}} +``` + +GraphQL request: + +```graphql +mutation { + createSnippet(input: { + title: "Title" + visibilityLevel: public + blobActions: [ + { + action: create + filePath: "BlobPath" + content: "BlobContent" + } + ] + }) { + snippet { + id + title + } + errors + } +} +``` + +GraphQL response: + +```json +{ + "data": { + "createSnippet": null + }, + "errors": [ + { + "message": "Request denied. Solve CAPTCHA challenge and retry", + "locations": [ + { + "line": 22, + "column": 5 + } + ], + "path": [ + "createSnippet" + ], + "extensions": { + "needs_captcha_response": true, + "spam_log_id": 140, + "captcha_site_key": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" + } + } + ] +} +``` + +#### Second request + +This request succeeds because a CAPTCHA response is provided. + +REST request: + +```shell +export CAPTCHA_RESPONSE="<CAPTCHA response obtained from HTML page to render CAPTCHA>" +export SPAM_LOG_ID="<spam_log_id obtained from initial REST response>" +curl --request POST --header "PRIVATE-TOKEN: $PRIVATE_TOKEN" --header "X-GitLab-Captcha-Response: $CAPTCHA_RESPONSE" --header "X-GitLab-Spam-Log-Id: $SPAM_LOG_ID" "http://localhost:3000/api/v4/snippets?title=Title&file_name=FileName&content=Content&visibility=public" +``` + +REST response: + +```shell +{"id":42,"title":"Title","description":null,"visibility":"public", "other_fields": "..."} +``` + +GraphQL request: + +NOTE: +The GitLab GraphiQL implementation doesn't allow passing of headers, so we must write +this as a `curl` query. Here, `--data-binary` is used to properly handle escaped double quotes +in the JSON-embedded query. + +```shell +export CAPTCHA_RESPONSE="<CAPTCHA response obtained from HTML page to render CAPTCHA>" +export SPAM_LOG_ID="<spam_log_id obtained from initial REST response>" +curl --include "http://localhost:3000/api/graphql" --header "Authorization: Bearer $PRIVATE_TOKEN" --header "Content-Type: application/json" --header "X-GitLab-Captcha-Response: $CAPTCHA_RESPONSE" --header "X-GitLab-Spam-Log-Id: $SPAM_LOG_ID" --request POST --data-binary '{"query": "mutation {createSnippet(input: {title: \"Title\" visibilityLevel: public blobActions: [ { action: create filePath: \"BlobPath\" content: \"BlobContent\" } ] }) { snippet { id title } errors }}"}' +``` + +GraphQL response: + +```json +{"data":{"createSnippet":{"snippet":{"id":"gid://gitlab/PersonalSnippet/42","title":"Title"},"errors":[]}}} +``` + +### Scenario: Akismet enabled, CAPTCHA disabled + +For this scenario, ensure you clear **Enable reCAPTCHA** in the Admin Area settings as described above. +If CAPTCHA is not enabled, any request flagged as potential spam fails with no chance to resubmit, +even if it could otherwise be resubmitted if CAPTCHA were enabled and successfully solved. + +The REST request is the same as if CAPTCHA was enabled: + +```shell +curl --request POST --header "PRIVATE-TOKEN: $PRIVATE_TOKEN" "http://localhost:3000/api/v4/snippets?title=Title&file_name=FileName&content=Content&visibility=public" +``` + +REST response: + +```shell +{"message":{"error":"Your snippet has been recognized as spam and has been discarded."}} +``` + +GraphQL request: + +```graphql +mutation { + createSnippet(input: { + title: "Title" + visibilityLevel: public + blobActions: [ + { + action: create + filePath: "BlobPath" + content: "BlobContent" + } + ] + }) { + snippet { + id + title + } + errors + } +} +``` + +GraphQL response: + +```json +{ + "data": { + "createSnippet": null + }, + "errors": [ + { + "message": "Request denied. Spam detected", + "locations": [ + { + "line": 22, + "column": 5 + } + ], + "path": [ + "createSnippet" + ], + "extensions": { + "spam": true + } + } + ] +} +``` + +### Scenario: allow_possible_spam feature flag enabled + +With the `allow_possible_spam` feature flag enabled, the API returns a 200 response. Any +valid request is successful and no CAPTCHA is presented, even if the request is considered +spam. diff --git a/doc/development/spam_protection_and_captcha/graphql_api.md b/doc/development/spam_protection_and_captcha/graphql_api.md new file mode 100644 index 00000000000..b47e3f84320 --- /dev/null +++ b/doc/development/spam_protection_and_captcha/graphql_api.md @@ -0,0 +1,66 @@ +--- +stage: Manage +group: Authentication and Authorization +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# GraphQL API spam protection and CAPTCHA support + +If the model can be modified via the GraphQL API, you must also add support to all of the +relevant GraphQL mutations which may modify spammable or spam-related attributes. This +definitely includes the `Create` and `Update` mutations, but may also include others, such as those +related to changing a model's confidential/public flag. + +## Add support to the GraphQL mutations + +This implementation is very similar to the controller implementation. You create a `spam_params` +instance based on the request, and pass it to the relevant Service class constructor. + +The three main differences from the controller implementation are: + +1. Use `include Mutations::SpamProtection` instead of `...JsonFormatActionsSupport`. +1. Obtain the request from the context via `context[:request]` when creating the `SpamParams` + instance. +1. After you create or updated the `Spammable` model instance, call `#check_spam_action_response!` + and pass it the model instance. This call will: + 1. Perform the necessary spam checks on the model. + 1. If spam is detected: + - Raise a `GraphQL::ExecutionError` exception. + - Include the relevant information added as error fields to the response via the `extensions:` parameter. + For more details on these fields, refer to the section on + [Spam and CAPTCHA support in the GraphQL API](../../api/graphql/index.md#resolve-mutations-detected-as-spam). + + NOTE: + If you use the standard ApolloLink or Axios interceptor CAPTCHA support described + above, the field details are unimportant. They become important if you + attempt to use the GraphQL API directly to process a failed check for potential spam, and + resubmit the request with a solved CAPTCHA response. + +For example: + +```ruby +module Mutations + module Widgets + class Create < BaseMutation + include Mutations::SpamProtection + + def resolve(args) + spam_params = ::Spam::SpamParams.new_from_request(request: context[:request]) + + service_response = ::Widgets::CreateService.new( + project: project, + current_user: current_user, + params: args, + spam_params: spam_params + ).execute + + widget = service_response.payload[:widget] + check_spam_action_response!(widget) + + # If possible spam wasdetected, an exception would have been thrown by + # `#check_spam_action_response!`, so the normal resolve return logic can follow below. + end + end + end +end +``` diff --git a/doc/development/spam_protection_and_captcha/index.md b/doc/development/spam_protection_and_captcha/index.md new file mode 100644 index 00000000000..9b195df536d --- /dev/null +++ b/doc/development/spam_protection_and_captcha/index.md @@ -0,0 +1,53 @@ +--- +stage: Manage +group: Authentication and Authorization +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Spam protection and CAPTCHA + +This guide provides an overview of how to add spam protection and CAPTCHA support to new areas of the +GitLab application. + +## Add spam protection and CAPTCHA support to a new area + +To add this support, you must implement the following areas as applicable: + +1. [Model and Services](model_and_services.md): The basic prerequisite + changes to the backend code which are required to add spam or CAPTCHA API and UI support + for a feature which does not yet have support. +1. REST API (Supported, documentation coming soon): The changes needed to add + spam or CAPTCHA support to Grape REST API endpoints. Refer to the related + [REST API documentation](../../api/index.md#resolve-requests-detected-as-spam). +1. [GraphQL API](graphql_api.md): The changes needed to add spam or CAPTCHA support to GraphQL + mutations. Refer to the related + [GraphQL API documentation](../../api/graphql/index.md#resolve-mutations-detected-as-spam). +1. [Web UI](web_ui.md): The various possible scenarios encountered when adding + spam/CAPTCHA support to the web UI, depending on whether the UI is JavaScript API-based (Vue or + plain JavaScript) or HTML-form (HAML) based. + +You should also perform manual exploratory testing of the new feature. Refer to +[Exploratory testing](exploratory_testing.md) for more information. + +## Spam-related model and API fields + +Multiple levels of spam flagging determine how spam is handled. These levels are referenced in +[`Spam::SpamConstants`](https://gitlab.com/gitlab-org/gitlab/blob/master/app/services/spam/spam_constants.rb#L4-4), +and used various places in the application, such as +[`Spam::SpamActionService#perform_spam_service_check`](https://gitlab.com/gitlab-org/gitlab/blob/d7585b56c9e7dc69414af306d82906e28befe7da/app/services/spam/spam_action_service.rb#L61-61). + +The possible values include: + +- `BLOCK_USER` +- `DISALLOW` +- `CONDITIONAL_ALLOW` +- `OVERRIDE_VIA_ALLOW_POSSIBLE_SPAM` +- `ALLOW` +- `NOOP` + +## Related topics + +- [Spam and CAPTCHA support in the GraphQL API](../../api/graphql/index.md#resolve-mutations-detected-as-spam) +- [Spam and CAPTCHA support in the REST API](../../api/index.md#resolve-requests-detected-as-spam) +- [reCAPTCHA Spam and Anti-bot Protection](../../integration/recaptcha.md) +- [Akismet and Spam Logs](../../integration/akismet.md) diff --git a/doc/development/spam_protection_and_captcha/model_and_services.md b/doc/development/spam_protection_and_captcha/model_and_services.md new file mode 100644 index 00000000000..d0519cba68f --- /dev/null +++ b/doc/development/spam_protection_and_captcha/model_and_services.md @@ -0,0 +1,155 @@ +--- +stage: Manage +group: Authentication and Authorization +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Model and services spam protection and CAPTCHA support + +Before adding any spam or CAPTCHA support to the REST API, GraphQL API, or Web UI, you must +first add the necessary support to: + +1. The backend ActiveRecord models. +1. The services layer. + +All or most of the following changes are required, regardless of the type of spam or CAPTCHA request +implementation you are supporting. Some newer features which are completely based on the GraphQL API +may not have any controllers, and don't require you to add the `mark_as_spam` action to the controller. + +To do this: + +1. [Add `Spammable` support to the ActiveRecord model](#add-spammable-support-to-the-activerecord-model). +1. [Add support for the `mark_as_spam` action to the controller](#add-support-for-the-mark_as_spam-action-to-the-controller). +1. [Add a call to SpamActionService to the execute method of services](#add-a-call-to-spamactionservice-to-the-execute-method-of-services). + +## Add `Spammable` support to the ActiveRecord model + +1. Include the `Spammable` module in the model class: + + ```ruby + include Spammable + ``` + +1. Add: `attr_spammable` to indicate which fields can be checked for spam. Up to + two fields per model are supported: a "`title`" and a "`description`". You can + designate which fields to consider the "`title`" or "`description`". For example, + this line designates the `content` field as the `description`: + + ```ruby + attr_spammable :content, spam_description: true + ``` + +1. Add a `#check_for_spam?` method implementation: + + ```ruby + def check_for_spam?(user:) + # Return a boolean result based on various applicable checks, which may include + # which attributes have changed, the type of user, whether the data is publicly + # visible, and other criteria. This may vary based on the type of model, and + # may change over time as spam checking requirements evolve. + end + ``` + + Refer to other existing `Spammable` models' + implementations of this method for examples of the required logic checks. + +## Add support for the `mark_as_spam` action to the controller + +The `SpammableActions::AkismetMarkAsSpamAction` module adds support for a `#mark_as_spam` action +to a controller. This controller allows administrators to manage spam for the associated +`Spammable` model in the [Spam Log section](../../integration/akismet.md) of the Admin Area page. + +1. Include the `SpammableActions::AkismetMarkAsSpamAction` module in the controller. + + ```ruby + include SpammableActions::AkismetMarkAsSpamAction + ``` + +1. Add a `#spammable_path` method implementation. The spam administration page redirects + to this page after edits. Refer to other existing controllers' implementations + of this method for examples of the type of path logic required. In general, it should + be the `#show` action for the `Spammable` model's controller. + + ```ruby + def spammable_path + widget_path(widget) + end + ``` + +NOTE: +There may be other changes needed to controllers, depending on how the feature is +implemented. See [Web UI](web_ui.md) for more details. + +## Add a call to SpamActionService to the execute method of services + +This approach applies to any service which can persist spammable attributes: + +1. In the relevant Create or Update service under `app/services`, pass in a populated + `Spam::SpamParams` instance. (Refer to instructions later on in this page.) +1. Use it and the `Spammable` model instance to execute a `Spam::SpamActionService` instance. +1. If the spam check fails: + - An error is added to the model, which causes it to be invalid and prevents it from being saved. + - The `needs_recaptcha` property is set to `true`. + + These changes to the model enable it for handling by the subsequent backend and frontend CAPTCHA logic. + +Make these changes to each relevant service: + +1. Change the constructor to take a `spam_params:` argument as a required named argument. + + Using named arguments for the constructor helps you identify all the calls to + the constructor that need changing. It's less risky because the interpreter raises + type errors unless the caller is changed to pass the `spam_params` argument. + If you use an IDE (such as RubyMine) which supports this, your + IDE flags it as an error in the editor. + +1. In the constructor, set the `@spam_params` instance variable from the `spam_params` constructor + argument. Add an `attr_reader: :spam_params` in the `private` section of the class. + +1. In the `execute` method, add a call to execute the `Spam::SpamActionService`. + (You can also use `before_create` or `before_update`, if the service + uses that pattern.) This method uses named arguments, so its usage is clear if + you refer to existing examples. However, two important considerations exist: + 1. The `SpamActionService` must be executed _after_ all necessary changes are made to + the unsaved (and dirty) `Spammable` model instance. This ordering ensures + spammable attributes exist to be spam-checked. + 1. The `SpamActionService` must be executed _before_ the model is checked for errors and + attempting a `save`. If potential spam is detected in the model's changed attributes, we must prevent a save. + +```ruby +module Widget + class CreateService < ::Widget::BaseService + # NOTE: We require the spam_params and do not default it to nil, because + # spam_checking is likely to be necessary. However, if there is not a request available in scope + # in the caller (for example, a note created via email) and the required arguments to the + # SpamParams constructor are not otherwise available, spam_params: must be explicitly passed as nil. + def initialize(project:, current_user: nil, params: {}, spam_params:) + super(project: project, current_user: current_user, params: params) + + @spam_params = spam_params + end + + def execute + widget = Widget::BuildService.new(project, current_user, params).execute + + # More code that may manipulate dirty model before it is spam checked. + + # NOTE: do this AFTER the spammable model is instantiated, but BEFORE + # it is validated or saved. + Spam::SpamActionService.new( + spammable: widget, + spam_params: spam_params, + user: current_user, + # Or `action: :update` for a UpdateService or service for an existing model. + action: :create + ).execute + + # Possibly more code related to saving model, but should not change any attributes. + + widget.save + end + + private + + attr_reader :spam_params +``` diff --git a/doc/development/spam_protection_and_captcha/web_ui.md b/doc/development/spam_protection_and_captcha/web_ui.md new file mode 100644 index 00000000000..6aa01f401bd --- /dev/null +++ b/doc/development/spam_protection_and_captcha/web_ui.md @@ -0,0 +1,196 @@ +--- +stage: Manage +group: Authentication and Authorization +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Web UI spam protection and CAPTCHA support + +The approach for adding spam protection and CAPTCHA support to a new UI area of the GitLab application +depends upon how the existing code is implemented. + +## Supported scenarios of request submissions + +Three different scenarios are supported. Two are used with JavaScript XHR/Fetch requests +for either Apollo or Axios, and one is used only with standard HTML form requests: + +1. A JavaScript-based submission (possibly via Vue) + 1. Using Apollo (GraphQL API via Fetch/XHR request) + 1. Using Axios (REST API via Fetch/XHR request) +1. A standard HTML form submission (HTML request) + +Some parts of the implementation depend upon which of these scenarios you must support. + +## Implementation tasks specific to JavaScript XHR/Fetch requests + +Two approaches are fully supported: + +1. Apollo, using the GraphQL API. +1. Axios, using either the GraphQL API. + +The spam and CAPTCHA-related data communication between the frontend and backend requires no +additional fields being added to the models. Instead, communication is handled: + +- Through custom header values in the request. +- Through top-level JSON fields in the response. + +The spam and CAPTCHA-related logic is also cleanly abstracted into reusable modules and helper methods +which can wrap existing logic, and only alter the existing flow if potential spam +is detected or a CAPTCHA display is needed. This approach allows the spam and CAPTCHA +support to be easily added to new areas of the application with minimal changes to +existing logic. In the case of the frontend, potentially **zero** changes are needed! + +On the frontend, this is handled abstractly and transparently using `ApolloLink` for Apollo, and an +Axios interceptor for Axios. The CAPTCHA display is handled by a standard GitLab UI / Pajamas modal +component. You can find all the relevant frontend code under `app/assets/javascripts/captcha`. + +However, even though the actual handling of the request interception and +modal is transparent, without any mandatory changes to the involved JavaScript or Vue components +for the form or page, changes in request or error handling may be required. Changes are needed +because the existing behavior may not work correctly: for example, if a failed or cancelled +CAPTCHA display interrupts the normal request flow or UI updates. +Careful exploratory testing of all scenarios is important to uncover any potential +problems. + +This sequence diagram illustrates the normal CAPTCHA flow for JavaScript XHR/Fetch requests +on the frontend: + +```mermaid +sequenceDiagram + participant U as User + participant V as Vue/JS Application + participant A as ApolloLink or Axios Interceptor + participant G as GitLab API + U->>V: Save model + V->>A: Request + A->>G: Request + G--xA: Response with error and spam/CAPTCHA related fields + A->>U: CAPTCHA presented in modal + U->>A: CAPTCHA solved to obtain valid CAPTCHA response + A->>G: Request with valid CAPTCHA response and SpamLog ID in headers + G-->>A: Response with success + A-->>V: Response with success +``` + +The backend is also cleanly abstracted via mixin modules and helper methods. The three main +changes required to the relevant backend controller actions (normally just `create`/`update`) are: + +1. Create a `SpamParams` parameter object instance based on the request, using the simple static + `#new_from_request` factory method. This method takes a request, and returns a `SpamParams` instance. +1. Pass the created `SpamParams` instance as the `spam_params` named argument to the + Service class constructor, which you should have already added. If the spam check indicates + the changes to the model are possibly spam, then: + - An error is added to the model. + - The `needs_recaptcha` property on the model is set to true. +1. Wrap the existing controller action return value (rendering or redirecting) in a block passed to + a `#with_captcha_check_json_format` helper method, which transparently handles: + 1. Check if CAPTCHA is enabled, and if so, proceeding with the next step. + 1. Checking if there the model contains an error, and the `needs_recaptcha` flag is true. + - If yes: Add the appropriate spam or CAPTCHA fields to the JSON response, and return + a `409 - Conflict` HTTP status code. + - If no (if CAPTCHA is disabled or if no spam was detected): The normal request return + logic passed in the block is run. + +Thanks to the abstractions, it's more straightforward to implement than it is to explain it. +You don't have to worry much about the hidden details! + +Make these changes: + +## Add support to the controller actions + +If the feature's frontend submits directly to controller actions, and does not only use the GraphQL +API, then you must add support to the appropriate controllers. + +The action methods may be directly in the controller class, or they may be abstracted +to a module included in the controller class. Our example uses a module. The +only difference when directly modifying the controller: +`extend ActiveSupport::Concern` is not required. + +```ruby +module WidgetsActions + # NOTE: This `extend` probably already exists, but it MUST be moved to occur BEFORE all + # `include` statements. Otherwise, confusing bugs may occur in which the methods + # in the included modules cannot be found. + extend ActiveSupport::Concern + + include SpammableActions::CaptchaCheck::JsonFormatActionsSupport + + def create + spam_params = ::Spam::SpamParams.new_from_request(request: request) + widget = ::Widgets::CreateService.new( + project: project, + current_user: current_user, + params: params, + spam_params: spam_params + ).execute + + respond_to do |format| + format.json do + with_captcha_check_json_format do + # The action's existing `render json: ...` (or wrapper method) and related logic. Possibly + # including different rendering cases if the model is valid or not. It's all wrapped here + # within the `with_captcha_check_json_format` block. For example: + if widget.valid? + render json: serializer.represent(widget) + else + render json: { errors: widget.errors.full_messages }, status: :unprocessable_entity + end + end + end + end + end +end +``` + +## Implementation tasks specific to HTML form requests + +Some areas of the application have not been converted to use the GraphQL API via +a JavaScript client, but instead rely on standard Rails HAML form submissions via an +`HTML` MIME type request. In these areas, the action returns a pre-rendered HTML (HAML) page +as the response body. Unfortunately, in this case +[it is not possible](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/66427#note_636989204) +to use any of the JavaScript-based frontend support as described above. Instead we must use an +alternate approach which handles the rendering of the CAPTCHA form via a HAML template. + +Everything is still cleanly abstracted, and the implementation in the backend +controllers is virtually identical to the JavaScript/JSON based approach. Replace the +word `JSON` with `HTML` (using the appropriate case) in the module names and helper methods. + +The action methods might be directly in the controller, or they +might be in a module. In this example, they are directly in the +controller, and we also do an `update` method instead of `create`: + +```ruby +class WidgetsController < ApplicationController + include SpammableActions::CaptchaCheck::HtmlFormatActionsSupport + + def update + # Existing logic to find the `widget` model instance... + + spam_params = ::Spam::SpamParams.new_from_request(request: request) + ::Widgets::UpdateService.new( + project: project, + current_user: current_user, + params: params, + spam_params: spam_params + ).execute(widget) + + respond_to do |format| + format.html do + if widget.valid? + # NOTE: `spammable_path` is required by the `SpammableActions::AkismetMarkAsSpamAction` + # module, and it should have already been implemented on this controller according to + # the instructions above. It is reused here to avoid duplicating the route helper call. + redirect_to spammable_path + else + # If we got here, there were errors on the model instance - from a failed spam check + # and/or other validation errors on the model. Either way, we'll re-render the form, + # and if a CAPTCHA render is necessary, it will be automatically handled by + # `with_captcha_check_html_format` + with_captcha_check_html_format { render :edit } + end + end + end + end +end +``` diff --git a/doc/development/testing_guide/end_to_end/best_practices.md b/doc/development/testing_guide/end_to_end/best_practices.md index 405ff40ef6a..e0f6cbe632d 100644 --- a/doc/development/testing_guide/end_to_end/best_practices.md +++ b/doc/development/testing_guide/end_to_end/best_practices.md @@ -17,13 +17,16 @@ In case custom inflection logic is needed, custom inflectors are added in the [q ## Link a test to its test case Every test should have a corresponding test case in the [GitLab project Test Cases](https://gitlab.com/gitlab-org/gitlab/-/quality/test_cases) as well as a results issue in the [Quality Test Cases project](https://gitlab.com/gitlab-org/quality/testcases/-/issues). -It's recommended that you reuse the issue created to plan the test as the results issue. If a test case or results issue does not already exist you -can create them yourself. Alternatively, you can run the test in a pipeline that has reporting -enabled and the test-case reporter will automatically create a new test case and/or results issue and link the results issue to it's corresponding test case. +If a test case issue does not yet exist you can create one yourself. To do so, create a new +issue in the [Test Cases](https://gitlab.com/gitlab-org/gitlab/-/quality/test_cases) GitLab project +with a placeholder title. After the test case URL is linked to a test in the code, when the test is +run in a pipeline that has reporting enabled, the `report-results` script automatically updates the +test case and the results issue. +If a results issue does not yet exist, the `report-results` script automatically creates one and +links it to its corresponding test case. -Whether you create a new test case or one is created automatically, you will need to manually add -a `testcase` RSpec metadata tag. In most cases, a single test will be associated with a single test case - ([see below for exceptions](#exceptions)). +To link a test case to a test in the code, you must manually add a `testcase` RSpec metadata tag. +In most cases, a single test is associated with a single test case. For example: @@ -41,7 +44,7 @@ RSpec.describe 'Stage' do end ``` -### Exceptions +### For shared tests Most tests are defined by a single line of a `spec` file, which is why those tests can be linked to a single test case via the `testcase` tag. @@ -54,24 +57,19 @@ multiple tests, including: - Templated tests. - Tests in shared examples that include more than one example. -In those and similar cases we can't assign a single `testcase` tag and so we rely on the test-case -reporter to programmatically determine the correct test case based on the name and description of -the test. In such cases, the test-case reporter will automatically create a test case and/or results issue -the first time the test runs, if none exist already. +In those and similar cases we need to include the test case link by other means. -In such a case, if you create the test case or results issue yourself or want to reuse an existing issue, -you must use this [end-to-end test issue template](https://gitlab.com/gitlab-org/quality/testcases/-/blob/master/.gitlab/issue_templates/End-to-end%20Test.md) -to format the issue description. (Note you must copy/paste this for test cases as templates aren't currently available.) - -To illustrate, there are two tests in the shared examples in [`qa/specs/features/ee/browser_ui/3_create/repository/restrict_push_protected_branch_spec.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/47b17db82c38ab704a23b5ba5d296ea0c6a732c8/qa/qa/specs/features/ee/browser_ui/3_create/repository/restrict_push_protected_branch_spec.rb): +To illustrate, there are two tests in the shared examples in [`qa/specs/features/ee/browser_ui/3_create/repository/restrict_push_protected_branch_spec.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/qa/specs/features/ee/browser_ui/3_create/repository/restrict_push_protected_branch_spec.rb): ```ruby -shared_examples 'only user with access pushes and merges' do - it 'unselected maintainer user fails to push' do +shared_examples 'unselected maintainer' do |testcase| + it 'user fails to push', testcase: testcase do ... end +end - it 'selected developer user pushes and merges' do +shared_examples 'selected developer' do |testcase| + it 'user pushes and merges', testcase: testcase do ... end end @@ -84,112 +82,22 @@ RSpec.describe 'Create' do describe 'Restricted protected branch push and merge' do context 'when only one user is allowed to merge and push to a protected branch' do ... - it_behaves_like 'only user with access pushes and merges' - end - end -end -``` - -There would be two associated test cases, one for each shared example, with the following content: - -[Test 1 Test Case](https://gitlab.com/gitlab-org/gitlab/-/quality/test_cases/347774): - -````markdown -```markdown -Title: browser_ui/3_create/repository/restrict_push_protected_branch_spec.rb | Create Restricted -protected branch push and merge when only one user is allowed to merge and push to a protected -branch behaves like only user with access pushes and merges selecte... - -Description: -### Full description - -Create Restricted protected branch push and merge when only one user is allowed to merge and push -to a protected branch behaves like only user with access pushes and merges selected developer user -pushes and merges - -### File path - -./qa/specs/features/ee/browser_ui/3_create/repository/restrict_push_protected_branch_spec.rb - -### DO NOT EDIT BELOW THIS LINE - -Active and historical test results: - -https://gitlab.com/gitlab-org/quality/testcases/-/issues/2177 - -``` -```` - -[Test 1 Results Issue](https://gitlab.com/gitlab-org/quality/testcases/-/issues/2177): - -````markdown -```markdown -Title: browser_ui/3_create/repository/restrict_push_protected_branch_spec.rb | Create Restricted -protected branch push and merge when only one user is allowed to merge and push to a protected -branch behaves like only user with access pushes and merges selecte... - -Description: -### Full description - -Create Restricted protected branch push and merge when only one user is allowed to merge and push -to a protected branch behaves like only user with access pushes and merges selected developer user -pushes and merges - -### File path - -./qa/specs/features/ee/browser_ui/3_create/repository/restrict_push_protected_branch_spec.rb -``` -```` - -[Test 2 Test Case](https://gitlab.com/gitlab-org/gitlab/-/quality/test_cases/347775): - -````markdown -```markdown -Title: browser_ui/3_create/repository/restrict_push_protected_branch_spec.rb | Create Restricted -protected branch push and merge when only one user is allowed to merge and push to a protected -branch behaves like only user with access pushes and merges unselec... - -Description: -### Full description - -Create Restricted protected branch push and merge when only one user is allowed to merge and push -to a protected branch behaves like only user with access pushes and merges unselected maintainer -user fails to push - -### File path - -./qa/specs/features/ee/browser_ui/3_create/repository/restrict_push_protected_branch_spec.rb - -### DO NOT EDIT BELOW THIS LINE - -Active and historical test results: + it_behaves_like 'unselected maintainer', 'https://gitlab.com/gitlab-org/gitlab/-/quality/test_cases/347775' + it_behaves_like 'selected developer', 'https://gitlab.com/gitlab-org/gitlab/-/quality/test_cases/347774' + end -https://gitlab.com/gitlab-org/quality/testcases/-/issues/2176 + context 'when only one group is allowed to merge and push to a protected branch' do + ... + it_behaves_like 'unselected maintainer', 'https://gitlab.com/gitlab-org/gitlab/-/quality/test_cases/347772' + it_behaves_like 'selected developer', 'https://gitlab.com/gitlab-org/gitlab/-/quality/test_cases/347773' + end + end +end ``` -```` - -[Test 2 Results Issue](https://gitlab.com/gitlab-org/quality/testcases/-/issues/2176): - -````markdown -```markdown -Title: browser_ui/3_create/repository/restrict_push_protected_branch_spec.rb | Create Restricted -protected branch push and merge when only one user is allowed to merge and push to a protected -branch behaves like only user with access pushes and merges unselec... - -Description: -### Full description - -Create Restricted protected branch push and merge when only one user is allowed to merge and push -to a protected branch behaves like only user with access pushes and merges unselected maintainer -user fails to push -### File path - -./qa/specs/features/ee/browser_ui/3_create/repository/restrict_push_protected_branch_spec.rb -``` -```` +We recommend creating four associated test cases, two for each shared example. ## Prefer API over UI @@ -518,3 +426,12 @@ Neither problem is present if we create a custom negatable matcher because the ` would be used, which would wait only as long as necessary for the job to disappear. Lastly, negatable matchers are preferred over using matchers of the form `have_no_*` because it's a common and familiar practice to negate matchers using `not_to`. If we facilitate that practice by adding negatable matchers, we make it easier for subsequent test authors to write efficient tests. + +## Use logger over puts + +We currently use Rails `logger` to handle logs in both GitLab QA application and end-to-end tests. +This provides additional functionalities when compared with `puts`, such as: + +- Ability to specify the logging level. +- Ability to tag similar logs. +- Auto-formatting log messages. diff --git a/doc/development/testing_guide/end_to_end/feature_flags.md b/doc/development/testing_guide/end_to_end/feature_flags.md index 238b1ccd8f9..c3e3f117c2b 100644 --- a/doc/development/testing_guide/end_to_end/feature_flags.md +++ b/doc/development/testing_guide/end_to_end/feature_flags.md @@ -130,7 +130,7 @@ You can set this variable inside the `fabricate_via_api` call. For a consistent - Add the word `activated` to the end of a variable's name. - Inside the `initialize` method, set the variable's default value. -For example: +For example: ```ruby def initialize diff --git a/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md b/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md index 07f7e9582ee..f9b505a8271 100644 --- a/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md +++ b/doc/development/testing_guide/end_to_end/rspec_metadata_tests.md @@ -20,7 +20,7 @@ This is a partial list of the [RSpec metadata](https://relishapp.com/rspec/rspec | `:github` | The test requires a GitHub personal access token. | | `:group_saml` | The test requires a GitLab instance that has SAML SSO enabled at the group level. Interacts with an external SAML identity provider. Paired with the `:orchestrated` tag. | | `:instance_saml` | The test requires a GitLab instance that has SAML SSO enabled at the instance level. Interacts with an external SAML identity provider. Paired with the `:orchestrated` tag. | -| `:integrations` | This aims to test the available [integrations](../../../user/project/integrations/overview.md#integrations-listing). The test requires Docker to be installed in the run context. It will provision the containers and can be run against a local instance or using the `gitlab-qa` scenario `Test::Integration::Integrations` | +| `:integrations` | This aims to test the available [integrations](../../../user/project/integrations/overview.md#integrations-listing). The test requires Docker to be installed in the run context. It will provision the containers and can be run against a local instance or using the `gitlab-qa` scenario `Test::Integration::Integrations` | | `:service_ping_disabled` | The test interacts with the GitLab configuration service ping at the instance level to turn admin setting service ping checkbox on or off. This tag will have the test run only in the `service_ping_disabled` job and must be paired with the `:orchestrated` and `:requires_admin` tags. | | `:jira` | The test requires a Jira Server. [GitLab-QA](https://gitlab.com/gitlab-org/gitlab-qa) provisions the Jira Server in a Docker container when the `Test::Integration::Jira` test scenario is run. | `:kubernetes` | The test includes a GitLab instance that is configured to be run behind an SSH tunnel, allowing a TLS-accessible GitLab. This test also includes provisioning of at least one Kubernetes cluster to test against. _This tag is often be paired with `:orchestrated`._ | @@ -42,6 +42,7 @@ This is a partial list of the [RSpec metadata](https://relishapp.com/rspec/rspec | `:requires_praefect` | The test requires that the GitLab instance uses [Gitaly Cluster](../../../administration/gitaly/praefect.md) (a.k.a. Praefect) as the repository storage . It's assumed to be used by default but if not the test can be skipped by setting `QA_CAN_TEST_PRAEFECT` to `false`. | | `:runner` | The test depends on and sets up a GitLab Runner instance, typically to run a pipeline. | | `:skip_live_env` | The test is excluded when run against live deployed environments such as Staging, Canary, and Production. | +| `:skip_fips_env` | The test is excluded when run against an environment in FIPS mode. | | `:skip_signup_disabled` | The test uses UI to sign up a new user and is skipped in any environment that does not allow new user registration via the UI. | | `:smoke` | The test belongs to the test suite which verifies basic functionality of a GitLab instance.| | `:smtp` | The test requires a GitLab instance to be configured to use an SMTP server. Tests SMTP notification email delivery from GitLab by using MailHog. | diff --git a/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md b/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md index 7fb95769fc2..49a9124253d 100644 --- a/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md +++ b/doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md @@ -169,135 +169,6 @@ docker run \ Where `interface_ip_address` is your local network's interface IP, which you can find with the `ifconfig` command. The same would apply to GDK running with the instance address as `localhost` too. -## Guide to run and debug Monitor tests - -### How to set up - -To run the Monitor tests locally, against the GDK, please follow the preparation steps below: - -1. Complete the [Prerequisites](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/howto/auto_devops/index.md#prerequisites-for-gitlab-team-members-only), at least through step 5. Note that the monitor tests do not require permissions to work with GKE because they use [k3s as a Kubernetes cluster provider](https://github.com/rancher/k3s). -1. The test setup deploys the app in a Kubernetes cluster, using the Auto DevOps deployment strategy. -To enable Auto DevOps in GDK, follow the [associated setup](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/howto/auto_devops/index.md#setup) instructions. If you have problems, review the [troubleshooting guide](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/howto/auto_devops/tips_and_troubleshooting.md) or reach out to the `#gdk` channel in the internal GitLab Slack. -1. Do [secure your GitLab instance](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/howto/auto_devops/index.md#secure-your-gitlab-instance) since it is now publicly accessible on `https://[YOUR-PORT].qa-tunnel.gitlab.info`. -1. Install the Kubernetes command line tool known as `kubectl`. Use the [official installation instructions](https://kubernetes.io/docs/tasks/tools/). - -You might see NGINX issues when you run `gdk start` or `gdk restart`. In that case, run `sft login` to revalidate your credentials and regain access the QA Tunnel. - -### How to run - -Navigate to the folder in `/your-gdk/gitlab/qa` and issue the command: - -```shell -QA_DEBUG=true WEBDRIVER_HEADLESS=false GITLAB_ADMIN_USERNAME=rootusername GITLAB_ADMIN_PASSWORD=rootpassword GITLAB_QA_ACCESS_TOKEN=your_token_here GITLAB_QA_ADMIN_ACCESS_TOKEN=your_token_here CLUSTER_API_URL=https://kubernetes.docker.internal:6443 bundle exec bin/qa Test::Instance::All https://[YOUR-PORT].qa-tunnel.gitlab.info/ -- qa/specs/features/browser_ui/8_monitor/all_monitor_core_features_spec.rb --tag kubernetes --tag orchestrated --tag requires_admin -``` - -The following includes more information on the command: - -- `QA_DEBUG` - Set to `true` to verbosely log page object actions. -- `WEBDRIVER_HEADLESS` - When running locally, set to `false` to allow browser tests to be visible - watch your tests being run. -- `GITLAB_ADMIN_USERNAME` - Administrator username to use when adding a license. -- `GITLAB_ADMIN_PASSWORD` - Administrator password to use when adding a license. -- `GITLAB_QA_ACCESS_TOKEN` and `GITLAB_QA_ADMIN_ACCESS_TOKEN` - A valid personal access token with the `api` scope. This is used for API access during tests, and is used in the version that staging is currently running. The `ADMIN_ACCESS_TOKEN` is from a user with administrator access. Used for API access as an administrator during tests. -- `CLUSTER_API_URL` - Use the address `https://kubernetes.docker.internal:6443` . This address is used to enable the cluster to be network accessible while deploying using Auto DevOps. -- `https://[YOUR-PORT].qa-tunnel.gitlab.info/` - The address of your local GDK -- `qa/specs/features/browser_ui/8_monitor/all_monitor_core_features_spec.rb` - The path to the monitor core specs -- `--tag` - the meta-tags used to filter the specs correctly - -At the moment of this writing, there are two specs which run monitor tests: - -- `qa/specs/features/browser_ui/8_monitor/all_monitor_core_features_spec.rb` - has the specs of features in GitLab Free -- `qa/specs/features/ee/browser_ui/8_monitor/all_monitor_features_spec.rb` - has the specs of features for paid GitLab (Enterprise Edition) - -### How to debug - -The monitor tests follow this setup flow: - -1. Creates a k3s cluster on your local machine. -1. Creates a project that has Auto DevOps enabled and uses an Express template (NodeJS) for the app to be deployed. -1. Associates the created cluster to the project and installs GitLab Runner, Prometheus and Ingress which are the needed components for a successful deployment. -1. Creates a CI pipeline with 2 jobs (`build` and `production`) to deploy the app on the Kubernetes cluster. -1. Goes to Operation > Metrics menu to verify data is being received and the app is being monitored successfully. - -The test requires a number of components. The setup requires time to collect the metrics of a real deployment. -The complexity of the setup may lead to problems unrelated to the app. The following sections include common strategies to debug possible issues. - -#### Deployment with Auto DevOps - -When debugging issues in the CI or locally in the CLI, open the Kubernetes job in the pipeline. -In the job log window, click on the top right icon labeled as *"Show complete raw"* to reveal raw job logs. -You can now search through the logs for *Job log*, which matches delimited sections like this one: - -```shell -------- Job log: ------- -``` - -A Job log is a subsection within these logs, related to app deployment. We use two jobs: `build` and `production`. -You can find the root causes of deployment failures in these logs, which can compromise the entire test. -If a `build` job fails, the `production` job doesn't run, and the test fails. - -The long test setup does not take screenshots of failures, which is a known [issue](https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/270). -However, if the spec fails (after a successful deployment) then you should be able to find screenshots which display the feature failure. -To access them in CI, go to the main job log window, look on the left side panel's Job artifacts section, and click Browse. - -#### Common issues - -**Container Registry** - -When enabling Auto DevOps in the GDK, you may see issues with the Container Registry, which stores -images of the app to be deployed. - -You can access the Registry is available by opening an existing project. On the left hand menu, -select `Packages & Registries > Container Registries`. If the Registry is available, this page should load normally. - -Also, the Registry should be running in Docker: - -```shell -$ docker ps - -CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES -f035f339506c registry.gitlab.com/gitlab-org/build/cng/gitlab-container-registry:v2.9.1-gitlab "/bin/sh -c 'exec /b…" 3 hours ago Up 3 hours 0.0.0.0:5000->5000/tcp jovial_proskuriakova -``` - -The `gdk status` command shows if the registry is running: - -```shell -run: ./services/registry: (pid 2662) 10875s, normally down; run: log: (pid 65148) 177993s -run: ./services/tunnel_gitlab: (pid 2650) 10875s, normally down; run: log: (pid 65154) 177993s -run: ./services/tunnel_registry: (pid 2651) 10875s, normally down; run: log: (pid 65155) 177993s -``` - -Also, restarting Docker and then, on the Terminal, issue the command -`docker login https://[YOUR-REGISTRY-PORT].qa-tunnel.gitlab.info:443` and use -the GDK credentials to sign in. Note that the Registry port and GDK port aren't -the same. When configuring Auto DevOps in GDK, the `gdk reconfigure` command -outputs the port of the Registry: - -```shell -********************************************* -Tunnel URLs - -GitLab: https://[PORT].qa-tunnel.gitlab.info -Registry: https://[PORT].qa-tunnel.gitlab.info -********************************************* -``` - -These Tunnel URLs are used by the QA SSH Tunnel generated when enabling Auto DevOps on the GDK. - -**Pod Eviction** - -Pod eviction happens when a node in a Kubernetes cluster is running out of memory or disk. After many local deployments this issue can happen. The UI shows that installing Prometheus, GitLab Runner and Ingress failed. How to be sure it is an Eviction? While the test is running, open another Terminal window and debug the current Kubernetes cluster by `kubectl get pods --all-namespaces`. If you observe that Pods have *Evicted status* such as the install-runner here: - -```shell -$ kubectl get pods --all-namespaces - -NAMESPACE NAME READY STATUS RESTARTS AGE -gitlab-managed-apps install-ingress 0/1 Pending 0 25s -gitlab-managed-apps install-prometheus 0/1 Pending 0 12s -gitlab-managed-apps install-runner 0/1 Evicted 0 75s -``` - -You can free some memory with either of the following commands: `docker prune system` or `docker prune volume`. - ## Geo tests Geo end-to-end tests can run locally against a [Geo GDK setup](https://gitlab.com/gitlab-org/gitlab-development-kit/-/blob/main/doc/howto/geo.md) or on Geo spun up in Docker containers. diff --git a/doc/development/testing_guide/flaky_tests.md b/doc/development/testing_guide/flaky_tests.md index 09fc8f4d33e..3c8df7d3416 100644 --- a/doc/development/testing_guide/flaky_tests.md +++ b/doc/development/testing_guide/flaky_tests.md @@ -117,14 +117,17 @@ reproduction. - [Don't wait for AJAX when no AJAX request is fired](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/30461): <https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10454> - [Bis](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/34647): <https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/12626> -#### PhantomJS / WebKit related issues - -- Memory is through the roof! (Load images but block images requests!): <https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/12003> - #### Capybara expectation times out - [Test imports a project (via Sidekiq) that is growing over time, leading to timeouts when the import takes longer than 60 seconds](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/22599) +### Hanging specs + +If a spec hangs, it might be caused by a [bug in Rails](https://github.com/rails/rails/issues/34310): + +- <https://gitlab.com/gitlab-org/gitlab/-/merge_requests/81112> +- <https://gitlab.com/gitlab-org/gitlab/-/issues/337039> + ## Resources - [Flaky Tests: Are You Sure You Want to Rerun Them?](https://semaphoreci.com/blog/2017/04/20/flaky-tests.html) diff --git a/doc/development/uploads.md b/doc/development/uploads.md index 14ecf38e21c..1860f898a26 100644 --- a/doc/development/uploads.md +++ b/doc/development/uploads.md @@ -1,430 +1,9 @@ --- -stage: none -group: unassigned -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +redirect_to: 'uploads/index.md' +remove_date: '2022-04-30' --- -# Uploads development documentation +This document was moved to [another location](uploads/index.md). -[GitLab Workhorse](https://gitlab.com/gitlab-org/gitlab-workhorse) has special rules for handling uploads. -We process the upload in Workhorse to prevent occupying a Ruby process on I/O operations and because it is cheaper. -This process can also directly upload to object storage. - -## The problem description - -The following graph explains machine boundaries in a scalable GitLab installation. Without any Workhorse optimization in place, we can expect incoming requests to follow the numbers on the arrows. - -```mermaid -graph TB - subgraph "load balancers" - LB(Proxy) - end - - subgraph "Shared storage" - nfs(NFS) - end - - subgraph "redis cluster" - r(persisted redis) - end - LB-- 1 -->Workhorse - - subgraph "web or API fleet" - Workhorse-- 2 -->rails - end - rails-- "3 (write files)" -->nfs - rails-- "4 (schedule a job)" -->r - - subgraph sidekiq - s(sidekiq) - end - s-- "5 (fetch a job)" -->r - s-- "6 (read files)" -->nfs -``` - -We have three challenges here: performance, availability, and scalability. - -### Performance - -Rails process are expensive in terms of both CPU and memory. Ruby [global interpreter lock](https://en.wikipedia.org/wiki/Global_interpreter_lock) adds to cost too because the Ruby process spends time on I/O operations on step 3 causing incoming requests to pile up. - -In order to improve this, [disk buffered upload](#disk-buffered-upload) was implemented. With this, Rails no longer deals with writing uploaded files to disk. - -```mermaid -graph TB - subgraph "load balancers" - LB(HA Proxy) - end - - subgraph "Shared storage" - nfs(NFS) - end - - subgraph "redis cluster" - r(persisted redis) - end - LB-- 1 -->Workhorse - - subgraph "web or API fleet" - Workhorse-- "3 (without files)" -->rails - end - Workhorse -- "2 (write files)" -->nfs - rails-- "4 (schedule a job)" -->r - - subgraph sidekiq - s(sidekiq) - end - s-- "5 (fetch a job)" -->r - s-- "6 (read files)" -->nfs -``` - -### Availability - -There's also an availability problem in this setup, NFS is a [single point of failure](https://en.wikipedia.org/wiki/Single_point_of_failure). - -To address this problem an HA object storage can be used and it's supported by [direct upload](#direct-upload) - -### Scalability - -Scaling NFS is outside of our support scope, and NFS is not a part of cloud native installations. - -All features that require Sidekiq and do not use direct upload doesn't work without NFS. In Kubernetes, machine boundaries translate to PODs, and in this case the uploaded file is written into the POD private disk. Since Sidekiq POD cannot reach into other pods, the operation fails to read it. - -## How to select the proper level of acceleration? - -Selecting the proper acceleration is a tradeoff between speed of development and operational costs. - -We can identify three major use-cases for an upload: - -1. **storage:** if we are uploading for storing a file (like artifacts, packages, or discussion attachments). In this case [direct upload](#direct-upload) is the proper level as it's the less resource-intensive operation. Additional information can be found on [File Storage in GitLab](file_storage.md). -1. **in-controller/synchronous processing:** if we allow processing **small files** synchronously, using [disk buffered upload](#disk-buffered-upload) may speed up development. -1. **Sidekiq/asynchronous processing:** Asynchronous processing must implement [direct upload](#direct-upload), the reason being that it's the only way to support Cloud Native deployments without a shared NFS. - -For more details about currently broken feature see [epic &1802](https://gitlab.com/groups/gitlab-org/-/epics/1802). - -### Handling repository uploads - -Some features involves Git repository uploads without using a regular Git client. -Some examples are uploading a repository file from the web interface and [design management](../user/project/issues/design_management.md). - -Those uploads requires the rails controller to act as a Git client in lieu of the user. -Those operation falls into _in-controller/synchronous processing_ category, but we have no warranties on the file size. - -In case of a LFS upload, the file pointer is committed synchronously, but file upload to object storage is performed asynchronously with Sidekiq. - -## Upload encodings - -By upload encoding we mean how the file is included within the incoming request. - -We have three kinds of file encoding in our uploads: - -1. <i class="fa fa-check-circle"></i> **multipart**: `multipart/form-data` is the most common, a file is encoded as a part of a multipart encoded request. -1. <i class="fa fa-check-circle"></i> **body**: some APIs uploads files as the whole request body. -1. <i class="fa fa-times-circle"></i> **JSON**: some JSON APIs upload files as base64-encoded strings. This requires a change to GitLab Workhorse, - which is tracked [in this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/325068). - -## Uploading technologies - -By uploading technologies we mean how all the involved services interact with each other. - -GitLab supports 3 kinds of uploading technologies, here follows a brief description with a sequence diagram for each one. Diagrams are not meant to be exhaustive. - -### Rack Multipart upload - -This is the default kind of upload, and it's the most expensive in terms of resources. - -In this case, Workhorse is unaware of files being uploaded and acts as a regular proxy. - -When a multipart request reaches the rails application, `Rack::Multipart` leaves behind temporary files in `/tmp` and uses valuable Ruby process time to copy files around. - -```mermaid -sequenceDiagram - participant c as Client - participant w as Workhorse - participant r as Rails - - activate c - c ->>+w: POST /some/url/upload - w->>+r: POST /some/url/upload - - r->>r: save the incoming file on /tmp - r->>r: read the file for processing - - r-->>-c: request result - deactivate c - deactivate w -``` - -### Disk buffered upload - -This kind of upload avoids wasting resources caused by handling upload writes to `/tmp` in rails. - -This optimization is not active by default on REST API requests. - -When enabled, Workhorse looks for files in multipart MIME requests, uploading -any it finds to a temporary file on shared storage. The MIME data in the request -is replaced with the path to the corresponding file before it is forwarded to -Rails. - -To prevent abuse of this feature, Workhorse signs the modified request with a -special header, stating which entries it modified. Rails ignores any -unsigned path entries. - -```mermaid -sequenceDiagram - participant c as Client - participant w as Workhorse - participant r as Rails - participant s as NFS - - activate c - c ->>+w: POST /some/url/upload - - w->>+s: save the incoming file on a temporary location - s-->>-w: request result - - w->>+r: POST /some/url/upload - Note over w,r: file was replaced with its location<br>and other metadata - - opt requires async processing - r->>+redis: schedule a job - redis-->>-r: job is scheduled - end - - r-->>-c: request result - deactivate c - w->>-w: cleanup - - opt requires async processing - activate sidekiq - sidekiq->>+redis: fetch a job - redis-->>-sidekiq: job - - sidekiq->>+s: read file - s-->>-sidekiq: file - - sidekiq->>sidekiq: process file - - deactivate sidekiq - end -``` - -### Direct upload - -This is the more advanced acceleration technique we have in place. - -Workhorse asks Rails for temporary pre-signed object storage URLs and directly uploads to object storage. - -In this setup, an extra Rails route must be implemented in order to handle authorization. Examples of this can be found in: - -- [`Projects::LfsStorageController`](https://gitlab.com/gitlab-org/gitlab/-/blob/cc723071ad337573e0360a879cbf99bc4fb7adb9/app/controllers/projects/lfs_storage_controller.rb) - and [its routes](https://gitlab.com/gitlab-org/gitlab/-/blob/cc723071ad337573e0360a879cbf99bc4fb7adb9/config/routes/git_http.rb#L31-32). -- [API endpoints for uploading packages](packages.md#file-uploads). - -Direct upload falls back to _disk buffered upload_ when `direct_upload` is disabled inside the [object storage setting](../administration/uploads.md#object-storage-settings). -The answer to the `/authorize` call contains only a file system path. - -```mermaid -sequenceDiagram - participant c as Client - participant w as Workhorse - participant r as Rails - participant os as Object Storage - - activate c - c ->>+w: POST /some/url/upload - - w ->>+r: POST /some/url/upload/authorize - Note over w,r: this request has an empty body - r-->>-w: presigned OS URL - - w->>+os: PUT file - Note over w,os: file is stored on a temporary location. Rails select the destination - os-->>-w: request result - - w->>+r: POST /some/url/upload - Note over w,r: file was replaced with its location<br>and other metadata - - r->>+os: move object to final destination - os-->>-r: request result - - opt requires async processing - r->>+redis: schedule a job - redis-->>-r: job is scheduled - end - - r-->>-c: request result - deactivate c - w->>-w: cleanup - - opt requires async processing - activate sidekiq - sidekiq->>+redis: fetch a job - redis-->>-sidekiq: job - - sidekiq->>+os: get object - os-->>-sidekiq: file - - sidekiq->>sidekiq: process file - - deactivate sidekiq - end -``` - -## How to add a new upload route - -In this section, we describe how to add a new upload route [accelerated](#uploading-technologies) by Workhorse for [body and multipart](#upload-encodings) encoded uploads. - -Upload routes belong to one of these categories: - -1. Rails controllers: uploads handled by Rails controllers. -1. Grape API: uploads handled by a Grape API endpoint. -1. GraphQL API: uploads handled by a GraphQL resolve function. - -WARNING: -GraphQL uploads do not support [direct upload](#direct-upload) yet. Depending on the use case, the feature may not work on installations without NFS (like GitLab.com or Kubernetes installations). Uploading to object storage inside the GraphQL resolve function may result in timeout errors. For more details please follow [issue #280819](https://gitlab.com/gitlab-org/gitlab/-/issues/280819). - -### Update Workhorse for the new route - -For both the Rails controller and Grape API uploads, Workhorse has to be updated in order to get the -support for the new upload route. - -1. Open a new issue in the [Workhorse tracker](https://gitlab.com/gitlab-org/gitlab-workhorse/-/issues/new) describing precisely the new upload route: - - The route's URL. - - The [upload encoding](#upload-encodings). - - If possible, provide a dump of the upload request. -1. Implement and get the MR merged for this issue above. -1. Ask the Maintainers of [Workhorse](https://gitlab.com/gitlab-org/gitlab-workhorse) to create a new release. You can do that in the MR - directly during the maintainer review or ask for it in the `#workhorse` Slack channel. -1. Bump the [Workhorse version file](https://gitlab.com/gitlab-org/gitlab/-/blob/master/GITLAB_WORKHORSE_VERSION) - to the version you have from the previous points, or bump it in the same merge request that contains - the Rails changes (see [Implementing the new route with a Rails controller](#implementing-the-new-route-with-a-rails-controller) or [Implementing the new route with a Grape API endpoint](#implementing-the-new-route-with-a-grape-api-endpoint) below). - -### Implementing the new route with a Rails controller - -For a Rails controller upload, we usually have a [multipart](#upload-encodings) upload and there are a -few things to do: - -1. The upload is available under the parameter name you're using. For example, it could be an `artifact` - or a nested parameter such as `user[avatar]`. Let's say that we have the upload under the - `file` parameter, reading `params[:file]` should get you an [`UploadedFile`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/uploaded_file.rb) instance. -1. Generally speaking, it's a good idea to check if the instance is from the [`UploadedFile`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/uploaded_file.rb) class. For example, see how we checked -[that the parameter is indeed an `UploadedFile`](https://gitlab.com/gitlab-org/gitlab/-/commit/ea30fe8a71bf16ba07f1050ab4820607b5658719#51c0cc7a17b7f12c32bc41cfab3649ff2739b0eb_79_77). - -WARNING: -**Do not** call `UploadedFile#from_params` directly! Do not build an [`UploadedFile`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/uploaded_file.rb) -instance using `UploadedFile#from_params`! This method can be unsafe to use depending on the `params` -passed. Instead, use the [`UploadedFile`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/uploaded_file.rb) -instance that [`multipart.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/middleware/multipart.rb) -builds automatically for you. - -### Implementing the new route with a Grape API endpoint - -For a Grape API upload, we can have [body or a multipart](#upload-encodings) upload. Things are slightly more complicated: two endpoints are needed. One for the -Workhorse pre-upload authorization and one for accepting the upload metadata from Workhorse: - -1. Implement an endpoint with the URL + `/authorize` suffix that will: - - Check that the request is coming from Workhorse with the `require_gitlab_workhorse!` from the [API helpers](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/helpers.rb). - - Check user permissions. - - Set the status to `200` with `status 200`. - - Set the content type with `content_type Gitlab::Workhorse::INTERNAL_API_CONTENT_TYPE`. - - Use your dedicated `Uploader` class (let's say that it's `FileUploader`) to build the response with `FileUploader.workhorse_authorize(params)`. -1. Implement the endpoint for the upload request that will: - - Require all the `UploadedFile` objects as parameters. - - For example, if we expect a single parameter `file` to be an [`UploadedFile`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/uploaded_file.rb) instance, -use `requires :file, type: ::API::Validations::Types::WorkhorseFile`. - - Body upload requests have their upload available under the parameter `file`. - - Check that the request is coming from Workhorse with the `require_gitlab_workhorse!` from the -[API helpers](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/helpers.rb). - - Check the user permissions. - - The remaining code of the processing. This is where the code must be reading the parameter (for -our example, it would be `params[:file]`). - -WARNING: -**Do not** call `UploadedFile#from_params` directly! Do not build an [`UploadedFile`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/uploaded_file.rb) -object using `UploadedFile#from_params`! This method can be unsafe to use depending on the `params` -passed. Instead, use the [`UploadedFile`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/uploaded_file.rb) -object that [`multipart.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/middleware/multipart.rb) -builds automatically for you. - -### Document Object Storage buckets and CarrierWave integration - -When using Object Storage, GitLab expects each kind of upload to maintain its own bucket in the respective -Object Storage destination. Moreover, the integration with CarrierWave is not used all the time. -The [Object Storage Working Group](https://about.gitlab.com/company/team/structure/working-groups/object-storage/) -is investigating an approach that unifies Object Storage buckets into a single one and removes CarrierWave -so as to simplify implementation and administration of uploads. - -Therefore, document new uploads here by slotting them into the following tables: - -- [Feature bucket details](#feature-bucket-details) -- [CarrierWave integration](#carrierwave-integration) - -#### Feature bucket details - -| Feature | Upload technology | Uploader | Bucket structure | -|------------------------------------------|-------------------|-----------------------|-----------------------------------------------------------------------------------------------------------| -| Job artifacts | `direct upload` | `workhorse` | `/artifacts/<proj_id_hash>/<date>/<job_id>/<artifact_id>` | -| Pipeline artifacts | `carrierwave` | `sidekiq` | `/artifacts/<proj_id_hash>/pipelines/<pipeline_id>/artifacts/<artifact_id>` | -| Live job traces | `fog` | `sidekiq` | `/artifacts/tmp/builds/<job_id>/chunks/<chunk_index>.log` | -| Job traces archive | `carrierwave` | `sidekiq` | `/artifacts/<proj_id_hash>/<date>/<job_id>/<artifact_id>/job.log` | -| Autoscale runner caching | N/A | `gitlab-runner` | `/gitlab-com-[platform-]runners-cache/???` | -| Backups | N/A | `s3cmd`, `awscli`, or `gcs` | `/gitlab-backups/???` | -| Git LFS | `direct upload` | `workhorse` | `/lsf-objects/<lfs_obj_oid[0:2]>/<lfs_obj_oid[2:2]>` | -| Design management files | `disk buffering` | `rails controller` | `/lsf-objects/<lfs_obj_oid[0:2]>/<lfs_obj_oid[2:2]>` | -| Design management thumbnails | `carrierwave` | `sidekiq` | `/uploads/design_management/action/image_v432x230/<model_id>` | -| Generic file uploads | `direct upload` | `workhorse` | `/uploads/@hashed/[0:2]/[2:4]/<hash1>/<hash2>/file` | -| Generic file uploads - personal snippets | `direct upload` | `workhorse` | `/uploads/personal_snippet/<snippet_id>/<filename>` | -| Global appearance settings | `disk buffering` | `rails controller` | `/uploads/appearance/...` | -| Topics | `disk buffering` | `rails controller` | `/uploads/projects/topic/...` | -| Avatar images | `direct upload` | `workhorse` | `/uploads/[user,group,project]/avatar/<model_id>` | -| Import/export | `direct upload` | `workhorse` | `/uploads/import_export_upload/???` | -| GitLab Migration | `carrierwave` | `sidekiq` | `/uploads/bulk_imports/???` | -| MR diffs | `carrierwave` | `sidekiq` | `/external-diffs/merge_request_diffs/mr-<mr_id>/diff-<diff_id>` | -| Package manager archives | `direct upload` | `sidekiq` | `/packages/<proj_id_hash>/packages/<pkg_segment>/files/<pkg_file_id>` | -| Package manager archives | `direct upload` | `sidekiq` | `/packages/<container_id_hash>/debian_*_component_file/<component_file_id>` | -| Package manager archives | `direct upload` | `sidekiq` | `/packages/<container_id_hash>/debian_*_distribution/<distribution_file_id>` | -| Container image cache (?) | `direct upload` | `workhorse` | `/dependency-proxy/<group_id_hash>/dependency_proxy/<group_id>/files/<proxy_id>/<blob_id or manifest_id>` | -| Terraform state files | `carrierwave` | `rails controller` | `/terraform/<proj_id_hash>/<terraform_state_id>` | -| Pages content archives | `carrierwave` | `sidekiq` | `/gitlab-gprd-pages/<proj_id_hash>/pages_deployments/<deployment_id>/` | -| Secure Files | `carrierwave` | `sidekiq` | `/ci-secure-files/<proj_id_hash>/secure_files/<secure_file_id>/` | - -#### CarrierWave integration - -| File | Carrierwave usage | Categorized | -|---------------------------------------------------------|----------------------------------------------------------------------------------|---------------------| -| `app/models/project.rb` | `include Avatarable` | :white_check_mark: | -| `app/models/projects/topic.rb` | `include Avatarable` | :white_check_mark: | -| `app/models/group.rb` | `include Avatarable` | :white_check_mark: | -| `app/models/user.rb` | `include Avatarable` | :white_check_mark: | -| `app/models/terraform/state_version.rb` | `include FileStoreMounter` | :white_check_mark: | -| `app/models/ci/job_artifact.rb` | `include FileStoreMounter` | :white_check_mark: | -| `app/models/ci/pipeline_artifact.rb` | `include FileStoreMounter` | :white_check_mark: | -| `app/models/pages_deployment.rb` | `include FileStoreMounter` | :white_check_mark: | -| `app/models/lfs_object.rb` | `include FileStoreMounter` | :white_check_mark: | -| `app/models/dependency_proxy/blob.rb` | `include FileStoreMounter` | :white_check_mark: | -| `app/models/dependency_proxy/manifest.rb` | `include FileStoreMounter` | :white_check_mark: | -| `app/models/packages/composer/cache_file.rb` | `include FileStoreMounter` | :white_check_mark: | -| `app/models/packages/package_file.rb` | `include FileStoreMounter` | :white_check_mark: | -| `app/models/concerns/packages/debian/component_file.rb` | `include FileStoreMounter` | :white_check_mark: | -| `ee/app/models/issuable_metric_image.rb` | `include FileStoreMounter` | | -| `ee/app/models/vulnerabilities/remediation.rb` | `include FileStoreMounter` | | -| `ee/app/models/vulnerabilities/export.rb` | `include FileStoreMounter` | | -| `app/models/packages/debian/project_distribution.rb` | `include Packages::Debian::Distribution` | :white_check_mark: | -| `app/models/packages/debian/group_distribution.rb` | `include Packages::Debian::Distribution` | :white_check_mark: | -| `app/models/packages/debian/project_component_file.rb` | `include Packages::Debian::ComponentFile` | :white_check_mark: | -| `app/models/packages/debian/group_component_file.rb` | `include Packages::Debian::ComponentFile` | :white_check_mark: | -| `app/models/merge_request_diff.rb` | `mount_uploader :external_diff, ExternalDiffUploader` | :white_check_mark: | -| `app/models/note.rb` | `mount_uploader :attachment, AttachmentUploader` | :white_check_mark: | -| `app/models/appearance.rb` | `mount_uploader :logo, AttachmentUploader` | :white_check_mark: | -| `app/models/appearance.rb` | `mount_uploader :header_logo, AttachmentUploader` | :white_check_mark: | -| `app/models/appearance.rb` | `mount_uploader :favicon, FaviconUploader` | :white_check_mark: | -| `app/models/project.rb` | `mount_uploader :bfg_object_map, AttachmentUploader` | | -| `app/models/import_export_upload.rb` | `mount_uploader :import_file, ImportExportUploader` | :white_check_mark: | -| `app/models/import_export_upload.rb` | `mount_uploader :export_file, ImportExportUploader` | :white_check_mark: | -| `app/models/ci/deleted_object.rb` | `mount_uploader :file, DeletedObjectUploader` | | -| `app/models/design_management/action.rb` | `mount_uploader :image_v432x230, DesignManagement::DesignV432x230Uploader` | :white_check_mark: | -| `app/models/concerns/packages/debian/distribution.rb` | `mount_uploader :signed_file, Packages::Debian::DistributionReleaseFileUploader` | :white_check_mark: | -| `app/models/bulk_imports/export_upload.rb` | `mount_uploader :export_file, ExportUploader` | :white_check_mark: | -| `ee/app/models/user_permission_export_upload.rb` | `mount_uploader :file, AttachmentUploader` | | -| `app/models/ci/secure_file.rb` | `include FileStoreMounter` | | +<!-- This redirect file can be deleted after 2022-04-30. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/development/uploads/background.md b/doc/development/uploads/background.md new file mode 100644 index 00000000000..e68e4127b57 --- /dev/null +++ b/doc/development/uploads/background.md @@ -0,0 +1,154 @@ +--- +stage: none +group: unassigned +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Uploads guide: Why GitLab uses custom upload logic + +This page is for developers trying to better understand the history behind GitLab uploads and the +technical challenges associated with uploads. + +## Problem description + +GitLab and [GitLab Workhorse](https://gitlab.com/gitlab-org/gitlab-workhorse) use special rules for handling file uploads, +because in an ordinary Rails application file uploads can become expensive as files grow in size. +Rails often sacrifices performance to provide a better developer experience, including how it handles +`multipart/form-post` uploads. In any Rack server, Rails applications included, when such a request arrives at the application server, +several things happen: + +1. A [Rack middleware](https://github.com/rack/rack/blob/main/lib/rack/multipart.rb) intercepts the request and parses the request body. +1. The middleware writes each file in the multipart request to a temporary directory on disk. +1. A `params` hash is constructed with entries pointing to the respective files on disk. +1. A Rails controller acts on the file contents. + +While this is convenient for developers, it is costly for the Ruby server process to buffer large files on disk. +Because of Ruby's [global interpreter lock](https://en.wikipedia.org/wiki/Global_interpreter_lock), +only a single thread of execution of a given Ruby process can be on CPU. This means the amount of CPU +time spent doing this is not available to other worker threads serving user requests. +Buffering files to disk also means spending more time in I/O routines and mode switches, which are expensive operations. + +The following diagram shows how GitLab handled such a request prior to putting optimizations in place. + +```mermaid +graph TB + subgraph "load balancers" + LB(Proxy) + end + + subgraph "Shared storage" + nfs(NFS) + end + + subgraph "redis cluster" + r(persisted redis) + end + LB-- 1 -->Workhorse + + subgraph "web or API fleet" + Workhorse-- 2 -->rails + end + rails-- "3 (write files)" -->nfs + rails-- "4 (schedule a job)" -->r + + subgraph sidekiq + s(sidekiq) + end + s-- "5 (fetch a job)" -->r + s-- "6 (read files)" -->nfs +``` + +We went through two major iterations of our uploads architecture to improve on these problems: + +1. [Moving disk buffering to Workhorse.](#moving-disk-buffering-to-workhorse) +1. [Uploading to Object Storage from Workhorse.](#moving-to-object-storage-and-direct-uploads) + +### Moving disk buffering to Workhorse + +To address the performance issues resulting from buffering files in Ruby, we moved this logic to Workhorse instead, +our reverse proxy fronting the GitLab Rails application. +Workhorse is written in Go, and is much better at dealing with stream processing and I/O than Rails. + +There are two parts to this implementation: + +1. In Workhorse, a request handler detects `multipart/form-data` content in an incoming user request. + If such a request is detected, Workhorse hijacks the request body before forwarding it to Rails. + Workhorse writes all files to disk, rewrites the multipart form fields to point to the new locations, signs the + request, then forwards it to Rails. +1. In Rails, a [custom multipart Rack middleware](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/middleware/multipart.rb) + identifies any signed multipart requests coming from Workhorse and prepares the `params` hash Rails + would expect, now pointing to the files cached by Workhorse. This makes it a drop-in replacement for `Rack::Multipart`. + +The diagram below shows how GitLab handles such a request today: + +```mermaid +graph TB + subgraph "load balancers" + LB(HA Proxy) + end + + subgraph "Shared storage" + nfs(NFS) + end + + subgraph "redis cluster" + r(persisted redis) + end + LB-- 1 -->Workhorse + + subgraph "web or API fleet" + Workhorse-- "3 (without files)" -->rails + end + Workhorse -- "2 (write files)" -->nfs + rails-- "4 (schedule a job)" -->r + + subgraph sidekiq + s(sidekiq) + end + s-- "5 (fetch a job)" -->r + s-- "6 (read files)" -->nfs +``` + +While this "one-size-fits-all" solution greatly improves performance for multipart uploads without compromising +developer ergonomics, it severely limits GitLab [availability](#availability-challenges) +and [scalability](#scalability-challenges). + +#### Availability challenges + +Moving file buffering to Workhorse addresses the immediate performance problems stemming from Ruby not being good at +handling large file uploads. However, a remaining issue of this solution is its reliance on attached storage, +whether via ordinary hard drives or network attached storage like NFS. +NFS is a [single point of failure](https://en.wikipedia.org/wiki/Single_point_of_failure), and is unsuitable for +deploying GitLab in highly available, cloud native environments. + +#### Scalability challenges + +NFS is not a part of cloud native installations, such as those running in Kubernetes. +In Kubernetes, machine boundaries translate to pods, and without network-attached storage, disk-buffered uploads +must be written directly to the pod's file system. + +Using disk buffering presents us with a scalability challenge here. If Workhorse can only +write files to a pod's private file system, then these files are inaccessible outside of this particular pod. +With disk buffering, a Rails controller will accept a file upload and enqueue it for upload in a Sidekiq +background job. Therefore, Sidekiq requires access to these files. +However, in a cloud native environment all Sidekiq instances run on separate pods, so they are +not able to access files buffered to disk on a web server pod. + +Therefore, all features that involve Sidekiq uploading disk-buffered files severely limit the scalability of GitLab. + +## Moving to object storage and direct uploads + +To address these availability and scalability problems, +instead of buffering files to disk, we have added support for uploading files directly +from Workhorse to a given destination. While it remains possible to upload to local or network-attached storage +this way, you should use a highly available +[object store](https://en.wikipedia.org/wiki/Object_storage), +such as AWS S3, Google GCS, or Azure, for scalability reasons. + +With direct uploads, Workhorse does not buffer files to disk. Instead, it first authorizes the request with +the Rails application to find out where to upload it, then streams the file directly to its ultimate destination. + +To learn more about how disk buffering and direct uploads are implemented, see: + +- [How uploads work technically](implementation.md) +- [Adding new uploads](working_with_uploads.md) diff --git a/doc/development/uploads/implementation.md b/doc/development/uploads/implementation.md new file mode 100644 index 00000000000..13a875cd1af --- /dev/null +++ b/doc/development/uploads/implementation.md @@ -0,0 +1,190 @@ +--- +stage: none +group: unassigned +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Uploads guide: How uploads work technically + +This page is for developers trying to better understand what kinds of uploads exist in GitLab and how they are implemented. + +## Kinds of uploads and how to choose between them + +We can identify three major use-cases for an upload: + +1. **storage:** if we are uploading for storing a file (like artifacts, packages, or discussion attachments). In this case [direct upload](#direct-upload) is the proper level as it's the less resource-intensive operation. Additional information can be found on [File Storage in GitLab](../file_storage.md). +1. **in-controller/synchronous processing:** if we allow processing **small files** synchronously, using [disk buffered upload](#disk-buffered-upload) may speed up development. +1. **Sidekiq/asynchronous processing:** Asynchronous processing must implement [direct upload](#direct-upload), the reason being that it's the only way to support Cloud Native deployments without a shared NFS. + +Selecting the proper acceleration is a tradeoff between speed of development and operational costs. + +For more details about currently broken feature see [epic &1802](https://gitlab.com/groups/gitlab-org/-/epics/1802). + +### Handling repository uploads + +Some features involves Git repository uploads without using a regular Git client. +Some examples are uploading a repository file from the web interface and [design management](../../user/project/issues/design_management.md). + +Those uploads requires the rails controller to act as a Git client in lieu of the user. +Those operation falls into _in-controller/synchronous processing_ category, but we have no warranties on the file size. + +In case of a LFS upload, the file pointer is committed synchronously, but file upload to object storage is performed asynchronously with Sidekiq. + +## Upload encodings + +By upload encoding we mean how the file is included within the incoming request. + +We have three kinds of file encoding in our uploads: + +1. <i class="fa fa-check-circle"></i> **multipart**: `multipart/form-data` is the most common, a file is encoded as a part of a multipart encoded request. +1. <i class="fa fa-check-circle"></i> **body**: some APIs uploads files as the whole request body. +1. <i class="fa fa-times-circle"></i> **JSON**: some JSON APIs upload files as base64-encoded strings. This requires a change to GitLab Workhorse, + which is tracked [in this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/325068). + +## Uploading technologies + +By uploading technologies we mean how all the involved services interact with each other. + +GitLab supports 3 kinds of uploading technologies, here follows a brief description with a sequence diagram for each one. Diagrams are not meant to be exhaustive. + +### Rack Multipart upload + +This is the default kind of upload, and it's the most expensive in terms of resources. + +In this case, Workhorse is unaware of files being uploaded and acts as a regular proxy. + +When a multipart request reaches the rails application, `Rack::Multipart` leaves behind temporary files in `/tmp` and uses valuable Ruby process time to copy files around. + +```mermaid +sequenceDiagram + participant c as Client + participant w as Workhorse + participant r as Rails + + activate c + c ->>+w: POST /some/url/upload + w->>+r: POST /some/url/upload + + r->>r: save the incoming file on /tmp + r->>r: read the file for processing + + r-->>-c: request result + deactivate c + deactivate w +``` + +### Disk buffered upload + +This kind of upload avoids wasting resources caused by handling upload writes to `/tmp` in rails. + +This optimization is not active by default on REST API requests. + +When enabled, Workhorse looks for files in multipart MIME requests, uploading +any it finds to a temporary file on shared storage. The MIME data in the request +is replaced with the path to the corresponding file before it is forwarded to +Rails. + +To prevent abuse of this feature, Workhorse signs the modified request with a +special header, stating which entries it modified. Rails ignores any +unsigned path entries. + +```mermaid +sequenceDiagram + participant c as Client + participant w as Workhorse + participant r as Rails + participant s as NFS + + activate c + c ->>+w: POST /some/url/upload + + w->>+s: save the incoming file on a temporary location + s-->>-w: request result + + w->>+r: POST /some/url/upload + Note over w,r: file was replaced with its location<br>and other metadata + + opt requires async processing + r->>+redis: schedule a job + redis-->>-r: job is scheduled + end + + r-->>-c: request result + deactivate c + w->>-w: cleanup + + opt requires async processing + activate sidekiq + sidekiq->>+redis: fetch a job + redis-->>-sidekiq: job + + sidekiq->>+s: read file + s-->>-sidekiq: file + + sidekiq->>sidekiq: process file + + deactivate sidekiq + end +``` + +### Direct upload + +This is the more advanced acceleration technique we have in place. + +Workhorse asks Rails for temporary pre-signed object storage URLs and directly uploads to object storage. + +In this setup, an extra Rails route must be implemented in order to handle authorization. Examples of this can be found in: + +- [`Projects::LfsStorageController`](https://gitlab.com/gitlab-org/gitlab/-/blob/cc723071ad337573e0360a879cbf99bc4fb7adb9/app/controllers/projects/lfs_storage_controller.rb) + and [its routes](https://gitlab.com/gitlab-org/gitlab/-/blob/cc723071ad337573e0360a879cbf99bc4fb7adb9/config/routes/git_http.rb#L31-32). +- [API endpoints for uploading packages](../packages.md#file-uploads). + +Direct upload falls back to _disk buffered upload_ when `direct_upload` is disabled inside the [object storage setting](../../administration/uploads.md#object-storage-settings). +The answer to the `/authorize` call contains only a file system path. + +```mermaid +sequenceDiagram + participant c as Client + participant w as Workhorse + participant r as Rails + participant os as Object Storage + + activate c + c ->>+w: POST /some/url/upload + + w ->>+r: POST /some/url/upload/authorize + Note over w,r: this request has an empty body + r-->>-w: presigned OS URL + + w->>+os: PUT file + Note over w,os: file is stored on a temporary location. Rails select the destination + os-->>-w: request result + + w->>+r: POST /some/url/upload + Note over w,r: file was replaced with its location<br>and other metadata + + r->>+os: move object to final destination + os-->>-r: request result + + opt requires async processing + r->>+redis: schedule a job + redis-->>-r: job is scheduled + end + + r-->>-c: request result + deactivate c + w->>-w: cleanup + + opt requires async processing + activate sidekiq + sidekiq->>+redis: fetch a job + redis-->>-sidekiq: job + + sidekiq->>+os: get object + os-->>-sidekiq: file + + sidekiq->>sidekiq: process file + + deactivate sidekiq + end +``` diff --git a/doc/development/uploads/index.md b/doc/development/uploads/index.md new file mode 100644 index 00000000000..c486f2d3689 --- /dev/null +++ b/doc/development/uploads/index.md @@ -0,0 +1,14 @@ +--- +stage: none +group: unassigned +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Uploads development guide + +Uploads are an integral part of many GitLab features. To understand how GitLab handles uploads, refer to +the following pages: + +- [Why GitLab uses custom upload logic.](background.md) +- [How uploads work technically.](implementation.md) +- [How to add new uploads.](working_with_uploads.md) diff --git a/doc/development/uploads/working_with_uploads.md b/doc/development/uploads/working_with_uploads.md new file mode 100644 index 00000000000..99c04888804 --- /dev/null +++ b/doc/development/uploads/working_with_uploads.md @@ -0,0 +1,163 @@ +--- +stage: none +group: unassigned +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Uploads guide: Adding new uploads + +In this section, we describe how to add a new upload route [accelerated](implementation.md#uploading-technologies) by Workhorse for [body and multipart](implementation.md#upload-encodings) encoded uploads. + +Upload routes belong to one of these categories: + +1. Rails controllers: uploads handled by Rails controllers. +1. Grape API: uploads handled by a Grape API endpoint. +1. GraphQL API: uploads handled by a GraphQL resolve function. + +WARNING: +GraphQL uploads do not support [direct upload](implementation.md#direct-upload) yet. Depending on the use case, the feature may not work on installations without NFS (like GitLab.com or Kubernetes installations). Uploading to object storage inside the GraphQL resolve function may result in timeout errors. For more details please follow [issue #280819](https://gitlab.com/gitlab-org/gitlab/-/issues/280819). + +## Update Workhorse for the new route + +For both the Rails controller and Grape API uploads, Workhorse has to be updated in order to get the +support for the new upload route. + +1. Open a new issue in the [Workhorse tracker](https://gitlab.com/gitlab-org/gitlab-workhorse/-/issues/new) describing precisely the new upload route: + - The route's URL. + - The [upload encoding](implementation.md#upload-encodings). + - If possible, provide a dump of the upload request. +1. Implement and get the MR merged for this issue above. +1. Ask the Maintainers of [Workhorse](https://gitlab.com/gitlab-org/gitlab-workhorse) to create a new release. You can do that in the MR + directly during the maintainer review or ask for it in the `#workhorse` Slack channel. +1. Bump the [Workhorse version file](https://gitlab.com/gitlab-org/gitlab/-/blob/master/GITLAB_WORKHORSE_VERSION) + to the version you have from the previous points, or bump it in the same merge request that contains + the Rails changes (see [Implementing the new route with a Rails controller](#implementing-the-new-route-with-a-rails-controller) or [Implementing the new route with a Grape API endpoint](#implementing-the-new-route-with-a-grape-api-endpoint) below). + +## Implementing the new route with a Rails controller + +For a Rails controller upload, we usually have a [multipart](implementation.md#upload-encodings) upload and there are a +few things to do: + +1. The upload is available under the parameter name you're using. For example, it could be an `artifact` + or a nested parameter such as `user[avatar]`. Let's say that we have the upload under the + `file` parameter, reading `params[:file]` should get you an [`UploadedFile`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/uploaded_file.rb) instance. +1. Generally speaking, it's a good idea to check if the instance is from the [`UploadedFile`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/uploaded_file.rb) class. For example, see how we checked +[that the parameter is indeed an `UploadedFile`](https://gitlab.com/gitlab-org/gitlab/-/commit/ea30fe8a71bf16ba07f1050ab4820607b5658719#51c0cc7a17b7f12c32bc41cfab3649ff2739b0eb_79_77). + +WARNING: +**Do not** call `UploadedFile#from_params` directly! Do not build an [`UploadedFile`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/uploaded_file.rb) +instance using `UploadedFile#from_params`! This method can be unsafe to use depending on the `params` +passed. Instead, use the [`UploadedFile`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/uploaded_file.rb) +instance that [`multipart.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/middleware/multipart.rb) +builds automatically for you. + +## Implementing the new route with a Grape API endpoint + +For a Grape API upload, we can have [body or a multipart](implementation.md#upload-encodings) upload. Things are slightly more complicated: two endpoints are needed. One for the +Workhorse pre-upload authorization and one for accepting the upload metadata from Workhorse: + +1. Implement an endpoint with the URL + `/authorize` suffix that will: + - Check that the request is coming from Workhorse with the `require_gitlab_workhorse!` from the [API helpers](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/helpers.rb). + - Check user permissions. + - Set the status to `200` with `status 200`. + - Set the content type with `content_type Gitlab::Workhorse::INTERNAL_API_CONTENT_TYPE`. + - Use your dedicated `Uploader` class (let's say that it's `FileUploader`) to build the response with `FileUploader.workhorse_authorize(params)`. +1. Implement the endpoint for the upload request that will: + - Require all the `UploadedFile` objects as parameters. + - For example, if we expect a single parameter `file` to be an [`UploadedFile`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/uploaded_file.rb) instance, +use `requires :file, type: ::API::Validations::Types::WorkhorseFile`. + - Body upload requests have their upload available under the parameter `file`. + - Check that the request is coming from Workhorse with the `require_gitlab_workhorse!` from the +[API helpers](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/api/helpers.rb). + - Check the user permissions. + - The remaining code of the processing. This is where the code must be reading the parameter (for +our example, it would be `params[:file]`). + +WARNING: +**Do not** call `UploadedFile#from_params` directly! Do not build an [`UploadedFile`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/uploaded_file.rb) +object using `UploadedFile#from_params`! This method can be unsafe to use depending on the `params` +passed. Instead, use the [`UploadedFile`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/uploaded_file.rb) +object that [`multipart.rb`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/middleware/multipart.rb) +builds automatically for you. + +## Document Object Storage buckets and CarrierWave integration + +When using Object Storage, GitLab expects each kind of upload to maintain its own bucket in the respective +Object Storage destination. Moreover, the integration with CarrierWave is not used all the time. +The [Object Storage Working Group](https://about.gitlab.com/company/team/structure/working-groups/object-storage/) +is investigating an approach that unifies Object Storage buckets into a single one and removes CarrierWave +so as to simplify implementation and administration of uploads. + +Therefore, document new uploads here by slotting them into the following tables: + +- [Feature bucket details](#feature-bucket-details) +- [CarrierWave integration](#carrierwave-integration) + +### Feature bucket details + +| Feature | Upload technology | Uploader | Bucket structure | +|------------------------------------------|-------------------|-----------------------|-----------------------------------------------------------------------------------------------------------| +| Job artifacts | `direct upload` | `workhorse` | `/artifacts/<proj_id_hash>/<date>/<job_id>/<artifact_id>` | +| Pipeline artifacts | `carrierwave` | `sidekiq` | `/artifacts/<proj_id_hash>/pipelines/<pipeline_id>/artifacts/<artifact_id>` | +| Live job traces | `fog` | `sidekiq` | `/artifacts/tmp/builds/<job_id>/chunks/<chunk_index>.log` | +| Job traces archive | `carrierwave` | `sidekiq` | `/artifacts/<proj_id_hash>/<date>/<job_id>/<artifact_id>/job.log` | +| Autoscale runner caching | N/A | `gitlab-runner` | `/gitlab-com-[platform-]runners-cache/???` | +| Backups | N/A | `s3cmd`, `awscli`, or `gcs` | `/gitlab-backups/???` | +| Git LFS | `direct upload` | `workhorse` | `/lsf-objects/<lfs_obj_oid[0:2]>/<lfs_obj_oid[2:2]>` | +| Design management files | `disk buffering` | `rails controller` | `/lsf-objects/<lfs_obj_oid[0:2]>/<lfs_obj_oid[2:2]>` | +| Design management thumbnails | `carrierwave` | `sidekiq` | `/uploads/design_management/action/image_v432x230/<model_id>` | +| Generic file uploads | `direct upload` | `workhorse` | `/uploads/@hashed/[0:2]/[2:4]/<hash1>/<hash2>/file` | +| Generic file uploads - personal snippets | `direct upload` | `workhorse` | `/uploads/personal_snippet/<snippet_id>/<filename>` | +| Global appearance settings | `disk buffering` | `rails controller` | `/uploads/appearance/...` | +| Topics | `disk buffering` | `rails controller` | `/uploads/projects/topic/...` | +| Avatar images | `direct upload` | `workhorse` | `/uploads/[user,group,project]/avatar/<model_id>` | +| Import/export | `direct upload` | `workhorse` | `/uploads/import_export_upload/???` | +| GitLab Migration | `carrierwave` | `sidekiq` | `/uploads/bulk_imports/???` | +| MR diffs | `carrierwave` | `sidekiq` | `/external-diffs/merge_request_diffs/mr-<mr_id>/diff-<diff_id>` | +| Package manager archives | `direct upload` | `sidekiq` | `/packages/<proj_id_hash>/packages/<pkg_segment>/files/<pkg_file_id>` | +| Package manager archives | `direct upload` | `sidekiq` | `/packages/<container_id_hash>/debian_*_component_file/<component_file_id>` | +| Package manager archives | `direct upload` | `sidekiq` | `/packages/<container_id_hash>/debian_*_distribution/<distribution_file_id>` | +| Container image cache (?) | `direct upload` | `workhorse` | `/dependency-proxy/<group_id_hash>/dependency_proxy/<group_id>/files/<proxy_id>/<blob_id or manifest_id>` | +| Terraform state files | `carrierwave` | `rails controller` | `/terraform/<proj_id_hash>/<terraform_state_id>` | +| Pages content archives | `carrierwave` | `sidekiq` | `/gitlab-gprd-pages/<proj_id_hash>/pages_deployments/<deployment_id>/` | +| Secure Files | `carrierwave` | `sidekiq` | `/ci-secure-files/<proj_id_hash>/secure_files/<secure_file_id>/` | + +### CarrierWave integration + +| File | Carrierwave usage | Categorized | +|---------------------------------------------------------|----------------------------------------------------------------------------------|---------------------| +| `app/models/project.rb` | `include Avatarable` | :white_check_mark: | +| `app/models/projects/topic.rb` | `include Avatarable` | :white_check_mark: | +| `app/models/group.rb` | `include Avatarable` | :white_check_mark: | +| `app/models/user.rb` | `include Avatarable` | :white_check_mark: | +| `app/models/terraform/state_version.rb` | `include FileStoreMounter` | :white_check_mark: | +| `app/models/ci/job_artifact.rb` | `include FileStoreMounter` | :white_check_mark: | +| `app/models/ci/pipeline_artifact.rb` | `include FileStoreMounter` | :white_check_mark: | +| `app/models/pages_deployment.rb` | `include FileStoreMounter` | :white_check_mark: | +| `app/models/lfs_object.rb` | `include FileStoreMounter` | :white_check_mark: | +| `app/models/dependency_proxy/blob.rb` | `include FileStoreMounter` | :white_check_mark: | +| `app/models/dependency_proxy/manifest.rb` | `include FileStoreMounter` | :white_check_mark: | +| `app/models/packages/composer/cache_file.rb` | `include FileStoreMounter` | :white_check_mark: | +| `app/models/packages/package_file.rb` | `include FileStoreMounter` | :white_check_mark: | +| `app/models/concerns/packages/debian/component_file.rb` | `include FileStoreMounter` | :white_check_mark: | +| `ee/app/models/issuable_metric_image.rb` | `include FileStoreMounter` | | +| `ee/app/models/vulnerabilities/remediation.rb` | `include FileStoreMounter` | | +| `ee/app/models/vulnerabilities/export.rb` | `include FileStoreMounter` | | +| `app/models/packages/debian/project_distribution.rb` | `include Packages::Debian::Distribution` | :white_check_mark: | +| `app/models/packages/debian/group_distribution.rb` | `include Packages::Debian::Distribution` | :white_check_mark: | +| `app/models/packages/debian/project_component_file.rb` | `include Packages::Debian::ComponentFile` | :white_check_mark: | +| `app/models/packages/debian/group_component_file.rb` | `include Packages::Debian::ComponentFile` | :white_check_mark: | +| `app/models/merge_request_diff.rb` | `mount_uploader :external_diff, ExternalDiffUploader` | :white_check_mark: | +| `app/models/note.rb` | `mount_uploader :attachment, AttachmentUploader` | :white_check_mark: | +| `app/models/appearance.rb` | `mount_uploader :logo, AttachmentUploader` | :white_check_mark: | +| `app/models/appearance.rb` | `mount_uploader :header_logo, AttachmentUploader` | :white_check_mark: | +| `app/models/appearance.rb` | `mount_uploader :favicon, FaviconUploader` | :white_check_mark: | +| `app/models/project.rb` | `mount_uploader :bfg_object_map, AttachmentUploader` | | +| `app/models/import_export_upload.rb` | `mount_uploader :import_file, ImportExportUploader` | :white_check_mark: | +| `app/models/import_export_upload.rb` | `mount_uploader :export_file, ImportExportUploader` | :white_check_mark: | +| `app/models/ci/deleted_object.rb` | `mount_uploader :file, DeletedObjectUploader` | | +| `app/models/design_management/action.rb` | `mount_uploader :image_v432x230, DesignManagement::DesignV432x230Uploader` | :white_check_mark: | +| `app/models/concerns/packages/debian/distribution.rb` | `mount_uploader :signed_file, Packages::Debian::DistributionReleaseFileUploader` | :white_check_mark: | +| `app/models/bulk_imports/export_upload.rb` | `mount_uploader :export_file, ExportUploader` | :white_check_mark: | +| `ee/app/models/user_permission_export_upload.rb` | `mount_uploader :file, AttachmentUploader` | | +| `app/models/ci/secure_file.rb` | `include FileStoreMounter` | | diff --git a/doc/development/value_stream_analytics/img/object_hierarchy_example_V14_10.png b/doc/development/value_stream_analytics/img/object_hierarchy_example_V14_10.png Binary files differindex 0e3c760fa3c..9c4d4ae7718 100644 --- a/doc/development/value_stream_analytics/img/object_hierarchy_example_V14_10.png +++ b/doc/development/value_stream_analytics/img/object_hierarchy_example_V14_10.png diff --git a/doc/development/workspaces/index.md b/doc/development/workspace/index.md index b36c79e84d7..b01a7826b3d 100644 --- a/doc/development/workspaces/index.md +++ b/doc/development/workspace/index.md @@ -4,19 +4,19 @@ type: index, dev stage: none group: Development info: "See the Technical Writers assigned to Development Guidelines: https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments-to-development-guidelines" -description: "Development Guidelines: learn about workspaces when developing GitLab." +description: "Development Guidelines: learn about workspace when developing GitLab." --- -# Workspaces +# Workspace -The [Workspaces initiative](../../user/workspace/index.md) focuses on reaching feature parity between +The [Workspace initiative](../../user/workspace/index.md) focuses on reaching feature parity between SaaS and self-managed installations. ## Consolidate groups and projects - [Architecture blueprint](../../architecture/blueprints/consolidating_groups_and_projects/index.md) -One facet of the workspaces initiative is to consolidate groups and projects, +One facet of the workspace initiative is to consolidate groups and projects, addressing the feature disparity between them. Some features, such as epics, are only available at the group level. Some features, such as issues, are only available at the project level. Other features, such as milestones, are available to both groups diff --git a/doc/install/aws/manual_install_aws.md b/doc/install/aws/manual_install_aws.md index 26d93ea06b7..b0e71cbc77e 100644 --- a/doc/install/aws/manual_install_aws.md +++ b/doc/install/aws/manual_install_aws.md @@ -390,7 +390,7 @@ persistence and is used to store session data, temporary cache information, and chance to deploy Redis in multiple availability zones. 1. In the settings section: 1. Give the cluster a name (`gitlab-redis`) and a description. - 1. For the version, select the latest of the `5.0` series (for example, `5.0.6`). + 1. For the version, select the latest. 1. Leave the port as `6379` since this is what we used in our Redis security group above. 1. Select the node type (at least `cache.t3.medium`, but adjust to your needs) and the number of replicas. 1. In the advanced settings section: @@ -827,7 +827,7 @@ to request additional material: Geo is the solution for widely distributed development teams. - [Omnibus GitLab](https://docs.gitlab.com/omnibus/) - Everything you need to know about administering your GitLab instance. -- [Upload a license](../../user/admin_area/license.md): +- [Add a license](../../user/admin_area/license.md): Activate all GitLab Enterprise Edition functionality with a license. - [Pricing](https://about.gitlab.com/pricing/): Pricing for the different tiers. diff --git a/doc/install/docker.md b/doc/install/docker.md index ed5e1dda5d5..a25ed629681 100644 --- a/doc/install/docker.md +++ b/doc/install/docker.md @@ -189,7 +189,7 @@ services: external_url 'http://gitlab.example.com:8929' gitlab_rails['gitlab_shell_ssh_port'] = 2224 ports: - - '8929:8929' + - '8929:80' - '2224:22' volumes: - '$GITLAB_HOME/config:/etc/gitlab' @@ -198,7 +198,7 @@ services: shm_size: '256m' ``` -This is the same as using `--publish 8929:8929 --publish 2224:22`. +This is the same as using `--publish 8929:80 --publish 2224:22`. ### Install GitLab using Docker swarm mode @@ -513,6 +513,22 @@ To update GitLab that was [installed using Docker Compose](#install-gitlab-using If you have used [tags](#use-tagged-versions-of-gitlab) instead, you'll need to first edit `docker-compose.yml`. +### Convert Community Edition to Enterprise Edition + +You can convert an existing Docker-based GitLab Community Edition (CE) container +to a GitLab [Enterprise Edition](https://about.gitlab.com/pricing/) (EE) container +using the same approach as [updating the version](#update). + +We recommend you convert from the same version of CE to EE (for example, CE 14.1 to EE 14.1). +This is not explicitly necessary, and any standard upgrade (for example, CE 14.0 to EE 14.1) should work. +The following steps assume that you are upgrading the same version. + +1. Take a [backup](#back-up-gitlab). +1. Stop the current CE container, and remove or rename it. +1. To create a new container with GitLab EE, + replace `ce` with `ee` in your `docker run` command or `docker-compose.yml` file. + However, reuse the CE container name, port and file mappings, and version. + ## Back up GitLab You can create a GitLab backup with: diff --git a/doc/install/google_cloud_platform/index.md b/doc/install/google_cloud_platform/index.md index b3d0863f6a3..76c1da74108 100644 --- a/doc/install/google_cloud_platform/index.md +++ b/doc/install/google_cloud_platform/index.md @@ -7,7 +7,7 @@ description: 'Learn how to install a GitLab instance on Google Cloud Platform.' # Installing GitLab on Google Cloud Platform **(FREE SELF)** -This guide will help you install GitLab on a [Google Cloud Platform (GCP)](https://cloud.google.com/) using the official GitLab Linux package. You should customize it to accommodate your needs. +You can install GitLab on a [Google Cloud Platform (GCP)](https://cloud.google.com/) using the official GitLab Linux package. You should customize it to accommodate your needs. NOTE: To deploy production-ready GitLab on @@ -19,20 +19,20 @@ the [Cloud native GitLab Helm chart](https://docs.gitlab.com/charts/). ## Prerequisites -There are only two prerequisites in order to install GitLab on GCP: +There are two prerequisites to install GitLab on GCP: -1. You need to have a Google account. -1. You need to sign up for the GCP program. If this is your first time, Google +1. You must have a Google account. +1. You must sign up for the GCP program. If this is your first time, Google gives you [$300 credit for free](https://console.cloud.google.com/freetrial) to consume over a 60-day period. -Once you have performed those two steps, you can [create a VM](#creating-the-vm). +After you have performed those two steps, you can [create a VM](#creating-the-vm). ## Creating the VM -To deploy GitLab on GCP you first need to create a virtual machine: +To deploy GitLab on GCP you must create a virtual machine: 1. Go to <https://console.cloud.google.com/compute/instances> and log in with your Google credentials. -1. Click on **Create** +1. Select **Create**  @@ -43,9 +43,9 @@ To deploy GitLab on GCP you first need to create a virtual machine:  1. To select the size, type, and desired [operating system](../requirements.md#supported-linux-distributions), - click **Change** under `Boot disk`. Click **Select** when finished. + select **Change** under `Boot disk`. select **Select** when finished. -1. As a last step allow HTTP and HTTPS traffic, then click **Create**. The process finishes in a few seconds. +1. As a last step allow HTTP and HTTPS traffic, then select **Create**. The process finishes in a few seconds. ## Installing GitLab @@ -54,7 +54,7 @@ After a few seconds, the instance is created and available to log in. The next s  1. Make a note of the external IP address of the instance, as you will need that in a later step. <!-- using future tense is okay here --> -1. Click on the SSH button to connect to the instance. +1. Select **SSH** under the connect column to connect to the instance. 1. A new window appears, with you logged into the instance.  @@ -84,7 +84,7 @@ Assuming you have a domain name in your possession and you have correctly set up DNS to point to the static IP you configured in the previous step, here's how you configure GitLab to be aware of the change: -1. SSH into the VM. You can easily use the **SSH** button in the Google console +1. SSH into the VM. You can select **SSH** in the Google console and a new window pops up.  @@ -122,7 +122,7 @@ certificate. Follow the steps in the [Omnibus documentation](https://docs.gitlab ### Configuring the email SMTP settings -You need to configure the email SMTP settings correctly otherwise GitLab cannot send notification emails, like comments, and password changes. +You must configure the email SMTP settings correctly otherwise GitLab cannot send notification emails, like comments, and password changes. Check the [Omnibus documentation](https://docs.gitlab.com/omnibus/settings/smtp.html#smtp-settings) how to do so. ## Further reading diff --git a/doc/install/index.md b/doc/install/index.md index 9ffed87fd61..b27d3683cd5 100644 --- a/doc/install/index.md +++ b/doc/install/index.md @@ -27,8 +27,8 @@ install GitLab: | Installation method | Description | When to choose | |----------------------------------------------------------------|-------------|----------------| -| [Linux package](https://docs.gitlab.com/omnibus/installation/) | The official deb/rpm packages (also known as Omnibus GitLab) that contains a bundle of GitLab and the components it depends on, including PostgreSQL, Redis, and Sidekiq. | This is the recommended method for getting started. The Linux packages are mature, scalable, and are used today on GitLab.com. If you need additional flexibility and resilience, we recommend deploying GitLab as described in the [reference architecture documentation](../administration/reference_architectures/index.md). | -| [Helm charts](https://docs.gitlab.com/charts/) | The cloud native Helm chart for installing GitLab and all of its components on Kubernetes. | When installing GitLab on Kubernetes, there are some trade-offs that you need to be aware of: <br/>- Administration and troubleshooting requires Kubernetes knowledge.<br/>- It can be more expensive for smaller installations. The default installation requires more resources than a single node Linux package deployment, as most services are deployed in a redundant fashion.<br/>- There are some feature [limitations to be aware of](https://docs.gitlab.com/charts/#limitations).<br/><br/> Use this method if your infrastructure is built on Kubernetes and you're familiar with how it works. The methods for management, observability, and some concepts are different than traditional deployments. | +| [Linux package](https://docs.gitlab.com/omnibus/installation/) | The official deb/rpm packages (also known as Omnibus GitLab) that contains a bundle of GitLab and the components it depends on, including PostgreSQL, Redis, and Sidekiq. | This method is recommended for getting started. The Linux packages are mature, scalable, and are used today on GitLab.com. If you need additional flexibility and resilience, we recommend deploying GitLab as described in the [reference architecture documentation](../administration/reference_architectures/index.md). | +| [Helm charts](https://docs.gitlab.com/charts/) | The cloud native Helm chart for installing GitLab and all of its components on Kubernetes. | When installing GitLab on Kubernetes, there are some trade-offs that you need to be aware of: <br/>- Administration and troubleshooting requires Kubernetes knowledge.<br/>- It can be more expensive for smaller installations. The default installation requires more resources than a single node Linux package deployment, as most services are deployed in a redundant fashion.<br/><br/> Use this method if your infrastructure is built on Kubernetes and you're familiar with how it works. The methods for management, observability, and some concepts are different than traditional deployments. | | [Docker](https://docs.gitlab.com/omnibus/docker/) | The GitLab packages, Dockerized. | Use this method if you're familiar with Docker. | | [Source](installation.md) | Install GitLab and all of its components from scratch. | Use this method if none of the previous methods are available for your platform. Useful for unsupported systems like \*BSD.| | [GitLab Environment Toolkit (GET)](https://gitlab.com/gitlab-org/gitlab-environment-toolkit#documentation) | The GitLab Environment toolkit provides a set of automation tools to deploy a [reference architecture](../administration/reference_architectures/index.md) on most major cloud providers. | Customers are very welcome to trial and evaluate GET today, however be aware of [key limitations](https://gitlab.com/gitlab-org/gitlab-environment-toolkit#missing-features-to-be-aware-of) of the current iteration. For production environments further manual setup will be required based on your specific requirements. | diff --git a/doc/install/next_steps.md b/doc/install/next_steps.md index 1db2bf9b7b6..2fc60c3af53 100644 --- a/doc/install/next_steps.md +++ b/doc/install/next_steps.md @@ -50,7 +50,7 @@ installation. ## License -- [Upload a license](../user/admin_area/license.md) or [start a free trial](https://about.gitlab.com/free-trial/): +- [Add a license](../user/admin_area/license.md) or [start a free trial](https://about.gitlab.com/free-trial/): Activate all GitLab Enterprise Edition functionality with a license. - [Pricing](https://about.gitlab.com/pricing/): Pricing for the different tiers. diff --git a/doc/install/openshift_and_gitlab/index.md b/doc/install/openshift_and_gitlab/index.md index e102235c4f0..364c27f089f 100644 --- a/doc/install/openshift_and_gitlab/index.md +++ b/doc/install/openshift_and_gitlab/index.md @@ -20,7 +20,7 @@ Some components (documented on the GitLab Operator doc) are not supported yet. ## Deploy to and integrate with OpenShift from GitLab -Deploying custom or COTS applications on top of OpenShift from GitLab is supported using [the GitLab Agent](../../user/clusters/agent/index.md). +Deploying custom or COTS applications on top of OpenShift from GitLab is supported using [the GitLab agent](../../user/clusters/agent/index.md). ## Use OpenShift to run a GitLab Runner Fleet @@ -29,6 +29,12 @@ The GitLab Operator does not include the GitLab Runner. To install and manage a ## Unsupported GitLab features +### Secure and Protect + +- License Compliance +- Code Quality scanning +- Cluster Image Scanning + ### Docker-in-Docker When using OpenShift to run a GitLab Runner Fleet, we do not support some GitLab features given OpenShift's security model. diff --git a/doc/install/pivotal/index.md b/doc/install/pivotal/index.md index ee379a3c95f..56dde411884 100644 --- a/doc/install/pivotal/index.md +++ b/doc/install/pivotal/index.md @@ -6,4 +6,6 @@ remove_date: '2022-03-08' This document was removed. For information about installing GitLab, see [this page](../index.md). <!-- This redirect file can be deleted after <2022-03-08>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/install/relative_url.md b/doc/install/relative_url.md index 43f2414e8f9..831e33870bd 100644 --- a/doc/install/relative_url.md +++ b/doc/install/relative_url.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Install GitLab under a relative URL **(FREE SELF)** -While it is recommended to install GitLab on its own (sub)domain, sometimes +While we recommend to install GitLab on its own (sub)domain, sometimes this is not possible due to a variety of reasons. In that case, GitLab can also be installed under a relative URL, for example `https://example.com/gitlab`. @@ -19,8 +19,8 @@ first time. There is no limit to how deeply nested the relative URL can be. For example you could serve GitLab under `/foo/bar/gitlab/git` without any issues. -Note that by changing the URL on an existing GitLab installation, all remote -URLs will change, so you'll have to manually edit them in any local repository +Changing the URL on an existing GitLab installation, changes all remote +URLs, so you have to manually edit them in any local repository that points to your GitLab instance. The list of configuration files you must change to serve GitLab from a @@ -32,7 +32,7 @@ relative URL is: - `/home/git/gitlab-shell/config.yml` - `/etc/default/gitlab` -After all the changes you need to recompile the assets and [restart GitLab](../administration/restart_gitlab.md#installations-from-source). +After all the changes, you must recompile the assets and [restart GitLab](../administration/restart_gitlab.md#installations-from-source). ## Relative URL requirements diff --git a/doc/install/requirements.md b/doc/install/requirements.md index bce9702b032..11f623641c1 100644 --- a/doc/install/requirements.md +++ b/doc/install/requirements.md @@ -258,7 +258,7 @@ works. ### Puma per worker maximum memory By default, each Puma worker will be limited to 1024 MB of memory. -This setting [can be adjusted](../administration/operations/puma.md#puma-worker-killer) and should be considered +This setting [can be adjusted](../administration/operations/puma.md#change-the-memory-limit-setting) and should be considered if you need to increase the number of Puma workers. ## Redis and Sidekiq @@ -303,7 +303,7 @@ The GitLab Runner server requirements depend on: Since the nature of the jobs varies for each use case, you need to experiment by adjusting the job concurrency to get the optimum setting. -For reference, the [SaaS runners on Linux](../ci/runners/build_cloud/linux_build_cloud.md) +For reference, the [SaaS runners on Linux](../ci/runners/saas/linux_saas_runner.md) are configured so that a **single job** runs in a **single instance** with: - 1 vCPU. diff --git a/doc/integration/azure.md b/doc/integration/azure.md index 8d69881699b..5749e638164 100644 --- a/doc/integration/azure.md +++ b/doc/integration/azure.md @@ -4,135 +4,45 @@ group: Integrations info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Microsoft Azure OAuth 2.0 OmniAuth Provider **(FREE SELF)** +# Use Microsoft Azure as an authentication provider **(FREE SELF)** -NOTE: -Per Microsoft, this provider uses the [older Azure Active Directory v1.0 endpoint](https://docs.microsoft.com/en-us/azure/active-directory/azuread-dev/v1-protocols-oauth-code). -Microsoft documentation suggests that you should use the [OpenID Connect protocol to use the v2 endpoints](../administration/auth/oidc.md#microsoft-azure) for new projects. -To use v2 endpoints via OmniAuth, please follow [Microsoft Azure OAuth 2.0 OmniAuth Provider v2 instructions](#microsoft-azure-oauth-20-omniauth-provider-v2). - -To enable the Microsoft Azure OAuth 2.0 OmniAuth provider, you must register -your application with Azure. Azure generates a client ID and secret key for you -to use. - -Sign in to the [Azure Portal](https://portal.azure.com), and follow the -instructions in the [Microsoft Quickstart documentation](https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app). - -As you go through the Microsoft procedure, keep the following in mind: - -- If you have multiple instances of Azure Active Directory, you can switch to the desired tenant. -- You're setting up a Web application. -- The redirect URI requires the URL of the Azure OAuth callback of your GitLab - installation. For example, `https://gitlab.mycompany.com/users/auth/azure_oauth2/callback`. - The type dropdown should be set to **Web**. -- The `client ID` and `client secret` are terms associated with OAuth 2.0. In some Microsoft documentation, - the terms may be listed as `Application ID` and `Application Secret`. -- If you have to generate a new client secret, follow the Microsoft documentation - for [creating a new application secret](https://docs.microsoft.com/en-us/azure/active-directory/develop/howto-create-service-principal-portal#create-a-new-application-secret). -- Save the client ID and client secret for your new app, as the client secret is only - displayed one time. - -1. On your GitLab server, open the configuration file. - - For Omnibus GitLab: - - ```shell - sudo editor /etc/gitlab/gitlab.rb - ``` - - For installations from source: - - ```shell - cd /home/git/gitlab - - sudo -u git -H editor config/gitlab.yml - ``` - -1. Refer to [Configure initial settings](omniauth.md#configure-initial-settings) - for initial settings. - -1. Add the provider configuration: - - For Omnibus GitLab: - - ```ruby - gitlab_rails['omniauth_providers'] = [ - { - name: "azure_oauth2", - # label: "Provider name", # optional label for login button, defaults to "Azure AD" - args: { - client_id: "CLIENT ID", - client_secret: "CLIENT SECRET", - tenant_id: "TENANT ID", - } - } - ] - ``` - - For installations from source: +You can enable the Microsoft Azure OAuth 2.0 OmniAuth provider and sign in to +GitLab with your Microsoft Azure credentials. You can configure the provider that uses +[the earlier Azure Active Directory v1.0 endpoint](https://docs.microsoft.com/en-us/azure/active-directory/azuread-dev/v1-protocols-oauth-code), +or the provider that uses the v2.0 endpoint. - ```yaml - - { name: 'azure_oauth2', - # label: 'Provider name', # optional label for login button, defaults to "Azure AD" - args: { client_id: 'CLIENT ID', - client_secret: 'CLIENT SECRET', - tenant_id: 'TENANT ID' } } - ``` - - The `base_azure_url` is optional and can be added for different locales; - such as `base_azure_url: "https://login.microsoftonline.de"`. - -1. Replace `CLIENT ID`, `CLIENT SECRET` and `TENANT ID` with the values you got above. - -1. Save the configuration file. - -1. Reconfigure or restart GitLab, depending on your installation method: - - - *If you installed from Omnibus GitLab,* - [reconfigure](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) GitLab. - - *If you installed from source,* - [restart GitLab](../administration/restart_gitlab.md#installations-from-source). - -On the sign-in page, you should now see a Microsoft icon below the regular -sign-in form. Click the icon to begin the authentication process. Microsoft then -asks you to sign in and authorize the GitLab application. If successful, you are -returned to GitLab and signed in. - -Read [Enable OmniAuth for an Existing User](omniauth.md#enable-omniauth-for-an-existing-user) -for information on how existing GitLab users can connect to their newly-available Azure AD accounts. - -## Microsoft Azure OAuth 2.0 OmniAuth Provider v2 - -To use v2 endpoints provided by Microsoft Azure Active Directory you must to -configure it via Azure OAuth 2.0 OmniAuth Provider v2. +NOTE: +For new projects, Microsoft suggests you use the +[OpenID Connect protocol](../administration/auth/oidc.md#microsoft-azure), +which uses the Microsoft identity platform (v2.0) endpoint. -### Registering an Azure application +## Register an Azure application To enable the Microsoft Azure OAuth 2.0 OmniAuth provider, you must register -your application with Azure. Azure generates a client ID and secret key for you -to use. +an Azure application and get a client ID and secret key. -Sign in to the [Azure Portal](https://portal.azure.com), and follow the -instructions in the [Microsoft Quickstart documentation](https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app). +1. Sign in to the [Azure portal](https://portal.azure.com). +1. If you have multiple Azure Active Directory tenants, switch to the desired tenant. +1. [Register an application](https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-register-app) + and provide the following information: + - The redirect URI, which requires the URL of the Azure OAuth callback of your GitLab + installation. For example: + - For the v1.0 endpoint: `https://gitlab.example.com/users/auth/azure_oauth2/callback`. + - For the v2.0 endpoint: `https://gitlab.example.com/users/auth/azure_activedirectory_v2/callback`. + - The application type, which must be set to **Web**. +1. Save the client ID and client secret. The client secret is only + displayed once. -As you go through the Microsoft procedure, keep the following in mind: + If required, you can [create a new application secret](https://docs.microsoft.com/en-us/azure/active-directory/develop/howto-create-service-principal-portal#option-2-create-a-new-application-secret). -- If you have multiple instances of Azure Active Directory, you can switch to - the desired tenant. -- You're setting up a Web application. -- The redirect URI requires the URL of the Azure OAuth callback of your GitLab - installation. For example, `https://gitlab.example.com/users/auth/azure_activedirectory_v2/callback`. - The type dropdown should be set to **Web**. -- The `client ID` and `client secret` are terms associated with OAuth 2.0. In some Microsoft documentation, - the terms may be listed as `Application ID` and `Application Secret`. -- If you have to generate a new client secret, follow the Microsoft documentation - for [creating a new application secret](https://docs.microsoft.com/en-us/azure/active-directory/develop/howto-create-service-principal-portal#create-a-new-application-secret). -- Save the client ID and client secret for your new app, as the client secret is only - displayed one time. +`client ID` and `client secret` are terms associated with OAuth 2.0. +In some Microsoft documentation, the terms are named `Application ID` and +`Application Secret`. -### Adding API permissions (scopes) +## Add API permissions (scopes) -After you have created an application, follow the [Microsoft Quickstart documentation to expose a web API](https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-configure-app-expose-web-apis). Be sure to add the following delegated permissions under the Microsoft Graph API: +If you're using the v2.0 endpoint, after you create the application, [configure it to expose a web API](https://docs.microsoft.com/en-us/azure/active-directory/develop/quickstart-configure-app-expose-web-apis). +Add the following delegated permissions under the Microsoft Graph API: - `email` - `openid` @@ -140,75 +50,100 @@ After you have created an application, follow the [Microsoft Quickstart document Alternatively, add the `User.Read.All` application permission. -### Configuring GitLab +## Enable Microsoft OAuth in GitLab 1. On your GitLab server, open the configuration file. - For Omnibus GitLab: + - **For Omnibus installations** + + ```shell + sudo editor /etc/gitlab/gitlab.rb + ``` - ```shell - sudo editor /etc/gitlab/gitlab.rb - ``` + - **For installations from source** - For installations from source: + ```shell + cd /home/git/gitlab - ```shell - cd /home/git/gitlab + sudo -u git -H editor config/gitlab.yml + ``` - sudo -u git -H editor config/gitlab.yml - ``` +1. [Configure the initial settings](omniauth.md#configure-initial-settings). -1. Refer to [Configure initial settings](omniauth.md#configure-initial-settings) - for initial settings. +1. Add the provider configuration. Replace `CLIENT ID`, `CLIENT SECRET`, and `TENANT ID` + with the values you got when you registered the Azure application. -1. Add the provider configuration: + - **For Omnibus installations** - For Omnibus GitLab: + For the v1.0 endpoint: - ```ruby - gitlab_rails['omniauth_providers'] = [ - { - "name" => "azure_activedirectory_v2", - "label" => "Provider name", # optional label for login button, defaults to "Azure AD v2" - "args" => { - "client_id" => "CLIENT ID", - "client_secret" => "CLIENT SECRET", - "tenant_id" => "TENANT ID", + ```ruby + gitlab_rails['omniauth_providers'] = [ + { + name: "azure_oauth2", + # label: "Provider name", # optional label for login button, defaults to "Azure AD" + args: { + client_id: "CLIENT ID", + client_secret: "CLIENT SECRET", + tenant_id: "TENANT ID", + } } - } - ] - ``` + ] + ``` + + For the v2.0 endpoint: + + ```ruby + gitlab_rails['omniauth_providers'] = [ + { + "name" => "azure_activedirectory_v2", + "label" => "Provider name", # optional label for login button, defaults to "Azure AD v2" + "args" => { + "client_id" => "CLIENT ID", + "client_secret" => "CLIENT SECRET", + "tenant_id" => "TENANT ID", + } + } + ] + ``` + + - **For installations from source** - For installations from source: + For the v1.0 endpoint: - ```yaml - - { name: 'azure_activedirectory_v2', - label: 'Provider name', # optional label for login button, defaults to "Azure AD v2" - args: { client_id: "CLIENT ID", - client_secret: "CLIENT SECRET", - tenant_id: "TENANT ID" } } - ``` + ```yaml + - { name: 'azure_oauth2', + # label: 'Provider name', # optional label for login button, defaults to "Azure AD" + args: { client_id: 'CLIENT ID', + client_secret: 'CLIENT SECRET', + tenant_id: 'TENANT ID' } } + ``` - The `base_azure_url` is optional and can be added for different locales; - such as `base_azure_url: "https://login.microsoftonline.de"`. + For the v2.0 endpoint: + + ```yaml + - { name: 'azure_activedirectory_v2', + label: 'Provider name', # optional label for login button, defaults to "Azure AD v2" + args: { client_id: "CLIENT ID", + client_secret: "CLIENT SECRET", + tenant_id: "TENANT ID" } } + ``` - The `scope` parameter is optional and can be added to `args`. Default `scope` is: `openid profile email`. + You can optionally add the following parameters: -1. Replace `CLIENT ID`, `CLIENT SECRET`, and `TENANT ID` with the values you got - above. + - `base_azure_url` for different locales. For example, `base_azure_url: "https://login.microsoftonline.de"`. + - `scope`, which you add to `args`. The default is `openid profile email`. 1. Save the configuration file. -1. Reconfigure or restart GitLab, depending on your installation method: +1. [Reconfigure GitLab](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) + if you installed using Omnibus, or [restart GitLab](../administration/restart_gitlab.md#installations-from-source) + if you installed from source. - - *If you installed from Omnibus GitLab,* - [reconfigure](../administration/restart_gitlab.md#omnibus-gitlab-reconfigure) GitLab. - - *If you installed from source,* - [restart GitLab](../administration/restart_gitlab.md#installations-from-source). +1. Refresh the GitLab sign-in page. A Microsoft icon should display below the + sign-in form. -On the sign-in page, you should now see a Microsoft icon below the regular sign-in form. -Select the icon to begin the authentication process. Microsoft then asks you to -sign in and authorize the GitLab application. If successful, you are returned to GitLab and signed in. +1. Select the icon. Sign in to Microsoft and authorize the GitLab application. -Read [Enable OmniAuth for an Existing User](omniauth.md#enable-omniauth-for-an-existing-user) -for information on how existing GitLab users can connect to their newly available Azure AD accounts. +Read [Enable OmniAuth for an existing user](omniauth.md#enable-omniauth-for-an-existing-user) +for information on how existing GitLab users can connect to their new Azure AD accounts. diff --git a/doc/integration/elasticsearch.md b/doc/integration/elasticsearch.md index 4976f7c2664..5265a24d299 100644 --- a/doc/integration/elasticsearch.md +++ b/doc/integration/elasticsearch.md @@ -58,6 +58,9 @@ source. You must [install it separately](https://www.elastic.co/guide/en/elastic You can install Elasticsearch yourself, or use a cloud hosted offering such as [Elasticsearch Service](https://www.elastic.co/elasticsearch/service) (available on AWS, GCP, or Azure) or the [Amazon OpenSearch](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/gsg.html) service. + +If using the Amazon OpenSearch service, ensure that you select `Elasticsearch 7.10` when configuring Deployment type. As noted in [Versions not supported](#versions-not-supported), Amazon's non-Elasticsearch versions are not yet supported. + You should install Elasticsearch on a separate server. Running Elasticsearch on the same server as GitLab is not recommended and can cause a degradation in GitLab instance performance. For a single node Elasticsearch cluster, the functional cluster health status is always yellow due to the allocation of the primary shard. Elasticsearch cannot assign replica shards to the same node as primary shards. @@ -206,7 +209,7 @@ The following Elasticsearch settings are available: | Parameter | Description | |-------------------------------------------------------|-------------| -| `Elasticsearch indexing` | Enables or disables Elasticsearch indexing and creates an empty index if one does not already exist. You may want to enable indexing but disable search in order to give the index time to be fully completed, for example. Also, keep in mind that this option doesn't have any impact on existing data, this only enables/disables the background indexer which tracks data changes and ensures new data is indexed. | +| `Elasticsearch indexing` | Enables or disables Elasticsearch indexing and creates an empty index if one does not already exist. You may want to enable indexing but disable search to give the index time to be fully completed, for example. Also, keep in mind that this option doesn't have any impact on existing data, this only enables/disables the background indexer which tracks data changes and ensures new data is indexed. | | `Pause Elasticsearch indexing` | Enables or disables temporary indexing pause. This is useful for cluster migration/reindexing. All changes are still tracked, but they are not committed to the Elasticsearch index until resumed. | | `Search with Elasticsearch enabled` | Enables or disables using Elasticsearch in search. | | `URL` | The URL of your Elasticsearch instance. Use a comma-separated list to support clustering (for example, `http://host1, https://host2:9200`). If your Elasticsearch instance is password-protected, use the `Username` and `Password` fields described below. Alternatively, use inline credentials such as `http://<username>:<password>@<elastic_host>:9200/`. | @@ -221,8 +224,8 @@ The following Elasticsearch settings are available: | `AWS Secret Access Key` | The AWS secret access key. | | `Maximum file size indexed` | See [the explanation in instance limits.](../administration/instance_limits.md#maximum-file-size-indexed). | | `Maximum field length` | See [the explanation in instance limits.](../administration/instance_limits.md#maximum-field-length). | -| `Maximum bulk request size (MiB)` | The Maximum Bulk Request size is used by the GitLab Golang-based indexer processes and indicates how much data it ought to collect (and store in memory) in a given indexing process before submitting the payload to Elasticsearch's Bulk API. This setting should be used with the Bulk request concurrency setting (see below) and needs to accommodate the resource constraints of both the Elasticsearch host(s) and the host(s) running the GitLab Golang-based indexer either from the `gitlab-rake` command or the Sidekiq tasks. | -| `Bulk request concurrency` | The Bulk request concurrency indicates how many of the GitLab Golang-based indexer processes (or threads) can run in parallel to collect data to subsequently submit to Elasticsearch's Bulk API. This increases indexing performance, but fills the Elasticsearch bulk requests queue faster. This setting should be used together with the Maximum bulk request size setting (see above) and needs to accommodate the resource constraints of both the Elasticsearch host(s) and the host(s) running the GitLab Golang-based indexer either from the `gitlab-rake` command or the Sidekiq tasks. | +| `Maximum bulk request size (MiB)` | The Maximum Bulk Request size is used by the GitLab Golang-based indexer processes and indicates how much data it ought to collect (and store in memory) in a given indexing process before submitting the payload to Elasticsearch's Bulk API. This setting should be used with the Bulk request concurrency setting (see below) and needs to accommodate the resource constraints of both the Elasticsearch hosts and the hosts running the GitLab Golang-based indexer either from the `gitlab-rake` command or the Sidekiq tasks. | +| `Bulk request concurrency` | The Bulk request concurrency indicates how many of the GitLab Golang-based indexer processes (or threads) can run in parallel to collect data to subsequently submit to Elasticsearch's Bulk API. This increases indexing performance, but fills the Elasticsearch bulk requests queue faster. This setting should be used together with the Maximum bulk request size setting (see above) and needs to accommodate the resource constraints of both the Elasticsearch hosts and the hosts running the GitLab Golang-based indexer either from the `gitlab-rake` command or the Sidekiq tasks. | | `Client request timeout` | Elasticsearch HTTP client request timeout value in seconds. `0` means using the system default timeout value, which depends on the libraries that GitLab application is built upon. | WARNING: @@ -259,16 +262,16 @@ from the Elasticsearch index as expected. You can improve the language support for Chinese and Japanese languages by utilizing [`smartcn`](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-smartcn.html) and/or [`kuromoji`](https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-kuromoji.html) analysis plugins from Elastic. -To enable language(s) support: +To enable languages support: -1. Install the desired plugin(s), please refer to [Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsearch/plugins/7.9/installation.html) for plugins installation instructions. The plugin(s) must be installed on every node in the cluster, and each node must be restarted after installation. For a list of plugins, see the table later in this section. +1. Install the desired plugins, please refer to [Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsearch/plugins/7.9/installation.html) for plugins installation instructions. The plugins must be installed on every node in the cluster, and each node must be restarted after installation. For a list of plugins, see the table later in this section. 1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Settings > Advanced Search**. 1. Locate **Custom analyzers: language support**. -1. Enable plugin(s) support for **Indexing**. +1. Enable plugins support for **Indexing**. 1. Click **Save changes** for the changes to take effect. 1. Trigger [Zero downtime reindexing](#zero-downtime-reindexing) or reindex everything from scratch to create a new index with updated mappings. -1. Enable plugin(s) support for **Searching** after the previous step is completed. +1. Enable plugins support for **Searching** after the previous step is completed. For guidance on what to install, see the following Elasticsearch language plugin options: @@ -662,7 +665,7 @@ Sidekiq processes](../administration/operations/extra_sidekiq_processes.md). ``` You can also use the `gitlab:elastic:clear_index_status` Rake task to force the - indexer to "forget" all progress, so it will retry the indexing process from the + indexer to "forget" all progress, so it retries the indexing process from the start. 1. Personal snippets are not associated with a project and need to be indexed separately: @@ -831,9 +834,9 @@ for the changes to take effect. ## Reverting to Basic Search Sometimes there may be issues with your Elasticsearch index data and as such -GitLab will allow you to revert to "basic search" when there are no search +GitLab allows you to revert to "basic search" when there are no search results and assuming that basic search is supported in that scope. This "basic -search" will behave as though you don't have Advanced Search enabled at all for +search" behaves as though you don't have Advanced Search enabled at all for your instance and search using other data sources (such as PostgreSQL data and Git data). @@ -847,7 +850,7 @@ the Elasticsearch data store is ever corrupted for whatever reason, you can rein ## Troubleshooting One of the most valuable tools for identifying issues with the Elasticsearch -integration will be logs. The most relevant logs for this integration are: +integration are logs. The most relevant logs for this integration are: 1. [`sidekiq.log`](../administration/logs.md#sidekiqlog) - All of the indexing happens in Sidekiq, so much of the relevant logs for the @@ -863,7 +866,7 @@ Here are some common pitfalls and how to overcome them. There are a couple of ways to achieve that: -- Whenever you perform a search there will be a link on the search results page +- Whenever you perform a search there is a link on the search results page in the top right hand corner saying "Advanced search functionality is enabled". This is always correctly identifying whether the current project/namespace being searched is using Elasticsearch. @@ -923,7 +926,7 @@ See [Elasticsearch Index Scopes](#advanced-search-index-scopes) for more informa ### I indexed all the repositories but then switched Elasticsearch servers and now I can't find anything -You will need to re-run all the Rake tasks to reindex the database, repositories, and wikis. +You must re-run all the Rake tasks to reindex the database, repositories, and wikis. ### The indexing process is taking a very long time @@ -938,7 +941,7 @@ You can run `sudo gitlab-rake gitlab:elastic:projects_not_indexed` to display pr NOTE: This was [fixed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/35936) in GitLab 13.2 and the Rake task is not available for versions greater than that. -When performing the initial indexing of blobs, we lock all projects until the project finishes indexing. It could happen that an error during the process causes one or multiple projects to remain locked. In order to unlock them, run: +When performing the initial indexing of blobs, we lock all projects until the project finishes indexing. It could happen that an error during the process causes one or multiple projects to remain locked. To unlock them, run: ```shell sudo gitlab-rake gitlab:elastic:clear_locked_projects @@ -946,8 +949,8 @@ sudo gitlab-rake gitlab:elastic:clear_locked_projects ### `Can't specify parent if no parent field has been configured` error -If you enabled Elasticsearch before GitLab 8.12 and have not rebuilt indexes you will get -exception in lots of different cases: +If you enabled Elasticsearch before GitLab 8.12 and have not rebuilt indexes, you get +exceptions in lots of different cases: ```plaintext Elasticsearch::Transport::Transport::Errors::BadRequest([400] { @@ -983,12 +986,12 @@ AWS has [fixed limits](https://docs.aws.amazon.com/opensearch-service/latest/dev ### My single node Elasticsearch cluster status never goes from `yellow` to `green` even though everything seems to be running properly -**For a single node Elasticsearch cluster the functional cluster health status will be yellow** (never green) because the primary shard is allocated but replicas cannot be as there is no other node to which Elasticsearch can assign a replica. This also applies if you are using the [Amazon OpenSearch](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/aes-handling-errors.html#aes-handling-errors-yellow-cluster-status) service. +**For a single node Elasticsearch cluster the functional cluster health status is yellow** (never green) because the primary shard is allocated but replicas cannot be as there is no other node to which Elasticsearch can assign a replica. This also applies if you are using the [Amazon OpenSearch](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/aes-handling-errors.html#aes-handling-errors-yellow-cluster-status) service. WARNING: -Setting the number of replicas to `0` is discouraged (this is not allowed in the GitLab Elasticsearch Integration menu). If you are planning to add more Elasticsearch nodes (for a total of more than 1 Elasticsearch) the number of replicas will need to be set to an integer value larger than `0`. Failure to do so will result in lack of redundancy (losing one node will corrupt the index). +Setting the number of replicas to `0` is discouraged (this is not allowed in the GitLab Elasticsearch Integration menu). If you are planning to add more Elasticsearch nodes (for a total of more than 1 Elasticsearch) the number of replicas needs to be set to an integer value larger than `0`. Failure to do so results in lack of redundancy (losing one node corrupts the index). -If you have a **hard requirement to have a green status for your single node Elasticsearch cluster**, please make sure you understand the risks outlined in the previous paragraph and then run the following query to set the number of replicas to `0`(the cluster will no longer try to create any shard replicas): +If you have a **hard requirement to have a green status for your single node Elasticsearch cluster**, please make sure you understand the risks outlined in the previous paragraph and then run the following query to set the number of replicas to `0`(the cluster no longer tries to create any shard replicas): ```shell curl --request PUT localhost:9200/gitlab-production/_settings --header 'Content-Type: application/json' \ @@ -1008,7 +1011,7 @@ Gitlab::Elastic::Indexer::Error: time="2020-01-23T09:13:00Z" level=fatal msg="he ``` You probably have not used either `http://` or `https://` as part of your value in the **"URL"** field of the Elasticsearch Integration Menu. Please make sure you are using either `http://` or `https://` in this field as the [Elasticsearch client for Go](https://github.com/olivere/elastic) that we are using [needs the prefix for the URL to be accepted as valid](https://github.com/olivere/elastic/commit/a80af35aa41856dc2c986204e2b64eab81ccac3a). -Once you have corrected the formatting of the URL, delete the index (via the [dedicated Rake task](#gitlab-advanced-search-rake-tasks)) and [reindex the content of your instance](#enable-advanced-search). +After you have corrected the formatting of the URL, delete the index (via the [dedicated Rake task](#gitlab-advanced-search-rake-tasks)) and [reindex the content of your instance](#enable-advanced-search). ### My Elasticsearch cluster has a plugin and the integration is not working @@ -1041,8 +1044,8 @@ using the above [troubleshooting](#troubleshooting) steps. If there are no other options, then you always have the option of recreating the entire index from scratch. If you have a small GitLab installation, this can sometimes be a quick way to resolve a problem, but if you have a large GitLab -installation, then this will likely take a very long time to complete. Until the -index is fully recreated, your index will not be serving correct search results, +installation, then this might take a very long time to complete. Until the +index is fully recreated, your index does not serve correct search results, so you may want to disable **Search with Elasticsearch** while it is running. If you are sure you've read the above caveats and want to proceed, then you @@ -1065,9 +1068,9 @@ sudo -u git -H bundle exec rake gitlab:elastic:index ### How does Advanced Search handle private projects? -Advanced Search will store all the projects in the same Elasticsearch indexes, -however searches will only surface results that can be viewed by the user. -Advanced Search will honor all permission checks in the application by +Advanced Search stores all the projects in the same Elasticsearch indexes, +however, searches only surface results that can be viewed by the user. +Advanced Search honors all permission checks in the application by filtering out projects that a user does not have access to at search time. ### Indexing fails with `error: elastic: Error 429 (Too Many Requests)` diff --git a/doc/integration/external-issue-tracker.md b/doc/integration/external-issue-tracker.md index 19f789832b9..ac470291c27 100644 --- a/doc/integration/external-issue-tracker.md +++ b/doc/integration/external-issue-tracker.md @@ -6,27 +6,26 @@ info: To determine the technical writer assigned to the Stage/Group associated w # External issue tracker **(FREE)** -GitLab has a great [issue tracker](../user/project/issues/index.md) but you can also use an external -one. External issue trackers are configurable per GitLab project. +GitLab has an [issue tracker](../user/project/issues/index.md), but you can +configure an external issue tracker per GitLab project. -Once configured, you can reference external issues using the format `CODE-123`, where: +After you configure the external tracker, you can reference external issues +in GitLab merge requests, commits, and comments +using the format `CODE-123`, where: - `CODE` is a unique code for the tracker. - `123` is the issue number in the tracker. -These references in GitLab merge requests, commits, or comments are automatically converted to links to the issues. +The references are automatically converted to links to the issues. You can keep the GitLab issue tracker enabled in parallel or disable it. When enabled, the **Issues** link in the GitLab menu always opens the internal issue tracker. When disabled, the link is not visible in the menu. -## Configuration +## Configure an external issue tracker -The configuration is done via a project's **Settings > Integrations**. +To enable an external issue tracker, you must configure the appropriate [integration](../user/project/integrations/index.md). -### Integration - -To enable an external issue tracker you must configure the appropriate **Integration**. -Visit the links below for details: +The following external issue tracker integrations are available: - [Bugzilla](../user/project/integrations/bugzilla.md) - [Custom Issue Tracker](../user/project/integrations/custom_issue_tracker.md) @@ -34,3 +33,4 @@ Visit the links below for details: - [Jira](../integration/jira/index.md) - [Redmine](../user/project/integrations/redmine.md) - [YouTrack](../user/project/integrations/youtrack.md) +- [ZenTao](../user/project/integrations/zentao.md) diff --git a/doc/integration/gitlab.md b/doc/integration/gitlab.md index 74ae9bb1998..132006ab996 100644 --- a/doc/integration/gitlab.md +++ b/doc/integration/gitlab.md @@ -62,9 +62,7 @@ GitLab.com generates an application ID and secret key for you to use. # label: "Provider name", # optional label for login button, defaults to "GitLab.com" app_id: "YOUR_APP_ID", app_secret: "YOUR_APP_SECRET", - args: { scope: "read_user" # optional: defaults to the scopes of the application - , client_options: { site: "https://gitlab.example.com/api/v4" } - } + args: { scope: "read_user" } # optional: defaults to the scopes of the application } ] ``` @@ -91,7 +89,6 @@ GitLab.com generates an application ID and secret key for you to use. # label: 'Provider name', # optional label for login button, defaults to "GitLab.com" app_id: 'YOUR_APP_ID', app_secret: 'YOUR_APP_SECRET', - args: { "client_options": { "site": 'https://gitlab.example.com/api/v4' } } ``` Or, for installations from source to authenticate against a different GitLab instance: @@ -125,7 +122,7 @@ signed in. FLAG: On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `omniauth_login_minimal_scopes`. On GitLab.com, this feature is not available. -If you use a GitLab instance for authentication, you can reduce access rights when an OAuth application is used for sign in. +If you use a GitLab instance for authentication, you can reduce access rights when an OAuth application is used for sign in. Any OAuth application can advertise the purpose of the application with the authorization parameter: `gl_auth_type=login`. If the application is diff --git a/doc/integration/jenkins.md b/doc/integration/jenkins.md index b86643c7279..54d235b5357 100644 --- a/doc/integration/jenkins.md +++ b/doc/integration/jenkins.md @@ -45,7 +45,7 @@ Grant a GitLab user access to the relevant GitLab projects. If you're integrating Jenkins with many GitLab projects, consider granting the user administrator access. Otherwise, add the user to each project - and grant the Developer role. + and grant the Maintainer role. ## Grant Jenkins access to the GitLab API @@ -242,7 +242,7 @@ To enable job logs in Jenkins: 1. Go to **Dashboard > Manage Jenkins > System Log**. 1. Select **Add new log recorder**. 1. Enter a name for the log recorder. -1. On the next screen, select **Add** and enter `org.jenkinsci.plugins.workflow.job`. +1. On the next screen, select **Add** and enter `com.dabsquared.gitlabjenkins`. 1. Make sure that the Log Level is **All** and select **Save**. To view your logs: diff --git a/doc/integration/jira/configure.md b/doc/integration/jira/configure.md index 979d8a52fb8..2033ddbad6f 100644 --- a/doc/integration/jira/configure.md +++ b/doc/integration/jira/configure.md @@ -50,7 +50,7 @@ To configure your project: If you enable Jira issues with this setting, all users with access to this GitLab project can view all issues from the specified Jira project. -1. To enable [issue creation for vulnerabilities](../../user/application_security/vulnerabilities/index.md#create-a-jira-issue-for-a-vulnerability), select **Enable Jira issues creation from vulnerabilities**. +1. To enable [issue creation for vulnerabilities](../../user/application_security/vulnerabilities/index.md#create-a-jira-issue-for-a-vulnerability), select **Enable Jira issue creation from vulnerabilities**. 1. Select the **Jira issue type**. If the dropdown is empty, select refresh (**{retry}**) and try again. 1. To verify the Jira connection is working, select **Test settings**. 1. Select **Save changes**. diff --git a/doc/integration/jira/connect-app.md b/doc/integration/jira/connect-app.md index ebe8a0f1af4..59cdba93543 100644 --- a/doc/integration/jira/connect-app.md +++ b/doc/integration/jira/connect-app.md @@ -101,7 +101,7 @@ from outside the Marketplace, which allows you to install the application:  1. For **App descriptor URL**, provide the full URL to your manifest file, based - on your instance configuration. For example: `https://your.domain/your-path/-/jira_connect/app_descriptor.json`. + on your instance configuration. By default, your manifest file is located at `/-/jira_connect/app_descriptor.json`. For example, if your GitLab self-managed instance domain is `app.pet-store.cloud`, your manifest file is located at `https://app.pet-store.cloud/-/jira_connect/app_descriptor.json`. 1. Select **Upload**. Jira fetches the content of your `app_descriptor` file and installs it. 1. If the upload is successful, Jira displays a modal panel: **Installed and ready to go!** diff --git a/doc/integration/jira/development_panel.md b/doc/integration/jira/development_panel.md index 8f66edcffa8..66810945d19 100644 --- a/doc/integration/jira/development_panel.md +++ b/doc/integration/jira/development_panel.md @@ -20,7 +20,7 @@ The information displayed in the Jira Development panel depends on where you men | Your mention of Jira issue ID in GitLab context | Automated effect in Jira issue | |---------------------------------------------------|--------------------------------------------------------------------------------------------------------| -| In a merge request | Link to the MR is displayed in Development panel. | +| In a merge request title or description | Link to the MR is displayed in Development panel. | | In a branch name | Link to the branch is displayed in Development panel. | | In a commit message | Link to the commit is displayed in Development panel. | | In a commit message with Jira [Smart Commits](https://confluence.atlassian.com/fisheye/using-smart-commits-960155400.html) | Displays your custom comment or logged time spent and/or performs specified issue transition on merge. | diff --git a/doc/integration/omniauth.md b/doc/integration/omniauth.md index 4c05f94148c..7a4bcba25e4 100644 --- a/doc/integration/omniauth.md +++ b/doc/integration/omniauth.md @@ -26,7 +26,7 @@ GitLab supports the following OmniAuth providers. | [Auth0](auth0.md) | `auth0` | | [Authentiq](../administration/auth/authentiq.md) | `authentiq` | | [AWS Cognito](../administration/auth/cognito.md) | `cognito` | -| [Azure v2](azure.md#microsoft-azure-oauth-20-omniauth-provider-v2) | `azure_activedirectory_v2` | +| [Azure v2](azure.md) | `azure_activedirectory_v2` | | [Azure v1](azure.md) | `azure_oauth2` | | [Bitbucket Cloud](bitbucket.md) | `bitbucket` | | [CAS](cas.md) | `cas3` | @@ -326,7 +326,7 @@ configuration to redirect login requests to your OmniAuth provider for authentication. This removes the need to select the provider before signing in. For example, to enable automatic sign-in for the -[Azure v2 integration](azure.md#microsoft-azure-oauth-20-omniauth-provider-v2): +[Azure v2 integration](azure.md): - **For Omnibus package** diff --git a/doc/integration/security_partners/index.md b/doc/integration/security_partners/index.md index 2c7641124a0..50a7b3b717b 100644 --- a/doc/integration/security_partners/index.md +++ b/doc/integration/security_partners/index.md @@ -1,7 +1,7 @@ --- stage: Secure group: Static Analysis -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: index --- @@ -12,16 +12,16 @@ each security partner: <!-- vale gitlab.Spelling = NO --> -- [Accurics](https://readme.accurics.com/1409/) - [Anchore](https://docs.anchore.com/current/docs/using/integration/ci_cd/gitlab/) - [Bridgecrew](https://docs.bridgecrew.io/docs/integrate-with-gitlab-self-managed) - [Checkmarx](https://checkmarx.atlassian.net/wiki/spaces/SD/pages/1929937052/GitLab+Integration) - [Deepfactor](https://docs.deepfactor.io/hc/en-us/articles/1500008981941) - [GrammaTech](https://www.grammatech.com/codesonar-gitlab-integration) -- [Indeni](https://cloudrail.app/doc/integrate-with-ci-cd/gitlab-instructions/) +- [Indeni](https://docs.cloudrail.app/#/integrations/gitlab) - [JScrambler](https://docs.jscrambler.com/code-integrity/documentation/gitlab-ci-integration) - [Semgrep](https://semgrep.dev/for/gitlab) - [StackHawk](https://docs.stackhawk.com/continuous-integration/gitlab.html) +- [Tenable](https://docs.tenable.com/tenableio/Content/ContainerSecurity/GetStarted.htm) - [Venafi](https://marketplace.venafi.com/details/gitlab-ci-cd/) - [Veracode](https://community.veracode.com/s/knowledgeitem/gitlab-ci-MCEKSYPRWL35BRTGOVI55SK5RI4A) - [WhiteSource](https://www.whitesourcesoftware.com/gitlab/) diff --git a/doc/integration/twitter.md b/doc/integration/twitter.md index e1f67df76c3..3aba6b70b94 100644 --- a/doc/integration/twitter.md +++ b/doc/integration/twitter.md @@ -9,9 +9,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w To enable the Twitter OmniAuth provider you must register your application with Twitter. Twitter generates a client ID and secret key for you to use. -1. Sign in to [Twitter Application Management](https://developer.twitter.com/apps). +1. Sign in to [Twitter Application Management](https://apps.twitter.com). -1. Select "Create new app" +1. Select "Create new app". 1. Fill in the application details. - Name: This can be anything. Consider something like `<Organization>'s GitLab` or `<Your Name>'s GitLab` or diff --git a/doc/operations/error_tracking.md b/doc/operations/error_tracking.md index 6f97f002a32..907c59adacb 100644 --- a/doc/operations/error_tracking.md +++ b/doc/operations/error_tracking.md @@ -41,8 +41,10 @@ least Maintainer [permissions](../user/permissions.md) to enable the Sentry inte 1. Sign up to Sentry.io or [deploy your own](#deploying-sentry) Sentry instance. 1. [Create](https://docs.sentry.io/product/sentry-basics/guides/integrate-frontend/create-new-project/) a new Sentry project. For each GitLab project that you want to integrate, we recommend that you create a new Sentry project. -1. [Find or generate](https://docs.sentry.io/api/auth/) a Sentry auth token for your Sentry project. - Make sure to give the token at least the following scopes: `event:read`, `project:read`, and `event:write` (for resolving events). +1. Find or generate a [Sentry auth token](https://docs.sentry.io/api/auth/#auth-tokens). + For the SaaS version of Sentry, you can find or generate the auth token at [https://sentry.io/api/](https://sentry.io/api/). + Make sure to give the token at least the following scopes: `project:read`, `event:read`, and + `event:write` (for resolving events). 1. In GitLab, enable error tracking: 1. On the top bar, select **Menu > Projects** and find your project. 1. On the left sidebar, select **Monitor > Error Tracking**. @@ -127,7 +129,17 @@ If another event occurs, the error reverts to unresolved. ## Integrated error tracking -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/329596) in GitLab 14.4. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/329596) in GitLab 14.4. +> - [Disabled](https://gitlab.com/gitlab-org/gitlab/-/issues/353639) in GitLab 14.9 [with a flag](../administration/feature_flags.md) named `integrated_error_tracking`. Disabled by default. + +FLAG: +By default this feature is not available. To make it available on self-managed GitLab, ask an +administrator to [enable the feature flag](../administration/feature_flags.md) +named `integrated_error_tracking`. The feature is not ready for production use. +On GitLab.com, this feature is not available. + +WARNING: +Turning on integrated error tracking may impact performance, depending on your error rates. Integrated error tracking is a lightweight alternative to Sentry backend. You still use Sentry SDK with your application. But you don't need to deploy Sentry diff --git a/doc/operations/feature_flags.md b/doc/operations/feature_flags.md index 8903e99ab3b..e7d78beb0b9 100644 --- a/doc/operations/feature_flags.md +++ b/doc/operations/feature_flags.md @@ -43,12 +43,7 @@ To create and enable a feature flag: 1. Enter a name that starts with a letter and contains only lowercase letters, digits, underscores (`_`), or dashes (`-`), and does not end with a dash (`-`) or underscore (`_`). 1. Optional. Enter a description (255 characters maximum). -1. Enter details about how the flag should be applied: - - In GitLab 13.0 and earlier, add **Environment specs**. For each environment, - include the **Status** (default enabled) and [**Rollout strategy**](#rollout-strategy-legacy) - (defaults to **All users**). - - In GitLab 13.1 and later, add Feature Flag [**Strategies**](#feature-flag-strategies). - For each strategy, include the **Type** (defaults to [**All users**](#all-users)) +1. Add Feature Flag [**Strategies**](#feature-flag-strategies) to define how the flag should be applied. For each strategy, include the **Type** (defaults to [**All users**](#all-users)) and **Environments** (defaults to all environments). 1. Select **Create feature flag**. @@ -163,21 +158,6 @@ WARNING: The Unleash client **must** be given a user ID for the feature to be enabled for target users. See the [Ruby example](#ruby-application-example) below. -## Search for Code References **(PREMIUM)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/300299) in GitLab 14.4. - -Search your project and find any references of a feature flag in your -code so that you can clean it up when it's time to remove the feature flag. - -To search for code references of a feature flag: - -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Deployments > Feature Flags**. -1. Edit the feature flag you want to remove. -1. Select **More actions** (**{ellipsis_v}**). -1. Select **Search code references**. - ### User List > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35930) in GitLab 13.1. @@ -235,34 +215,20 @@ To remove users from a user list: 1. Select **Edit** (**{pencil}**) next to the list you want to change. 1. Select **Remove** (**{remove}**) next to the ID you want to remove. -## Rollout strategy (legacy) - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/8240) in GitLab 12.2. -> - [Made read-only](https://gitlab.com/gitlab-org/gitlab/-/issues/220228) in GitLab 13.4. - -In GitLab 13.0 and earlier, the **Rollout strategy** setting affects which users experience -the feature as enabled. Choose the percentage of users that the feature is enabled -for. The rollout strategy has no effect if the environment spec is disabled. - -It can be set to: - -- All users -- [Percent of users](#percent-of-users) - - Optionally, you can click the **Include additional user IDs** checkbox and add a list - of specific users IDs to enable the feature for. -- [User IDs](#user-ids) +## Search for Code References **(PREMIUM)** -## Legacy feature flag migration +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/300299) in GitLab 14.4. -Legacy feature flags became read-only in GitLab 13.4. GitLab 14.0 removes support for legacy feature -flags. You must migrate your legacy feature flags to the new version. To do so, follow these steps: +Search your project and find any references of a feature flag in your +code so that you can clean it up when it's time to remove the feature flag. -1. Take a screenshot of the legacy flag for tracking. -1. Delete the flag through the API or UI (you don't need to alter the code). -1. Create a new feature flag with the same name as the legacy flag you deleted. Make sure the - strategies and environments match the deleted flag. +To search for code references of a feature flag: -See [this video tutorial](https://www.youtube.com/watch?v=CAJY2IGep7Y) for help with this migration. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Deployments > Feature Flags**. +1. Edit the feature flag you want to remove. +1. Select **More actions** (**{ellipsis_v}**). +1. Select **Search code references**. ## Disable a feature flag for a specific environment @@ -441,3 +407,49 @@ click the `+` button and input the issue reference number or the full URL of the The issues then appear in the related feature flag and the other way round. This feature is similar to the [linked issues](../user/project/issues/related_issues.md) feature. + +## Performance factors + +In general, GitLab Feature Flags can be used in any applications, +however, if it's a large application, it could require an additional configuration in advance. +This section explains the performance factors to help your organization to identify +what's needed to be done before using the feature. +Please read [How it works](#how-it-works) section before diving into the details. + +### Maximum supported clients in application nodes + +GitLab accepts client requests as much as possible until it hits the [rate limiting](../security/rate_limits.md). +At the moment, the Feature Flag API falls into **Unauthenticated traffic (from a given IP address)** +in the [GitLab.com specific limits](../user/gitlab_com/index.md), +so it's **500 requests per minute**. + +Please note that the polling rate is configurable in SDKs. Provided that all clients are requesting from the same IP: + +- Request once per minute ... 500 clients can be supported. +- Request once per 15 sec ... 125 clients can be supported. + +For applications looking for more scalable solution, we recommend to use [Unleash Proxy](#unleash-proxy-example). +This proxy server sits between the server and clients. It requests to the server as a behalf of the client groups, +so the nubmer of outbound requests can be greatly reduced. + +There is also an [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/295472) to give more +capacity to the current rate limit. + +### Recovering from network errors + +In general, [Unleash clients](https://github.com/Unleash/unleash#unleash-sdks) have +a fall-back mechanism when the server returns an error code. +For example, `unleash-ruby-client` reads flag data from the local backup so that +application can keep running in the current state. + +Please reads the documentation in a SDK project for more information. + +### Self-managed GitLab + +Functionality-wise, there are no differences. Both SaaS and self-managed behave the same. + +In terms of scalability, it's up to the spec of the GitLab instance. +For example, GitLab.com runs on HA architecture so that it can handle a lot of requests concurrently, +however, a self-managed instance runs on a low spec machine can't expect the same result. +Please see [Reference architectures](../administration/reference_architectures/index.md) +for more information. diff --git a/doc/operations/incident_management/alerts.md b/doc/operations/incident_management/alerts.md index b03955edf87..0ad2b5ecf3b 100644 --- a/doc/operations/incident_management/alerts.md +++ b/doc/operations/incident_management/alerts.md @@ -144,6 +144,7 @@ The following actions result in a system note: - [Updating the status of an alert](#update-an-alerts-status) - [Creating an incident based on an alert](#create-an-incident-from-an-alert) - [Assignment of an alert to a user](#assign-an-alert) +- [Escalation of an alert to on-call responders](paging.md#escalating-an-alert)  @@ -153,8 +154,26 @@ There are different actions available in GitLab to help triage and respond to al ### Update an alert's status -The Alert detail view enables you to update the Alert Status. -See [Create and manage alerts in GitLab](alerts.md) for more details. +**Triggered** is the default status for new alerts. For users with the Developer role or higher, the +alert status can be updated from these locations: + +- [Alert list](#alert-list): select the status dropdown corresponding to an alert, then select an + alternate status. +- [Alert details page](#alert-details-page): select **Edit** in the right-hand side bar, then select + an alternate status. + +To stop email notifications for alert reoccurrences in projects with [email notifications enabled](paging.md#email-notifications-for-alerts), +[change the alert's status](alerts.md#update-an-alerts-status) away from **Triggered**. + +In projects with GitLab Premium, on-call responders can respond to [alert pages](paging.md#escalating-an-alert) +by changing the status. Setting the status to: + +- **Resolved** silences all on-call pages for the alert. +- **Acknowledged** limits on-call pages based on the project's [escalation policy](escalation_policies.md). +- **Triggered** from **Resolved** restarts the alert escalating from the beginning. + +For [alerts with an associated incident](alerts.md#create-an-incident-from-an-alert), +updating the alert status also updates the incident status. ### Create an incident from an alert @@ -165,8 +184,10 @@ description populated from an alert. To create the issue, select the **Create Issue** button. You can then view the issue from the alert by selecting the **View Issue** button. -Closing a GitLab issue associated with an alert changes the alert's status to -Resolved. See [Create and manage alerts in GitLab](alerts.md) for more details +You can also [create incidents for alerts automatically](incidents.md#create-incidents-automatically). + +Closing a GitLab issue associated with an alert [changes the alert's status](#update-an-alerts-status) to +**Resolved**. See [Alert List](#alert-list) for more details about alert statuses. ### Assign an alert @@ -196,7 +217,7 @@ To assign an alert: After completing their portion of investigating or fixing the alert, users can unassign themselves from the alert. To remove an assignee, select **Edit** next to the **Assignee** dropdown menu -and deselect the user from the list of assignees, or select **Unassigned**. +and clear the user from the list of assignees, or select **Unassigned**. ### Create a to-do item from an alert @@ -217,13 +238,3 @@ add a to-do item:  To view your To-Do List, on the top bar, select **To-Do List** (**{todo-done}**). - -## View the environment that generated the alert - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/232492) in GitLab 13.5 behind a feature flag, disabled by default. -> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/232492) in GitLab 13.6. - -WARNING: -This feature might not be available to you. Check the **version history** note above for details. - -The environment information and the link are displayed in the [Alert Details tab](#alert-details-tab). diff --git a/doc/operations/incident_management/img/incident_list_v14_9.png b/doc/operations/incident_management/img/incident_list_v14_9.png Binary files differnew file mode 100644 index 00000000000..4cf6f0e6522 --- /dev/null +++ b/doc/operations/incident_management/img/incident_list_v14_9.png diff --git a/doc/operations/incident_management/img/incident_metrics_tab_text_link_modal_v14_9.png b/doc/operations/incident_management/img/incident_metrics_tab_text_link_modal_v14_9.png Binary files differnew file mode 100644 index 00000000000..1b045a13fc5 --- /dev/null +++ b/doc/operations/incident_management/img/incident_metrics_tab_text_link_modal_v14_9.png diff --git a/doc/operations/incident_management/img/metric_image_url_dialog_v13_8.png b/doc/operations/incident_management/img/metric_image_url_dialog_v13_8.png Binary files differdeleted file mode 100644 index 732921bbb9f..00000000000 --- a/doc/operations/incident_management/img/metric_image_url_dialog_v13_8.png +++ /dev/null diff --git a/doc/operations/incident_management/incidents.md b/doc/operations/incident_management/incidents.md index e142d060d87..c7ed386f505 100644 --- a/doc/operations/incident_management/incidents.md +++ b/doc/operations/incident_management/incidents.md @@ -54,7 +54,7 @@ GitLab to create incident automatically whenever an alert is triggered: 1. Check the **Create an incident** checkbox. 1. To customize the incident, select an [issue template](../../user/project/description_templates.md#create-an-issue-template). -1. To send [an email notification](paging.md#email-notifications) to users +1. To send [an email notification](paging.md#email-notifications-for-alerts) to users with the Developer role, select **Send a separate email notification to Developers**. Email notifications are also sent to users with the **Maintainer** and **Owner** roles. @@ -89,9 +89,9 @@ For users with at least Guest [permissions](../../user/permissions.md), the Incident list is available at **Monitor > Incidents** in your project's sidebar. The list contains the following metrics: - + -- **Status** - To filter incidents by their status, click **Open**, **Closed**, +- **State** - To filter incidents by their state, select **Open**, **Closed**, or **All** above the incident list. - **Search** - The Incident list supports a simple free text search, which filters on the **Title** and **Incident** fields. @@ -108,6 +108,13 @@ in your project's sidebar. The list contains the following metrics: - **Incident** - The description of the incident, which attempts to capture the most meaningful data. +- **Status** - The status of the incident, which can be one of the following values: + - **Triggered** + - **Acknowledged** + - **Resolved** + + In GitLab Premium, this field is also linked to [on-call escalation](paging.md#escalating-an-incident) for the incident. + - **Date created** - How long ago the incident was created. This field uses the standard GitLab pattern of `X time ago`, but is supported by a granular date/time tooltip depending on the user's locale. @@ -173,9 +180,11 @@ charts in the **Metrics** tab:  -When you upload an image, you can associate it with a URL to the original graph. Users can access the original graph by clicking the image: +When you upload an image, you can associate the image with text or a link to the original graph. - + + +If you add a link, you can access the original graph by clicking the hyperlink above the uploaded image. ### Alert details @@ -226,7 +235,7 @@ There are different actions available to help triage and respond to incidents. ### Assign incidents Assign incidents to users that are actively responding. Select **Edit** in the -right-hand side bar to select or deselect assignees. +right-hand side bar to select or clear assignees. ### Associate a milestone @@ -244,6 +253,40 @@ You can also change the severity using the [`/severity` quick action](../../user Add a to-do for incidents that you want to track in your to-do list. Click the **Add a to do** button at the top of the right-hand side bar to add a to-do item. +### Change incident status + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5716) in GitLab 14.9. + +For users with the Developer role or higher, select **Edit** in the **Status** section of the +right-hand side bar of an incident, then select a status. **Triggered** is the default status for +new incidents. + +In projects with GitLab Premium, on-call responders can respond to [incident pages](paging.md#escalating-an-incident) +by changing the status. Setting the status to: + +- **Resolved** silences on-call pages for the alert. +- **Acknowledged** limits on-call pages based on the selected [escalation policy](#change-escalation-policy). +- **Triggered** from **Resolved** restarts the incident escalating from the beginning. + +For [incidents created from alerts](alerts.md#create-an-incident-from-an-alert), +updating the incident status also updates the alert status. + +## Change escalation policy **(PREMIUM)** + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5716) in GitLab 14.9. + +For users with the Developer role or higher, select **Edit** in the **Escalation policy** section of +the right-hand side bar of an incident, then select a policy. By default, new incidents do not have +an escalation policy selected. + +Selecting an escalation policy updates the incident status to **Triggered** and begins +[escalating the incident to on-call responders](paging.md#escalating-an-incident). +Deselecting an escalation policy halts escalation. Refer to the [incident status](#change-incident-status) +to manage on-call paging once escalation has begun. + +For [incidents created from alerts](alerts.md#create-an-incident-from-an-alert), +the incident's escalation policy reflects the alert's escalation policy and cannot be changed. + ### Manage incidents from Slack Slack slash commands allow you to control GitLab and view GitLab content without leaving Slack. diff --git a/doc/operations/incident_management/oncall_schedules.md b/doc/operations/incident_management/oncall_schedules.md index 84ab5da77df..9b2e9159429 100644 --- a/doc/operations/incident_management/oncall_schedules.md +++ b/doc/operations/incident_management/oncall_schedules.md @@ -10,8 +10,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w Use on-call schedule management to create schedules for responders to rotate on-call responsibilities. Maintain the availability of your software services by putting your teams on-call. -With an on-call schedule, your team is notified immediately when things go wrong so they can quickly -respond to service outages and disruptions. +With [escalation policies](escalation_policies.md) and on-call schedules, your team is notified immediately +when things go wrong so they can quickly respond to service outages and disruptions. To use on-call schedules: @@ -111,9 +111,7 @@ Hover over any rotation shift participants in the schedule to view their individ ## Page an on-call responder -When an alert is created in a project, GitLab sends an email to the on-call responder(s) in the -on-call schedule for that project. If there is no schedule or no one on-call in that schedule at the -time the alert is triggered, no email is sent. +See [Paging](paging.md#paging) for more details. ## Removal or deletion of on-call user diff --git a/doc/operations/incident_management/paging.md b/doc/operations/incident_management/paging.md index b6f77de3b4f..27854a9e201 100644 --- a/doc/operations/incident_management/paging.md +++ b/doc/operations/incident_management/paging.md @@ -21,7 +21,7 @@ receive a **single** page via Slack. To set up Slack notifications on your mobil device, make sure to enable notifications for the Slack app on your phone so you never miss a page. -## Email notifications +## Email notifications for alerts Email notifications are available in projects for triggered alerts. Project members with the **Owner** or **Maintainer** roles have the option to receive @@ -34,8 +34,30 @@ a single email notification for new alerts. **Send a single email notification to Owners and Maintainers for new alerts** checkbox. 1. Select **Save changes**. +[Update the alert's status](alerts.md#update-an-alerts-status) to manage email notifications for an alert. + ## Paging **(PREMIUM)** -In projects that have an [on-call schedule](oncall_schedules.md) configured, on-call responders are -paged through email for triggered alerts. The on-call responder(s) receive one email for triggered -alerts. +In projects that have an [escalation policy](escalation_policies.md) configured, on-call responder(s) +can be automatically paged about critical problems through email. + +### Escalating an alert + +When an alert is triggered, it begins escalating to the on-call responders immediately. +For each escalation rule in the project's escalation policy, the designated on-call +responders receive one email when the rule fires. You can respond to a page +or stop alert escalations by [updating the alert's status](alerts.md#update-an-alerts-status). + +### Escalating an incident + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5716) in GitLab 14.9. + +For incidents, paging on-call responders is optional for each individual incident. +To begin escalating the incident, [set the incident's escalation policy](incidents.md#change-escalation-policy). +For each escalation rule, the designated on-call responders receive one email when +the rule fires. You can respond to a page or stop incident escalations by +[updating the incident's status](incidents.md#change-incident-status) or, if applicable, +[unsetting the incident's escalation policy](incidents.md#change-escalation-policy). + +To avoid duplicate pages, [incidents created from alerts](alerts.md#create-an-incident-from-an-alert) do not support independent escalation. +Instead, the status and escalation policy fields are synced between the alert and the incident. diff --git a/doc/operations/metrics/dashboards/panel_types.md b/doc/operations/metrics/dashboards/panel_types.md index 09e969e8af6..734b560bf13 100644 --- a/doc/operations/metrics/dashboards/panel_types.md +++ b/doc/operations/metrics/dashboards/panel_types.md @@ -234,7 +234,7 @@ For example, if you have a query value of `53.6`, adding `%` as the unit results ## Gauge WARNING: -This panel type is an _alpha_ feature, and is subject to change at any time +This panel type is an [Alpha](../../../policy/alpha-beta-support.md#alpha-features) feature, and is subject to change at any time without prior notice! > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207044) in GitLab 13.3. diff --git a/doc/operations/metrics/dashboards/templating_variables.md b/doc/operations/metrics/dashboards/templating_variables.md index 531693d032f..d751a96f379 100644 --- a/doc/operations/metrics/dashboards/templating_variables.md +++ b/doc/operations/metrics/dashboards/templating_variables.md @@ -27,7 +27,7 @@ described [in Using Variables](variables.md). ## `text` variable type WARNING: -This variable type is an _alpha_ feature, and is subject to change at any time +This variable type is an [Alpha](../../../policy/alpha-beta-support.md#alpha-features) feature, and is subject to change at any time without prior notice! For each `text` variable defined in the dashboard YAML, a free text field displays @@ -64,7 +64,7 @@ templating: ## `custom` variable type WARNING: -This variable type is an _alpha_ feature, and is subject to change at any time +This variable type is an [Alpha](../../../policy/alpha-beta-support.md#alpha-features) feature, and is subject to change at any time without prior notice! Each `custom` variable defined in the dashboard YAML creates a dropdown @@ -112,7 +112,7 @@ templating: ## `metric_label_values` variable type WARNING: -This variable type is an _alpha_ feature, and is subject to change at any time +This variable type is an [Alpha](../../../policy/alpha-beta-support.md#alpha-features) feature, and is subject to change at any time without prior notice! ### Full syntax diff --git a/doc/push_rules/push_rules.md b/doc/push_rules/push_rules.md index 3c4b0e10bf9..ab87cba9da3 100644 --- a/doc/push_rules/push_rules.md +++ b/doc/push_rules/push_rules.md @@ -6,4 +6,6 @@ remove_date: '2022-05-10' This document was moved to [another location](../user/project/repository/push_rules.md). <!-- This redirect file can be deleted after <2022-05-10>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/raketasks/backup_restore.md b/doc/raketasks/backup_restore.md index 30cd0f8f511..cef52cc61d7 100644 --- a/doc/raketasks/backup_restore.md +++ b/doc/raketasks/backup_restore.md @@ -434,7 +434,7 @@ gitlab_rails['backup_upload_storage_options'] = { ###### SSE-KMS To enable SSE-KMS, you'll need the [KMS key via its Amazon Resource Name (ARN) -in the `arn:aws:kms:region:acct-id:key/key-id` format](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingKMSEncryption.html). Under the `backup_upload_storage_options` config setting, set: +in the `arn:aws:kms:region:acct-id:key/key-id` format](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingKMSEncryption.html). Under the `backup_upload_storage_options` configuration setting, set: - `server_side_encryption` to `aws:kms`. - `server_side_encryption_kms_key_id` to the ARN of the key. @@ -1191,7 +1191,7 @@ has a longer discussion explaining the potential problems. To prevent writes to the Git repository data, there are two possible approaches: -- Use [maintenance mode](../administration/maintenance_mode/index.md) **(PREMIUM SELF)** to place GitLab in a read-only state. +- Use [maintenance mode](../administration/maintenance_mode/index.md) to place GitLab in a read-only state. - Create explicit downtime by stopping all Gitaly services before backing up the repositories: ```shell @@ -1354,15 +1354,13 @@ To prepare the new server: ```shell sudo rm -f /var/opt/gitlab/redis/dump.rdb - sudo chown <your-linux-username> /var/opt/gitlab/redis - sudo mkdir /var/opt/gitlab/backups - sudo chown <your-linux-username> /var/opt/gitlab/backups + sudo chown <your-linux-username> /var/opt/gitlab/redis /var/opt/gitlab/backups ``` ### Prepare and transfer content from the old server 1. Ensure you have an up-to-date system-level backup or snapshot of the old server. -1. Enable [maintenance mode](../administration/maintenance_mode/index.md) **(PREMIUM SELF)**, +1. Enable [maintenance mode](../administration/maintenance_mode/index.md), if supported by your GitLab edition. 1. Block new CI/CD jobs from starting: 1. Edit `/etc/gitlab/gitlab.rb`, and set the following: @@ -1461,11 +1459,11 @@ To prepare the new server: 1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Monitoring > Background Jobs**. 1. Under the Sidekiq dashboard, verify that the numbers - match with what was shown on the old server. + match with what was shown on the old server. 1. While still under the Sidekiq dashboard, select **Cron** and then **Enable All** to re-enable periodic background jobs. 1. Test that read-only operations on the GitLab instance work as expected. For example, browse through project repository files, merge requests, and issues. -1. Disable [Maintenance Mode](../administration/maintenance_mode/index.md) **(PREMIUM SELF)**, if previously enabled. +1. Disable [Maintenance Mode](../administration/maintenance_mode/index.md), if previously enabled. 1. Test that the GitLab instance is working as expected. 1. If applicable, re-enable [incoming email](../administration/incoming_email.md) and test it is working as expected. 1. Update your DNS or load balancer to point at the new server. @@ -1856,3 +1854,22 @@ To enable it: ```ruby Feature.enable(:gitaly_backup) ``` + +### Incremental repository backups + +> Introduced in GitLab 14.9 [with a flag](../administration/feature_flags.md) named `incremental_repository_backup`. Disabled by default. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `incremental_repository_backup`. +On GitLab.com, this feature is not available. +This feature is not ready for production use. + +Incremental backups can be faster than full backups because they only pack changes since the last backup into the backup +bundle for each repository. Because incremental backups require access to the previous backup, you can't use incremental +backups with tar files. + +To create an incremental backup, run: + +```shell +sudo gitlab-backup create SKIP=tar INCREMENTAL=yes +``` diff --git a/doc/raketasks/features.md b/doc/raketasks/features.md index 3248f7370e8..e2554c1c18d 100644 --- a/doc/raketasks/features.md +++ b/doc/raketasks/features.md @@ -1,34 +1,9 @@ --- -stage: Enablement -group: Distribution -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +redirect_to: 'index.md' +remove_date: '2022-05-24' --- -# Namespaces **(FREE SELF)** +This document was moved to [another location](index.md). -This Rake task enables [namespaces](../user/group/index.md#namespaces) for projects. - -## Enable usernames and namespaces for user projects - -This command enables the namespaces feature. It moves every project in its -namespace folder. - -The **repository location changes as part of this task**, so you must **update all your Git URLs** to -point to the new location. - -To change your username: - -1. In the top-right corner, select your avatar. -1. Select **Edit profile**. -1. On the left sidebar, select **Account**. -1. In the **Change username** section, type the new username. -1. Select **Update username**. - -For example: - -- Old path: `git@example.org:myrepo.git`. -- New path: `git@example.org:username/myrepo.git` or `git@example.org:groupname/myrepo.git`. - -```shell -bundle exec rake gitlab:enable_namespaces RAILS_ENV=production -``` +<!-- This redirect file can be deleted after <2022-05-24>. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/raketasks/list_repos.md b/doc/raketasks/list_repos.md index 35c5e1e3357..57020491197 100644 --- a/doc/raketasks/list_repos.md +++ b/doc/raketasks/list_repos.md @@ -24,7 +24,7 @@ The results use the default ordering of the GitLab Rails application. ## Limit search results To list only projects with recent activity, pass a date with the `SINCE` environment variable. The -time you specify is parsed by the Rails [TimeZone#parse function](https://api.rubyonrails.org/classes/ActiveSupport/TimeZone.html#method-i-parse). +time you specify is parsed by the Rails [`TimeZone#parse` function](https://api.rubyonrails.org/classes/ActiveSupport/TimeZone.html#method-i-parse). ```shell # Omnibus diff --git a/doc/security/rate_limits.md b/doc/security/rate_limits.md index a9b066631e7..cdf99e8377d 100644 --- a/doc/security/rate_limits.md +++ b/doc/security/rate_limits.md @@ -93,7 +93,7 @@ The **rate limit** is 5 requests per minute per user. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/339151) in GitLab 14.7. There is a rate limit per IP address on the `/users/sign_up` endpoint. This is to mitigate attempts to misuse the endpoint. For example, to mass -discover usernames or email addresses in use. +discover usernames or email addresses in use. The **rate limit** is 20 calls per minute per IP address. @@ -113,7 +113,7 @@ The **rate limit** is 10 calls per minute per signed-in user. There is a rate limit for the internal endpoint `/users/:username/exists`, used upon sign up to check if a chosen username has already been taken. This is to mitigate the risk of misuses, such as mass discovery of usernames in use. -The **rate limit** is 20 calls per minute per IP address. +The **rate limit** is 20 calls per minute per IP address. ## Troubleshooting diff --git a/doc/security/two_factor_authentication.md b/doc/security/two_factor_authentication.md index e8bb627ccbd..cab9f6a957e 100644 --- a/doc/security/two_factor_authentication.md +++ b/doc/security/two_factor_authentication.md @@ -33,10 +33,10 @@ To enable 2FA for all users: If you want 2FA enforcement to take effect during the next sign-in attempt, change the grace period to `0`. -## Disable 2FA enforcement through rails console +## Disable 2FA enforcement through Rails console -Using the [rails console](../administration/operations/rails_console.md), enforcing 2FA for -all user can be disabled. Connect to the rails console and run: +Using the [Rails console](../administration/operations/rails_console.md), enforcing 2FA for +all user can be disabled. Connect to the Rails console and run: ```ruby Gitlab::CurrentSettings.update!('require_two_factor_authentication': false) @@ -74,7 +74,7 @@ The following are important notes about 2FA: 2FA enabled, 2FA is **not** required for those individually added members. - If there are multiple 2FA requirements (for example, group + all users, or multiple groups) the shortest grace period is used. -- It is possible to disallow subgroups from setting up their own 2FA requirements: +- It is possible to prevent subgroups from setting up their own 2FA requirements: 1. Go to the top-level group's **Settings > General**. 1. Expand the **Permissions and group features** section. 1. Uncheck the **Allow subgroups to set up their own two-factor authentication rule** field. @@ -108,13 +108,10 @@ reactivate 2FA from scratch if they want to use it again. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/270554) in GitLab 13.7. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/299088) from GitLab Free to GitLab Premium in 13.9. -> - It's [deployed behind a feature flag](../user/feature_flags.md), disabled by default. -> - It's disabled on GitLab.com. -> - It's not recommended for production use. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-2fa-for-git-operations). +> - It's deployed behind a feature flag, disabled by default. -WARNING: -This feature might not be available to you. Check the **version history** note above for details. +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `two_factor_for_cli`. On GitLab.com, this feature is not available. The feature is not ready for production use. This feature flag also affects [session duration for Git Operations when 2FA is enabled](../user/admin_area/settings/account_and_limit_settings.md#customize-session-duration-for-git-operations-when-2fa-is-enabled). Two-factor authentication can be enforced for Git over SSH operations. However, we recommend using [ED25519_SK](../ssh/index.md#ed25519_sk-ssh-keys) or [ECDSA_SK](../ssh/index.md#ecdsa_sk-ssh-keys) SSH keys instead. @@ -135,30 +132,6 @@ After the OTP is verified, Git over SSH operations can be used for a session dur Once an OTP is verified, anyone can run Git over SSH with that private SSH key for the configured [session duration](../user/admin_area/settings/account_and_limit_settings.md#customize-session-duration-for-git-operations-when-2fa-is-enabled). -### Enable or disable 2FA for Git operations - -2FA for Git operations is under development and not -ready for production use. It is deployed behind a feature flag that is -**disabled by default**. [GitLab administrators with access to the GitLab Rails console](../administration/feature_flags.md) -can enable it. - -To enable it: - -```ruby -Feature.enable(:two_factor_for_cli) -``` - -To disable it: - -```ruby -Feature.disable(:two_factor_for_cli) -``` - -The feature flag affects these features: - -- [Two-factor Authentication (2FA) for Git over SSH operations](#2fa-for-git-over-ssh-operations). -- [Customize session duration for Git Operations when 2FA is enabled](../user/admin_area/settings/account_and_limit_settings.md#customize-session-duration-for-git-operations-when-2fa-is-enabled). - <!-- ## Troubleshooting Include any troubleshooting steps that you can foresee. If you know beforehand what issues diff --git a/doc/ssh/index.md b/doc/ssh/index.md index 35ca9a23179..846e5c369bb 100644 --- a/doc/ssh/index.md +++ b/doc/ssh/index.md @@ -408,7 +408,7 @@ If you are using [EGit](https://www.eclipse.org/egit/), you can [add your SSH ke ## Use SSH on Microsoft Windows If you're running Windows 10, you can either use the [Windows Subsystem for Linux (WSL)](https://docs.microsoft.com/en-us/windows/wsl/install) -with [WSL 2](https://docs.microsoft.com/en-us/windows/wsl/install-win10#update-to-wsl-2) which +with [WSL 2](https://docs.microsoft.com/en-us/windows/wsl/install#update-to-wsl-2) which has both `git` and `ssh` preinstalled, or install [Git for Windows](https://gitforwindows.org) to use SSH through Powershell. @@ -479,3 +479,19 @@ ssh: Could not resolve hostname gitlab.example.com: nodename nor servname provid ``` If you receive this error, restart your terminal and try the command again. + +### `Key enrollment failed: invalid format` error + +You may receive the following error when [generating an SSH key pair for a FIDO/U2F hardware security key](#generate-an-ssh-key-pair-for-a-fidou2f-hardware-security-key): + +```shell +Key enrollment failed: invalid format +``` + +You can troubleshoot this by trying the following: + +- Run the `ssh-keygen` command using `sudo`. +- Verify your IDO/U2F hardware security key supports + the key type provided. +- Verify the version of OpenSSH is 8.2 or greater by + running `ssh -v`. diff --git a/doc/subscriptions/index.md b/doc/subscriptions/index.md index 3f9a1898505..6c5543edb58 100644 --- a/doc/subscriptions/index.md +++ b/doc/subscriptions/index.md @@ -154,7 +154,7 @@ To change the namespace linked to a subscription: for that group. 1. Select **Proceed to checkout**. -Subscription charges are calculated based on the total number of users in a group, including its subgroups and nested projects. If the [total number of users](gitlab_com/index.md#view-seat-usage) exceeds the number of seats in your subscription, your account is charged for the additional users and you need to pay for the overage before you can change the linked namespace. +Subscription charges are calculated based on the total number of users in a group, including its subgroups and nested projects. If the [total number of users](gitlab_com/index.md#view-seat-usage) exceeds the number of seats in your subscription, your account is charged for the additional users and you need to pay for the overage before you can change the linked namespace. Only one namespace can be linked to a subscription. diff --git a/doc/subscriptions/self_managed/index.md b/doc/subscriptions/self_managed/index.md index cb9db3673ac..d38b56bb1f8 100644 --- a/doc/subscriptions/self_managed/index.md +++ b/doc/subscriptions/self_managed/index.md @@ -153,13 +153,13 @@ See the [quarterly subscription reconciliation section](../quarterly_reconciliat ### How cloud licensing works -#### Activate your license +#### Add your license 1. When you purchase a GitLab self-managed plan, an activation code is generated. This activation code is sent to the email address associated with the Customers Portal account. 1. In GitLab, on the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Subscription** and paste the activation code in the text field. -1. Select **Activate**. +1. Select **Add license**. The page displays the details of the subscription. @@ -255,8 +255,8 @@ to IP address `104.18.26.123:443` (`customers.gitlab.com`). To subscribe to GitLab through a GitLab self-managed installation: 1. Go to the [Customers Portal](https://customers.gitlab.com/) and purchase a GitLab self-managed plan. -1. After purchase, a license file is sent to the email address associated to the Customers Portal account, - which must be [uploaded to your GitLab instance](../../user/admin_area/license.md#upload-your-license). +1. After purchase, an activation code is sent to the email address associated with the Customers Portal account. + You must [add this code to your GitLab instance](../../user/admin_area/license.md). NOTE: If you're purchasing a subscription for an existing **Free** GitLab self-managed @@ -380,7 +380,7 @@ To add seats to a subscription: The following items are emailed to you: - A payment receipt. You can also access this information in the Customers Portal under [**View invoices**](https://customers.gitlab.com/receipts). -- A new license. [Upload this license](../../user/admin_area/license.md#upload-your-license) to your instance to use it. +- An activation code. [Add this code](../../user/admin_area/license.md) to your instance to use it. ### Renew a subscription @@ -388,7 +388,7 @@ Starting 30 days before a subscription expires, GitLab notifies administrators o We recommend following these steps during renewal: -1. Prune any inactive or unwanted users by [blocking them](../../user/admin_area/moderate_users.md#block-a-user). +1. Prior to the renewal date, prune any inactive or unwanted users by [blocking them](../../user/admin_area/moderate_users.md#block-a-user). 1. Determine if you have a need for user growth in the upcoming subscription. 1. Log in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in) and select the **Renew** button beneath your existing subscription. The **Renew** button remains disabled (grayed-out) until 15 days before a subscription expires. @@ -400,8 +400,8 @@ You can hover your mouse on the **Renew** button to see the date when it will be 1. In the first box, enter the total number of user licenses you'll need for the upcoming year. Be sure this number is at least **equal to, or greater than** the number of billable users in the system at the time of performing the renewal. 1. Enter the number of [users over license](#users-over-license) in the second box for the user overage incurred in your previous subscription term. 1. Review your renewal details and complete the payment process. -1. A license for the renewal term is available for download on the [Manage Purchases](https://customers.gitlab.com/subscriptions) page on the relevant subscription card. Select **Copy license to clipboard** or **Download license** to get a copy. -1. [Upload](../../user/admin_area/license.md#upload-your-license) your new license to your instance. +1. An activation code for the renewal term is available on the [Manage Purchases](https://customers.gitlab.com/subscriptions) page on the relevant subscription card. Select **Copy activation code** to get a copy. +1. [Add the activation code](../../user/admin_area/license.md) to your instance. An invoice is generated for the renewal and available for viewing or download on the [View invoices](https://customers.gitlab.com/receipts) page. If you have difficulty during the renewal process, contact our [support team](https://support.gitlab.com/hc/en-us/requests/new?ticket_form_id=360000071293) for assistance. @@ -421,10 +421,10 @@ The following is emailed to you: - A payment receipt. You can also access this information in the Customers Portal under [**View invoices**](https://customers.gitlab.com/receipts). -- A new license. +- A new activation code for your license. -[Upload the new license](../../user/admin_area/license.md#upload-your-license) to your instance. -The new tier takes effect when the new license is uploaded. +[Add the activation code](../../user/admin_area/license.md) to your instance. +The new tier takes effect when the new license is activated. ## Add or change the contacts for your subscription @@ -447,7 +447,7 @@ an expiration message is displayed to all administrators. For GitLab self-managed instances, you have a 14-day grace period before this occurs. -- To resume functionality, upload a new license. +- To resume functionality, acticate a new license. - To fall back to Free features, delete the expired license. ## Contact Support diff --git a/doc/topics/autodevops/customize.md b/doc/topics/autodevops/customize.md index 089f9f8e7cc..503774ef6b5 100644 --- a/doc/topics/autodevops/customize.md +++ b/doc/topics/autodevops/customize.md @@ -26,7 +26,7 @@ used for the build. Specify either: -- The CI/CD variable `BUILDPACK_URL` according to [`pack`'s specifications](https://buildpacks.io/docs/app-developer-guide/specific-buildpacks/). +- The CI/CD variable `BUILDPACK_URL` according to [`pack`'s specifications](https://buildpacks.io/docs/app-developer-guide/specify-buildpacks/). - A [`project.toml` project descriptor](https://buildpacks.io/docs/app-developer-guide/using-project-descriptor/) with the buildpacks you would like to include. ### Custom buildpacks with Herokuish @@ -190,7 +190,7 @@ You can override the default values in the `values.yaml` file in the `HELM_UPGRADE_VALUES_FILE` [CI/CD variable](#cicd-variables) with the path and name. -Some values can not be overridden with the options above. Settings like `replicaCount` should instead be overridden with the `REPLICAS` +Some values can not be overridden with the options above. Settings like `replicaCount` should instead be overridden with the `REPLICAS` [build and deployment](#build-and-deployment) CI/CD variable. Follow [this issue](https://gitlab.com/gitlab-org/cluster-integration/auto-deploy-image/-/issues/31) for more information. NOTE: @@ -431,7 +431,7 @@ applications. | `HELM_UPGRADE_EXTRA_ARGS` | Allows extra options in `helm upgrade` commands when deploying the application. Note that using quotes doesn't prevent word splitting. | | `INCREMENTAL_ROLLOUT_MODE` | If present, can be used to enable an [incremental rollout](#incremental-rollout-to-production) of your application for the production environment. Set to `manual` for manual deployment jobs or `timed` for automatic rollout deployments with a 5 minute delay each one. | | `K8S_SECRET_*` | Any variable prefixed with [`K8S_SECRET_`](#application-secret-variables) is made available by Auto DevOps as environment variables to the deployed application. | -| `KUBE_CONTEXT` | From GitLab 14.5, can be used to select which context to use from `KUBECONFIG`. When `KUBE_CONTEXT` is blank, the default context in `KUBECONFIG` (if any) will be used. A context must be selected when using the [CI/CD tunnel](../../user/clusters/agent/ci_cd_tunnel.md). | +| `KUBE_CONTEXT` | From GitLab 14.5, can be used to select a context to use from `KUBECONFIG`. When `KUBE_CONTEXT` is blank, the default context in `KUBECONFIG` (if any) is used. A context must be selected when used [with the agent for Kubernetes](../../user/clusters/agent/ci_cd_tunnel.md). | | `KUBE_INGRESS_BASE_DOMAIN` | Can be used to set a domain per cluster. See [cluster domains](../../user/project/clusters/gitlab_managed_clusters.md#base-domain) for more information. | | `KUBE_NAMESPACE` | The namespace used for deployments. When using certificate-based clusters, [this value should not be overwritten directly](../../user/project/clusters/deploy_to_cluster.md#custom-namespace). | | `KUBECONFIG` | The kubeconfig to use for deployments. User-provided values take priority over GitLab-provided values. | diff --git a/doc/topics/autodevops/index.md b/doc/topics/autodevops/index.md index 585d484d3be..c8efc40a4cd 100644 --- a/doc/topics/autodevops/index.md +++ b/doc/topics/autodevops/index.md @@ -7,7 +7,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Auto DevOps **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/38366) in GitLab 11.0. -> - Support for the GitLab Agent was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/299350) in GitLab 14.5. +> - Support for the GitLab agent was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/299350) in GitLab 14.5. GitLab Auto DevOps is a collection of pre-configured features and integrations that work together to support your software delivery process. @@ -172,7 +172,7 @@ To disable it, follow the same process and clear the > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/52447) in GitLab 11.10. When you enable Auto DevOps at group level, the subgroups and projects in that -group inherit the configuration. This saves you time by batch-enabling it +group inherit the configuration. This saves you some time by batch-enabling it rather than enabling individually for each subgroup or project. When enabled for a group, you can still disable Auto DevOps @@ -207,7 +207,7 @@ instance become enabled. This is convenient when you want to run Auto DevOps by default for all projects. You can still disable Auto DevOps individually for the groups and projects where you don't want to run it. -Only GitLab administrators can enable or disable Auto DevOps in the instance +Only GitLab administrators can enable or disable Auto DevOps at the instance level. Even when disabled for an instance, group owners and project maintainers @@ -234,7 +234,7 @@ and clear the **Default to Auto DevOps pipeline** checkbox. ### Quick start -To guide your through the process of setting up Auto DevOps to deploy to a Kubernetes cluster on +To guide you through the process of setting up Auto DevOps to deploy to a Kubernetes cluster on Google Kubernetes Engine (GKE), see the [quick start guide](quick_start_guide.md). You can also follow the quick start for the general steps, but deploy to diff --git a/doc/topics/autodevops/quick_start_guide.md b/doc/topics/autodevops/quick_start_guide.md index 1fc2bd7c2c0..13d831aa00d 100644 --- a/doc/topics/autodevops/quick_start_guide.md +++ b/doc/topics/autodevops/quick_start_guide.md @@ -76,17 +76,18 @@ to deploy this project to.  -1. On the **Add a Kubernetes cluster integration** page, select the **Create new cluster** tab, - then select **Google GKE**. +1. On the **Kubernetes clusters** page, select the **Create a new cluster** option from the **Actions** dropdown menu. + +1. On the **Connect a Kubernetes cluster** page, select **Google GKE**. 1. Connect with your Google account, and select **Allow** to allow access to your Google account. (This authorization request is only displayed the first time you connect GitLab with your Google account.) - After authorizing access, the **Add a Kubernetes cluster integration** page + After authorizing access, the **Connect a Kubernetes cluster** page is displayed. -1. In the **Enter the details for your Kubernetes cluster** section, provide +1. In the **Enter your Kubernetes cluster certificate details** section, provide details about your cluster: - **Kubernetes cluster name** diff --git a/doc/topics/autodevops/stages.md b/doc/topics/autodevops/stages.md index 8b3966526ec..790b46b6310 100644 --- a/doc/topics/autodevops/stages.md +++ b/doc/topics/autodevops/stages.md @@ -112,7 +112,7 @@ Herokuish, with the following caveats: converted to a Cloud Native Buildpack using Heroku's [`cnb-shim`](https://github.com/heroku/cnb-shim). - `BUILDPACK_URL` must be in a format - [supported by `pack`](https://buildpacks.io/docs/app-developer-guide/specific-buildpacks/). + [supported by `pack`](https://buildpacks.io/docs/app-developer-guide/specify-buildpacks/). - The `/bin/herokuish` command is not present in the built image, and prefixing commands with `/bin/herokuish procfile exec` is no longer required (nor possible). Instead, custom commands should be prefixed with `/cnb/lifecycle/launcher` @@ -659,7 +659,7 @@ ciliumNetworkPolicy: #### Enabling Alerts You can also enable alerts. Network policies with alerts are considered only if -[Agent](../../user/clusters/agent/index.md) +the [agent](../../user/clusters/agent/index.md) has been integrated. You can enable alerts as follows: diff --git a/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md b/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md index 8c460247734..b0814ac5076 100644 --- a/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md +++ b/doc/topics/autodevops/upgrading_auto_deploy_dependencies.md @@ -180,7 +180,7 @@ used the `v0.17.0` chart, add `AUTO_DEVOPS_FORCE_DEPLOY_V2`. ## Early adopters -If you want to use the latest beta or unstable version of `auto-deploy-image`, include +If you want to use the latest [Beta](../../policy/alpha-beta-support.md#beta-features) or unstable version of `auto-deploy-image`, include the latest Auto Deploy template into your `.gitlab-ci.yml`: ```yaml @@ -190,7 +190,7 @@ include: ``` WARNING: -Using a beta or unstable `auto-deploy-image` could cause unrecoverable damage to +Using a [Beta](../../policy/alpha-beta-support.md#beta-features) or unstable `auto-deploy-image` could cause unrecoverable damage to your environments. Do not test it with important projects or environments. The next stable template update is [planned for GitLab v14.0](https://gitlab.com/gitlab-org/gitlab/-/issues/232788). diff --git a/doc/topics/cron/index.md b/doc/topics/cron/index.md index affd746f66f..34a51d3d535 100644 --- a/doc/topics/cron/index.md +++ b/doc/topics/cron/index.md @@ -38,7 +38,10 @@ are valid: - Run once a day at midnight: `0 0 * * *` - Run once a week at midnight on Sunday morning: `0 0 * * 0` - Run once a month at midnight of the first day of the month: `0 0 1 * *` +- Run once a month on the 22nd: `0 0 22 * *`) +- Run once a month on the 2nd Monday: `0 0 * * 1#2` - Run once a year at midnight of 1 January: `0 0 1 1 *` +- Run every other Sunday at 0900 hours: `0 9 * * sun%2` For complete cron documentation, refer to the [crontab(5) — Linux manual page](https://man7.org/linux/man-pages/man5/crontab.5.html). diff --git a/doc/topics/git/tags.md b/doc/topics/git/tags.md index 6e0622273bb..8576bcd09ed 100644 --- a/doc/topics/git/tags.md +++ b/doc/topics/git/tags.md @@ -36,6 +36,6 @@ git tag git push origin --tags ``` -## Additional resources +## Related topics - [Tagging](https://git-scm.com/book/en/v2/Git-Basics-Tagging) Git reference page diff --git a/doc/topics/gitlab_flow.md b/doc/topics/gitlab_flow.md index 3bca33b32b7..d35eba0d782 100644 --- a/doc/topics/gitlab_flow.md +++ b/doc/topics/gitlab_flow.md @@ -190,7 +190,7 @@ By doing this, you minimize the length of time during which you have to apply bu After announcing a release branch, only add serious bug fixes to the branch. If possible, first merge these bug fixes into `main`, and then cherry-pick them into the release branch. If you start by merging into the release branch, you might forget to cherry-pick them into `main`, and then you'd encounter the same bug in subsequent releases. -Merging into `main` and then cherry-picking into release is called an "upstream first" policy, which is also practiced by [Google](https://www.chromium.org/chromium-os/chromiumos-design-docs/upstream-first) and [Red Hat](https://www.redhat.com/en/blog/a-community-for-using-openstack-with-red-hat-rdo). +Merging into `main` and then cherry-picking into release is called an "upstream first" policy, which is also practiced by [Google](https://www.chromium.org/chromium-os/chromiumos-design-docs/upstream-first/) and [Red Hat](https://www.redhat.com/en/blog/a-community-for-using-openstack-with-red-hat-rdo). Every time you include a bug fix in a release branch, increase the patch version (to comply with [Semantic Versioning](https://semver.org/)) by setting a new tag. Some projects also have a stable branch that points to the same commit as the latest released branch. In this flow, it is not common to have a production branch (or Git flow `main` branch). diff --git a/doc/topics/index.md b/doc/topics/index.md deleted file mode 100644 index 6d2c839430b..00000000000 --- a/doc/topics/index.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: '../index.md' -remove_date: '2022-02-03' ---- - -This document was moved to [another location](../index.md). - -<!-- This redirect file can be deleted after <2022-02-03>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/topics/offline/quick_start_guide.md b/doc/topics/offline/quick_start_guide.md index 09fae2b1fd5..d1be775268f 100644 --- a/doc/topics/offline/quick_start_guide.md +++ b/doc/topics/offline/quick_start_guide.md @@ -12,12 +12,49 @@ instance entirely offline. ## Installation NOTE: -This guide assumes the server is Ubuntu 18.04. Instructions for other servers may vary. -This guide also assumes the server host resolves as `my-host`, which you should replace with your -server's name. +This guide assumes the server is Ubuntu 20.04 using the [Omnibus installation method](https://docs.gitlab.com/omnibus/) and will be running GitLab [Enterprise Edition](https://about.gitlab.com/install/ce-or-ee/). Instructions for other servers may vary. +This guide also assumes the server host resolves as `my-host.internal`, which you should replace with your +server's FQDN, and that you have access to a different server with Internet access to download the required package files. -Follow the installation instructions [as outlined in the omnibus install -guide](https://about.gitlab.com/install/#ubuntu), but make sure to specify an `http` +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For a video walkthrough of this process, see [Offline GitLab Installation: Downloading & Installing](https://www.youtube.com/watch?v=TJaq4ua2Prw). + +### Download the GitLab package + +You should [manually download the GitLab package](../../update/package/index.md#upgrade-using-a-manually-downloaded-package) and relevant dependencies using a server of the same operating system type that has access to the Internet. + +If your offline environment has no local network access, you must manually transport across the relevant package files through physical media, such as a USB drive or writable DVD. + +In Ubuntu, this can be performed on a server with Internet access using the following commands: + +```shell +# Download the bash script to prepare the repository +curl --silent "https://packages.gitlab.com/install/repositories/gitlab/gitlab-ee/script.deb.sh" | sudo bash + +# Download the gitlab-ee package and dependencies to /var/cache/apt/archives +sudo apt-get install --download-only gitlab-ee + +# Copy the contents of the apt download folder to a mounted media device +sudo cp /var/cache/apt/archives/*.deb /path/to/mount +``` + +### Install the GitLab package + +Prerequisites: + +- Before installing the GitLab package on your offline environment, ensure that you have installed all required dependencies first. + +If you are using Ubuntu, you can install the dependency `.deb` packages you copied across with `dpkg`. Do not install the GitLab package yet. + +```shell +# Navigate to the physical media device +sudo cd /path/to/mount + +# Install the dependency packages +sudo dpkg -i <package_name>.deb +``` + +[Use the relevant commands for your operating system to install the package](../../update/package/index.md#upgrade-using-a-manually-downloaded-package) but make sure to specify an `http` URL for the `EXTERNAL_URL` installation step. Once installed, we can manually configure the SSL ourselves. @@ -25,8 +62,10 @@ It is strongly recommended to setup a domain for IP resolution rather than bind to the server's IP address. This better ensures a stable target for our certs' CN and makes long-term resolution simpler. +The following example for Ubuntu specifies the `EXTERNAL_URL` using HTTP and installs the GitLab package: + ```shell -sudo EXTERNAL_URL="http://my-host.internal" apt-get install gitlab-ee +sudo EXTERNAL_URL="http://my-host.internal" dpkg -i <gitlab_package_name>.deb ``` ## Enabling SSL @@ -38,7 +77,7 @@ Follow these steps to enable SSL for your fresh instance. Note that these steps ```ruby # Update external_url from "http" to "https" - external_url "https://gitlab.example.com" + external_url "https://my-host.internal" # Set Let's Encrypt to false letsencrypt['enable'] = false @@ -159,3 +198,11 @@ If all goes well, this is what you should see: Running hooks in /etc/ca-certificates/update.d... done. ``` + +### Disable Version Check and Service Ping + +The Version Check and Service Ping services improve the GitLab user experience and ensure that +users are on the most up-to-date instances of GitLab. These two services can be turned off for air-gapped +environments so that they do not attempt and fail to reach out to GitLab services. + +Learn more about [disabling usage statistics](../../user/admin_area/settings/usage_statistics.md#enable-or-disable-usage-statistics). diff --git a/doc/topics/release_your_application.md b/doc/topics/release_your_application.md index 64decba33ad..7ed227adcac 100644 --- a/doc/topics/release_your_application.md +++ b/doc/topics/release_your_application.md @@ -26,18 +26,18 @@ deployment using GitLab CI/CD. ### Deploy applications to Kubernetes clusters With the extensive integration between GitLab and Kubernetes, you can safely deploy your applications -to Kubernetes clusters using the [GitLab Agent](../user/clusters/agent/install/index.md). +to Kubernetes clusters using the [GitLab agent](../user/clusters/agent/install/index.md). #### GitOps deployments **(PREMIUM)** -With the [GitLab Agent](../user/clusters/agent/install/index.md), you can perform [pull-based -deployments of Kubernetes manifests](../user/clusters/agent/repository.md#synchronize-manifest-projects). This provides a scalable, secure, and cloud-native +With the [GitLab agent for Kubernetes](../user/clusters/agent/install/index.md), you can perform [pull-based +deployments of Kubernetes manifests](../user/clusters/agent/gitops.md). This provides a scalable, secure, and cloud-native approach to manage Kubernetes deployments. -#### Deploy to Kubernetes with the CI/CD Tunnel +#### Deploy to Kubernetes from GitLab CI/CD -With the [GitLab Agent](../user/clusters/agent/install/index.md), you can perform push-based -deployments with the [CI/CD Tunnel](../user/clusters/agent/ci_cd_tunnel.md). It provides +With the [GitLab agent for Kubernetes](../user/clusters/agent/install/index.md), you can perform [push-based +deployments](../user/clusters/agent/ci_cd_tunnel.md) from GitLab CI/CD. The agent provides a secure and reliable connection between GitLab and your Kubernetes cluster. ### Deploy to AWS with GitLab CI/CD diff --git a/doc/topics/use_gitlab.md b/doc/topics/use_gitlab.md index a7189284fe6..59a933b1441 100644 --- a/doc/topics/use_gitlab.md +++ b/doc/topics/use_gitlab.md @@ -14,7 +14,7 @@ organize your work, create and secure your application, and analyze its performa - [Plan and track work](plan_and_track.md) - [Build your application](build_your_application.md) - [Secure your application](../user/application_security/index.md) -- [Release your application](release_your_application.md) +- [Deploy and release your application](release_your_application.md) - [Monitor application performance](../operations/index.md) - [Monitor runner performance](https://docs.gitlab.com/runner/monitoring/index.html) - [Manage your infrastructure](../user/infrastructure/index.md) diff --git a/doc/tutorials/index.md b/doc/tutorials/index.md index ac05b9945db..91913813256 100644 --- a/doc/tutorials/index.md +++ b/doc/tutorials/index.md @@ -15,6 +15,7 @@ and running quickly. | Topic | Description | Good for beginners | |-------|-------------|--------------------| +| <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> [Introduction to GitLab](https://youtu.be/_4SmIyQ5eis?t=90) (59m 51s) | Walk through recommended processes and example workflows for using GitLab. | **{star}** | | [GitLab 101](https://gitlab.edcast.com/pathways/copy-of-gitlab-certification) | Learn the basics of GitLab in this certification course. | **{star}** | | <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> [Use GitLab for DevOps](https://www.youtube.com/watch?v=7q9Y1Cv-ib0) (12m 34s) | Use GitLab through the entire DevOps lifecycle, from planning to monitoring. | **{star}** | | [Use Markdown at GitLab](../user/markdown.md) | GitLab Flavored Markdown (GFM) is used in many areas of GitLab, for example, in merge requests. | **{star}** | diff --git a/doc/update/deprecations.md b/doc/update/deprecations.md index 2f5146b8161..e89c45c522c 100644 --- a/doc/update/deprecations.md +++ b/doc/update/deprecations.md @@ -36,9 +36,15 @@ For deprecation reviewers (Technical Writers only): https://about.gitlab.com/handbook/marketing/blog/release-posts/#update-the-deprecations-doc --> -## 14.0 +## 14.9 -### NFS for Git repository storage +### GitLab Pages running as daemon + +In 15.0, support for daemon mode for GitLab Pages will be removed. + +**Planned removal milestone: 15.0 (2022-05-22)** + +### GitLab self-monitoring WARNING: This feature will be changed or removed in 15.0 @@ -46,19 +52,11 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -With the general availability of Gitaly Cluster ([introduced in GitLab 13.0](https://about.gitlab.com/releases/2020/05/22/gitlab-13-0-released/)), we have deprecated development (bugfixes, performance improvements, etc) for NFS for Git repository storage in GitLab 14.0. We will continue to provide technical support for NFS for Git repositories throughout 14.x, but we will remove all support for NFS in GitLab 15.0. Please see our official [Statement of Support](https://about.gitlab.com/support/statement-of-support.html#gitaly-and-nfs) for further information. - -Gitaly Cluster offers tremendous benefits for our customers such as: - -- [Variable replication factors](https://docs.gitlab.com/ee/administration/gitaly/index.html#replication-factor). -- [Strong consistency](https://docs.gitlab.com/ee/administration/gitaly/index.html#strong-consistency). -- [Distributed read capabilities](https://docs.gitlab.com/ee/administration/gitaly/index.html#distributed-reads). - -We encourage customers currently using NFS for Git repositories to plan their migration by reviewing our documentation on [migrating to Gitaly Cluster](https://docs.gitlab.com/ee/administration/gitaly/index.html#migrate-to-gitaly-cluster). +GitLab self-monitoring gives administrators of self-hosted GitLab instances the tools to monitor the health of their instances. This feature is deprecated in GitLab 14.9, and is scheduled for removal in 15.0. **Planned removal milestone: 15.0 (2022-05-22)** -### OAuth implicit grant +### GraphQL permissions change for Package settings WARNING: This feature will be changed or removed in 15.0 @@ -66,29 +64,38 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -The OAuth implicit grant authorization flow will be removed in our next major release, GitLab 15.0. Any applications that use OAuth implicit grant should switch to alternative [supported OAuth flows](https://docs.gitlab.com/ee/api/oauth2.html). +The GitLab Package stage offers a Package Registry, Container Registry, and Dependency Proxy to help you manage all of your dependencies using GitLab. Each of these product categories has a variety of settings that can be adjusted using the API. -**Planned removal milestone: 15.0 (2022-05-22)** +The permissions model for GraphQL is being updated. After 15.0, users with the Guest, Reporter, and Developer role can no longer update these settings: -## 14.2 +- [Package Registry settings](https://docs.gitlab.com/ee/api/graphql/reference/#packagesettings) +- [Container Registry cleanup policy](https://docs.gitlab.com/ee/api/graphql/reference/#containerexpirationpolicy) +- [Dependency Proxy time-to-live policy](https://docs.gitlab.com/ee/api/graphql/reference/#dependencyproxyimagettlgrouppolicy) +- [Enabling the Dependency Proxy for your group](https://docs.gitlab.com/ee/api/graphql/reference/#dependencyproxysetting) -### Release CLI distributed as a generic package +**Planned removal milestone: 15.0 (2022-05-22)** -The [release-cli](https://gitlab.com/gitlab-org/release-cli) will be released as a [generic package](https://gitlab.com/gitlab-org/release-cli/-/packages) starting in GitLab 14.2. We will continue to deploy it as a binary to S3 until GitLab 14.5 and stop distributing it in S3 in GitLab 14.6. +### Move `custom_hooks_dir` setting from GitLab Shell to Gitaly -**Planned removal milestone: 14.6 (2021-12-22)** +The [`custom_hooks_dir`](https://docs.gitlab.com/ee/administration/server_hooks.html#create-a-global-server-hook-for-all-repositories) setting is now configured in Gitaly, and will be removed from GitLab Shell in GitLab 15.0. -### Rename Task Runner pod to Toolbox +**Planned removal milestone: 15.0 ()** -The Task Runner pod is used to execute periodic housekeeping tasks within the GitLab application and is often confused with the GitLab Runner. Thus, [Task Runner will be renamed to Toolbox](https://gitlab.com/groups/gitlab-org/charts/-/epics/25). +### Permissions change for downloading Composer dependencies -This will result in the rename of the sub-chart: `gitlab/task-runner` to `gitlab/toolbox`. Resulting pods will be named along the lines of `{{ .Release.Name }}-toolbox`, which will often be `gitlab-toolbox`. They will be locatable with the label `app=toolbox`. +WARNING: +This feature will be changed or removed in 15.0 +as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#breaking-changes). +Before updating GitLab, review the details carefully to determine if you need to make any +changes to your code, settings, or workflow. -**Planned removal milestone: 14.5 (2021-11-22)** +The GitLab Composer repository can be used to push, search, fetch metadata about, and download PHP dependencies. All these actions require authentication, except for downloading dependencies. -## 14.3 +Downloading Composer dependencies without authentication is deprecated in GitLab 14.9, and will be removed in GitLab 15.0. Starting with GitLab 15.0, you must authenticate to download Composer dependencies. -### Audit events for repository push events +**Planned removal milestone: 15.0 (2022-05-22)** + +### htpasswd Authentication for the Container Registry WARNING: This feature will be changed or removed in 15.0 @@ -96,15 +103,15 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -Audit events for [repository events](https://docs.gitlab.com/ee/administration/audit_events.html#repository-push-deprecated) are now deprecated and will be removed in GitLab 15.0. +The Container Registry supports [authentication](https://gitlab.com/gitlab-org/container-registry/-/blob/master/docs/configuration.md#auth) with `htpasswd`. It relies on an [Apache `htpasswd` file](https://httpd.apache.org/docs/2.4/programs/htpasswd.html), with passwords hashed using `bcrypt`. -These events have always been disabled by default and had to be manually enabled with a -feature flag. Enabling them can cause too many events to be generated which can -dramatically slow down GitLab instances. For this reason, they are being removed. +Since it isn't used in the context of GitLab (the product), `htpasswd` authentication will be deprecated in GitLab 14.9 and removed in GitLab 15.0. **Planned removal milestone: 15.0 (2022-05-22)** -### GitLab Serverless +## 14.8 + +### Changes to the `CI_JOB_JWT` WARNING: This feature will be changed or removed in 15.0 @@ -112,13 +119,20 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -[GitLab Serverless](https://docs.gitlab.com/ee/user/project/clusters/serverless/) is a feature set to support Knative-based serverless development with automatic deployments and monitoring. - -We decided to remove the GitLab Serverless features as they never really resonated with our users. Besides, given the continuous development of Kubernetes and Knative, our current implementations do not even work with recent versions. +The `CI_JOB_JWT` will be updated to support a wider variety of cloud providers. It will be changed to match [`CI_JOB_JWT_V2`](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html), but this change may not be backwards compatible for all users, including Hashicorp Vault users. To maintain the current behavior, users can switch to using `CI_JOB_JWT_V1`, or update their configuration in GitLab 15.0 to use the improved `CI_JOB_JWT`. **Planned removal milestone: 15.0 (2022-05-22)** -### Legacy database configuration +### Configurable Gitaly `per_repository` election strategy + +Configuring the `per_repository` Gitaly election strategy is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/352612). +`per_repository` has been the only option since GitLab 14.0. + +This change is part of regular maintenance to keep our codebase clean. + +**Planned removal milestone: 14.9 (2022-03-22)** + +### Container Network and Host Security WARNING: This feature will be changed or removed in 15.0 @@ -126,15 +140,20 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -The syntax of [GitLabs database](https://docs.gitlab.com/omnibus/settings/database.html) -configuration located in `database.yml` is changing and the legacy format is deprecated. The legacy format -supported using a single PostgreSQL adapter, whereas the new format is changing to support multiple databases. The `main:` database needs to be defined as a first configuration item. +All functionality related to GitLab's Container Network Security and Container Host Security categories is deprecated in GitLab 14.8 and scheduled for removal in GitLab 15.0. Users who need a replacement for this functionality are encouraged to evaluate the following open source projects as potential solutions that can be installed and managed outside of GitLab: [AppArmor](https://gitlab.com/apparmor/apparmor), [Cilium](https://github.com/cilium/cilium), [Falco](https://github.com/falcosecurity/falco), [FluentD](https://github.com/fluent/fluentd), [Pod Security Admission](https://kubernetes.io/docs/concepts/security/pod-security-admission/). To integrate these technologies into GitLab, add the desired Helm charts into your copy of the [Cluster Management Project Template](https://docs.gitlab.com/ee/user/clusters/management_project_template.html). Deploy these Helm charts in production by calling commands through the GitLab [Secure CI/CD Tunnel](https://docs.gitlab.com/ee/user/clusters/agent/repository.html#run-kubectl-commands-using-the-cicd-tunnel). -This deprecation mainly impacts users compiling GitLab from source because Omnibus will handle this configuration automatically. +As part of this change, the following specific capabilities within GitLab are now deprecated, and are scheduled for removal in GitLab 15.0: + +- The **Security & Compliance > Threat Monitoring** page. +- The `Network Policy` security policy type, as found on the **Security & Compliance > Policies** page. +- The ability to manage integrations with the following technologies through GitLab: AppArmor, Cilium, Falco, FluentD, and Pod Security Policies. +- All APIs related to the above functionality. + +For additional context, or to provide feedback regarding this change, please reference our open [deprecation issue](https://gitlab.com/groups/gitlab-org/-/epics/7476). **Planned removal milestone: 15.0 (2022-05-22)** -### OmniAuth Kerberos gem +### Dependency Scanning Python 3.9 and 3.6 image deprecation WARNING: This feature will be changed or removed in 15.0 @@ -142,41 +161,88 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -The `omniauth-kerberos` gem will be removed in our next major release, GitLab 15.0. +For those using Dependency Scanning for Python projects, we are deprecating the default `gemnasium-python:2` image which uses Python 3.6 as well as the custom `gemnasium-python:2-python-3.9` image which uses Python 3.9. The new default image as of GitLab 15.0 will be for Python 3.9 as it is a [supported version](https://endoflife.date/python) and 3.6 [is no longer supported](https://endoflife.date/python). -This gem has not been maintained and has very little usage. We therefore plan to remove support for this authentication method and recommend using the Kerberos [SPNEGO](https://en.wikipedia.org/wiki/SPNEGO) integration instead. You can follow the [upgrade instructions](https://docs.gitlab.com/ee/integration/kerberos.html#upgrading-from-password-based-to-ticket-based-kerberos-sign-ins) to upgrade from the `omniauth-kerberos` integration to the supported one. +For users using Python 3.9 or 3.9-compatible projects, you should not need to take action and dependency scanning should begin to work in GitLab 15.0. If you wish to test the new container now please run a test pipeline in your project with this container (which will be removed in 15.0). Use the Python 3.9 image: -Note that we are not deprecating the Kerberos SPNEGO integration, only the old password-based Kerberos integration. +```yaml +gemnasium-python-dependency_scanning: + image: + name: registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python:2-python-3.9 +``` + +For users using Python 3.6, as of GitLab 15.0 you will no longer be able to use the default template for dependency scanning. You will need to switch to use the deprecated `gemnasium-python:2` analyzer image. If you are impacted by this please comment in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/351503) so we can extend the removal if needed. + +For users using the 3.9 special exception image, you must instead use the default value and no longer override your container. To verify if you are using the 3.9 special exception image, check your `.gitlab-ci.yml` file for the following reference: + +```yaml +gemnasium-python-dependency_scanning: + image: + name: registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python:2-python-3.9 +``` + +**Planned removal milestone: 15.0 (2021-05-22)** + +### Deprecate Geo Admin UI Routes + +In GitLab 13.0, we introduced new project and design replication details routes in the Geo Admin UI. These routes are `/admin/geo/replication/projects` and `/admin/geo/replication/designs`. We kept the legacy routes and redirected them to the new routes. In GitLab 15.0, we will remove support for the legacy routes `/admin/geo/projects` and `/admin/geo/designs`. Please update any bookmarks or scripts that may use the legacy routes. **Planned removal milestone: 15.0 (2022-05-22)** -## 14.5 +### Deprecate custom Geo:db:* Rake tasks -### Certificate-based integration with Kubernetes +In GitLab 14.8, we are [replacing the `geo:db:*` Rake tasks with built-in tasks](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/77269/diffs) that are now possible after [switching the Geo tracking database to use Rails' 6 support of multiple databases](https://gitlab.com/groups/gitlab-org/-/epics/6458). +The following `geo:db:*` tasks will be replaced with their corresponding `db:*:geo` tasks: + +- `geo:db:drop` -> `db:drop:geo` +- `geo:db:create` -> `db:create:geo` +- `geo:db:setup` -> `db:setup:geo` +- `geo:db:migrate` -> `db:migrate:geo` +- `geo:db:rollback` -> `db:rollback:geo` +- `geo:db:version` -> `db:version:geo` +- `geo:db:reset` -> `db:reset:geo` +- `geo:db:seed` -> `db:seed:geo` +- `geo:schema:load:geo` -> `db:schema:load:geo` +- `geo:db:schema:dump` -> `db:schema:dump:geo` +- `geo:db:migrate:up` -> `db:migrate:up:geo` +- `geo:db:migrate:down` -> `db:migrate:down:geo` +- `geo:db:migrate:redo` -> `db:migrate:redo:geo` +- `geo:db:migrate:status` -> `db:migrate:status:geo` +- `geo:db:test:prepare` -> `db:test:prepare:geo` +- `geo:db:test:load` -> `db:test:load:geo` +- `geo:db:test:purge` -> `db:test:purge:geo` + +**Planned removal milestone: 15.0 (2022-05-22)** + +### Deprecate feature flag PUSH_RULES_SUPERSEDE_CODE_OWNERS WARNING: -This feature will be changed or removed in 15.6 +This feature will be changed or removed in 15.0 as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#breaking-changes). Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -[We are deprecating the certificate-based integration with Kubernetes](https://about.gitlab.com/blog/2021/11/15/deprecating-the-cert-based-kubernetes-integration/). -The timeline of removal of the integration from the product is planned to happen in two steps, starting with milestone 15.0 and finishing in GitLab version 15.6. +The feature flag `PUSH_RULES_SUPERSEDE_CODE_OWNERS` is being removed in GitLab 15.0. Upon its removal, push rules will supersede CODEOWNERS. The CODEOWNERS feature will no longer be available for access control. -In 15.0, we plan to introduce a feature flag that will allow GitLab Self-Managed customers to keep the certificate-based integration enabled, it will be disabled by default. We plan to remove this feature flag together with the underlying code in GitLab version 15.6. -The certificate-based integration will continue to receive security and -critical fixes, and features built on the integration will continue to work with supported Kubernetes -versions until the final removal in 15.6. +**Planned removal milestone: 15.0 (2022-05-22)** -For a more robust, secure, forthcoming, and reliable integration with Kubernetes, we recommend the use of the -[Kubernetes Agent](https://docs.gitlab.com/ee/user/clusters/agent/) to connect Kubernetes clusters with GitLab. -We provide [migration plans](https://docs.gitlab.com/ee/user/infrastructure/clusters/migrate_to_gitlab_agent.html) in the documentation. +### Deprecate legacy Gitaly configuration methods -For updates and details around this deprecation, follow this [epic](https://gitlab.com/groups/gitlab-org/configure/-/epics/8). +WARNING: +This feature will be changed or removed in 15.0 +as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#breaking-changes). +Before updating GitLab, review the details carefully to determine if you need to make any +changes to your code, settings, or workflow. -**Planned removal milestone: 15.6 (2022-11-22)** +Using environment variables `GIT_CONFIG_SYSTEM` and `GIT_CONFIG_GLOBAL` to configure Gitaly is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/352609). +These variables are being replaced with standard [`config.toml` Gitaly configuration](https://docs.gitlab.com/ee/administration/gitaly/reference.html). -### Converting an instance (shared) runner to a project (specific) runner +GitLab instances that use `GIT_CONFIG_SYSTEM` and `GIT_CONFIG_GLOBAL` to configure Gitaly should switch to configuring using +`config.toml`. + +**Planned removal milestone: 15.0 (2022-05-22)** + +### Elasticsearch 6.8 WARNING: This feature will be changed or removed in 15.0 @@ -184,11 +250,15 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -In GitLab 15.0, we will remove the feature that enables you to convert an instance (shared) runner to a project (specific) runner. Users who need to add a runner to only a particular project can register a runner to the project directly. +Elasticsearch 6.8 is deprecated in GitLab 14.8 and scheduled for removal in GitLab 15.0. +Customers using Elasticsearch 6.8 need to upgrade their Elasticsearch version to 7.x prior to upgrading to GitLab 15.0. +We recommend using the latest version of Elasticsearch 7 to benefit from all Elasticsearch improvements. + +Elasticsearch 6.8 is also incompatible with Amazon OpenSearch, which we [plan to support in GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/327560). **Planned removal milestone: 15.0 (2022-05-22)** -### Known host required for GitLab Runner SSH executor +### External status check API breaking changes WARNING: This feature will be changed or removed in 15.0 @@ -196,13 +266,25 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -In [GitLab 14.3](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/3074), we added a configuration setting in the GitLab Runner `config.toml` file. This setting, [`[runners.ssh.disable_strict_host_key_checking]`](https://docs.gitlab.com/runner/executors/ssh.html#security), controls whether or not to use strict host key checking with the SSH executor. +The [external status check API](https://docs.gitlab.com/ee/api/status_checks.html) was originally implemented to +support pass-by-default requests to mark a status check as passing. Pass-by-default requests are now deprecated. +Specifically, the following are deprecated: -In GitLab 15.0 and later, the default value for this configuration option will change from `true` to `false`. This means that strict host key checking will be enforced when using the GitLab Runner SSH executor. +- Requests that do not contain the `status` field. +- Requests that have the `status` field set to `approved`. + +Beginning in GitLab 15.0, status checks will only be updated to a passing state if the `status` field is both present +and set to `pass`. Requests that: + +- Do not contain the `status` field will be rejected with a `422` error. For more information, see [the relevant issue](https://gitlab.com/gitlab-org/gitlab/-/issues/338827). +- Contain any value other than `pass` will cause the status check to fail. For more information, see [the relevant issue](https://gitlab.com/gitlab-org/gitlab/-/issues/339039). + +To align with this change, API calls to list external status checks will also return the value of `pass` rather than +`approved` for status checks that have passed. **Planned removal milestone: 15.0 (2022-05-22)** -### Must explicitly assign `AuthenticationType` for `[runners.cache.s3]` +### GraphQL ID and GlobalID compatibility WARNING: This feature will be changed or removed in 15.0 @@ -210,13 +292,61 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -In GitLab 15.0 and later, to access the AWS S3 cache, you must specify the `AuthenticationType` for [`[runners.cache.s3]`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runnerscaches3-section). The `AuthenticationType` must be `IAM` or `credentials`. +We are removing a non-standard extension to our GraphQL processor, which we added for backwards compatibility. This extension modifies the validation of GraphQL queries, allowing the use of the `ID` type for arguments where it would normally be rejected. +Some arguments originally had the type `ID`. These were changed to specific +kinds of `ID`. This change may be a breaking change if you: -Prior to 14.5, if you did not define the `AuthenticationType`, GitLab Runner chose a type for you. +- Use GraphQL. +- Use the `ID` type for any argument in your query signatures. -**Planned removal milestone: 15.0 (2022-05-22)** +Some field arguments still have the `ID` type. These are typically for +IID values, or namespace paths. An example is `Query.project(fullPath: ID!)`. -### Package pipelines in API payload is paginated +For a list of affected and unaffected field arguments, +see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352832). + +You can test if this change affects you by validating +your queries locally, using schema data fetched from a GitLab server. +You can do this by using the GraphQL explorer tool for the relevant GitLab +instance. For example: `https://gitlab.com/-/graphql-explorer`. + +For example, the following query illustrates the breaking change: + +```graphql +# a query using the deprecated type of Query.issue(id:) +# WARNING: This will not work after GitLab 15.0 +query($id: ID!) { + deprecated: issue(id: $id) { + title, description + } +} +``` + +The query above will not work after GitLab 15.0 is released, because the type +of `Query.issue(id:)` is actually `IssueID!`. + +Instead, you should use one of the following two forms: + +```graphql +# This will continue to work +query($id: IssueID!) { + a: issue(id: $id) { + title, description + } + b: issue(id: "gid://gitlab/Issue/12345") { + title, description + } +} +``` + +This query works now, and will continue to work after GitLab 15.0. +You should convert any queries in the first form (using `ID` as a named type in the signature) +to one of the other two forms (using the correct appropriate type in the signature, or using +an inline argument expression). + +**Planned removal milestone: 15.0 (2022-04-22)** + +### OAuth tokens without expiration WARNING: This feature will be changed or removed in 15.0 @@ -224,13 +354,19 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -A request to the API for `/api/v4/projects/:id/packages` returns a paginated result of packages. Each package lists all of its pipelines in this response. This is a performance concern, as it's possible for a package to have hundreds or thousands of associated pipelines. +By default, all new applications expire access tokens after 2 hours. In GitLab 14.2 and earlier, OAuth access tokens +had no expiration. In GitLab 15.0, an expiry will be automatically generated for any existing token that does not +already have one. -In milestone 15.0, we will remove the `pipelines` attribute from the API response. +You should [opt in](https://docs.gitlab.com/ee/integration/oauth_provider.html#expiring-access-tokens) to expiring +tokens before GitLab 15.0 is released: + +1. Edit the application. +1. Select **Expire access tokens** to enable them. Tokens must be revoked or they don’t expire. **Planned removal milestone: 15.0 (2022-05-22)** -### REST and GraphQL API Runner status will not return `paused` +### Optional enforcement of PAT expiration WARNING: This feature will be changed or removed in 15.0 @@ -238,17 +374,13 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -The GitLab Runner REST and GraphQL API endpoints will not return `paused` or `active` as a status in GitLab 15.0. - -A runner's status will only relate to runner contact status, such as: -`online`, `offline`, or `not_connected`. Status `paused` or `active` will no longer appear. - -When checking if a runner is `paused`, API users are advised to check the boolean attribute -`paused` to be `true` instead. When checking if a runner is `active`, check if `paused` is `false`. +The feature to disable enforcement of PAT expiration is unusual from a security perspective. +We have become concerned that this unusual feature could create unexpected behavior for users. +Unexpected behavior in a security feature is inherently dangerous, so we have decided to remove this feature. **Planned removal milestone: 15.0 (2022-05-22)** -### Support for SLES 12 SP2 +### Optional enforcement of SSH expiration WARNING: This feature will be changed or removed in 15.0 @@ -256,11 +388,13 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -Long term service and support (LTSS) for SUSE Linux Enterprise Server (SLES) 12 SP2 [ended on March 31, 2021](https://www.suse.com/lifecycle/). The CA certificates on SP2 include the expired DST root certificate, and it's not getting new CA certificate package updates. We have implemented some [workarounds](https://gitlab.com/gitlab-org/gitlab-omnibus-builder/-/merge_requests/191), but we will not be able to continue to keep the build running properly. +The feature to disable enforcement of SSH expiration is unusual from a security perspective. +We have become concerned that this unusual feature could create unexpected behavior for users. +Unexpected behavior in a security feature is inherently dangerous, so we have decided to remove this feature. **Planned removal milestone: 15.0 (2022-05-22)** -### Update to the Container Registry group-level API +### Out-of-the-box SAST support for Java 8 WARNING: This feature will be changed or removed in 15.0 @@ -268,13 +402,22 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -In milestone 15.0, support for the `tags` and `tags_count` parameters will be removed from the Container Registry API that [gets registry repositories from a group](../api/container_registry.md#within-a-group). +The [GitLab SAST SpotBugs analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) scans [Java, Scala, Groovy, and Kotlin code](https://docs.gitlab.com/ee/user/application_security/sast/#supported-languages-and-frameworks) for security vulnerabilities. +For technical reasons, the analyzer must first compile the code before scanning. +Unless you use the [pre-compilation strategy](https://docs.gitlab.com/ee/user/application_security/sast/#pre-compilation), the analyzer attempts to automatically compile your project's code. -The `GET /groups/:id/registry/repositories` endpoint will remain, but won't return any info about tags. To get the info about tags, you can use the existing `GET /registry/repositories/:id` endpoint, which will continue to support the `tags` and `tag_count` options as it does today. The latter must be called once per image repository. +In GitLab versions prior to 15.0, the analyzer image includes Java 8 and Java 11 runtimes to facilitate compilation. + +In GitLab 15.0, we will: + +- Remove Java 8 from the analyzer image to reduce the size of the image. +- Add Java 17 to the analyzer image to make it easier to compile with Java 17. + +If you rely on Java 8 being present in the analyzer environment, you must take action as detailed in the [deprecation issue for this change](https://gitlab.com/gitlab-org/gitlab/-/issues/352549#breaking-change). **Planned removal milestone: 15.0 (2022-05-22)** -### Value Stream Analytics filtering calculation change +### Querying Usage Trends via the `instanceStatisticsMeasurements` GraphQL node WARNING: This feature will be changed or removed in 15.0 @@ -282,13 +425,71 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -We are changing how the date filter works in Value Stream Analytics. Instead of filtering by the time that the issue or merge request was created, the date filter will filter by the end event time of the given stage. This will result in completely different figures after this change has rolled out. - -If you monitor Value Stream Analytics metrics and rely on the date filter, to avoid losing data, you must save the data prior to this change. +The `instanceStatisticsMeasurements` GraphQL node has been renamed to `usageTrendsMeasurements` in 13.10 and the old field name has been marked as deprecated. To fix the existing GraphQL queries, replace `instanceStatisticsMeasurements` with `usageTrendsMeasurements`. **Planned removal milestone: 15.0 (2022-05-22)** -### `Versions` on base `PackageType` +### REST API Runner will not accept `status` filter values of `active` or `paused` + +WARNING: +This feature will be changed or removed in 16.0 +as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#breaking-changes). +Before updating GitLab, review the details carefully to determine if you need to make any +changes to your code, settings, or workflow. + +The GitLab Runner REST endpoints will stop accepting `paused` or `active` as a status value in GitLab 16.0. + +A runner's status will only relate to runner contact status, such as: `online`, `offline`. +Status values `paused` or `active` will no longer be accepted and will be replaced by the `paused` query parameter. + +When checking for paused runners, API users are advised to specify `paused=true` as the query parameter. +When checking for active runners, specify `paused=false`. + +**Planned removal milestone: 16.0 (2023-04-22)** + +### REST API endpoint to list group runners no longer accepts `project_type` value for `type` argument + +WARNING: +This feature will be changed or removed in 16.0 +as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#breaking-changes). +Before updating GitLab, review the details carefully to determine if you need to make any +changes to your code, settings, or workflow. + +The `GET /groups/:id/runners?type=project_type` endpoint will be removed in GitLab 16.0. The endpoint always returned an empty collection. + +**Planned removal milestone: 16.0 (2023-04-22)** + +### REST and GraphQL API Runner usage of `active` replaced by `paused` + +WARNING: +This feature will be changed or removed in 16.0 +as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#breaking-changes). +Before updating GitLab, review the details carefully to determine if you need to make any +changes to your code, settings, or workflow. + +Occurrences of the `active` identifier in the GitLab Runner REST and GraphQL API endpoints will be +renamed to `paused` in GitLab 16.0, namely: + +- GraphQL API: + - the `CiRunner` property; + - the `RunnerUpdateInput` input type for the `runnerUpdate` mutation; + - the `runners` and `Group.runners` queries. +- REST API: + - endpoints taking or returning `active` properties, such as: + - `GET /runners` + - `GET /runners/all` + - `GET /runners/:id` / `PUT /runners/:id` + - `PUT --form "active=false" /runners/:runner_id` + - `GET /projects/:id/runners` / `POST /projects/:id/runners` + - `GET /groups/:id/runners` + +The 16.0 release of the GitLab Runner will start using the `paused` property when registering runners, and therefore +will only be compatible with GitLab 16.0 and later. Until 16.0, GitLab will accept the deprecated `active` flag from +existing runners. + +**Planned removal milestone: 16.0 (2023-04-22)** + +### Reminder: support for NFS repository storage WARNING: This feature will be changed or removed in 15.0 @@ -296,13 +497,22 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -As part of the work to create a [Package Registry GraphQL API](https://gitlab.com/groups/gitlab-org/-/epics/6318), the Package group deprecated the `Version` type for the basic `PackageType` type and moved it to [`PackageDetailsType`](https://docs.gitlab.com/ee/api/graphql/reference/index.html#packagedetailstype). +As [announced](https://about.gitlab.com/releases/2021/06/22/gitlab-14-0-released/#nfs-for-git-repository-storage-deprecated) at the +release of GitLab 14.0, technical support for NFS storage for Git repositories is being removed. Please see our official +[Statement of Support](https://about.gitlab.com/support/statement-of-support.html#gitaly-and-nfs) for additional information. -In milestone 15.0, we will completely remove `Version` from `PackageType`. +We encourage customers currently using NFS for Git repositories to plan their migration by reviewing our documentation on +[migrating to Gitaly Cluster](https://docs.gitlab.com/ee/administration/gitaly/#migrating-to-gitaly-cluster). + +Gitaly Cluster offers tremendous benefits for our customers such as: + +- [Variable replication factors](https://docs.gitlab.com/ee/administration/gitaly/praefect.html#configure-replication-factor) +- [Strong consistency](https://docs.gitlab.com/ee/administration/gitaly/#strong-consistency) +- [Distributed read capabilities](https://docs.gitlab.com/ee/administration/gitaly/#distributed-reads) **Planned removal milestone: 15.0 (2022-05-22)** -### `defaultMergeCommitMessageWithDescription` GraphQL API field +### Request profiling WARNING: This feature will be changed or removed in 15.0 @@ -310,11 +520,17 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -The GraphQL API field `defaultMergeCommitMessageWithDescription` has been deprecated and will be removed in GitLab 15.0. For projects with a commit message template set, it will ignore the template. +[Request profiling](https://docs.gitlab.com/ee/administration/monitoring/performance/request_profiling.html) is deprecated in GitLab 14.8 and scheduled for removal in GitLab 15.0. + +We're working on [consolidating our profiling tools](https://gitlab.com/groups/gitlab-org/-/epics/7327) and making them more easily accessible. +We [evaluated](https://gitlab.com/gitlab-org/gitlab/-/issues/350152) the use of this feature and we found that it is not widely used. +It also depends on a few third-party gems that are not actively maintained anymore, have not been updated for the latest version of Ruby, or crash frequently when profiling heavy page loads. + +For more information, check the [summary section of the deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352488#deprecation-summary). **Planned removal milestone: 15.0 (2022-05-22)** -### `dependency_proxy_for_private_groups` feature flag +### Required pipeline configurations in Premium tier WARNING: This feature will be changed or removed in 15.0 @@ -322,13 +538,16 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -We added a feature flag because [GitLab-#11582](https://gitlab.com/gitlab-org/gitlab/-/issues/11582) changed how public groups use the Dependency Proxy. Prior to this change, you could use the Dependency Proxy without authentication. The change requires authentication to use the Dependency Proxy. +The [required pipeline configuration](https://docs.gitlab.com/ee/user/admin_area/settings/continuous_integration.html#required-pipeline-configuration) feature is deprecated in GitLab 14.8 for Premium customers and is scheduled for removal in GitLab 15.0. This feature is not deprecated for GitLab Ultimate customers. -In milestone 15.0, we will remove the feature flag entirely. Moving forward, you must authenticate when using the Dependency Proxy. +This change to move the feature to GitLab's Ultimate tier is intended to help our features better align with our [pricing philosophy](https://about.gitlab.com/company/pricing/#three-tiers) as we see demand for this feature originating primarily from executives. + +This change will also help GitLab remain consistent in its tiering strategy with the other related Ultimate-tier features of: +[Security policies](https://docs.gitlab.com/ee/user/application_security/policies/) and [compliance framework pipelines](https://docs.gitlab.com/ee/user/project/settings/index.html#compliance-pipeline-configuration). **Planned removal milestone: 15.0 (2022-05-22)** -### `pipelines` field from the `version` field +### Retire-JS Dependency Scanning tool WARNING: This feature will be changed or removed in 15.0 @@ -336,16 +555,13 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -In GraphQL, there are two `pipelines` fields that you can use in a [`PackageDetailsType`](https://docs.gitlab.com/ee/api/graphql/reference/#packagedetailstype) to get the pipelines for package versions: - -- The `versions` field's `pipelines` field. This returns all the pipelines associated with all the package's versions, which can pull an unbounded number of objects in memory and create performance concerns. -- The `pipelines` field of a specific `version`. This returns only the pipelines associated with that single package version. +As of 14.8 the retire.js job is being deprecated from Dependency Scanning. It will continue to be included in our CI/CD template while deprecated. We are removing retire.js from Dependency Scanning on May 22, 2022 in GitLab 15.0. JavaScript scanning functionality will not be affected as it is still being covered by Gemnasium. -To mitigate possible performance problems, we will remove the `versions` field's `pipelines` field in milestone 15.0. Although you will no longer be able to get all pipelines for all versions of a package, you can still get the pipelines of a single version through the remaining `pipelines` field for that version. +If you have explicitly excluded retire.js using DS_EXCLUDED_ANALYZERS you will need to clean up (remove the reference) in 15.0. If you have customized your pipeline's Dependency Scanning configuration related to the `retire-js-dependency_scanning` job you will want to switch to gemnasium-dependency_scanning before the removal in 15.0, to prevent your pipeline from failing. If you have not used the DS_EXCLUDED_ANALYZERS to reference retire.js, or customized your template specifically for retire.js, you will not need to take action. **Planned removal milestone: 15.0 (2022-05-22)** -### `promote-db` command from `gitlab-ctl` +### SAST analyzer consolidation and CI/CD template changes WARNING: This feature will be changed or removed in 15.0 @@ -353,11 +569,30 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -In GitLab 14.5, we introduced the command `gitlab-ctl promote` to promote any Geo secondary node to a primary during a failover. This command replaces `gitlab-ctl promote-db` which is used to promote database nodes in multi-node Geo secondary sites. `gitlab-ctl promote-db` will continue to function as-is and be available until GitLab 15.0. We recommend that Geo customers begin testing the new `gitlab-ctl promote` command in their staging environments and incorporating the new command in their failover procedures. +GitLab SAST uses various [analyzers](https://docs.gitlab.com/ee/user/application_security/sast/analyzers/) to scan code for vulnerabilities. + +We are reducing the number of analyzers used in GitLab SAST as part of our long-term strategy to deliver a better and more consistent user experience. +Streamlining the set of analyzers will also enable faster [iteration](https://about.gitlab.com/handbook/values/#iteration), better [results](https://about.gitlab.com/handbook/values/#results), and greater [efficiency](https://about.gitlab.com/handbook/values/#results) (including a reduction in CI runner usage in most cases). + +In GitLab 15.0, GitLab SAST will no longer use the following analyzers: + +- [ESLint](https://gitlab.com/gitlab-org/security-products/analyzers/eslint) (JavaScript, TypeScript, React) +- [Gosec](https://gitlab.com/gitlab-org/security-products/analyzers/gosec) (Go) +- [Bandit](https://gitlab.com/gitlab-org/security-products/analyzers/bandit) (Python) + +These analyzers will be removed from the [GitLab-managed SAST CI/CD template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml) and replaced with the [Semgrep-based analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep). +They will no longer receive routine updates, except for security issues. +We will not delete container images previously published for these analyzers; any such change would be announced as a [deprecation, removal, or breaking change announcement](https://about.gitlab.com/handbook/marketing/blog/release-posts/#deprecations-removals-and-breaking-changes). + +We will also remove Java from the scope of the [SpotBugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) analyzer and replace it with the [Semgrep-based analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep). +This change will make it simpler to scan Java code; compilation will no longer be required. +This change will be reflected in the automatic language detection portion of the [GitLab-managed SAST CI/CD template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml). + +If you applied customizations to any of the affected analyzers, you must take action as detailed in the [deprecation issue for this change](https://gitlab.com/gitlab-org/gitlab/-/issues/352554#breaking-change). **Planned removal milestone: 15.0 (2022-05-22)** -### `promote-to-primary-node` command from `gitlab-ctl` +### SAST support for .NET 2.1 WARNING: This feature will be changed or removed in 15.0 @@ -365,21 +600,50 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -In GitLab 14.5, we introduced the command `gitlab-ctl promote` to promote any Geo secondary node to a primary during a failover. This command replaces `gitlab-ctl promote-to-primary-node` which was only usable for single-node Geo sites. `gitlab-ctl promote-to-primary-node` will continue to function as-is and be available until GitLab 15.0. We recommend that Geo customers begin testing the new `gitlab-ctl promote` command in their staging environments and incorporating the new command in their failover procedures. +The GitLab SAST Security Code Scan analyzer scans .NET code for security vulnerabilities. +For technical reasons, the analyzer must first build the code to scan it. + +In GitLab versions prior to 15.0, the default analyzer image (version 2) includes support for: + +- .NET 2.1 +- .NET 3.0 and .NET Core 3.0 +- .NET Core 3.1 +- .NET 5.0 + +In GitLab 15.0, we will change the default major version for this analyzer from version 2 to version 3. This change: + +- Adds [severity values for vulnerabilities](https://gitlab.com/gitlab-org/gitlab/-/issues/350408) along with [other new features and improvements](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan/-/blob/master/CHANGELOG.md). +- Removes .NET 2.1 support. +- Adds support for .NET 6.0, Visual Studio 2019, and Visual Studio 2022. + +Version 3 was [announced in GitLab 14.6](https://about.gitlab.com/releases/2021/12/22/gitlab-14-6-released/#sast-support-for-net-6) and made available as an optional upgrade. + +If you rely on .NET 2.1 support being present in the analyzer image by default, you must take action as detailed in the [deprecation issue for this change](https://gitlab.com/gitlab-org/gitlab/-/issues/352553#breaking-change). **Planned removal milestone: 15.0 (2022-05-22)** -### openSUSE Leap 15.2 packages +### Secret Detection configuration variables deprecated -Distribution support and security updates for openSUSE Leap 15.2 are [ending December 2021](https://en.opensuse.org/Lifetime#openSUSE_Leap). +To make it simpler and more reliable to [customize GitLab Secret Detection](https://docs.gitlab.com/ee/user/application_security/secret_detection/#customizing-settings), we're deprecating some of the variables that you could previously set in your CI/CD configuration. -Starting in 14.5 we are providing packages for openSUSE Leap 15.3, and will stop providing packages for openSUSE Leap 15.2 in the 14.8 milestone. +The following variables currently allow you to customize the options for historical scanning, but interact poorly with the [GitLab-managed CI/CD template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/Secret-Detection.gitlab-ci.yml) and are now deprecated: -**Planned removal milestone: 14.8 (2022-02-22)** +- `SECRET_DETECTION_COMMIT_FROM` +- `SECRET_DETECTION_COMMIT_TO` +- `SECRET_DETECTION_COMMITS` +- `SECRET_DETECTION_COMMITS_FILE` -## 14.6 +The `SECRET_DETECTION_ENTROPY_LEVEL` previously allowed you to configure rules that only considered the entropy level of strings in your codebase, and is now deprecated. +This type of entropy-only rule created an unacceptable number of incorrect results (false positives) and is no longer supported. -### API: `stale` status returned instead of `offline` or `not_connected` +In GitLab 15.0, we'll update the Secret Detection [analyzer](https://docs.gitlab.com/ee/user/application_security/terminology/#analyzer) to ignore these deprecated options. +You'll still be able to configure historical scanning of your commit history by setting the [`SECRET_DETECTION_HISTORIC_SCAN` CI/CD variable](https://docs.gitlab.com/ee/user/application_security/secret_detection/#available-cicd-variables). + +For further details, see [the deprecation issue for this change](https://gitlab.com/gitlab-org/gitlab/-/issues/352565). + +**Planned removal milestone: 15.0 (2022-05-22)** + +### Secure and Protect analyzer images published in new location WARNING: This feature will be changed or removed in 15.0 @@ -387,13 +651,25 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -A breaking change will occur for the Runner [API](https://docs.gitlab.com/ee/api/runners.html#runners-api) endpoints in 15.0. +GitLab uses various [analyzers](https://docs.gitlab.com/ee/user/application_security/terminology/#analyzer) to [scan for security vulnerabilities](https://docs.gitlab.com/ee/user/application_security/). +Each analyzer is distributed as a container image. -Instead of the GitLab Runner API endpoints returning `offline` and `not_connected` for runners that have not contacted the GitLab instance in the past three months, the API endpoints will return the `stale` value, which was introduced in 14.6. +Starting in GitLab 14.8, new versions of GitLab Secure and Protect analyzers are published to a new registry location under `registry.gitlab.com/security-products`. + +We will update the default value of [GitLab-managed CI/CD templates](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates/Security) to reflect this change: + +- For all analyzers except Container Scanning, we will update the variable `SECURE_ANALYZERS_PREFIX` to the new image registry location. +- For Container Scanning, the default image address is already updated. There is no `SECURE_ANALYZERS_PREFIX` variable for Container Scanning. + +In a future release, we will stop publishing images to `registry.gitlab.com/gitlab-org/security-products/analyzers`. +Once this happens, you must take action if you manually pull images and push them into a separate registry. This is commonly the case for [offline deployments](https://docs.gitlab.com/ee/user/application_security/offline_deployments/index.html). +Otherwise, you won't receive further updates. + +See the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352564) for more details. **Planned removal milestone: 15.0 (2022-05-22)** -### CI/CD job name length limit +### Secure and Protect analyzer major version update WARNING: This feature will be changed or removed in 15.0 @@ -401,11 +677,42 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -In GitLab 15.0 we are going to limit the number of characters in CI/CD job names to 255. Any pipeline with job names that exceed the 255 character limit will stop working after the 15.0 release. +The Secure and Protect stages will be bumping the major versions of their analyzers in tandem with the GitLab 15.0 release. This major bump will enable a clear delineation for analyzers, between: + +- Those released prior to May 22, 2022, which generate reports that _are not_ subject to stringent schema validation. +- Those released after May 22, 2022, which generate reports that _are_ subject to stringent schema validation. + +If you are not using the default inclusion templates, or have pinned your analyzer version(s) you will need to update your CI/CD job definition to either remove the pinned version or to update the latest major version. +Users of GitLab 12.0-14.10 will continue to experience analyzer updates as normal until the release of GitLab 15.0, following which all newly fixed bugs and newly released features in the new major versions of the analyzers will not be available in the deprecated versions because we do not backport bugs and new features as per our [maintenance policy](https://docs.gitlab.com/ee/policy/maintenance.html). As required security patches will be backported within the latest 3 minor releases. +Specifically, the following are being deprecated and will no longer be updated after 15.0 GitLab release: + +- API Security: version 1 +- Container Scanning: version 4 +- Coverage-guided fuzz testing: version 2 +- Dependency Scanning: version 2 +- Dynamic Application Security Testing (DAST): version 2 +- Infrastructure as Code (IaC) Scanning: version 1 +- License Scanning: version 3 +- Secret Detection: version 3 +- Static Application Security Testing (SAST): version 2 of [all analyzers](https://docs.gitlab.com/ee/user/application_security/sast/#supported-languages-and-frameworks), except `gosec` which is currently at version 3 + - `bandit`: version 2 + - `brakeman`: version 2 + - `eslint`: version 2 + - `flawfinder`: version 2 + - `gosec`: version 3 + - `kubesec`: version 2 + - `mobsf`: version 2 + - `nodejs-scan`: version 2 + - `phpcs-security-audit`: version 2 + - `pmd-apex`: version 2 + - `security-code-scan`: version 2 + - `semgrep`: version 2 + - `sobelow`: version 2 + - `spotbugs`: version 2 **Planned removal milestone: 15.0 (2022-05-22)** -### Legacy approval status names from License Compliance API +### Support for gRPC-aware proxy deployed between Gitaly and rest of GitLab WARNING: This feature will be changed or removed in 15.0 @@ -413,13 +720,21 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -We deprecated legacy names for approval status of license policy (blacklisted, approved) in the `managed_licenses` API but they are still used in our API queries and responses. They will be removed in 15.0. +Although not recommended or documented, it was possible to deploy a gRPC-aware proxy between Gitaly and +the rest of GitLab. For example, NGINX and Envoy. The ability to deploy a gRPC-aware proxy is +[deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/352517). If you currently use a gRPC-aware proxy for +Gitaly connections, you should change your proxy configuration to use TCP or TLS proxying (OSI layer 4) instead. -If you are using our License Compliance API you should stop using the `approved` and `blacklisted` query parameters, they are now `allowed` and `denied`. In 15.0 the responses will also stop using `approved` and `blacklisted` so you need to adjust any of your custom tools to use the old and new values so they do not break with the 15.0 release. +Gitaly Cluster became incompatible with gRPC-aware proxies in GitLab 13.12. Now all GitLab installations will be incompatible with +gRPC-aware proxies, even without Gitaly Cluster. + +By sending some of our internal RPC traffic through a custom protocol (instead of gRPC) we +increase throughput and reduce Go garbage collection latency. For more information, see +the [relevant epic](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/463). **Planned removal milestone: 15.0 (2022-05-22)** -### Runner status `not_connected` API value +### Test coverage project CI/CD setting WARNING: This feature will be changed or removed in 15.0 @@ -427,14 +742,16 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -The GitLab Runner REST and GraphQL [API](https://docs.gitlab.com/ee/api/runners.html#runners-api) endpoints -will return `never_contacted` instead of `not_connected` as the status values in 15.0. +To simplify setting a test coverage pattern, in GitLab 15.0 the +[project setting for test coverage parsing](https://docs.gitlab.com/ee/ci/pipelines/settings.html#add-test-coverage-results-using-project-settings-deprecated) +is being removed. -Runners that have never contacted the GitLab instance will also return `stale` if created more than 3 months ago. +Instead, using the project’s `.gitlab-ci.yml`, provide a regular expression with the `coverage` keyword to set +testing coverage results in merge requests. **Planned removal milestone: 15.0 (2022-05-22)** -### `pipelines` fields in the Package GraphQL types +### Vulnerability Check WARNING: This feature will be changed or removed in 15.0 @@ -442,13 +759,18 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -As part of the work to create a [Package Registry GraphQL API](https://gitlab.com/groups/gitlab-org/-/epics/6318), the Package group deprecated the `pipelines` fields in all Package-related GraphQL types. As of GitLab 14.6, the `pipelines` field is deprecated in [`Package`](https://docs.gitlab.com/ee/api/graphql/reference/index.html#package) and [`PackageDetailsType`](https://docs.gitlab.com/ee/api/graphql/reference/index.html#packagedetailstype) due to scalability and performance concerns. +The vulnerability check feature is deprecated in GitLab 14.8 and scheduled for removal in GitLab 15.0. We encourage you to migrate to the new security approvals feature instead. You can do so by navigating to **Security & Compliance > Policies** and creating a new Scan Result Policy. -In milestone 15.0, we will completely remove `pipelines` from `Package` and `PackageDetailsType`. You can follow and contribute to work on a replacement in the epic [GitLab-#7214](https://gitlab.com/groups/gitlab-org/-/epics/7214). +The new security approvals feature is similar to vulnerability check. For example, both can require approvals for MRs that contain security vulnerabilities. However, security approvals improve the previous experience in several ways: + +- Users can choose who is allowed to edit security approval rules. An independent security or compliance team can therefore manage rules in a way that prevents development project maintainers from modifying the rules. +- Multiple rules can be created and chained together to allow for filtering on different severity thresholds for each scanner type. +- A two-step approval process can be enforced for any desired changes to security approval rules. +- A single set of security policies can be applied to multiple development projects to allow for ease in maintaining a single, centralized ruleset. **Planned removal milestone: 15.0 (2022-05-22)** -### `type` and `types` keyword in CI/CD configuration +### `CI_BUILD_*` predefined variables WARNING: This feature will be changed or removed in 15.0 @@ -456,11 +778,23 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -The `type` and `types` CI/CD keywords will be removed in GitLab 15.0. Pipelines that use these keywords will stop working, so you must switch to `stage` and `stages`, which have the same behavior. +The predefined CI/CD variables that start with `CI_BUILD_*` were deprecated in GitLab 9.0, and will be removed in GitLab 15.0. If you still use these variables, be sure to change to the current [`CI_JOB_*` predefined variables](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html) which are identical (except for the updated name). **Planned removal milestone: 15.0 (2022-05-22)** -### apiFuzzingCiConfigurationCreate GraphQL mutation +### `fixup!` commit messages setting draft status of associated Merge Request + +The use of `fixup!` as a commit message to trigger draft status +of the associated Merge Request is generally unused, and can cause +confusion with other uses of the term. "Draft" is the preferred +and supported trigger for triggering draft status from commit +messages, as part of our streamlining of the feature. +Support for `fixup!` is now considered deprecated, and will be +removed in GitLab 15.0. + +**Planned removal milestone: 15.0 (2022-06-22)** + +### `projectFingerprint` in `PipelineSecurityReportFinding` GraphQL WARNING: This feature will be changed or removed in 15.0 @@ -468,13 +802,14 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -The API Fuzzing configuration snippet is now being generated client-side and does not require an -API request anymore. We are therefore deprecating the `apiFuzzingCiConfigurationCreate` mutation -which isn't being used in GitLab anymore. +The `projectFingerprint` field in the [PipelineSecurityReportFinding](https://docs.gitlab.com/ee/api/graphql/reference/index.html#pipelinesecurityreportfinding) +GraphQL object is being deprecated. This field contains a "fingerprint" of security findings used to determine uniqueness. +The method for calculating fingerprints has changed, resulting in different values. Going forward, the new values will be +exposed in the UUID field. Data previously available in the projectFingerprint field will eventually be removed entirely. **Planned removal milestone: 15.0 (2022-05-22)** -### bundler-audit Dependency Scanning tool +### `started` iterations API field WARNING: This feature will be changed or removed in 15.0 @@ -482,9 +817,7 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -As of 14.6 bundler-audit is being deprecated from Dependency Scanning. It will continue to be in our CI/CD template while deprecated. We are removing bundler-audit from Dependency Scanning on May 22, 2022 in 15.0. After this removal Ruby scanning functionality will not be affected as it is still being covered by Gemnasium. - -If you have explicitly excluded bundler-audit using DS_EXCLUDED_ANALYZERS you will need to clean up (remove the reference) in 15.0. If you have customized your pipeline's Dependency Scanning configuration, for example to edit the `bundler-audit-dependency_scanning` job, you will want to switch to gemnasium-dependency_scanning before removal in 15.0, to prevent your pipeline from failing. If you have not used the DS_EXCLUDED_ANALYZERS to reference bundler-audit, or customized your template specifically for bundler-audit, you will not need to take action. +The `started` field in the [iterations API](https://docs.gitlab.com/ee/api/iterations.html#list-project-iterations) is being deprecated and will be removed in GitLab 15.0. This field is being replaced with the `current` field (already available) which aligns with the naming for other time-based entities, such as milestones. **Planned removal milestone: 15.0 (2022-05-22)** @@ -725,9 +1058,9 @@ The `merged_by` field in the [merge request API](https://docs.gitlab.com/ee/api/ **Planned removal milestone: 15.0 (2022-05-22)** -## 14.8 +## 14.6 -### Changes to the `CI_JOB_JWT` +### API: `stale` status returned instead of `offline` or `not_connected` WARNING: This feature will be changed or removed in 15.0 @@ -735,20 +1068,13 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -The `CI_JOB_JWT` will be updated to support a wider variety of cloud providers. It will be changed to match [`CI_JOB_JWT_V2`](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html), but this change may not be backwards compatible for all users, including Hashicorp Vault users. To maintain the current behavior, users can switch to using `CI_JOB_JWT_V1`, or update their configuration in GitLab 15.0 to use the improved `CI_JOB_JWT`. - -**Planned removal milestone: 15.0 ()** - -### Configurable Gitaly `per_repository` election strategy - -Configuring the `per_repository` Gitaly election strategy is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/352612). -`per_repository` has been the only option since GitLab 14.0. +A breaking change will occur for the Runner [API](https://docs.gitlab.com/ee/api/runners.html#runners-api) endpoints in 15.0. -This change is part of regular maintenance to keep our codebase clean. +Instead of the GitLab Runner API endpoints returning `offline` and `not_connected` for runners that have not contacted the GitLab instance in the past three months, the API endpoints will return the `stale` value, which was introduced in 14.6. -**Planned removal milestone: 14.9 (2022-03-22)** +**Planned removal milestone: 15.0 (2022-05-22)** -### Container Network and Host Security +### CI/CD job name length limit WARNING: This feature will be changed or removed in 15.0 @@ -756,20 +1082,11 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -All functionality related to GitLab's Container Network Security and Container Host Security categories is deprecated in GitLab 14.8 and scheduled for removal in GitLab 15.0. Users who need a replacement for this functionality are encouraged to evaluate the following open source projects as potential solutions that can be installed and managed outside of GitLab: [AppArmor](https://gitlab.com/apparmor/apparmor), [Cilium](https://github.com/cilium/cilium), [Falco](https://github.com/falcosecurity/falco), [FluentD](https://github.com/fluent/fluentd), [Pod Security Admission](https://kubernetes.io/docs/concepts/security/pod-security-admission/). To integrate these technologies into GitLab, add the desired Helm charts into your copy of the [Cluster Management Project Template](https://docs.gitlab.com/ee/user/clusters/management_project_template.html). Deploy these Helm charts in production by calling commands through the GitLab [Secure CI/CD Tunnel](https://docs.gitlab.com/ee/user/clusters/agent/repository.html#run-kubectl-commands-using-the-cicd-tunnel). - -As part of this change, the following specific capabilities within GitLab are now deprecated, and are scheduled for removal in GitLab 15.0: - -- The **Security & Compliance > Threat Monitoring** page. -- The `Network Policy` security policy type, as found on the **Security & Compliance > Policies** page. -- The ability to manage integrations with the following technologies through GitLab: AppArmor, Cilium, Falco, FluentD, and Pod Security Policies. -- All APIs related to the above functionality. - -For additional context, or to provide feedback regarding this change, please reference our open [deprecation issue](https://gitlab.com/groups/gitlab-org/-/epics/7476). +In GitLab 15.0 we are going to limit the number of characters in CI/CD job names to 255. Any pipeline with job names that exceed the 255 character limit will stop working after the 15.0 release. **Planned removal milestone: 15.0 (2022-05-22)** -### Dependency Scanning Python 3.9 and 3.6 image deprecation +### Legacy approval status names from License Compliance API WARNING: This feature will be changed or removed in 15.0 @@ -777,60 +1094,13 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -For those using Dependency Scanning for Python projects, we are deprecating the default `gemnasium-python:2` image which uses Python 3.6 as well as the custom `gemnasium-python:2-python-3.9` image which uses Python 3.9. The new default image as of GitLab 15.0 will be for Python 3.9 as it is a [supported version](https://endoflife.date/python) and 3.6 [is no longer supported](https://endoflife.date/python). - -For users using Python 3.9 or 3.9-compatible projects, you should not need to take action and dependency scanning should begin to work in GitLab 15.0. If you wish to test the new container now please run a test pipeline in your project with this container (which will be removed in 15.0). Use the Python 3.9 image: - -```yaml -gemnasium-python-dependency_scanning: - image: - name: registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python:2-python-3.9 -``` - -For users using Python 3.6, as of GitLab 15.0 you will no longer be able to use the default template for dependency scanning. You will need to switch to use the deprecated `gemnasium-python:2` analyzer image. If you are impacted by this please comment in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/351503) so we can extend the removal if needed. - -For users using the 3.9 special exception image, you must instead use the default value and no longer override your container. To verify if you are using the 3.9 special exception image, check your `.gitlab-ci.yml` file for the following reference: - -```yaml -gemnasium-python-dependency_scanning: - image: - name: registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python:2-python-3.9 -``` - -**Planned removal milestone: 15.0 (2021-05-22)** - -### Deprecate Geo Admin UI Routes - -In GitLab 13.0, we introduced new project and design replication details routes in the Geo Admin UI. These routes are `/admin/geo/replication/projects` and `/admin/geo/replication/designs`. We kept the legacy routes and redirected them to the new routes. In GitLab 15.0, we will remove support for the legacy routes `/admin/geo/projects` and `/admin/geo/designs`. Please update any bookmarks or scripts that may use the legacy routes. - -**Planned removal milestone: 15.0 (2022-05-22)** - -### Deprecate custom Geo:db:* Rake tasks - -In GitLab 14.8, we are [replacing the `geo:db:*` Rake tasks with built-in tasks](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/77269/diffs) that are now possible after [switching the Geo tracking database to use Rails' 6 support of multiple databases](https://gitlab.com/groups/gitlab-org/-/epics/6458). -The following `geo:db:*` tasks will be replaced with their corresponding `db:*:geo` tasks: +We deprecated legacy names for approval status of license policy (blacklisted, approved) in the `managed_licenses` API but they are still used in our API queries and responses. They will be removed in 15.0. -- `geo:db:drop` -> `db:drop:geo` -- `geo:db:create` -> `db:create:geo` -- `geo:db:setup` -> `db:setup:geo` -- `geo:db:migrate` -> `db:migrate:geo` -- `geo:db:rollback` -> `db:rollback:geo` -- `geo:db:version` -> `db:version:geo` -- `geo:db:reset` -> `db:reset:geo` -- `geo:db:seed` -> `db:seed:geo` -- `geo:schema:load:geo` -> `db:schema:load:geo` -- `geo:db:schema:dump` -> `db:schema:dump:geo` -- `geo:db:migrate:up` -> `db:migrate:up:geo` -- `geo:db:migrate:down` -> `db:migrate:down:geo` -- `geo:db:migrate:redo` -> `db:migrate:redo:geo` -- `geo:db:migrate:status` -> `db:migrate:status:geo` -- `geo:db:test:prepare` -> `db:test:prepare:geo` -- `geo:db:test:load` -> `db:test:load:geo` -- `geo:db:test:purge` -> `db:test:purge:geo` +If you are using our License Compliance API you should stop using the `approved` and `blacklisted` query parameters, they are now `allowed` and `denied`. In 15.0 the responses will also stop using `approved` and `blacklisted` so you need to adjust any of your custom tools to use the old and new values so they do not break with the 15.0 release. **Planned removal milestone: 15.0 (2022-05-22)** -### Deprecate feature flag PUSH_RULES_SUPERSEDE_CODE_OWNERS +### Runner status `not_connected` API value WARNING: This feature will be changed or removed in 15.0 @@ -838,11 +1108,14 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -The feature flag `PUSH_RULES_SUPERSEDE_CODE_OWNERS` is being removed in GitLab 15.0. Upon its removal, push rules will supersede CODEOWNERS. The CODEOWNERS feature will no longer be available for access control. +The GitLab Runner REST and GraphQL [API](https://docs.gitlab.com/ee/api/runners.html#runners-api) endpoints +will return `never_contacted` instead of `not_connected` as the status values in 15.0. + +Runners that have never contacted the GitLab instance will also return `stale` if created more than 3 months ago. **Planned removal milestone: 15.0 (2022-05-22)** -### Deprecate legacy Gitaly configuration methods +### `pipelines` fields in the Package GraphQL types WARNING: This feature will be changed or removed in 15.0 @@ -850,15 +1123,13 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -Using environment variables `GIT_CONFIG_SYSTEM` and `GIT_CONFIG_GLOBAL` to configure Gitaly is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/352609). -These variables are being replaced with standard [`config.toml` Gitaly configuration](https://docs.gitlab.com/ee/administration/gitaly/reference.html). +As part of the work to create a [Package Registry GraphQL API](https://gitlab.com/groups/gitlab-org/-/epics/6318), the Package group deprecated the `pipelines` fields in all Package-related GraphQL types. As of GitLab 14.6, the `pipelines` field is deprecated in [`Package`](https://docs.gitlab.com/ee/api/graphql/reference/index.html#package) and [`PackageDetailsType`](https://docs.gitlab.com/ee/api/graphql/reference/index.html#packagedetailstype) due to scalability and performance concerns. -GitLab instances that use `GIT_CONFIG_SYSTEM` and `GIT_CONFIG_GLOBAL` to configure Gitaly should switch to configuring using -`config.toml`. +In milestone 15.0, we will completely remove `pipelines` from `Package` and `PackageDetailsType`. You can follow and contribute to work on a replacement in the epic [GitLab-#7214](https://gitlab.com/groups/gitlab-org/-/epics/7214). **Planned removal milestone: 15.0 (2022-05-22)** -### Elasticsearch 6.8 +### `type` and `types` keyword in CI/CD configuration WARNING: This feature will be changed or removed in 15.0 @@ -866,15 +1137,11 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -Elasticsearch 6.8 is deprecated in GitLab 14.8 and scheduled for removal in GitLab 15.0. -Customers using Elasticsearch 6.8 need to upgrade their Elasticsearch version to 7.x prior to upgrading to GitLab 15.0. -We recommend using the latest version of Elasticsearch 7 to benefit from all Elasticsearch improvements. - -Elasticsearch 6.8 is also incompatible with Amazon OpenSearch, which we [plan to support in GitLab 15.0](https://gitlab.com/gitlab-org/gitlab/-/issues/327560). +The `type` and `types` CI/CD keywords will be removed in GitLab 15.0. Pipelines that use these keywords will stop working, so you must switch to `stage` and `stages`, which have the same behavior. **Planned removal milestone: 15.0 (2022-05-22)** -### External status check API breaking changes +### apiFuzzingCiConfigurationCreate GraphQL mutation WARNING: This feature will be changed or removed in 15.0 @@ -882,25 +1149,13 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -The [external status check API](https://docs.gitlab.com/ee/api/status_checks.html) was originally implemented to -support pass-by-default requests to mark a status check as passing. Pass-by-default requests are now deprecated. -Specifically, the following are deprecated: - -- Requests that do not contain the `status` field. -- Requests that have the `status` field set to `approved`. - -Beginning in GitLab 15.0, status checks will only be updated to a passing state if the `status` field is both present -and set to `pass`. Requests that: - -- Do not contain the `status` field will be rejected with a `422` error. For more information, see [the relevant issue](https://gitlab.com/gitlab-org/gitlab/-/issues/338827). -- Contain any value other than `pass` will cause the status check to fail. For more information, see [the relevant issue](https://gitlab.com/gitlab-org/gitlab/-/issues/339039). - -To align with this change, API calls to list external status checks will also return the value of `pass` rather than -`approved` for status checks that have passed. +The API Fuzzing configuration snippet is now being generated client-side and does not require an +API request anymore. We are therefore deprecating the `apiFuzzingCiConfigurationCreate` mutation +which isn't being used in GitLab anymore. **Planned removal milestone: 15.0 (2022-05-22)** -### GraphQL ID and GlobalID compatibility +### bundler-audit Dependency Scanning tool WARNING: This feature will be changed or removed in 15.0 @@ -908,81 +1163,42 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -We are removing a non-standard extension to our GraphQL processor, which we added for backwards compatibility. This extension modifies the validation of GraphQL queries, allowing the use of the `ID` type for arguments where it would normally be rejected. -Some arguments originally had the type `ID`. These were changed to specific -kinds of `ID`. This change may be a breaking change if you: - -- Use GraphQL. -- Use the `ID` type for any argument in your query signatures. - -Some field arguments still have the `ID` type. These are typically for -IID values, or namespace paths. An example is `Query.project(fullPath: ID!)`. - -For a list of affected and unaffected field arguments, -see the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352832). - -You can test if this change affects you by validating -your queries locally, using schema data fetched from a GitLab server. -You can do this by using the GraphQL explorer tool for the relevant GitLab -instance. For example: `https://gitlab.com/-/graphql-explorer`. - -For example, the following query illustrates the breaking change: - -```graphql -# a query using the deprecated type of Query.issue(id:) -# WARNING: This will not work after GitLab 15.0 -query($id: ID!) { - deprecated: issue(id: $id) { - title, description - } -} -``` - -The query above will not work after GitLab 15.0 is released, because the type -of `Query.issue(id:)` is actually `IssueID!`. - -Instead, you should use one of the following two forms: +As of 14.6 bundler-audit is being deprecated from Dependency Scanning. It will continue to be in our CI/CD template while deprecated. We are removing bundler-audit from Dependency Scanning on May 22, 2022 in 15.0. After this removal Ruby scanning functionality will not be affected as it is still being covered by Gemnasium. -```graphql -# This will continue to work -query($id: IssueID!) { - a: issue(id: $id) { - title, description - } - b: issue(id: "gid://gitlab/Issue/12345") { - title, description - } -} -``` +If you have explicitly excluded bundler-audit using DS_EXCLUDED_ANALYZERS you will need to clean up (remove the reference) in 15.0. If you have customized your pipeline's Dependency Scanning configuration, for example to edit the `bundler-audit-dependency_scanning` job, you will want to switch to gemnasium-dependency_scanning before removal in 15.0, to prevent your pipeline from failing. If you have not used the DS_EXCLUDED_ANALYZERS to reference bundler-audit, or customized your template specifically for bundler-audit, you will not need to take action. -This query works now, and will continue to work after GitLab 15.0. -You should convert any queries in the first form (using `ID` as a named type in the signature) -to one of the other two forms (using the correct appropriate type in the signature, or using -an inline argument expression). +**Planned removal milestone: 15.0 (2022-05-22)** -**Planned removal milestone: 15.0 (2022-04-22)** +## 14.5 -### OAuth tokens without expiration +### Certificate-based integration with Kubernetes WARNING: -This feature will be changed or removed in 15.0 +This feature will be changed or removed in 15.6 as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#breaking-changes). Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -By default, all new applications expire access tokens after 2 hours. In GitLab 14.2 and earlier, OAuth access tokens -had no expiration. In GitLab 15.0, an expiry will be automatically generated for any existing token that does not -already have one. +[The certificate-based integration with Kubernetes will be deprecated and removed](https://about.gitlab.com/blog/2021/11/15/deprecating-the-cert-based-kubernetes-integration/). -You should [opt in](https://docs.gitlab.com/ee/integration/oauth_provider.html#expiring-access-tokens) to expiring -tokens before GitLab 15.0 is released: +If you are a self-managed customer, in GitLab 15.0, a feature flag will be introduced so you can keep +certificate-based integration enabled. The flag will be disabled by default. +The flag and the related code will be removed in GitLab 15.6. -1. Edit the application. -1. Select **Expire access tokens** to enable them. Tokens must be revoked or they don’t expire. +Until the final removal in 15.6, features built on the integration will continue to work, and +GitLab will continue to fix security and critical issues. -**Planned removal milestone: 15.0 (2022-05-22)** +If you use GitLab.com, certificate-based integrations will cease functioning in 15.0. -### Optional enforcement of PAT expiration +For a more robust, secure, forthcoming, and reliable integration with Kubernetes, we recommend you use the +[agent for Kubernetes](https://docs.gitlab.com/ee/user/clusters/agent/) to connect Kubernetes clusters with GitLab. +See the documentation for [how to migrate](https://docs.gitlab.com/ee/user/infrastructure/clusters/migrate_to_gitlab_agent.html). + +For updates and details about this deprecation, follow [this epic](https://gitlab.com/groups/gitlab-org/configure/-/epics/8). + +**Planned removal milestone: 15.6 (2022-11-22)** + +### Converting an instance (shared) runner to a project (specific) runner WARNING: This feature will be changed or removed in 15.0 @@ -990,13 +1206,11 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -The feature to disable enforcement of PAT expiration is unusual from a security perspective. -We have become concerned that this unusual feature could create unexpected behavior for users. -Unexpected behavior in a security feature is inherently dangerous, so we have decided to remove this feature. +In GitLab 15.0, we will remove the feature that enables you to convert an instance (shared) runner to a project (specific) runner. Users who need to add a runner to only a particular project can register a runner to the project directly. **Planned removal milestone: 15.0 (2022-05-22)** -### Optional enforcement of SSH expiration +### Known host required for GitLab Runner SSH executor WARNING: This feature will be changed or removed in 15.0 @@ -1004,13 +1218,13 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -The feature to disable enforcement of SSH expiration is unusual from a security perspective. -We have become concerned that this unusual feature could create unexpected behavior for users. -Unexpected behavior in a security feature is inherently dangerous, so we have decided to remove this feature. +In [GitLab 14.3](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/3074), we added a configuration setting in the GitLab Runner `config.toml` file. This setting, [`[runners.ssh.disable_strict_host_key_checking]`](https://docs.gitlab.com/runner/executors/ssh.html#security), controls whether or not to use strict host key checking with the SSH executor. + +In GitLab 15.0 and later, the default value for this configuration option will change from `true` to `false`. This means that strict host key checking will be enforced when using the GitLab Runner SSH executor. **Planned removal milestone: 15.0 (2022-05-22)** -### Out-of-the-box SAST support for Java 8 +### Must explicitly assign `AuthenticationType` for `[runners.cache.s3]` WARNING: This feature will be changed or removed in 15.0 @@ -1018,22 +1232,13 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -The [GitLab SAST SpotBugs analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) scans [Java, Scala, Groovy, and Kotlin code](https://docs.gitlab.com/ee/user/application_security/sast/#supported-languages-and-frameworks) for security vulnerabilities. -For technical reasons, the analyzer must first compile the code before scanning. -Unless you use the [pre-compilation strategy](https://docs.gitlab.com/ee/user/application_security/sast/#pre-compilation), the analyzer attempts to automatically compile your project's code. - -In GitLab versions prior to 15.0, the analyzer image includes Java 8 and Java 11 runtimes to facilitate compilation. - -In GitLab 15.0, we will: - -- Remove Java 8 from the analyzer image to reduce the size of the image. -- Add Java 17 to the analyzer image to make it easier to compile with Java 17. +In GitLab 15.0 and later, to access the AWS S3 cache, you must specify the `AuthenticationType` for [`[runners.cache.s3]`](https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runnerscaches3-section). The `AuthenticationType` must be `IAM` or `credentials`. -If you rely on Java 8 being present in the analyzer environment, you must take action as detailed in the [deprecation issue for this change](https://gitlab.com/gitlab-org/gitlab/-/issues/352549#breaking-change). +Prior to 14.5, if you did not define the `AuthenticationType`, GitLab Runner chose a type for you. **Planned removal milestone: 15.0 (2022-05-22)** -### Querying Usage Trends via the `instanceStatisticsMeasurements` GraphQL node +### Package pipelines in API payload is paginated WARNING: This feature will be changed or removed in 15.0 @@ -1041,71 +1246,57 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -The `instanceStatisticsMeasurements` GraphQL node has been renamed to `usageTrendsMeasurements` in 13.10 and the old field name has been marked as deprecated. To fix the existing GraphQL queries, replace `instanceStatisticsMeasurements` with `usageTrendsMeasurements`. +A request to the API for `/api/v4/projects/:id/packages` returns a paginated result of packages. Each package lists all of its pipelines in this response. This is a performance concern, as it's possible for a package to have hundreds or thousands of associated pipelines. + +In milestone 15.0, we will remove the `pipelines` attribute from the API response. **Planned removal milestone: 15.0 (2022-05-22)** -### REST API Runner will not accept `status` filter values of `active` or `paused` +### REST and GraphQL API Runner status will not return `paused` WARNING: -This feature will be changed or removed in 16.0 +This feature will be changed or removed in 15.0 as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#breaking-changes). Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -The GitLab Runner REST endpoints will stop accepting `paused` or `active` as a status value in GitLab 16.0. +The GitLab Runner REST and GraphQL API endpoints will not return `paused` or `active` as a status in GitLab 15.0. -A runner's status will only relate to runner contact status, such as: `online`, `offline`. -Status values `paused` or `active` will no longer be accepted and will be replaced by the `paused` query parameter. +A runner's status will only relate to runner contact status, such as: +`online`, `offline`, or `not_connected`. Status `paused` or `active` will no longer appear. -When checking for paused runners, API users are advised to specify `paused=true` as the query parameter. -When checking for active runners, specify `paused=false`. +When checking if a runner is `paused`, API users are advised to check the boolean attribute +`paused` to be `true` instead. When checking if a runner is `active`, check if `paused` is `false`. -**Planned removal milestone: 16.0 (2023-04-22)** +**Planned removal milestone: 15.0 (2022-05-22)** -### REST API endpoint to list group runners no longer accepts `project_type` value for `type` argument +### Support for SLES 12 SP2 WARNING: -This feature will be changed or removed in 16.0 +This feature will be changed or removed in 15.0 as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#breaking-changes). Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -The `GET /groups/:id/runners?type=project_type` endpoint will be removed in GitLab 16.0. The endpoint always returned an empty collection. +Long term service and support (LTSS) for SUSE Linux Enterprise Server (SLES) 12 SP2 [ended on March 31, 2021](https://www.suse.com/lifecycle/). The CA certificates on SP2 include the expired DST root certificate, and it's not getting new CA certificate package updates. We have implemented some [workarounds](https://gitlab.com/gitlab-org/gitlab-omnibus-builder/-/merge_requests/191), but we will not be able to continue to keep the build running properly. -**Planned removal milestone: 16.0 (2023-04-22)** +**Planned removal milestone: 15.0 (2022-05-22)** -### REST and GraphQL API Runner usage of `active` replaced by `paused` +### Update to the Container Registry group-level API WARNING: -This feature will be changed or removed in 16.0 +This feature will be changed or removed in 15.0 as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#breaking-changes). Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -Occurrences of the `active` identifier in the GitLab Runner REST and GraphQL API endpoints will be -renamed to `paused` in GitLab 16.0, namely: - -- GraphQL API: - - the `CiRunner` property; - - the `RunnerUpdateInput` input type for the `runnerUpdate` mutation; - - the `runners` and `Group.runners` queries. -- REST API: - - endpoints taking or returning `active` properties, such as: - - `GET /runners` - - `GET /runners/all` - - `GET /runners/:id` / `PUT /runners/:id` - - `PUT --form "active=false" /runners/:runner_id` - - `GET /projects/:id/runners` / `POST /projects/:id/runners` - - `GET /groups/:id/runners` +In milestone 15.0, support for the `tags` and `tags_count` parameters will be removed from the Container Registry API that [gets registry repositories from a group](../api/container_registry.md#within-a-group). -The 16.0 release of the GitLab Runner will start using the `paused` property when registering runners, and therefore -will only be compatible with GitLab 16.0 and later. Until 16.0, GitLab will accept the deprecated `active` flag from -existing runners. +The `GET /groups/:id/registry/repositories` endpoint will remain, but won't return any info about tags. To get the info about tags, you can use the existing `GET /registry/repositories/:id` endpoint, which will continue to support the `tags` and `tag_count` options as it does today. The latter must be called once per image repository. -**Planned removal milestone: 16.0 (2023-04-22)** +**Planned removal milestone: 15.0 (2022-05-22)** -### Reminder: support for NFS repository storage +### Value Stream Analytics filtering calculation change WARNING: This feature will be changed or removed in 15.0 @@ -1113,22 +1304,13 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -As [announced](https://about.gitlab.com/releases/2021/06/22/gitlab-14-0-released/#nfs-for-git-repository-storage-deprecated) at the -release of GitLab 14.0, technical support for NFS storage for Git repositories is being removed. Please see our official -[Statement of Support](https://about.gitlab.com/support/statement-of-support.html#gitaly-and-nfs) for additional information. - -We encourage customers currently using NFS for Git repositories to plan their migration by reviewing our documentation on -[migrating to Gitaly Cluster](https://docs.gitlab.com/ee/administration/gitaly/#migrating-to-gitaly-cluster). - -Gitaly Cluster offers tremendous benefits for our customers such as: +We are changing how the date filter works in Value Stream Analytics. Instead of filtering by the time that the issue or merge request was created, the date filter will filter by the end event time of the given stage. This will result in completely different figures after this change has rolled out. -- [Variable replication factors](https://docs.gitlab.com/ee/administration/gitaly/praefect.html#configure-replication-factor) -- [Strong consistency](https://docs.gitlab.com/ee/administration/gitaly/#strong-consistency) -- [Distributed read capabilities](https://docs.gitlab.com/ee/administration/gitaly/#distributed-reads) +If you monitor Value Stream Analytics metrics and rely on the date filter, to avoid losing data, you must save the data prior to this change. **Planned removal milestone: 15.0 (2022-05-22)** -### Request profiling +### `Versions` on base `PackageType` WARNING: This feature will be changed or removed in 15.0 @@ -1136,17 +1318,13 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -[Request profiling](https://docs.gitlab.com/ee/administration/monitoring/performance/request_profiling.html) is deprecated in GitLab 14.8 and scheduled for removal in GitLab 15.0. - -We're working on [consolidating our profiling tools](https://gitlab.com/groups/gitlab-org/-/epics/7327) and making them more easily accessible. -We [evaluated](https://gitlab.com/gitlab-org/gitlab/-/issues/350152) the use of this feature and we found that it is not widely used. -It also depends on a few third-party gems that are not actively maintained anymore, have not been updated for the latest version of Ruby, or crash frequently when profiling heavy page loads. +As part of the work to create a [Package Registry GraphQL API](https://gitlab.com/groups/gitlab-org/-/epics/6318), the Package group deprecated the `Version` type for the basic `PackageType` type and moved it to [`PackageDetailsType`](https://docs.gitlab.com/ee/api/graphql/reference/index.html#packagedetailstype). -For more information, check the [summary section of the deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352488#deprecation-summary). +In milestone 15.0, we will completely remove `Version` from `PackageType`. **Planned removal milestone: 15.0 (2022-05-22)** -### Required pipeline configurations in Premium tier +### `defaultMergeCommitMessageWithDescription` GraphQL API field WARNING: This feature will be changed or removed in 15.0 @@ -1154,16 +1332,11 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -The [required pipeline configuration](https://docs.gitlab.com/ee/user/admin_area/settings/continuous_integration.html#required-pipeline-configuration) feature is deprecated in GitLab 14.8 for Premium customers and is scheduled for removal in GitLab 15.0. This feature is not deprecated for GitLab Ultimate customers. - -This change to move the feature to GitLab's Ultimate tier is intended to help our features better align with our [pricing philosophy](https://about.gitlab.com/company/pricing/#three-tiers) as we see demand for this feature originating primarily from executives. - -This change will also help GitLab remain consistent in its tiering strategy with the other related Ultimate-tier features of: -[Security policies](https://docs.gitlab.com/ee/user/application_security/policies/) and [compliance framework pipelines](https://docs.gitlab.com/ee/user/project/settings/index.html#compliance-pipeline-configuration). +The GraphQL API field `defaultMergeCommitMessageWithDescription` has been deprecated and will be removed in GitLab 15.0. For projects with a commit message template set, it will ignore the template. **Planned removal milestone: 15.0 (2022-05-22)** -### Retire-JS Dependency Scanning tool +### `dependency_proxy_for_private_groups` feature flag WARNING: This feature will be changed or removed in 15.0 @@ -1171,13 +1344,13 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -As of 14.8 the retire.js job is being deprecated from Dependency Scanning. It will continue to be included in our CI/CD template while deprecated. We are removing retire.js from Dependency Scanning on May 22, 2022 in GitLab 15.0. JavaScript scanning functionality will not be affected as it is still being covered by Gemnasium. +We added a feature flag because [GitLab-#11582](https://gitlab.com/gitlab-org/gitlab/-/issues/11582) changed how public groups use the Dependency Proxy. Prior to this change, you could use the Dependency Proxy without authentication. The change requires authentication to use the Dependency Proxy. -If you have explicitly excluded retire.js using DS_EXCLUDED_ANALYZERS you will need to clean up (remove the reference) in 15.0. If you have customized your pipeline's Dependency Scanning configuration related to the `retire-js-dependency_scanning` job you will want to switch to gemnasium-dependency_scanning before the removal in 15.0, to prevent your pipeline from failing. If you have not used the DS_EXCLUDED_ANALYZERS to reference retire.js, or customized your template specifically for retire.js, you will not need to take action. +In milestone 15.0, we will remove the feature flag entirely. Moving forward, you must authenticate when using the Dependency Proxy. **Planned removal milestone: 15.0 (2022-05-22)** -### SAST analyzer consolidation and CI/CD template changes +### `pipelines` field from the `version` field WARNING: This feature will be changed or removed in 15.0 @@ -1185,30 +1358,16 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -GitLab SAST uses various [analyzers](https://docs.gitlab.com/ee/user/application_security/sast/analyzers/) to scan code for vulnerabilities. - -We are reducing the number of analyzers used in GitLab SAST as part of our long-term strategy to deliver a better and more consistent user experience. -Streamlining the set of analyzers will also enable faster [iteration](https://about.gitlab.com/handbook/values/#iteration), better [results](https://about.gitlab.com/handbook/values/#results), and greater [efficiency](https://about.gitlab.com/handbook/values/#results) (including a reduction in CI runner usage in most cases). - -In GitLab 15.0, GitLab SAST will no longer use the following analyzers: - -- [ESLint](https://gitlab.com/gitlab-org/security-products/analyzers/eslint) (JavaScript, TypeScript, React) -- [Gosec](https://gitlab.com/gitlab-org/security-products/analyzers/gosec) (Go) -- [Bandit](https://gitlab.com/gitlab-org/security-products/analyzers/bandit) (Python) - -These analyzers will be removed from the [GitLab-managed SAST CI/CD template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml) and replaced with the [Semgrep-based analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep). -They will no longer receive routine updates, except for security issues. -We will not delete container images previously published for these analyzers; any such change would be announced as a [deprecation, removal, or breaking change announcement](https://about.gitlab.com/handbook/marketing/blog/release-posts/#deprecations-removals-and-breaking-changes). +In GraphQL, there are two `pipelines` fields that you can use in a [`PackageDetailsType`](https://docs.gitlab.com/ee/api/graphql/reference/#packagedetailstype) to get the pipelines for package versions: -We will also remove Java from the scope of the [SpotBugs](https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs) analyzer and replace it with the [Semgrep-based analyzer](https://gitlab.com/gitlab-org/security-products/analyzers/semgrep). -This change will make it simpler to scan Java code; compilation will no longer be required. -This change will be reflected in the automatic language detection portion of the [GitLab-managed SAST CI/CD template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST.gitlab-ci.yml). +- The `versions` field's `pipelines` field. This returns all the pipelines associated with all the package's versions, which can pull an unbounded number of objects in memory and create performance concerns. +- The `pipelines` field of a specific `version`. This returns only the pipelines associated with that single package version. -If you applied customizations to any of the affected analyzers, you must take action as detailed in the [deprecation issue for this change](https://gitlab.com/gitlab-org/gitlab/-/issues/352554#breaking-change). +To mitigate possible performance problems, we will remove the `versions` field's `pipelines` field in milestone 15.0. Although you will no longer be able to get all pipelines for all versions of a package, you can still get the pipelines of a single version through the remaining `pipelines` field for that version. **Planned removal milestone: 15.0 (2022-05-22)** -### SAST support for .NET 2.1 +### `promote-db` command from `gitlab-ctl` WARNING: This feature will be changed or removed in 15.0 @@ -1216,50 +1375,11 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -The GitLab SAST Security Code Scan analyzer scans .NET code for security vulnerabilities. -For technical reasons, the analyzer must first build the code to scan it. - -In GitLab versions prior to 15.0, the default analyzer image (version 2) includes support for: - -- .NET 2.1 -- .NET 3.0 and .NET Core 3.0 -- .NET Core 3.1 -- .NET 5.0 - -In GitLab 15.0, we will change the default major version for this analyzer from version 2 to version 3. This change: - -- Adds [severity values for vulnerabilities](https://gitlab.com/gitlab-org/gitlab/-/issues/350408) along with [other new features and improvements](https://gitlab.com/gitlab-org/security-products/analyzers/security-code-scan/-/blob/master/CHANGELOG.md). -- Removes .NET 2.1 support. -- Adds support for .NET 6.0, Visual Studio 2019, and Visual Studio 2022. - -Version 3 was [announced in GitLab 14.6](https://about.gitlab.com/releases/2021/12/22/gitlab-14-6-released/#sast-support-for-net-6) and made available as an optional upgrade. - -If you rely on .NET 2.1 support being present in the analyzer image by default, you must take action as detailed in the [deprecation issue for this change](https://gitlab.com/gitlab-org/gitlab/-/issues/352553#breaking-change). - -**Planned removal milestone: 15.0 (2022-05-22)** - -### Secret Detection configuration variables deprecated - -To make it simpler and more reliable to [customize GitLab Secret Detection](https://docs.gitlab.com/ee/user/application_security/secret_detection/#customizing-settings), we're deprecating some of the variables that you could previously set in your CI/CD configuration. - -The following variables currently allow you to customize the options for historical scanning, but interact poorly with the [GitLab-managed CI/CD template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/Secret-Detection.gitlab-ci.yml) and are now deprecated: - -- `SECRET_DETECTION_COMMIT_FROM` -- `SECRET_DETECTION_COMMIT_TO` -- `SECRET_DETECTION_COMMITS` -- `SECRET_DETECTION_COMMITS_FILE` - -The `SECRET_DETECTION_ENTROPY_LEVEL` previously allowed you to configure rules that only considered the entropy level of strings in your codebase, and is now deprecated. -This type of entropy-only rule created an unacceptable number of incorrect results (false positives) and is no longer supported. - -In GitLab 15.0, we'll update the Secret Detection [analyzer](https://docs.gitlab.com/ee/user/application_security/terminology/#analyzer) to ignore these deprecated options. -You'll still be able to configure historical scanning of your commit history by setting the [`SECRET_DETECTION_HISTORIC_SCAN` CI/CD variable](https://docs.gitlab.com/ee/user/application_security/secret_detection/#available-cicd-variables). - -For further details, see [the deprecation issue for this change](https://gitlab.com/gitlab-org/gitlab/-/issues/352565). +In GitLab 14.5, we introduced the command `gitlab-ctl promote` to promote any Geo secondary node to a primary during a failover. This command replaces `gitlab-ctl promote-db` which is used to promote database nodes in multi-node Geo secondary sites. `gitlab-ctl promote-db` will continue to function as-is and be available until GitLab 15.0. We recommend that Geo customers begin testing the new `gitlab-ctl promote` command in their staging environments and incorporating the new command in their failover procedures. **Planned removal milestone: 15.0 (2022-05-22)** -### Secure and Protect analyzer images published in new location +### `promote-to-primary-node` command from `gitlab-ctl` WARNING: This feature will be changed or removed in 15.0 @@ -1267,67 +1387,37 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -GitLab uses various [analyzers](https://docs.gitlab.com/ee/user/application_security/terminology/#analyzer) to [scan for security vulnerabilities](https://docs.gitlab.com/ee/user/application_security/). -Each analyzer is distributed as a container image. +In GitLab 14.5, we introduced the command `gitlab-ctl promote` to promote any Geo secondary node to a primary during a failover. This command replaces `gitlab-ctl promote-to-primary-node` which was only usable for single-node Geo sites. `gitlab-ctl promote-to-primary-node` will continue to function as-is and be available until GitLab 15.0. We recommend that Geo customers begin testing the new `gitlab-ctl promote` command in their staging environments and incorporating the new command in their failover procedures. -Starting in GitLab 14.8, new versions of GitLab Secure and Protect analyzers are published to a new registry location under `registry.gitlab.com/security-products`. +**Planned removal milestone: 15.0 (2022-05-22)** -We will update the default value of [GitLab-managed CI/CD templates](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates/Security) to reflect this change: - -- For all analyzers except Container Scanning, we will update the variable `SECURE_ANALYZERS_PREFIX` to the new image registry location. -- For Container Scanning, the default image address is already updated. There is no `SECURE_ANALYZERS_PREFIX` variable for Container Scanning. +### openSUSE Leap 15.2 packages -In a future release, we will stop publishing images to `registry.gitlab.com/gitlab-org/security-products/analyzers`. -Once this happens, you must take action if you manually pull images and push them into a separate registry. This is commonly the case for [offline deployments](https://docs.gitlab.com/ee/user/application_security/offline_deployments/index.html). -Otherwise, you won't receive further updates. +Distribution support and security updates for openSUSE Leap 15.2 are [ending December 2021](https://en.opensuse.org/Lifetime#openSUSE_Leap). -See the [deprecation issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352564) for more details. +Starting in 14.5 we are providing packages for openSUSE Leap 15.3, and will stop providing packages for openSUSE Leap 15.2 in the 14.8 milestone. -**Planned removal milestone: 15.0 (2022-05-22)** +**Planned removal milestone: 14.8 (2022-02-22)** -### Secure and Protect analyzer major version update +## 14.3 + +### Audit events for repository push events WARNING: -This feature will be changed or removed in 15.00 +This feature will be changed or removed in 15.0 as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#breaking-changes). Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -The Secure and Protect stages will be bumping the major versions of their analyzers in tandem with the GitLab 15.0 release. This major bump will enable a clear delineation for analyzers, between: - -- Those released prior to May 22, 2022, which generate reports that _are not_ subject to stringent schema validation. -- Those released after May 22, 2022, which generate reports that _are_ subject to stringent schema validation. - -If you are not using the default inclusion templates, or have pinned your analyzer version(s) you will need to update your CI/CD job definition to either remove the pinned version or to update the latest major version. -Users of GitLab 12.0-14.10 will continue to experience analyzer updates as normal until the release of GitLab 15.0, following which all newly fixed bugs and newly released features in the new major versions of the analyzers will not be available in the deprecated versions because we do not backport bugs and new features as per our [maintenance policy](https://docs.gitlab.com/ee/policy/maintenance.html). As required security patches will be backported within the latest 3 minor releases. -Specifically, the following are being deprecated and will no longer be updated after 15.0 GitLab release: +Audit events for [repository events](https://docs.gitlab.com/ee/administration/audit_events.html#repository-push-deprecated) are now deprecated and will be removed in GitLab 15.0. -- API Security: version 1 -- Container Scanning: version 4 -- Coverage-guided fuzz testing: version 2 -- Dependency Scanning: version 2 -- Dynamic Application Security Testing (DAST): version 2 -- License Scanning: version 3 -- Secret Detection: version 3 -- Static Application Security Testing (SAST): version 2, except security-code-scan which is version 3 - - `bandit`: version 2 - - `brakeman`: version 2 - - `eslint`: version 2 - - `flawfinder`: version 2 - - `gosec`: version 3 - - `kubesec`: version 2 - - `mobsf`: version 2 - - `nodejs-scan`: version 2 - - `phpcs-security-audit`: version 2 - - `pmd-apex`: version 2 - - `security-code-scan`: version 3 - - `semgrep`: version 2 - - `sobelow`: version 2 - - `spotbugs`: version 2 +These events have always been disabled by default and had to be manually enabled with a +feature flag. Enabling them can cause too many events to be generated which can +dramatically slow down GitLab instances. For this reason, they are being removed. -**Planned removal milestone: 15.00 ()** +**Planned removal milestone: 15.0 (2022-05-22)** -### Support for gRPC-aware proxy deployed between Gitaly and rest of GitLab +### GitLab Serverless WARNING: This feature will be changed or removed in 15.0 @@ -1335,21 +1425,13 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -Although not recommended or documented, it was possible to deploy a gRPC-aware proxy between Gitaly and -the rest of GitLab. For example, NGINX and Envoy. The ability to deploy a gRPC-aware proxy is -[deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/352517). If you currently use a gRPC-aware proxy for -Gitaly connections, you should change your proxy configuration to use TCP or TLS proxying (OSI layer 4) instead. - -Gitaly Cluster became incompatible with gRPC-aware proxies in GitLab 13.12. Now all GitLab installations will be incompatible with -gRPC-aware proxies, even without Gitaly Cluster. +[GitLab Serverless](https://docs.gitlab.com/ee/user/project/clusters/serverless/) is a feature set to support Knative-based serverless development with automatic deployments and monitoring. -By sending some of our internal RPC traffic through a custom protocol (instead of gRPC) we -increase throughput and reduce Go garbage collection latency. For more information, see -the [relevant epic](https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/463). +We decided to remove the GitLab Serverless features as they never really resonated with our users. Besides, given the continuous development of Kubernetes and Knative, our current implementations do not even work with recent versions. **Planned removal milestone: 15.0 (2022-05-22)** -### Test coverage project CI/CD setting +### Legacy database configuration WARNING: This feature will be changed or removed in 15.0 @@ -1357,16 +1439,15 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -To simplify setting a test coverage pattern, in GitLab 15.0 the -[project setting for test coverage parsing](https://docs.gitlab.com/ee/ci/pipelines/settings.html#add-test-coverage-results-to-a-merge-request-deprecated) -is being removed. +The syntax of [GitLabs database](https://docs.gitlab.com/omnibus/settings/database.html) +configuration located in `database.yml` is changing and the legacy format is deprecated. The legacy format +supported using a single PostgreSQL adapter, whereas the new format is changing to support multiple databases. The `main:` database needs to be defined as a first configuration item. -Instead, using the project’s `.gitlab-ci.yml`, provide a regular expression with the `coverage` keyword to set -testing coverage results in merge requests. +This deprecation mainly impacts users compiling GitLab from source because Omnibus will handle this configuration automatically. **Planned removal milestone: 15.0 (2022-05-22)** -### Vulnerability Check +### OmniAuth Kerberos gem WARNING: This feature will be changed or removed in 15.0 @@ -1374,42 +1455,33 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -The vulnerability check feature is deprecated in GitLab 14.8 and scheduled for removal in GitLab 15.0. We encourage you to migrate to the new security approvals feature instead. You can do so by navigating to **Security & Compliance > Policies** and creating a new Scan Result Policy. +The `omniauth-kerberos` gem will be removed in our next major release, GitLab 15.0. -The new security approvals feature is similar to vulnerability check. For example, both can require approvals for MRs that contain security vulnerabilities. However, security approvals improve the previous experience in several ways: +This gem has not been maintained and has very little usage. We therefore plan to remove support for this authentication method and recommend using the Kerberos [SPNEGO](https://en.wikipedia.org/wiki/SPNEGO) integration instead. You can follow the [upgrade instructions](https://docs.gitlab.com/ee/integration/kerberos.html#upgrading-from-password-based-to-ticket-based-kerberos-sign-ins) to upgrade from the `omniauth-kerberos` integration to the supported one. -- Users can choose who is allowed to edit security approval rules. An independent security or compliance team can therefore manage rules in a way that prevents development project maintainers from modifying the rules. -- Multiple rules can be created and chained together to allow for filtering on different severity thresholds for each scanner type. -- A two-step approval process can be enforced for any desired changes to security approval rules. -- A single set of security policies can be applied to multiple development projects to allow for ease in maintaining a single, centralized ruleset. +Note that we are not deprecating the Kerberos SPNEGO integration, only the old password-based Kerberos integration. **Planned removal milestone: 15.0 (2022-05-22)** -### `CI_BUILD_*` predefined variables +## 14.2 -WARNING: -This feature will be changed or removed in 15.0 -as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#breaking-changes). -Before updating GitLab, review the details carefully to determine if you need to make any -changes to your code, settings, or workflow. +### Release CLI distributed as a generic package -The predefined CI/CD variables that start with `CI_BUILD_*` were deprecated in GitLab 9.0, and will be removed in GitLab 15.0. If you still use these variables, be sure to change to the current [`CI_JOB_*` predefined variables](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html) which are identical (except for the updated name). +The [release-cli](https://gitlab.com/gitlab-org/release-cli) will be released as a [generic package](https://gitlab.com/gitlab-org/release-cli/-/packages) starting in GitLab 14.2. We will continue to deploy it as a binary to S3 until GitLab 14.5 and stop distributing it in S3 in GitLab 14.6. -**Planned removal milestone: 15.0 (2022-05-22)** +**Planned removal milestone: 14.6 (2021-12-22)** -### `fixup!` commit messages setting draft status of associated Merge Request +### Rename Task Runner pod to Toolbox -The use of `fixup!` as a commit message to trigger draft status -of the associated Merge Request is generally unused, and can cause -confusion with other uses of the term. "Draft" is the preferred -and supported trigger for triggering draft status from commit -messages, as part of our streamlining of the feature. -Support for `fixup!` is now considered deprecated, and will be -removed in GitLab 15.0. +The Task Runner pod is used to execute periodic housekeeping tasks within the GitLab application and is often confused with the GitLab Runner. Thus, [Task Runner will be renamed to Toolbox](https://gitlab.com/groups/gitlab-org/charts/-/epics/25). -**Planned removal milestone: 15.0 (2022-06-22)** +This will result in the rename of the sub-chart: `gitlab/task-runner` to `gitlab/toolbox`. Resulting pods will be named along the lines of `{{ .Release.Name }}-toolbox`, which will often be `gitlab-toolbox`. They will be locatable with the label `app=toolbox`. -### `projectFingerprint` in `PipelineSecurityReportFinding` GraphQL +**Planned removal milestone: 14.5 (2021-11-22)** + +## 14.0 + +### NFS for Git repository storage WARNING: This feature will be changed or removed in 15.0 @@ -1417,14 +1489,19 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -The `projectFingerprint` field in the [PipelineSecurityReportFinding](https://docs.gitlab.com/ee/api/graphql/reference/index.html#pipelinesecurityreportfinding) -GraphQL object is being deprecated. This field contains a "fingerprint" of security findings used to determine uniqueness. -The method for calculating fingerprints has changed, resulting in different values. Going forward, the new values will be -exposed in the UUID field. Data previously available in the projectFingerprint field will eventually be removed entirely. +With the general availability of Gitaly Cluster ([introduced in GitLab 13.0](https://about.gitlab.com/releases/2020/05/22/gitlab-13-0-released/)), we have deprecated development (bugfixes, performance improvements, etc) for NFS for Git repository storage in GitLab 14.0. We will continue to provide technical support for NFS for Git repositories throughout 14.x, but we will remove all support for NFS in GitLab 15.0. Please see our official [Statement of Support](https://about.gitlab.com/support/statement-of-support.html#gitaly-and-nfs) for further information. + +Gitaly Cluster offers tremendous benefits for our customers such as: + +- [Variable replication factors](https://docs.gitlab.com/ee/administration/gitaly/index.html#replication-factor). +- [Strong consistency](https://docs.gitlab.com/ee/administration/gitaly/index.html#strong-consistency). +- [Distributed read capabilities](https://docs.gitlab.com/ee/administration/gitaly/index.html#distributed-reads). + +We encourage customers currently using NFS for Git repositories to plan their migration by reviewing our documentation on [migrating to Gitaly Cluster](https://docs.gitlab.com/ee/administration/gitaly/index.html#migrate-to-gitaly-cluster). **Planned removal milestone: 15.0 (2022-05-22)** -### `started` iterations API field +### OAuth implicit grant WARNING: This feature will be changed or removed in 15.0 @@ -1432,6 +1509,6 @@ as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#brea Before updating GitLab, review the details carefully to determine if you need to make any changes to your code, settings, or workflow. -The `started` field in the [iterations API](https://docs.gitlab.com/ee/api/iterations.html#list-project-iterations) is being deprecated and will be removed in GitLab 15.0. This field is being replaced with the `current` field (already available) which aligns with the naming for other time-based entities, such as milestones. +The OAuth implicit grant authorization flow will be removed in our next major release, GitLab 15.0. Any applications that use OAuth implicit grant should switch to alternative [supported OAuth flows](https://docs.gitlab.com/ee/api/oauth2.html). **Planned removal milestone: 15.0 (2022-05-22)** diff --git a/doc/update/index.md b/doc/update/index.md index 45b5f36bc5a..5a00a728535 100644 --- a/doc/update/index.md +++ b/doc/update/index.md @@ -148,9 +148,22 @@ If you get this error, [check the batched background migration options](../user/ ### What do I do if my background migrations are stuck? WARNING: -The following operations can disrupt your GitLab performance. +The following operations can disrupt your GitLab performance. They run a number of Sidekiq jobs that perform various database or file updates. -It is safe to re-execute these commands, especially if you have 1000+ pending jobs which would likely overflow your runtime memory. +#### Background migrations remain in the Sidekiq queue + +Run the following check. If it returns non-zero and the count does not decrease over time, follow the rest of the steps in this section. + +```shell +# For Omnibus installations: +sudo gitlab-rails runner -e production 'puts Gitlab::BackgroundMigration.remaining' + +# For installations from source: +cd /home/git/gitlab +sudo -u git -H bundle exec rails runner -e production 'puts Gitlab::BackgroundMigration.remaining' +``` + +It is safe to re-execute the following commands, especially if you have 1000+ pending jobs which would likely overflow your runtime memory. **For Omnibus installations** @@ -176,7 +189,52 @@ pending_job_classes = scheduled_queue.select { |job| job["class"] == "Background pending_job_classes.each { |job_class| Gitlab::BackgroundMigration.steal(job_class) } ``` -**Batched migrations (GitLab 14.0 and newer):** +#### Background migrations stuck in 'pending' state + +GitLab 13.6 introduced an issue where a background migration named `BackfillJiraTrackerDeploymentType2` can be permanently stuck in a **pending** state across upgrades. To clean up this stuck migration, see the [13.6.0 version-specific instructions](#1360). + +For other background migrations stuck in pending, run the following check. If it returns non-zero and the count does not decrease over time, follow the rest of the steps in this section. + +```shell +# For Omnibus installations: +sudo gitlab-rails runner -e production 'puts Gitlab::Database::BackgroundMigrationJob.pending.count' + +# For installations from source: +cd /home/git/gitlab +sudo -u git -H bundle exec rails runner -e production 'puts Gitlab::Database::BackgroundMigrationJob.pending.count' +``` + +It is safe to re-attempt these migrations to clear them out from a pending status: + +**For Omnibus installations** + +```shell +# Start the rails console +sudo gitlab-rails c + +# Execute the following in the rails console +Gitlab::Database::BackgroundMigrationJob.pending.find_each do |job| + puts "Running pending job '#{job.class_name}' with arguments #{job.arguments}" + result = Gitlab::BackgroundMigration.perform(job.class_name, job.arguments) + puts "Result: #{result}" +end +``` + +**For installations from source** + +```shell +# Start the rails console +sudo -u git -H bundle exec rails RAILS_ENV=production + +# Execute the following in the rails console +Gitlab::Database::BackgroundMigrationJob.pending.find_each do |job| + puts "Running pending job '#{job.class_name}' with arguments #{job.arguments}" + result = Gitlab::BackgroundMigration.perform(job.class_name, job.arguments) + puts "Result: #{result}" +end +``` + +#### Batched migrations (GitLab 14.0 and later) See [troubleshooting batched background migrations](../user/admin_area/monitoring/background_migrations.md#troubleshooting). @@ -192,6 +250,8 @@ To address the above two scenario's, it is advised to do the following prior to 1. Pause your runners. 1. Wait until all jobs are finished. 1. Upgrade GitLab. +1. [Update GitLab Runner](https://docs.gitlab.com/runner/install/index.html) to the same version + as your GitLab version. Both versions [should be the same](https://docs.gitlab.com/runner/#gitlab-runner-versions). ## Checking for pending Advanced Search migrations **(PREMIUM SELF)** @@ -230,14 +290,12 @@ Backward-incompatible changes and migrations are reserved for major versions. Follow the directions carefully as we cannot guarantee that upgrading between major versions is seamless. -It is required to follow the following upgrade steps to ensure a successful *major* version upgrade: +A *major* upgrade requires the following steps: +1. Start by identifying a [supported upgrade path](#upgrade-paths). This is essential for a successful *major* version upgrade. 1. Upgrade to the latest minor version of the preceding major version. -1. Upgrade to the next major version (`X.0.Z`). -1. Upgrade to its first minor version (`X.1.Z`). -1. Proceed with upgrading to a newer releases of that major version. - -Identify a [supported upgrade path](#upgrade-paths). +1. Upgrade to the "dot zero" release of the next major version (`X.0.Z`). +1. Optional. Follow the [upgrade path](#upgrade-paths), and proceed with upgrading to newer releases of that major version. It's also important to ensure that any [background migrations have been fully completed](#checking-for-background-migrations-before-upgrading) before upgrading to a new major version. @@ -261,7 +319,7 @@ Find where your version sits in the upgrade path below, and upgrade GitLab accordingly, while also consulting the [version-specific upgrade instructions](#version-specific-upgrading-instructions): -`8.11.Z` -> `8.12.0` -> `8.17.7` -> `9.5.10` -> `10.8.7` -> [`11.11.8`](#1200) -> `12.0.12` -> [`12.1.17`](#1210) -> `12.10.14` -> `13.0.14` -> [`13.1.11`](#1310) -> [`13.8.8`](#1388) -> [latest `13.12.Z`](https://about.gitlab.com/releases/categories/releases/) -> [latest `14.0.Z`](#1400) -> [latest `14.1.Z`](#1410) -> [latest `14.Y.Z`](https://about.gitlab.com/releases/categories/releases/) +`8.11.Z` -> `8.12.0` -> `8.17.7` -> `9.5.10` -> `10.8.7` -> [`11.11.8`](#1200) -> `12.0.12` -> [`12.1.17`](#1210) -> `12.10.14` -> `13.0.14` -> [`13.1.11`](#1310) -> [`13.8.8`](#1388) -> [`13.12.15`](#13120) -> [`14.0.12`](#1400) -> [latest `14.Y.Z`](https://gitlab.com/gitlab-org/gitlab/-/releases) The following table, while not exhaustive, shows some examples of the supported upgrade paths. @@ -269,8 +327,8 @@ Additional steps between the mentioned versions are possible. We list the minima | Target version | Your version | Supported upgrade path | Note | | -------------- | ------------ | ---------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | -| `14.2.6` | `13.10.2` | `13.10.2` -> `13.12.15` -> `14.0.11` -> `14.1.8` -> `14.2.6` | Three intermediate versions are required: `13.12`, `14.0`, and `14.1`, then `14.2.6`. | -| `14.1.8` | `13.9.2` | `13.9.2` -> `13.12.15` -> `14.0.11` -> `14.1.8` | Two intermediate versions are required: `13.12` and `14.0`, then `14.1.8`. | +| `14.6.2` | `13.10.2` | `13.10.2` -> `13.12.15` -> `14.0.12` -> `14.6.2` | Two intermediate versions are required: `13.12` and `14.0`, then `14.6.2`. | +| `14.1.8` | `13.9.2` | `13.9.2` -> `13.12.15` -> `14.0.12` -> `14.1.8` | Two intermediate versions are required: `13.12` and `14.0`, then `14.1.8`. | | `13.12.15` | `12.9.2` | `12.9.2` -> `12.10.14` -> `13.0.14` -> `13.1.11` -> `13.8.8` -> `13.12.15` | Four intermediate versions are required: `12.10`, `13.0`, `13.1` and `13.8.8`, then `13.12.15`. | | `13.2.10` | `11.5.0` | `11.5.0` -> `11.11.8` -> `12.0.12` -> `12.1.17` -> `12.10.14` -> `13.0.14` -> `13.1.11` -> `13.2.10` | Six intermediate versions are required: `11.11`, `12.0`, `12.1`, `12.10`, `13.0` and `13.1`, then `13.2.10`. | | `12.10.14` | `11.3.4` | `11.3.4` -> `11.11.8` -> `12.0.12` -> `12.1.17` -> `12.10.14` | Three intermediate versions are required: `11.11`, `12.0` and `12.1`, then `12.10.14`. | @@ -300,6 +358,8 @@ Edition, follow the guides below based on the installation method: script, start the application and check its status. - [Omnibus CE to EE](package/convert_to_ee.md) - Follow this guide to update your Omnibus GitLab Community Edition to the Enterprise Edition. +- [Docker CE to EE](../install/docker.md#convert-community-edition-to-enterprise-edition) - + Follow this guide to update your GitLab Community Edition container to an Enterprise Edition container. ### Enterprise to Community Edition @@ -336,6 +396,36 @@ NOTE: Specific information that follow related to Ruby and Git versions do not apply to [Omnibus installations](https://docs.gitlab.com/omnibus/) and [Helm Chart deployments](https://docs.gitlab.com/charts/). They come with appropriate Ruby and Git versions and are not using system binaries for Ruby and Git. There is no need to install Ruby or Git when utilizing these two approaches. +### 14.8.0 + +- The agent server for Kubernetes [is enabled by default](https://about.gitlab.com/releases/2022/02/22/gitlab-14-8-released/#the-agent-server-for-kubernetes-is-enabled-by-default) + on Omnibus installations. If you run GitLab at scale, + such as [the reference architectures](../administration/reference_architectures/index.md), + you must disable the agent on the following server types, **if the agent is not required**. + + - Praefect + - Gitaly + - Sidekiq + - Redis (if configured using `redis['enable'] = true` and not via `roles`) + - Container registry + - Any other server types based on `roles(['application_role'])`, such as the GitLab Rails nodes + + [The reference architectures](../administration/reference_architectures/index.md) have been updated + with this configuration change and a specific role for standalone Redis servers. + + Steps to disable the agent: + + 1. Add `gitlab_kas['enable'] = false` to `gitlab.rb`. + 1. If the server is already upgraded to 14.8, run `gitlab-ctl reconfigure`. + +### 14.7.0 + +- See [LFS objects import and mirror issue in GitLab 14.6.0 to 14.7.2](#lfs-objects-import-and-mirror-issue-in-gitlab-1460-to-1472). + +### 14.6.0 + +- See [LFS objects import and mirror issue in GitLab 14.6.0 to 14.7.2](#lfs-objects-import-and-mirror-issue-in-gitlab-1460-to-1472). + ### 14.5.0 - When `make` is run, Gitaly builds are now created in `_build/bin` and no longer in the root directory of the source directory. If you @@ -364,6 +454,15 @@ or [init scripts](upgrading_from_source.md#configure-sysv-init-script) by [follo For more information, refer to [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/331823). +### 14.4.4 + +- For [zero-downtime upgrades](zero_downtime.md) on a GitLab cluster with separate Web and API nodes, you need to enable the `paginated_tree_graphql_query` [feature flag](../administration/feature_flags.md#enable-or-disable-the-feature) _before_ upgrading GitLab Web nodes to 14.4. + This is because we [enabled `paginated_tree_graphql_query by default in 14.4](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/70913/diffs), so if GitLab UI is on 14.4 and its API is on 14.3, the frontend will have this feature enabled but the backend will have it disabled. This will result in the following error: + + ```shell + bundle.esm.js:63 Uncaught (in promise) Error: GraphQL error: Field 'paginatedTree' doesn't exist on type 'Repository' + ``` + ### 14.4.0 - Git 2.33.x and later is required. We recommend you use the @@ -585,6 +684,19 @@ Ruby 2.7.2 is required. GitLab does not start with Ruby 2.6.6 or older versions. The required Git version is Git v2.29 or higher. +GitLab 13.6 includes a +[background migration `BackfillJiraTrackerDeploymentType2`](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/46368) +that may remain stuck permanently in a **pending** state despite completion of work +due to a bug. + +To clean up this stuck job, run the following in the [GitLab Rails Console](../administration/operations/rails_console.md): + +```ruby +Gitlab::Database::BackgroundMigrationJob.pending.where(class_name: "BackfillJiraTrackerDeploymentType2").find_each do |job| + puts Gitlab::Database::BackgroundMigrationJob.mark_all_as_succeeded("BackfillJiraTrackerDeploymentType2", job.arguments) +end +``` + ### 13.4.0 GitLab 13.4.0 includes a background migration to [move all remaining repositories in legacy storage to hashed storage](../administration/raketasks/storage.md#migrate-to-hashed-storage). There are [known issues with this migration](https://gitlab.com/gitlab-org/gitlab/-/issues/259605) which are fixed in GitLab 13.5.4 and later. If possible, skip 13.4.0 and upgrade to 13.5.4 or higher instead. Note that the migration can take quite a while to run, depending on how many repositories must be moved. Be sure to check that all background migrations have completed before upgrading further. @@ -684,6 +796,12 @@ Users who were signed in before Maintenance mode was enabled will continue to be [This bug](https://gitlab.com/gitlab-org/gitlab/-/issues/329261) was fixed in GitLab 14.5.0 and backported into 14.4.3 and 14.3.5. +### LFS objects import and mirror issue in GitLab 14.6.0 to 14.7.2 + +When Geo is enabled, LFS objects fail to be saved for imported or mirrored projects. + +[This bug](https://gitlab.com/gitlab-org/gitlab/-/issues/352368) was fixed in GitLab 14.8.0 and backported into 14.7.3. + ## Miscellaneous - [MySQL to PostgreSQL](mysql_to_postgresql.md) guides you through migrating diff --git a/doc/update/package/convert_to_ee.md b/doc/update/package/convert_to_ee.md index 2cc54e2c8cf..d5a71ba3e80 100644 --- a/doc/update/package/convert_to_ee.md +++ b/doc/update/package/convert_to_ee.md @@ -1,7 +1,7 @@ --- stage: Enablement group: Distribution -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Convert Community Edition to Enterprise Edition **(FREE SELF)** @@ -91,8 +91,8 @@ The steps can be summed up to: sudo gitlab-ctl reconfigure ``` -1. Now go to the GitLab admin panel of your server (`/admin/license/new`) and - upload your license file. +1. Now go to the GitLab admin panel of your server (`/admin/subscription`) and + [add your license](../../user/admin_area/license.md). 1. After you confirm that GitLab is working as expected, you may remove the old Community Edition repository: diff --git a/doc/update/package/downgrade.md b/doc/update/package/downgrade.md index 96b17a7632b..81f7d089bea 100644 --- a/doc/update/package/downgrade.md +++ b/doc/update/package/downgrade.md @@ -1,7 +1,7 @@ --- stage: Enablement group: Distribution -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Downgrade **(FREE SELF)** diff --git a/doc/update/package/index.md b/doc/update/package/index.md index d7c18f15e44..b6417e10917 100644 --- a/doc/update/package/index.md +++ b/doc/update/package/index.md @@ -1,7 +1,7 @@ --- stage: Enablement group: Distribution -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Upgrade GitLab by using the GitLab package **(FREE SELF)** diff --git a/doc/update/plan_your_upgrade.md b/doc/update/plan_your_upgrade.md index 665d2f6783e..6eebfd1b278 100644 --- a/doc/update/plan_your_upgrade.md +++ b/doc/update/plan_your_upgrade.md @@ -94,6 +94,8 @@ to roll back GitLab to a working state if there's a problem with the upgrade: ### Restore GitLab +If you have a test environment that mimics your production one, we recommend testing the restoration to ensure that everything works as you expect. + To restore your GitLab backup: - Before restoring, make sure to read about the diff --git a/doc/update/removals.md b/doc/update/removals.md index 9a28adf1f96..7e2b4f84fa1 100644 --- a/doc/update/removals.md +++ b/doc/update/removals.md @@ -28,6 +28,79 @@ For removal reviewers (Technical Writers only): https://about.gitlab.com/handbook/marketing/blog/release-posts/#update-the-removals-doc --> +## 14.9 + +### Integrated error tracking disabled by default + +WARNING: +This feature was changed or removed in 14.9 +as a [breaking change](https://docs.gitlab.com/ee/development/contributing/#breaking-changes). +Before updating GitLab, review the details carefully to determine if you need to make any +changes to your code, settings, or workflow. + +In GitLab 14.4, GitLab released an integrated error tracking backend that replaces Sentry. This feature caused database performance issues. In GitLab 14.9, integrated error tracking is removed from GitLab.com, and turned off by default in GitLab self-managed. While we explore the future development of this feature, please consider switching to the Sentry backend by [changing your error tracking to Sentry in your project settings](https://docs.gitlab.com/ee/operations/error_tracking.html#sentry-error-tracking). + +For additional background on this removal, please reference [Disable Integrated Error Tracking by Default](https://gitlab.com/groups/gitlab-org/-/epics/7580). If you have feedback please add a comment to [Feedback: Removal of Integrated Error Tracking](https://gitlab.com/gitlab-org/gitlab/-/issues/355493). + +## 14.6 + +### Limit the number of triggered pipeline to 25K in free tier + +A large amount of triggered pipelines in a single project impacts the performance of GitLab.com. In GitLab 14.6, we are limiting the number of triggered pipelines in a single project on GitLab.com at any given moment to 25,000. This change applies to projects in the free tier only, Premium and Ultimate are not affected by this change. + +### Release CLI distributed as a generic package + +The [release-cli](https://gitlab.com/gitlab-org/release-cli) will be released as a [generic package](https://gitlab.com/gitlab-org/release-cli/-/packages) starting in GitLab 14.2. We will continue to deploy it as a binary to S3 until GitLab 14.5 and stop distributing it in S3 in GitLab 14.6. + +## 14.3 + +### Introduced limit of 50 tags for jobs + +GitLab values efficiency and is prioritizing reliability for [GitLab.com in FY22](https://about.gitlab.com/direction/#gitlab-hosted-first). In 14.3, GitLab CI/CD jobs must have less than 50 [tags](https://docs.gitlab.com/ee/ci/yaml/index.html#tags). If a pipeline contains a job with 50 or more tags, you will receive an error and the pipeline will not be created. + +### List project pipelines API endpoint removes `name` support in 14.3 + +In GitLab 14.3, we will remove the ability to filter by `name` in the [list project pipelines API endpoint](https://docs.gitlab.com/ee/api/pipelines.html#list-project-pipelines) to improve performance. If you currently use this parameter with this endpoint, you must switch to `username`. + +### Use of legacy storage setting + +The support for [`gitlab_pages['use_legacy_storage']` setting](https://docs.gitlab.com/ee/administration/pages/index.html#domain-source-configuration-before-140) in Omnibus installations has been removed. + +In 14.0 we removed [`domain_config_source`](https://docs.gitlab.com/ee/administration/pages/index.html#domain-source-configuration-before-140) which had been previously deprecated, and allowed users to specify disk storage. In 14.0 we added `use_legacy_storage` as a **temporary** flag to unblock upgrades, and allow us to debug issues with our users and it was deprecated and communicated for removal in 14.3. + +## 14.2 + +### Max job log file size of 100 MB + +GitLab values efficiency for all users in our wider community of contributors, so we're always working hard to make sure the application performs at a high level with a lovable UX. + In GitLab 14.2, we have introduced a [job log file size limit](https://docs.gitlab.com/ee/administration/instance_limits.html#maximum-file-size-for-job-logs), set to 100 megabytes by default. Administrators of self-managed GitLab instances can customize this to any value. All jobs that exceed this limit are dropped and marked as failed, helping prevent performance impacts or over-use of resources. This ensures that everyone using GitLab has the best possible experience. + +## 14.1 + +### Remove support for `prometheus.listen_address` and `prometheus.enable` + +The support for `prometheus.listen_address` and `prometheus.enable` has been removed from `gitlab.yml`. Use `prometheus.enabled` and `prometheus.server_address` to set up Prometheus server that GitLab instance connects to. Refer to [our documentation](https://docs.gitlab.com/ee/install/installation.html#prometheus-server-setup) for details. + +This only affects new installations from source where users might use the old configurations. + +### Remove support for older browsers + +In GitLab 14.1, we are cleaning up and [removing old code](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63994) that was specific for browsers that we no longer support. This has no impact on users when one of our [supported web browsers](https://docs.gitlab.com/ee/install/requirements.html#supported-web-browsers) is used. + +Most notably, support for the following browsers has been removed: + +- Apple Safari 13 and older. +- Mozilla Firefox 68. +- Pre-Chromium Microsoft Edge. + +The minimum supported browser versions are: + +- Apple Safari 13.1. +- Mozilla Firefox 78. +- Google Chrome 84. +- Chromium 84. +- Microsoft Edge 84. + ## 14.0 ### Auto Deploy CI template v1 @@ -510,8 +583,8 @@ As we continuously [develop GitLab's Terraform integrations](https://gitlab.com/ At every major release of GitLab, the "latest version" template becomes the "major version" template, inheriting the "latest template" setup. As we have added many new features to the Terraform integration, the new setup for the "major version" template can be considered a breaking change. -The latest template supports the [Terraform Merge Request widget](https://docs.gitlab.com/ee/user/infrastructure/mr_integration.html) and -doesn't need additional setup to work with the [GitLab managed Terraform state](https://docs.gitlab.com/ee/user/infrastructure/terraform_state.html). +The latest template supports the [Terraform Merge Request widget](https://docs.gitlab.com/ee/user/infrastructure/iac/mr_integration.html) and +doesn't need additional setup to work with the [GitLab managed Terraform state](https://docs.gitlab.com/ee/user/infrastructure/iac/terraform_state.html). To check the new changes, see the [new "major version" template](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform.gitlab-ci.yml). @@ -671,62 +744,3 @@ Before updating GitLab, review the details carefully to determine if you need to changes to your code, settings, or workflow. GitLab Runner was updated in GitLab 13.4 to internally stop passing the `trace` parameter to the `/api/jobs/:id` endpoint. GitLab 14.0 deprecates the `trace` parameter entirely for all other requests of this endpoint. Make sure your [GitLab Runner version matches your GitLab version](https://docs.gitlab.com/runner/#gitlab-runner-versions) to ensure consistent behavior. - -## 14.1 - -### Remove support for `prometheus.listen_address` and `prometheus.enable` - -The support for `prometheus.listen_address` and `prometheus.enable` has been removed from `gitlab.yml`. Use `prometheus.enabled` and `prometheus.server_address` to set up Prometheus server that GitLab instance connects to. Refer to [our documentation](https://docs.gitlab.com/ee/install/installation.html#prometheus-server-setup) for details. - -This only affects new installations from source where users might use the old configurations. - -### Remove support for older browsers - -In GitLab 14.1, we are cleaning up and [removing old code](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63994) that was specific for browsers that we no longer support. This has no impact on users when one of our [supported web browsers](https://docs.gitlab.com/ee/install/requirements.html#supported-web-browsers) is used. - -Most notably, support for the following browsers has been removed: - -- Apple Safari 13 and older. -- Mozilla Firefox 68. -- Pre-Chromium Microsoft Edge. - -The minimum supported browser versions are: - -- Apple Safari 13.1. -- Mozilla Firefox 78. -- Google Chrome 84. -- Chromium 84. -- Microsoft Edge 84. - -## 14.2 - -### Max job log file size of 100 MB - -GitLab values efficiency for all users in our wider community of contributors, so we're always working hard to make sure the application performs at a high level with a lovable UX. - In GitLab 14.2, we have introduced a [job log file size limit](https://docs.gitlab.com/ee/administration/instance_limits.html#maximum-file-size-for-job-logs), set to 100 megabytes by default. Administrators of self-managed GitLab instances can customize this to any value. All jobs that exceed this limit are dropped and marked as failed, helping prevent performance impacts or over-use of resources. This ensures that everyone using GitLab has the best possible experience. - -## 14.3 - -### Introduced limit of 50 tags for jobs - -GitLab values efficiency and is prioritizing reliability for [GitLab.com in FY22](https://about.gitlab.com/direction/#gitlab-hosted-first). In 14.3, GitLab CI/CD jobs must have less than 50 [tags](https://docs.gitlab.com/ee/ci/yaml/index.html#tags). If a pipeline contains a job with 50 or more tags, you will receive an error and the pipeline will not be created. - -### List project pipelines API endpoint removes `name` support in 14.3 - -In GitLab 14.3, we will remove the ability to filter by `name` in the [list project pipelines API endpoint](https://docs.gitlab.com/ee/api/pipelines.html#list-project-pipelines) to improve performance. If you currently use this parameter with this endpoint, you must switch to `username`. - -### Use of legacy storage setting - -The support for [`gitlab_pages['use_legacy_storage']` setting](https://docs.gitlab.com/ee/administration/pages/index.html#domain-source-configuration-before-140) in Omnibus installations has been removed. - -In 14.0 we removed [`domain_config_source`](https://docs.gitlab.com/ee/administration/pages/index.html#domain-source-configuration-before-140) which had been previously deprecated, and allowed users to specify disk storage. In 14.0 we added `use_legacy_storage` as a **temporary** flag to unblock upgrades, and allow us to debug issues with our users and it was deprecated and communicated for removal in 14.3. - -## 14.6 - -### Limit the number of triggered pipeline to 25K in free tier - -A large amount of triggered pipelines in a single project impacts the performance of GitLab.com. In GitLab 14.6, we are limiting the number of triggered pipelines in a single project on GitLab.com at any given moment to 25,000. This change applies to projects in the free tier only, Premium and Ultimate are not affected by this change. - -### Release CLI distributed as a generic package - -The [release-cli](https://gitlab.com/gitlab-org/release-cli) will be released as a [generic package](https://gitlab.com/gitlab-org/release-cli/-/packages) starting in GitLab 14.2. We will continue to deploy it as a binary to S3 until GitLab 14.5 and stop distributing it in S3 in GitLab 14.6. diff --git a/doc/update/zero_downtime.md b/doc/update/zero_downtime.md index 050c5909934..d69d60fe73d 100644 --- a/doc/update/zero_downtime.md +++ b/doc/update/zero_downtime.md @@ -41,6 +41,8 @@ you check the release posts of any releases between your current and target version just in case they include any migrations that may require you to upgrade one release at a time. +We also recommend you verify the [version specific upgrading instructions](index.md#version-specific-upgrading-instructions) relevant to your [upgrade path](index.md#upgrade-paths). + Some releases may also include so called "background migrations". These migrations are performed in the background by Sidekiq and are often used for migrating data. Background migrations are only added in the monthly releases. diff --git a/doc/user/admin_area/analytics/usage_trends.md b/doc/user/admin_area/analytics/usage_trends.md index a9c5adcf838..5f46908adb0 100644 --- a/doc/user/admin_area/analytics/usage_trends.md +++ b/doc/user/admin_area/analytics/usage_trends.md @@ -12,9 +12,6 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - It's enabled on GitLab.com. > - It's recommended for production use. -WARNING: -This feature might not be available to you. Check the **version history** note above for details. - Usage Trends gives you an overview of how much data your instance contains, and how quickly this volume is changing over time. Usage Trends data refreshes daily. diff --git a/doc/user/admin_area/broadcast_messages.md b/doc/user/admin_area/broadcast_messages.md index 987d7444ae0..69cb2f04c4d 100644 --- a/doc/user/admin_area/broadcast_messages.md +++ b/doc/user/admin_area/broadcast_messages.md @@ -7,7 +7,9 @@ type: reference, howto # Broadcast messages **(FREE SELF)** -GitLab can display broadcast messages to all users of a GitLab instance. There are two types of broadcast messages: +> Target roles [introduced](https://gitlab.com/gitlab-org/growth/team-tasks/-/issues/461) in GitLab 14.8 [with a flag](../../administration/feature_flags.md) named `role_targeted_broadcast_messages`. Disabled by default. + +GitLab can display broadcast messages to users of a GitLab instance. There are two types of broadcast messages: - Banners - Notifications @@ -66,6 +68,7 @@ To add a broadcast message: - `text-decoration` 1. Select one of the suggested background colors, or add the hex code of a different color. The default color is orange. 1. Select the **Dismissable** checkbox to enable users to dismiss the broadcast message. +1. Optional. Select **Target roles** to only show the broadcast message to users with the selected roles. The message displays on group, subgroup, and project pages, and does not display in Git remote responses. 1. If required, add a **Target Path** to only show the broadcast message on URLs matching that path. You can use the wildcard character `*` to match multiple URLs, for example `mygroup/myproject*`. 1. Select a date for the message to start and end. 1. Select **Add broadcast message**. diff --git a/doc/user/admin_area/credentials_inventory.md b/doc/user/admin_area/credentials_inventory.md index 437a72da767..21ac0f720ec 100644 --- a/doc/user/admin_area/credentials_inventory.md +++ b/doc/user/admin_area/credentials_inventory.md @@ -7,7 +7,8 @@ type: howto # Credentials inventory **(ULTIMATE SELF)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20912) in GitLab 12.6. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/20912) in GitLab 12.6. +> - [Bot-created access tokens not displayed in personal access token list](https://gitlab.com/gitlab-org/gitlab/-/issues/351759) in GitLab 14.9. GitLab administrators are responsible for the overall security of their instance. To assist, GitLab provides a Credentials inventory to keep track of all the credentials that can be used to access @@ -50,6 +51,8 @@ If you see a **Revoke** button, you can revoke that user's PAT. Whether you see When a PAT is revoked from the credentials inventory, the instance notifies the user by email. + + ## Revoke a user's project access token > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/243833) in GitLab 14.8. @@ -59,6 +62,8 @@ The **Revoke** button next to a project access token can be selected to revoke t - Revoke the token project access token. - Enqueue a background worker to delete the project bot user. + + ## Delete a user's SSH key > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/225248) in GitLab 13.5. @@ -66,7 +71,7 @@ The **Revoke** button next to a project access token can be selected to revoke t You can **Delete** a user's SSH key by navigating to the credentials inventory's SSH Keys tab. The instance then notifies the user. - + ## Review existing GPG keys @@ -80,4 +85,4 @@ credentials inventory GPG Keys tab, as well as the following properties: - The ID of the GPG key. - Whether the GPG key is [verified or unverified](../project/repository/gpg_signed_commits/index.md) - + diff --git a/doc/user/admin_area/img/credentials_inventory_gpg_keys_v13_10.png b/doc/user/admin_area/img/credentials_inventory_gpg_keys_v13_10.png Binary files differdeleted file mode 100644 index a88d80a72b6..00000000000 --- a/doc/user/admin_area/img/credentials_inventory_gpg_keys_v13_10.png +++ /dev/null diff --git a/doc/user/admin_area/img/credentials_inventory_gpg_keys_v14_9.png b/doc/user/admin_area/img/credentials_inventory_gpg_keys_v14_9.png Binary files differnew file mode 100644 index 00000000000..6c4c8f30df6 --- /dev/null +++ b/doc/user/admin_area/img/credentials_inventory_gpg_keys_v14_9.png diff --git a/doc/user/admin_area/img/credentials_inventory_personal_access_tokens_v14_9.png b/doc/user/admin_area/img/credentials_inventory_personal_access_tokens_v14_9.png Binary files differnew file mode 100644 index 00000000000..254d520d538 --- /dev/null +++ b/doc/user/admin_area/img/credentials_inventory_personal_access_tokens_v14_9.png diff --git a/doc/user/admin_area/img/credentials_inventory_project_access_tokens_v14_9.png b/doc/user/admin_area/img/credentials_inventory_project_access_tokens_v14_9.png Binary files differnew file mode 100644 index 00000000000..ae204ac09ef --- /dev/null +++ b/doc/user/admin_area/img/credentials_inventory_project_access_tokens_v14_9.png diff --git a/doc/user/admin_area/img/credentials_inventory_ssh_keys_v13_5.png b/doc/user/admin_area/img/credentials_inventory_ssh_keys_v13_5.png Binary files differdeleted file mode 100644 index f5edf513bbf..00000000000 --- a/doc/user/admin_area/img/credentials_inventory_ssh_keys_v13_5.png +++ /dev/null diff --git a/doc/user/admin_area/img/credentials_inventory_ssh_keys_v14_9.png b/doc/user/admin_area/img/credentials_inventory_ssh_keys_v14_9.png Binary files differnew file mode 100644 index 00000000000..8f2f11515eb --- /dev/null +++ b/doc/user/admin_area/img/credentials_inventory_ssh_keys_v14_9.png diff --git a/doc/user/admin_area/index.md b/doc/user/admin_area/index.md index 57a4a746ff0..4a334c28496 100644 --- a/doc/user/admin_area/index.md +++ b/doc/user/admin_area/index.md @@ -30,7 +30,7 @@ The Admin Area is made up of the following sections: | **{hook}** System Hooks | Configure [system hooks](../../system_hooks/system_hooks.md) for many events. | | **{applications}** Applications | Create system [OAuth applications](../../integration/oauth_provider.md) for integrations with other services. | | **{slight-frown}** Abuse Reports | Manage [abuse reports](review_abuse_reports.md) submitted by your users. | -| **{license}** License | Upload, display, and remove [licenses](license.md). | +| **{license}** License | Add, display, and remove [licenses](license.md). | | **{cloud-gear}** Kubernetes | Create and manage instance-level [Kubernetes clusters](../instance/clusters/index.md). | | **{push-rules}** Push rules | Configure pre-defined Git [push rules](../project/repository/push_rules.md) for projects. Also, configure [merge requests approvers rules](merge_requests_approvals.md). | | **{location-dot}** Geo | Configure and maintain [Geo nodes](geo_nodes.md). | @@ -218,6 +218,17 @@ You must be an administrator to manually add emails to users: The [Cohorts](user_cohorts.md) tab displays the monthly cohorts of new users and their activities over time. +### Prevent a user from creating groups + +By default, users can create groups. To prevent a user from creating groups: + +1. On the top bar, select **Menu > Admin**. +1. On the left sidebar, select **Overview > Users** (`/admin/users`). +1. Locate the user and select them. +1. Select **Edit**. +1. Clear the **Can create group** checkbox. +1. Select **Save changes**. + ### Administering Groups You can administer all groups in the GitLab instance from the Admin Area's Groups page. diff --git a/doc/user/admin_area/license.md b/doc/user/admin_area/license.md index 22133e30aa0..bee784e850b 100644 --- a/doc/user/admin_area/license.md +++ b/doc/user/admin_area/license.md @@ -1,183 +1,57 @@ --- -stage: Growth -group: Conversion +stage: Fulfillment +group: License info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Activate GitLab Enterprise Edition (EE) **(PREMIUM SELF)** -When you install a new GitLab instance without a license, it only has the Free features -enabled. To enable all features of GitLab Enterprise Edition (EE), activate -your instance with an activation code or a license file. When [the license expires](#what-happens-when-your-license-expires), -some functionality is locked. - -## Verify your GitLab edition +When you install a new GitLab instance without a license, only Free features +are enabled. To enable more features in GitLab Enterprise Edition (EE), activate +your instance with an activation code. -To activate your instance, make sure you are running GitLab Enterprise Edition (EE). +## Activate GitLab EE -To verify the edition, sign in to GitLab and select -**Help** (**{question-o}**) > **Help**. The GitLab edition and version are listed -at the top of the page. - -If you are running GitLab Community Edition (CE), upgrade your installation to GitLab -EE. For more details, see [Upgrading between editions](../../update/index.md#upgrading-between-editions). -If you have questions or need assistance upgrading from GitLab CE to EE, -[contact GitLab Support](https://about.gitlab.com/support/#contact-support). +In GitLab Enterprise Edition 14.1 and later, you need an activation code to activate +your instance. -## Activate GitLab EE with an activation code +Prerequisite: -In GitLab Enterprise Edition 14.1 and later, you need an activation code to activate -your instance. To get an activation code you have to [purchase a license](https://about.gitlab.com/pricing/). -The activation code is a 24-character alphanumeric string you receive in a confirmation email. -You can also sign in to the [Customers Portal](https://customers.gitlab.com/customers/sign_in) -to copy the activation code to your clipboard. +- You must [purchase a subscription](https://about.gitlab.com/pricing/). +- You must be running GitLab Enterprise Edition (EE). +- You must have GitLab 14.1 or later. +- Your instance must be connected to the internet. To activate your instance with an activation code: +1. Copy the activation code, a 24-character alphanumeric string, from either: + - Your subscription confirmation email. + - The [Customers Portal](https://customers.gitlab.com/customers/sign_in), on the **Manage Purchases** page. 1. Sign in to your GitLab self-managed instance. 1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Subscription**. -1. Enter the activation code in **Activation code**. +1. Paste the activation code in **Activation code**. 1. Read and accept the terms of service. 1. Select **Activate**. -## Activate GitLab EE with a license file - -If you receive a license file from GitLab (for example, for a trial), you can -upload it to your instance or add it during installation. The license file is -a base64-encoded ASCII text file with a `.gitlab-license` extension. +The subscription is activated. -## Upload your license - -The first time you sign in to your GitLab instance, a note with a -link to the **Upload license** page should be displayed. - -Otherwise, to upload your license: - -1. Sign in to GitLab as an administrator. -1. On the top bar, select **Menu > Admin**. -1. On the left sidebar, select **Settings**. -1. In the **License file** area, select **Upload a license**. -1. Upload a license: - - For a file, either: - - Select **Upload `.gitlab-license` file**, then **Choose File** and - select the license file from your local machine. - - Drag and drop the license file to the **Drag your license file here** area. - - For plain text, select **Enter license key** and paste the contents in - **License key**. -1. Select the **Terms of Service** checkbox. -1. Select **Upload License**. - -## Add your license during installation - -You can import a license file when you install GitLab. - -- **For installations from source** - - Place the `Gitlab.gitlab-license` file in the `config/` directory. - - To specify a custom location and filename for the license, set the - `GITLAB_LICENSE_FILE` environment variable with the path to the file: - - ```shell - export GITLAB_LICENSE_FILE="/path/to/license/file" - ``` - -- **For Omnibus package** - - Place the `Gitlab.gitlab-license` file in the `/etc/gitlab/` directory. - - To specify a custom location and filename for the license, add this entry to `gitlab.rb`: - - ```ruby - gitlab_rails['initial_license_file'] = "/path/to/license/file" - ``` - -WARNING: -These methods only add a license at the time of installation. To renew or upgrade -a license, upload the license in the **Admin Area** in the web user interface. - -## What happens when your license expires - -Fifteen days before the license expires, a notification banner with the upcoming expiration -date displays to GitLab administrators. - -When your license expires, GitLab locks features, like Git pushes -and issue creation. Your instance becomes read-only and -an expiration message displays to all administrators. You have a 14-day grace period -before this occurs. - -To resume functionality, [upload a new license](#upload-your-license). - -To go back to Free features, [delete all expired licenses](#remove-a-license-file). - -## Remove a license file - -To remove a license file from a self-managed instance: - -1. On the top bar, select **Menu > Admin**. -1. On the left sidebar, select **Subscription**. -1. Select **Remove license**. - -Repeat these steps to remove all licenses, including those applied in the past. - -## View license details and history - -To view your license details: - -1. On the top bar, select **Menu > Admin**. -1. On the left sidebar, select **Subscription**. - -You can upload and view more than one license, but only the latest license in -the current date range is the active license. - -When you upload a future-dated license, it doesn't take effect until its applicable date. -You can view all active subscriptions in the **Subscription history** table. - -You can also [export](../../subscriptions/self_managed/index.md) your license usage information to a CSV file. - -NOTE: -In GitLab 13.6 and earlier, a banner about an expiring license may continue to display -when you upload a new license. This happens when the start date of the new license -is in the future and the expiring one is still active. -The banner disappears after the new license becomes active. - -## Troubleshooting - -### No Subscription area in the Admin Area - -You cannot upload your license because there is no **Subscription** area. -This issue might occur if: - -- You're running GitLab Community Edition. Before you upload your license, you - must [upgrade to Enterprise Edition](../../update/index.md#community-to-enterprise-edition). -- You're using GitLab.com. You cannot upload a self-managed license to GitLab.com. - To use paid features on GitLab.com, [purchase a separate subscription](../../subscriptions/gitlab_com/index.md). - -### Users exceed license limit upon renewal - -GitLab displays a message prompting you to purchase -additional users. This issue occurs if you upload a license that does not have enough -users to cover the number of users in your instance. - -To fix this issue, purchase additional seats to cover those users. -For more information, read the [licensing FAQ](https://about.gitlab.com/pricing/licensing-faq/). - -In GitLab 14.2 and later, for instances that use a license file, the following -rules apply: - -- If the users over license are less than or equal to 10% of the users in the license - file, the license is applied and you pay the overage in the next renewal. -- If the users over license are more than 10% of the users in the license file, - you cannot apply the license without purchasing more users. +If you have an offline or airgapped environment, +[activate GitLab EE with a license file or key](license_file.md) instead. -For example, if you purchase a license for 100 users, you can have 110 users when you activate -your license. However, if you have 111 users, you must purchase more users before you can activate -the license. +If you have questions or need assistance activating your instance, +[contact GitLab Support](https://about.gitlab.com/support/#contact-support). -### Cannot activate instance due to connectivity error +When [the license expires](license_file.md#what-happens-when-your-license-expires), +some functionality is locked. -In GitLab 14.1 and later, to activate your subscription with an activation code, -your GitLab instance must be connected to the internet. +## Verify your GitLab edition -If you have an offline or airgapped environment, -[upload a license file](license.md#activate-gitlab-ee-with-a-license-file) instead. +To verify the edition, sign in to GitLab and select +**Help** (**{question-o}**) > **Help**. The GitLab edition and version are listed +at the top of the page. -If you have questions or need assistance activating your instance, +If you are running GitLab Community Edition (CE), you can upgrade your installation to GitLab +EE. For more details, see [Upgrading between editions](../../update/index.md#upgrading-between-editions). +If you have questions or need assistance upgrading from GitLab CE to EE, [contact GitLab Support](https://about.gitlab.com/support/#contact-support). diff --git a/doc/user/admin_area/license_file.md b/doc/user/admin_area/license_file.md new file mode 100644 index 00000000000..e0332e70681 --- /dev/null +++ b/doc/user/admin_area/license_file.md @@ -0,0 +1,127 @@ +--- +stage: Fulfillment +group: License +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Activate GitLab EE with a license file or key + +If you receive a license file from GitLab (for example, for a trial), you can +upload it to your instance or add it during installation. The license file is +a base64-encoded ASCII text file with a `.gitlab-license` extension. + +The first time you sign in to your GitLab instance, a note with a +link to the **Add license** page should be displayed. + +Otherwise, to add your license: + +1. Sign in to GitLab as an administrator. +1. On the top bar, select **Menu > Admin**. +1. On the left sidebar, select **Settings > General**. +1. In the **License file** area, select **Add a license**. +1. Add a license by either uploading the file or pasting the key. +1. Select the **Terms of Service** checkbox. +1. Select **Add license**. + +## Add your license during installation + +You can import a license file when you install GitLab. + +- **For installations from source** + - Place the `Gitlab.gitlab-license` file in the `config/` directory. + - To specify a custom location and filename for the license, set the + `GITLAB_LICENSE_FILE` environment variable with the path to the file: + + ```shell + export GITLAB_LICENSE_FILE="/path/to/license/file" + ``` + +- **For Omnibus package** + - Place the `Gitlab.gitlab-license` file in the `/etc/gitlab/` directory. + - To specify a custom location and filename for the license, add this entry to `gitlab.rb`: + + ```ruby + gitlab_rails['initial_license_file'] = "/path/to/license/file" + ``` + +WARNING: +These methods only add a license at the time of installation. To renew or upgrade +a license, add the license in the **Admin Area** in the web user interface. + +## What happens when your license expires + +Fifteen days before the license expires, a notification banner with the upcoming expiration +date displays to GitLab administrators. + +When your license expires, GitLab locks features, like Git pushes +and issue creation. Your instance becomes read-only and +an expiration message displays to all administrators. You have a 14-day grace period +before this occurs. + +To resume functionality, activate a new subscription. + +To go back to Free features, [delete all expired licenses](#remove-a-license). + +## Remove a license + +To remove a license from a self-managed instance: + +1. On the top bar, select **Menu > Admin**. +1. On the left sidebar, select **Subscription**. +1. Select **Remove license**. + +Repeat these steps to remove all licenses, including those applied in the past. + +## View license details and history + +To view your license details: + +1. On the top bar, select **Menu > Admin**. +1. On the left sidebar, select **Subscription**. + +You can add and view more than one license, but only the latest license in +the current date range is the active license. + +When you add a future-dated license, it doesn't take effect until its applicable date. +You can view all active subscriptions in the **Subscription history** table. + +You can also [export](../../subscriptions/self_managed/index.md) your license usage information to a CSV file. + +NOTE: +In GitLab 13.6 and earlier, a banner about an expiring license may continue to display +when you add a new license. This happens when the start date of the new license +is in the future and the expiring one is still active. +The banner disappears after the new license becomes active. + +## Troubleshooting + +### No Subscription area in the Admin Area + +You cannot add your license because there is no **Subscription** area. +This issue might occur if: + +- You're running GitLab Community Edition. Before you add your license, you + must [upgrade to Enterprise Edition](../../update/index.md#community-to-enterprise-edition). +- You're using GitLab.com. You cannot add a self-managed license to GitLab.com. + To use paid features on GitLab.com, [purchase a separate subscription](../../subscriptions/gitlab_com/index.md). + +### Users exceed license limit upon renewal + +GitLab displays a message prompting you to purchase +additional users. This issue occurs if you add a license that does not have enough +users to cover the number of users in your instance. + +To fix this issue, purchase additional seats to cover those users. +For more information, read the [licensing FAQ](https://about.gitlab.com/pricing/licensing-faq/). + +In GitLab 14.2 and later, for instances that use a license file, the following +rules apply: + +- If the users over license are less than or equal to 10% of the users in the license + file, the license is applied and you pay the overage in the next renewal. +- If the users over license are more than 10% of the users in the license file, + you cannot apply the license without purchasing more users. + +For example, if you purchase a license for 100 users, you can have 110 users when you add +your license. However, if you have 111 users, you must purchase more users before you can add +the license. diff --git a/doc/user/admin_area/reporting/spamcheck.md b/doc/user/admin_area/reporting/spamcheck.md index 02d7cd01139..559235fe322 100644 --- a/doc/user/admin_area/reporting/spamcheck.md +++ b/doc/user/admin_area/reporting/spamcheck.md @@ -1,13 +1,16 @@ --- stage: Enablement group: Distribution -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Spamcheck anti-spam service **(PREMIUM SELF)** +# Spamcheck anti-spam service **(FREE SELF)** > [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6259) in GitLab 14.8. +WARNING: +Spamcheck is available to all tiers, but only on instances using GitLab Enterprise Edition (EE). For [licensing reasons](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/6259#note_726605397), it is not included in the GitLab Community Edition (CE) package. You can [migrate from CE to EE](../../../update/package/convert_to_ee.md). + [Spamcheck](https://gitlab.com/gitlab-org/spamcheck) is an anti-spam engine developed by GitLab originally to combat rising amount of spam in GitLab.com, and later made public to be used in self-managed GitLab instances. @@ -47,7 +50,7 @@ Spamcheck is only available for package-based installations: 1. Select **Save changes**. NOTE: -In single-node instances, Spamcehck runs over `localhost`, and hence is running +In single-node instances, Spamcheck runs over `localhost`, and hence is running in an unauthenticated mode. If on multi-node instances where GitLab runs on one server and Spamcheck runs on another server listening over a public endpoint, it is recommended to enforce some sort of authentication using a reverse proxy in diff --git a/doc/user/admin_area/settings/account_and_limit_settings.md b/doc/user/admin_area/settings/account_and_limit_settings.md index 1d982196228..2f30298644a 100644 --- a/doc/user/admin_area/settings/account_and_limit_settings.md +++ b/doc/user/admin_area/settings/account_and_limit_settings.md @@ -178,14 +178,9 @@ nginx['client_max_body_size'] = "200m" > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/296669) in GitLab 13.9. > - It's deployed behind a feature flag, disabled by default. -> - It's disabled on GitLab.com. -> - It's not recommended for production use. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](../../../security/two_factor_authentication.md#enable-or-disable-2fa-for-git-operations). -NOTE: -This feature is under development and not ready for production use. It is deployed -behind a feature flag that is **disabled by default**. To use it in GitLab -self-managed instances, ask a GitLab administrator to [enable it](../../../security/two_factor_authentication.md#enable-or-disable-2fa-for-git-operations). +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `two_factor_for_cli`. On GitLab.com, this feature is not available. This feature is not ready for production use. This feature flag also affects [2FA for Git over SSH operations](../../../security/two_factor_authentication.md#2fa-for-git-over-ssh-operations). GitLab administrators can choose to customize the session duration (in minutes) for Git operations when 2FA is enabled. The default is 15 and this can be set to a value between 1 and 10080. diff --git a/doc/user/admin_area/settings/continuous_integration.md b/doc/user/admin_area/settings/continuous_integration.md index 18379471bcf..0330d89aedc 100644 --- a/doc/user/admin_area/settings/continuous_integration.md +++ b/doc/user/admin_area/settings/continuous_integration.md @@ -55,31 +55,28 @@ can be set at: - The instance level. - [From GitLab 12.4](https://gitlab.com/gitlab-org/gitlab/-/issues/21688), the project and group level. -The value is: +For the setting on GitLab.com, see [Artifacts maximum size](../../gitlab_com/index.md#gitlab-cicd). -- In *MB* and the default is 100MB per job. -- [Set to 1G](../../gitlab_com/index.md#gitlab-cicd) on GitLab.com. - -To change it at the: +The value is in MB and the default is 100MB per job. To change it at the: - Instance level: 1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD**. 1. Change the value of maximum artifacts size (in MB). - 1. Click **Save changes** for the changes to take effect. + 1. Select **Save changes** for the changes to take effect. - Group level (this overrides the instance setting): 1. Go to the group's **Settings > CI/CD > General Pipelines**. 1. Change the value of **maximum artifacts size (in MB)**. - 1. Click **Save changes** for the changes to take effect. + 1. Select **Save changes** for the changes to take effect. - Project level (this overrides the instance and group settings): 1. Go to the project's **Settings > CI/CD > General Pipelines**. 1. Change the value of **maximum artifacts size (in MB)**. - 1. Click **Save changes** for the changes to take effect. + 1. Select **Save changes** for the changes to take effect. NOTE: The setting at all levels is only available to GitLab administrators. @@ -94,7 +91,7 @@ and the default value is `30 days`. 1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Settings > CI/CD**. 1. Change the value of default expiration time. -1. Click **Save changes** for the changes to take effect. +1. Select **Save changes** for the changes to take effect. This setting is set per job and can be overridden in [`.gitlab-ci.yml`](../../../ci/yaml/index.md#artifactsexpire_in). @@ -126,7 +123,7 @@ To disable the setting: 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Continuous Integration and Deployment**. 1. Clear the **Keep the latest artifacts for all jobs in the latest successful pipelines** checkbox. -1. Click **Save changes** +1. Select **Save changes** When you disable the feature, the latest artifacts do not immediately expire. A new pipeline must run before the latest artifacts can expire and be deleted. @@ -156,7 +153,7 @@ After that time passes, the jobs are archived and no longer able to be retried. Make it empty to never expire jobs. It has to be no less than 1 day, for example: <code>15 days</code>, <code>1 month</code>, <code>2 years</code>. -As of June 22, 2020 the [value is set](../../gitlab_com/index.md#gitlab-cicd) to 3 months on GitLab.com. Jobs created before that date were archived after September 22, 2020. +For the value set for GitLab.com, see [Scheduled job archiving](../../gitlab_com/index.md#gitlab-cicd). ## Protect CI/CD variables by default @@ -233,7 +230,7 @@ To select a CI/CD template for the required pipeline configuration: 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand the **Required pipeline configuration** section. 1. Select a CI/CD template from the dropdown. -1. Click **Save changes**. +1. Select **Save changes**. ## Package Registry configuration @@ -272,7 +269,7 @@ To set the maximum file size: 1. Expand the **Package Registry** section. 1. Find the package type you would like to adjust. 1. Enter the maximum file size, in bytes. -1. Click **Save size limits**. +1. Select **Save size limits**. ## Prevent users from registering runners diff --git a/doc/user/admin_area/settings/gitaly_timeouts.md b/doc/user/admin_area/settings/gitaly_timeouts.md index fac23cb48af..42e0c9faf9f 100644 --- a/doc/user/admin_area/settings/gitaly_timeouts.md +++ b/doc/user/admin_area/settings/gitaly_timeouts.md @@ -22,6 +22,6 @@ The following timeouts are available. | Timeout | Default | Description | |:--------|:-----------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| Default | 55 seconds | Timeout for most Gitaly calls (not enforced for `git` `fetch` and `push` operations, or Sidekiq jobs). For example, checking if a repository exists on disk. Makes sure that Gitaly calls made within a web request cannot exceed the entire request timeout. It should be shorter than the [worker timeout](../../../administration/operations/puma.md#worker-timeout) that can be configured for [Puma](../../../install/requirements.md#puma-settings). If a Gitaly call timeout exceeds the worker timeout, the remaining time from the worker timeout is used to avoid having to terminate the worker. | +| Default | 55 seconds | Timeout for most Gitaly calls (not enforced for `git` `fetch` and `push` operations, or Sidekiq jobs). For example, checking if a repository exists on disk. Makes sure that Gitaly calls made within a web request cannot exceed the entire request timeout. It should be shorter than the [worker timeout](../../../administration/operations/puma.md#change-the-worker-timeout) that can be configured for [Puma](../../../install/requirements.md#puma-settings). If a Gitaly call timeout exceeds the worker timeout, the remaining time from the worker timeout is used to avoid having to terminate the worker. | | Fast | 10 seconds | Timeout for fast Gitaly operations used within requests, sometimes multiple times. For example, checking if a repository exists on disk. If fast operations exceed this threshold, there may be a problem with a storage shard. Failing fast can help maintain the stability of the GitLab instance. | | Medium | 30 seconds | Timeout for Gitaly operations that should be fast (possibly within requests) but preferably not used multiple times within a request. For example, loading blobs. Timeout that should be set between Default and Fast. | diff --git a/doc/user/admin_area/settings/index.md b/doc/user/admin_area/settings/index.md index a581fd4aebc..052b6e26c07 100644 --- a/doc/user/admin_area/settings/index.md +++ b/doc/user/admin_area/settings/index.md @@ -130,6 +130,7 @@ The **Network** settings contain: Git LFS requests that supersede the user and IP rate limits. - [Files API Rate Limits](files_api_rate_limits.md) - Configure specific limits for Files API requests that supersede the user and IP rate limits. + - [Search rate limits](../../../administration/instance_limits.md#search-rate-limit) - Configure global search request rate limits for authenticated and unauthenticated users. - [Deprecated API Rate Limits](deprecated_api_rate_limits.md) - Configure specific limits for deprecated API requests that supersede the user and IP rate limits. - [Outbound requests](../../../security/webhooks.md) - Allow requests to the local network from hooks and services. @@ -170,6 +171,8 @@ The **Repository** settings contain: - [Repository's custom initial branch name](../../project/repository/branches/default.md#instance-level-custom-initial-branch-name) - Set a custom branch name for new repositories created in your instance. +- [Repository's initial default branch protection](../../project/repository/branches/default.md#instance-level-default-branch-protection) - + Configure the branch protections to apply to every repository's default branch. - [Repository mirror](visibility_and_access_controls.md#enable-project-mirroring) - Configure repository mirroring. - [Repository storage](../../../administration/repository_storage_types.md) - Configure storage path settings. diff --git a/doc/user/admin_area/settings/user_and_ip_rate_limits.md b/doc/user/admin_area/settings/user_and_ip_rate_limits.md index 88be73c3215..56e240a8d39 100644 --- a/doc/user/admin_area/settings/user_and_ip_rate_limits.md +++ b/doc/user/admin_area/settings/user_and_ip_rate_limits.md @@ -107,7 +107,7 @@ attached into the response headers. | `RateLimit-Limit` | `60` | The request quota for the client **each minute**. If the rate limit period set in the admin area is different from 1 minute, the value of this header is adjusted to approximately the nearest 60-minute period. | | `RateLimit-Name` | `throttle_authenticated_web` | Name of the throttle blocking the requests. | | `RateLimit-Observed` | `67` | Number of requests associated to the client in the time window. | -| `RateLimit-Remaining` | `0` | Remaining quota in the time window. The result of `RateLimit-Limit` - `RateLimit-Remaining`. | +| `RateLimit-Remaining` | `0` | Remaining quota in the time window. The result of `RateLimit-Limit` - `RateLimit-Observed`. | | `RateLimit-Reset` | `1609844400` | [Unix time](https://en.wikipedia.org/wiki/Unix_time)-formatted time when the request quota is reset. | | `RateLimit-ResetTime` | `Tue, 05 Jan 2021 11:00:00 GMT` | [RFC2616](https://tools.ietf.org/html/rfc2616#section-3.3.1)-formatted date and time when the request quota is reset. | | `Retry-After` | `30` | Remaining duration **in seconds** until the quota is reset. This is a [standard HTTP header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After). | diff --git a/doc/user/admin_area/settings/visibility_and_access_controls.md b/doc/user/admin_area/settings/visibility_and_access_controls.md index c38b2455a8d..2165dc54899 100644 --- a/doc/user/admin_area/settings/visibility_and_access_controls.md +++ b/doc/user/admin_area/settings/visibility_and_access_controls.md @@ -17,54 +17,6 @@ To access the visibility and access control options: 1. On the left sidebar, select **Settings > General**. 1. Expand the **Visibility and access controls** section. -## Protect default branches - -With this option, you can define [branch protections](../../project/protected_branches.md) -to apply to every repository's [default branch](../../project/repository/branches/default.md). -These protections specify the user roles with permission to push to default branches. - -This setting applies only to each repository's default branch. To protect other branches, -you must configure [branch protection in the repository](../../project/protected_branches.md), -or configure [branch protection for groups](../../group/index.md#change-the-default-branch-protection-of-a-group). - -To change the default branch protection for the entire instance: - -1. Sign in to GitLab as a user with Administrator access level. -1. On the top bar, select **Menu > Admin**. -1. On the left sidebar, select **Settings > General**. -1. Expand the **Visibility and access controls** section. -1. Select a **Default branch protection**: - - **Not protected** - Both developers and maintainers can push new commits - and force push. - - **Protected against pushes** - Developers cannot push new commits, but are - allowed to accept merge requests to the branch. Maintainers can push to the branch. - - **Partially protected** - Both developers and maintainers can push new commits, - but cannot force push. - - **Fully protected** - Developers cannot push new commits, but maintainers can. - No one can force push. -1. To allow group owners to override the instance's default branch protection, select - [**Allow owners to manage default branch protection per group**](#prevent-overrides-of-default-branch-protection). -1. Select **Save changes**. - -### Prevent overrides of default branch protection **(PREMIUM SELF)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211944) in GitLab 13.0. - -Instance-level protections for [default branch](../../project/repository/branches/default.md) -can be overridden on a per-group basis by the group's owner. In -[GitLab Premium or higher](https://about.gitlab.com/pricing/), GitLab administrators can -disable this privilege for group owners, enforcing the instance-level protection rule: - -1. Sign in to GitLab as a user with Administrator access level. -1. On the top bar, select **Menu > Admin**. -1. On the left sidebar, select **Settings > General**. -1. Expand the **Visibility and access controls** section. -1. Deselect the **Allow owners to manage default branch protection per group** checkbox. -1. Select **Save changes**. - -NOTE: -GitLab administrators can still update the default branch protection of a group. - ## Define which roles can create projects Instance-level protections for project creation define which roles can diff --git a/doc/user/analytics/ci_cd_analytics.md b/doc/user/analytics/ci_cd_analytics.md index 1bc0c2b8fb0..8e231d18f41 100644 --- a/doc/user/analytics/ci_cd_analytics.md +++ b/doc/user/analytics/ci_cd_analytics.md @@ -4,13 +4,13 @@ group: Release info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# CI/CD Analytics **(FREE)** +# CI/CD analytics **(FREE)** ## Pipeline success and duration charts > [Renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/38318) to CI/CD Analytics in GitLab 12.8. -CI/CD Analytics shows the history of your pipeline successes and failures, as well as how long each pipeline +CI/CD analytics shows the history of your pipeline successes and failures, as well as how long each pipeline ran. View successful pipelines: @@ -39,21 +39,14 @@ To view CI/CD analytics: > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/275991) in GitLab 13.7. > - [Added support](https://gitlab.com/gitlab-org/gitlab/-/issues/291746) for lead time for changes in GitLab 13.10. -Customer experience is a key metric. Users want to measure platform stability and other -post-deployment performance KPIs, and set targets for customer behavior, experience, and financial -impact. Tracking and measuring these indicators solves an important pain point. Similarly, creating -views that manage products, not projects or repositories, provides users with a more relevant data set. -Since GitLab is a tool for the entire DevOps life-cycle, information from different workflows is -integrated and can be used to measure the success of the teams. - The DevOps Research and Assessment ([DORA](https://cloud.google.com/blog/products/devops-sre/the-2019-accelerate-state-of-devops-elite-performance-productivity-and-scaling)) -team developed four key metrics that the industry has widely adopted. You can use these metrics as -performance indicators for software development teams: +team developed several key metrics that you can use as performance indicators for software development +teams: - Deployment frequency: How often an organization successfully releases to production. - Lead time for changes: The amount of time it takes for code to reach production. - Change failure rate: The percentage of deployments that cause a failure in production. -- Time to restore service: How long it takes an organization to recover from a failure in +- Time to restore service: How long it takes for an organization to recover from a failure in production. ### Supported metrics in GitLab @@ -62,39 +55,48 @@ The following table shows the supported metrics, at which level they are support | Metric | Level | API version | Chart (UI) version | Comments | |---------------------------|---------------------|--------------------------------------|---------------------------------------|-----------| -| `deployment_frequency` | Project-level | [13.7+](../../api/dora/metrics.md) | [13.8+](#deployment-frequency-charts) | The [old API endpoint](../../api/dora4_project_analytics.md) was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/323713) in 13.10. | -| `deployment_frequency` | Group-level | [13.10+](../../api/dora/metrics.md) | [13.12+](#deployment-frequency-charts) | | -| `lead_time_for_changes` | Project-level | [13.10+](../../api/dora/metrics.md) | [13.11+](#lead-time-charts) | Unit in seconds. Aggregation method is median. | -| `lead_time_for_changes` | Group-level | [13.10+](../../api/dora/metrics.md) | [14.0+](#lead-time-charts) | Unit in seconds. Aggregation method is median. | +| `deployment_frequency` | Project-level | [13.7+](../../api/dora/metrics.md) | [13.8+](#view-deployment-frequency-chart) | The [old API endpoint](../../api/dora4_project_analytics.md) was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/323713) in 13.10. | +| `deployment_frequency` | Group-level | [13.10+](../../api/dora/metrics.md) | [13.12+](#view-deployment-frequency-chart) | | +| `lead_time_for_changes` | Project-level | [13.10+](../../api/dora/metrics.md) | [13.11+](#view-lead-time-for-changes-chart) | Unit in seconds. Aggregation method is median. | +| `lead_time_for_changes` | Group-level | [13.10+](../../api/dora/metrics.md) | [14.0+](#view-lead-time-for-changes-chart) | Unit in seconds. Aggregation method is median. | | `change_failure_rate` | Project/Group-level | To be supported | To be supported | | | `time_to_restore_service` | Project/Group-level | To be supported | To be supported | | -### Deployment frequency charts +## View deployment frequency chart **(ULTIMATE)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/275991) in GitLab 13.8. -The **Analytics > CI/CD Analytics** page shows information about the deployment +The deployment frequency charts show information about the deployment frequency to the `production` environment. The environment must be part of the [production deployment tier](../../ci/environments/index.md#deployment-tier-of-environments) for its deployment information to appear on the graphs. - +The deployment frequency chart is available for groups and projects. + +To view the deployment frequency chart: -These charts are available for both groups and projects. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Analytics > CI/CD Analytics**. +1. Select the **Deployment frequency** tab. -### Lead time charts + + +## View lead time for changes chart **(ULTIMATE)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/250329) in GitLab 13.11. -The charts in the **Lead Time** tab show information about how long it takes -merge requests to be deployed to a production environment. +The lead time for changes chart shows information about how long it takes for +merge requests to be deployed to a production environment. This chart is available for groups and projects. - +- Small lead times indicate fast, efficient deployment + processes. +- For time periods in which no merge requests were deployed, the charts render a + red, dashed line. -Smaller values are better. Small lead times indicate fast, efficient deployment -processes. +To view the lead time for changes chart: -For time periods in which no merge requests were deployed, the charts render a -red, dashed line. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Analytics > CI/CD Analytics**. +1. Select the **Lead time** tab. -These charts are available for both groups and projects. + diff --git a/doc/user/analytics/code_review_analytics.md b/doc/user/analytics/code_review_analytics.md index 18a6ca20bc7..dc02512702a 100644 --- a/doc/user/analytics/code_review_analytics.md +++ b/doc/user/analytics/code_review_analytics.md @@ -40,7 +40,7 @@ To view Code Review Analytics: ## Use cases -This feature is designed for [development team leaders](https://about.gitlab.com/handbook/marketing/strategic-marketing/roles-personas/#delaney-development-team-lead) +This feature is designed for [development team leaders](https://about.gitlab.com/handbook/product/personas/#delaney-development-team-lead) and others who want to understand broad code review dynamics, and identify patterns to explain them. You can use Code Review Analytics to: diff --git a/doc/user/analytics/img/mr_throughput_chart_v13_3.png b/doc/user/analytics/img/mr_throughput_chart_v13_3.png Binary files differdeleted file mode 100644 index 100c9a8557c..00000000000 --- a/doc/user/analytics/img/mr_throughput_chart_v13_3.png +++ /dev/null diff --git a/doc/user/analytics/img/project_vsa_filter_v14_3.png b/doc/user/analytics/img/project_vsa_filter_v14_3.png Binary files differdeleted file mode 100644 index d3fcad31909..00000000000 --- a/doc/user/analytics/img/project_vsa_filter_v14_3.png +++ /dev/null diff --git a/doc/user/analytics/img/project_vsa_stage_table_v14_4.png b/doc/user/analytics/img/project_vsa_stage_table_v14_4.png Binary files differdeleted file mode 100644 index e65296062f6..00000000000 --- a/doc/user/analytics/img/project_vsa_stage_table_v14_4.png +++ /dev/null diff --git a/doc/user/analytics/merge_request_analytics.md b/doc/user/analytics/merge_request_analytics.md index f9ca06c0ef9..06774c3f16a 100644 --- a/doc/user/analytics/merge_request_analytics.md +++ b/doc/user/analytics/merge_request_analytics.md @@ -40,7 +40,7 @@ To view the number of merge requests merged per month: 1. On the top bar, select **Menu > Projects** and find your project. 1. On the left sidebar, select **Analytics > Merge request**. -1. Optional. Filter results: +1. Optional. Filter results: 1. Select the filter bar. 1. Select a parameter. 1. Select a value or enter text to refine the results. diff --git a/doc/user/analytics/value_stream_analytics.md b/doc/user/analytics/value_stream_analytics.md index 6bad374c371..92c4d447ed9 100644 --- a/doc/user/analytics/value_stream_analytics.md +++ b/doc/user/analytics/value_stream_analytics.md @@ -10,215 +10,197 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12077) in GitLab Premium 12.3 at the group level. > - [Renamed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/23427) from cycle analytics to value stream analytics in GitLab 12.8. -Value stream analytics measures the time spent to go from an -[idea to production](https://about.gitlab.com/blog/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#from-idea-to-production-with-gitlab) -(also known as cycle time) for each of your projects or groups. Value stream analytics displays the median time -spent in each stage defined in the process. +Value stream analytics provides metrics about each stage of your software development process. -You can use value stream analytics to determine the velocity of a given -project. It points to bottlenecks in the development process, enabling management -to uncover, triage, and identify the root cause of slowdowns in the software development life cycle. +Use value stream analytics to identify: -For information about how to contribute to the development of value stream analytics, see our [contributor documentation](../../development/value_stream_analytics.md). +- The amount of time it takes to go from an idea to production. +- The velocity of a given project. +- Bottlenecks in the development process. +- Factors that cause your software development lifecycle to slow down. -To access value stream analytics for a project: +Value stream analytics is also available for [groups](../group/value_stream_analytics). -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Analytics > Value stream**. - -NOTE: -[Value stream analytics for groups](../group/value_stream_analytics) is also available. - -## Default stages +## View value stream analytics -The stages tracked by value stream analytics by default represent the [GitLab flow](../../topics/gitlab_flow.md). You can customize these stages in value stream analytics for groups. +> - Filtering [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326701) in GitLab 14.3 +> - Sorting [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335974) in GitLab 14.4. -- **Issue** (Tracker) - - Time to schedule an issue (by milestone or by adding it to an issue board) -- **Plan** (Board) - - Time to first commit -- **Code** (IDE) - - Time to create a merge request -- **Test** (CI) - - Time it takes GitLab CI/CD to test your code -- **Review** (Merge request) - - Time spent on code review -- **Staging** (Continuous Deployment) - - Time between merging and deploying to production +To view value stream analytics for your project: -## Filter value stream analytics data +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Analytics > Value stream**. +1. To view metrics for each stage, above the **Filter results** text box, select a stage. +1. Optional. Filter the results: + 1. Select the **Filter results** text box. + 1. Select a parameter. + 1. Select a value or enter text to refine the results. + 1. To adjust the date range: + - In the **From** field, select a start date. + - In the **To** field, select an end date. +1. Optional. Sort results by ascending or descending: + - To sort by most recent or oldest workflow item, select the **Merge requests** or **Issues** + header. The header name differs based on the stage you select. + - To sort by most or least amount of time spent in each stage, select the **Time** header. + +The table shows a list of related workflow items for the selected stage. Based on the stage you choose, this can be: -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326701) in GitLab 14.3 +- CI/CD jobs +- Issues +- Merge requests +- Pipelines -You can filter analytics based on the following parameters: +A badge next to the workflow items table header shows the number of workflow items that completed the selected stage. -- Milestones (Group level) -- Labels (Group level) -- Author -- Assignees +## View time spent in each development stage -To filter results: +Value stream analytics shows the median time spent by issues or merge requests in each development stage. -1. Select the **Filter results** text box. -1. Select a parameter. -1. Select a value. To find a value in the list, enter the value name. +To view the median time spent in each stage: - +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Analytics > Value stream**. +1. Optional. Filter the results: + 1. Select the **Filter results** text box. + 1. Select a parameter. + 1. Select a value or enter text to refine the results. + 1. To adjust the date range: + - In the **From** field, select a start date. + - In the **To** field, select an end date. +1. To view the median time for each stage, above the **Filter results** text box, point to a stage. -### Date ranges +## View the lead time and cycle time for issues -To filter analytics results based on a date range, -select different **From** and **To** days -from the date picker (default: last 30 days). +Value stream analytics shows the lead time and cycle time for issues in your project: -### Stage table +- Lead time: Median time from when the issue was created to when it was closed. +- Cycle time: Median time from first commit to issue closed. Commits are associated with issues when users [cross-link them in the commit message](../project/issues/crosslinking_issues.md#from-commit-messages). -> Sorting the stage table [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335974) in GitLab 14.4. +To view the lead time and cycle time for issues: - +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Analytics > Value stream**. +1. Optional. Filter the results: + 1. Select the **Filter results** text box. + 1. Select a parameter. + 1. Select a value or enter text to refine the results. + 1. To adjust the date range: + - In the **From** field, select a start date. + - In the **To** field, select an end date. -The stage table shows a list of related workflow items for the selected stage. This can include: +The **Lead Time** and **Cycle Time** metrics display below the **Filter results** text box. -- CI/CD jobs -- Issues -- Merge requests -- Pipelines +## View lead time for changes for merge requests **(ULTIMATE)** -A little badge next to the workflow items table header shows the number of workflow items that -completed the selected stage. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340150) in GitLab 14.5. -The stage table also includes the **Time** column, which shows how long it takes each item to pass -through the selected value stream stage. +Lead time for changes is the median duration between when a merge request is merged and when it's deployed to production. -To sort the stage table by a table column, select the table header. -You can sort in ascending or descending order. To find items that spent the most time in a stage, -potentially causing bottlenecks in your value stream, sort the table by the **Time** column. -From there, select individual items to drill in and investigate how delays are happening. -To see which items most recently exited the stage, sort by the work item column on the left. +To view the lead time for changes for merge requests in your project: -The table displays 20 items per page. If there are more than 20 items, you can use the -**Prev** and **Next** buttons to navigate through the pages. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Analytics > Value stream**. +1. Optional. Filter the results: + 1. Select the **Filter results** text box. + 1. Select a parameter. + 1. Select a value or enter text to refine the results. + 1. To adjust the date range: + - In the **From** field, select a start date. + - In the **To** field, select an end date. -## How Time metrics are measured +The **Lead Time for Changes** metrics display below the **Filter results** text box. -The **Time** metrics near the top of the page are measured as follows: +## View number of successful deployments **(PREMIUM)** -- **Lead time**: Median time from issue created to issue closed. -- **Cycle time**: Median time from first commit to issue closed. (You can associate a commit with an issue by [crosslinking in the commit message](../project/issues/crosslinking_issues.md#from-commit-messages).) -- **Lead Time for Changes**: median duration between merge request merge and deployment to a production environment for all MRs deployed in the given time period. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340150) in GitLab 14.5 (Ultimate only). +To view deployment metrics, you must have a +[production environment configured](../../ci/environments/index.md#deployment-tier-of-environments). -## Deployment metrics (**PREMIUM**) +Value stream analytics shows the following deployment metrics for your project: -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/337256) in GitLab 11.3. +- Deploys: The number of successful deployments in the date range. +- Deployment Frequency: The average number of successful deployments per day in the date range. -Value stream analytics exposes two deployment related metrics near the top of the page: +To view deployment metrics for your project: -- **Deploys:** The number of successful deployments in the date range. -- **Deployment Frequency:** The average number of successful deployments. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Analytics > Value stream**. +1. Optional. Filter the results: + 1. Select the **Filter results** text box. + 1. Select a parameter. + 1. Select a value or enter text to refine the results. + 1. To adjust the date range: + - In the **From** field, select a start date. + - In the **To** field, select an end date. -The deployment metrics calculation uses the same method as the -[value stream analytics for groups](../group/value_stream_analytics/index.md#how-metrics-are-measured). -Both of them are based on the [DORA API](../../api/dora/metrics.md#devops-research-and-assessment-dora-key-metrics-api). +The **Deploys** and **Deployment Frequency** metrics display below the **Filter results** text box. -## How the stages are measured +Deployment metrics are calculated based on data from the +[DORA API](../../api/dora/metrics.md#devops-research-and-assessment-dora-key-metrics-api). -Value stream analytics uses start events and end events to measure the time that an issue or merge request spends in each stage. -For example, a stage might start when one label is added to an issue and end when another label is added. -Items aren't included in the stage time calculation if they have not reached the end event. +NOTE: +In GitLab 13.9 and later, metrics are calculated based on when the deployment was finished. +In GitLab 13.8 and earlier, metrics are calculated based on when the deployment was created. -| Stage | Description | -|---------|---------------| -| Issue | Measures the median time between creating an issue and taking action to solve it, by either labeling it or adding it to a milestone, whichever comes first. The label is tracked only if it already includes an [issue board list](../project/issue_board.md) created for it. | -| Plan | Measures the median time between the action you took for the previous stage, and pushing the first commit to the branch. That first branch commit triggers the separation between **Plan** and **Code**, and at least one of the commits in the branch must include the related issue number (such as `#42`). If the issue number is *not* included in a commit, that data is not included in the measurement time of the stage. | -| Code | Measures the median time between pushing a first commit (previous stage) and creating a merge request (MR). The process is tracked with the [issue closing pattern](../project/issues/managing_issues.md#closing-issues-automatically) in the description of the merge request. For example, if the issue is closed with `Closes #xxx`, it's assumed that `xxx` is issue number for the merge request). If there is no closing pattern, the start time is set to the create time of the first commit. | -| Test | Essentially the start to finish time for all pipelines. Measures the median time to run the entire pipeline for that project. Related to the time required by GitLab CI/CD to run every job for the commits pushed to that merge request, as defined in the previous stage. | -| Review | Measures the median time taken to review merge requests with a closing issue pattern, from creation to merge. | -| Staging | Measures the median time between merging the merge request (with a closing issue pattern) to the first deployment to a [production environment](#how-the-production-environment-is-identified). Data not collected without a production environment. | +## Access permissions for value stream analytics -How this works: +Access permissions for value stream analytics depend on the project type. -1. Issues and merge requests are grouped in pairs, where the merge request has the - [closing pattern](../project/issues/managing_issues.md#closing-issues-automatically) - for the corresponding issue. Issue and merge request pairs without closing patterns are - not included. -1. Issue and merge request pairs are filtered by the last XX days, specified through the UI - (default is `90` days). Pairs outside the filtered range are not included. -1. For the remaining pairs, review information needed for stages, including - issue creation date and merge request merge time. +| Project type | Permissions | +|--------------|----------------------------------------| +| Public | Anyone can access. | +| Internal | Any authenticated user can access. | +| Private | Any member Guest and above can access. | -In short, the value stream analytics dashboard tracks data related to [GitLab flow](../../topics/gitlab_flow.md). It does not include data for: +## How value stream analytics measures each stage -- Merge requests that do not close an issue. -- Issues that do not include labels present in the issue board. -- Issues without a milestone. -- Staging stages, in projects without a [production environment](#how-the-production-environment-is-identified). +Value stream analytics uses start and end events to measure the time that an issue or merge request +spends in each stage. -## How the production environment is identified +For example, a stage might start when a user adds a label to an issue, and ends when they add another label. +Items aren't included in the stage time calculation if they have not reached the end event. -Value stream analytics identifies production environments based on the -[deployment tier of environments](../../ci/environments/index.md#deployment-tier-of-environments). +| Stage | Measurement method | +|---------|----------------------| +| Issue | The median time between creating an issue and taking action to solve it, by either labeling it or adding it to a milestone. The label is tracked only if it already includes an [issue board list](../project/issue_board.md) that has been created for the label. | +| Plan | The median time between the action you took for the previous stage, and when you push the first commit to the branch. The first branch commit triggers the transition from **Plan** to **Code**, and at least one of the commits in the branch must include the related issue number (such as `#42`). If the issue number is not included in a commit, that data is not included in the measurement time of the stage. | +| Code | The median time between pushing a first commit (previous stage) and creating a merge request. The process is tracked with the [issue closing pattern](../project/issues/managing_issues.md#closing-issues-automatically) in the description of the merge request. For example, if the issue is closed with `Closes #xxx`, `xxx` is the issue number for the merge request. If there is no closing pattern, the start time is set to the create time of the first commit. | +| Test | The time from start to finish for all pipelines. Measures the median time to run the entire pipeline for that project. Related to the time required by GitLab CI/CD to run every job for the commits pushed to that merge request, as defined in the previous stage. | +| Review | The median time taken to review merge requests with a closing issue pattern, from creation to merge. | +| Staging | The median time between merging the merge request (with a closing issue pattern) to the first deployment to a [production environment](../../ci/environments/index.md#deployment-tier-of-environments). Data is not collected without a production environment. | ## Example workflow -Here's a fictional workflow of a single cycle that happens in a -single day, passing through all seven stages. If a stage doesn't have -a start and a stop mark, it isn't measured and hence isn't calculated in the median -time. It's assumed that milestones are created, and CI for testing and setting -environments is configured. - -1. Issue is created at 09:00 (start of **Issue** stage). -1. Issue is added to a milestone at 11:00 (stop of **Issue** stage and start of - **Plan** stage). -1. Start working on the issue, create a branch locally, and make one commit at - 12:00. -1. Make a second commit to the branch that mentions the issue number at 12:30 - (stop of **Plan** stage and start of **Code** stage). -1. Push branch, and create a merge request that contains the [issue closing pattern](../project/issues/managing_issues.md#closing-issues-automatically) - in its description at 14:00 (stop of **Code** stage and start of **Test** and - **Review** stages). -1. The CI starts running your scripts defined in [`.gitlab-ci.yml`](../../ci/yaml/index.md) and - takes 5 minutes (stop of **Test** stage). -1. Review merge request, ensure that everything is okay, and then merge the merge - request at 19:00 (stop of **Review** stage and start of **Staging** stage). -1. The merge request is merged, and a deployment to the `production` - environment starts and finishes at 19:30 (stop of **Staging** stage). - -From the previous example we see the time used for each stage: - -- **Issue**: 2 hrs (09:00 to 11:00) -- **Plan**: 1 hr (11:00 to 12:00) -- **Code**: 2 hrs (12:00 to 14:00) +This example shows a workflow through all seven stages in one day. In this +example, milestones have been created and CI for testing and setting environments is configured. + +- 09:00: Create issue. **Issue** stage starts. +- 11:00: Add issue to a milestone, start work on the issue, and create a branch locally. +**Issue** stage stops and **Plan** stage starts. +- 12:00: Make the first commit. +- 12:30: Make the second commit to the branch that mentions the issue number. **Plan** stage stops and **Code** stage starts. +- 14:00: Push branch and create a merge request that contains the [issue closing pattern](../project/issues/managing_issues.md#closing-issues-automatically). **Code** stage stops and **Test** and **Review** stages start. +- The CI takes 5 minutes to run scripts defined in [`.gitlab-ci.yml`](../../ci/yaml/index.md). +**Test** stage stops. +- Review merge request. +- 19:00: Merge the merge request. **Review** stage stops and **Staging** stage starts. +- 19:30: Deployment to the `production` environment starts and finishes. **Staging** stops. + +Value stream analytics records the following times for each stage: + +- **Issue**: 09:00 to 11:00: 2 hrs +- **Plan**: 11:00 to 12:00: 1 hr +- **Code**: 12:00 to 14:00: 2 hrs - **Test**: 5 minutes -- **Review**: 5 hrs (14:00 to 19:00) -- **Staging**: 30 minutes (19:00 to 19:30) - -More information: - -- Although the previous example specifies the issue number in a later commit, the process - still collects analytics data for the issue. -- The time required in the **Test** stage isn't included in the overall time of - the cycle. The time is included in the **Review** process, as every merge request should be - tested. -- The previous example illustrates only one cycle of the multiple stages. Value - stream analytics, on its dashboard, shows the calculated median elapsed time - for these issues. - -## Permissions - -The permissions for the value stream analytics for projects dashboard include: - -| Project type | Permissions | -|--------------|---------------------------------------| -| Public | Anyone can access | -| Internal | Any authenticated user can access | -| Private | Any member Guest and above can access | - -You can [read more about permissions](../../user/permissions.md) in general. - -## More resources - -Learn more about value stream analytics with the following resources: - -- [Value stream analytics feature page](https://about.gitlab.com/stages-devops-lifecycle/value-stream-analytics/). -- [Value stream analytics feature preview](https://about.gitlab.com/blog/2016/09/16/feature-preview-introducing-cycle-analytics/). -- [Value stream analytics feature highlight](https://about.gitlab.com/blog/2016/09/21/cycle-analytics-feature-highlight/). +- **Review**: 14:00 to 19:00: 5 hrs +- **Staging**: 19:00 to 19:30: 30 minutes + +There are some additional considerations for this example: + +- Although this example specifies the issue number in a later commit, the process +still collects analytics data for the issue. +- The time required in the **Test** stage is included in the **Review** process, +as every merge request should be tested. +- This example illustrates only one cycle of multiple stages. The value +stream analytics dashboard shows the calculated median elapsed time for these issues. +- Value stream analytics identifies production environments based on the +[deployment tier of environments](../../ci/environments/index.md#deployment-tier-of-environments). diff --git a/doc/user/application_security/api_fuzzing/create_har_files.md b/doc/user/application_security/api_fuzzing/create_har_files.md index db0b2a32bcf..1ba19359fde 100644 --- a/doc/user/application_security/api_fuzzing/create_har_files.md +++ b/doc/user/application_security/api_fuzzing/create_har_files.md @@ -1,7 +1,7 @@ --- stage: Secure group: Dynamic Analysis -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments type: howto --- diff --git a/doc/user/application_security/api_fuzzing/index.md b/doc/user/application_security/api_fuzzing/index.md index 4eb721f8832..5413c28912a 100644 --- a/doc/user/application_security/api_fuzzing/index.md +++ b/doc/user/application_security/api_fuzzing/index.md @@ -87,9 +87,6 @@ In GitLab 14.0 and later, API fuzzing configuration files must be in your reposi > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/299234) in GitLab 13.10. -WARNING: -This feature might not be available to you. Check the **version history** note above for details. - The API fuzzing configuration form helps you create or modify your project's API fuzzing configuration. The form lets you choose values for the most common API fuzzing options and builds a YAML snippet that you can paste in your GitLab CI/CD configuration. @@ -804,7 +801,7 @@ variables: If the value must be generated or regenerated on expiration, you can provide a program or script for the API fuzzer to execute on a specified interval. The provided script runs in an Alpine Linux -container that has Python 3 and Bash installed. +container that has Python 3 and Bash installed. You have to set the environment variable `FUZZAPI_OVERRIDES_CMD` to the program or script you would like to execute. The provided command creates the overrides JSON file as defined previously. @@ -813,7 +810,7 @@ You might want to install other scripting runtimes like NodeJS or Ruby, or maybe your overrides command. In this case, we recommend setting the `FUZZAPI_PRE_SCRIPT` to the file path of a script which provides those prerequisites. The script provided by `FUZZAPI_PRE_SCRIPT` is executed once, before the analyzer starts. -See the [Alpine Linux package management](https://wiki.alpinelinux.org/wiki/Alpine_Linux_package_management) +See the [Alpine Linux package management](https://wiki.alpinelinux.org/wiki/Alpine_Linux_package_management) page for information about installing Alpine Linux packages. You must provide three CI/CD variables, each set for correct operation: diff --git a/doc/user/application_security/cluster_image_scanning/index.md b/doc/user/application_security/cluster_image_scanning/index.md index 0db9af7a0d3..293645b8de6 100644 --- a/doc/user/application_security/cluster_image_scanning/index.md +++ b/doc/user/application_security/cluster_image_scanning/index.md @@ -29,7 +29,7 @@ To integrate GitLab with security scanners other than those listed here, see You can use cluster image scanning through the following methods: - [The cluster image scanning analyzer](#use-the-cluster-image-scanning-analyzer) -- [The GitLab Agent](#cluster-image-scanning-with-the-gitlab-agent) +- [The GitLab agent](#cluster-image-scanning-with-the-gitlab-agent) ## Use the cluster image scanning analyzer @@ -46,7 +46,7 @@ To enable cluster image scanning in your pipeline, you need the following: - [GitLab Runner](https://docs.gitlab.com/runner/) with the [`docker`](https://docs.gitlab.com/runner/executors/docker.html) or [`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) - executor. + executor on Linux/amd64. - Docker `18.09.03` or later installed on the same computer as the runner. If you're using the shared runners on GitLab.com, then this is already the case. - [Starboard Operator](https://aquasecurity.github.io/starboard/v0.10.3/operator/installation/kubectl/) @@ -277,22 +277,22 @@ Here's an example cluster image scanning report: } ``` -## Cluster image scanning with the GitLab Agent +## Cluster image scanning with the GitLab agent -You can use the [GitLab Agent](../../clusters/agent/index.md) to +You can use the [GitLab agent](../../clusters/agent/index.md) to scan images from within your Kubernetes cluster and record the vulnerabilities in GitLab. ### Prerequisites - [Starboard Operator](https://aquasecurity.github.io/starboard/v0.10.3/operator/installation/kubectl/) installed and configured in your cluster. -- [GitLab Agent](../../clusters/agent/install/index.md) +- [GitLab agent](../../clusters/agent/install/index.md) set up in GitLab, installed in your cluster, and configured using a configuration repository. ### Configuration -The Agent runs the cluster image scanning once the `cluster_image_scanning` -directive is added to your [Agent's configuration repository](../../clusters/agent/repository.md#scan-your-container-images-for-vulnerabilities). +The agent runs the cluster image scanning once the `cluster_image_scanning` +directive is added to your [agent's configuration repository](../../clusters/agent/vulnerabilities.md). ## Security Dashboard @@ -302,7 +302,7 @@ the security vulnerabilities in your groups, projects, and pipelines. ## Interacting with the vulnerabilities After you find a vulnerability, you can address it in the [vulnerability report](../vulnerabilities/index.md) -or the [GitLab Agent's](../../clusters/agent/install/index.md#view-vulnerabilities-in-cluster-images) +or the [GitLab agent's](../../clusters/agent/vulnerabilities.md) details section. ## Troubleshooting diff --git a/doc/user/application_security/configuration/index.md b/doc/user/application_security/configuration/index.md index 430f8e1a2a2..61a2121b9c6 100644 --- a/doc/user/application_security/configuration/index.md +++ b/doc/user/application_security/configuration/index.md @@ -49,7 +49,9 @@ You can configure the following security controls: - Select **Configure with a merge request** to create a merge request with the changes required to enable Dependency Scanning. For more details, see [Enable Dependency Scanning via an automatic merge request](../dependency_scanning/index.md#enable-dependency-scanning-via-an-automatic-merge-request). - [Container Scanning](../container_scanning/index.md) - - Can be configured with `.gitlab-ci.yml`. For more details, read [Container Scanning](../../../user/application_security/container_scanning/index.md#configuration). + - Select **Configure with a merge request** to create a merge request with the changes required to + enable Container Scanning. For more details, see + [Enable Container Scanning through an automatic merge request](../container_scanning/index.md#enable-container-scanning-through-an-automatic-merge-request). - [Cluster Image Scanning](../cluster_image_scanning/index.md) - Can be configured with `.gitlab-ci.yml`. For more details, read [Cluster Image Scanning](../../../user/application_security/cluster_image_scanning/#configuration). - [Secret Detection](../secret_detection/index.md) @@ -66,3 +68,6 @@ You can configure the following security controls: - [License Compliance](../../../user/compliance/license_compliance/index.md) - Can be configured with `.gitlab-ci.yml`. For more details, read [License Compliance](../../../user/compliance/license_compliance/index.md#enable-license-compliance). + +- [Security Training](../../../user/application_security/vulnerabilities/index.md#enable-security-training-for-vulnerabilities) + - Enable **Security training** for the current project. For more details, read [security training](../../../user/application_security/vulnerabilities/index.md#enable-security-training-for-vulnerabilities). diff --git a/doc/user/application_security/container_scanning/index.md b/doc/user/application_security/container_scanning/index.md index 08a8c46cc72..f2d6cef669d 100644 --- a/doc/user/application_security/container_scanning/index.md +++ b/doc/user/application_security/container_scanning/index.md @@ -50,7 +50,7 @@ To enable container scanning in your pipeline, you need the following: - Container Scanning runs in the `test` stage, which is available by default. If you redefine the stages in the `.gitlab-ci.yml` file, the `test` stage is required. - [GitLab Runner](https://docs.gitlab.com/runner/) with the [`docker`](https://docs.gitlab.com/runner/executors/docker.html) - or [`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor. + or [`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor on Linux/amd64. - Docker `18.09.03` or higher installed on the same computer as the runner. If you're using the shared runners on GitLab.com, then this is already the case. - An image matching the [supported distributions](#supported-distributions). @@ -145,7 +145,7 @@ For example, to scan an image from AWS Elastic Container Registry: ```yaml container_scanning: before_script: - - curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" --output "awscliv2.zip" + - ruby -r open-uri -e "IO.copy_stream(URI.open('https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip'), 'awscliv2.zip')" - unzip awscliv2.zip - ./aws/install - aws --version @@ -253,6 +253,24 @@ images. To configure the images, set the `CS_ANALYZER_IMAGE` variable to the sta | Grype | `registry.gitlab.com/security-products/container-scanning/grype:4-ubi` | | Trivy | `registry.gitlab.com/security-products/container-scanning/trivy:4-ubi` | +### Enable Container Scanning through an automatic merge request + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6334) in GitLab 14.9. + +To enable Container Scanning in a project, create a merge request from the Security Configuration +page: + +1. In the project where you want to enable Container Scanning, go to + **Security & Compliance > Configuration**. +1. In the **Container Scanning** row, select **Configure with a merge request**. + +This automatically creates a merge request with the changes necessary to enable Container Scanning. +To complete the configuration, review and merge this merge request. + +The configuration tool works best with no existing `.gitlab-ci.yml` file, or with a minimal +configuration file. If you have a complex GitLab configuration file, it may not be parsed +successfully and an error may occur. + ### Overriding the container scanning template If you want to override the job definition (for example, to change properties like `variables`), you diff --git a/doc/user/application_security/coverage_fuzzing/index.md b/doc/user/application_security/coverage_fuzzing/index.md index 290d4a06dcc..14e98766f0f 100644 --- a/doc/user/application_security/coverage_fuzzing/index.md +++ b/doc/user/application_security/coverage_fuzzing/index.md @@ -121,7 +121,7 @@ Use the following variables to configure coverage-guided fuzz testing in your CI | `COVFUZZ_URL_PREFIX` | Path to the `gitlab-cov-fuzz` repository cloned for use with an offline environment. You should only change this value when using an offline environment. Default: `https://gitlab.com/gitlab-org/security-products/analyzers/gitlab-cov-fuzz/-/raw`. | | `COVFUZZ_USE_REGISTRY` | Set to `true` to have the corpus stored in the GitLab corpus registry. The variables `COVFUZZ_CORPUS_NAME` and `COVFUZZ_GITLAB_TOKEN` are required if this variable is set to `true`. Default: `false`. [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5017) in GitLab 14.8. | | `COVFUZZ_CORPUS_NAME` | Name of the corpus to be used in the job. [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5017) in GitLab 14.8. | -| `COVFUZZ_GITLAB_TOKEN` | Environment variable configured with [Personal Access Token](../../../user/profile/personal_access_tokens.md#create-a-personal-access-token) with API read/write access. [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5017) in GitLab 14.8. | +| `COVFUZZ_GITLAB_TOKEN` | Environment variable configured with [Personal Access Token](../../../user/profile/personal_access_tokens.md#create-a-personal-access-token) or [Project Access Token](../../../user/project/settings/project_access_tokens.md#create-a-project-access-token) with API read/write access. [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5017) in GitLab 14.8. | #### Seed corpus @@ -144,12 +144,8 @@ You can download the JSON report file from the CI/CD pipelines page. For more in ## Corpus registry -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5017) in GitLab 14.8. - -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, ask an -administrator to [disable the feature flags](../../../administration/feature_flags.md) named -`corpus_management` and `corpus_management_ui`. On GitLab.com, this feature is available. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5017) in GitLab 14.8. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/347187) in GitLab 14.9. [Feature flags `corpus_management` and `corpus_management_ui`](https://gitlab.com/gitlab-org/gitlab/-/issues/328418) removed. The corpus registry is a library of corpuses. Corpuses in a project's registry are available to all jobs in that project. A project-wide registry is a more efficient way to manage corpuses than diff --git a/doc/user/application_security/dast/checks/200.1.md b/doc/user/application_security/dast/checks/200.1.md index 98a482b4a0f..9795ad11b0b 100644 --- a/doc/user/application_security/dast/checks/200.1.md +++ b/doc/user/application_security/dast/checks/200.1.md @@ -8,13 +8,13 @@ info: To determine the technical writer assigned to the Stage/Group associated w ## Description -A private RFC 1918 was identified in the target application. Public facing websites should not be issuing -requests to private IP Addresses. Attackers attempting to execute subsequent attacks, such as Server-Side +A private RFC 1918 was identified in the target application. Public facing websites should not be issuing +requests to private IP Addresses. Attackers attempting to execute subsequent attacks, such as Server-Side Request Forgery (SSRF), may be able to use this information to identify additional internal targets. ## Remediation -Identify the resource that is incorrectly specifying an internal IP address and replace it with it's public +Identify the resource that is incorrectly specifying an internal IP address and replace it with it's public facing version, or remove the reference from the target application. ## Details diff --git a/doc/user/application_security/dast/checks/548.1.md b/doc/user/application_security/dast/checks/548.1.md index 94f747739c5..d6371c5491d 100644 --- a/doc/user/application_security/dast/checks/548.1.md +++ b/doc/user/application_security/dast/checks/548.1.md @@ -8,8 +8,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w ## Description -The target web server is configured to list the contents of directories that do not contain an index file -such as `index.html`. This could lead to accidental exposure of sensitive information, or give an attacker +The target web server is configured to list the contents of directories that do not contain an index file +such as `index.html`. This could lead to accidental exposure of sensitive information, or give an attacker details on how filenames and directories are structured and stored. ## Remediation @@ -17,11 +17,11 @@ details on how filenames and directories are structured and stored. Directory indexing should be disabled. Apache: -For Apache based web sites, ensure all `<Directory>` definitions have `Options -Indexes` configured in the +For Apache based web sites, ensure all `<Directory>` definitions have `Options -Indexes` configured in the `apache2.conf` or `httpd.conf` configuration file. NGINX: -For NGINX based websites, ensure all `location` definitions have the `autoindex off` directive set in the +For NGINX based websites, ensure all `location` definitions have the `autoindex off` directive set in the `nginx.conf` file. IIS: diff --git a/doc/user/application_security/dast/checks/598.1.md b/doc/user/application_security/dast/checks/598.1.md new file mode 100644 index 00000000000..817e20ec413 --- /dev/null +++ b/doc/user/application_security/dast/checks/598.1.md @@ -0,0 +1,31 @@ +--- +stage: Secure +group: Dynamic Analysis +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Use of GET request method with sensitive query strings (session ID) + +## Description + +A session ID was identified in the request URL as well as a cookie value. Session +IDs should not be sent in GET requests as they maybe captured by proxy systems, stored in +browser history, or stored in log files. If an attacker were to get access to the session +ID they would potentially be able to gain access to the target account. + +## Remediation + +As request headers are rarely logged or captured by third party systems, ensure session ID +values are only sent in cookies (assigned via `Set-Cookie` response headers) and never sent +in the request URL. + +## Details + +| ID | Aggregated | CWE | Type | Risk | +|:---|:--------|:--------|:--------|:--------| +| 598.1 | true | 598 | Passive | Medium | + +## Links + +- [OWASP](https://owasp.org/www-community/vulnerabilities/Information_exposure_through_query_strings_in_url) +- [CWE](https://cwe.mitre.org/data/definitions/598.html) diff --git a/doc/user/application_security/dast/checks/index.md b/doc/user/application_security/dast/checks/index.md index 97224554723..435bc28c4aa 100644 --- a/doc/user/application_security/dast/checks/index.md +++ b/doc/user/application_security/dast/checks/index.md @@ -19,5 +19,6 @@ The [DAST browser-based crawler](../browser_based.md) provides a number of vulne | [16.6](16.6.md) | AspNetMvc header exposes version information | Low | Passive | | [200.1](200.1.md) | Exposure of sensitive information to an unauthorized actor (private IP address) | Low | Passive | | [548.1](548.1.md) | Exposure of information through directory listing | Low | Passive | +| [598.1](598.1.md) | Use of GET request method with sensitive query strings (session ID) | Medium | Passive | | [614.1](614.1.md) | Sensitive cookie without Secure attribute | Low | Passive | | [693.1](693.1.md) | Missing X-Content-Type-Options: nosniff | Low | Passive | diff --git a/doc/user/application_security/dast/index.md b/doc/user/application_security/dast/index.md index 0865cc10691..fd6c39ffbf1 100644 --- a/doc/user/application_security/dast/index.md +++ b/doc/user/application_security/dast/index.md @@ -51,7 +51,7 @@ results. On failure, the analyzer outputs an ## Prerequisites - [GitLab Runner](../../../ci/runners/index.md) available, with the -[`docker` executor](https://docs.gitlab.com/runner/executors/docker.html). +[`docker` executor](https://docs.gitlab.com/runner/executors/docker.html) on Linux/amd64. - Target application deployed. For more details, read [Deployment options](#deployment-options). - DAST runs in the `dast` stage, which must be added manually to your `.gitlab-ci.yml`. @@ -105,7 +105,7 @@ services: # use services to link your app container to the dast job variables: DAST_FULL_SCAN_ENABLED: "true" # do a full scan - DAST_ZAP_USE_AJAX_SPIDER: "true" # use the ajax spider + DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler ``` Most applications depend on multiple services such as databases or caching services. By default, services defined in the services fields cannot communicate @@ -314,6 +314,7 @@ include: variables: DAST_FULL_SCAN_ENABLED: "true" + DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler ``` If your DAST job exceeds the job timeout and you need to reduce the scan duration, we shared some @@ -455,6 +456,7 @@ include: variables: GIT_STRATEGY: fetch DAST_PATHS_FILE: url_file.txt # url_file.txt lives in the root directory of the project + DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler ``` ##### Use `DAST_PATHS` CI/CD variable @@ -470,6 +472,7 @@ include: variables: DAST_PATHS: "/page1.html,/category1/page1.html,/page3.html" + DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler ``` When using `DAST_PATHS` and `DAST_PATHS_FILE`, note the following: @@ -547,6 +550,7 @@ include: variables: DAST_WEBSITE: https://example.com DAST_SPIDER_MINS: 120 + DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler ``` Because the template is [evaluated before](../../../ci/yaml/index.md#include) the pipeline @@ -628,7 +632,7 @@ These CI/CD variables are specific to DAST. They can be used to customize the be | `DAST_AUTH_VERIFICATION_SELECTOR` <sup>2</sup> | selector | Verifies successful authentication by checking for presence of a selector once the login form has been submitted. Example: `css:.user-photo`. | | `DAST_AUTH_VERIFICATION_URL` <sup>1,2</sup> | URL | A URL only accessible to logged in users that DAST can use to confirm successful authentication. If provided, DAST exits if it cannot access the URL. Example: `"http://example.com/loggedin_page"`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207335) in GitLab 13.8. | | `DAST_AUTO_UPDATE_ADDONS` | boolean | ZAP add-ons are pinned to specific versions in the DAST Docker image. Set to `true` to download the latest versions when the scan starts. Default: `false`. | -| `DAST_BROWSER_PATH_TO_LOGIN_FORM` <sup>1,2</sup> | selector | Comma-separated list of selectors that will be clicked on prior to attempting to enter `DAST_USERNAME` and `DAST_PASSWORD` into the login form. Example: `"css:.navigation-menu,css:.login-menu-item"`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326633) in GitLab 14.1. | +| `DAST_BROWSER_PATH_TO_LOGIN_FORM` <sup>1,2</sup> | selector | Comma-separated list of selectors that are clicked on prior to attempting to enter `DAST_USERNAME` and `DAST_PASSWORD` into the login form. Example: `"css:.navigation-menu,css:.login-menu-item"`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326633) in GitLab 14.1. | | `DAST_DEBUG` <sup>1</sup> | boolean | Enable debug message output. Default: `false`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12652) in GitLab 13.1. | | `DAST_EXCLUDE_RULES` | string | Set to a comma-separated list of Vulnerability Rule IDs to exclude them from running during the scan. Rule IDs are numbers and can be found from the DAST log or on the [ZAP project](https://www.zaproxy.org/docs/alerts/). For example, `HTTP Parameter Override` has a rule ID of `10026`. Cannot be used when `DAST_ONLY_INCLUDE_RULES` is set. **Note:** In earlier versions of GitLab the excluded rules were executed but vulnerabilities they generated were suppressed. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118641) in GitLab 12.10. | | `DAST_EXCLUDE_URLS` <sup>1,2</sup> | URLs | The URLs to skip during the authenticated scan; comma-separated. Regular expression syntax can be used to match multiple URLs. For example, `.*` matches an arbitrary character sequence. Not supported for API scans. Example, `http://example.com/sign-out`. | @@ -737,7 +741,7 @@ Only run an authenticated scan against a test server. ### Log in using automatic detection of the login form -By providing a `DAST_USERNAME`, `DAST_PASSWORD`, and `DAST_AUTH_URL`, DAST will attempt to authenticate to the +By providing a `DAST_USERNAME`, `DAST_PASSWORD`, and `DAST_AUTH_URL`, DAST attempts to authenticate to the target application by locating the login form based on a determination about whether or not the form contains username or password fields. Automatic detection is "best-effort", and depending on the application being scanned may provide either a resilient login experience or one that fails to authenticate the user. @@ -753,8 +757,8 @@ Login process: ### Log in using explicit selection of the login form By providing a `DAST_USERNAME_FIELD`, `DAST_PASSWORD_FIELD`, and `DAST_SUBMIT_FIELD`, in addition to the fields required for automatic login, -DAST will attempt to authenticate to the target application by locating the login form based on the selectors provided. -Most applications will benefit from this approach to authentication. +DAST attempts to authenticate to the target application by locating the login form based on the selectors provided. +Most applications benefit from this approach to authentication. Login process: @@ -790,6 +794,7 @@ include: dast: variables: DAST_WEBSITE: "https://example.com" + DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler ... DAST_AUTH_VERIFICATION_URL: "https://example.com/user/welcome" ``` @@ -808,6 +813,7 @@ include: dast: variables: DAST_WEBSITE: "https://example.com" + DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler ... DAST_AUTH_VERIFICATION_SELECTOR: "css:.welcome-user" ``` @@ -826,6 +832,7 @@ include: dast: variables: DAST_WEBSITE: "https://example.com" + DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler ... DAST_AUTH_VERIFICATION_LOGIN_FORM: "true" ``` @@ -847,6 +854,7 @@ include: dast: variables: DAST_WEBSITE: "https://my.site.com" + DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler ... DAST_AUTH_URL: "https://my.site.com/admin" DAST_BROWSER_PATH_TO_LOGIN_FORM: "css:.navigation-menu,css:.login-menu-item" @@ -875,6 +883,7 @@ An example configuration where the authentication debug report is exported may l dast: variables: DAST_WEBSITE: "https://example.com" + DAST_BROWSER_SCAN: "true" # use the browser-based GitLab DAST crawler ... DAST_AUTH_REPORT: "true" artifacts: @@ -885,7 +894,7 @@ dast: ### Selectors Selectors are used by CI/CD variables to specify the location of an element displayed on a page in a browser. -Selectors have the format `type`:`search string`. The crawler will search for the selector using the search string based on the type. +Selectors have the format `type`:`search string`. The crawler searches for the selector using the search string based on the type. | Selector type | Example | Description | | ------------- | ---------------------------------- | ----------- | diff --git a/doc/user/application_security/dast_api/index.md b/doc/user/application_security/dast_api/index.md index cc20b49764f..839833d9d98 100644 --- a/doc/user/application_security/dast_api/index.md +++ b/doc/user/application_security/dast_api/index.md @@ -479,8 +479,8 @@ Follow these steps to provide the bearer token with `DAST_API_OVERRIDES_ENV`: `{"headers":{"Authorization":"Bearer dXNlcm5hbWU6cGFzc3dvcmQ="}}` (substitute your token). You can create CI/CD variables from the GitLab projects page at **Settings > CI/CD**, in the **Variables** section. - Due to the format of `TEST_API_BEARERAUTH` it's not possible to mask the variable. - To mask the token's value, you can create a second variable with the token value's, and define + Due to the format of `TEST_API_BEARERAUTH` it's not possible to mask the variable. + To mask the token's value, you can create a second variable with the token value's, and define `TEST_API_BEARERAUTH` with the value `{"headers":{"Authorization":"Bearer $MASKED_VARIABLE"}}`. 1. In your `.gitlab-ci.yml` file, set `DAST_API_OVERRIDES_ENV` to the variable you just created: @@ -876,7 +876,7 @@ variables: If the value must be generated or regenerated on expiration, you can provide a program or script for the DAST API scanner to execute on a specified interval. The provided command runs in an Alpine Linux -container that has Python 3 and Bash installed. +container that has Python 3 and Bash installed. You have to set the environment variable `DAST_API_OVERRIDES_CMD` to the program or script you would like to execute. The provided command creates the overrides JSON file as defined previously. @@ -885,7 +885,7 @@ You might want to install other scripting runtimes like NodeJS or Ruby, or maybe your overrides command. In this case, we recommend setting the `DAST_API_PRE_SCRIPT` to the file path of a script which provides those prerequisites. The script provided by `DAST_API_PRE_SCRIPT` is executed once, before the analyzer starts. -See the [Alpine Linux package management](https://wiki.alpinelinux.org/wiki/Alpine_Linux_package_management) +See the [Alpine Linux package management](https://wiki.alpinelinux.org/wiki/Alpine_Linux_package_management) page for information about installing Alpine Linux packages. You must provide three CI/CD variables, each set for correct operation: diff --git a/doc/user/application_security/dependency_list/index.md b/doc/user/application_security/dependency_list/index.md index baafdcda6e0..78de740c96d 100644 --- a/doc/user/application_security/dependency_list/index.md +++ b/doc/user/application_security/dependency_list/index.md @@ -15,7 +15,7 @@ details about those dependencies, including their known vulnerabilities. It is a To see the dependency list, go to your project and select **Security & Compliance > Dependency List**. -This information is sometimes referred to as a Software Bill of Materials or SBoM / BOM. +This information is sometimes referred to as a Software Bill of Materials, SBOM, or BOM. The dependency list only shows the results of the last successful pipeline to run on the default branch. This is why we recommend not changing the default behavior of allowing the secure jobs to fail. diff --git a/doc/user/application_security/dependency_scanning/analyzers.md b/doc/user/application_security/dependency_scanning/analyzers.md index 551488c0dc0..665d29c4017 100644 --- a/doc/user/application_security/dependency_scanning/analyzers.md +++ b/doc/user/application_security/dependency_scanning/analyzers.md @@ -50,7 +50,7 @@ Any custom change to the official analyzers can be achieved by using a You can switch to a custom Docker registry that provides the official analyzer images under a different prefix. For instance, the following instructs Dependency Scanning to pull `my-docker-registry/gl-images/gemnasium` -instead of `registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium`. +instead of `registry.gitlab.com/security-products/gemnasium`. In `.gitlab-ci.yml` define: ```yaml diff --git a/doc/user/application_security/dependency_scanning/index.md b/doc/user/application_security/dependency_scanning/index.md index a169b78a193..a4a7e6703ab 100644 --- a/doc/user/application_security/dependency_scanning/index.md +++ b/doc/user/application_security/dependency_scanning/index.md @@ -69,7 +69,8 @@ stages in the `.gitlab-ci.yml` file, the `test` stage is required. To run dependency scanning jobs, by default, you need GitLab Runner with the [`docker`](https://docs.gitlab.com/runner/executors/docker.html) or [`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor. -If you're using the shared runners on GitLab.com, this is enabled by default. +If you're using the shared runners on GitLab.com, this is enabled by default. The analyzer images +provided are for the Linux/amd64 architecture. WARNING: If you use your own runners, make sure your installed version of Docker @@ -181,7 +182,7 @@ table.supported-languages ul { </tr> <tr> <td rowspan="2">Java</td> - <td rowspan="2">8, 11, 13, 14, 15, or 16</td> + <td rowspan="2">8, 11, 13, 14, 15, 16, or 17</td> <td><a href="https://gradle.org/">Gradle</a><sup><b><a href="#notes-regarding-supported-languages-and-package-managers-1">1</a></b></sup></td> <td> <ul> @@ -335,27 +336,61 @@ To support the following package managers, the GitLab analyzers proceed in two s 1. Execute the package manager or a specific task, to export the dependency information. 1. Parse the exported dependency information. -| Package Manager | Preinstalled Versions | Tested Versions | -| ------ | ------ | ------ | -| Bundler | [2.1.4](https://gitlab.com/gitlab-org/security-products/analyzers/bundler-audit/-/blob/v2.11.3/Dockerfile#L15)<sup><b><a href="#exported-dependency-information-notes-1">1</a></b></sup> | [1.17.3](https://gitlab.com/gitlab-org/security-products/tests/ruby-bundler/-/blob/master/Gemfile.lock#L118), [2.1.4](https://gitlab.com/gitlab-org/security-products/tests/ruby-bundler/-/blob/bundler2-FREEZE/Gemfile.lock#L118) | -| sbt | [1.6.1](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.24.6/config/.tool-versions#L4) | [1.0.4](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.24.6/.gitlab-ci.yml#L330), [1.1.4](https://gitlab.com/gitlab-org/security-products/tests/scala-sbt-multiproject/-/blob/main/project/build.properties#L1), [1.1.6](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.24.6/.gitlab-ci.yml#L339), [1.2.8](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.24.6/.gitlab-ci.yml#L348), [1.3.12](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.24.6/.gitlab-ci.yml#L357), [1.4.6](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.24.6/.gitlab-ci.yml#L366), [1.6.1](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.24.6/.gitlab-ci.yml#L384) | -| Maven | [3.6.3](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.23.0/config/.tool-versions#L3) | [3.6.3](https://gitlab.com/gitlab-org/security-products/tests/java-maven/-/blob/master/pom.xml#L3) | -| Gradle | [6.7.1](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.23.0/config/.tool-versions#L5) | [5.6.4](https://gitlab.com/gitlab-org/security-products/tests/java-gradle/-/blob/master/gradle/wrapper/gradle-wrapper.properties#L3), [6.5](https://gitlab.com/gitlab-org/security-products/tests/java-gradle/-/blob/java-14/gradle/wrapper/gradle-wrapper.properties#L3), [6.7-rc-1](https://gitlab.com/gitlab-org/security-products/tests/java-gradle/-/blob/java-15/gradle/wrapper/gradle-wrapper.properties#L3), [6.9](https://gitlab.com/gitlab-org/security-products/tests/java-gradle/-/blob/java-14-gradle-6-9/gradle/wrapper/gradle-wrapper.properties#L3), [7.0-rc-2](https://gitlab.com/gitlab-org/security-products/tests/java-gradle/-/blob/java-16/gradle/wrapper/gradle-wrapper.properties#L3) | -| setuptools | [50.3.2](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/v2.29.9/Dockerfile#L27) | [57.5.0](https://gitlab.com/gitlab-org/security-products/tests/python-setuptools/-/blob/main/setup.py) | -| pip | [20.2.4](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/v2.29.9/Dockerfile#L26) | [20.x](https://gitlab.com/gitlab-org/security-products/tests/python-pip/-/blob/master/requirements.txt) | -| Pipenv | [2018.11.26](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/blob/v2.18.4/requirements.txt#L13) | [2018.11.26](https://gitlab.com/gitlab-org/security-products/tests/python-pipenv/-/blob/pipfile-lock-FREEZE/Pipfile.lock#L6)<sup><b><a href="#exported-dependency-information-notes-2">2</a></b></sup>, [2018.11.26](https://gitlab.com/gitlab-org/security-products/tests/python-pipenv/-/blob/master/Pipfile) | +| Package Manager | Pre-installed Versions | Tested Versions | +| ------ | ------ | ------ | +| Bundler | [2.1.4](https://gitlab.com/gitlab-org/security-products/analyzers/bundler-audit/-/blob/v2.11.3/Dockerfile#L15)<sup><b><a href="#exported-dependency-information-notes-1">1</a></b></sup> | [1.17.3](https://gitlab.com/gitlab-org/security-products/tests/ruby-bundler/-/blob/master/Gemfile.lock#L118), [2.1.4](https://gitlab.com/gitlab-org/security-products/tests/ruby-bundler/-/blob/bundler2-FREEZE/Gemfile.lock#L118) | +| sbt | [1.6.1](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.24.6/config/.tool-versions#L4) | [1.0.4](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.24.6/.gitlab-ci.yml#L330), [1.1.4](https://gitlab.com/gitlab-org/security-products/tests/scala-sbt-multiproject/-/blob/main/project/build.properties#L1), [1.1.6](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.24.6/.gitlab-ci.yml#L339), [1.2.8](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.24.6/.gitlab-ci.yml#L348), [1.3.12](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.24.6/.gitlab-ci.yml#L357), [1.4.6](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.24.6/.gitlab-ci.yml#L366), [1.6.1](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.24.6/.gitlab-ci.yml#L384) | +| Maven | [3.6.3](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.23.0/config/.tool-versions#L3) | [3.6.3](https://gitlab.com/gitlab-org/security-products/tests/java-maven/-/blob/master/pom.xml#L3) | +| Gradle | [6.7.1](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.23.0/config/.tool-versions#L5)<sup><b><a href="#exported-dependency-information-notes-2">2</a></b></sup>, [7.3.3](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.26.0/config/.tool-versions#L5)<sup><b><a href="#exported-dependency-information-notes-2">2</a></b></sup> | [5.6.4](https://gitlab.com/gitlab-org/security-products/tests/java-gradle/-/blob/master/gradle/wrapper/gradle-wrapper.properties#L3), [6.5](https://gitlab.com/gitlab-org/security-products/tests/java-gradle/-/blob/java-14/gradle/wrapper/gradle-wrapper.properties#L3), [6.7-rc-1](https://gitlab.com/gitlab-org/security-products/tests/java-gradle/-/blob/java-15/gradle/wrapper/gradle-wrapper.properties#L3), [6.7.1](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.27.1/.gitlab-ci.yml#L289-297)<sup><b><a href="#exported-dependency-information-notes-3">3</a></b></sup>, [6.9](https://gitlab.com/gitlab-org/security-products/tests/java-gradle/-/blob/java-14-gradle-6-9/gradle/wrapper/gradle-wrapper.properties#L3), [7.0-rc-2](https://gitlab.com/gitlab-org/security-products/tests/java-gradle/-/blob/java-16/gradle/wrapper/gradle-wrapper.properties#L3), [7.3](https://gitlab.com/gitlab-org/security-products/tests/java-gradle/-/blob/java-14-gradle-7-3/gradle/wrapper/gradle-wrapper.properties#L3), [7.3.3](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven/-/blob/v2.27.1/.gitlab-ci.yml#L299-317)<sup><b><a href="#exported-dependency-information-notes-3">3</a></b></sup> | +| setuptools | [50.3.2](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/v2.29.9/Dockerfile#L27) | [57.5.0](https://gitlab.com/gitlab-org/security-products/tests/python-setuptools/-/blob/main/setup.py) | +| pip | [20.2.4](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/v2.29.9/Dockerfile#L26) | [20.x](https://gitlab.com/gitlab-org/security-products/tests/python-pip/-/blob/master/requirements.txt) | +| Pipenv | [2018.11.26](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/blob/v2.18.4/requirements.txt#L13) | [2018.11.26](https://gitlab.com/gitlab-org/security-products/tests/python-pipenv/-/blob/pipfile-lock-FREEZE/Pipfile.lock#L6)<sup><b><a href="#exported-dependency-information-notes-4">4</a></b></sup>, [2018.11.26](https://gitlab.com/gitlab-org/security-products/tests/python-pipenv/-/blob/master/Pipfile) | <!-- markdownlint-disable MD044 --> <ol> <li> <a id="exported-dependency-information-notes-1"></a> <p> - The installed version of <code>Bundler</code> is only used for the <a href="https://gitlab.com/gitlab-org/security-products/analyzers/bundler-audit">bundler-audit</a> analyzer, and is not used for <a href="https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium">gemnasium</a> + The pre-installed version of <code>Bundler</code> is only used for the <a href="https://gitlab.com/gitlab-org/security-products/analyzers/bundler-audit">bundler-audit</a> analyzer, and is not used for <a href="https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium">gemnasium</a>. </p> </li> <li> <a id="exported-dependency-information-notes-2"></a> <p> + Different versions of Java require different versions of Gradle. The versions of Gradle listed in the above table are pre-installed + in the analyzer image. The version of Gradle used by the analyzer depends on whether your project uses a <code>gradlew</code> + (Gradle wrapper) file or not: + </p> + <ul> + <li> + <p> + If your project <i>does not use</i> a <code>gradlew</code> file, then the analyzer automatically switches to one of the + pre-installed Gradle versions, based on the version of Java specified by the + <a href="#configuring-specific-analyzers-used-by-dependency-scanning"><code>DS_JAVA_VERSION</code></a> variable. + </p> + <p>You can view the + <a href="https://docs.gradle.org/current/userguide/compatibility.html#java">Gradle Java compatibility matrix</a> to see which version + of Gradle is selected for each Java version. Note that we only support switching to one of these pre-installed Gradle versions + for Java versions 13 to 17. + </p> + </li> + <li> + <p> + If your project <i>does use</i> a <code>gradlew</code> file, then the version of Gradle pre-installed in the analyzer image is + ignored, and the version specified in your <code>gradlew</code> file is used instead. + </p> + </li> + </ul> + </li> + <li> + <a id="exported-dependency-information-notes-3"></a> + <p> + These tests confirm that if a <code>gradlew</code> file does not exist, the version of <code>Gradle</code> pre-installed in the analyzer image is used. + </p> + </li> + <li> + <a id="exported-dependency-information-notes-4"></a> + <p> This test confirms that if a <code>Pipfile.lock</code> file is found, it will be used by <a href="https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium">Gemnasium</a> to scan the exact package versions listed in this file. </p> </li> @@ -563,7 +598,7 @@ The following variables are used for configuring specific analyzers (used for a | `GEMNASIUM_DB_REF_NAME` | `gemnasium` | `master` | Branch name for remote repository database. `GEMNASIUM_DB_REMOTE_URL` is required. | | `DS_REMEDIATE` | `gemnasium` | `"true"` | Enable automatic remediation of vulnerable dependencies. | | `GEMNASIUM_LIBRARY_SCAN_ENABLED` | `gemnasium` | `"true"` | Enable detecting vulnerabilities in vendored JavaScript libraries. For now, `gemnasium` leverages [`Retire.js`](https://github.com/RetireJS/retire.js) to do this job. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/350512) in GitLab 14.8. | -| `DS_JAVA_VERSION` | `gemnasium-maven` | `11` | Version of Java. Available versions: `8`, `11`, `13`, `14`, `15`, `16`. | +| `DS_JAVA_VERSION` | `gemnasium-maven` | `11` | Version of Java. Available versions: `8`, `11`, `13`, `14`, `15`, `16`, `17`. | | `MAVEN_CLI_OPTS` | `gemnasium-maven` | `"-DskipTests --batch-mode"` | List of command line arguments that are passed to `maven` by the analyzer. See an example for [using private repositories](../index.md#using-private-maven-repositories). | | `GRADLE_CLI_OPTS` | `gemnasium-maven` | | List of command line arguments that are passed to `gradle` by the analyzer. | | `SBT_CLI_OPTS` | `gemnasium-maven` | | List of command-line arguments that the analyzer passes to `sbt`. | @@ -767,13 +802,13 @@ Here's an example dependency scanning report: } ``` -### CycloneDX reports +### CycloneDX Software Bill of Materials > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/350509) in GitLab 14.8 in [Beta](../../../policy/alpha-beta-support.md#beta-features). In addition to the [JSON report file](#reports-json-format), the [Gemnasium](https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium) -Dependency Scanning tool outputs a [CycloneDX](https://cyclonedx.org/) report for -each supported lock or build file it detects. These CycloneDX reports are named +Dependency Scanning tool outputs a [CycloneDX](https://cyclonedx.org/) Software Bill of Materials (SBOM) for +each supported lock or build file it detects. These CycloneDX SBOMs are named `cyclonedx-<package-type>-<package-manager>.json`, and are saved in the same directory as the detected lock or build files. @@ -791,7 +826,7 @@ For example, if your project has the following structure: └── go.sum ``` -Then the Gemnasium scanner generates the following CycloneDX reports: +Then the Gemnasium scanner generates the following CycloneDX SBOMs: ```plaintext . @@ -809,23 +844,23 @@ Then the Gemnasium scanner generates the following CycloneDX reports: └── cyclonedx-go-go.json ``` -The CycloneDX reports can be downloaded [the same way as other job artifacts](../../../ci/pipelines/job_artifacts.md#download-job-artifacts). +The CycloneDX SBOMs can be downloaded [the same way as other job artifacts](../../../ci/pipelines/job_artifacts.md#download-job-artifacts). -### Merging multiple CycloneDX Reports +### Merging multiple CycloneDX SBOMs -You can use a CI/CD job to merge multiple CycloneDX Reports into a single report. +You can use a CI/CD job to merge multiple CycloneDX SBOMs into a single SBOM. For example: ```yaml stages: - test - - merge-cyclonedx-reports + - merge-cyclonedx-sboms include: - template: Security/Dependency-Scanning.gitlab-ci.yml -merge cyclonedx reports: - stage: merge-cyclonedx-reports +merge cyclonedx sboms: + stage: merge-cyclonedx-sboms image: alpine:latest script: - wget https://github.com/CycloneDX/cyclonedx-cli/releases/download/v0.22.0/cyclonedx-linux-musl-x64 -O /usr/local/bin/cyclonedx-cli @@ -838,14 +873,14 @@ merge cyclonedx reports: ``` GitLab uses [CycloneDX Properties](https://cyclonedx.org/use-cases/#properties--name-value-store) -to store implementation-specific details in the metadata of each CycloneDX report, -such as the location of build and lock files. If multiple CycloneDX reports are merged together, +to store implementation-specific details in the metadata of each CycloneDX SBOM, +such as the location of build and lock files. If multiple CycloneDX SBOMs are merged together, this information is removed from the resulting merged file. NOTE: -CycloneDX reports are a [Beta](../../../policy/alpha-beta-support.md#beta-features) feature, +CycloneDX SBOMs are a [Beta](../../../policy/alpha-beta-support.md#beta-features) feature, and the reports are subject to change during the beta period. Do not build integrations -that rely on the format of these reports staying consistent, as the format might change +that rely on the format of these SBOMs staying consistent, as the format might change before the feature is made generally available. ## Versioning and release process @@ -892,11 +927,11 @@ import the following default dependency scanning analyzer images from `registry. your [local Docker container registry](../../packages/container_registry/index.md): ```plaintext -registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium:2 -registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium-maven:2 -registry.gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python:2 -registry.gitlab.com/gitlab-org/security-products/analyzers/retire.js:2 -registry.gitlab.com/gitlab-org/security-products/analyzers/bundler-audit:2 +registry.gitlab.com/security-products/gemnasium:2 +registry.gitlab.com/security-products/gemnasium-maven:2 +registry.gitlab.com/security-products/gemnasium-python:2 +registry.gitlab.com/security-products/retire.js:2 +registry.gitlab.com/security-products/bundler-audit:2 ``` The process for importing Docker images into a local offline Docker registry depends on @@ -961,7 +996,13 @@ BUNDLER_AUDIT_ADVISORY_DB_REF_NAME: "master" BUNDLER_AUDIT_ADVISORY_DB_URL: "gitlab.example.com/ruby-advisory-db.git" ``` -#### Python (setup tools) +#### Python (pip) + +If you need to install Python packages before the analyzer runs, you should use `pip install --user` in the `before_script` of the scanning job. The `--user` flag causes project dependencies to be installed in the user directory. If you do not pass the `--user` option, packages are installed globally, and they are not scanned and don't show up when listing project dependencies. + +#### Python (setuptools) + +If you need to install Python packages before the analyzer runs, you should use `python setup.py install --user` in the `before_script` of the scanning job. The `--user` flag causes project dependencies to be installed in the user directory. If you do not pass the `--user` option, packages are installed globally, and they are not scanned and don't show up when listing project dependencies. When using self-signed certificates for your private PyPi repository, no extra job configuration (aside from the template `.gitlab-ci.yml` above) is needed. However, you must update your `setup.py` to diff --git a/doc/user/application_security/iac_scanning/index.md b/doc/user/application_security/iac_scanning/index.md index 89d3531bccd..b72f54b4493 100644 --- a/doc/user/application_security/iac_scanning/index.md +++ b/doc/user/application_security/iac_scanning/index.md @@ -22,7 +22,7 @@ To run IaC scanning jobs, by default, you need GitLab Runner with the If you're using the shared runners on GitLab.com, this is enabled by default. WARNING: -Our IaC scanning jobs require a Linux container type. Windows containers are not yet supported. +Our IaC scanning jobs require a Linux/amd64 container type. Windows containers are not yet supported. WARNING: If you use your own runners, make sure the Docker version installed @@ -58,7 +58,7 @@ as shown in the following table: |:---------------------------------------------------------------------------------------|:--------------------|:-------------------| | [Configure IaC Scanners](#configuration) | **{check-circle}** | **{check-circle}** | | View [JSON Report](#reports-json-format) | **{check-circle}** | **{check-circle}** | -| Presentation of JSON Report in Merge Request | **{dotted-circle}** | **{check-circle}** | +| Presentation of JSON Report in merge request | **{dotted-circle}** | **{check-circle}** | | [Address vulnerabilities](../../application_security/vulnerabilities/index.md) | **{dotted-circle}** | **{check-circle}** | | [Access to Security Dashboard](../../application_security/security_dashboard/index.md) | **{dotted-circle}** | **{check-circle}** | @@ -76,7 +76,12 @@ To configure IaC Scanning for a project you can: ### Configure IaC Scanning manually To enable IaC Scanning you must [include](../../../ci/yaml/index.md#includetemplate) the -[`SAST-IaC.latest.gitlab-ci.yml template`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST-IaC.latest.gitlab-ci.yml) provided as part of your GitLab installation. +[`SAST-IaC.latest.gitlab-ci.yml template`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Security/SAST-IaC.latest.gitlab-ci.yml) provided as part of your GitLab installation. Here is an example of how to include it: + +```yaml +include: + - template: Security/SAST-IaC.latest.gitlab-ci.yml +``` The included template creates IaC scanning jobs in your CI/CD pipeline and scans your project's configuration files for possible vulnerabilities. diff --git a/doc/user/application_security/index.md b/doc/user/application_security/index.md index 6a0b81335fd..ff548f1d29f 100644 --- a/doc/user/application_security/index.md +++ b/doc/user/application_security/index.md @@ -110,11 +110,9 @@ For more details about each of the security scanning tools, see their respective ### Override the default registry base address -By default, GitLab security scanners use `registry.gitlab.com/gitlab-org/security-products/analyzers` as the +By default, GitLab security scanners use `registry.gitlab.com/security-products` as the base address for Docker images. You can override this globally by setting the CI/CD variable -`SECURE_ANALYZERS_PREFIX` to another location. Note that this affects all scanners at once, except -the container-scanning analyzer which uses -`registry.gitlab.com/security-products/container-scanning` as its registry. +`SECURE_ANALYZERS_PREFIX` to another location. Note that this affects all scanners at once. ### Use security scanning tools with merge request pipelines @@ -221,7 +219,7 @@ security issues: WARNING: This feature is in its end-of-life process. It is [deprecated](../../update/deprecations.md#vulnerability-check) for use in GitLab 14.8, and is planned for removal in GitLab 15.0. Users should migrate to the new -[Security Approval Policies](policies/#scan-result-policy-editor). +[Security Approval Policies](policies/scan-result-policies.md). To prevent a merge request introducing a security vulnerability in a project, enable the Vulnerability-Check rule. While this rule is enabled, additional merge request approval by @@ -397,6 +395,8 @@ any report artifacts that failed validation. ### Enable security report validation +> [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/354928) in GitLab 14.9, and planned for removal in GitLab 15.0. + To enable report artifacts validation, set the `VALIDATE_SCHEMA` environment variable to `"true"` for the desired jobs in the `.gitlab-ci.yml` file. diff --git a/doc/user/application_security/offline_deployments/index.md b/doc/user/application_security/offline_deployments/index.md index 915e43d0fa5..7aeb094093c 100644 --- a/doc/user/application_security/offline_deployments/index.md +++ b/doc/user/application_security/offline_deployments/index.md @@ -179,7 +179,7 @@ set -ux # Specify needed analyzer images analyzers=${SAST_ANALYZERS:-"bandit eslint gosec"} -gitlab=registry.gitlab.com/gitlab-org/security-products/analyzers/ +gitlab=registry.gitlab.com/security-products/ for i in "${analyzers[@]}" do diff --git a/doc/user/application_security/policies/img/container_policy_rule_mode_v14_3.png b/doc/user/application_security/policies/img/container_policy_rule_mode_v14_3.png Binary files differdeleted file mode 100644 index b21d0330b2f..00000000000 --- a/doc/user/application_security/policies/img/container_policy_rule_mode_v14_3.png +++ /dev/null diff --git a/doc/user/application_security/policies/img/container_policy_yaml_mode_v14_3.png b/doc/user/application_security/policies/img/container_policy_yaml_mode_v14_3.png Binary files differdeleted file mode 100644 index 31d5eb57228..00000000000 --- a/doc/user/application_security/policies/img/container_policy_yaml_mode_v14_3.png +++ /dev/null diff --git a/doc/user/application_security/policies/img/policy_rule_mode_v14_9.png b/doc/user/application_security/policies/img/policy_rule_mode_v14_9.png Binary files differnew file mode 100644 index 00000000000..8ca7547a33c --- /dev/null +++ b/doc/user/application_security/policies/img/policy_rule_mode_v14_9.png diff --git a/doc/user/application_security/policies/img/policy_yaml_mode_v14_9.png b/doc/user/application_security/policies/img/policy_yaml_mode_v14_9.png Binary files differnew file mode 100644 index 00000000000..1d71e8684e9 --- /dev/null +++ b/doc/user/application_security/policies/img/policy_yaml_mode_v14_9.png diff --git a/doc/user/application_security/policies/img/scan_result_policy_yaml_mode_v14_6.png b/doc/user/application_security/policies/img/scan_result_policy_yaml_mode_v14_6.png Binary files differdeleted file mode 100644 index 57649c58d8b..00000000000 --- a/doc/user/application_security/policies/img/scan_result_policy_yaml_mode_v14_6.png +++ /dev/null diff --git a/doc/user/application_security/policies/index.md b/doc/user/application_security/policies/index.md index 803f3983b96..8a39220da35 100644 --- a/doc/user/application_security/policies/index.md +++ b/doc/user/application_security/policies/index.md @@ -64,13 +64,13 @@ The policy editor has two modes: - The visual _Rule_ mode allows you to construct and preview policy rules using rule blocks and related controls. -  +  - YAML mode allows you to enter a policy definition in `.yaml` format and is aimed at expert users and cases that the Rule mode doesn't support. -  +  You can use both modes interchangeably and switch between them at any time. If a YAML resource is incorrect or contains data not supported diff --git a/doc/user/application_security/policies/scan-execution-policies.md b/doc/user/application_security/policies/scan-execution-policies.md index 4e44162d5c5..c3778ac97de 100644 --- a/doc/user/application_security/policies/scan-execution-policies.md +++ b/doc/user/application_security/policies/scan-execution-policies.md @@ -8,14 +8,20 @@ info: To determine the technical writer assigned to the Stage/Group associated w Project owners can use scan execution policies to require that security scans run on a specified schedule or with the project pipeline. Required scans are injected into the CI pipeline as new jobs -with a long, random job name. In the unlikely event of a job name collision, the security policy job -overwrites any pre-existing job in the pipeline. +with a long, random job name. In the unlikely event of a job name collision, the security policy job overwrites +any pre-existing job in the pipeline. This feature has some overlap with [compliance framework pipelines](../../project/settings/#compliance-pipeline-configuration), as we have not [unified the user experience for these two features](https://gitlab.com/groups/gitlab-org/-/epics/7312). For details on the similarities and differences between these features, see [Enforce scan execution](../#enforce-scan-execution). +NOTE: +Policy jobs are created in the `test` stage of the pipeline. If you modify the default pipeline +[`stages`](../../../ci/yaml/index.md#stages), +you must ensure that the `test` stage exists in the list. Otherwise, the pipeline fails to run and +an error appears that states `chosen stage does not exist`. + ## Scan execution policy editor NOTE: diff --git a/doc/user/application_security/policies/scan-result-policies.md b/doc/user/application_security/policies/scan-result-policies.md index 7857de8c780..8215316bcab 100644 --- a/doc/user/application_security/policies/scan-result-policies.md +++ b/doc/user/application_security/policies/scan-result-policies.md @@ -13,7 +13,7 @@ job is fully executed. ## Scan result policy editor -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/77814) in GitLab 14.8 with a flag named `scan_result_policy`. Disabled by default. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/77814) in GitLab 14.8. NOTE: Only project Owners have the [permissions](../../permissions.md#project-members-permissions) @@ -28,10 +28,7 @@ the bottom of the editor. All scan result policy changes are applied through a background job that runs once every 10 minutes. Allow up to 10 minutes for any policy changes committed to this project to take effect. -The policy editor only supports YAML mode. To follow work on Rule mode, see the epic -[Allow Users to Edit Rule-mode scan result policies in the Policy UI](https://gitlab.com/groups/gitlab-org/-/epics/5363). - - +The [policy editor](index.md#policy-editor) supports YAML mode and rule mode. ## Scan result policies schema diff --git a/doc/user/application_security/sast/index.md b/doc/user/application_security/sast/index.md index 3c0a2caf114..d3a79410eea 100644 --- a/doc/user/application_security/sast/index.md +++ b/doc/user/application_security/sast/index.md @@ -57,7 +57,7 @@ To run SAST jobs, by default, you need GitLab Runner with the If you're using the shared runners on GitLab.com, this is enabled by default. WARNING: -Our SAST jobs require a Linux container type. Windows containers are not yet supported. +Our SAST jobs require a Linux/amd64 container type. Windows containers are not yet supported. WARNING: If you use your own runners, make sure the Docker version installed @@ -315,7 +315,6 @@ To disable analyzer rules: 1. In one or more `ruleset.identifier` sub sections, list the rules that you want disabled. Every `ruleset.identifier` section has: - a `type` field, to name the predefined rule identifier that the targeted analyzer uses. - - a `value` field, to name the rule to be disabled. ##### Example: Disable predefined rules of SAST analyzers @@ -345,6 +344,9 @@ and `sobelow` by matching the `type` and `value` of identifiers: value = "sql_injection" ``` +Those vulnerabilities containing the provided type and value are now disabled, meaning +they won't be displayed in Merge Request nor the Vulnerability Report. + #### Override predefined analyzer rules To override analyzer rules: @@ -365,30 +367,40 @@ To override analyzer rules: ##### Example: Override predefined rules of SAST analyzers -In the following example, rules from `eslint` -and `gosec` are matched by the `type` and `value` of identifiers and -then overridden: +Before adding a ruleset, we verify which vulnerability will be overwritten by viewing the [`gl-sast-report.json`](#reports-json-format): + +```json +"identifiers": [ + { + "type": "gosec_rule_id", + "name": "Gosec Rule ID G307", + "value": "G307" + }, + { + "type": "CWE", + "name": "CWE-703", + "value": "703", + "url": "https://cwe.mitre.org/data/definitions/703.html" + } + ] +``` + +In the following example, rules from `gosec` are matched by the `type` +and `value` of identifiers and then overridden: ```toml -[eslint] - [[eslint.ruleset]] - [eslint.ruleset.identifier] - type = "eslint_rule_id" - value = "security/detect-object-injection" - [eslint.ruleset.override] - description = "OVERRIDDEN description" - message = "OVERRIDDEN message" - name = "OVERRIDDEN name" - severity = "Critical" [gosec] [[gosec.ruleset]] [gosec.ruleset.identifier] type = "CWE" - value = "CWE-79" + value = "703" [gosec.ruleset.override] severity = "Critical" ``` +If a vulnerability is found with a type `CWE` with a value of `703` then +the vulnerability severity is overwritten to `Critical`. + #### Synthesize a custom configuration To create a custom configuration, you can use passthrough chains. @@ -661,6 +673,25 @@ repositories and thus require credentials like username and password to download Depending on the analyzer, such credentials can be provided to it via [custom CI/CD variables](#custom-cicd-variables). +#### Using a CI/CD variable to pass username and password to a private Go repository + +If your Go project depends on private modules, see +[Fetch modules from private projects](../../packages/go_proxy/index.md#fetch-modules-from-private-projects) +for how to provide authentication over HTTPS. + +To specify credentials via `~/.netrc` provide a `before_script` containing the following: + +```yaml +gosec-sast: + before_script: + - | + cat <<EOF > ~/.netrc + machine gitlab.com + login $CI_DEPLOY_USER + password $CI_DEPLOY_PASSWORD + EOF +``` + #### Using a CI/CD variable to pass username and password to a private Maven repository If your private Maven repository requires login credentials, @@ -878,12 +909,12 @@ variables: ## Reports JSON format -SAST outputs a report file in JSON format. The report file contains details of all found vulnerabilities. -To download the report file, you can either: +SAST outputs a report file in JSON format. The report file contains details of all found vulnerabilities. +To download the report file, you can either: - Download the file from the CI/CD pipelines page. -- In the pipelines tab on merge requests, set [`artifacts: paths`](../../../ci/yaml/index.md#artifactspaths) to `gl-sast-report.json`. - +- In the pipelines tab on merge requests, set [`artifacts: paths`](../../../ci/yaml/index.md#artifactspaths) to `gl-sast-report.json`. + For information, see [Download job artifacts](../../../ci/pipelines/job_artifacts.md#download-job-artifacts). For details of the report file's schema, see diff --git a/doc/user/application_security/secret_detection/index.md b/doc/user/application_security/secret_detection/index.md index 2ce2d59898f..582497eb465 100644 --- a/doc/user/application_security/secret_detection/index.md +++ b/doc/user/application_security/secret_detection/index.md @@ -42,7 +42,7 @@ To run Secret Detection jobs, by default, you need GitLab Runner with the If you're using the shared runners on GitLab.com, this is enabled by default. WARNING: -Our Secret Detection jobs expect a Linux container type. Windows containers are not supported. +Our Secret Detection jobs expect a Linux/amd64 container type. Windows containers are not supported. WARNING: If you use your own runners, make sure the Docker version installed @@ -328,14 +328,6 @@ as part of your normal job definition. A new configuration variable ([`SECRET_DETECTION_HISTORIC_SCAN`](#available-cicd-variables)) can be set to change the behavior of the GitLab Secret Detection scan to run on the entire Git history of a repository. -We have created a [short video walkthrough](https://youtu.be/wDtc_K00Y0A) showcasing how you can perform a full history secret detection scan. -<div class="video-fallback"> - See the video: <a href="https://www.youtube.com/watch?v=wDtc_K00Y0A">Walkthrough of historical secret detection scan</a>. -</div> -<figure class="video-container"> - <iframe src="https://www.youtube.com/embed/wDtc_K00Y0A" frameborder="0" allowfullscreen="true"> </iframe> -</figure> - ## Running Secret Detection in an offline environment For self-managed GitLab instances in an environment with limited, restricted, or intermittent access @@ -450,9 +442,9 @@ secret_detection: ### `secret-detection` job fails with `ERR fatal: ambiguous argument` message -Your `secret-detection` job can fail with `ERR fatal: ambiguous argument` error if your -repository's default branch is unrelated to the branch the job was triggered for. +Your `secret-detection` job can fail with `ERR fatal: ambiguous argument` error if your +repository's default branch is unrelated to the branch the job was triggered for. See issue [!352014](https://gitlab.com/gitlab-org/gitlab/-/issues/352014) for more details. -To resolve the issue, make sure to correctly [set your default branch](../../project/repository/branches/default.md#change-the-default-branch-name-for-a-project) on your repository. You should set it to a branch -that has related history with the branch you run the `secret-detection` job on. +To resolve the issue, make sure to correctly [set your default branch](../../project/repository/branches/default.md#change-the-default-branch-name-for-a-project) on your repository. You should set it to a branch +that has related history with the branch you run the `secret-detection` job on. diff --git a/doc/user/application_security/secret_detection/post_processing.md b/doc/user/application_security/secret_detection/post_processing.md index 972558c3b95..643da47d876 100644 --- a/doc/user/application_security/secret_detection/post_processing.md +++ b/doc/user/application_security/secret_detection/post_processing.md @@ -56,7 +56,7 @@ A vendor revocation receiver service integrates with a GitLab instance to receiv a web notification and respond to leaked token requests. To implement a receiver service to revoke leaked tokens: - + 1. Create a publicly accessible HTTP service matching the corresponding API contract below. Your service should be idempotent and rate-limited. 1. When a pipeline corresponding to its revocable token type (in the example, `my_api_token`) diff --git a/doc/user/application_security/vulnerabilities/index.md b/doc/user/application_security/vulnerabilities/index.md index 7b39002bac3..0b27760b4bb 100644 --- a/doc/user/application_security/vulnerabilities/index.md +++ b/doc/user/application_security/vulnerabilities/index.md @@ -27,8 +27,9 @@ On the vulnerability's page, you can: - [Change the vulnerability's status](#change-vulnerability-status). - [Create an issue](#create-an-issue-for-a-vulnerability). - [Link issues to the vulnerability](#linked-issues). -- [Resolve a vulnerability](#resolve-a-vulnerability), if a solution is - available. +- [Resolve a vulnerability](#resolve-a-vulnerability) if a solution is + available. +- [View security training specific to the detected vulnerability](#view-security-training-for-a-vulnerability). ## Vulnerability status values @@ -80,7 +81,7 @@ The issue is then opened so you can take further action. Prerequisites: - [Enable Jira integration](../../../integration/jira/index.md). - The **Enable Jira issues creation from vulnerabilities** option must be selected as part of the configuration. + The **Enable Jira issue creation from vulnerabilities** option must be selected as part of the configuration. - Each user must have a personal Jira user account with permission to create issues in the target project. To create a Jira issue for a vulnerability: @@ -159,3 +160,29 @@ To manually apply the patch that GitLab generated for a vulnerability: 1. Ensure your local project has the same commit checked out that was used to generate the patch. 1. Run `git apply remediation.patch`. 1. Verify and commit the changes to your branch. + +## Enable security training for vulnerabilities + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6176) in GitLab 14.9. + +Security training helps your developers learn how to fix vulnerabilities. Developers can view security training from selected educational providers, relevant to the detected vulnerability. + +To enable security training for vulnerabilities in your project: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Security & Compliance > Configuration**. +1. On the tab bar, select **Vulnerability Management**. +1. To enable a security training provider, turn on the toggle. + +## View security training for a vulnerability + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6176) in GitLab 14.9. + +If security training is enabled, the vulnerability page includes a training link relevant to the detected vulnerability. + +To view the security training for a vulnerability: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Security & Compliance > Vulnerability report**. +1. Select the vulnerability for which you want to view security training. +1. Select **View training**. diff --git a/doc/user/application_security/vulnerability_report/index.md b/doc/user/application_security/vulnerability_report/index.md index ba1455ab70a..eb59c289700 100644 --- a/doc/user/application_security/vulnerability_report/index.md +++ b/doc/user/application_security/vulnerability_report/index.md @@ -7,7 +7,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Vulnerability Report **(ULTIMATE)** -The Vulnerability Report provides information about vulnerabilities from scans of the default branch. It contains cumulative results of all successful jobs, regardless of whether the pipeline was successful. +The Vulnerability Report provides information about vulnerabilities from scans of the default branch. It contains cumulative results of all successful jobs, regardless of whether the pipeline was successful. The scan results from a pipeline are only ingested after all the jobs in the pipeline complete. Partial results for a pipeline with jobs in progress can be seen in the pipeline security tab. @@ -52,6 +52,7 @@ From the Vulnerability Report you can: - [View an issue raised for a vulnerability](#view-issues-raised-for-a-vulnerability). - [Change the status of vulnerabilities](#change-status-of-vulnerabilities). - [Export details of vulnerabilities](#export-vulnerability-details). +- [Manually add a vulnerability finding](#manually-add-a-vulnerability-finding). ## Vulnerability Report filters @@ -219,6 +220,25 @@ You can dismiss a vulnerability for the entire project: To undo this action, select a different status from the same menu. +## Manually add a vulnerability finding + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/301003) in GitLab 14.9. Disabled by default. + +FLAG: +This feature is not enabled by default. To make it available, ask an administrator to +[enable the feature flag](../../feature_flags.md) named `new_vulnerability_form`. +On GitLab.com, this feature is not yet available. + +To add a new vulnerability finding from your project level Vulnerability Report page: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Security & Compliance > Vulnerability Report**. +1. Click on **Submit Vulnerability**. +1. Complete the fields and submit the form. + +You will be brought to the newly created vulnerability's detail page. Manually created records appear in the +Group, Project, and Security Center Vulnerability Reports. To filter them, use the Generic Tool filter. + ## Operational vulnerabilities > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6345) in GitLab 14.6. diff --git a/doc/user/clusters/agent/ci_cd_tunnel.md b/doc/user/clusters/agent/ci_cd_tunnel.md index 5fe772d9686..73a8470e025 100644 --- a/doc/user/clusters/agent/ci_cd_tunnel.md +++ b/doc/user/clusters/agent/ci_cd_tunnel.md @@ -4,60 +4,254 @@ group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# CI/CD Tunnel **(FREE)** +# Using a GitLab CI/CD workflow for Kubernetes **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327409) in GitLab 14.1. > - The pre-configured `KUBECONFIG` was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/324275) in GitLab 14.2. > - The ability to authorize groups was [introduced](https://gitlab.com/groups/gitlab-org/-/epics/5784) in GitLab 14.3. > - [Moved](https://gitlab.com/groups/gitlab-org/-/epics/6290) to GitLab Free in 14.5. > - Support for Omnibus installations was [introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/5686) in GitLab 14.5. +> - The ability to switch between certificate-based clusters and agents was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/335089) in GitLab 14.9. The certificate-based cluster context is always called `gitlab-deploy`. -To use GitLab CI/CD to safely deploy your application to a cluster, you can use the CI/CD Tunnel. +You can use a GitLab CI/CD workflow to safely deploy to and update your Kubernetes clusters. -You can authorize multiple projects to access the same cluster, so you -can keep your application's codebase in one repository and configure -your cluster in another. This method is scalable and can save you resources. +To do so, you must first [install an agent in your cluster](install/index.md). When done, you have a Kubernetes context and can +run Kubernetes API commands in your GitLab CI/CD pipeline. -To ensure access to your cluster is safe, only the projects you -authorize can access your Agent through the CI/CD Tunnel. +To ensure access to your cluster is safe: -## Prerequisites +- Each agent has a separate context (`kubecontext`). +- Only the project where the agent is configured, and any additional projects you authorize, can access the agent in your cluster. -To use the CI/CD Tunnel, you need an existing Kubernetes cluster connected to GitLab through the -[GitLab Agent](install/index.md#install-the-agent-onto-the-cluster). +You do not need to have a runner in the cluster with the agent. -To run your CI/CD jobs using the CI/CD Tunnel, you do not need to have a runner in the same cluster. +## GitLab CI/CD workflow steps -## How the CI/CD Tunnel works +To update a Kubernetes cluster by using GitLab CI/CD, complete the following steps. -When you authorize a project to use an Agent, the Tunnel automatically -injects a `KUBECONFIG` variable into its CI/CD jobs. This way, you can -run `kubectl` commands from GitLab CI/CD scripts that belong to the -authorized project. +1. Ensure you have a working Kubernetes cluster and the manifests are in a GitLab project. +1. In the same GitLab project, [register and install the GitLab agent](install/index.md). +1. [Update your `.gitlab-ci.yml` file](#update-your-gitlab-ciyml-file-to-run-kubectl-commands) to + select the agent's Kubernetes context and run the Kubernetes API commands. +1. Run your pipeline to deploy to or update the cluster. -When you authorize a group, all the projects that belong to that group -become authorized to access the selected Agent. +If you have multiple GitLab projects that contain Kubernetes manifests: -An Agent can only authorize projects or groups in the same group -hierarchy as the Agent's configuration project. You can authorize -up to 100 projects and 100 groups per Agent. +1. [Install the GitLab agent](install/index.md) in its own project, or in one of the + GitLab projects where you keep Kubernetes manifests. +1. [Authorize the agent](#authorize-the-agent) to access your GitLab projects. +1. Optional. For added security, [use impersonation](#use-impersonation-to-restrict-project-and-group-access). +1. [Update your `.gitlab-ci.yml` file](#update-your-gitlab-ciyml-file-to-run-kubectl-commands) to + select the agent's Kubernetes context and run the Kubernetes API commands. +1. Run your pipeline to deploy to or update the cluster. -Also, each Agent has a separate context (`kubecontext`). -The Tunnel uses this information to safely allow access to the cluster from -jobs running in the projects you authorized. +## Authorize the agent -### `~/.kube/cache` permissions - -`kubectl` and other tools based on the same libraries (such as Helm, `kpt`, and `kustomize`) cache information about +You must authorize the agent to access the project where you keep your Kubernetes manifests. +You can authorize the agent to access individual projects, or authorize a group or subgroup, +so all projects within have access. For added security, you can also +[use impersonation](#use-impersonation-to-restrict-project-and-group-access). + +### Authorize the agent to access your projects + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327850) in GitLab 14.4. + +To authorize the agent to access the GitLab project where you keep Kubernetes manifests: + +1. On the top bar, select **Menu > Projects** and find the project that contains the agent configuration file (`config.yaml`). +1. Edit the file. Under the `ci_access` keyword, add the `projects` attribute. +1. For the `id`, add the path: + + ```yaml + ci_access: + projects: + - id: path/to/project + ``` + + - The Kubernetes projects must be in the same group hierarchy as the project where the agent's configuration is. + - You can authorize up to 100 projects. + +All CI/CD jobs now include a `KUBECONFIG` with contexts for every shared agent connection. +Choose the context to run `kubectl` commands from your CI/CD scripts. + +### Authorize the agent to access projects in your groups + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5784) in GitLab 14.3. + +To authorize the agent to access all of the GitLab projects in a group or subgroup: + +1. On the top bar, select **Menu > Projects** and find the project that contains the agent configuration file (`config.yaml`). +1. Edit the file. Under the `ci_access` keyword, add the `groups` attribute. +1. For the `id`, add the path: + + ```yaml + ci_access: + groups: + - id: path/to/group/subgroup + ``` + + - The Kubernetes projects must be in the same group hierarchy as the project where the agent's configuration is. + - You can authorize up to 100 groups. + +All the projects that belong to the group are now authorized to access the agent. +All CI/CD jobs now include a `KUBECONFIG` with contexts for every shared agent connection. +Choose the context to run `kubectl` commands from your CI/CD scripts. + +## Update your `.gitlab-ci.yml` file to run `kubectl` commands + +In the project where you want to run Kubernetes commands, edit your project's `.gitlab-ci.yml` file. + +In the first command under the `script` keyword, set your agent's context. +Use the format `path/to/agent/repository:agent-name`. For example: + +```yaml + deploy: + image: + name: bitnami/kubectl:latest + entrypoint: [""] + script: + - kubectl config get-contexts + - kubectl config use-context path/to/agent/repository:agent-name + - kubectl get pods +``` + +If you are not sure what your agent's context is, open a terminal and connect to your cluster. +Run `kubectl config get-contexts`. + +### Environments with both certificate-based and agent-based connections + +When you deploy to an environment that has both a [certificate-based +cluster](../../infrastructure/clusters/index.md) (deprecated) and an agent connection: + +- The certificate-based cluster's context is called `gitlab-deploy`. This context + is always selected by default. +- In GitLab 14.9 and later, agent contexts are included in the + `KUBECONFIG`. You can select them by using `kubectl config use-context + path/to/agent/repository:agent-name`. +- In GitLab 14.8 and earlier, you can still use agent connections, but for environments that + already have a certificate-based cluster, the agent connections are not included in the `KUBECONFIG`. + +To use an agent connection when certificate-based connections are present, you can manually configure a new `kubectl` +configuration context. For example: + + ```yaml + deploy: + variables: + KUBE_CONTEXT: my-context # The name to use for the new context + AGENT_ID: 1234 # replace with your agent's numeric ID + K8S_PROXY_URL: wss://kas.gitlab.com/k8s-proxy/ # replace with your agent server (KAS) Kubernetes proxy URL + # ... any other variables you have configured + before_script: + - kubectl config set-credentials agent:$AGENT_ID --token="ci:${AGENT_ID}:${CI_JOB_TOKEN}" + - kubectl config set-cluster gitlab --server="${K8S_PROXY_URL}" + - kubectl config set-context "$KUBE_CONTEXT" --cluster=gitlab --user="agent:${AGENT_ID}" + - kubectl config use-context "$KUBE_CONTEXT" + # ... rest of your job configuration + ``` + +## Use impersonation to restrict project and group access **(PREMIUM)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345014) in GitLab 14.5. + +By default, your CI/CD job inherits all the permissions from the service account used to install the +agent in the cluster. +To restrict access to your cluster, you can use [impersonation](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#user-impersonation). + +To specify impersonations, use the `access_as` attribute in your agent configuration file and use Kubernetes RBAC rules to manage impersonated account permissions. + +You can impersonate: + +- The agent itself (default). +- The CI/CD job that accesses the cluster. +- A specific user or system account defined within the cluster. + +### Impersonate the agent + +The agent is impersonated by default. You don't need to do anything to impersonate it. + +### Impersonate the CI/CD job that accesses the cluster + +To impersonate the CI/CD job that accesses the cluster, under the `access_as` key, add the `ci_job: {}` key-value. + +When the agent makes the request to the actual Kubernetes API, it sets the +impersonation credentials in the following way: + +- `UserName` is set to `gitlab:ci_job:<job id>`. Example: `gitlab:ci_job:1074499489`. +- `Groups` is set to: + - `gitlab:ci_job` to identify all requests coming from CI jobs. + - The list of IDs of groups the project is in. + - The project ID. + - The slug of the environment this job belongs to. + + Example: for a CI job in `group1/group1-1/project1` where: + + - Group `group1` has ID 23. + - Group `group1/group1-1` has ID 25. + - Project `group1/group1-1/project1` has ID 150. + - Job running in a prod environment. + + Group list would be `[gitlab:ci_job, gitlab:group:23, gitlab:group:25, gitlab:project:150, gitlab:project_env:150:prod]`. + +- `Extra` carries extra information about the request. The following properties are set on the impersonated identity: + +| Property | Description | +| -------- | ----------- | +| `agent.gitlab.com/id` | Contains the agent ID. | +| `agent.gitlab.com/config_project_id` | Contains the agent's configuration project ID. | +| `agent.gitlab.com/project_id` | Contains the CI project ID. | +| `agent.gitlab.com/ci_pipeline_id` | Contains the CI pipeline ID. | +| `agent.gitlab.com/ci_job_id` | Contains the CI job ID. | +| `agent.gitlab.com/username` | Contains the username of the user the CI job is running as. | +| `agent.gitlab.com/environment_slug` | Contains the slug of the environment. Only set if running in an environment. | + +Example to restrict access by the CI/CD job's identity: + +```yaml +ci_access: + projects: + - id: path/to/project + access_as: + ci_job: {} +``` + +### Impersonate a static identity + +For a given connection, you can use a static identity for the impersonation. + +Under the `access_as` key, add the `impersonate` key to make the request using the provided identity. + +The identity can be specified with the following keys: + +- `username` (required) +- `uid` +- `groups` +- `extra` + +See the [official Kubernetes documentation for details](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#user-impersonation). + +## Troubleshooting + +### `kubectl` commands not supported + +The commands `kubectl exec`, `kubectl cp`, and `kubectl attach` are not supported. +Anything that uses these API endpoints does not work, because they use the deprecated +SPDY protocol. +[An issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/346248) to add support for these commands. + +### Grant write permissions to `~/.kube/cache` + +Tools like `kubectl`, Helm, `kpt`, and `kustomize` cache information about the cluster in `~/.kube/cache`. If this directory is not writable, the tool fetches information on each invocation, -making interactions slower and creating unnecessary load on the cluster. Make sure that this directory in the container image -you use is writable for the best experience. +making interactions slower and creating unnecessary load on the cluster. For the best experience, in the +image you use in your .`gitlab-ci.yml` file, ensure this directory is writable. + +### Enable TLS -## Configure the CI/CD Tunnel +If you are on a self-managed GitLab instance, ensure your instance is configured with Transport Layer Security (TLS). -The CI/CD Tunnel is configured directly through the -Agent's configuration file ([`config.yaml`](repository.md)) to: +If you attempt to use `kubectl` without TLS, you might get an error like: -- Authorize [projects](repository.md#authorize-projects-to-use-an-agent) and [groups](repository.md#authorize-groups-to-use-an-agent) to use the same Agent. -- [Run `kubectl` commands using the CI/CD Tunnel](repository.md#run-kubectl-commands-using-the-cicd-tunnel). -- [Restrict access of authorized projects and groups through impersonation strategies](repository.md#use-impersonation-to-restrict-project-and-group-access). +```shell +$ kubectl get pods +error: You must be logged in to the server (the server has asked for the client to provide credentials) +``` diff --git a/doc/user/clusters/agent/gitops.md b/doc/user/clusters/agent/gitops.md new file mode 100644 index 00000000000..8f0e2255121 --- /dev/null +++ b/doc/user/clusters/agent/gitops.md @@ -0,0 +1,140 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Using a GitOps workflow for Kubernetes **(PREMIUM)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/259669) in GitLab 13.7. + +With GitOps, you can manage containerized clusters and applications from a Git repository that: + +- Is the single source of truth of your system. +- Is the single place where you operate your system. + +By combining GitLab, Kubernetes, and GitOps, you can have: + +- GitLab as the GitOps operator. +- Kubernetes as the automation and convergence system. +- GitLab CI/CD for Continuous Integration and the agent for Continuous Deployment. + +This diagram shows the repositories and main actors in a GitOps deployment: + +```mermaid +sequenceDiagram + participant D as Developer + participant A as Application code repository + participant M as Manifest repository + participant K as GitLab agent + participant C as Agent configuration repository + loop Regularly + K-->>C: Grab the configuration + end + D->>+A: Pushing code changes + A->>M: Updating manifest + loop Regularly + K-->>M: Watching changes + M-->>K: Pulling and applying changes + end +``` + +For details, view the [architecture documentation](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/architecture.md#high-level-architecture). + +## GitOps workflow steps + +To update a Kubernetes cluster by using GitOps, complete the following steps. + +1. Ensure you have a working Kubernetes cluster, and that the manifests are in a GitLab project. +1. In the same project, [register and install the GitLab agent](install/index.md). +1. Configure the agent configuration file so that the agent monitors the project for changes to the Kubernetes manifests. + Use the [GitOps configuration reference](#gitops-configuration-reference) for guidance. + +Any time you commit updates to your Kubernetes manifests, the agent updates the cluster. + +## GitOps configuration reference + +The following snippet shows an example of the possible keys and values for the GitOps section of an agent configuration file. + +```yaml +gitops: + manifest_projects: + - id: gitlab-org/cluster-integration/gitlab-agent + default_namespace: my-ns + paths: + # Read all YAML files from this directory. + - glob: '/team1/app1/*.yaml' + # Read all .yaml files from team2/apps and all subdirectories. + - glob: '/team2/apps/**/*.yaml' + # If 'paths' is not specified or is an empty list, the configuration below is used. + - glob: '/**/*.{yaml,yml,json}' + reconcile_timeout: 3600s + dry_run_strategy: none + prune: true + prune_timeout: 3600s + prune_propagation_policy: foreground + inventory_policy: must_match +``` + +| Keyword | Description | +|--|--| +| `manifest_projects` | Projects where your Kubernetes manifests are stored. The agent monitors the files in the repositories in these projects. When manifest files change, the agent deploys the changes to the cluster. | +| `id` | Required. Path to a Git repository that has Kubernetes manifests in YAML or JSON format. No authentication mechanisms are currently supported. | +| `default_namespace` | Namespace to use if not set explicitly in object manifest. Also used for inventory `ConfigMap` objects. | +| `paths` | Repository paths to scan for manifest files. Directories with names that start with a dot `(.)` are ignored. | +| `paths[].glob` | Required. See [doublestar](https://github.com/bmatcuk/doublestar#about) and [the match function](https://pkg.go.dev/github.com/bmatcuk/doublestar/v2#Match) for globbing rules. | +| `reconcile_timeout` | Determines whether the applier should wait until all applied resources have been reconciled, and if so, how long to wait. Default is 3600 seconds (1 hour). | +| `dry_run_strategy` | Determines whether changes [should be performed](https://github.com/kubernetes-sigs/cli-utils/blob/d6968048dcd80b1c7b55d9e4f31fc25f71c9b490/pkg/common/common.go#L68-L89). Can be: `none`, `client`, or `server`. Default is `none`.| +| `prune` | Determines whether pruning of previously applied objects should happen after apply. Default is `true`. | +| `prune_timeout` | Determines whether to wait for all resources to be fully deleted after pruning, and if so, how long to wait. Default is 3600 seconds (1 hour). | +| `prune_propagation_policy` | The deletion propagation policy that [should be used for pruning](https://github.com/kubernetes/apimachinery/blob/44113beed5d39f1b261a12ec398a356e02358307/pkg/apis/meta/v1/types.go#L456-L470). Can be: `orphan`, `background`, or `foreground`. Default is `foreground`. | +| `inventory_policy` | Determines whether an inventory object can take over objects that belong to another inventory object or don't belong to any inventory object. This is done by determining if the apply/prune operation can go through for a resource based on comparison of the `inventory-id` value in the package and the `owning-inventory` annotation (`config.k8s.io/owning-inventory`) [in the live object](https://github.com/kubernetes-sigs/cli-utils/blob/d6968048dcd80b1c7b55d9e4f31fc25f71c9b490/pkg/inventory/policy.go#L12-L66). Can be: `must_match`, `adopt_if_no_inventory`, or `adopt_all`. Default is `must_match`. | + +## Additional resources + +The following documentation and examples can help you get started with a GitOps workflow. + +- [Managing Kubernetes secrets in a GitOps workflow](gitops/secrets_management.md) +- [Application and manifest repository example](https://gitlab.com/gitlab-examples/ops/gitops-demo/hello-world-service-gitops) + +## Troubleshooting + +### Avoiding conflicts when you have multiple projects + +The agent watches each glob pattern set under a project's `paths` section independently, and makes updates to the cluster concurrently. +If changes are found at multiple paths, when the agent attempts to update the cluster, +a conflict can occur. + +To prevent this from happening, consider storing a logical group of manifests in a single place and reference them only once to avoid overlapping globs. + +For example, both of these globs match `*.yaml` files in the root directory +and could cause conflicts: + +```yaml +gitops: + manifest_projects: + - id: project1 + paths: + - glob: '/**/*.yaml' + - glob: '/*.yaml' +``` + +Instead, specify a single glob that matches all `*.yaml` files recursively: + +```yaml +gitops: + manifest_projects: + - id: project1 + paths: + - glob: '/**/*.yaml' +``` + +### Use multiple agents or projects + +If you store your Kubernetes manifests in separate GitLab projects, +update your agent configuration file with the location of these projects. + +WARNING: +The project with the agent's +configuration file can be private or public. Other projects with Kubernetes manifests must be public. Support for private manifest projects is tracked +in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/283885). diff --git a/doc/user/clusters/agent/gitops/secrets_management.md b/doc/user/clusters/agent/gitops/secrets_management.md new file mode 100644 index 00000000000..cf520c881bf --- /dev/null +++ b/doc/user/clusters/agent/gitops/secrets_management.md @@ -0,0 +1,61 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Managing Kubernetes secrets in a GitOps workflow + +You should never store Kubernetes secrets in unencrypted form in a `git` repository. If you use a GitOps workflow, you can follow these steps to securely manage your secrets. + +1. Set up the Sealed Secrets controller to manage secrets. +1. Deploy Docker credentials so the cluster can pull images from the GitLab Container Registry. + +## Prerequisites + +This setup requires: + +- A [GitLab agent for Kubernetes configured for the GitOps workflow](../gitops.md). +- Access to the cluster to finish the setup. + +## Set up the Sealed Secrets controller to manage secrets + +You can use the [Sealed Secrets controller](https://github.com/bitnami-labs/sealed-secrets) to store encrypted secrets securely in a `git` repository. The controller decrypts the secret into a standard Kubernetes `Secret` kind resource. + +1. Go to [the Sealed Secrets release page](https://github.com/bitnami-labs/sealed-secrets/releases) and download the most recent `controller.yaml` file. +1. In GitLab, go to the project that contains your Kubernetes manifests and upload the `controller.yaml` file. +1. Open the agent configuration file (`config.yaml`) and if needed, update the `paths.glob` pattern to match the Sealed Secrets manifest. +1. Commit and push the changes to GitLab. +1. Confirm that the Sealed Secrets controller was installed successfully: + + ```shell + kubectl get pods -lname=sealed-secrets-controller -n kube-system + ``` + +1. Install the `kubeseal` command line utility by following [the Sealed Secrets instructions](https://github.com/bitnami-labs/sealed-secrets#homebrew). +1. Get the public key you need to encrypt secrets without direct access to the cluster: + + ```shell + kubeseal --fetch-cert > public.pem + ``` + +1. Commit the public key to the repository. + +For more details on how the Sealed Secrets controller works, view [the usage instructions](https://github.com/bitnami-labs/sealed-secrets/blob/main/README.md#usage). + +## Deploy Docker credentials + +To deploy containers from the GitLab Container Registry, you must configure the cluster with the proper Docker registry credentials. You can achieve this by deploying a `docker-registry` type secret. + +1. Generate a GitLab token with at least `read-registry` rights. The token can be either a Personal or a Project Access Token. +1. Create a Kubernetes secret manifest YAML file. Update the values as needed: + + ```shell + kubectl create secret docker-registry gitlab-credentials --docker-server=registry.gitlab.example.com --docker-username=<gitlab-username> --docker-password=<gitlab-token> --docker-email=<gitlab-user-email> -n <namespace> --dry-run=client -o yaml > gitlab-credentials.yaml + ``` + +1. Encrypt the secret into a `SealedSecret` manifest: + + ```shell + kubeseal --format=yaml --cert=public.pem < gitlab-credentials.yaml > gitlab-credentials.sealed.yaml + ``` diff --git a/doc/user/clusters/agent/index.md b/doc/user/clusters/agent/index.md index 3fb141e402f..a8ad19df2e1 100644 --- a/doc/user/clusters/agent/index.md +++ b/doc/user/clusters/agent/index.md @@ -8,23 +8,36 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/223061) in GitLab 13.4. > - Support for `grpcs` [introduced](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/issues/7) in GitLab 13.6. -> - Agent Server [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/300960) on GitLab.com under `wss://kas.gitlab.com` through an Early Adopter Program in GitLab 13.10. +> - Agent server [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/300960) on GitLab.com under `wss://kas.gitlab.com` through an Early Adopter Program in GitLab 13.10. > - The agent became available to every project on GitLab.com in GitLab 13.11. > - [Moved](https://gitlab.com/groups/gitlab-org/-/epics/6290) from GitLab Premium to GitLab Free in 14.5. > - [Renamed](https://gitlab.com/groups/gitlab-org/-/epics/7167) from "GitLab Kubernetes Agent" to "GitLab agent for Kubernetes" in GitLab 14.6. You can connect your Kubernetes cluster with GitLab to deploy, manage, -and monitor your cloud-native solutions. You can choose from two primary workflows. +and monitor your cloud-native solutions. -In a **GitOps workflow**, you keep your Kubernetes manifests in GitLab. You install a GitLab agent in your cluster, and +To connect a Kubernetes cluster to GitLab, you must first [install an agent in your cluster](install/index.md). + +The agent runs in the cluster, and you can use it to: + +- Communicate with a cluster, which is behind a firewall or NAT. +- Access API endpoints in a cluster in real time. +- Push information about events happening in the cluster. +- Enable a cache of Kubernetes objects, which are kept up-to-date with very low latency. + +For more details about the agent's purpose and architecture, see the [architecture documentation](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/architecture.md). + +## Workflows + +You can choose from two primary workflows. + +In a [**GitOps** workflow](gitops.md), you keep your Kubernetes manifests in GitLab. You install a GitLab agent in your cluster, and any time you update your manifests, the agent updates the cluster. This workflow is fully driven with Git and is considered pull-based, because the cluster is pulling updates from your GitLab repository. -In a **CI/CD** workflow, you use GitLab CI/CD to query and update your cluster by using the Kubernetes API. +In a [**CI/CD** workflow](ci_cd_tunnel.md), you use GitLab CI/CD to query and update your cluster by using the Kubernetes API. This workflow is considered push-based, because GitLab is pushing requests from GitLab CI/CD to your cluster. -Both of these workflows require you to [install an agent in your cluster](install/index.md). - ## Supported cluster versions GitLab supports the following Kubernetes versions. You can upgrade your @@ -32,8 +45,6 @@ Kubernetes version to a supported version at any time: - 1.20 (support ends on July 22, 2022) - 1.19 (support ends on February 22, 2022) -- 1.18 (support ends on November 22, 2021) -- 1.17 (support ends on September 22, 2021) GitLab supports at least two production-ready Kubernetes minor versions at any given time. GitLab regularly reviews the supported versions and @@ -47,174 +58,15 @@ version. The list of supported versions is based on: Some GitLab features might work on versions not listed here. -## Using Kubernetes with GitOps **(PREMIUM)** - -With GitOps, you can manage containerized clusters and applications from a Git repository that: - -- Is the single source of truth of your system. -- Is the single place where you operate your system. - -By combining GitLab, Kubernetes, and GitOps, you can have: - -- GitLab as the GitOps operator. -- Kubernetes as the automation and convergence system. -- GitLab CI/CD for Continuous Integration and the agent for Continuous Deployment. - -Beyond that, you can use all the features offered by GitLab as -the all-in-one DevOps platform for your product and your team. - -### GitOps workflow **(PREMIUM)** - -The agent uses multiple GitLab projects to provide a flexible workflow -that can suit various needs. This diagram shows these repositories and the main -The agent uses multiple GitLab projects to provide a flexible workflow. -This diagram shows these repositories and the main -actors involved in a deployment: - -```mermaid -sequenceDiagram - participant D as Developer - participant A as Application code repository - participant M as Manifest repository - participant K as GitLab agent - participant C as Agent configuration repository - loop Regularly - K-->>C: Grab the configuration - end - D->>+A: Pushing code changes - A->>M: Updating manifest - loop Regularly - K-->>M: Watching changes - M-->>K: Pulling and applying changes - end -``` - -For details, view the [architecture documentation](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/architecture.md#high-level-architecture). - -To perform GitOps deployments, you need: - -- A properly-configured Kubernetes cluster where the GitLab agent is running. -- A project that contains the agent's configuration file (`config.yaml`) in the repository. - This file tells the agent which repositories to synchronize with the cluster. -- A project that contains Kubernetes manifests. Any changes to manifests are applied to the cluster. - -You can keep the agent's configuration file and Kubernetes manifests in one project, or you can use multiple. - -- One GitLab project (recommended): When you use one project for both the Kubernetes manifests - and the agent's configuration file, the projects can be either private or public. -- Two GitLab projects: When you use two different GitLab projects (one for Kubernetes - manifests and another for the agent's configuration file), the project with Kubernetes manifests must - be public. The project with the agent's configuration file can be either private or public. - -Support for separate private projects is tracked in [this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/283885). - -## Remove an agent - -You can remove an agent by using the [GitLab UI](#remove-an-agent-through-the-gitlab-ui) or the [GraphQL API](#remove-an-agent-with-the-gitlab-graphql-api). - -### Remove an agent through the GitLab UI - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/323055) in GitLab 14.7. - -To remove an agent from the UI: - -1. On the top bar, select **Menu > Projects** and find the project that contains the agent's configuration file. -1. From the left sidebar, select **Infrastructure > Kubernetes clusters**. -1. In the table, in the row for your agent, in the **Options** column, select the vertical ellipsis (**{ellipsis_v}**). -1. Select **Delete agent**. - -### Remove an agent with the GitLab GraphQL API - -1. Get the `<cluster-agent-token-id>` from a query in the interactive GraphQL explorer. - - For GitLab.com, go to <https://gitlab.com/-/graphql-explorer> to open GraphQL Explorer. - - For self-managed GitLab, go to `https://gitlab.example.com/-/graphql-explorer`, replacing `gitlab.example.com` with your instance's URL. - - ```graphql - query{ - project(fullPath: "<full-path-to-agent-configuration-project>") { - clusterAgent(name: "<agent-name>") { - id - tokens { - edges { - node { - id - } - } - } - } - } - } - ``` - -1. Remove an agent record with GraphQL by deleting the `clusterAgentToken`. - - ```graphql - mutation deleteAgent { - clusterAgentDelete(input: { id: "<cluster-agent-id>" } ) { - errors - } - } - - mutation deleteToken { - clusterAgentTokenDelete(input: { id: "<cluster-agent-token-id>" }) { - errors - } - } - ``` - -1. Verify whether the removal occurred successfully. If the output in the Pod logs includes `unauthenticated`, it means that the agent was successfully removed: - - ```json - { - "level": "warn", - "time": "2021-04-29T23:44:07.598Z", - "msg": "GetConfiguration.Recv failed", - "error": "rpc error: code = Unauthenticated desc = unauthenticated" - } - ``` - -1. Delete the agent in your cluster: - - ```shell - kubectl delete -n gitlab-kubernetes-agent -f ./resources.yml - ``` - -## Migrating to the agent from the legacy certificate-based integration - -Find out how to [migrate to the agent for Kubernetes](../../infrastructure/clusters/migrate_to_gitlab_agent.md) from the certificate-based integration. - -## Kubernetes network security alerts **(ULTIMATE)** - -> [Deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) in GitLab 15.0. - -WARNING: -Cilium integration is in its end-of-life process. It's [deprecated](https://gitlab.com/groups/gitlab-org/-/epics/7476) -for use in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) -in GitLab 15.0. - -The agent for Kubernetes also provides an integration with Cilium. This integration provides a simple way to -generate network policy-related alerts and to surface those alerts in GitLab. - -Several components work in concert for the agent to generate the alerts: - -- A working Kubernetes cluster. -- Cilium integration through either of these options: - - Installation through [cluster management template](../../project/clusters/protect/container_network_security/quick_start_guide.md#use-the-cluster-management-template-to-install-cilium). - - Enablement of [hubble-relay](https://docs.cilium.io/en/v1.8/concepts/overview/#hubble) on an - existing installation. -- One or more network policies through any of these options: - - Use the [Container Network Policy editor](../../application_security/policies/index.md#container-network-policy-editor) to create and manage policies. - - Use an [AutoDevOps](../../application_security/policies/index.md#container-network-policy) configuration. - - Add the required labels and annotations to existing network policies. -- A configuration repository with [Cilium configured in `config.yaml`](repository.md#surface-network-security-alerts-from-cluster-to-gitlab) +## Migrate to the agent from the legacy certificate-based integration -The setup process follows the same [agent's installation steps](install/index.md), -with the following differences: - -- When you define a configuration repository, you must do so with [Cilium settings](repository.md#surface-network-security-alerts-from-cluster-to-gitlab). -- You do not need to specify the `gitops` configuration section. +Read about how to [migrate to the agent for Kubernetes](../../infrastructure/clusters/migrate_to_gitlab_agent.md) from the certificate-based integration. ## Related topics +- [GitOps workflow](gitops.md) +- [GitLab CI/CD workflow](ci_cd_tunnel.md) +- [Install the agent](install/index.md) +- [Work with the agent](repository.md) - [Troubleshooting](troubleshooting.md) -- [Contribute to the GitLab agent's development](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/tree/master/doc) +- [Contribute to the agent's development](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/tree/master/doc) diff --git a/doc/user/clusters/agent/install/index.md b/doc/user/clusters/agent/install/index.md index 4d196e57f8f..fca80a4a291 100644 --- a/doc/user/clusters/agent/install/index.md +++ b/doc/user/clusters/agent/install/index.md @@ -4,92 +4,94 @@ group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Install the GitLab Agent **(FREE)** +# Installing the agent for Kubernetes **(FREE)** > - [Moved](https://gitlab.com/groups/gitlab-org/-/epics/6290) from GitLab Premium to GitLab Free in 14.5. > - [Introduced](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/merge_requests/594) multi-arch images in GitLab 14.8. The first multi-arch release is `v14.8.1`. It supports AMD64 and ARM64 architectures. +> - [Introduced](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/merge_requests/603) ARM architecture support in GitLab 14.9. -To connect a cluster to GitLab, you need to install the GitLab Agent -onto your cluster. +To connect a Kubernetes cluster to GitLab, you must install an agent in your cluster. ## Prerequisites -- An existing Kubernetes cluster. If you don't have a cluster yet, you can create a new cluster on cloud providers, such as: +Before you can install the agent in your cluster, you need: + +- An existing Kubernetes cluster. If you don't have a cluster, you can create one on a cloud provider, like: - [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine/docs/quickstart) - [Amazon Elastic Kubernetes Service (EKS)](https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html) - [Digital Ocean](https://docs.digitalocean.com/products/kubernetes/quickstart/) -- On self-managed GitLab instances, a GitLab administrator needs to set up the [GitLab Agent Server (KAS)](../../../../administration/clusters/kas.md). +- On self-managed GitLab instances, a GitLab administrator must set up the [agent server](../../../../administration/clusters/kas.md). + On GitLab.com, the agent server is available at `wss://kas.gitlab.com`. ## Installation steps -To install the GitLab Agent on your cluster: +To install the agent in your cluster: -1. [Define a configuration repository](#define-a-configuration-repository). -1. [Register the Agent with GitLab](#register-the-agent-with-gitlab). -1. [Install the Agent onto the cluster](#install-the-agent-onto-the-cluster). +1. [Register the agent with GitLab](#register-the-agent-with-gitlab). +1. [Install the agent in your cluster](#install-the-agent-in-the-cluster). -<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Watch a GitLab 14.2 [walking-through video](https://www.youtube.com/watch?v=XuBpKtsgGkE) with this process. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> Watch a GitLab 14.2 [walk-through of this process](https://www.youtube.com/watch?v=XuBpKtsgGkE). -When you complete the installation process, you can -[view your Agent's status and activity information](#view-your-agents). -You can also [configure](#configure-the-agent) it to your needs. +### Register the agent with GitLab -### Define a configuration repository +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5786) in GitLab 14.1, you can create a new agent record directly from the GitLab UI. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/347240) in GitLab 14.9, the agent can be registered without creating an agent configuration file. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/259669) in GitLab 13.7, the Agent manifest configuration can be added to multiple directories (or subdirectories) of its repository. -> - Group authorization was [introduced](https://gitlab.com/groups/gitlab-org/-/epics/5784) in GitLab 14.3. +You must register an agent with GitLab. -To create an Agent, you need a GitLab repository to hold its -configuration file. If you already have a repository holding your -cluster's manifest files, you can use it to store your -Agent's configuration file and sync them with no further steps. +Prerequisites: -#### Create the Agent's configuration file +- For a [GitLab CI/CD workflow](../ci_cd_tunnel.md), ensure that + [GitLab CI/CD is enabled](../../../../ci/enable_or_disable_ci.md#enable-cicd-in-a-project). -To create an Agent, go to the repository where you want to store -it and add the Agent's configuration file under: +To register an agent with GitLab: -```plaintext -.gitlab/agents/<agent-name>/config.yaml -``` +1. On the top bar, select **Menu > Projects** and find your project. +1. From the left sidebar, select **Infrastructure > Kubernetes clusters**. +1. Select **Actions**. +1. From the **Select an agent** dropdown list: + - If you want to create a configuration with CI/CD defaults, type a name for the agent. + - If you already have an [agent configuration file](#create-an-agent-configuration-file), select it from the list. +1. Select **Register an agent**. +1. GitLab generates an access token for the agent. Securely store this token. You need it to install the agent in your cluster and to [update the agent](#update-the-agent-version) to another version. +1. Copy the command under **Recommended installation method**. You need it when you use the one-liner installation method to install the agent in your cluster. -You **don't have to add any content** to this file at the moment you -create it. The fact that the file exists tells GitLab that this is -an Agent. You can edit it later to [configure the Agent](#configure-the-agent). +### Create an agent configuration file -When creating this file, pay special attention to: +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/259669) in GitLab 13.7, the agent configuration file can be added to multiple directories (or subdirectories) of the repository. +> - Group authorization was [introduced](https://gitlab.com/groups/gitlab-org/-/epics/5784) in GitLab 14.3. -- The [Agent's naming format](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/identity_and_auth.md#agent-identity-and-name). -- The file extension: use the `.yaml` extension (`config.yaml`). The `.yml` extension is **not** recognized. +You can use an agent configuration file to specify details about your implementation. +Creating a file is optional but is needed if: -### Register the Agent with GitLab +- You use [a GitOps workflow](../gitops.md#gitops-configuration-reference) and you want a more advanced configuration. +- You use a GitLab CI/CD workflow. In that workflow, you must [authorize the agent](../ci_cd_tunnel.md#authorize-the-agent). -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5786) in GitLab 14.1, you can create a new Agent record directly from the GitLab UI. +To create an agent configuration file, go to the GitLab project. In the repository, create a file called `config.yaml` at this path: -Now that you've created your Agent's configuration file, register it -with GitLab. -When you register the Agent, GitLab generates a token that you need for -installing the Agent onto your cluster. +```plaintext +.gitlab/agents/<agent-name>/config.yaml +``` -In GitLab, go to the project where you added your Agent's configuration -file and: +- Ensure the agent name follows the [naming convention](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/identity_and_auth.md#agent-identity-and-name). +- Ensure the filename has the `.yaml` file extension (`config.yaml`). The `.yml` extension is not accepted. +- Add content to the `config.yaml` file: + - For a GitOps workflow, view [the configuration reference](../gitops.md#gitops-configuration-reference) for details. + - For a GitLab CI/CD workflow, you can leave the file blank for now. -1. Ensure that [GitLab CI/CD is enabled in your project](../../../../ci/enable_or_disable_ci.md#enable-cicd-in-a-project). -1. From your project's sidebar, select **Infrastructure > Kubernetes clusters**. -1. Select **Actions**. -1. From the **Select an Agent** dropdown list, select the Agent you want to register and select **Register an Agent**. -1. GitLab generates a registration token for this Agent. Securely store this secret token, as you need it to install the Agent onto your cluster and to [update the Agent](#update-the-agent-version) to another version. -1. Copy the command under **Recommended installation method**. You need it to install the Agent onto your cluster through the one-liner installation method. +The agent bootstraps with the GitLab installation URL and an access token, +and you provide the rest of the configuration in your repository, following +Infrastructure as Code (IaaC) best practices. -### Install the Agent onto the cluster +### Install the agent in the cluster -To connect your cluster to GitLab, install the registered Agent -onto your cluster. To install it, you can use either: +To connect your cluster to GitLab, install the registered agent +in your cluster. To install it, you can use either: - [The one-liner installation method](#one-liner-installation). - [The advanced installation method](#advanced-installation). -You can use the one-liner installation for trying to use the Agent for the first time, to do internal setups with +You can use the one-liner installation for trying to use the agent for the first time, to do internal setups with high trust, and to quickly get started. For long-term production usage, you may want to use the advanced installation method to benefit from more configuration options. @@ -99,35 +101,45 @@ The one-liner installation is the simplest process, but you need Docker installed locally. If you don't have it, you can either install it or opt to the [advanced installation method](#advanced-installation). -To install the Agent on your cluster using the one-liner installation: +To install the agent on your cluster using the one-liner installation: -1. In your computer, open the terminal and connect to your cluster. +1. In your computer, open a terminal and [connect to your cluster](https://kubernetes.io/docs/tasks/access-application-cluster/access-cluster/). 1. Run the command you copied when registering your cluster in the previous step. Optionally, you can [customize the one-liner installation command](#customize-the-one-liner-installation). ##### Customize the one-liner installation -The one-liner command generated by GitLab: +By default, the one-liner command generated by GitLab: -- Creates a namespace for the deployment (`gitlab-kubernetes-agent`). +- Creates a namespace for the deployment (`gitlab-agent`). - Sets up a service account with `cluster-admin` rights (see [how to restrict this service account](#customize-the-permissions-for-the-agentk-service-account)). -- Creates a `Secret` resource for the Agent's registration token. +- Creates a `Secret` resource for the agent's access token. - Creates a `Deployment` resource for the `agentk` pod. -You can edit these parameters according to your needs to customize the -one-liner installation command at the command line. To find all available -options, run in your terminal: +You can edit these parameters to customize the one-liner installation command. +To view all available options, open a terminal and run this command: ```shell docker run --pull=always --rm registry.gitlab.com/gitlab-org/cluster-integration/gitlab-agent/cli:stable generate --help + +Usage: + cli generate [flags] + +Flags: + --agent-token string Access token registered for agent + --agent-version string Version of the agentk image to use (default "v14.8.1") + -h, --help help for generate + --kas-address string GitLab agent server for Kubernetes address + --name-prefix string The prefix to use for names of Kubernetes objects + --namespace string Kubernetes namespace to create resources in (default "gitlab-agent") + --no-rbac Do not include corresponding Roles and RoleBindings for the agent service account ``` WARNING: -`--agent-version stable` can be used to refer to the latest stable -release at the time when the command runs. It's fine for testing -purposes but for production please make sure to specify a matching -version explicitly. +Use `--agent-version stable` to refer to the latest stable +release at the time when the command runs. For production, however, +you should explicitly specify a matching version. #### Advanced installation @@ -135,105 +147,76 @@ For advanced installation options, use [the `kpt` installation method](https://g ##### Customize the permissions for the `agentk` service account -The GitLab Agent allows you to fully own your cluster and grant GitLab -the permissions you want. Still, to facilitate the process, by default the -generated manifests provide `cluster-admin` rights to the Agent. +You own your cluster and can grant GitLab the permissions you want. +By default, however, the generated manifests provide `cluster-admin` rights to the agent. -You can restrict the Agent's access rights using Kustomize overlays. [An example is commented out](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/build/deployment/gitlab-agent/cluster/kustomization.yaml) in the `kpt` package you retrieved as part of the installation. +You can restrict the agent's access rights by using Kustomize overlays. [An example is commented out](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/build/deployment/gitlab-agent/cluster/kustomization.yaml) in the `kpt` package you retrieved as part of the installation. -To create restricted permissions: +To restrict permissions: 1. Copy the `cluster` directory. 1. Edit the `kustomization.yaml` and `components/*` files based on your requirements. 1. Run `kustomize build <your copied directory> | kubectl apply -f -` to apply your configuration. -The above setup allows you to regularly update from the upstream package using `kpt pkg update gitlab-agent --strategy resource-merge` and maintain your customizations at the same time. +#### Update the advanced installation base layer -## Configure the Agent +Now you can update from the upstream package by using `kpt pkg update gitlab-agent --strategy resource-merge`. +When the advanced installation setup changes, you will not need to change your custom overlays. -When successfully installed, you can [configure the Agent](../repository.md) -by editing its configuration file. -When you update the configuration file, GitLab transmits the -information to the cluster automatically without downtime. +## Install multiple agents in your cluster -## View your Agents +For total separation between teams, you might need to run multiple `agentk` instances in your cluster. +You might want multiple agents so you can restrict RBAC for every `agentk` deployment. -> The version of installed `agentk` shown on the Agent tab [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340882) in GitLab 14.8. -If you have at least the Developer role, you can access the Agent's -configuration repository and view the Agent's list: +To install multiple agents, follow the +[advanced installation steps](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/tree/master/build/deployment/gitlab-agent) +a second time and: -1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. -1. Select **Agent** tab to view clusters connected to GitLab through the Agent. +1. Change the agent name and create a new configuration file. +1. Register the new agent. You receive a new access token. Each token should be used only with one agent. +1. Change the namespace or prefix you use for the installation. -On this page, you can view: +You should also change the RBAC for the installed `agentk`. -- All the registered Agents for the current project. -- The connection status. -- The version of `agentk` installed on your cluster. -- The path to each Agent's configuration file. - -Furthermore, if you select one of the Agents on your list, you can view its -[activity information](#view-the-agents-activity-information). - -### View the Agent's activity information - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/277323) in GitLab 14.6. - -The activity logs help you to identify problems and get the information -you need for troubleshooting. You can see events from a week before the -current date. To access an Agent's activity: - -1. In your Agent's repository, go to the Agents list as described [above](#view-your-agents). -1. Select the Agent you want to see the activity. - -The activity list includes: - -- Agent registration events: when a new token is **created**. -- Connection events: when an Agent is successfully **connected** to a cluster. - -Note that the connection status is logged when you connect an Agent for -the first time or after more than an hour of inactivity. - -To check what else is planned for the Agent's UI and provide feedback, -see the [related epic](https://gitlab.com/groups/gitlab-org/-/epics/4739). +## Example projects -### View vulnerabilities in cluster images **(ULTIMATE)** +The following example projects can help you get started with the agent. -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6346) in GitLab 14.8 [with a flag](../../../../administration/feature_flags.md) named `cluster_vulnerabilities`. Enabled by default. +- [Configuration repository with minimal manifests](https://gitlab.com/gitlab-examples/ops/gitops-demo/k8s-agents) +- [Distinct application and manifest repository example](https://gitlab.com/gitlab-examples/ops/gitops-demo/hello-world-service-gitops) +- [Auto DevOps setup that uses the CI/CD workflow](https://gitlab.com/gitlab-examples/ops/gitops-demo/hello-world-service) +- [Cluster management project template example that uses the CI/CD workflow](https://gitlab.com/gitlab-examples/ops/gitops-demo/cluster-management) -Users with at least the [Developer role](../../../permissions.md) -can view cluster vulnerabilities. You can access them through the [vulnerability report](../../../application_security/vulnerabilities/index.md) -or in your cluster's image through the following process: +## Upgrades and version compatibility -1. Configure [cluster image scanning](../../../application_security/cluster_image_scanning/index.md) - to your build process. -1. Go to your Agent's configuration repository. -1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. -1. Select the **Agent** tab. -1. Select the Agent you want to see the vulnerabilities for. +The agent has two major components: `agentk` and `kas`. +GitLab provides `kas` installers built into the various GitLab installation methods. +The required `kas` version corresponds to the GitLab `major.minor` (X.Y) versions. - +At the same time, `agentk` and `kas` can differ by 1 minor version in either direction. For example, +`agentk` 14.4 supports `kas` 14.3, 14.4, and 14.5 (regardless of the patch). -## Create multiple Agents +A feature introduced in a given GitLab minor version might work with other `agentk` or `kas` versions. +To ensure it works, use at least the same `agentk` and `kas` minor version. For example, +if your GitLab version is 14.2, use at least `agentk` 14.2 and `kas` 14.2. -You can create and install multiple Agents using the same process -documented above. Give each Agent's configuration file a unique name -and you're good to go. You can create multiple Agents, for example: +We recommend upgrading your `kas` installations together with GitLab instances' upgrades, and to +[upgrade the `agentk` installations](#update-the-agent-version) after upgrading GitLab. -- To reach your cluster from different projects. -- To connect multiple clusters to GitLab. +The available `agentk` and `kas` versions are available in +[the Container Registry](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/container_registry/). -## Update the Agent version +### Update the agent version -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340882) in GitLab 14.8, GitLab warns you on the Agent's list page to update the Agent version installed on your cluster. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340882) in GitLab 14.8, GitLab warns you on the agent's list page to update the agent version installed on your cluster. -To update the Agent's version on your cluster, you need to re-run the [installation command](#install-the-agent-onto-the-cluster) +To update the agent's version, re-run the [installation command](#install-the-agent-in-the-cluster) with a newer `--agent-version`. Make sure to specify the other required parameters: `--kas-address`, `--namespace`, and `--agent-token`. -You can find the available `agentk` versions in [the container registry](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/container_registry/1223205?sort=desc). +The available `agentk` versions are in [the Container Registry](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/container_registry/1223205?sort=desc). -If you don't have access to your Agent's token, you can retrieve it from your cluster: +If you don't have access to your agent's access token, you can retrieve it from your cluster: -1. On your computer, open the terminal and connect to your cluster. +1. Open a terminal and connect to your cluster. 1. To retrieve the namespace, run: ```shell @@ -246,33 +229,8 @@ If you don't have access to your Agent's token, you can retrieve it from your cl kubectl -n <namespace> get secrets ``` -1. To retrieve the token, run: +1. To retrieve the access token, run: ```shell kubectl -n <namespace> get secret <secret-name> --template={{.data.token}} | base64 --decode ``` - -## Example projects - -The following example projects can help you get started with the Agent. - -- [Configuration repository](https://gitlab.com/gitlab-org/configure/examples/kubernetes-agent) -- This basic GitOps example deploys NGINX: [Manifest repository](https://gitlab.com/gitlab-org/configure/examples/gitops-project) - -## Upgrades and version compatibility - -The Agent is comprised of two major components: `agentk` and `kas`. -As we provide `kas` installers built into the various GitLab installation methods, the required `kas` version corresponds to the GitLab `major.minor` (X.Y) versions. - -At the same time, `agentk` and `kas` can differ by 1 minor version in either direction. For example, -`agentk` 14.4 supports `kas` 14.3, 14.4, and 14.5 (regardless of the patch). - -A feature introduced in a given GitLab minor version might work with other `agentk` or `kas` versions. -To make sure that it works, use at least the same `agentk` and `kas` minor version. For example, -if your GitLab version is 14.2, use at least `agentk` 14.2 and `kas` 14.2. - -We recommend upgrading your `kas` installations together with GitLab instances' upgrades, and to -[upgrade the `agentk` installations](#update-the-agent-version) after upgrading GitLab. - -The available `agentk` and `kas` versions can be found in -[the container registry](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/container_registry/). diff --git a/doc/user/clusters/agent/repository.md b/doc/user/clusters/agent/repository.md index a9d9fd1c13d..3f743d34e0a 100644 --- a/doc/user/clusters/agent/repository.md +++ b/doc/user/clusters/agent/repository.md @@ -1,335 +1,169 @@ --- stage: Configure group: Configure -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Agent configuration repository **(FREE)** +# Working with the agent for Kubernetes **(FREE)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/259669) in GitLab 13.7. -> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3834) in GitLab 13.11, the GitLab Agent became available on GitLab.com. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/3834) in GitLab 13.11, the GitLab agent became available on GitLab.com. > - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5784) the `ci_access` attribute in GitLab 14.3. > - [Moved](https://gitlab.com/groups/gitlab-org/-/epics/6290) from GitLab Premium to GitLab Free in 14.5. > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/332227) in GitLab 14.0, the `resource_inclusions` and `resource_exclusions` attributes were removed and `reconcile_timeout`, `dry_run_strategy`, `prune`, `prune_timeout`, `prune_propagation_policy`, and `inventory_policy` attributes were added. -The [GitLab Agent](index.md) supports hosting your configuration for -multiple agents in a single repository. These agents can be running -in the same cluster or in multiple clusters, and potentially with more than one agent per cluster. +## View your agents -The Agent bootstraps with the GitLab installation URL and an authentication token, -and you provide the rest of the configuration in your repository, following -Infrastructure as Code (IaaC) best practices. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340882) in GitLab 14.8, the installed `agentk` version is displayed on the **Agent** tab. -A minimal repository layout looks like this, with `my-agent-1` as the name -of your Agent: +Prerequisite: -```plaintext -|- .gitlab - |- agents - |- my-agent-1 - |- config.yaml -``` - -Make sure that `<agent-name>` conforms to the [Agent's naming format](https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/blob/master/doc/identity_and_auth.md#agent-identity-and-name). +- You must have at least the Developer role. -## Synchronize manifest projects **(PREMIUM)** +To view the list of agents: -Your `config.yaml` file contains a `gitops` section, which contains a `manifest_projects` -section. Each `id` in the `manifest_projects` section is the path to a Git repository -with Kubernetes resource definitions in YAML or JSON format. The Agent monitors -each project you declare, and when the project changes, GitLab deploys the changes -using the Agent. +1. On the top bar, select **Menu > Projects** and find the project that contains your agent configuration file. +1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. +1. Select **Agent** tab to view clusters connected to GitLab through the agent. -WARNING: -When using separate GitLab projects for manifest files and configuration repository, the manifests project must be public. +On this page, you can view: -To use multiple YAML files, specify a `paths` attribute in the `gitops.manifest_projects` section. +- All the registered agents for the current project. +- The connection status. +- The version of `agentk` installed on your cluster. +- The path to each agent configuration file. -```yaml -gitops: - # Manifest projects are watched by the agent. Whenever a project changes, - # GitLab deploys the changes using the agent. - manifest_projects: - # No authentication mechanisms are currently supported. - # The `id` is a path to a Git repository with Kubernetes resource definitions - # in YAML or JSON format. - - id: gitlab-org/cluster-integration/gitlab-agent - # Namespace to use if not set explicitly in object manifest. - # Also used for inventory ConfigMap objects. - default_namespace: my-ns - # Paths inside of the repository to scan for manifest files. - # Directories with names starting with a dot are ignored. - paths: - # Read all .yaml files from team1/app1 directory. - # See https://github.com/bmatcuk/doublestar#about and - # https://pkg.go.dev/github.com/bmatcuk/doublestar/v2#Match for globbing rules. - - glob: '/team1/app1/*.yaml' - # Read all .yaml files from team2/apps and all subdirectories - - glob: '/team2/apps/**/*.yaml' - # If 'paths' is not specified or is an empty list, the configuration below is used - - glob: '/**/*.{yaml,yml,json}' - # Reconcile timeout defines whether the applier should wait - # until all applied resources have been reconciled, and if so, - # how long to wait. - reconcile_timeout: 3600s # 1 hour by default - # Dry run strategy defines whether changes should actually be performed, - # or if it is just talk and no action. - # https://github.com/kubernetes-sigs/cli-utils/blob/d6968048dcd80b1c7b55d9e4f31fc25f71c9b490/pkg/common/common.go#L68-L89 - # Can be: none, client, server - dry_run_strategy: none # 'none' by default - # Prune defines whether pruning of previously applied - # objects should happen after apply. - prune: true # enabled by default - # Prune timeout defines whether we should wait for all resources - # to be fully deleted after pruning, and if so, how long we should - # wait. - prune_timeout: 3600s # 1 hour by default - # Prune propagation policy defines the deletion propagation policy - # that should be used for pruning. - # https://github.com/kubernetes/apimachinery/blob/44113beed5d39f1b261a12ec398a356e02358307/pkg/apis/meta/v1/types.go#L456-L470 - # Can be: orphan, background, foreground - prune_propagation_policy: foreground # 'foreground' by default - # Inventory policy defines if an inventory object can take over - # objects that belong to another inventory object or don't - # belong to any inventory object. - # This is done by determining if the apply/prune operation - # can go through for a resource based on the comparison - # the inventory-id value in the package and the owning-inventory - # annotation (config.k8s.io/owning-inventory) in the live object. - # https://github.com/kubernetes-sigs/cli-utils/blob/d6968048dcd80b1c7b55d9e4f31fc25f71c9b490/pkg/inventory/policy.go#L12-L66 - # Can be: must_match, adopt_if_no_inventory, adopt_all - inventory_policy: must_match # 'must_match' by default -``` +## View an agent's activity information -### Using multiple manifest projects +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/277323) in GitLab 14.6. -Storing Kubernetes manifests in more than one repository can be handy, for example: +The activity logs help you to identify problems and get the information +you need for troubleshooting. You can see events from a week before the +current date. To view an agent's activity: -- You may store manifests for different applications in separate repositories. -- Different teams can work on manifests of independent projects in separate repositories. +1. On the top bar, select **Menu > Projects** and find the project that contains your agent configuration file. +1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. +1. Select the agent you want to see activity for. -To use multiple repositories as the source of Kubernetes manifests, specify them in the list of -`manifest_projects` in your `config.yaml`: +The activity list includes: -```yaml -gitops: - manifest_projects: - - id: group1/project1 - - id: group2/project2 -``` +- Agent registration events: When a new token is **created**. +- Connection events: When an agent is successfully **connected** to a cluster. -Note that repositories are synchronized **concurrently** and **independently** from each other, -which means that, ideally, there should **not** be any dependencies shared by these repositories. -Storing a logical group of manifests in a single repository may work better than distributing it across several -repositories. +The connection status is logged when you connect an agent for +the first time or after more than an hour of inactivity. -You cannot use a single repository as a source for multiple concurrent synchronization -operations. If such functionality is needed, you may use multiple agents reading -manifests from the same repository. +View and provide feedback about the UI in [this epic](https://gitlab.com/groups/gitlab-org/-/epics/4739). -Ensure not to specify "overlapping" globs to avoid synchronizing the same files more than once. -This is detected by the Agent and leads to an error. +## Debug the agent -INCORRECT - both globs match `*.yaml` files in the root directory: +To debug the cluster-side component (`agentk`) of the agent, set the log +level according to the available options: -```yaml -gitops: - manifest_projects: - - id: project1 - paths: - - glob: '/**/*.yaml' - - glob: '/*.yaml' -``` +- `off` +- `warning` +- `error` +- `info` +- `debug` -CORRECT - single globs matches all `*.yaml` files recursively: +The log level defaults to `info`. You can change it by using a top-level `observability` +section in the configuration file, for example: ```yaml -gitops: - manifest_projects: - - id: project1 - paths: - - glob: '/**/*.yaml' +observability: + logging: + level: debug ``` -## Authorize projects and groups to use an Agent - -With the [CI/CD Tunnel](ci_cd_tunnel.md), you can authorize [projects](#authorize-projects-to-use-an-agent) -and [groups](#authorize-groups-to-use-an-agent) to use an Agent. - -Then, you can reach your cluster from authorized projects and [run Kubernetes commands from GitLab CI/CD scripts](#run-kubectl-commands-using-the-cicd-tunnel) -in these projects. - -### Authorize projects to use an Agent - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327850) in GitLab 14.4. - -To grant projects access to the Agent through the CI/CD Tunnel: - -1. Go to your Agent's configuration repository. -1. Edit the Agent's configuration file (`config.yaml`). -1. Add the `projects` attribute into `ci_access`. -1. Identify the project through its path: - - ```yaml - ci_access: - projects: - - id: path/to/project +## Reset the agent token + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327152) in GitLab 14.9. + +To reset the agent token without downtime: + +1. Create a new token: + 1. On the top bar, select **Menu > Projects** and find your project. + 1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. + 1. Select the agent you want to create a token for. + 1. On the **Tokens** tab, select **Create token**. + 1. Enter token's name and description (optional) and select **Create token**. +1. Securely store the generated token. +1. Use the token to [install the agent in your cluster](install/index.md#install-the-agent-in-the-cluster) and to [update the agent](install/index.md#update-the-agent-version) to another version. +1. Delete the token you're no longer using. + +## Remove an agent + +You can remove an agent by using the [GitLab UI](#remove-an-agent-through-the-gitlab-ui) or the +[GraphQL API](#remove-an-agent-with-the-gitlab-graphql-api). The agent and any associated tokens +are removed from GitLab, but no changes are made in your Kubernetes cluster. You must +clean up those resources manually. + +### Remove an agent through the GitLab UI + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/323055) in GitLab 14.7. + +To remove an agent from the UI: + +1. On the top bar, select **Menu > Projects** and find the project that contains the agent configuration file. +1. From the left sidebar, select **Infrastructure > Kubernetes clusters**. +1. In the table, in the row for your agent, in the **Options** column, select the vertical ellipsis (**{ellipsis_v}**). +1. Select **Delete agent**. + +### Remove an agent with the GitLab GraphQL API + +1. Get the `<cluster-agent-token-id>` from a query in the interactive GraphQL explorer. + - For GitLab.com, go to <https://gitlab.com/-/graphql-explorer> to open GraphQL Explorer. + - For self-managed GitLab, go to `https://gitlab.example.com/-/graphql-explorer`, replacing `gitlab.example.com` with your instance's URL. + + ```graphql + query{ + project(fullPath: "<full-path-to-agent-configuration-project>") { + clusterAgent(name: "<agent-name>") { + id + tokens { + edges { + node { + id + } + } + } + } + } + } ``` -### Authorize groups to use an Agent +1. Remove an agent record with GraphQL by deleting the `clusterAgentToken`. -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/5784) in GitLab 14.3. + ```graphql + mutation deleteAgent { + clusterAgentDelete(input: { id: "<cluster-agent-id>" } ) { + errors + } + } -To grant access to all projects within a group: - -1. Go to your Agent's configuration repository. -1. Edit the Agent's configuration file (`config.yaml`). -1. Add the `groups` attribute into `ci_access`. -1. Identify the group or subgroup through its path: - - ```yaml - ci_access: - groups: - - id: path/to/group/subgroup + mutation deleteToken { + clusterAgentTokenDelete(input: { id: "<cluster-agent-token-id>" }) { + errors + } + } ``` -## Run `kubectl` commands using the CI/CD Tunnel - -After you authorize your project or group to use the Agent, you need to -configure the project's `.gitlab-ci.yaml` file to access the Agent. -This makes it possible to deploy applications to your cluster and run -any Kubernetes-specific commands from the authorized project. - -First, configure your Agent: - -1. Go to your Agent's configuration repository. -1. Edit your Agent's `config.yaml` file authorizing the [project](#authorize-projects-to-use-an-agent) or [group](#authorize-groups-to-use-an-agent) you want to run Kubernetes commands from. - -Then, configure the other project: - -1. Go to the project where you want to run Kubernetes commands from. -1. Edit your project's `.gitlab-ci.yml` file. -1. Set your Agent's context in the first command of the script with the format `path/to/agent/repository:agent-name`. -1. Run Kubernetes commands. - -For example: - -```yaml - deploy: - image: - name: bitnami/kubectl:latest - entrypoint: [""] - script: - - kubectl config use-context path/to/agent/repository:agent-name - - kubectl get pods -``` - -When you use the Agent, KubeContexts are named as `path/to/agent/repository:agent-name`. - -To get the list of available contexts: - -1. Open your terminal and connect to your cluster. -1. Run `kubectl config get-contexts`. - -### `kubectl` commands not supported - -The commands `kubectl exec`, `kubectl cp`, and `kubectl attach` are not supported by the CI/CD tunnel. -Anything else that uses the same API endpoints does not work either as they use the deprecated -SPDY protocol. -We [plan to add support for these features](https://gitlab.com/gitlab-org/gitlab/-/issues/346248) -in a future version of GitLab. - -### `kubectl` requires TLS - -`kubectl` would never send credentials over an unencrypted connection. Self-managed users should ensure that their -GitLab instance is configured with TLS for the CI/CD tunnel feature to work. Trying to use it without TLS -would produce errors: - -```shell -$ kubectl get pods -error: You must be logged in to the server (the server has asked for the client to provide credentials) -``` - -## Use impersonation to restrict project and group access **(PREMIUM)** - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345014) in GitLab 14.5. - -By default, the [CI/CD Tunnel](ci_cd_tunnel.md) inherits all the permissions from the service account used to install the -Agent in the cluster. -To restrict access to your cluster, you can use [impersonation](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#user-impersonation). - -To specify impersonations, use the `access_as` attribute in your Agent's configuration file and use Kubernetes RBAC rules to manage impersonated account permissions. - -You can impersonate: - -- The Agent itself (default). -- The CI job that accesses the cluster. -- A specific user or system account defined within the cluster. - -### Impersonate the Agent - -The Agent is impersonated by default. You don't need to do anything to impersonate it. - -### Impersonate the CI job that accesses the cluster - -To impersonate the CI job that accesses the cluster, add the `ci_job: {}` key-value -under the `access_as` key. - -When the agent makes the request to the actual Kubernetes API, it sets the -impersonation credentials in the following way: - -- `UserName` is set to `gitlab:ci_job:<job id>`. Example: `gitlab:ci_job:1074499489`. -- `Groups` is set to: - - `gitlab:ci_job` to identify all requests coming from CI jobs. - - The list of IDs of groups the project is in. - - The project ID. - - The slug of the environment this job belongs to. +1. Verify whether the removal occurred successfully. If the output in the Pod logs includes `unauthenticated`, it means that the agent was successfully removed: - Example: for a CI job in `group1/group1-1/project1` where: - - - Group `group1` has ID 23. - - Group `group1/group1-1` has ID 25. - - Project `group1/group1-1/project1` has ID 150. - - Job running in a prod environment. - - Group list would be `[gitlab:ci_job, gitlab:group:23, gitlab:group:25, gitlab:project:150, gitlab:project_env:150:prod]`. - -- `Extra` carries extra information about the request. The following properties are set on the impersonated identity: - -| Property | Description | -| -------- | ----------- | -| `agent.gitlab.com/id` | Contains the agent ID. | -| `agent.gitlab.com/config_project_id` | Contains the agent's configuration project ID. | -| `agent.gitlab.com/project_id` | Contains the CI project ID. | -| `agent.gitlab.com/ci_pipeline_id` | Contains the CI pipeline ID. | -| `agent.gitlab.com/ci_job_id` | Contains the CI job ID. | -| `agent.gitlab.com/username` | Contains the username of the user the CI job is running as. | -| `agent.gitlab.com/environment_slug` | Contains the slug of the environment. Only set if running in an environment. | - -Example to restrict access by the CI job's identity: - -```yaml -ci_access: - projects: - - id: path/to/project - access_as: - ci_job: {} -``` - -### Impersonate a static identity - -For the given CI/CD Tunnel connection, you can use a static identity for the impersonation. - -Add the `impersonate` key under the `access_as` key to make the request using the provided identity. - -The identity can be specified with the following keys: + ```json + { + "level": "warn", + "time": "2021-04-29T23:44:07.598Z", + "msg": "GetConfiguration.Recv failed", + "error": "rpc error: code = Unauthenticated desc = unauthenticated" + } + ``` -- `username` (required) -- `uid` -- `groups` -- `extra` +1. Delete the agent in your cluster: -See the [official Kubernetes documentation for more details](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#user-impersonation) on the usage of these keys. + ```shell + kubectl delete -n gitlab-kubernetes-agent -f ./resources.yml + ``` ## Surface network security alerts from cluster to GitLab **(ULTIMATE)** @@ -340,7 +174,28 @@ Cilium integration is in its end-of-life process. It's [deprecated](https://gitl for use in GitLab 14.8, and planned for [removal](https://gitlab.com/groups/gitlab-org/-/epics/7477) in GitLab 15.0. -The GitLab Agent provides an [integration with Cilium](index.md#kubernetes-network-security-alerts). +The agent for Kubernetes also provides an integration with Cilium. This integration provides a simple way to +generate network policy-related alerts and to surface those alerts in GitLab. + +Several components work in concert for the agent to generate the alerts: + +- A working Kubernetes cluster. +- Cilium integration through either of these options: + - Installation through [cluster management template](../../project/clusters/protect/container_network_security/quick_start_guide.md#use-the-cluster-management-template-to-install-cilium). + - Enablement of [hubble-relay](https://docs.cilium.io/en/v1.8/concepts/overview/#hubble) on an + existing installation. +- One or more network policies through any of these options: + - Use the [Container Network Policy editor](../../application_security/policies/index.md#container-network-policy-editor) to create and manage policies. + - Use an [AutoDevOps](../../application_security/policies/index.md#container-network-policy) configuration. + - Add the required labels and annotations to existing network policies. +- A configuration repository with [Cilium configured in `config.yaml`](repository.md#surface-network-security-alerts-from-cluster-to-gitlab) + +The setup process follows the same [agent's installation steps](install/index.md), +with the following differences: + +- When you define a configuration repository, you must do so with [Cilium settings](repository.md#surface-network-security-alerts-from-cluster-to-gitlab). +- You do not need to specify the `gitops` configuration section. + To integrate, add a top-level `cilium` section to your `config.yml` file. Currently, the only configuration option is the Hubble relay address: @@ -357,107 +212,3 @@ you can use `hubble-relay.gitlab-managed-apps.svc.cluster.local:80` as the addre cilium: hubble_relay_address: "hubble-relay.gitlab-managed-apps.svc.cluster.local:80" ``` - -## Scan your container images for vulnerabilities **(ULTIMATE)** - -You can use [cluster image scanning](../../application_security/cluster_image_scanning/index.md) -to scan container images in your cluster for security vulnerabilities. - -To begin scanning all resources in your cluster, add a `starboard` -configuration block to your agent's `config.yaml` with no `filters`: - -```yaml -starboard: - vulnerability_report: - filters: [] -``` - -The namespaces that are able to be scanned depend on the [Starboard Operator install mode](https://aquasecurity.github.io/starboard/latest/operator/configuration/#install-modes). -By default, the Starboard Operator only scans resources in the `default` namespace. To change this -behavior, edit the `STARBOARD_OPERATOR` environment variable in the `starboard-operator` deployment -definition. - -By adding filters, you can limit scans by: - -- Resource name -- Kind -- Container name -- Namespace - -```yaml -starboard: - vulnerability_report: - filters: - - namespaces: - - staging - - production - kinds: - - Deployment - - DaemonSet - containers: - - ruby - - postgres - - nginx - resources: - - my-app-name - - postgres - - ingress-nginx -``` - -A resource is scanned if the resource matches any of the given names and all of the given filter -types (`namespaces`, `kinds`, `containers`, `resources`). If a filter type is omitted, then all -names are scanned. In this example, a resource isn't scanned unless it has a container named `ruby`, -`postgres`, or `nginx`, and it's a `Deployment`: - -```yaml -starboard: - vulnerability_report: - filters: - - kinds: - - Deployment - containers: - - ruby - - postgres - - nginx -``` - -There is also a global `namespaces` field that applies to all filters: - -```yaml -starboard: - vulnerability_report: - namespaces: - - production - filters: - - kinds: - - Deployment - - kinds: - - DaemonSet - resources: - - log-collector -``` - -In this example, the following resources are scanned: - -- All deployments (`Deployment`) in the `production` namespace -- All daemon sets (`DaemonSet`) named `log-collector` in the `production` namespace - -## Debugging - -To debug the cluster-side component (`agentk`) of the Agent, set the log -level according to the available options: - -- `off` -- `warning` -- `error` -- `info` -- `debug` - -The log level defaults to `info`. You can change it by using a top-level `observability` -section in the configuration file, for example: - -```yaml -observability: - logging: - level: debug -``` diff --git a/doc/user/clusters/agent/runner.md b/doc/user/clusters/agent/runner.md deleted file mode 100644 index c40733bd7a5..00000000000 --- a/doc/user/clusters/agent/runner.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: 'https://docs.gitlab.com/runner/install/kubernetes-agent.html' -remove_date: '2022-02-01' ---- - -This document was moved to [another location](https://docs.gitlab.com/runner/install/kubernetes-agent.html). - -<!-- This redirect file can be deleted after <2022-02-01>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page -->
\ No newline at end of file diff --git a/doc/user/clusters/agent/troubleshooting.md b/doc/user/clusters/agent/troubleshooting.md index 2c9f98b7c45..a5e568837ad 100644 --- a/doc/user/clusters/agent/troubleshooting.md +++ b/doc/user/clusters/agent/troubleshooting.md @@ -4,9 +4,9 @@ group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Troubleshooting the GitLab Agent for Kubernetes +# Troubleshooting the GitLab agent for Kubernetes -When you are using the GitLab Agent for Kubernetes, you might experience issues you need to troubleshoot. +When you are using the GitLab agent for Kubernetes, you might experience issues you need to troubleshoot. You can start by viewing the service logs: @@ -14,7 +14,7 @@ You can start by viewing the service logs: kubectl logs -f -l=app=gitlab-agent -n gitlab-kubernetes-agent ``` -If you are a GitLab administrator, you can also view the [GitLab Agent Server logs](../../../administration/clusters/kas.md#troubleshooting). +If you are a GitLab administrator, you can also view the [GitLab agent server logs](../../../administration/clusters/kas.md#troubleshooting). ## Transport: Error while dialing failed to WebSocket dial @@ -28,7 +28,7 @@ If you are a GitLab administrator, you can also view the [GitLab Agent Server lo ``` This error is shown if there are some connectivity issues between the address -specified as `kas-address`, and your Agent pod. To fix it, make sure that you +specified as `kas-address`, and your agent pod. To fix it, make sure that you specified the `kas-address` correctly. ```json @@ -188,6 +188,5 @@ Alternatively, you can mount the certificate file at a different location and in } ``` -This error is shown if the manifest project is not public. To fix it, -[make sure your manifest project is public](repository.md#synchronize-manifest-projects) or your manifest files -are stored in the Agent's configuration repository. +This error is shown if the manifest project is not public. To fix it, make sure your manifest project is public or your manifest files +are stored in the agent's configuration repository. diff --git a/doc/user/clusters/agent/vulnerabilities.md b/doc/user/clusters/agent/vulnerabilities.md new file mode 100644 index 00000000000..480b09ff2ab --- /dev/null +++ b/doc/user/clusters/agent/vulnerabilities.md @@ -0,0 +1,113 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Container vulnerability scanning **(ULTIMATE)** + +> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6346) in GitLab 14.8. + +To view cluster vulnerabilities, you can view the [vulnerability report](../../application_security/vulnerabilities/index.md). +You can also configure your agent so the vulnerabilities are displayed with other agent information in GitLab. + +## View cluster vulnerabilities + +Prerequisite: + +- You must have at least the Developer role. +- [Cluster image scanning](../../application_security/cluster_image_scanning/index.md) + must be part of your build process. + +To view vulnerability information in GitLab: + +1. On the top bar, select **Menu > Projects** and find the project that contains the agent configuration file. +1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. +1. Select the **Agent** tab. +1. Select the agent you want to see the vulnerabilities for. + + + +## Enable cluster vulnerability scanning **(ULTIMATE)** + +You can use [cluster image scanning](../../application_security/cluster_image_scanning/index.md) +to scan container images in your cluster for security vulnerabilities. + +To begin scanning all resources in your cluster, add a `starboard` +configuration block to your agent configuration file with no `filters`: + +```yaml +starboard: + vulnerability_report: + filters: [] +``` + +The namespaces that are able to be scanned depend on the [Starboard Operator install mode](https://aquasecurity.github.io/starboard/latest/operator/configuration/#install-modes). +By default, the Starboard Operator only scans resources in the `default` namespace. To change this +behavior, edit the `STARBOARD_OPERATOR` environment variable in the `starboard-operator` deployment +definition. + +By adding filters, you can limit scans by: + +- Resource name +- Kind +- Container name +- Namespace + +```yaml +starboard: + vulnerability_report: + filters: + - namespaces: + - staging + - production + kinds: + - Deployment + - DaemonSet + containers: + - ruby + - postgres + - nginx + resources: + - my-app-name + - postgres + - ingress-nginx +``` + +A resource is scanned if the resource matches any of the given names and all of the given filter +types (`namespaces`, `kinds`, `containers`, `resources`). If a filter type is omitted, then all +names are scanned. In this example, a resource isn't scanned unless it has a container named `ruby`, +`postgres`, or `nginx`, and it's a `Deployment`: + +```yaml +starboard: + vulnerability_report: + filters: + - kinds: + - Deployment + containers: + - ruby + - postgres + - nginx +``` + +There is also a global `namespaces` field that applies to all filters: + +```yaml +starboard: + vulnerability_report: + namespaces: + - production + filters: + - kinds: + - Deployment + - kinds: + - DaemonSet + resources: + - log-collector +``` + +In this example, the following resources are scanned: + +- All deployments (`Deployment`) in the `production` namespace. +- All daemon sets (`DaemonSet`) named `log-collector` in the `production` namespace. diff --git a/doc/user/clusters/applications.md b/doc/user/clusters/applications.md index f880e603133..2bcfea50ee3 100644 --- a/doc/user/clusters/applications.md +++ b/doc/user/clusters/applications.md @@ -980,7 +980,7 @@ podAnnotations: The only information to be changed here is the profile name which is `profile-one` in this example. Refer to the -[AppArmor tutorial](https://kubernetes.io/docs/tutorials/clusters/apparmor/#securing-a-pod) +[AppArmor tutorial](https://kubernetes.io/docs/tutorials/security/apparmor/#securing-a-pod) for more information on how AppArmor is integrated in Kubernetes. #### Using PodSecurityPolicy in your deployments @@ -1017,7 +1017,7 @@ securityPolicies: ``` This example creates a single policy named `example` with the provided specification, -and enables [AppArmor annotations](https://kubernetes.io/docs/tutorials/clusters/apparmor/#podsecuritypolicy-annotations) on it. +and enables [AppArmor annotations](https://kubernetes.io/docs/tutorials/security/apparmor/#podsecuritypolicy-annotations) on it. Support for installing the AppArmor managed application is provided by the GitLab Container Security group. If you run into unknown issues, diff --git a/doc/user/clusters/create/index.md b/doc/user/clusters/create/index.md new file mode 100644 index 00000000000..bee622ac50a --- /dev/null +++ b/doc/user/clusters/create/index.md @@ -0,0 +1,13 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Create Kubernetes clusters + +You can use Infrastructure as Code (IaC) to create clusters on cloud providers. +You connect the clusters to GitLab by using the agent for Kubernetes. + +- [Create a cluster on Google GKE](../../infrastructure/clusters/connect/new_gke_cluster.md) +- [Create a cluster on Amazon EKS](../../infrastructure/clusters/connect/new_eks_cluster.md) diff --git a/doc/user/clusters/crossplane.md b/doc/user/clusters/crossplane.md index e6540e68f71..9e4c672ac45 100644 --- a/doc/user/clusters/crossplane.md +++ b/doc/user/clusters/crossplane.md @@ -80,7 +80,7 @@ provided can manage resources in the `database.crossplane.io` API group: ## Configure Crossplane with a cloud provider -See [Configure Your Cloud Provider Account](https://crossplane.github.io/docs/v0.4/cloud-providers.html) +See [Configure Your Cloud Provider Account](https://crossplane.github.io/docs/v1.6/) to configure the installed cloud provider stack with a user account. The Secret, and the Provider resource referencing the Secret, must be diff --git a/doc/user/clusters/img/kubernetes-agent-ui-list_v14_8.png b/doc/user/clusters/img/kubernetes-agent-ui-list_v14_8.png Binary files differdeleted file mode 100644 index 5b5ba3a9804..00000000000 --- a/doc/user/clusters/img/kubernetes-agent-ui-list_v14_8.png +++ /dev/null diff --git a/doc/user/clusters/integrations.md b/doc/user/clusters/integrations.md index ee7452537fd..74f6ec283ea 100644 --- a/doc/user/clusters/integrations.md +++ b/doc/user/clusters/integrations.md @@ -32,9 +32,9 @@ to automate this step. Prometheus and Elastic Stack cluster integrations can only be enabled for clusters [connected through cluster certificates](../project/clusters/add_existing_cluster.md). -To enable Prometheus for your cluster connected through the [GitLab Agent](agent/index.md), you can [integrate it manually](../project/integrations/prometheus.md#manual-configuration-of-prometheus). +To enable Prometheus for your cluster connected through the [GitLab agent](agent/index.md), you can [integrate it manually](../project/integrations/prometheus.md#manual-configuration-of-prometheus). -There is no option to enable Elastic Stack for your cluster if it is connected with the GitLab Agent. +There is no option to enable Elastic Stack for your cluster if it is connected with the GitLab agent. Follow this [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/300230) for updates. ## Prometheus cluster integration @@ -44,7 +44,7 @@ Follow this [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/300230) for up WARNING: This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. However, you can **still use** Prometheus for Kubernetes clusters connected to GitLab through the -[Agent](agent/index.md) by [enabling Prometheus manually](../project/integrations/prometheus.md#manual-configuration-of-prometheus). +[agent](agent/index.md) by [enabling Prometheus manually](../project/integrations/prometheus.md#manual-configuration-of-prometheus). You can integrate your Kubernetes cluster with [Prometheus](https://prometheus.io/) for monitoring key metrics of your @@ -113,9 +113,9 @@ To use this integration: `gitlab-managed-apps` namespace. 1. The `Service` resource must be called `elastic-stack-elasticsearch-master` and expose the Elasticsearch API on port `9200`. -1. The logs are expected to be [Filebeat container logs](https://www.elastic.co/guide/en/beats/filebeat/7.x/filebeat-input-container.html) - following the [7.x log structure](https://www.elastic.co/guide/en/beats/filebeat/7.x/exported-fields-log.html) - and include [Kubernetes metadata](https://www.elastic.co/guide/en/beats/filebeat/7.x/add-kubernetes-metadata.html). +1. The logs are expected to be [Filebeat container logs](https://www.elastic.co/guide/en/beats/filebeat/7.16/filebeat-input-container.html) + following the [7.x log structure](https://www.elastic.co/guide/en/beats/filebeat/7.16/exported-fields-log.html) + and include [Kubernetes metadata](https://www.elastic.co/guide/en/beats/filebeat/7.16/add-kubernetes-metadata.html). You can manage your Elastic Stack however you like, but as an example, you can use [this Elastic Stack chart](https://gitlab.com/gitlab-org/charts/elastic-stack) to get up and diff --git a/doc/user/clusters/management_project.md b/doc/user/clusters/management_project.md index e28f9275b50..7658bc41dd9 100644 --- a/doc/user/clusters/management_project.md +++ b/doc/user/clusters/management_project.md @@ -11,7 +11,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w WARNING: The cluster management project was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. -To manage cluster applications, use the [GitLab Agent](agent/index.md) +To manage cluster applications, use the [GitLab agent](agent/index.md) with the [Cluster Management Project Template](management_project_template.md). A project can be designated as the management project for a cluster. diff --git a/doc/user/clusters/management_project_template.md b/doc/user/clusters/management_project_template.md index 2d7e89ef765..ab17e462c6a 100644 --- a/doc/user/clusters/management_project_template.md +++ b/doc/user/clusters/management_project_template.md @@ -8,89 +8,53 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/25318) in GitLab 12.10 with Helmfile support via Helm v2. > - Helm v2 support was [dropped](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/63577) in GitLab 14.0. Use Helm v3 instead. -> - [Migrated](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/merge_requests/24) to the GitLab Agent in GitLab 14.5. +> - [Migrated](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/merge_requests/24) to the GitLab agent in GitLab 14.5. -Use a repository to install, manage, and deploy clusters applications through code. +GitLab provides a cluster management project template, which you use +to create a project. The project includes cluster applications that integrate with GitLab +and extend GitLab functionality. You can use the pattern shown in the project to extend +your custom cluster applications. -## Cluster Management Project Template +## Use one project for the agent and your manifests -The Cluster Management Project Template provides you a baseline to get -started and flexibility to customize your project to your cluster's needs. -For instance, you can: +If you **have not yet** used the agent to connect your cluster with GitLab: -- Extend the CI/CD configuration. -- Configure the built-in cluster applications. -- Remove the built-in cluster applications you don't need. -- Add other cluster applications using the same structure as the ones already available. +1. [Create a project from the cluster management project template](#create-a-project-based-on-the-cluster-management-project-template). +1. [Configure the project for the agent](agent/install/index.md). +1. In your project's settings, create an + [environment variable](../../ci/variables/index.md#add-a-cicd-variable-to-a-project) named `$KUBE_CONTEXT` + and set the value to `path/to/agent-configuration-project:your-agent-name`. +1. [Configure the files](#configure-the-project) as needed. -The template contains the following [components](#configure-the-available-components): +## Use separate projects for the agent and your manifests -- A pre-configured GitLab CI/CD file so that you can configure CI/CD pipelines using the [CI/CD Tunnel](agent/ci_cd_tunnel.md). -- A pre-configured [Helmfile](https://github.com/roboll/helmfile) so that -you can manage cluster applications with [Helm v3](https://helm.sh/). -- An `applications` directory with a `helmfile.yaml` configured for each -application available in the template. +If you have already configured the agent and connected a cluster with GitLab: -## Use the Agent with the Cluster Management Project Template +1. [Create a project from the cluster management project template](#create-a-project-based-on-the-cluster-management-project-template). +1. In the project where you configured your agent, + [grant the agent access to the new project](agent/ci_cd_tunnel.md#authorize-the-agent). +1. In the new project, create an + [environment variable](../../ci/variables/index.md#add-a-cicd-variable-to-a-project) named `$KUBE_CONTEXT` + and set the value to `path/to/agent-configuration-project:your-agent-name`. +1. In the new project, [configure the files](#configure-the-project) as needed. -To use a new project created from the Cluster Management Project Template -with a cluster connected to GitLab through the [GitLab Agent](agent/index.md), -you have two options: +## Create a project based on the cluster management project template -- [Use one single project](#single-project) to configure the Agent and manage cluster applications. -- [Use separate projects](#separate-projects) - one to configure the Agent and another to manage cluster applications. +To create a project from the cluster management project template: -### Single project +1. On the top bar, select **Menu > Projects > Create new project**. +1. Select **Create from template**. +1. From the list of templates, next to **GitLab Cluster Management**, select **Use template**. +1. Enter the project details. +1. Select **Create project**. -This setup is particularly useful when you haven't connected your cluster -to GitLab through the Agent yet and you want to use the Cluster Management -Project Template to manage cluster applications. +If you use self-managed GitLab, your instance might not include the latest version of the template. +In that case, select **Import project**, **Repo by URL** and for the **Git repository URL**, enter +`https://gitlab.com/gitlab-org/project-templates/cluster-management.git`. -To use one single project to configure the Agent and to manage cluster applications: +## Configure the project -1. [Create a new project from the Cluster Management Project Template](#create-a-new-project-based-on-the-cluster-management-template). -1. Configure the new project as the [Agent's configuration repository](agent/repository.md) -(where the Agent is registered and its `config.yaml` is stored). -1. From your project's settings, add a [new environment variable](../../ci/variables/index.md#add-a-cicd-variable-to-a-project) `$KUBE_CONTEXT` and set it to `path/to/agent-configuration-project:your-agent-name`. -1. [Configure the components](#configure-the-available-components) inherited from the template. - -### Separate projects - -This setup is particularly useful **when you already have a cluster** connected -to GitLab through the Agent and want to use the Cluster Management -Project Template to manage cluster applications. - -To use one project to configure the Agent ("project A") and another project to -manage cluster applications ("project B"), follow the steps below. - -We assume that you already have a cluster connected through the Agent and -[configured through the Agent's configuration repository](agent/repository.md) -("project A"). - -1. [Create a new project from the Cluster Management Project Template](#create-a-new-project-based-on-the-cluster-management-template). -This new project is "project B". -1. In your "project A", [grant the Agent access to the new project (B) through the CI/CD Tunnel](agent/repository.md#authorize-projects-to-use-an-agent). -1. From the "project's B" settings, add a [new environment variable](../../ci/variables/index.md#add-a-cicd-variable-to-a-project) `$KUBE_CONTEXT` and set it to `path/to/agent-configuration-project:your-agent-name`. -1. In "project B", [configure the components](#configure-the-available-components) inherited from the template. - -## Create a new project based on the Cluster Management Template - -To get started, create a new project based on the Cluster Management -project template to use as a cluster management project. - -You can either create the new project from the template or import the -project from the URL. Importing the project is useful if you are using -a GitLab self-managed instance that may not have the latest version of -the template. - -To [create the new project](../project/working_with_projects.md#create-a-project): - -- From the template: select the **GitLab Cluster Management** project template. -- Importing from the URL: use `https://gitlab.com/gitlab-org/project-templates/cluster-management.git`. - -## Configure the available components - -Use the available components to configure your cluster applications: +After you use the cluster management template to create a project, you can configure: - [The `.gitlab-ci.yml` file](#the-gitlab-ciyml-file). - [The main `helmfile.yml` file](#the-main-helmfileyml-file). @@ -98,22 +62,22 @@ Use the available components to configure your cluster applications: ### The `.gitlab-ci.yml` file -The base image used in your pipeline is built by the [cluster-applications](https://gitlab.com/gitlab-org/cluster-integration/cluster-applications) -project. This image consists of a set of Bash utility scripts to support [Helm v3 releases](https://helm.sh/docs/intro/using_helm/#three-big-concepts): +The `.gitlab-ci.yml` file: -- `gl-fail-if-helm2-releases-exist {namespace}`: It tries to detect whether you have apps deployed through Helm v2 - releases for a given namespace. If so, it will fail the pipeline and ask you to manually - [migrate your Helm v2 releases to Helm v3](https://helm.sh/docs/topics/v2_v3_migration/). -- `gl-ensure-namespace {namespace}`: It creates the given namespace if it does not exist and adds the necessary label - for the [Cilium](https://github.com/cilium/cilium/) app network policies to work. -- `gl-adopt-resource-with-helm-v3 {arguments}`: Used only internally in the [cert-manager's](https://cert-manager.io/) Helmfile to - facilitate the GitLab Managed Apps adoption. -- `gl-adopt-crds-with-helm-v3 {arguments}`: Used only internally in the [cert-manager's](https://cert-manager.io/) Helmfile to - facilitate the GitLab Managed Apps adoption. -- `gl-helmfile {arguments}`: A thin wrapper that triggers the [Helmfile](https://github.com/roboll/helmfile) command. +- Ensures you are on Helm version 3. +- Deploys the enabled applications from the project. + +You can edit and extend the pipeline definitions. + +The base image used in the pipeline is built by the +[cluster-applications](https://gitlab.com/gitlab-org/cluster-integration/cluster-applications) project. +This image contains a set of Bash utility scripts to support [Helm v3 releases](https://helm.sh/docs/intro/using_helm/#three-big-concepts). ### The main `helmfile.yml` file +The template contains a [Helmfile](https://github.com/roboll/helmfile) you can use to manage +cluster applications with [Helm v3](https://helm.sh/). + This file has a list of paths to other Helmfiles for each app. They're all commented out by default, so you must uncomment the paths for the apps that you would like to use in your cluster. @@ -124,6 +88,9 @@ from your cluster. [Read more](https://github.com/roboll/helmfile) about how Hel ### Built-in applications +The template contains an `applications` directory with a `helmfile.yaml` configured for each +application in the template. + The [built-in supported applications](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/tree/master/applications) are: - [Apparmor](../infrastructure/clusters/manage/management_project_applications/apparmor.md) @@ -138,8 +105,8 @@ The [built-in supported applications](https://gitlab.com/gitlab-org/project-temp - [Sentry](../infrastructure/clusters/manage/management_project_applications/sentry.md) - [Vault](../infrastructure/clusters/manage/management_project_applications/vault.md) -#### Customize your applications +Each application has an `applications/{app}/values.yaml` file. +For GitLab Runner, the file is `applications/{app}/values.yaml.gotmpl`. -Each app has an `applications/{app}/values.yaml` file (`applications/{app}/values.yaml.gotmpl` in case of GitLab Runner). This is the -place where you can define default values for your app's Helm chart. Some apps already have defaults -pre-defined by GitLab. +In this file, you can define default values for your app's Helm chart. +Some apps already have defaults defined. diff --git a/doc/user/clusters/migrating_from_gma_to_project_template.md b/doc/user/clusters/migrating_from_gma_to_project_template.md index b2ba1bef338..09453262fbb 100644 --- a/doc/user/clusters/migrating_from_gma_to_project_template.md +++ b/doc/user/clusters/migrating_from_gma_to_project_template.md @@ -20,7 +20,7 @@ To migrate from GitLab Managed Apps to a Cluster Management Project, follow the steps below. See also [video walk-throughs](#video-walk-throughs) with examples. -1. Create a new project based on the [Cluster Management Project template](management_project_template.md#create-a-new-project-based-on-the-cluster-management-template). +1. Create a new project based on the [Cluster Management Project template](management_project_template.md#create-a-project-based-on-the-cluster-management-project-template). 1. [Associate your new Cluster Management Project with your cluster](management_project.md#associate-the-cluster-management-project-with-the-cluster). 1. Detect apps deployed through Helm v2 releases by using the pre-configured [`.gitlab-ci.yml`](management_project_template.md#the-gitlab-ciyml-file) file: - In case you had overwritten the default GitLab Managed Apps namespace, edit `.gitlab-ci.yml`, @@ -120,7 +120,7 @@ you want to manage with the Cluster Management Project. ## Backup and uninstall cert-manager v0.10 -1. Follow the [official docs](https://docs.cert-manager.io/en/release-0.10/tasks/backup-restore-crds.html) on how to +1. Follow the [official docs](https://cert-manager.io/docs/tutorials/backup/) on how to backup your cert-manager v0.10 data. 1. Uninstall cert-manager by editing the setting all the occurrences of `installed: true` to `installed: false` in the `applications/cert-manager/helmfile.yaml` file. diff --git a/doc/user/compliance/compliance_report/index.md b/doc/user/compliance/compliance_report/index.md index d98a0a145f2..27783a063da 100644 --- a/doc/user/compliance/compliance_report/index.md +++ b/doc/user/compliance/compliance_report/index.md @@ -105,3 +105,64 @@ You can generate a commit-specific Chain of Custody report for a given commit SH NOTE: The Chain of Custody report download is a CSV file, with a maximum size of 15 MB. The remaining records are truncated when this limit is reached. + +## Merge request violations + +> - Introduced in GitLab 14.6. [Deployed behind the `compliance_violations_report` flag](../../../administration/feature_flags.md). Disabled by default. +> - GraphQL API [introduced](https://gitlab.com/groups/gitlab-org/-/epics/7222) in GitLab 14.9. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, +ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `compliance_violations_report`. +On GitLab.com, this feature is not available. This feature is not ready for production use. + +Merge request violations provide a view of all the [separation of duties](#approval-status-and-separation-of-duties) compliance violations +that exist in projects in a specific group. For each separation of duties compliance violation, you can see: + +- A list of compliance violations. +- The severity of each compliance violation. +- Reason for the compliance violation. +- A link to the merge request that caused the compliance violation. + +Merge request violations can be accessed: + +- In the GitLab UI. +- Using the [GraphQL API](../../../api/graphql/reference/index.md#complianceviolation) (GitLab 14.9 and later). + +### View merge request violations + +To view merge request violations: + +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Security & Compliance > Compliance report**. + +### Severity levels scale + +The following is a list of available violation severity levels, ranked from most to least severe: + +| Icon | Severity level | +|:----------------------------------------------|:---------------| +| **{severity-critical, 18, gl-fill-red-800}** | Critical | +| **{severity-high, 18, gl-fill-red-600}** | High | +| **{severity-medium, 18, gl-fill-orange-400}** | Medium | +| **{severity-low, 18, gl-fill-orange-300}** | Low | +| **{severity-info, 18, gl-fill-blue-400}** | Info | + +### Violation types + +The following is a list of violations that are either: + +- Already available. +- Aren't available, but which we are tracking in issues. + +| Violation | Severity level | Category | Description | Availability | +|:-------------------------------------|:----------------|:----------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------| +| Author approved merge request | High | [Separation of duties](#approval-status-and-separation-of-duties) | The author of the merge request approved their own merge request. [Learn more](../../project/merge_requests/approvals/settings.md#prevent-approval-by-author). | [Unavailable](https://gitlab.com/groups/gitlab-org/-/epics/6870) | +| Committers approved merge request | High | [Separation of duties](#approval-status-and-separation-of-duties) | The committers of the merge request approved the merge request they contributed to. [Learn more](../../project/merge_requests/approvals/settings.md#prevent-approvals-by-users-who-add-commits). | [Unavailable](https://gitlab.com/groups/gitlab-org/-/epics/6870) | +| Fewer than two approvals | High | [Separation of duties](#approval-status-and-separation-of-duties) | The merge request was merged with fewer than two approvals. [Learn more](../../project/merge_requests/approvals/rules.md). | [Unavailable](https://gitlab.com/groups/gitlab-org/-/epics/6870) | +| Pipeline failed | Medium | [Pipeline results](../../../ci/pipelines/index.md) | The merge requests pipeline failed and was merged. | [Unavailable](https://gitlab.com/gitlab-org/gitlab/-/issues/346011) | +| Pipeline passed with warnings | Info | [Pipeline results](../../../ci/pipelines/index.md) | The merge request pipeline passed with warnings and was merged. | [Unavailable](https://gitlab.com/gitlab-org/gitlab/-/issues/346011) | +| Code coverage down more than 10% | High | [Code coverage](../../../ci/pipelines/settings.md#merge-request-test-coverage-results) | The code coverage report for the merge request indicates a reduction in coverage of more than 10%. | [Unavailable](https://gitlab.com/gitlab-org/gitlab/-/issues/346011) | +| Code coverage down between 5% to 10% | Medium | [Code coverage](../../../ci/pipelines/settings.md#merge-request-test-coverage-results) | The code coverage report for the merge request indicates a reduction in coverage of between 5% to 10%. | [Unavailable](https://gitlab.com/gitlab-org/gitlab/-/issues/346011) | +| Code coverage down between 1% to 5% | Low | [Code coverage](../../../ci/pipelines/settings.md#merge-request-test-coverage-results) | The code coverage report for the merge request indicates a reduction in coverage of between 1% to 5%. | [Unavailable](https://gitlab.com/gitlab-org/gitlab/-/issues/346011) | +| Code coverage down less than 1% | Info | [Code coverage](../../../ci/pipelines/settings.md#merge-request-test-coverage-results) | The code coverage report for the merge request indicates a reduction in coverage of less than 1%. | [Unavailable](https://gitlab.com/gitlab-org/gitlab/-/issues/346011) | diff --git a/doc/user/compliance/license_compliance/index.md b/doc/user/compliance/license_compliance/index.md index 18de33ea03b..a2172b72572 100644 --- a/doc/user/compliance/license_compliance/index.md +++ b/doc/user/compliance/license_compliance/index.md @@ -55,7 +55,7 @@ You can view and modify existing policies from the [policies](#policies) tab. ## License expressions -GitLab has limited support for [composite licenses](https://spdx.github.io/spdx-spec/appendix-IV-SPDX-license-expressions/). +GitLab has limited support for [composite licenses](https://spdx.github.io/spdx-spec/SPDX-license-expressions/). License compliance can read multiple licenses, but always considers them combined using the `AND` operator. For example, if a dependency has two licenses, and one of them is allowed and the other is denied by the project [policy](#policies), GitLab evaluates the composite license as _denied_, as this is the safer option. @@ -90,7 +90,7 @@ The reported licenses might be incomplete or inaccurate. | Objective-C, Swift | [Carthage](https://github.com/Carthage/Carthage), [CocoaPods](https://cocoapods.org/) v0.39 and below | | Elixir | [Mix](https://elixir-lang.org/getting-started/mix-otp/introduction-to-mix.html) | | C++/C | [Conan](https://conan.io/) | -| Rust | [Cargo](https://crates.io) | +| Rust | [Cargo](https://crates.io/) | | PHP | [Composer](https://getcomposer.org/) | ## Enable License Compliance @@ -219,7 +219,7 @@ license_scanning: MAVEN_CLI_OPTS: --debug ``` -`mvn install` runs through all of the [build life cycle](http://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html) +`mvn install` runs through all of the [build life cycle](https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html) stages prior to `install`, including `test`. Running unit tests is not directly necessary for the license scanning purposes and consumes time, so it's skipped by having the default value of `MAVEN_CLI_OPTS` as `-DskipTests`. If you want @@ -249,7 +249,7 @@ license_scanning: Alternatively, you can use a Java key store to verify the TLS connection. For instructions on how to generate a key store file, see the -[Maven Guide to Remote repository access through authenticated HTTPS](http://maven.apache.org/guides/mini/guide-repository-ssl.html). +[Maven Guide to Remote repository access through authenticated HTTPS](https://maven.apache.org/guides/mini/guide-repository-ssl.html). ### Selecting the version of Java @@ -650,7 +650,7 @@ import the following default License Compliance analyzer images from `registry.g offline [local Docker container registry](../../packages/container_registry/index.md): ```plaintext -registry.gitlab.com/gitlab-org/security-products/analyzers/license-finder:latest +registry.gitlab.com/security-products/license-finder:latest ``` The process for importing Docker images into a local offline Docker registry depends on @@ -734,7 +734,7 @@ Note, the merge request is not able to be merged until the `denied` license is r You may add a [`License-Check` approval rule](#enabling-license-approvals-within-a-project), which enables a designated approver that can approve and then merge a merge request with `denied` license. - + The **Policies** tab in the project's license compliance section displays your project's license policies. Project maintainers can specify policies in this section. @@ -853,7 +853,7 @@ A full list of variables can be found in [CI/CD variables](#available-cicd-varia To find out what tools are pre-installed in the `license_scanning` Docker image use the following command: ```shell -$ docker run --entrypoint='' registry.gitlab.com/gitlab-org/security-products/analyzers/license-finder:3 /bin/bash -lc 'asdf list' +$ docker run --entrypoint='' registry.gitlab.com/security-products/license-finder:3 /bin/bash -lc 'asdf list' golang 1.14 gradle @@ -880,7 +880,7 @@ sbt To interact with the `license_scanning` runtime environment use the following command: ```shell -$ docker run -it --entrypoint='' registry.gitlab.com/gitlab-org/security-products/analyzers/license-finder:3 /bin/bash -l +$ docker run -it --entrypoint='' registry.gitlab.com/security-products/license-finder:3 /bin/bash -l root@6abb70e9f193:~# ``` diff --git a/doc/user/crm/index.md b/doc/user/crm/index.md index 305cca33dd5..1fb628cf505 100644 --- a/doc/user/crm/index.md +++ b/doc/user/crm/index.md @@ -6,15 +6,18 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Customer relations management (CRM) **(FREE)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/2256) in GitLab 14.6 [with a flag](../../administration/feature_flags.md) named `customer_relations`. Disabled by default. - FLAG: On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `customer_relations`. On GitLab.com, this feature is not available. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/2256) in GitLab 14.6 [with a flag](../../administration/feature_flags.md) named `customer_relations`. Disabled by default. +> - In GitLab 14.8 and later, you can [create contacts and organizations only in root groups](https://gitlab.com/gitlab-org/gitlab/-/issues/350634). + With customer relations management (CRM) you can create a record of contacts (individuals) and organizations (companies) and relate them to issues. +Contacts and organizations can only be created for root groups. + You can use contacts and organizations to tie work to customers for billing and reporting purposes. To read more about what is planned for the future, see [issue 2256](https://gitlab.com/gitlab-org/gitlab/-/issues/2256). diff --git a/doc/user/discussions/index.md b/doc/user/discussions/index.md index 37047e9f8d0..a6343e0d1cf 100644 --- a/doc/user/discussions/index.md +++ b/doc/user/discussions/index.md @@ -36,10 +36,9 @@ Each object can have as many as 5,000 comments. ## Mentions -You can mention a user or a group present in your GitLab instance with `@username` or -`@groupname`. All mentioned users are notified with to-do items and emails. -Users can change this setting for themselves in the -[notification settings](../profile/notifications.md). +You can mention a user or a group (including [subgroups](../group/subgroups/index.md#mention-subgroups)) in your GitLab +instance with `@username` or `@groupname`. All mentioned users are notified with to-do items and emails. +Users can change this setting for themselves in the [notification settings](../profile/notifications.md). You can quickly see which comments involve you, because mentions for yourself (the user currently signed in) are highlighted @@ -47,6 +46,8 @@ in a different color. Avoid mentioning `@all` in issues and merge requests, because it sends an email notification to all the members of that project's group. This might be interpreted as spam. +Notifications and mentions can be disabled in +[a group's settings](../group/index.md#disable-email-notifications). ## Add a comment to a merge request diff diff --git a/doc/user/gitlab_com/index.md b/doc/user/gitlab_com/index.md index 80ee8d23a35..5cf6a505bee 100644 --- a/doc/user/gitlab_com/index.md +++ b/doc/user/gitlab_com/index.md @@ -85,7 +85,10 @@ are included when cloning. Top-level groups created after August 12, 2021 have delayed project deletion enabled by default. Projects are permanently deleted after a seven-day delay. -You can disable this by changing the [group setting](../group/index.md#enable-delayed-project-deletion). +If you are on: + +- Premium tier and above, you can disable this by changing the [group setting](../group/index.md#enable-delayed-project-deletion). +- Free tier, you cannot disable this setting or restore projects. ## Alternative SSH port @@ -130,20 +133,20 @@ Below are the current settings regarding [GitLab CI/CD](../../ci/index.md). Any settings or feature limits not listed here are using the defaults listed in the related documentation. -| Setting | GitLab.com | Default | -|-------------------------------------|-------------|---------| -| Artifacts maximum size (compressed) | 1 GB | 100 MB | -| Artifacts [expiry time](../../ci/yaml/index.md#artifactsexpire_in) | From June 22, 2020, deleted after 30 days unless otherwise specified (artifacts created before that date have no expiry). | deleted after 30 days unless otherwise specified | -| Scheduled Pipeline Cron | `*/5 * * * *` | `3-59/10 * * * *` | -| [Max jobs in active pipelines](../../administration/instance_limits.md#number-of-jobs-in-active-pipelines) | `500` for Free tier, unlimited otherwise | Unlimited | -| [Max CI/CD subscriptions to a project](../../administration/instance_limits.md#number-of-cicd-subscriptions-to-a-project) | `2` | Unlimited | -| [Max number of pipeline triggers in a project](../../administration/instance_limits.md#limit-the-number-of-pipeline-triggers) | `25000` for Free tier, Unlimited for all paid tiers | Unlimited | -| [Max pipeline schedules in projects](../../administration/instance_limits.md#number-of-pipeline-schedules) | `10` for Free tier, `50` for all paid tiers | Unlimited | -| [Max pipelines per schedule](../../administration/instance_limits.md#limit-the-number-of-pipelines-created-by-a-pipeline-schedule-per-day) | `24` for Free tier, `288` for all paid tiers | Unlimited | -| [Scheduled Job Archival](../../user/admin_area/settings/continuous_integration.md#archive-jobs) | 3 months | Never | -| Max test cases per [unit test report](../../ci/unit_test_reports.md) | `500_000` | Unlimited | -| [Max registered runners](../../administration/instance_limits.md#number-of-registered-runners-per-scope) | Free tier: `50` per-group / `50` per-project <br/> All paid tiers: `1_000` per-group / `1_000` per-project | `1_000` per-group / `1_000` per-project | -| [Limit dotenv variables](../../administration/instance_limits.md#limit-dotenv-variables) | Free tier: `50` / Premium tier: `100` / Ultimate tier: `150` | 150 | +| Setting | GitLab.com | Default (self-managed) | +|:-------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| Artifacts maximum size (compressed) | 1 GB | See [Maximum artifacts size](../../user/admin_area/settings/continuous_integration.md#maximum-artifacts-size) | +| Artifacts [expiry time](../../ci/yaml/index.md#artifactsexpire_in) | From June 22, 2020, deleted after 30 days unless otherwise specified (artifacts created before that date have no expiry). | See [Default artifacts expiration](../admin_area/settings/continuous_integration.md#default-artifacts-expiration) | +| Scheduled Pipeline Cron | `*/5 * * * *` | See [Pipeline schedules advanced configuration](../../administration/cicd.md#change-maximum-scheduled-pipeline-frequency) | +| Maximum jobs in active pipelines | `500` for Free tier, unlimited otherwise | See [Number of jobs in active pipelines](../../administration/instance_limits.md#number-of-jobs-in-active-pipelines) | +| Maximum CI/CD subscriptions to a project | `2` | See [Number of CI/CD subscriptions to a project](../../administration/instance_limits.md#number-of-cicd-subscriptions-to-a-project) | +| Maximum number of pipeline triggers in a project | `25000` for Free tier, Unlimited for all paid tiers | See [Limit the number of pipeline triggers](../../administration/instance_limits.md#limit-the-number-of-pipeline-triggers) | +| Maximum pipeline schedules in projects | `10` for Free tier, `50` for all paid tiers | See [Number of pipeline schedules](../../administration/instance_limits.md#number-of-pipeline-schedules) | +| Maximum pipelines per schedule | `24` for Free tier, `288` for all paid tiers | See [Limit the number of pipelines created by a pipeline schedule per day](../../administration/instance_limits.md#limit-the-number-of-pipelines-created-by-a-pipeline-schedule-per-day) | +| Scheduled job archiving | 3 months (from June 22, 2020). Jobs created before that date were archived after September 22, 2020. | Never | +| Maximum test cases per [unit test report](../../ci/unit_test_reports.md) | `500000` | Unlimited | +| Maximum registered runners | Free tier: `50` per-group / `50` per-project<br/>All paid tiers: `1000` per-group / `1000` per-project | See [Number of registered runners per scope](../../administration/instance_limits.md#number-of-registered-runners-per-scope) | +| Limit of dotenv variables | Free tier: `50` / Premium tier: `100` / Ultimate tier: `150` | See [Limit dotenv variables](../../administration/instance_limits.md#limit-dotenv-variables) | ## Package registry limits @@ -188,7 +191,7 @@ GitLab.com uses the IP ranges `34.74.90.64/28` and `34.74.226.0/24` for traffic fleet. This whole range is solely allocated to GitLab. You can expect connections from webhooks or repository mirroring to come from those IPs and allow them. -GitLab.com is fronted by Cloudflare. For incoming connections to GitLab.com, you might need to allow CIDR blocks of Cloudflare ([IPv4](https://www.cloudflare.com/ips-v4) and [IPv6](https://www.cloudflare.com/ips-v6)). +GitLab.com is fronted by Cloudflare. For incoming connections to GitLab.com, you might need to allow CIDR blocks of Cloudflare ([IPv4](https://www.cloudflare.com/ips-v4/) and [IPv6](https://www.cloudflare.com/ips-v6/)). For outgoing connections from CI/CD runners, we are not providing static IP addresses. All GitLab.com shared runners are deployed into Google Cloud Platform (GCP). Any @@ -296,7 +299,7 @@ for `shared_buffers` is quite high, and we are ## Puma -GitLab.com uses the default of 60 seconds for [Puma request timeouts](../../administration/operations/puma.md#worker-timeout). +GitLab.com uses the default of 60 seconds for [Puma request timeouts](../../administration/operations/puma.md#change-the-worker-timeout). ## GitLab.com-specific rate limits diff --git a/doc/user/group/clusters/index.md b/doc/user/group/clusters/index.md index b2b16321488..0da7eaa4d55 100644 --- a/doc/user/group/clusters/index.md +++ b/doc/user/group/clusters/index.md @@ -12,7 +12,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w WARNING: This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. To connect clusters to GitLab, -use the [GitLab Agent](../../clusters/agent/index.md). +use the [GitLab agent](../../clusters/agent/index.md). Similar to [project-level](../../project/clusters/index.md) and [instance-level](../../instance/clusters/index.md) Kubernetes clusters, diff --git a/doc/user/group/epics/img/related_epic_block_v14_9.png b/doc/user/group/epics/img/related_epic_block_v14_9.png Binary files differnew file mode 100644 index 00000000000..7b5824b84d1 --- /dev/null +++ b/doc/user/group/epics/img/related_epic_block_v14_9.png diff --git a/doc/user/group/epics/img/related_epics_add_v14_9.png b/doc/user/group/epics/img/related_epics_add_v14_9.png Binary files differnew file mode 100644 index 00000000000..3da6eeaff43 --- /dev/null +++ b/doc/user/group/epics/img/related_epics_add_v14_9.png diff --git a/doc/user/group/epics/img/related_epics_remove_v14_9.png b/doc/user/group/epics/img/related_epics_remove_v14_9.png Binary files differnew file mode 100644 index 00000000000..2cdded5965f --- /dev/null +++ b/doc/user/group/epics/img/related_epics_remove_v14_9.png diff --git a/doc/user/group/epics/index.md b/doc/user/group/epics/index.md index d6f87a026b8..149c5362ac9 100644 --- a/doc/user/group/epics/index.md +++ b/doc/user/group/epics/index.md @@ -62,6 +62,7 @@ You can also consult the [group permissions table](../../permissions.md#group-me ## Related topics - [Manage epics](manage_epics.md) and multi-level child epics. +- Link [related epics](linked_epics.md) based on a type of relationship. - Create workflows with [epic boards](epic_boards.md). - [Turn on notifications](../../profile/notifications.md) for about epic events. - [Award an emoji](../../award_emojis.md) to an epic or its comments. diff --git a/doc/user/group/epics/linked_epics.md b/doc/user/group/epics/linked_epics.md new file mode 100644 index 00000000000..b695bae39e4 --- /dev/null +++ b/doc/user/group/epics/linked_epics.md @@ -0,0 +1,79 @@ +--- +stage: Plan +group: Product Planning +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Linked epics **(ULTIMATE)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/353473) in GitLab 14.9 [with a flag](../../../administration/feature_flags.md) named `related_epics_widget`. Enabled by default. + +FLAG: +On self-managed GitLab, by default this feature is available. To hide the feature, +ask an administrator to [disable the feature flag](../../../administration/feature_flags.md) +named `related_epics_widget`. On GitLab.com, this feature is available. + +Linked epics are a bi-directional relationship between any two epics and appear in a block below +the epic description. You can link epics in different groups. + +The relationship only shows up in the UI if the user can see both epics. When you try to close an +epic that has open blockers, a warning is displayed. + +NOTE: +To manage linked epics through our API, visit the [epic links API documentation](../../../api/linked_epics.md). + +## Add a linked epic + +Prerequisites: + +- You must have at least the Reporter role for both groups. +- For GitLab SaaS: the epic that you're editing must be in a group on GitLab Ultimate. + The epics you're linking can be in a group on a lower tier. + +To link one epic to another: + +1. In the **Linked epics** section of an epic, + select the add linked epic button (**{plus}**). +1. Select the relationship between the two epics. Either: + - **relates to** + - **[blocks](#blocking-epics)** + - **[is blocked by](#blocking-epics)** +1. Enter the epic number or paste in the full URL of the epic. + +  + + Epics of the same group can be specified just by the reference number. + Epics from a different group require additional information like the + group name. For example: + + - The same group: `&44` + - Different group: `group&44` + + Valid references are added to a temporary list that you can review. + +1. Select **Add**. + +The linked epics are then displayed on the epic grouped by relationship. + + + +## Remove a linked epic + +Prerequisites: + +- You must have at least the Reporter role for the epic's group. + +To remove a linked epic, in the **Linked epics** section of an epic, +select **Remove** (**{close}**) next to +each epic. + +The relationship is removed from both epics. + + + +## Blocking epics + +When you [add a linked epic](#add-a-linked-epic), you can show that it **blocks** or +**is blocked by** another epic. + +If you try to close a blocked epic using the "Close epic" button, a confirmation message appears. diff --git a/doc/user/group/import/img/import_panel_v14_1.png b/doc/user/group/import/img/import_panel_v14_1.png Binary files differdeleted file mode 100644 index 52791a82c3c..00000000000 --- a/doc/user/group/import/img/import_panel_v14_1.png +++ /dev/null diff --git a/doc/user/group/import/img/new_group_navigation_v13_8.png b/doc/user/group/import/img/new_group_navigation_v13_8.png Binary files differdeleted file mode 100644 index 40be3dd41d2..00000000000 --- a/doc/user/group/import/img/new_group_navigation_v13_8.png +++ /dev/null diff --git a/doc/user/group/import/index.md b/doc/user/group/import/index.md index cdc3fe02a53..0c16b535ed1 100644 --- a/doc/user/group/import/index.md +++ b/doc/user/group/import/index.md @@ -132,3 +132,25 @@ migrated: - image URL - Boards - Board Lists + +## Troubleshooting Group Migration + +In a [rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session), +you can find the failure or error messages for the group import attempt using: + +```shell +# Get relevant import records +import = BulkImports::Entity.where(namespace_id: Group.id).bulk_import + +# Alternative lookup by user +import = BulkImport.where(user_id: User.find(...)).last + +# Get list of import entities. Each entity represents either a group or a project +entities = import.entities + +# Get a list of entity failures +entities.map(&:failures).flatten + +# Alternative failure lookup by status +entities.where(status: [-1]).pluck(:destination_name, :destination_namespace, :status) +``` diff --git a/doc/user/group/index.md b/doc/user/group/index.md index ec76dc52516..4b9ff7f64e8 100644 --- a/doc/user/group/index.md +++ b/doc/user/group/index.md @@ -46,19 +46,16 @@ the immediate parent group. ### Namespaces -In GitLab, a namespace is a unique name and URL for a user, a group, or subgroup. - -- `http://gitlab.example.com/username` -- `http://gitlab.example.com/groupname` -- `http://gitlab.example.com/groupname/subgroup_name` +In GitLab, a namespace is a unique name for a user, a group, or subgroup under +which a project can be created. For example, consider a user named Alex: -1. Alex creates an account with the username `alex`: `https://gitlab.example.com/alex` -1. Alex creates a group for their team with the group name `alex-team`. - The group and its projects are available at: `https://gitlab.example.com/alex-team` -1. Alex creates a subgroup of `alex-team` with the subgroup name `marketing`. - The subgroup and its projects are available at: `https://gitlab.example.com/alex-team/marketing` +| GitLab URL | Namespace | +| ---------- | --------- | +| Alex creates an account with the username `alex`: `https://gitlab.example.com/alex`. | The namespace in this case is `alex`. | +| Alex creates a group for their team with the group name `alex-team`. The group and its projects are available at: `https://gitlab.example.com/alex-team`. | The namespace in this cases is `alex-team`. | +| Alex creates a subgroup of `alex-team` with the subgroup name `marketing`. The subgroup and its projects are available at: `https://gitlab.example.com/alex-team/marketing`. | The namespace in this case is `alex-team/marketing`. | ## Create a group @@ -87,6 +84,7 @@ You can give a user access to all projects in a group. 1. On the top bar, select **Menu > Groups** and find your group. 1. On the left sidebar, select **Group information > Members**. +1. Select **Invite members**. 1. Fill in the fields. - The role applies to all projects in the group. [Learn more about permissions](../permissions.md). - On the **Access expiration date**, the user can no longer access projects in the group. @@ -174,6 +172,7 @@ Filter a group to find members. By default, all members in the group and subgrou - To view members in the group only, select **Membership = Direct**. - To view members of the group and its subgroups, select **Membership = Inherited**. - To view members with two-factor authentication enabled or disabled, select **2FA = Enabled** or **Disabled**. + - [In GitLab 14.0 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/349887), to view GitLab users created by [SAML SSO](saml_sso/index.md) or [SCIM provisioning](saml_sso/scim_setup.md) select **Enterprise = true**. ### Search a group @@ -207,31 +206,24 @@ A to-do item is created for all the group and subgroup members. ## Change the default branch protection of a group -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7583) in GitLab 12.9. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7583) in GitLab 12.9. +> - [Settings moved and renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/340403) in GitLab 14.9. By default, every group inherits the branch protection set at the global level. -To change this setting for a specific group: - -1. On the top bar, select **Menu > Groups**. -1. Select **Your Groups**. -1. Find the group and select it. -1. From the left menu, select **Settings > General**. -1. Expand the **Permissions and group features** section. -1. Select the desired option in the **Default branch protection** dropdown list. -1. Select **Save changes**. +To change this setting for a specific group, see [group level default branch protection](../project/repository/branches/default.md#group-level-default-branch-protection). -To change this setting globally, see [Default branch protection](../admin_area/settings/visibility_and_access_controls.md#protect-default-branches). +To change this setting globally, see [initial default branch protection](../project/repository/branches/default.md#instance-level-default-branch-protection). NOTE: -In [GitLab Premium or higher](https://about.gitlab.com/pricing/), GitLab administrators can choose to [disable group owners from updating the default branch protection](../admin_area/settings/visibility_and_access_controls.md#prevent-overrides-of-default-branch-protection). +In [GitLab Premium or higher](https://about.gitlab.com/pricing/), GitLab administrators can choose to [disable group owners from updating the default branch protection](../project/repository/branches/default.md#prevent-overrides-of-default-branch-protection). ## Add projects to a group There are two different ways to add a new project to a group: -- Select a group, and then click **New project**. You can then continue [creating your project](../../user/project/working_with_projects.md#create-a-project). -- While you are creating a project, select a group from the dropdown menu. +- Select a group, and then select **New project**. You can then continue [creating your project](../../user/project/working_with_projects.md#create-a-project). +- While you are creating a project, select a group from the dropdown list.  @@ -256,7 +248,7 @@ To change this setting globally, see [Default project creation protection](../ad ## Group activity analytics **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207164) in GitLab 12.10 as a [beta feature](https://about.gitlab.com/handbook/product/#beta). +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207164) in GitLab 12.10 as a [Beta feature](../../policy/alpha-beta-support.md#beta-features). For a group, you can view how many merge requests, issues, and members were created in the last 90 days. @@ -283,12 +275,8 @@ To view the activity feed in Atom format, select the > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/18328) in GitLab 12.7. > - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/247208) in GitLab 13.11 from a form to a modal window [with a flag](../feature_flags.md). Disabled by default. > - Modal window [enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/247208) in GitLab 14.8. - -FLAG: -On self-managed GitLab, by default the modal window feature is available. -To hide the feature, ask an administrator to [disable the feature flag](../../administration/feature_flags.md) -named `invite_members_group_modal`. -On GitLab.com, this feature is available. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/352526) in GitLab 14.9. + [Feature flag `invite_members_group_modal`](https://gitlab.com/gitlab-org/gitlab/-/issues/352526) removed. Similar to how you [share a project with a group](../project/members/share_project_with_groups.md), you can share a group with another group. Members get direct access @@ -308,7 +296,7 @@ All the members of the `Engineering` group are added to the `Frontend` group. ## Manage group memberships via LDAP **(PREMIUM SELF)** -Group syncing allows LDAP groups to be mapped to GitLab groups. This provides more control over per-group user management. To configure group syncing, edit the `group_base` **DN** (`'OU=Global Groups,OU=GitLab INT,DC=GitLab,DC=org'`). This **OU** contains all groups that will be associated with GitLab groups. +Group syncing allows LDAP groups to be mapped to GitLab groups. This provides more control over per-group user management. To configure group syncing, edit the `group_base` **DN** (`'OU=Global Groups,OU=GitLab INT,DC=GitLab,DC=org'`). This **OU** contains all groups that are associated with GitLab groups. Group links can be created by using either a CN or a filter. To create these group links, go to the group's **Settings > LDAP Synchronization** page. After configuring the link, it may take more than an hour for the users to sync with the GitLab group. @@ -325,9 +313,9 @@ To create group links via CN: 1. Select the **LDAP Server** for the link. 1. As the **Sync method**, select `LDAP Group cn`. -1. In the **LDAP Group cn** field, begin typing the CN of the group. There is a dropdown menu with matching CNs in the configured `group_base`. Select your CN from this list. +1. In the **LDAP Group cn** field, begin typing the CN of the group. There is a dropdown list with matching CNs in the configured `group_base`. Select your CN from this list. 1. In the **LDAP Access** section, select the [permission level](../permissions.md) for users synced in this group. -1. Select the **Add Synchronization** button. +1. Select **Add Synchronization**. <!-- vale gitlab.Spelling = YES --> @@ -339,7 +327,7 @@ To create group links via filter: 1. As the **Sync method**, select `LDAP user filter`. 1. Input your filter in the **LDAP User filter** box. Follow the [documentation on user filters](../../administration/auth/ldap/index.md#set-up-ldap-user-filter). 1. In the **LDAP Access** section, select the [permission level](../permissions.md) for users synced in this group. -1. Select the **Add Synchronization** button. +1. Select **Add Synchronization**. ### Override user permissions **(PREMIUM SELF)** @@ -347,7 +335,7 @@ LDAP user permissions can be manually overridden by an administrator. To overrid 1. Go to your group's **Group information > Members** page. 1. In the row for the user you are editing, select the pencil (**{pencil}**) icon. -1. Select the brown **Edit permissions** button in the modal. +1. Select **Edit permissions** in the modal. Now you can edit the user's permissions from the **Members** page. @@ -375,7 +363,7 @@ Changing a group's path (group URL) can have unintended side effects. Read before you proceed. If you are changing the path so it can be claimed by another group or user, -you may need to rename the group too. Both names and paths must +you must rename the group too. Both names and paths must be unique. To retain ownership of the original namespace and protect the URL redirects, @@ -384,7 +372,7 @@ create a new group and transfer projects to it instead. To change your group path (group URL): 1. Go to your group's **Settings > General** page. -1. Expand the **Path, transfer, remove** section. +1. Expand the **Advanced** section. 1. Under **Change group URL**, enter a new name. 1. Select **Change group URL**. @@ -467,7 +455,7 @@ To restore a group that is marked for deletion: This setting is only available on top-level groups. It affects all subgroups. -When checked, any group within the top-level group hierarchy can be shared only with other groups within the hierarchy. +When checked, any group in the top-level group hierarchy can be shared only with other groups in the hierarchy. For example, with these groups: @@ -496,7 +484,7 @@ To prevent a project from being shared with other groups: 1. Go to the group's **Settings > General** page. 1. Expand the **Permissions and group features** section. -1. Select **Prevent sharing a project within `<group_name>` with other groups**. +1. Select **Prevent sharing a project in `<group_name>` with other groups**. 1. Select **Save changes**. This setting applies to all subgroups unless overridden by a group owner. Groups already @@ -559,6 +547,8 @@ Decreasing the user cap does not approve pending members. When the number of billable users reaches the user cap, any new member is put in a pending state and must be approved. +Pending members do not count as billable. Members count as billable only after they have been approved and are no longer in a pending state. + Prerequisite: - You must be assigned the Owner role) for the group. @@ -586,7 +576,7 @@ To prevent members from being added to projects in a group: 1. Go to the group's **Settings > General** page. 1. Expand the **Permissions and group features** section. -1. Under **Member lock**, select **Prevent adding new members to project membership within this group**. +1. Under **Membership**, select **Prevent adding new members to projects within this group**. 1. Select **Save changes**. All users who previously had permissions can no longer add members to a group. @@ -601,7 +591,7 @@ You can export a list of members in a group or subgroup as a CSV. 1. Go to your group or subgroup and select either **Group information > Members** or **Subgroup information > Members**. 1. Select **Export as CSV**. -1. Once the CSV file has been generated, it is emailed as an attachment to the user that requested it. +1. After the CSV file has been generated, it is emailed as an attachment to the user that requested it. ## Restrict group access by IP address **(PREMIUM)** @@ -646,7 +636,7 @@ To restrict group access by IP address: > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7297) in GitLab 12.2. > - Support for specifying multiple email domains [added](https://gitlab.com/gitlab-org/gitlab/-/issues/33143) in GitLab 13.1. -> - Support for restricting access to projects within the group [added](https://gitlab.com/gitlab-org/gitlab/-/issues/14004) in GitLab 14.1.2. +> - Support for restricting access to projects in the group [added](https://gitlab.com/gitlab-org/gitlab/-/issues/14004) in GitLab 14.1.2. You can prevent users with email addresses in specific domains from being added to a group and its projects. @@ -663,7 +653,7 @@ Any time you attempt to add a new user, the user's [primary email](../profile/in Only users with a [primary email](../profile/index.md#change-your-primary-email) that matches any of the configured email domain restrictions can be added to the group. -Some domains cannot be restricted. These are the most popular public email domains, such as: +The most popular public email domains cannot be restricted, such as: - `gmail.com`, `yahoo.com`, `aol.com`, `icloud.com` - `hotmail.com`, `hotmail.co.uk`, `hotmail.fr` @@ -718,9 +708,10 @@ To disable email notifications: > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/21301) in GitLab 12.6. You can prevent users from being added to a conversation and getting notified when -anyone mentions a group in which those users are members. +anyone [mentions a group](../discussions/index.md#mentions) +in which those users are members. -Groups with disabled mentions are visualized accordingly in the autocompletion dropdown. +Groups with disabled mentions are visualized accordingly in the autocompletion dropdown list. This is particularly helpful for groups with a large number of users. @@ -806,9 +797,7 @@ The group's new subgroups have push rules set for them based on either: > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/285458) in GitLab 13.9. [Deployed behind the `group_merge_request_approval_settings_feature_flag` flag](../../administration/feature_flags.md), disabled by default. > - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/285410) in GitLab 14.5. - -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature per group, ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named `group_merge_request_approval_settings_feature_flag`. On GitLab.com, this feature is available. +> - [Feature flag `group_merge_request_approval_settings_feature_flag`](https://gitlab.com/gitlab-org/gitlab/-/issues/343872) removed in GitLab 14.9. Group approval rules manage [project merge request approval rules](../project/merge_requests/approvals/index.md) at the top-level group level. These rules [cascade to all projects](../project/merge_requests/approvals/settings.md#settings-cascading) diff --git a/doc/user/group/iterations/index.md b/doc/user/group/iterations/index.md index c0bd6f1a672..b5912a0b40e 100644 --- a/doc/user/group/iterations/index.md +++ b/doc/user/group/iterations/index.md @@ -142,6 +142,7 @@ To view an iteration report, go to the iterations list page and select an iterat > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/222750) in GitLab 13.6. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/269972) in GitLab 13.7. +> - Scoped burnup and burndown charts in subgroups and projects [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326029) in GitLab 14.9. The iteration report includes [burndown and burnup charts](../../project/milestones/burndown_and_burnup_charts.md), similar to how they appear when viewing a [milestone](../../project/milestones/index.md). @@ -149,6 +150,33 @@ similar to how they appear when viewing a [milestone](../../project/milestones/i Burndown charts help track completion progress of total scope, and burnup charts track the daily total count and weight of issues added to and completed in a given timebox. +#### Iteration charts scoped to subgroups or projects + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/326029) in GitLab 14.9. + +You can view burndown and burnup charts for iterations created for a group in any of its +subgroups or projects. +When you do this, the charts only count the issues that belong to the subgroup or project. + +For example, suppose a group has two projects named `Project 1` and `Project 2`. +Each project has a single issue assigned to the same iteration from the group. + +An iteration report generated for the group shows issue counts for all the group's projects: + +- Completed: 0 of 2 +- Incomplete: 0 of 2 +- Unstarted: 2 of 2 +- Burndown chart total issues: 2 +- Burnup chart total issues: 2 + +An iteration report generated for `Project 1` shows only issues that belong to this project: + +- Completed: 0 of 1 +- Incomplete: 0 of 1 +- Unstarted: 1 of 1 +- Burndown chart total issues: 1 +- Burnup chart total issues: 1 + ### Group issues by label > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/225500) in GitLab 13.8. diff --git a/doc/user/group/planning_hierarchy/img/epic-view-ancestors-in-sidebar_v14_6.png b/doc/user/group/planning_hierarchy/img/epic-view-ancestors-in-sidebar_v14_6.png Binary files differindex 373b861239b..d4ba8acf9b9 100644 --- a/doc/user/group/planning_hierarchy/img/epic-view-ancestors-in-sidebar_v14_6.png +++ b/doc/user/group/planning_hierarchy/img/epic-view-ancestors-in-sidebar_v14_6.png diff --git a/doc/user/group/planning_hierarchy/img/issue-view-parent-epic-in-sidebar_v14_6.png b/doc/user/group/planning_hierarchy/img/issue-view-parent-epic-in-sidebar_v14_6.png Binary files differindex 95a5777674a..f3b6a80ea66 100644 --- a/doc/user/group/planning_hierarchy/img/issue-view-parent-epic-in-sidebar_v14_6.png +++ b/doc/user/group/planning_hierarchy/img/issue-view-parent-epic-in-sidebar_v14_6.png diff --git a/doc/user/group/planning_hierarchy/img/view-project-work-item-hierarchy_v14_8.png b/doc/user/group/planning_hierarchy/img/view-project-work-item-hierarchy_v14_8.png Binary files differdeleted file mode 100644 index 2ddd551ee46..00000000000 --- a/doc/user/group/planning_hierarchy/img/view-project-work-item-hierarchy_v14_8.png +++ /dev/null diff --git a/doc/user/group/planning_hierarchy/index.md b/doc/user/group/planning_hierarchy/index.md index 934421e8a9a..6ffc47923f2 100644 --- a/doc/user/group/planning_hierarchy/index.md +++ b/doc/user/group/planning_hierarchy/index.md @@ -5,7 +5,7 @@ group: Product Planning info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Planning hierarchies **(FREE)** +# Planning hierarchies **(PREMIUM)** Planning hierarchies are an integral part of breaking down your work in GitLab. To understand how you can use epics and issues together in hierarchies, remember the following: @@ -20,21 +20,7 @@ To learn about hierarchies in general, common frameworks, and using GitLab for portfolio management, see [How to use GitLab for Agile portfolio planning and project management](https://about.gitlab.com/blog/2020/11/11/gitlab-for-agile-portfolio-planning-project-management/). -## View planning hierarchies - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340844/) in GitLab 14.8. - -To view the planning hierarchy in a project: - -1. On the top bar, select **Menu > Projects** and find your project. -1. On the left sidebar, select **Project information > Planning hierarchy**. - -Under **Current structure**, you can see a hierarchy diagram that matches your current planning hierarchy. -The work items outside your subscription plan show up below **Unavailable structure**. - - - -## Hierarchies with epics **(PREMIUM)** +## Hierarchies with epics With epics, you can achieve the following hierarchy: @@ -74,7 +60,7 @@ In an issue, you can view the parented epic above the issue in the right sidebar  -## View ancestry of an epic **(PREMIUM)** +## View ancestry of an epic In an epic, you can view the ancestors as parents in the right sidebar under **Ancestors**. diff --git a/doc/user/group/roadmap/img/epics_state_dropdown_v14_3.png b/doc/user/group/roadmap/img/epics_state_dropdown_v14_3.png Binary files differdeleted file mode 100644 index 171876e34a9..00000000000 --- a/doc/user/group/roadmap/img/epics_state_dropdown_v14_3.png +++ /dev/null diff --git a/doc/user/group/roadmap/index.md b/doc/user/group/roadmap/index.md index f10dc3bc22a..89c5c6ed466 100644 --- a/doc/user/group/roadmap/index.md +++ b/doc/user/group/roadmap/index.md @@ -24,12 +24,12 @@ When you hover over an epic bar, a popover appears with the epic's title, start weight completed. You can expand epics that contain child epics to show their child epics in the roadmap. -You can click the chevron (**{chevron-down}**) next to the epic title to expand and collapse the +You can select the chevron (**{chevron-down}**) next to the epic title to expand and collapse the child epics. On top of the milestone bars, you can see their title. When you hover over a milestone bar or title, a popover appears with its title, start date, and due -date. You can also click the chevron (**{chevron-down}**) next to the **Milestones** heading to +date. You can also select the chevron (**{chevron-down}**) next to the **Milestones** heading to toggle the list of the milestone bars.  @@ -47,10 +47,6 @@ Filtering roadmaps by milestone might not be available to you. Check the **versi When you want to explore a roadmap, there are several ways to make it easier by sorting epics or filtering them by what's important for you. -A dropdown menu lets you show only open or closed epics. By default, all epics are shown. - - - You can sort epics in the Roadmap view by: - Start date @@ -74,18 +70,15 @@ Roadmaps can also be [visualized inside an epic](../epics/index.md#roadmap-in-ep ### Roadmap settings -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345158) in GitLab 14.8 [with a flag](../../../administration/feature_flags.md) named `roadmap_settings`. Enabled by default. - -FLAG: -On self-managed GitLab, by default this feature is not available. To make it available, ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `roadmap_settings`. -On GitLab.com, this feature is available but can be configured by GitLab.com administrators only. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/345158) in GitLab 14.8 [with a flag](../../../administration/feature_flags.md) named `roadmap_settings`. Enabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/350830) in GitLab 14.9. Feature flag `roadmap_settings`removed. When you enable the roadmap settings sidebar, you can use it to refine epics shown in the roadmap. You can configure the following: - Select date range. -- Turn milestones on or off and select whether to show all, group, sub-group, or project milestones. +- Turn milestones on or off and select whether to show all, group, subgroup, or project milestones. - Show all, open, or closed epics. - Turn progress tracking for child issues on or off and select whether to use issue weights or counts. diff --git a/doc/user/group/saml_sso/group_managed_accounts.md b/doc/user/group/saml_sso/group_managed_accounts.md index aeb7db923a9..bffaef40800 100644 --- a/doc/user/group/saml_sso/group_managed_accounts.md +++ b/doc/user/group/saml_sso/group_managed_accounts.md @@ -65,7 +65,7 @@ This restriction also applies to projects forked from or to those groups. > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/34648) in GitLab 12.9. -Groups with group-managed accounts can disallow forking of projects to destinations outside the group. +Groups with group-managed accounts can prevent forking of projects to destinations outside the group. To do so, enable the "Prohibit outer forks" option in **Settings > SAML SSO**. When enabled **at the parent group level**, projects within the group can be forked only to other destinations within the group (including its subgroups). diff --git a/doc/user/group/saml_sso/index.md b/doc/user/group/saml_sso/index.md index 14c4447c5c6..8ebcd9f62d0 100644 --- a/doc/user/group/saml_sso/index.md +++ b/doc/user/group/saml_sso/index.md @@ -176,7 +176,7 @@ See the [troubleshooting page](../../../administration/troubleshooting/group_sam ### Okta setup notes -Please follow the Okta documentation on [setting up a SAML application in Okta](https://developer.okta.com/docs/guides/build-sso-integration/saml2/overview/) with the notes below for consideration. +Please follow the Okta documentation on [setting up a SAML application in Okta](https://developer.okta.com/docs/guides/build-sso-integration/saml2/main/) with the notes below for consideration. <i class="fa fa-youtube-play youtube" aria-hidden="true"></i> For a demo of the Okta SAML setup including SCIM, see [Demo: Okta Group SAML & SCIM setup](https://youtu.be/0ES9HsZq0AQ). @@ -214,6 +214,35 @@ we recommend the ["Use the OneLogin SAML Test Connector" documentation](https:// Recommended `NameID` value: `OneLogin ID`. +### Change the SAML app + +To change the SAML app used for sign in: + +- If the NameID is not identical in both the existing and new SAML apps, users must: + 1. [Unlink the current SAML identity](#unlinking-accounts). + 1. [Link their identity](#user-access-and-management) to the new SAML app. +- If the NameID is identical, no change is required. + +### Migrate to a different SAML provider + +You can migrate to a different SAML provider. During the migration process users will not be able to access any of the SAML groups. +To mitigate this, you can disable [SSO enforcement](#sso-enforcement). + +To migrate SAML providers: + +1. [Configure](#configure-your-identity-provider) the group with the new identity provider SAML app. +1. Ask users to [unlink their account from the group](#unlinking-accounts). +1. Ask users to [link their account to the new SAML app](#linking-saml-to-your-existing-gitlabcom-account). + +### Change email domains + +To migrate users to a new email domain, users must: + +1. Add their new email as the primary email to their accounts and verify it. +1. [Unlink their account from the group](#unlinking-accounts). +1. [Link their account to the group](#linking-saml-to-your-existing-gitlabcom-account). +1. (Optional) Remove their old email from the account. + ## User access and management > [Improved](https://gitlab.com/gitlab-org/gitlab/-/issues/268142) in GitLab 13.7. @@ -610,12 +639,6 @@ Alternatively, when users need to [link SAML to their existing GitLab.com accoun | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | As mentioned in the [NameID](#nameid) section, if the NameID changes for any user, the user can be locked out. This is a common problem when an email address is used as the identifier. | Follow the steps outlined in the ["SAML authentication failed: User has already been taken"](#message-saml-authentication-failed-user-has-already-been-taken) section. | -### I need to change my SAML app - -If the NameID is identical in both SAML apps, then no change is required. - -Otherwise, to change the SAML app used for sign in, users need to [unlink the current SAML identity](#unlinking-accounts) and then [link their identity](#user-access-and-management) to the new SAML app. - ### I need additional information to configure my identity provider Many SAML terms can vary between providers. It is possible that the information you are looking for is listed under another name. diff --git a/doc/user/group/saml_sso/scim_setup.md b/doc/user/group/saml_sso/scim_setup.md index d1e9ba29378..331288e33a1 100644 --- a/doc/user/group/saml_sso/scim_setup.md +++ b/doc/user/group/saml_sso/scim_setup.md @@ -51,7 +51,7 @@ Once [Group Single Sign-On](index.md) has been configured, we can: The SAML application that was created during [Single sign-on](index.md) setup for [Azure](https://docs.microsoft.com/en-us/azure/active-directory/manage-apps/view-applications-portal) now needs to be set up for SCIM. You can refer to [Azure SCIM setup documentation](https://docs.microsoft.com/en-us/azure/active-directory/app-provisioning/use-scim-to-provision-users-and-groups#getting-started). -1. In your app, go to the Provisioning tab, and set the **Provisioning Mode** to **Automatic**. +1. In your app, go to the Provisioning tab, and set the **Provisioning Mode** to **Automatic**. Then fill in the **Admin Credentials**, and save. The **Tenant URL** and **secret token** are the items retrieved in the [previous step](#gitlab-configuration). @@ -60,7 +60,7 @@ The SAML application that was created during [Single sign-on](index.md) setup fo - **Settings**: We recommend setting a notification email and selecting the **Send an email notification when a failure occurs** checkbox. You also control what is actually synced by selecting the **Scope**. For example, **Sync only assigned users and groups** only syncs the users and groups assigned to the application. Otherwise, it syncs the whole Active Directory. - - **Mappings**: We recommend keeping **Provision Azure Active Directory Users** enabled, and disable **Provision Azure Active Directory Groups**. + - **Mappings**: We recommend keeping **Provision Azure Active Directory Users** enabled, and disable **Provision Azure Active Directory Groups**. Leaving **Provision Azure Active Directory Groups** enabled does not break the SCIM user provisioning, but it causes errors in Azure AD that may be confusing and misleading. 1. You can then test the connection by selecting **Test Connection**. If the connection is successful, save your configuration before moving on. See below for [troubleshooting](#troubleshooting). @@ -100,12 +100,14 @@ Once synchronized, changing the field mapped to `id` and `externalId` may cause ### Okta configuration steps -Before you start this section, complete the [GitLab configuration](#gitlab-configuration) process. -Make sure that you've also set up a SAML application for [Okta](https://developer.okta.com/docs/guides/build-sso-integration/saml2/overview/), -as described in the [Okta setup notes](index.md#okta-setup-notes) +Before you start this section: -Make sure that the Okta setup matches our documentation exactly, especially the NameID -configuration. Otherwise, the Okta SCIM app may not work properly. +- Check that you are using Okta [Lifecycle Management](https://www.okta.com/products/lifecycle-management/) product. This product tier is required to use SCIM on Okta. To check which Okta product you are using, check your signed Okta contract, contact your Okta AE, CSM, or Okta support. +- Complete the [GitLab configuration](#gitlab-configuration) process. +- Complete the setup for SAML application for [Okta](https://developer.okta.com/docs/guides/build-sso-integration/saml2/overview/), as described in the [Okta setup notes](index.md#okta-setup-notes). +- Check that your Okta SAML setup matches our documentation exactly, especially the NameID configuration. Otherwise, the Okta SCIM app may not work properly. + +After the above steps are complete: 1. Sign in to Okta. 1. Ensure you are in the Admin section by selecting the **Admin** button located in the top right. The admin button is not visible from the admin page. @@ -169,7 +171,7 @@ We recommend users do this prior to turning on sync, because while synchronizati New users and existing users on subsequent visits can access the group through the identify provider's dashboard or by visiting links directly. -[In GitLab 14.0 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/325712), GitLab users created with a SCIM identity display with an **Enterprise** badge in the **Members** view. +[In GitLab 14.0 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/325712), GitLab users created by [SAML SSO](index.md#user-access-and-management) or SCIM provisioning display with an **Enterprise** badge in the **Members** view.  @@ -244,7 +246,7 @@ It is important not to update these to incorrect values, since this causes users ### I need to change my SCIM app -Individual users can follow the instructions in the ["SAML authentication failed: User has already been taken"](index.md#i-need-to-change-my-saml-app) section. +Individual users can follow the instructions in the ["SAML authentication failed: User has already been taken"](index.md#change-the-saml-app) section. Alternatively, users can be removed from the SCIM app which de-links all removed users. Sync can then be turned on for the new SCIM app to [link existing users](#user-access-and-linking-setup). diff --git a/doc/user/group/settings/group_access_tokens.md b/doc/user/group/settings/group_access_tokens.md index 769943cd5d8..6b20f1763d4 100644 --- a/doc/user/group/settings/group_access_tokens.md +++ b/doc/user/group/settings/group_access_tokens.md @@ -44,7 +44,7 @@ To create a group access token: 1. On the top bar, select **Menu > Groups** and find your group. 1. On the left sidebar, select **Settings > Access Tokens**. -1. Enter a name. +1. Enter a name. The token name is visible to any user with permissions to view the group. 1. Optional. Enter an expiry date for the token. The token will expire on that date at midnight UTC. 1. Select a role for the token. 1. Select the [desired scopes](#scopes-for-a-group-access-token). diff --git a/doc/user/group/subgroups/img/mention_subgroups.png b/doc/user/group/subgroups/img/mention_subgroups.png Binary files differdeleted file mode 100644 index ec370add4f9..00000000000 --- a/doc/user/group/subgroups/img/mention_subgroups.png +++ /dev/null diff --git a/doc/user/group/subgroups/index.md b/doc/user/group/subgroups/index.md index 46dea81ae9f..a98f213bb3f 100644 --- a/doc/user/group/subgroups/index.md +++ b/doc/user/group/subgroups/index.md @@ -2,189 +2,176 @@ stage: Manage group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: reference, howto, concepts --- # Subgroups **(FREE)** > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/2772) in GitLab 9.0. -GitLab supports up to 20 levels of subgroups, also known as nested groups or hierarchical groups. - -By using subgroups you can do the following: - -- **Separate internal / external organizations.** Since every group - can have its own visibility level ([public, internal, or private](../../../development/permissions.md#general-permissions)), - you're able to host groups for different purposes under the same umbrella. -- **Organize large projects.** For large projects, subgroups makes it - potentially easier to separate permissions on parts of the source code. -- **Make it easier to manage people and control visibility.** Give people - different [permissions](../../permissions.md#group-members-permissions) depending on their group [membership](#membership). - -For more information on allowed permissions in groups and projects, see -[visibility levels](../../../development/permissions.md#general-permissions). - -## Overview - -A group can have many subgroups inside it, and at the same time a group can have -only one immediate parent group. It resembles a directory behavior or a nested items list: - -- Group 1 - - Group 1.1 - - Group 1.2 - - Group 1.2.1 - - Group 1.2.2 - - Group 1.2.2.1 - -In a real world example, imagine maintaining a GNU/Linux distribution with the -first group being the name of the distribution, and subsequent groups split as follows: - -- Organization Group - GNU/Linux distro - - Category Subgroup - Packages - - (project) Package01 - - (project) Package02 - - Category Subgroup - Software - - (project) Core - - (project) CLI - - (project) Android app - - (project) iOS app - - Category Subgroup - Infra tools - - (project) Ansible playbooks - -Another example of GitLab as a company would be the following: - -- Organization Group - GitLab - - Category Subgroup - Marketing - - (project) Design - - (project) General - - Category Subgroup - Software - - (project) GitLab CE - - (project) GitLab EE - - (project) Omnibus GitLab - - (project) GitLab Runner - - (project) GitLab Pages daemon - - Category Subgroup - Infra tools - - (project) Chef cookbooks - - Category Subgroup - Executive team - -When performing actions such as transferring or importing a project between -subgroups, the behavior is the same as when performing these actions at the -`group/project` level. - -## Creating a subgroup - -Users can create subgroups if they are explicitly added as an Owner (or -Maintainer, if that setting is enabled) to an immediate parent group, even if group -creation is disabled by an administrator in their settings. +You can organize GitLab [groups](../index.md) into subgroups. You can use subgroups to: + +- Separate internal and external organizations. Because every subgroup can have its own + [visibility level](../../../development/permissions.md#general-permissions), you can host groups for different + purposes under the same parent group. +- Organize large projects. You can use subgroups to give different access to parts of + the source code. +- Manage people and control visibility. Give a user a different + [role](../../permissions.md#group-members-permissions) for each group they're [a member of](#subgroup-membership). + +Subgroups can: + +- Belong to one immediate parent group. +- Have many subgroups. +- Be nested up to 20 levels. +- Use [runners](../../../ci/runners/index.md) registered to parent groups: + - Secrets configured for the parent group are available to subgroup jobs. + - Users with the Maintainer role in projects that belong to subgroups can see the details of runners registered to + parent groups. + +For example: + +```mermaid +graph TD + subgraph "Parent group" + subgraph "Subgroup A" + subgraph "Subgroup A1" + G["Project E"] + end + C["Project A"] + D["Project B"] + E["Project C"] + end + subgraph "Subgroup B" + F["Project D"] + end + end +``` + +## Create a subgroup + +Users with the at least the Maintainer role on a group can create subgroups immediately below the group, unless +[configured otherwise](#change-who-can-create-subgroups). These users can create subgroups even if group creation is +[disabled by an Administrator](../../admin_area/index.md#prevent-a-user-from-creating-groups) in the user's settings. To create a subgroup: -1. On the top bar, select **Menu > Groups** and find the parent group. -1. In the top right, select **New subgroup**. +1. On the top bar, select **Menu > Groups** and find and select the parent group to add a subgroup to. +1. On the parent group's overview page, in the top right, select **New subgroup**. 1. Select **Create group**. -1. Fill in the fields. View a list of [reserved names](../../reserved_names.md) - that cannot be used as group names. +1. Fill in the fields. View a list of [reserved names](../../reserved_names.md) that cannot be used as group names. 1. Select **Create group**. ### Change who can create subgroups -To create a subgroup you must either be an Owner or a Maintainer of the -group, depending on the group's setting. +To create a subgroup, you must have at least the Maintainer role on the group, depending on the group's setting. By +default: -By default: +- In GitLab 12.2 or later, users with at least the Maintainer role can create subgroups. +- In GitLab 12.1 or earlier, only users with the Owner role can create subgroups. -- In GitLab 12.2 or later, both Owners and Maintainers can create subgroups. -- In GitLab 12.1 or earlier, only Owners can create subgroups. +To change who can create subgroups on a group: -You can change this setting: - -- As group owner: - 1. Select the group. +- As a user with the Owner role on the group: + 1. On the top bar, select **Menu > Groups** and find the group. 1. On the left sidebar, select **Settings > General**. 1. Expand **Permissions and group features**. + 1. Select a role from the **Allowed to create subgroups** dropdown. - As an administrator: 1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Overview > Groups**. 1. Select the group, and select **Edit**. + 1. Select a role from the **Allowed to create subgroups** dropdown. For more information, view the [permissions table](../../permissions.md#group-members-permissions). -## Membership - -When you add a member to a group, that member is also added to all subgroups. -Permission level is inherited from the group's parent. This model allows access to -subgroups if you have membership in one of its parents. +## Subgroup membership -Jobs for pipelines in subgroups can use [runners](../../../ci/runners/index.md) registered to the parent group(s). -This means secrets configured for the parent group are available to subgroup jobs. +When you add a member to a group, that member is also added to all subgroups. The user's permissions are inherited from +the group's parent. -In addition, maintainers of projects that belong to subgroups can see the details of runners registered to parent group(s). +Subgroup members can: -The group permissions for a member can be changed only by Owners, and only on -the **Members** page of the group the member was added. +1. Be [direct members](../../project/members/index.md#add-users-to-a-project) of the subgroup. +1. [Inherit membership](../../project/members/index.md#inherited-membership) of the subgroup from the subgroup's parent group. +1. Be a member of a group that was [shared with the subgroup's top-level group](../index.md#share-a-group-with-another-group). -You can tell if a member has inherited the permissions from a parent group by -looking at the group's **Members** page. +```mermaid +flowchart RL + subgraph Group A + A(Direct member) + B{{Shared member}} + subgraph Subgroup A + H(1. Direct member) + C{{2. Inherited member}} + D{{Inherited member}} + E{{3. Shared member}} + end + A-->|Direct membership of Group A\nInherited membership of Subgroup A|C + end + subgraph Group C + G(Direct member) + end + subgraph Group B + F(Direct member) + end + F-->|Group B\nshared with\nGroup A|B + B-->|Inherited membership of Subgroup A|D + G-->|Group C shared with Subgroup A|E +``` - +Group permissions for a member can be changed only by: -From the image above, we can deduce the following things: +- Users with the Owner role on the group. +- Changing the configuration of the group the member was added to. -- There are 5 members that have access to the group `four`. -- User 0 is a Reporter and has inherited their permissions from group `one` - which is above the hierarchy of group `four`. -- User 1 is a Developer and has inherited their permissions from group - `one/two` which is above the hierarchy of group `four`. -- User 2 is a Developer and has inherited their permissions from group - `one/two/three` which is above the hierarchy of group `four`. -- For User 3 the **Source** column indicates **Direct member**, therefore they belong to - group `four`, the one we're inspecting. -- Administrator is the Owner and member of **all** subgroups and for that reason, - as with User 3, the **Source** column indicates **Direct member**. +### Determine membership inheritance -Members can be [filtered by inherited or direct membership](../index.md#filter-a-group). +To see if a member has inherited the permissions from a parent group: -### Overriding the ancestor group membership +1. On the top bar, select **Menu > Groups** and find the group. +1. Select **Group information > Members**. -NOTE: -You must be an Owner of a group to be able to add members to it. +Members list for an example subgroup _Four_: -NOTE: -A user's permissions in a subgroup cannot be lower than in any of its ancestor groups. -Therefore, you cannot reduce a user's permissions in a subgroup with respect to its ancestor groups. + -To override a user's membership of an ancestor group (the first group they were -added to), add the user to the new subgroup again with a higher set of permissions. +In the screenshot above: + +- Five members have access to group _Four_. +- User 0 has the Reporter role on group _Four_, and has inherited their permissions from group _One_: + - User 0 is a direct member of group _One_. + - Group _One_ is above group _Four_ in the hierarchy. +- User 1 has the Developer role on group _Four_ and inherited their permissions from group _Two_: + - User 0 is a direct member of group _Two_, which is a subgroup of group _One_. + - Groups _One / Two_ are above group _Four_ in the hierarchy. +- User 2 has the Developer role on group _Four_ and has inherited their permissions from group _Three_: + - User 0 is a direct member of group _Three_, which is a subgroup of group _Two_. Group _Two_ is a subgroup of group + _One_. + - Groups _One / Two / Three_ are above group _Four_ the hierarchy. +- User 3 is a direct member of group _Four_. This means they get their Maintainer role directly from group _Four_. +- Administrator has the Owner role on group _Four_ and is a member of all subgroups. For that reason, as with User 3, + the **Source** column indicates they are a direct member. -For example, if User 1 was first added to group `one/two` with Developer -permissions, then they inherit those permissions in every other subgroup -of `one/two`. To give them the Maintainer role for group `one/two/three/four`, -you would add them again in that group as Maintainer. Removing them from that group, -the permissions fall back to those of the ancestor group. +Members can be [filtered by inherited or direct membership](../index.md#filter-a-group). -## Mentioning subgroups +### Override ancestor group membership -Mentioning groups (`@group`) in issues, commits and merge requests, would -notify all members of that group. Now with subgroups, there is more granular -support if you want to split your group's structure. Mentioning works as before -and you can choose the group of people to be notified. +Users with the Owner role on a subgroup can add members to it. - +You can't give a user a role on a subgroup that's lower than the roles they have on ancestor groups. To override a user's +role on an ancestor group, add the user to the subgroup again with a higher role. For example: -## Limitations +- If User 1 is added to group _Two_ with the Developer role, they inherit that role in every subgroup of group _Two_. +- To give User 1 the Maintainer role on group _Four_ (under _One / Two / Three_), add them again to group _Four_ with + the Maintainer role. +- If User 1 is removed from group _Four_, their role falls back to their role on group _Two_. They have the Developer + role on group _Four_ again. -Here's a list of what you can't do with subgroups: +## Mention subgroups -- [GitLab Pages](../../project/pages/index.md) supports projects hosted under - a subgroup, but not subgroup websites. - That means that only the highest-level group supports - [group websites](../../project/pages/getting_started_part_one.md#gitlab-pages-default-domain-names), - although you can have project websites under a subgroup. -- It is not possible to share a project with a group that's an ancestor of - the group the project is in. That means you can only share as you walk down - the hierarchy. For example, `group/subgroup01/project` **cannot** be shared - with `group`, but can be shared with `group/subgroup02` or - `group/subgroup01/subgroup03`. +Mentioning subgroups ([`@<subgroup_name>`](../../discussions/index.md#mentions)) in issues, commits, and merge requests +notifies all members of that group. Mentioning works the same as for projects and groups, and you can choose the group +of people to be notified. <!-- ## Troubleshooting diff --git a/doc/user/group/value_stream_analytics/img/vsa_stage_table_v14_7.png b/doc/user/group/value_stream_analytics/img/vsa_stage_table_v14_7.png Binary files differindex c9074cbb5ea..7c3305792bb 100644 --- a/doc/user/group/value_stream_analytics/img/vsa_stage_table_v14_7.png +++ b/doc/user/group/value_stream_analytics/img/vsa_stage_table_v14_7.png diff --git a/doc/user/group/value_stream_analytics/index.md b/doc/user/group/value_stream_analytics/index.md index ccf77f79fd9..5ea8512ed5a 100644 --- a/doc/user/group/value_stream_analytics/index.md +++ b/doc/user/group/value_stream_analytics/index.md @@ -2,84 +2,155 @@ type: reference stage: Manage group: Optimize -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Value stream analytics for groups **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/196455) in GitLab 12.9 for groups. -Value stream analytics measures the time spent to go from an -[idea to production](https://about.gitlab.com/blog/2016/08/05/continuous-integration-delivery-and-deployment-with-gitlab/#from-idea-to-production-with-gitlab) -(also known as cycle time) for each of your projects or groups. Value stream analytics displays the median time -spent in each stage defined in the process. +Value stream analytics provides metrics about each stage of your software development process. -Value stream analytics can help you quickly determine the velocity of a given -group. It points to bottlenecks in the development process, enabling management -to uncover, triage, and identify the root cause of slowdowns in the software development life cycle. +Use value stream analytics to identify: -For information on how to contribute to the development of value stream analytics, see our [contributor documentation](../../../development/value_stream_analytics.md). +- The amount of time it takes to go from an idea to production. +- The velocity of a given project. +- Bottlenecks in the development process. +- Factors that cause your software development lifecycle to slow down. -To view value stream analytics for groups: +Value stream analytics is also available for [projects](../../analytics/value_stream_analytics.md). + +## View value stream analytics + +> - Date range filter [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13216) in GitLab 12.4 +> - Filtering [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13216) in GitLab 13.3 +> - Horizontal stage path [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12196) in 13.0 and [feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/323982) in 13.12 + +You must have at least the Reporter role to view value stream analytics for groups. + +To view value stream analytics for your group: 1. On the top bar, select **Menu > Groups** and find your group. 1. On the left sidebar, select **Analytics > Value stream**. +1. To view metrics for each stage, above the **Filter results** text box, select a stage. +1. Optional. Filter the results: + 1. Select the **Filter results** text box. + 1. Select a parameter. + 1. Select a value or enter text to refine the results. + 1. To adjust the date range: + - In the **From** field, select a start date. + - In the **To** field, select an end date. The charts and list show workflow items created + during the date range. +1. Optional. Sort results by ascending or descending: + - To sort by most recent or oldest workflow item, select the **Merge requests** or **Issues** + header. The header name differs based on the stage you select. + - To sort by most or least amount of time spent in each stage, select the **Time** header. + +A badge next to the workflow items table header shows the number of workflow items that +completed during the selected stage. + +The table shows a list of related workflow items for the selected stage. Based on the stage you select, this can be: + +- CI/CD jobs +- Issues +- Merge requests +- Pipelines -Value stream analytics at the group level includes data for the selected group and its subgroups. +## View metrics for each development stage -NOTE: -[Value stream analytics for projects](../../analytics/value_stream_analytics.md) is also available. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/210315) in GitLab 13.0. +> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/323982) in GitLab 13.12. + +Value stream analytics shows the median time spent by issues or merge requests in each development stage. + +To view the median time spent in each stage by a group: + +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Analytics > Value stream**. +1. Optional. Filter the results: + 1. Select the **Filter results** text box. + 1. Select a parameter. + 1. Select a value or enter text to refine the results. + 1. To adjust the date range: + - In the **From** field, select a start date. + - In the **To** field, select an end date. +1. To view the metrics for each stage, above the **Filter results** text box, hover over a stage. + +## View the lead time and cycle time for issues + +Value stream analytics shows the lead time and cycle time for issues in your groups: + +- Lead time: Median time from when the issue was created to when it was closed. +- Cycle time: Median time from first commit to issue closed. Commits are associated with issues when users [cross-link them in the commit message](../../project/issues/crosslinking_issues.md#from-commit-messages). + +To view the lead time and cycle time for issues: + +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Analytics > Value stream**. +1. Optional. Filter the results: + 1. Select the **Filter results** text box. + 1. Select a parameter. + 1. Select a value or enter text to refine the results. + 1. To adjust the date range: + - In the **From** field, select a start date. + - In the **To** field, select an end date. + +The **Lead Time** and **Cycle Time** metrics display below the **Filter results** text box. + +## View lead time for changes for merge requests **(ULTIMATE)** -## Default stages +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340150) in GitLab 14.5. -The stages tracked by value stream analytics by default represent the [GitLab flow](../../../topics/gitlab_flow.md). -These stages can be customized in value stream analytics for groups. +Lead time for changes is the median duration between when a merge request is merged and when it's deployed to production. -- **Issue** (Tracker) - - Time to schedule an issue (by milestone or by adding it to an issue board) -- **Plan** (Board) - - Time to first commit -- **Code** (IDE) - - Time to create a merge request -- **Test** (CI) - - Time it takes GitLab CI/CD to test your code -- **Review** (Merge Request/MR) - - Time spent on code review -- **Staging** (Continuous Deployment) - - Time between merging and deploying to production +To view the lead time for changes for merge requests in your group: -## Filter the analytics data +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Analytics > Value stream**. +1. Optional. Filter the results: + 1. Select the **Filter results** text box. + 1. Select a parameter. + 1. Select a value or enter text to refine the results. + 1. To adjust the date range: + - In the **From** field, select a start date. + - In the **To** field, select an end date. -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13216) in GitLab 13.3 +The **Lead Time for Changes** metrics display below the **Filter results** text box. -GitLab provides the ability to filter analytics based on the following parameters: +## View number of successful deployments **(PREMIUM)** -- Milestones (Group level) -- Labels (Group level) -- Author -- Assignees +> DORA API-based deployment metrics for value stream analytics for groups were [moved](https://gitlab.com/gitlab-org/gitlab/-/issues/337256) from GitLab Ultimate to GitLab Premium in 14.3. -To filter results: +To view deployment metrics, you must have a +[production environment configured](../../../ci/environments/index.md#deployment-tier-of-environments). -1. Select a group. -1. Click on the filter bar. -1. Select a parameter to filter by. -1. Select a value from the autocompleted results, or type to refine the results. +Value stream analytics shows the following deployment metrics for your group: + +- Deploys: The number of successful deployments in the date range. +- Deployment Frequency: The average number of successful deployments per day in the date range. - +To view deployment metrics for your group: -### Date ranges +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Analytics > Value stream**. +1. Optional. Filter the results: + 1. Select the **Filter results** text box. + 1. Select a parameter. + 1. Select a value or enter text to refine the results. + 1. To adjust the date range: + - In the **From** field, select a start date. + - In the **To** field, select an end date. -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13216) in GitLab 12.4. +The **Deploys** and **Deployment Frequency** metrics display below the **Filter results** text box. -GitLab provides the ability to filter analytics based on a date range. -Data is shown for workflow items created during the selected date range. To filter results: +Deployment metrics are calculated based on data from the +[DORA API](../../../api/dora/metrics.md#devops-research-and-assessment-dora-key-metrics-api). -1. Select a group. -1. Optionally select a project. -1. Select a date range using the available date pickers. +NOTE: +In GitLab 13.9 and later, metrics are calculated based on when the deployment was finished. +In GitLab 13.8 and earlier, metrics are calculated based on when the deployment was created. -### Upcoming date filter change +## Upcoming date filter change In the [epics](https://gitlab.com/groups/gitlab-org/-/epics/6046), we plan to alter the date filter behavior to filter the end event time of the currently selected stage. @@ -92,82 +163,72 @@ If you were to look at the metrics for the last three months, this issue would n the stage metrics. With the new date filter, this item would be included. DISCLAIMER: -This page contains information related to upcoming products, features, and functionality. +This section contains information related to upcoming products, features, and functionality. It is important to note that the information presented is for informational purposes only. Please do not rely on this information for purchasing or planning purposes. As with all projects, the items mentioned on this page are subject to change or delay. The development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc. -## How metrics are measured - -> DORA API-based deployment metrics for value stream analytics for groups were [moved](https://gitlab.com/gitlab-org/gitlab/-/issues/337256) from GitLab Ultimate to GitLab Premium in 14.3. - -The "Time" metrics near the top of the page are measured as follows: +## How value stream analytics measures stages -- **Lead time**: median time from issue created to issue closed. -- **Cycle time**: median time from first commit to issue closed. (You can associate a commit with an - issue by [crosslinking in the commit message](../../project/issues/crosslinking_issues.md#from-commit-messages).) -- **Lead Time for Changes**: median time between when a merge request is merged and deployed to a -production environment for all merge requests deployed in the given time period. -[Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340150) in GitLab 14.5. - -- **Lead Time for Changes**: median duration between merge request merge and deployment to a production environment for all MRs deployed in the given time period. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/340150) in GitLab 14.5. - -The "Recent Activity" metrics near the top of the page are measured as follows: - -- **New Issues:** the number of issues created in the date range. -- **Deploys:** the number of deployments to production in the date range. -- **Deployment Frequency:** the average number of deployments to production - per day in the date range. - -To see deployment metrics, you must have a [production environment configured](../../../ci/environments/index.md#deployment-tier-of-environments). +Value stream analytics measures each stage from its start event to its end event. -NOTE: -In GitLab 13.9 and later, deployment metrics are calculated based on when the deployment was finished. -In GitLab 13.8 and earlier, deployment metrics are calculated based on when the deployment was created. +For example, a stage might start when a user adds a label to an issue, and ends when they add another label. +Items aren't included in the stage time calculation if they have not reached the end event. -You can learn more about these metrics in our [analytics definitions](../../analytics/index.md). +Each stage of value stream analytics is further described in the table below. - +| Stage | Measurement method | +| ------- | -------------------- | +| Issue | The median time between creating an issue and taking action to solve it, by either labeling it or adding it to a milestone, whichever comes first. The label is tracked only if it already has an [issue board list](../../project/issue_board.md) created for it. | +| Plan | The median time between the action you took for the previous stage, and pushing the first commit to the branch. The first commit on the branch triggers the separation between **Plan** and **Code**. At least one of the commits in the branch must contain the related issue number (for example, `#42`). If none of the commits in the branch mention the related issue number, it is not considered in the measurement time of the stage. | +| Code | The median time between pushing a first commit (previous stage) and creating a merge request (MR) related to that commit. The key to keep the process tracked is to include the [issue closing pattern](../../project/issues/managing_issues.md#default-closing-pattern) in the description of the merge request. For example, `Closes #xxx`, where `xxx` is the number of the issue related to this merge request. If the closing pattern is not present, then the calculation uses the creation time of the first commit in the merge request as the start time. | +| Test | The median time to run the entire pipeline for that project. It's related to the time GitLab CI/CD takes to run every job for the commits pushed to that merge request. It is basically the start->finish time for all pipelines. | +| Review | The median time taken to review a merge request that has a closing issue pattern, between its creation and until it's merged. | +| Staging | The median time between merging a merge request that has a closing issue pattern until the very first deployment to a [production environment](#how-value-stream-analytics-identifies-the-production-environment). If there isn't a production environment, this is not tracked. | -## How the stages are measured +## Example workflow -Value stream analytics measures each stage from its start event to its end event. -For example, a stage might start when one label is added to an issue, and end when another label is added. -Value stream analytics excludes work in progress, meaning it ignores any items that have not reached the end event. +This example shows a workflow through all seven stages in one day. -Each stage of value stream analytics is further described in the table below. +If a stage does not include a start and a stop time, its data is not included in the median time. +In this example, milestones have been created and CI/CD for testing and setting environments is configured. -| **Stage** | **Description** | -| --------- | --------------- | -| Issue | Measures the median time between creating an issue and taking action to solve it, by either labeling it or adding it to a milestone, whatever comes first. The label is tracked only if it already has an [issue board list](../../project/issue_board.md) created for it. | -| Plan | Measures the median time between the action you took for the previous stage, and pushing the first commit to the branch. The very first commit of the branch is the one that triggers the separation between **Plan** and **Code**, and at least one of the commits in the branch needs to contain the related issue number (for example, `#42`). If none of the commits in the branch mention the related issue number, it is not considered to the measurement time of the stage. | -| Code | Measures the median time between pushing a first commit (previous stage) and creating a merge request (MR) related to that commit. The key to keep the process tracked is to include the [issue closing pattern](../../project/issues/managing_issues.md#closing-issues-automatically) to the description of the merge request (for example, `Closes #xxx`, where `xxx` is the number of the issue related to this merge request). If the closing pattern is not present, then the calculation takes the creation time of the first commit in the merge request as the start time. | -| Test | Measures the median time to run the entire pipeline for that project. It's related to the time GitLab CI/CD takes to run every job for the commits pushed to that merge request defined in the previous stage. It is basically the start->finish time for all pipelines. | -| Review | Measures the median time taken to review the merge request that has a closing issue pattern, between its creation and until it's merged. | -| Staging | Measures the median time between merging the merge request with a closing issue pattern until the very first deployment to a [production environment](#how-the-production-environment-is-identified). If there isn't a production environment, this is not tracked. | +- 09:00: Create issue. **Issue** stage starts. +- 11:00: Add issue to a milestone, start work on the issue, and create a branch locally. + **Issue** stage stops and **Plan** stage starts. +- 12:00: Make the first commit. +- 12:30: Make the second commit to the branch that mentions the issue number. + **Plan** stage stops and **Code** stage starts. +- 14:00: Push branch and create a merge request that contains the + [issue closing pattern](../../project/issues/managing_issues.md#closing-issues-automatically). + **Code** stage stops and **Test** and **Review** stages start. +- GitLab CI/CD takes 5 minutes to run scripts defined in [`.gitlab-ci.yml`](../../../ci/yaml/index.md). +- 19:00: Merge the merge request. **Review** stage stops and **Staging** stage starts. +- 19:30: Deployment to the `production` environment finishes. **Staging** stops. -How this works, behind the scenes: +Value stream analytics records the following times for each stage: -1. Issues and merge requests are grouped together in pairs, such that for each - `<issue, merge request>` pair, the merge request has the [issue closing pattern](../../project/issues/managing_issues.md#closing-issues-automatically) - for the corresponding issue. All other issues and merge requests are **not** - considered. -1. Then the `<issue, merge request>` pairs are filtered out by last XX days (specified - by the UI - default is 90 days). So it prohibits these pairs from being considered. -1. For the remaining `<issue, merge request>` pairs, we check the information that - we need for the stages, like issue creation date, merge request merge time, - and so on. +- **Issue**: 09:00 to 11:00: 2 hrs +- **Plan**: 11:00 to 12:00: 1 hr +- **Code**: 12:00 to 14:00: 2 hrs +- **Test**: 5 minutes +- **Review**: 14:00 to 19:00: 5 hrs +- **Staging**: 19:00 to 19:30: 30 minutes -To sum up, anything that doesn't follow [GitLab flow](../../../topics/gitlab_flow.md) is not tracked and the -value stream analytics dashboard does not present any data for: +There are some additional considerations for this example: -- Merge requests that do not close an issue. -- Issues not labeled with a label present in the issue board or for issues not assigned a milestone. -- Staging stage, if the project has no [production environment](#how-the-production-environment-is-identified). +- This example demonstrates that it doesn't matter if your first + commit doesn't mention the issue number, you can do this later in any commit + on the branch you are working on. +- The **Test** stage is used in the calculation for the overall time of + the cycle. It is included in the **Review** process, as every MR should be + tested. +- This example illustrates only **one cycle** of the seven stages. The value stream analytics dashboard + shows the median time for multiple cycles. -## How the production environment is identified +## How value stream analytics identifies the production environment Value stream analytics identifies production environments by looking for project [environments](../../../ci/yaml/index.md#environment) with a name matching any of these patterns: @@ -179,149 +240,22 @@ These patterns are not case-sensitive. You can change the name of a project environment in your GitLab CI/CD configuration. -## Example workflow - -Below is an example workflow of a single cycle that happens in a -single day through all noted stages. Note that if a stage does not include a start -and a stop time, its data is not included in the median time. It is assumed that -milestones are created and a CI for testing and setting environments is configured. -a start and a stop mark, it is not measured and hence not calculated in the median -time. It is assumed that milestones are created and CI for testing and setting -environments is configured. - -1. Issue is created at 09:00 (start of **Issue** stage). -1. Issue is added to a milestone at 11:00 (stop of **Issue** stage / start of - **Plan** stage). -1. Start working on the issue, create a branch locally and make one commit at - 12:00. -1. Make a second commit to the branch which mentions the issue number at 12.30 - (stop of **Plan** stage / start of **Code** stage). -1. Push branch and create a merge request that contains the - [issue closing pattern](../../project/issues/managing_issues.md#closing-issues-automatically) - in its description at 14:00 (stop of **Code** stage / start of **Test** and - **Review** stages). -1. The CI starts running your scripts defined in [`.gitlab-ci.yml`](../../../ci/yaml/index.md) and - takes 5min (stop of **Test** stage). -1. Review merge request, ensure that everything is OK and merge the merge - request at 19:00. (stop of **Review** stage / start of **Staging** stage). -1. Now that the merge request is merged, a deployment to the `production` - environment starts and finishes at 19:30 (stop of **Staging** stage). - -From the above example you can conclude the time it took each stage to complete -as long as their total time: - -- **Issue**: 2h (11:00 - 09:00) -- **Plan**: 1h (12:00 - 11:00) -- **Code**: 2h (14:00 - 12:00) -- **Test**: 5min -- **Review**: 5h (19:00 - 14:00) -- **Staging**: 30min (19:30 - 19:00) - -A few notes: - -- In the above example we demonstrated that it doesn't matter if your first - commit doesn't mention the issue number, you can do this later in any commit - of the branch you are working on. -- You can see that the **Test** stage is not calculated to the overall time of - the cycle, because it is included in the **Review** process (every MR should be - tested). -- The example above was just **one cycle** of the seven stages. Add multiple - cycles, calculate their median time and the result is what the dashboard of - value stream analytics is showing. - ## Custom value streams > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12196) in GitLab 12.9. -The default stages are designed to work straight out of the box, but they might not be suitable for -all teams. Different teams use different approaches to building software, so some teams might want -to customize their value stream analytics. - -GitLab allows users to create multiple value streams, hide default stages and create custom stages -that align better to their development workflow. - -### Stage path - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/210315) in GitLab 13.0. -> - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/323982) in GitLab 13.12. - - - -Stages are visually depicted as a horizontal process flow. Selecting a stage updates the content -below the value stream. - -The stage time is displayed next to the name of each stage, in the following format: - -| Symbol | Description | -|--------|-------------| -| `m` | Minutes | -| `h` | Hours | -| `d` | Days | -| `w` | Weeks | -| `M` | Months | - -Hovering over a stage item displays a popover with the following information: - -- Start event description for the given stage -- End event description -- Median time items took to complete the stage -- Number of items that completed the stage - -### Stream overview - -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/321438) in GitLab 13.11. - - - -The stream overview provides access to key metrics and charts summarizing all the stages in the value stream -based on selected filters. - -Shown metrics and charts includes: - -- [Lead time](#how-metrics-are-measured) -- [Cycle time](#how-metrics-are-measured) -- [Days to completion chart](#days-to-completion-chart) -- [Tasks by type chart](#type-of-work---tasks-by-type-chart) - -### Stage table - -> Sorting the stage table [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/301082) in GitLab 13.12. +Use custom value streams to create custom stages that align with your own development processes, +and hide default stages. The dashboard depicts stages as a horizontal process +flow. - - -The stage table shows a list of related workflow items for the selected stage. This can include: - -- CI/CD jobs -- Issues -- Merge requests -- Pipelines - -A little badge next to the workflow items table header shows the number of workflow items that -completed the selected stage. - -The stage table also includes the **Time** column, which shows how long it takes each item to pass -through the selected value stream stage. - -The stage table is not displayed on the stream [Overview](#stream-overview). -The workflow item column (first column) is ordered by end event. - -To sort the stage table by a table column, select the table header. -You can sort in ascending or descending order. To find items that spent the most time in a stage, -potentially causing bottlenecks in your value stream, sort the table by the **Time** column. -From there, select individual items to drill in and investigate how delays are happening. -To see which items most recently exited the stage, sort by the work item column on the left. - -The table displays 20 items per page. If there are more than 20 items, you can use the -**Prev** and **Next** buttons to navigate through the pages. - -### Creating a value stream +### Create a value stream > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/221202) in GitLab 13.3 A default value stream is readily available for each group. You can create additional value streams based on the different areas of work that you would like to measure. -Once created, a new value stream includes the [seven stages](#default-stages) that follow +Once created, a new value stream includes the stages that follow [GitLab workflow](../../../topics/gitlab_flow.md) best practices. You can customize this flow by adding, hiding or re-ordering stages. @@ -331,20 +265,17 @@ To create a value stream: 1. On the left sidebar, select **Analytics > Value Stream**. 1. In the top right, select the dropdown list and then **Create new Value Stream**. 1. Enter a name for the new Value Stream. - - You can [customize the stages](#creating-a-value-stream-with-stages). + - You can [customize the stages](#create-a-value-stream-with-stages). 1. Select **Create Value Stream**.  -#### Creating a value stream with stages +### Create a value stream with stages > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/50229) in GitLab 13.7. > - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/55572) in GitLab 13.10. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/294190) in GitLab 13.11. -WARNING: -This feature might not be available to you. Check the **version history** note above for details. - You can create value streams with stages, starting with a default or a blank template. You can add stages as desired. @@ -364,7 +295,7 @@ To create a value stream with stages:  1. Select **Create Value Stream**. -#### Label-based stages +### Label-based stages The pre-defined start and end events can cover many use cases involving both issues and merge requests. @@ -379,7 +310,7 @@ In this example, we'd like to measure times for deployment from a staging enviro  -### Editing a value stream +### Edit a value stream > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/267537) in GitLab 13.10. @@ -388,7 +319,7 @@ After you create a value stream, you can customize it to suit your purposes. To 1. On the top bar, select **Menu > Groups** and find your group. 1. On the left sidebar, select **Analytics > Value Stream**. 1. In the top right, select the dropdown list and then select the relevant value stream. -1. Next to the value stream dropdown, select **Edit**. +1. Next to the value stream dropdown list, select **Edit**. The edit form is populated with the value stream details. 1. Optional: - Rename the value stream. @@ -399,7 +330,7 @@ After you create a value stream, you can customize it to suit your purposes. To 1. Optional. To undo any modifications, select **Restore value stream defaults**. 1. Select **Save Value Stream**. -### Deleting a value stream +### Delete a value stream > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/221205) in GitLab 13.4. @@ -413,19 +344,27 @@ To delete a custom value stream:  -## Days to completion chart +## View number of days for a cycle to complete > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/21631) in GitLab 12.6. > - Chart median line [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/235455) in GitLab 13.4. > - Totals [replaced](https://gitlab.com/gitlab-org/gitlab/-/issues/262070) with averages in GitLab 13.12. -This chart visually depicts the average number of days it takes for cycles to be completed. - -This chart uses the global page filters for displaying data based on the selected -group, projects, and time frame. In addition, specific stages can be selected -from the chart itself. +The **Total time chart** shows the average number of days it takes for development cycles to complete. +The chart shows data for the last 500 workflow items. -The chart data is limited to the last 500 items. +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Analytics > Value stream**. +1. Above the **Filter results** box, select a stage: + - To view a summary of the cycle time for all stages, select **Overview**. + - To view the cycle time for specific stage, select a stage. +1. Optional. Filter the results: + 1. Select the **Filter results** text box. + 1. Select a parameter. + 1. Select a value or enter text to refine the results. + 1. To adjust the date range: + - In the **From** field, select a start date. + - In the **To** field, select an end date. ## Type of work - Tasks by type chart @@ -439,17 +378,3 @@ toggled to show data for merge requests and further refined for specific group-l By default the top group-level labels (max. 10) are pre-selected, with the ability to select up to a total of 15 labels. - -## Permissions - -To access value stream analytics for groups, users must have at least the Reporter role. - -You can [read more about permissions](../../permissions.md) in general. - -## More resources - -Learn more about value stream analytics in the following resources: - -- [Value stream analytics feature page](https://about.gitlab.com/stages-devops-lifecycle/value-stream-analytics/). -- [Value stream analytics feature preview](https://about.gitlab.com/blog/2016/09/16/feature-preview-introducing-cycle-analytics/). -- [Value stream analytics feature highlight](https://about.gitlab.com/blog/2016/09/21/cycle-analytics-feature-highlight/). diff --git a/doc/user/index.md b/doc/user/index.md index f4a7cd6a5d9..4ce46c5b46f 100644 --- a/doc/user/index.md +++ b/doc/user/index.md @@ -47,6 +47,8 @@ GitLab is a Git-based platform that integrates a great number of essential tools - Reviewing code in [merge requests](project/merge_requests/index.md) with live-preview changes per branch with [Review Apps](../ci/review_apps/index.md). - Building, testing, and deploying with built-in [Continuous Integration](../ci/index.md). +- View the current health and status of each CI environment running on Kubernetes with [deploy boards](project/deploy_boards.md). +- Leverage continuous delivery method with [Canary Deployments](project/canary_deployments.md). - Deploying personal and professional static websites with [GitLab Pages](project/pages/index.md). - Integrating with Docker by using [GitLab Container Registry](packages/container_registry/index.md). - Tracking the development lifecycle by using [GitLab Value Stream Analytics](analytics/value_stream_analytics.md). @@ -66,8 +68,6 @@ With GitLab Enterprise Edition, you can also: - [Mirror a repository](project/repository/mirror/index.md) from elsewhere on your local server. - View your entire CI/CD pipeline involving more than one project with [Multiple-Project Pipelines](../ci/pipelines/multi_project_pipelines.md). - [Lock files](project/file_lock.md) to prevent conflicts. -- View the current health and status of each CI environment running on Kubernetes with [deploy boards](project/deploy_boards.md). -- Leverage continuous delivery method with [Canary Deployments](project/canary_deployments.md). - Scan your code for vulnerabilities and [display them in merge requests](application_security/sast/index.md). You can also [integrate](project/integrations/overview.md) GitLab with numerous third-party applications, such as Mattermost, Microsoft Teams, Trello, Slack, Bamboo CI, Jira, and a lot more. @@ -76,12 +76,15 @@ You can also [integrate](project/integrations/overview.md) GitLab with numerous There are several types of users in GitLab: -- Regular users and GitLab.com users. <!-- Note: further description TBA --> -- [Groups](group/index.md) of users. -- GitLab [administrator area](admin_area/index.md) user. +- Regular users. +- [Internal users](../development/internal_users.md) often referred to as bot or system users. +- [Auditor](permissions.md#auditor-users) with read access to self-managed instances. - [GitLab Administrator](../administration/index.md) with full access to - self-managed instances' features and settings. -- [Internal users](../development/internal_users.md). + self-managed instances including settings and the [Admin Area](admin_area/index.md). + +Each user can be a member in a [group](group/index.md). + +See the [permissions page](permissions.md) for details on how each user type is used. ## User activity @@ -159,8 +162,7 @@ it all at once, from one single project. ## Account -There is a lot you can customize and configure -to enjoy the best of GitLab. +There is a lot you can customize and configure to enjoy the best of GitLab. - [Settings](profile/index.md): Manage your user settings to change your personal information, personal access tokens, authorized applications, etc. @@ -209,8 +211,7 @@ requests you're assigned to. ## Snippets [Snippets](snippets.md) are code blocks that you want to store in GitLab, from which -you have quick access to. You can also gather feedback on them through -[Discussions](#discussions). +you have quick access to. You can also gather feedback on them through [Discussions](#discussions). ## GitLab CI/CD @@ -228,8 +229,7 @@ pages and accomplish tasks faster. ## Integrations -[Integrate GitLab](../integration/index.md) with your preferred tool, -such as Trello, Jira, etc. +[Integrate GitLab](../integration/index.md) with your preferred tool, such as Trello, Jira, etc. ## Webhooks @@ -251,5 +251,4 @@ See [various statistics](admin_area/analytics/index.md) of your GitLab instance. ## Operations Dashboard -See [Operations Dashboard](operations_dashboard/index.md) for a summary of each -project's operational health. +See [Operations Dashboard](operations_dashboard/index.md) for a summary of each project's operational health. diff --git a/doc/user/infrastructure/clusters/connect/index.md b/doc/user/infrastructure/clusters/connect/index.md index 06d1307984d..004f4409919 100644 --- a/doc/user/infrastructure/clusters/connect/index.md +++ b/doc/user/infrastructure/clusters/connect/index.md @@ -8,10 +8,10 @@ info: To determine the technical writer assigned to the Stage/Group associated w The [certificate-based Kubernetes integration with GitLab](../index.md) was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) -in GitLab 14.5. To connect your clusters, use the [GitLab Agent](../../../clusters/agent/index.md). +in GitLab 14.5. To connect your clusters, use the [GitLab agent](../../../clusters/agent/index.md). <!-- TBA: (We need to resolve https://gitlab.com/gitlab-org/gitlab/-/issues/343660 before adding this line) -If you don't have a cluster yet, create one and connect it to GitLab through the Agent. +If you don't have a cluster yet, create one and connect it to GitLab through the agent. You can also create a new cluster from GitLab using [Infrastructure as Code](../../iac/index.md#create-a-new-cluster-through-iac). --> diff --git a/doc/user/infrastructure/clusters/connect/new_eks_cluster.md b/doc/user/infrastructure/clusters/connect/new_eks_cluster.md new file mode 100644 index 00000000000..87b8f510289 --- /dev/null +++ b/doc/user/infrastructure/clusters/connect/new_eks_cluster.md @@ -0,0 +1,132 @@ +--- +stage: Configure +group: Configure +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Create an Amazon EKS cluster + +You can create a cluster on Amazon Elastic Kubernetes Service (EKS) through +[Infrastructure as Code (IaC)](../../index.md). This process uses the AWS and +Kubernetes Terraform providers to create EKS clusters. You connect the clusters to GitLab +by using the GitLab agent for Kubernetes. + +**Prerequisites:** + +- An Amazon Web Services (AWS) account, with a set of configured + [security credentials](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-prereqs.html). +- [A runner](https://docs.gitlab.com/runner/install/) you can use to run the GitLab CI/CD pipeline. + +**Steps:** + +1. [Import the example project](#import-the-example-project). +1. [Register the agent for Kubernetes](#register-the-agent). +1. [Configure your project](#configure-your-project). +1. [Provision your cluster](#provision-your-cluster). + +## Import the example project + +To create a cluster from GitLab using Infrastructure as Code, you must +create a project to manage the cluster from. In this tutorial, you start with +a sample project and modify it according to your needs. + +Start by [importing the example project by URL](../../../project/import/repo_by_url.md). + +To import the project: + +1. On the top bar, select **Menu > Create new project**. +1. Select **Import project**. +1. Select **Repo by URL**. +1. For the **Git repository URL**, enter `https://gitlab.com/gitlab-org/configure/examples/gitlab-terraform-eks.git`. +1. Complete the fields and select **Create project**. + +This project provides you with: + +- An Amazon [Virtual Private Cloud (VPC)](https://gitlab.com/gitlab-org/configure/examples/gitlab-terraform-eks/-/blob/main/vpc.tf). +- An Amazon [Elastic Kubernetes Service (EKS)](https://gitlab.com/gitlab-org/configure/examples/gitlab-terraform-eks/-/blob/main/eks.tf) cluster. +- The [GitLab agent for Kubernetes](https://gitlab.com/gitlab-org/configure/examples/gitlab-terraform-eks/-/blob/main/agent.tf) installed in the cluster. + +## Register the agent + +To create a GitLab agent for Kubernetes: + +1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. +1. Select **Actions**. +1. From the **Select an agent** dropdown list, select `eks-agent` and select **Register an agent**. +1. GitLab generates a registration token for the agent. Securely store this secret token, as you will need it later. +1. GitLab provides an address for the agent server (KAS), which you will also need later. + +## Configure your project + +Use CI/CD environment variables to configure your project. + +**Required configuration:** + +1. On the left sidebar, select **Settings > CI/CD**. +1. Expand **Variables**. +1. Set the variable `AWS_ACCESS_KEY_ID` to your AWS access key ID. +1. Set the variable `AWS_SECRET_ACCESS_KEY` to your AWS secret access key. +1. Set the variable `TF_VAR_agent_token` to the agent token displayed in the previous task. +1. Set the variable `TF_VAR_kas_address` to the agent server address displayed in the previous task. + +**Optional configuration:** + +The file [`variables.tf`](https://gitlab.com/gitlab-org/configure/examples/gitlab-terraform-eks/-/blob/main/variables.tf) +contains other variables that you can override according to your needs: + +- `TF_VAR_region`: Set your cluster's region. +- `TF_VAR_cluster_name`: Set your cluster's name. +- `TF_VAR_cluster_version`: Set the version of Kubernetes. +- `TF_VAR_instance_type`: Set the instance type for the Kubernetes nodes. +- `TF_VAR_instance_count`: Set the number of Kubernetes nodes. +- `TF_VAR_agent_version`: Set the version of the GitLab agent. +- `TF_VAR_agent_namespace`: Set the Kubernetes namespace for the GitLab agent. + +View the [AWS Terraform provider](https://registry.terraform.io/providers/hashicorp/aws/latest/docs) and the [Kubernetes Terraform provider](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs) documentation for further resource options. + +## Provision your cluster + +After configuring your project, manually trigger the provisioning of your cluster. In GitLab: + +1. On the left sidebar, go to **CI/CD > Pipelines**. +1. Next to **Play** (**{play}**), select the dropdown icon (**{angle-down}**). +1. Select **Deploy** to manually trigger the deployment job. + +When the pipeline finishes successfully, you can view the new cluster: + +- In AWS: From the [EKS console](https://console.aws.amazon.com/eks/home), select **Amazon EKS > Clusters**. +- In GitLab: On the left sidebar, select **Infrastructure > Kubernetes clusters**. + +## Use your cluster + +After you provision the cluster, it is connected to GitLab and is ready for deployments. To check the connection: + +1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. +1. In the list, view the **Connection status** column. + +For more information about the capabilities of the connection, see [the GitLab agent for Kubernetes documentation](../index.md). + +## Remove the cluster + +A cleanup job is not included in your pipeline by default. To remove all created resources, you +must modify your GitLab CI/CD template before running the cleanup job. + +To remove all resources: + +1. Add the following to your `.gitlab-ci.yml` file: + + ```yaml + stages: + - init + - validate + - build + - deploy + - cleanup + + destroy: + extends: .destroy + needs: [] + ``` + +1. On the left sidebar, select **CI/CD > Pipelines** and select the most recent pipeline. +1. For the `destroy` job, select **Play** (**{play}**). diff --git a/doc/user/infrastructure/clusters/connect/new_gke_cluster.md b/doc/user/infrastructure/clusters/connect/new_gke_cluster.md index d1e3bd47b89..1ed8b0ef350 100644 --- a/doc/user/infrastructure/clusters/connect/new_gke_cluster.md +++ b/doc/user/infrastructure/clusters/connect/new_gke_cluster.md @@ -4,65 +4,59 @@ group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# New GKE cluster through IaC (DEPRECATED) - -> [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. - -WARNING: -The process described on this page uses cluster certificates to connect the -new cluster to GitLab, [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. -You can still create a cluster and then connect it to GitLab through the [Agent](../index.md). -[An issue exists](https://gitlab.com/gitlab-org/gitlab/-/issues/343660) -to migrate this functionality to the [Agent](../index.md). +# Create a Google GKE cluster Learn how to create a new cluster on Google Kubernetes Engine (GKE) through -[Infrastructure as Code (IaC)](../../index.md). - -This process combines the GitLab Terraform and Google Terraform providers -with Kubernetes to help you create GKE clusters and deploy them through -GitLab. - -This document describes how to set up a [group-level cluster](../../../group/clusters/index.md) on GKE by importing an example project to get you started. -You can then modify the project files according to your needs. +[Infrastructure as Code (IaC)](../../index.md). This process uses the Google +and Kubernetes Terraform providers create GKE clusters. You connect the clusters to GitLab +by using the GitLab agent for Kubernetes. **Prerequisites:** -- A GitLab group. -- A GitLab user with the Maintainer role in the group. -- A [GitLab personal access token](../../../profile/personal_access_tokens.md) with `api` access, created by a user with at least the Maintainer role in the group. - A [Google Cloud Platform (GCP) service account](https://cloud.google.com/docs/authentication/getting-started). +- [A runner](https://docs.gitlab.com/runner/install/) you can use to run the GitLab CI/CD pipeline. **Steps:** 1. [Import the example project](#import-the-example-project). -1. [Create your GCP and GitLab credentials](#create-your-gcp-and-gitlab-credentials). +1. [Register the agent for Kubernetes](#register-the-agent). +1. [Create your GCP credentials](#create-your-gcp-credentials). 1. [Configure your project](#configure-your-project). -1. [Deploy your cluster](#deploy-your-cluster). +1. [Provision your cluster](#provision-your-cluster). ## Import the example project -To create a new group-level cluster from GitLab using Infrastructure as Code, it is necessary -to create a project to manage the cluster from. In this tutorial, we import a pre-configured -sample project to help you get started. +To create a cluster from GitLab using Infrastructure as Code, you must +create a project to manage the cluster from. In this tutorial, you start with +a sample project and modify it according to your needs. -Start by [importing the example project by URL](../../../project/import/repo_by_url.md). Use `https://gitlab.com/gitlab-org/configure/examples/gitlab-terraform-gke.git` as URL. +Start by [importing the example project by URL](../../../project/import/repo_by_url.md). -This project provides you with the following resources: +To import the project: + +1. On the top bar, select **Menu > Create new project**. +1. Select **Import project**. +1. Select **Repo by URL**. +1. For the **Git repository URL**, enter `https://gitlab.com/gitlab-org/configure/examples/gitlab-terraform-gke.git`. +1. Complete the fields and select **Create project**. + +This project provides you with: - A [cluster on Google Cloud Platform (GCP)](https://gitlab.com/gitlab-org/configure/examples/gitlab-terraform-gke/-/blob/master/gke.tf) with defaults for name, location, node count, and Kubernetes version. -- A [`gitlab-admin` K8s service account](https://gitlab.com/gitlab-org/configure/examples/gitlab-terraform-gke/-/blob/master/gitlab-admin.tf) with `cluster-admin` privileges. -- The new group-level cluster connected to GitLab. -- Pre-configures Terraform files: - - ```plaintext - ├── backend.tf # State file Location Configuration - ├── gke.tf # Google GKE Configuration - ├── gitlab-admin.tf # Adding kubernetes service account - └── group_cluster.tf # Registering kubernetes cluster to GitLab `apps` Group - ``` +- The [GitLab agent for Kubernetes](https://gitlab.com/gitlab-org/configure/examples/gitlab-terraform-gke/-/blob/master/agent.tf) installed in the cluster. -## Create your GCP and GitLab credentials +## Register the agent + +To create a GitLab agent for Kubernetes: + +1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. +1. Select **Actions**. +1. From the **Select an agent** dropdown list, select `gke-agent` and select **Register an agent**. +1. GitLab generates a registration token for the agent. Securely store this secret token, as you will need it later. +1. GitLab provides an address for the agent server (KAS), which you will also need later. + +## Create your GCP credentials To set up your project to communicate to GCP and the GitLab API: @@ -85,18 +79,14 @@ The Admin role creates a service account in the `kube-system` namespace. ## Configure your project -**Required configuration:** - -Use CI/CD environment variables to configure your project as detailed below. +Use CI/CD environment variables to configure your project. **Required configuration:** 1. On the left sidebar, select **Settings > CI/CD**. 1. Expand **Variables**. -1. Set the variable `TF_VAR_gitlab_token` to the GitLab personal access token you just created. 1. Set the variable `BASE64_GOOGLE_CREDENTIALS` to the `base64` encoded JSON file you just created. 1. Set the variable `TF_VAR_gcp_project` to your GCP's `project` name. -1. Set the variable `TF_VAR_gitlab_group` to the name of the group you want to connect your cluster to. If your group's URL is `https://gitlab.example.com/my-example-group`, `my-example-group` is your group's name. **Optional configuration:** @@ -105,22 +95,57 @@ contains other variables that you can override according to your needs: - `TF_VAR_gcp_region`: Set your cluster's region. - `TF_VAR_cluster_name`: Set your cluster's name. -- `TF_VAR_machine_type`: Set the machine type for the Kubernetes nodes. - `TF_VAR_cluster_description`: Set a description for the cluster. We recommend setting this to `$CI_PROJECT_URL` to create a reference to your GitLab project on your GCP cluster detail page. This way you know which project was responsible for provisioning the cluster you see on the GCP dashboard. -- `TF_VAR_base_domain`: Set to the base domain to provision resources under. -- `TF_VAR_environment_scope`: Set to the environment scope for your cluster. +- `TF_VAR_machine_type`: Set the machine type for the Kubernetes nodes. +- `TF_VAR_node_count`: Set the number of Kubernetes nodes. +- `TF_VAR_agent_version`: Set the version of the GitLab agent. +- `TF_VAR_agent_namespace`: Set the Kubernetes namespace for the GitLab agent. -Refer to the [GitLab Terraform provider](https://registry.terraform.io/providers/gitlabhq/gitlab/latest/docs) and the [Google Terraform provider](https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/provider_reference) documentation for further resource options. +Refer to the [Google Terraform provider](https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/provider_reference) and the [Kubernetes Terraform provider](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs) documentation for further resource options. -## Deploy your cluster +## Provision your cluster -After configuring your project, manually trigger the deployment of your cluster. In GitLab: +After configuring your project, manually trigger the provisioning of your cluster. In GitLab: -1. From your project's sidebar, go to **CI/CD > Pipelines**. -1. Select the dropdown icon (**{angle-down}**) next to the play icon (**{play}**). -1. Select **deploy** to manually trigger the deployment job. +1. On the left sidebar, go to **CI/CD > Pipelines**. +1. Next to **Play** (**{play}**), select the dropdown icon (**{angle-down}**). +1. Select **Deploy** to manually trigger the deployment job. When the pipeline finishes successfully, you can see your new cluster: - In GCP: on your [GCP console's Kubernetes list](https://console.cloud.google.com/kubernetes/list). - In GitLab: from your project's sidebar, select **Infrastructure > Kubernetes clusters**. + +## Use your cluster + +After you provision the cluster, it is connected to GitLab and is ready for deployments. To check the connection: + +1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. +1. In the list, view the **Connection status** column. + +For more information about the capabilities of the connection, see [the GitLab agent for Kubernetes documentation](../index.md). + +## Remove the cluster + +A cleanup job is not included in your pipeline by default. To remove all created resources, you +must modify your GitLab CI/CD template before running the cleanup job. + +To remove all resources: + +1. Add the following to your `.gitlab-ci.yml` file: + + ```yaml + stages: + - init + - validate + - build + - deploy + - cleanup + + destroy: + extends: .destroy + needs: [] + ``` + +1. On the left sidebar, select **CI/CD > Pipelines** and select the most recent pipeline. +1. For the `destroy` job, select **Play** (**{play}**). diff --git a/doc/user/infrastructure/clusters/deploy/inventory_object.md b/doc/user/infrastructure/clusters/deploy/inventory_object.md index 47063dcae96..d76ef0b9359 100644 --- a/doc/user/infrastructure/clusters/deploy/inventory_object.md +++ b/doc/user/infrastructure/clusters/deploy/inventory_object.md @@ -4,66 +4,74 @@ group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Inventory object **(PREMIUM)** +# Tracking cluster resources managed by GitLab **(PREMIUM)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/332227) in GitLab 14.0. -An inventory object is a `ConfigMap` object for keeping track of the set of objects applied to a cluster. -When you remove objects from a manifest repository, the Agent uses a corresponding inventory object to -prune (delete) objects from the cluster. +GitLab uses an inventory object to track the resources you deploy to your cluster. +The inventory object is a `ConfigMap` that contains a list of controlled objects. +The managed resources use the `cli-utils.sigs.k8s.io/inventory-id` annotation. -The Agent creates an inventory object for each manifest project specified in the -`gitops.manifest_projects` configuration section. The inventory object has to be stored somewhere in the cluster. -The default behavior is: +## Default location of the inventory object -- The `namespace` used comes from `gitops.manifest_projects[].default_namespace`. If you don't specify this parameter - explicitly, the inventory object is stored in the `default` namespace. -- The `name` is generated from the numeric project ID of the manifest project and the numeric agent ID. +In the agent configuration file, you specify a list of projects. For example: - This way the Agent constructs the name and local where the inventory object is - stored in the cluster. +```yaml +gitops: + manifest_projects: + - id: gitlab-org/cluster-integration/gitlab-agent + default_namespace: my-ns +``` -The Agent cannot locate the existing inventory object if you: +The agent creates an inventory object for every item in the `manifest_projects` list. +The inventory object is stored in the namespace you specify for `default_namespace`. -- Change `gitops.manifest_projects[].default_namespace` parameter. -- Move manifests into another project. +The name and location of the inventory object is based on: -## Inventory object template +- The `default_namespace`. If you don't specify this parameter, + the inventory object is stored in the `default` namespace. +- The `name`, which is the ID of the project with the manifest and the ID of the agent. -The inventory object template is a `ConfigMap` object that allows you to configure the namespace and the name of the inventory -object. Store this template with manifest files in a single group. +WARNING: +The agent cannot locate the existing inventory object if you change +the `default_namespace` parameter or move manifests to another project. -Example inventory object template: +## Change the location of the inventory object -```yaml -apiVersion: v1 -kind: ConfigMap -metadata: - name: unique-name-for-the-inventory - namespace: my-project-namespace - labels: - cli-utils.sigs.k8s.io/inventory-id: unique-name-for-the-inventory -``` +You can configure the namespace and the name of the inventory object. +This action changes the location of the object in the cluster. + +1. Create an inventory object template, which is a `ConfigMap` object. + For example: + + ```yaml + apiVersion: v1 + kind: ConfigMap + metadata: + name: unique-name-for-the-inventory + namespace: my-project-namespace + labels: + cli-utils.sigs.k8s.io/inventory-id: unique-name-for-the-inventory + ``` + +1. Specify a `namespace` and `name`. Ensure that the `name` is unique so it doesn't conflict with other + inventory objects in the same namespace in the future. +1. Ensure the value for `cli-utils.sigs.k8s.io/inventory-id` is unique. This value is used for objects + tracked by this inventory object. Their `config.k8s.io/owning-inventory` annotation is set to this value. + + The value doesn't have to match the `name` but it's convenient to set them to the same value. + +1. Save the file with the manifest files as a single logical group. -- The `namespace` and `name` fields configure where the real inventory object is created. -- The `cli-utils.sigs.k8s.io/inventory-id` label with its corresponding value is set on the inventory object, created - from this template. Make sure that the value is unique (for example, a string of random characters) and doesn't clash - with any existing or future inventory object templates. -- Objects tracked by this inventory object have the `config.k8s.io/owning-inventory` annotation set to the value of - the `cli-utils.sigs.k8s.io/inventory-id` label. -- The label's value doesn't have to match the `name` but it's convenient to have them set to the same value. -- Make sure that the `name` is unique so that it doesn't conflict with another inventory object in the same - namespace in the future. +## `inventory_policy` options -## Using GitOps with pre-existing Kubernetes objects +Sometimes your manifest changes affect resources that aren't tracked by the GitLab inventory object. -The Agent treats manifest files in the manifest repository as the source of truth. When it applies -objects from the files to the cluster, it tracks them in an inventory object. If an object already exists, -The Agent behaves differently based on the `gitops.manifest_projects[].inventory_policy` configuration. -Check the table below with the available options and when to use them. +To change how the agent behaves when it overwrites existing and previously untracked resources, +change the `inventory_policy` value. `inventory_policy` value | Description | ------------------------ | ------------------------------------------------------------------------------------------- | -`must_match` | This is the default policy. A live object must have the `config.k8s.io/owning-inventory` annotation set to the same value as the `cli-utils.sigs.k8s.io/inventory-id` label on the corresponding inventory object to be updated. Object is not updated and an error is reported if the values don't match or the object doesn't have the annotation. | -`adopt_if_no_inventory` | This mode allows to "adopt" an object if it doesn't have the `config.k8s.io/owning-inventory` annotation. Use this mode if you want to start managing existing objects using the GitOps feature. Once all objects have been "adopted", we recommend you to put the setting back into the default `must_match` mode to avoid any unexpected adoptions. | -`adopt_all` | This mode allows to "adopt" an object even if it has the `config.k8s.io/owning-inventory` annotation set to a different value. This mode can be useful if you want to migrate a set of objects from one agent to another one or from some other tool to the Agent. Once all objects have been "adopted", we recommend you to put the setting back into the default `must_match` mode to avoid any unexpected adoptions. | +`must_match` | The default policy. To be updated, a live object must have the `config.k8s.io/owning-inventory` annotation set to the same value as the `cli-utils.sigs.k8s.io/inventory-id` label on the corresponding inventory object. If the values don't match or the object doesn't have the annotation, the object is not updated and an error is reported. | +`adopt_if_no_inventory` | Adopt an object if it doesn't have the `config.k8s.io/owning-inventory` annotation. Use this mode if you want to start managing existing objects by using the GitOps feature. To avoid unexpected adoptions, after all objects have been adopted, put the setting back to the default `must_match` mode. | +`adopt_all` | Adopt an object even if it has the `config.k8s.io/owning-inventory` annotation set to a different value. Use this mode if you want to migrate a set of objects from one agent to another, or from some other tool to the agent. To avoid unexpected adoptions, after all objects have been adopted, put the setting back to the default `must_match` mode. | diff --git a/doc/user/infrastructure/clusters/index.md b/doc/user/infrastructure/clusters/index.md index 144a8cd2d31..cd7c5feb284 100644 --- a/doc/user/infrastructure/clusters/index.md +++ b/doc/user/infrastructure/clusters/index.md @@ -6,7 +6,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Kubernetes clusters **(FREE)** -To connect clusters to GitLab, use the [GitLab Agent](../../clusters/agent/index.md). +To connect clusters to GitLab, use the [GitLab agent](../../clusters/agent/index.md). ## Certificate-based Kubernetes integration (DEPRECATED) @@ -24,7 +24,7 @@ It had the following issues: - Users were constantly reporting issues with features based on this model. For this reason, we started to build features based on a new model, the -[GitLab Agent](../../clusters/agent/index.md). +[GitLab agent](../../clusters/agent/index.md). Maintaining both methods in parallel caused a lot of confusion and significantly increased the complexity to use, develop, maintain, and document them. For this reason, we decided to deprecate them to focus on the @@ -38,7 +38,7 @@ Follow this [epic](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) for updates. You can find technical information about why we moved away from cluster certificates into -the GitLab Agent model on the [Agent's blueprint documentation](../../../architecture/blueprints/gitlab_to_kubernetes_communication/index.md). +the GitLab agent model on the [agent's blueprint documentation](../../../architecture/blueprints/gitlab_to_kubernetes_communication/index.md). ## Deprecated features @@ -67,6 +67,5 @@ The concept of [project-level](../../project/clusters/index.md), [instance-level](../../instance/clusters/index.md) clusters becomes extinct in the new model, although the functionality remains to some extent. -The Agent is always configured in a single GitLab project, but you can use the CI/CD Tunnel to -[authorize other projects and groups to use the same Agent](../../clusters/agent/repository.md#authorize-projects-and-groups-to-use-an-agent). +The agent is always configured in a single GitLab project and you can expose the cluster connection to other projects and groups to [access it from GitLab CI/CD](../../clusters/agent/ci_cd_tunnel.md). By doing so, you are granting these projects and groups access to the same cluster, which is similar to group-level clusters' use case. diff --git a/doc/user/infrastructure/clusters/manage/management_project_applications/runner.md b/doc/user/infrastructure/clusters/manage/management_project_applications/runner.md index 841f2af7863..4faf5f46418 100644 --- a/doc/user/infrastructure/clusters/manage/management_project_applications/runner.md +++ b/doc/user/infrastructure/clusters/manage/management_project_applications/runner.md @@ -4,7 +4,7 @@ group: Runner info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Install GitLab Runner with a cluster management project +# Install GitLab Runner with a cluster management project **(FREE)** > [Introduced](https://gitlab.com/gitlab-org/project-templates/cluster-management/-/merge_requests/5) in GitLab 14.0. diff --git a/doc/user/infrastructure/clusters/migrate_to_gitlab_agent.md b/doc/user/infrastructure/clusters/migrate_to_gitlab_agent.md index 1dd1c760bcc..01530422e4a 100644 --- a/doc/user/infrastructure/clusters/migrate_to_gitlab_agent.md +++ b/doc/user/infrastructure/clusters/migrate_to_gitlab_agent.md @@ -4,85 +4,78 @@ group: Configure info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Migrate to the GitLab Agent for Kubernetes **(FREE)** +# Migrate to the GitLab agent for Kubernetes **(FREE)** -The first integration between GitLab and Kubernetes used cluster certificates -to connect the cluster to GitLab. -This method was [deprecated](https://about.gitlab.com/blog/2021/11/15/deprecating-the-cert-based-kubernetes-integration/) -in GitLab 14.5 in favor of the [GitLab Agent for Kubernetes](../../clusters/agent/index.md). +To connect your Kubernetes cluster with GitLab, you can use: -To make sure your clusters connected to GitLab do not break in the future, -we recommend you migrate to the GitLab Agent as soon as possible by following -the processes described in this document. +- [A GitOps workflow](../../clusters/agent/gitops.md). +- [A GitLab CI/CD workflow](../../clusters/agent/ci_cd_tunnel.md). +- [A certificate-based integration](index.md). -The certificate-based integration was used for some popular GitLab features such as, -GitLab Managed Apps, GitLab-managed clusters, and Auto DevOps. +The certificate-based integration is +[**deprecated**](https://about.gitlab.com/blog/2021/11/15/deprecating-the-cert-based-kubernetes-integration/) +in GitLab 14.5. It is expected to be +[turned off by default in 15.0](../../../update/deprecations.md#certificate-based-integration-with-kubernetes) +and removed in GitLab 15.6. + +If you are using the certificate-based integration, you should move to another workflow as soon as possible. -As a general rule, migrating clusters that rely on GitLab CI/CD can be -achieved using the [CI/CD Tunnel](../../clusters/agent/ci_cd_tunnel.md) -provided by the Agent. +As a general rule, to migrate clusters that rely on GitLab CI/CD, +you can use the [CI/CD workflow](../../clusters/agent/ci_cd_tunnel.md). +This workflow uses an agent to connect to your cluster. The agent: + +- Is not exposed to the internet. +- Does not require full cluster-admin access to GitLab. NOTE: -The GitLab Agent for Kubernetes does not intend to provide feature parity with the -certificate-based cluster integrations. As a result, the Agent doesn't support -all the features available to clusters connected through certificates. +The certificate-based integration was used for popular GitLab features like +GitLab Managed Apps, GitLab-managed clusters, and Auto DevOps. +Some features are currently available only when using certificate-based integration. ## Migrate cluster application deployments ### Migrate from GitLab-managed clusters With GitLab-managed clusters, GitLab creates separate service accounts and namespaces -for every branch and deploys using these resources. +for every branch and deploys by using these resources. -To achieve a similar result with the GitLab Agent, you can use [impersonation](../../clusters/agent/repository.md#use-impersonation-to-restrict-project-and-group-access) +The GitLab agent uses [impersonation](../../clusters/agent/ci_cd_tunnel.md#use-impersonation-to-restrict-project-and-group-access) strategies to deploy to your cluster with restricted account access. To do so: 1. Choose the impersonation strategy that suits your needs. 1. Use Kubernetes RBAC rules to manage impersonated account permissions in Kubernetes. -1. Use the `access_as` attribute in your Agent’s configuration file to define the impersonation. +1. Use the `access_as` attribute in your agent configuration file to define the impersonation. ### Migrate from Auto DevOps -To configure your Auto DevOps project to use the GitLab Agent: +To configure your Auto DevOps project to use the GitLab agent: -1. Follow the steps to [install an agent](../../clusters/agent/install/index.md) on your cluster. -1. Go to the project in which you use Auto DevOps. -1. From the sidebar, select **Settings > CI/CD** and expand **Variables**. +1. Follow the steps to [install an agent](../../clusters/agent/install/index.md) in your cluster. +1. Go to the project where you use Auto DevOps. +1. On the left sidebar, select **Settings > CI/CD** and expand **Variables**. 1. Select **Add new variable**. 1. Add `KUBE_CONTEXT` as the key, `path/to/agent/project:agent-name` as the value, and select the environment scope of your choice. 1. Select **Add variable**. 1. Repeat the process to add another variable, `KUBE_NAMESPACE`, setting the value for the Kubernetes namespace you want your deployments to target, and set the same environment scope from the previous step. -1. From the sidebar, select **Infrastructure > Kubernetes clusters**. +1. On the left sidebar, select **Infrastructure > Kubernetes clusters**. 1. From the certificate-based clusters section, open the cluster that serves the same environment scope. 1. Select the **Details** tab and disable the cluster. -1. To activate the changes, from the project's sidebar, select **CI/CD > Variables > Run pipeline**. +1. To activate the changes, on the left sidebar, select **CI/CD > Variables > Run pipeline**. -### Migrate generic deployments - -When you use Kubernetes contexts to reach the cluster from GitLab, you can use the [CI/CD Tunnel](../../clusters/agent/ci_cd_tunnel.md) -directly. It injects the available contexts into your CI environment automatically: +For an example, [view this project](https://gitlab.com/gitlab-examples/ops/gitops-demo/hello-world-service). -1. Follow the steps to [install an agent](../../clusters/agent/install/index.md) on your cluster. -1. Go to the project in which you use Auto DevOps. -1. From the sidebar, select **Settings > CI/CD** and expand **Variables**. -1. Select **Add new variable**. -1. Add `KUBE_CONTEXT` as the key, `path/to/agent-configuration-project:your-agent-name` as the value, and select the environment scope of your choice. -1. Edit your `.gitlab-ci.yml` file and set the Kubernetes context to the `KUBE_CONTEXT` you defined in the previous step: +### Migrate generic deployments - ```yaml - <your job name>: - script: - - kubectl config use-context $KUBE_CONTEXT - ``` +Follow the process for the [CI/CD workflow](../../clusters/agent/ci_cd_tunnel.md). -## Migrate from GitLab Managed Applications +## Migrate from GitLab Managed applications -Follow the process to [migrate from GitLab Managed Apps to the Cluster Management Project](../../clusters/migrating_from_gma_to_project_template.md). +Follow the process to [migrate from GitLab Managed Apps to the cluster management project](../../clusters/migrating_from_gma_to_project_template.md). -## Migrating a Cluster Management project +## Migrate a cluster management project -See [how to use a cluster management project with the GitLab Agent](../../clusters/management_project_template.md#use-the-agent-with-the-cluster-management-project-template). +See [how to use a cluster management project with the GitLab agent](../../clusters/management_project_template.md). ## Migrate cluster monitoring features -Cluster monitoring features are not supported by the GitLab Agent for Kubernetes yet. +Cluster monitoring features are not yet supported by the GitLab agent for Kubernetes. diff --git a/doc/user/infrastructure/iac/index.md b/doc/user/infrastructure/iac/index.md index 6fef1aa7879..3bc7495d4be 100644 --- a/doc/user/infrastructure/iac/index.md +++ b/doc/user/infrastructure/iac/index.md @@ -60,9 +60,9 @@ This video from January 2021 walks you through all the GitLab Terraform integrat ## GitLab-managed Terraform state -[Terraform remote backends](https://www.terraform.io/docs/language/settings/backends/index.html) +[Terraform remote backends](https://www.terraform.io/language/settings/backends) enable you to store the state file in a remote, shared store. GitLab uses the -[Terraform HTTP backend](https://www.terraform.io/docs/language/settings/backends/http.html) +[Terraform HTTP backend](https://www.terraform.io/language/settings/backends/http) to securely store the state files in local storage (the default) or [the remote store of your choice](../../../administration/terraform_state.md). @@ -105,16 +105,10 @@ owned by GitLab, where everyone can contribute. The [documentation of the provider](https://registry.terraform.io/providers/gitlabhq/gitlab/latest/docs) is available as part of the official Terraform provider documentation. -## Create a new cluster through IaC (DEPRECATED) +## Create a new cluster through IaC -Learn how to [create a new cluster on Google Kubernetes Engine (GKE)](../clusters/connect/new_gke_cluster.md). - -NOTE: -The linked tutorial connects the cluster to GitLab through cluster certificates, -and this method was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) -in GitLab 14.5. You can still create a cluster through IaC and then connect it to GitLab -through the [agent](../../clusters/agent/index.md), the default and fully supported -method to connect clusters to GitLab. +- Learn how to [create a new cluster on Amazon Elastic Kubernetes Service (EKS)](../clusters/connect/new_eks_cluster.md). +- Learn how to [create a new cluster on Google Kubernetes Engine (GKE)](../clusters/connect/new_gke_cluster.md). ## Troubleshooting diff --git a/doc/user/infrastructure/iac/mr_integration.md b/doc/user/infrastructure/iac/mr_integration.md index 8c135f18bc1..bcad5c9279a 100644 --- a/doc/user/infrastructure/iac/mr_integration.md +++ b/doc/user/infrastructure/iac/mr_integration.md @@ -151,8 +151,8 @@ apply: ### Multiple Terraform Plan reports -Starting with GitLab version 13.2, you can display multiple reports on a merge request. -The reports also display the `artifacts: name:`. See example below for a suggested setup. +Starting with GitLab version 13.2, you can display multiple reports on the merge request +page. The reports also display the `artifacts: name:`. See example below for a suggested setup. ```yaml default: diff --git a/doc/user/infrastructure/iac/terraform_state.md b/doc/user/infrastructure/iac/terraform_state.md index 8fd38215b04..39a57b60787 100644 --- a/doc/user/infrastructure/iac/terraform_state.md +++ b/doc/user/infrastructure/iac/terraform_state.md @@ -8,9 +8,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2673) in GitLab 13.0. -[Terraform remote backends](https://www.terraform.io/docs/language/settings/backends/index.html) +[Terraform remote backends](https://www.terraform.io/language/settings/backends) enable you to store the state file in a remote, shared store. GitLab uses the -[Terraform HTTP backend](https://www.terraform.io/docs/language/settings/backends/http.html) +[Terraform HTTP backend](https://www.terraform.io/language/settings/backends/http) to securely store the state files in local storage (the default) or [the remote store of your choice](../../../administration/terraform_state.md). @@ -222,9 +222,9 @@ See [this reference project](https://gitlab.com/gitlab-org/configure/examples/gi ## Using a GitLab-managed Terraform state backend as a remote data source You can use a GitLab-managed Terraform state as a -[Terraform data source](https://www.terraform.io/docs/language/state/remote-state-data.html). +[Terraform data source](https://www.terraform.io/language/state/remote-state-data). To use your existing Terraform state backend as a data source, provide the following details -as [Terraform input variables](https://www.terraform.io/docs/language/values/variables.html): +as [Terraform input variables](https://www.terraform.io/language/values/variables): - **address**: The URL of the remote state backend you want to use as a data source. For example, `https://gitlab.com/api/v4/projects/<TARGET-PROJECT-ID>/terraform/state/<TARGET-STATE-NAME>`. diff --git a/doc/user/infrastructure/index.md b/doc/user/infrastructure/index.md index d8e75928675..73e57ea3d59 100644 --- a/doc/user/infrastructure/index.md +++ b/doc/user/infrastructure/index.md @@ -30,11 +30,11 @@ Learn more about how GitLab can help you run [Infrastructure as Code](iac/index. ## Integrated Kubernetes management The GitLab integration with Kubernetes helps you to install, configure, manage, deploy, and troubleshoot -cluster applications. With the GitLab Agent, you can connect clusters behind a firewall, +cluster applications. With the GitLab agent, you can connect clusters behind a firewall, have real-time access to API endpoints, perform pull-based or push-based deployments for production and non-production environments, and much more. -Learn more about the [GitLab Agent](../clusters/agent/index.md). +Learn more about the [GitLab agent](../clusters/agent/index.md). ## Runbooks in GitLab diff --git a/doc/user/instance/clusters/index.md b/doc/user/instance/clusters/index.md index a184f00f6f6..a5c1402b9ec 100644 --- a/doc/user/instance/clusters/index.md +++ b/doc/user/instance/clusters/index.md @@ -11,7 +11,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w WARNING: This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. To connect clusters to GitLab, -use the [GitLab Agent](../../clusters/agent/index.md). +use the [GitLab agent](../../clusters/agent/index.md). Similar to [project-level](../../project/clusters/index.md) and [group-level](../../group/clusters/index.md) Kubernetes clusters, diff --git a/doc/user/markdown.md b/doc/user/markdown.md index a16f8fd39b1..c81fdc275d9 100644 --- a/doc/user/markdown.md +++ b/doc/user/markdown.md @@ -30,7 +30,7 @@ and the [main GitLab website](https://about.gitlab.com) use [Kramdown](https://k You should not view this page in the documentation, but instead [view these styles as they appear on GitLab](https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/user/markdown.md). GitLab Flavored Markdown extends the [CommonMark specification](https://spec.commonmark.org/current/). -It was inspired by [GitHub Flavored Markdown](https://docs.github.com/en/github/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax). +It was inspired by [GitHub Flavored Markdown](https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax). ## Where you can use GitLab Flavored Markdown @@ -67,15 +67,15 @@ Features not found in standard Markdown: Features [extended from standard Markdown](#features-extended-from-standard-markdown): -| Standard Markdown | Extended Markdown in GitLab | -| ------------------------------------- | ------------------------- | -| [blockquotes](#blockquotes) | [multi-line blockquotes](#multiline-blockquote) | -| [code blocks](#code-spans-and-blocks) | [colored code and syntax highlighting](#colored-code-and-syntax-highlighting) | -| [emphasis](#emphasis) | [multiple underscores in words](#multiple-underscores-in-words-and-mid-word-emphasis) -| [headers](#headers) | [linkable Header IDs](#header-ids-and-links) | -| [images](#images) | [embedded videos](#videos) and [audio](#audio) | -| [line breaks](#line-breaks) | [more line break control](#newlines) | -| [links](#links) | [automatically linking URLs](#url-auto-linking) | +| Standard Markdown | Extended Markdown in GitLab | +|---------------------------------------|---------------------------------------------------------------------------------------| +| [blockquotes](#blockquotes) | [multi-line blockquotes](#multiline-blockquote) | +| [code blocks](#code-spans-and-blocks) | [colored code and syntax highlighting](#colored-code-and-syntax-highlighting) | +| [emphasis](#emphasis) | [multiple underscores in words](#multiple-underscores-in-words-and-mid-word-emphasis) | +| [headers](#headers) | [linkable Header IDs](#header-ids-and-links) | +| [images](#images) | [embedded videos](#videos) and [audio](#audio) | +| [line breaks](#line-breaks) | [more line break control](#newlines) | +| [links](#links) | [automatically linking URLs](#url-auto-linking) | ## Features not found in standard Markdown @@ -262,7 +262,7 @@ The following delimiters are supported: --- title: About Front Matter example: - language: yaml + language: yaml --- ``` @@ -515,31 +515,31 @@ version to reference other projects from the same namespace. GitLab Flavored Markdown recognizes the following: -| references | input | cross-project reference | shortcut inside same namespace | -| :--------------------------------------------------- | :---------------------------- | :----------------------------------------- | :------------------------------- | -| specific user | `@user_name` | | | -| specific group | `@group_name` | | | -| entire team | `@all` | | | -| project | `namespace/project>` | | | -| issue | ``#123`` | `namespace/project#123` | `project#123` | -| merge request | `!123` | `namespace/project!123` | `project!123` | -| snippet | `$123` | `namespace/project$123` | `project$123` | -| [epic](group/epics/index.md) | `&123` | `group1/subgroup&123` | | -| vulnerability **(ULTIMATE)** <sup>1</sup> | `[vulnerability:123]` | `[vulnerability:namespace/project/123]` | `[vulnerability:project/123]` | -| feature flag | `[feature_flag:123]` | `[feature_flag:namespace/project/123]` | `[feature_flag:project/123]` | -| label by ID | `~123` | `namespace/project~123` | `project~123` | -| one-word label by name | `~bug` | `namespace/project~bug` | `project~bug` | -| multi-word label by name | `~"feature request"` | `namespace/project~"feature request"` | `project~"feature request"` | -| scoped label by name | `~"priority::high"` | `namespace/project~"priority::high"` | `project~"priority::high"` | -| project milestone by ID | `%123` | `namespace/project%123` | `project%123` | -| one-word milestone by name | `%v1.23` | `namespace/project%v1.23` | `project%v1.23` | -| multi-word milestone by name | `%"release candidate"` | `namespace/project%"release candidate"` | `project%"release candidate"` | -| specific commit | `9ba12248` | `namespace/project@9ba12248` | `project@9ba12248` | -| commit range comparison | `9ba12248...b19a04f5` | `namespace/project@9ba12248...b19a04f5` | `project@9ba12248...b19a04f5` | -| repository file references | `[README](doc/README.md)` | | | -| repository file line references | `[README](doc/README.md#L13)` | | | -| [alert](../operations/incident_management/alerts.md) | `^alert#123` | `namespace/project^alert#123` | `project^alert#123` | -| contact | `[contact:test@example.com]` | | | +| references | input | cross-project reference | shortcut inside same namespace | +|:----------------------------------------------------------------------------|:------------------------------|:----------------------------------------|:-------------------------------| +| specific user | `@user_name` | | | +| specific group | `@group_name` | | | +| entire team | `@all` | | | +| project | `namespace/project>` | | | +| issue | ``#123`` | `namespace/project#123` | `project#123` | +| merge request | `!123` | `namespace/project!123` | `project!123` | +| snippet | `$123` | `namespace/project$123` | `project$123` | +| [epic](group/epics/index.md) | `&123` | `group1/subgroup&123` | | +| [vulnerability](application_security/vulnerabilities/index.md) <sup>1</sup> | `[vulnerability:123]` | `[vulnerability:namespace/project/123]` | `[vulnerability:project/123]` | +| feature flag | `[feature_flag:123]` | `[feature_flag:namespace/project/123]` | `[feature_flag:project/123]` | +| label by ID | `~123` | `namespace/project~123` | `project~123` | +| one-word label by name | `~bug` | `namespace/project~bug` | `project~bug` | +| multi-word label by name | `~"feature request"` | `namespace/project~"feature request"` | `project~"feature request"` | +| scoped label by name | `~"priority::high"` | `namespace/project~"priority::high"` | `project~"priority::high"` | +| project milestone by ID | `%123` | `namespace/project%123` | `project%123` | +| one-word milestone by name | `%v1.23` | `namespace/project%v1.23` | `project%v1.23` | +| multi-word milestone by name | `%"release candidate"` | `namespace/project%"release candidate"` | `project%"release candidate"` | +| specific commit | `9ba12248` | `namespace/project@9ba12248` | `project@9ba12248` | +| commit range comparison | `9ba12248...b19a04f5` | `namespace/project@9ba12248...b19a04f5` | `project@9ba12248...b19a04f5` | +| repository file references | `[README](doc/README.md)` | | | +| repository file line references | `[README](doc/README.md#L13)` | | | +| [alert](../operations/incident_management/alerts.md) | `^alert#123` | `namespace/project^alert#123` | `project^alert#123` | +| contact | `[contact:test@example.com]` | | | 1. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/222483) in GitLab 13.7. @@ -1486,7 +1486,7 @@ but they do not render properly on `docs.gitlab.com`: #### Copy from spreadsheet and paste in Markdown -[Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27205) in GitLab 12.7. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27205) in GitLab 12.7. If you're working in spreadsheet software (for example, Microsoft Excel, Google Sheets, or Apple Numbers), GitLab creates a Markdown table when you copy-and-paste diff --git a/doc/user/packages/composer_repository/index.md b/doc/user/packages/composer_repository/index.md index 5cfb4278a2c..ea12b225717 100644 --- a/doc/user/packages/composer_repository/index.md +++ b/doc/user/packages/composer_repository/index.md @@ -22,6 +22,9 @@ Then, install the packages whenever you need to use them as a dependency. For documentation of the specific API endpoints that the Composer client uses, see the [Composer API documentation](../../../api/packages/composer.md). +Composer v2.0 is recommended. Composer v1.0 is supported, but it has lower performance when working +in groups with very large numbers of packages. + ## Create a Composer package If you do not have a Composer package, create one and check it in to diff --git a/doc/user/packages/container_registry/index.md b/doc/user/packages/container_registry/index.md index b28f0cbfb35..5b6c71d4dd2 100644 --- a/doc/user/packages/container_registry/index.md +++ b/doc/user/packages/container_registry/index.md @@ -553,13 +553,6 @@ this setting. However, disabling the Container Registry disables all Container R | Private project with Container Registry visibility <br/> set to **Only Project Members** (UI) or `private` (API) | View Container Registry <br/> and pull images | No | No | Yes | | Any project with Container Registry `disabled` | All operations on Container Registry | No | No | No | -## Manifest lists and garbage collection - -Manifest lists are commonly used for creating multi-architecture images. If you rely on manifest -lists, you should tag all the individual manifests referenced by a list in their respective -repositories, and not just the manifest list itself. This ensures that those manifests aren't -garbage collected, as long as they have at least one tag pointing to them. - ## Troubleshooting the GitLab Container Registry ### Docker connection error @@ -702,3 +695,10 @@ There may be some errors not properly cached. Follow these steps to investigate Once adjusted, trigger another tag deletion. You should be able to successfully delete tags. Follow [this issue](https://gitlab.com/gitlab-org/container-registry/-/issues/551) for details. + +### Tags temporarily cannot be marked for deletion + +GitLab is [migrating to the next generation of the Container Registry](https://gitlab.com/groups/gitlab-org/-/epics/5523). +During the migration, you may encounter difficulty deleting tags. +If you encounter an error, it's likely that your image repository is in the process of being migrated. +Please wait a few minutes and try again. diff --git a/doc/user/packages/container_registry/reduce_container_registry_storage.md b/doc/user/packages/container_registry/reduce_container_registry_storage.md index a2a22f138e3..7e8b2865b6e 100644 --- a/doc/user/packages/container_registry/reduce_container_registry_storage.md +++ b/doc/user/packages/container_registry/reduce_container_registry_storage.md @@ -49,20 +49,6 @@ Cleanup policies can be run on all projects, with these exceptions: There are performance risks with enabling it for all projects, especially if you are using an [external registry](#use-with-external-container-registries). -- For self-managed GitLab instances, you can enable or disable the cleanup policy for a specific - project. - - To enable it: - - ```ruby - Feature.enable(:container_expiration_policies_historic_entry, Project.find(<project id>)) - ``` - - To disable it: - - ```ruby - Feature.disable(:container_expiration_policies_historic_entry, Project.find(<project id>)) - ``` WARNING: For performance reasons, enabled cleanup policies are automatically disabled for projects on @@ -171,22 +157,32 @@ Here are examples of regex patterns you may want to use: ### Set cleanup limits to conserve resources -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/288812) in GitLab 13.9. -> - It's [deployed behind a feature flag](../../feature_flags.md), disabled by default. -> - It's enabled on GitLab.com. -> - It's not recommended for production use. -> - To use it in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-cleanup-policy-limits). +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/288812) in GitLab 13.9 [with a flag](../../../administration/feature_flags.md) named `container_registry_expiration_policies_throttling`. Disabled by default. +> - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/80815) in GitLab 14.9. + +FLAG: +By default this feature is available in GitLab 14.9. To disable the feature, an administrator can +[disable the feature flag](../../../administration/feature_flags.md) +named `container_registry_expiration_policies_throttling`. Cleanup policies are executed as a background process. This process is complex, and depending on the number of tags to delete, the process can take time to finish. To prevent server resource starvation, the following application settings are available: -- `container_registry_expiration_policies_worker_capacity`. The maximum number of cleanup workers running concurrently. This must be greater than `1`. - We recommend starting with a low number and increasing it after monitoring the resources used by the background workers. -- `container_registry_delete_tags_service_timeout`. The maximum time, in seconds, that the cleanup process can take to delete a batch of tags. -- `container_registry_cleanup_tags_service_max_list_size`. The maximum number of tags that can be deleted in a single execution. Additional tags must be deleted in another execution. - We recommend starting with a low number, like `100`, and increasing it after monitoring that container images are properly deleted. +- `container_registry_expiration_policies_worker_capacity`: the maximum number of cleanup workers + running concurrently. This must be greater than or equal to `0`. We recommend starting with a low + number and increasing it after monitoring the resources used by the background workers. To remove + all workers and not execute the cleanup policies, set this to `0`. The default value is `4`. +- `container_registry_delete_tags_service_timeout`: the maximum time (in seconds) that the cleanup + process can take to delete a batch of tags. The default value is `250`. +- `container_registry_cleanup_tags_service_max_list_size`: the maximum number of tags that can be + deleted in a single execution. Additional tags must be deleted in another execution. We recommend + starting with a low number and increasing it after monitoring that container images are properly + deleted. The default value is `200`. +- `container_registry_expiration_policies_caching`: enable or disable tag creation timestamp caching + during execution of policies. Cached timestamps are stored in [Redis](../../../development/architecture.md#redis). + Enabled by default. For self-managed instances, those settings can be updated in the [Rails console](../../../administration/operations/rails_console.md#starting-a-rails-console-session): @@ -194,31 +190,11 @@ For self-managed instances, those settings can be updated in the [Rails console] ApplicationSetting.last.update(container_registry_expiration_policies_worker_capacity: 3) ``` -Alternatively, once the limits are [enabled](#enable-or-disable-cleanup-policy-limits), -they are available in the [administrator area](../../admin_area/index.md): +They are also available in the [administrator area](../../admin_area/index.md): 1. On the top bar, select **Menu > Admin**. 1. Go to **Settings > CI/CD > Container Registry**. -#### Enable or disable cleanup policy limits - -The cleanup policies limits are under development and not ready for production use. They are -deployed behind a feature flag that is **disabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) -can enable it. - -To enable it: - -```ruby -Feature.enable(:container_registry_expiration_policies_throttling) -``` - -To disable it: - -```ruby -Feature.disable(:container_registry_expiration_policies_throttling) -``` - ### Use the cleanup policy API You can set, update, and disable the cleanup policies using the GitLab API. diff --git a/doc/user/packages/dependency_proxy/index.md b/doc/user/packages/dependency_proxy/index.md index 925a1b3e8e6..e431d4d7de3 100644 --- a/doc/user/packages/dependency_proxy/index.md +++ b/doc/user/packages/dependency_proxy/index.md @@ -312,3 +312,14 @@ services: - name: ${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/docker:18.09.7-dind alias: docker ``` + +### "Not Found" error when pulling image + +Docker errors similar to the following may indicate that the user running the build job doesn't have +a minimum of the Guest role in the specified Dependency Proxy group: + +```plaintext +ERROR: gitlab.example.com:443/group1/dependency_proxy/containers/alpine:latest: not found + +failed to solve with frontend dockerfile.v0: failed to create LLB definition: gitlab.example.com:443/group1/dependency_proxy/containers/alpine:latest: not found +``` diff --git a/doc/user/packages/generic_packages/index.md b/doc/user/packages/generic_packages/index.md index ada6f033288..9dc859a37e2 100644 --- a/doc/user/packages/generic_packages/index.md +++ b/doc/user/packages/generic_packages/index.md @@ -6,12 +6,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w # GitLab Generic Packages Repository **(FREE)** -> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4209) in GitLab 13.5. -> - It's [deployed behind a feature flag](../../../user/feature_flags.md), enabled by default. -> - It's enabled on GitLab.com. -> - It's able to be enabled or disabled per-project. -> - It's recommended for production use. -> - For GitLab self-managed instances, GitLab administrators can opt to [disable it](#enable-or-disable-generic-packages-in-the-package-registry). +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/4209) in GitLab 13.5 [with a flag](../../../administration/feature_flags.md) named `generic_packages`. Enabled by default. +> - [Feature flag `generic_packages`](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/80886) removed in GitLab 14.8. Publish generic files, like release binaries, in your project's Package Registry. Then, install the packages whenever you need to use them as a dependency. @@ -194,31 +190,6 @@ upload: - Invoke-RestMethod -Headers @{ "JOB-TOKEN"="$CI_JOB_TOKEN" } -InFile path/to/file.txt -uri "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/generic/my_package/0.0.1/file.txt" -Method put ``` -### Enable or disable generic packages in the Package Registry - -Support for generic packages is under development but ready for production use. -It is deployed behind a feature flag that is **enabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) -can opt to disable it. - -To enable it: - -```ruby -# For the instance -Feature.enable(:generic_packages) -# For a single project -Feature.enable(:generic_packages, Project.find(<project id>)) -``` - -To disable it: - -```ruby -# For the instance -Feature.disable(:generic_packages) -# For a single project -Feature.disable(:generic_packages, Project.find(<project id>)) -``` - ### Generic package sample project The [Write CI-CD Variables in Pipeline](https://gitlab.com/guided-explorations/cfg-data/write-ci-cd-variables-in-pipeline) project contains a working example you can use to create, upload, and download generic packages in GitLab CI/CD. diff --git a/doc/user/packages/package_registry/index.md b/doc/user/packages/package_registry/index.md index a2228148447..b980f6a5694 100644 --- a/doc/user/packages/package_registry/index.md +++ b/doc/user/packages/package_registry/index.md @@ -60,6 +60,12 @@ For most package types, the following credential types are valid: allows access to packages in the project running the job for the users running the pipeline. Access to other external projects can be configured. + NOTE: + There's an open issue, + [GitLab-#333444](https://gitlab.com/gitlab-org/gitlab/-/issues/333444), + which prevents you from using a job token with internal projects. This bug only impacts self-managed + GitLab instances. + ## Use GitLab CI/CD to build packages You can use [GitLab CI/CD](../../../ci/index.md) to build packages. diff --git a/doc/user/packages/package_registry/reduce_package_registry_storage.md b/doc/user/packages/package_registry/reduce_package_registry_storage.md index c2e4cd8d889..f8a1e63a228 100644 --- a/doc/user/packages/package_registry/reduce_package_registry_storage.md +++ b/doc/user/packages/package_registry/reduce_package_registry_storage.md @@ -16,7 +16,7 @@ We recommend deleting unnecessary packages and files. This page offers examples ## Check Package Registry Storage Use -The Usage Quotas page (**Settings > Usage Quotas > Storage**) displays storage usage for Packages. +The Usage Quotas page (**Settings > Usage Quotas > Storage**) displays storage usage for Packages. ## Delete a package diff --git a/doc/user/packages/pypi_repository/index.md b/doc/user/packages/pypi_repository/index.md index bf007572ac7..4d46032d229 100644 --- a/doc/user/packages/pypi_repository/index.md +++ b/doc/user/packages/pypi_repository/index.md @@ -254,7 +254,9 @@ Prerequisites: - The maximum allowed package size is 5 GB. - You can't upload the same version of a package multiple times. If you try, you receive the error `400 Bad Request`. -- You cannot publish PyPI packages to a group, only to a project. +- PyPI packages are published using your projectID. +- If your project is in a group, PyPI packages published to your project registry are also available + at the group-level registry (see [Install from the group level](#install-from-the-group-level)). You can then [publish a package by using twine](#publish-a-pypi-package-by-using-twine). diff --git a/doc/user/packages/terraform_module_registry/index.md b/doc/user/packages/terraform_module_registry/index.md index 6fc47a23373..a107d06a63c 100644 --- a/doc/user/packages/terraform_module_registry/index.md +++ b/doc/user/packages/terraform_module_registry/index.md @@ -36,7 +36,7 @@ PUT /projects/:id/packages/terraform/modules/:module-name/:module-system/:module | -------------------| --------------- | ---------| -------------------------------------------------------------------------------------------------------------------------------- | | `id` | integer/string | yes | The ID or [URL-encoded path of the project](../../../api/index.md#namespaced-path-encoding). | | `module-name` | string | yes | The package name. It can contain only lowercase letters (`a-z`), uppercase letter (`A-Z`), numbers (`0-9`), or hyphens (`-`) and cannot exceed 64 characters. -| `module-system` | string | yes | The package system. It can contain only lowercase letters (`a-z`) and numbers (`0-9`), and cannot exceed 64 characters. +| `module-system` | string | yes | The package system. It can contain only lowercase letters (`a-z`) and numbers (`0-9`), and cannot exceed 64 characters. More information can be found in the [Terraform Module Registry Protocol documentation](https://www.terraform.io/internals/module-registry-protocol). | `module-version` | string | yes | The package version. It must be valid according to the [Semantic Versioning Specification](https://semver.org/). Provide the file content in the request body. @@ -93,7 +93,7 @@ credentials "gitlab.com" { Where `gitlab.com` can be replaced with the hostname of your self-managed GitLab instance. -You can then reference your Terraform Module from a downstream Terraform project: +You can then refer to your Terraform Module from a downstream Terraform project: ```plaintext module "<module>" { @@ -101,6 +101,8 @@ module "<module>" { } ``` +Where `<namespace>` is the [namespace](../../../user/group/index.md#namespaces) of the Terraform module registry. + ## Publish a Terraform module by using CI/CD To work with Terraform modules in [GitLab CI/CD](../../../ci/index.md), you can use diff --git a/doc/user/permissions.md b/doc/user/permissions.md index 4476b0dd75b..c90f575e83a 100644 --- a/doc/user/permissions.md +++ b/doc/user/permissions.md @@ -4,7 +4,7 @@ group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Permissions and roles +# Permissions and roles **(FREE)** Users have different abilities depending on the role they have in a particular group or project. If a user is both in a project's group and the @@ -33,168 +33,176 @@ usernames. A GitLab administrator can configure the GitLab instance to ## Project members permissions +- [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/219299) in GitLab 14.8, personal namespace owners appear with Owner role in new projects in their namespace. Introduced [with a flag](../administration/feature_flags.md) named `personal_project_owner_with_owner_access`. Disabled by default. +- [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/351919) in GitLab 14.9. Feature flag `personal_project_owner_with_owner_access` [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/219299). + A user's role determines what permissions they have on a project. The Owner role provides all permissions but is available only: - For group owners. The role is inherited for a group's projects. - For Administrators. -Personal namespace owners have the same permissions as an Owner, but are displayed with the Maintainer role on projects created in their personal namespace. -For more information, see [projects members documentation](project/members/index.md). +Personal [namespace](group/index.md#namespaces) owners: + +- Are displayed as having the Maintainer role on projects in the namespace, but have the same permissions as a user with the Owner role. +- In GitLab 14.9 and later, for new projects in the namespace, are displayed as having the Owner role. + +For more information about how to manage project members, see +[members of a project](project/members/index.md). The following table lists project permissions available for each role: <!-- Keep this table sorted: By topic first, then by minimum role, then alphabetically. --> -| Action | Guest | Reporter | Developer | Maintainer | Owner | -|--------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|----------|-----------|------------|-------| -| [Analytics](analytics/index.md):<br>View issue analytics **(PREMIUM)** | ✓ | ✓ | ✓ | ✓ | ✓ | -| [Analytics](analytics/index.md):<br>View [merge request analytics](analytics/merge_request_analytics.md) **(PREMIUM)** | ✓ | ✓ | ✓ | ✓ | ✓ | -| [Analytics](analytics/index.md):<br>View value stream analytics | ✓ | ✓ | ✓ | ✓ | ✓ | -| [Analytics](analytics/index.md):<br>View [DORA metrics](analytics/ci_cd_analytics.md) | | ✓ | ✓ | ✓ | ✓ | -| [Analytics](analytics/index.md):<br>View [CI/CD analytics](analytics/ci_cd_analytics.md) | | ✓ | ✓ | ✓ | ✓ | -| [Analytics](analytics/index.md):<br>View [code review analytics](analytics/code_review_analytics.md) **(PREMIUM)** | | ✓ | ✓ | ✓ | ✓ | -| [Analytics](analytics/index.md):<br>View [repository analytics](analytics/repository_analytics.md) | | ✓ | ✓ | ✓ | ✓ | -| [Application security](application_security/index.md):<br>View licenses in [dependency list](application_security/dependency_list/index.md) **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | -| [Application security](application_security/index.md):<br>Create and run [on-demand DAST scans](application_security/dast/index.md#on-demand-scans) **(ULTIMATE)** | | | ✓ | ✓ | ✓ | -| [Application security](application_security/index.md):<br>Manage [security policy](application_security/policies/index.md) **(ULTIMATE)** | | | ✓ | ✓ | ✓ | -| [Application security](application_security/index.md):<br>View [dependency list](application_security/dependency_list/index.md) **(ULTIMATE)** | | | ✓ | ✓ | ✓ | -| [Application security](application_security/index.md):<br>View [threats list](application_security/threat_monitoring/index.md#threat-monitoring) **(ULTIMATE)** | | | ✓ | ✓ | ✓ | -| [Application security](application_security/index.md):<br>Create a [CVE ID Request](application_security/cve_id_request.md) **(FREE SAAS)** | | | | ✓ | ✓ | -| [Application security](application_security/index.md):<br>Create or assign [security policy project](application_security/policies/index.md) **(ULTIMATE)** | | | | | ✓ | -| [Clusters](infrastructure/clusters/index.md):<br>View [pod logs](project/clusters/kubernetes_pod_logs.md) | | | ✓ | ✓ | ✓ | -| [Clusters](infrastructure/clusters/index.md):<br>View clusters | | | ✓ | ✓ | ✓ | -| [Clusters](infrastructure/clusters/index.md):<br>Manage clusters | | | | ✓ | ✓ | -| [Container Registry](packages/container_registry/index.md):<br>Create, edit, delete cleanup policies | | | ✓ | ✓ | ✓ | -| [Container Registry](packages/container_registry/index.md):<br>Push an image to the Container Registry | | | ✓ | ✓ | ✓ | -| [Container Registry](packages/container_registry/index.md):<br>Pull an image from the Container Registry | ✓ (*20*) | ✓ (*20*) | ✓ | ✓ | ✓ | -| [Container Registry](packages/container_registry/index.md):<br>Remove a Container Registry image | | | ✓ | ✓ | ✓ | -| [GitLab Pages](project/pages/index.md):<br>View Pages protected by [access control](project/pages/introduction.md#gitlab-pages-access-control) | ✓ | ✓ | ✓ | ✓ | ✓ | -| [GitLab Pages](project/pages/index.md):<br>Manage | | | | ✓ | ✓ | -| [GitLab Pages](project/pages/index.md):<br>Manage GitLab Pages domains and certificates | | | | ✓ | ✓ | -| [GitLab Pages](project/pages/index.md):<br>Remove GitLab Pages | | | | ✓ | ✓ | -| [Incident Management](../operations/incident_management/index.md):<br>View [alerts](../operations/incident_management/alerts.md) | | ✓ | ✓ | ✓ | ✓ | -| [Incident Management](../operations/incident_management/index.md):<br>Assign an alert | ✓ | ✓ | ✓ | ✓ | ✓ | -| [Incident Management](../operations/incident_management/index.md):<br>View [incident](../operations/incident_management/incidents.md) | ✓ | ✓ | ✓ | ✓ | ✓ | -| [Incident Management](../operations/incident_management/index.md):<br>Create [incident](../operations/incident_management/incidents.md) | (*16*) | ✓ | ✓ | ✓ | ✓ | -| [Incident Management](../operations/incident_management/index.md):<br>View [on-call schedules](../operations/incident_management/oncall_schedules.md) | | ✓ | ✓ | ✓ | ✓ | -| [Incident Management](../operations/incident_management/index.md):<br>Participate in on-call rotation | ✓ | ✓ | ✓ | ✓ | ✓ | -| [Incident Management](../operations/incident_management/index.md):<br>View [escalation policies](../operations/incident_management/escalation_policies.md) | | ✓ | ✓ | ✓ | ✓ | -| [Incident Management](../operations/incident_management/index.md):<br>Manage [on-call schedules](../operations/incident_management/oncall_schedules.md) | | | | ✓ | ✓ | -| [Incident Management](../operations/incident_management/index.md):<br>Manage [escalation policies](../operations/incident_management/escalation_policies.md) | | | | ✓ | ✓ | -| [Issues](project/issues/index.md):<br>Add Labels | ✓ (*15*) | ✓ | ✓ | ✓ | ✓ | -| [Issues](project/issues/index.md):<br>Assign | ✓ (*15*) | ✓ | ✓ | ✓ | ✓ | -| [Issues](project/issues/index.md):<br>Create (*18*) | ✓ | ✓ | ✓ | ✓ | ✓ | -| [Issues](project/issues/index.md):<br>Create [confidential issues](project/issues/confidential_issues.md) | ✓ | ✓ | ✓ | ✓ | ✓ | -| [Issues](project/issues/index.md):<br>View [Design Management](project/issues/design_management.md) pages | ✓ | ✓ | ✓ | ✓ | ✓ | -| [Issues](project/issues/index.md):<br>View related issues | ✓ | ✓ | ✓ | ✓ | ✓ | -| [Issues](project/issues/index.md):<br>Set weight | ✓ (*15*) | ✓ | ✓ | ✓ | ✓ | -| [Issues](project/issues/index.md):<br>View [confidential issues](project/issues/confidential_issues.md) | (*2*) | ✓ | ✓ | ✓ | ✓ | -| [Issues](project/issues/index.md):<br>Close / reopen (*19*) | | ✓ | ✓ | ✓ | ✓ | -| [Issues](project/issues/index.md):<br>Lock threads | | ✓ | ✓ | ✓ | ✓ | -| [Issues](project/issues/index.md):<br>Manage related issues | | ✓ | ✓ | ✓ | ✓ | -| [Issues](project/issues/index.md):<br>Manage tracker | | ✓ | ✓ | ✓ | ✓ | -| [Issues](project/issues/index.md):<br>Move issues (*14*) | | ✓ | ✓ | ✓ | ✓ | -| [Issues](project/issues/index.md):<br>Set issue [time tracking](project/time_tracking.md) estimate and time spent | | ✓ | ✓ | ✓ | ✓ | -| [Issues](project/issues/index.md):<br>Upload [Design Management](project/issues/design_management.md) files | | | ✓ | ✓ | ✓ | -| [Issues](project/issues/index.md):<br>Delete | | | | | ✓ | -| [License Compliance](compliance/license_compliance/index.md):<br>View allowed and denied licenses **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | -| [License Compliance](compliance/license_compliance/index.md):<br>View License Compliance reports **(ULTIMATE)** | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | -| [License Compliance](compliance/license_compliance/index.md):<br>View License list **(ULTIMATE)** | | ✓ | ✓ | ✓ | ✓ | -| [License Compliance](compliance/license_compliance/index.md):<br>Manage license policy **(ULTIMATE)** | | | | ✓ | ✓ | -| [Merge requests](project/merge_requests/index.md):<br>Assign reviewer | | ✓ | ✓ | ✓ | ✓ | -| [Merge requests](project/merge_requests/index.md):<br>See list | | ✓ | ✓ | ✓ | ✓ | -| [Merge requests](project/merge_requests/index.md):<br>Apply code change suggestions | | | ✓ | ✓ | ✓ | -| [Merge requests](project/merge_requests/index.md):<br>Approve (*8*) | | | ✓ | ✓ | ✓ | -| [Merge requests](project/merge_requests/index.md):<br>Assign | | | ✓ | ✓ | ✓ | -| [Merge requests](project/merge_requests/index.md):<br>Create (*17*) | | | ✓ | ✓ | ✓ | -| [Merge requests](project/merge_requests/index.md):<br>Add labels | | | ✓ | ✓ | ✓ | -| [Merge requests](project/merge_requests/index.md):<br>Lock threads | | | ✓ | ✓ | ✓ | -| [Merge requests](project/merge_requests/index.md):<br>Manage or accept | | | ✓ | ✓ | ✓ | -| [Merge requests](project/merge_requests/index.md):<br>Manage merge approval rules (project settings) | | | | ✓ | ✓ | -| [Merge requests](project/merge_requests/index.md):<br>Delete | | | | | ✓ | -| [Metrics dashboards](../operations/metrics/dashboards/index.md):<br>Manage user-starred metrics dashboards (*6*) | ✓ | ✓ | ✓ | ✓ | ✓ | -| [Metrics dashboards](../operations/metrics/dashboards/index.md):<br>View metrics dashboard annotations | | ✓ | ✓ | ✓ | ✓ | -| [Metrics dashboards](../operations/metrics/dashboards/index.md):<br>Create/edit/delete metrics dashboard annotations | | | ✓ | ✓ | ✓ | -| [Package registry](packages/index.md):<br>Pull a package | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | -| [Package registry](packages/index.md):<br>Publish a package | | | ✓ | ✓ | ✓ | -| [Package registry](packages/index.md):<br>Delete a package | | | | ✓ | ✓ | -| [Package registry](packages/index.md):<br>Delete a file associated with a package | | | | ✓ | ✓ | -| [Project operations](../operations/index.md):<br>View [Error Tracking](../operations/error_tracking.md) list | | ✓ | ✓ | ✓ | ✓ | -| [Project operations](../operations/index.md):<br>Manage [Feature Flags](../operations/feature_flags.md) **(PREMIUM)** | | | ✓ | ✓ | ✓ | -| [Project operations](../operations/index.md):<br>Manage [Error Tracking](../operations/error_tracking.md) | | | | ✓ | ✓ | -| [Projects](project/index.md):<br>Download project | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | -| [Projects](project/index.md):<br>Leave comments | ✓ | ✓ | ✓ | ✓ | ✓ | -| [Projects](project/index.md):<br>Reposition comments on images (posted by any user) | ✓ (*9*) | ✓ (*9*) | ✓ (*9*) | ✓ | ✓ | -| [Projects](project/index.md):<br>View Insights **(ULTIMATE)** | ✓ | ✓ | ✓ | ✓ | ✓ | -| [Projects](project/index.md):<br>View [releases](project/releases/index.md) | ✓ (*5*) | ✓ | ✓ | ✓ | ✓ | -| [Projects](project/index.md):<br>View Requirements **(ULTIMATE)** | ✓ | ✓ | ✓ | ✓ | ✓ | -| [Projects](project/index.md):<br>View [time tracking](project/time_tracking.md) reports | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | -| [Projects](project/index.md):<br>View [wiki](project/wiki/index.md) pages | ✓ | ✓ | ✓ | ✓ | ✓ | -| [Projects](project/index.md):<br>Create [snippets](snippets.md) | | ✓ | ✓ | ✓ | ✓ | -| [Projects](project/index.md):<br>Manage labels | | ✓ | ✓ | ✓ | ✓ | -| [Projects](project/index.md):<br>View [project traffic statistics](../api/project_statistics.md) | | ✓ | ✓ | ✓ | ✓ | -| [Projects](project/index.md):<br>Create, edit, delete [milestones](project/milestones/index.md). | | | ✓ | ✓ | ✓ | -| [Projects](project/index.md):<br>Create, edit, delete [releases](project/releases/index.md) | | | ✓ (*12*) | ✓ (*12*) | ✓ (*12*) | -| [Projects](project/index.md):<br>Create, edit [wiki](project/wiki/index.md) pages | | | ✓ | ✓ | ✓ | -| [Projects](project/index.md):<br>Enable Review Apps | | | ✓ | ✓ | ✓ | -| [Projects](project/index.md):<br>View project [Audit Events](../administration/audit_events.md) | | | ✓ (*10*) | ✓ | ✓ | -| [Projects](project/index.md):<br>Add deploy keys | | | | ✓ | ✓ | -| [Projects](project/index.md):<br>Add new team members | | | | ✓ | ✓ | -| [Projects](project/index.md):<br>Change [project features visibility](../public_access/public_access.md) level | | | | ✓ (*13*) | ✓ | -| [Projects](project/index.md):<br>Configure [webhooks](project/integrations/webhooks.md) | | | | ✓ | ✓ | -| [Projects](project/index.md):<br>Delete [wiki](project/wiki/index.md) pages | | | ✓ | ✓ | ✓ | -| [Projects](project/index.md):<br>Edit comments (posted by any user) | | | | ✓ | ✓ | -| [Projects](project/index.md):<br>Edit project badges | | | | ✓ | ✓ | -| [Projects](project/index.md):<br>Edit project settings | | | | ✓ | ✓ | -| [Projects](project/index.md):<br>Export project | | | | ✓ | ✓ | -| [Projects](project/index.md):<br>Manage [project access tokens](project/settings/project_access_tokens.md) **(FREE SELF)** **(PREMIUM SAAS)** (*11*) | | | | ✓ | ✓ | -| [Projects](project/index.md):<br>Manage [Project Operations](../operations/index.md) | | | | ✓ | ✓ | -| [Projects](project/index.md):<br>Rename project | | | | ✓ | ✓ | -| [Projects](project/index.md):<br>Share (invite) projects with groups | | | | ✓ (*7*) | ✓ (*7*) | -| [Projects](project/index.md):<br>View 2FA status of members | | | | ✓ | ✓ | -| [Projects](project/index.md):<br>Administer project compliance frameworks | | | | | ✓ | -| [Projects](project/index.md):<br>Archive project | | | | | ✓ | -| [Projects](project/index.md):<br>Change project visibility level | | | | | ✓ | -| [Projects](project/index.md):<br>Delete project | | | | | ✓ | -| [Projects](project/index.md):<br>Disable notification emails | | | | | ✓ | -| [Projects](project/index.md):<br>Transfer project to another namespace | | | | | ✓ | -| [Repository](project/repository/index.md):<br>Pull project code | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | -| [Repository](project/repository/index.md):<br>View project code | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | -| [Repository](project/repository/index.md):<br>View a commit status | | ✓ | ✓ | ✓ | ✓ | -| [Repository](project/repository/index.md):<br>Add tags | | | ✓ | ✓ | ✓ | -| [Repository](project/repository/index.md):<br>Create new branches | | | ✓ | ✓ | ✓ | -| [Repository](project/repository/index.md):<br>Create or update commit status | | | ✓ (*4*) | ✓ | ✓ | -| [Repository](project/repository/index.md):<br>Force push to non-protected branches | | | ✓ | ✓ | ✓ | -| [Repository](project/repository/index.md):<br>Push to non-protected branches | | | ✓ | ✓ | ✓ | -| [Repository](project/repository/index.md):<br>Remove non-protected branches | | | ✓ | ✓ | ✓ | -| [Repository](project/repository/index.md):<br>Rewrite or remove Git tags | | | ✓ | ✓ | ✓ | -| [Repository](project/repository/index.md):<br>Enable or disable branch protection | | | | ✓ | ✓ | -| [Repository](project/repository/index.md):<br>Enable or disable tag protection | | | | ✓ | ✓ | -| [Repository](project/repository/index.md):<br>Manage [push rules](project/repository/push_rules.md) | | | | ✓ | ✓ | -| [Repository](project/repository/index.md):<br>Push to protected branches (*4*) | | | | ✓ | ✓ | -| [Repository](project/repository/index.md):<br>Turn on or off protected branch push for developers | | | | ✓ | ✓ | -| [Repository](project/repository/index.md):<br>Remove fork relationship | | | | | ✓ | -| [Repository](project/repository/index.md):<br>Force push to protected branches (*3*) | | | | | | -| [Repository](project/repository/index.md):<br>Remove protected branches (*3*) | | | | | | -| [Requirements Management](project/requirements/index.md):<br>Archive / reopen **(ULTIMATE)** | | ✓ | ✓ | ✓ | ✓ | -| [Requirements Management](project/requirements/index.md):<br>Create / edit **(ULTIMATE)** | | ✓ | ✓ | ✓ | ✓ | -| [Requirements Management](project/requirements/index.md):<br>Import / export **(ULTIMATE)** | | ✓ | ✓ | ✓ | ✓ | -| [Security dashboard](application_security/security_dashboard/index.md):<br>Create issue from vulnerability finding **(ULTIMATE)** | | | ✓ | ✓ | ✓ | -| [Security dashboard](application_security/security_dashboard/index.md):<br>Create vulnerability from vulnerability finding **(ULTIMATE)** | | | ✓ | ✓ | ✓ | -| [Security dashboard](application_security/security_dashboard/index.md):<br>Dismiss vulnerability **(ULTIMATE)** | | | ✓ | ✓ | ✓ | -| [Security dashboard](application_security/security_dashboard/index.md):<br>Dismiss vulnerability finding **(ULTIMATE)** | | | ✓ | ✓ | ✓ | -| [Security dashboard](application_security/security_dashboard/index.md):<br>Resolve vulnerability **(ULTIMATE)** | | | ✓ | ✓ | ✓ | -| [Security dashboard](application_security/security_dashboard/index.md):<br>Revert vulnerability to detected state **(ULTIMATE)** | | | ✓ | ✓ | ✓ | -| [Security dashboard](application_security/security_dashboard/index.md):<br>Use security dashboard **(ULTIMATE)** | | | ✓ | ✓ | ✓ | -| [Security dashboard](application_security/security_dashboard/index.md):<br>View vulnerability **(ULTIMATE)** | | | ✓ | ✓ | ✓ | -| [Security dashboard](application_security/security_dashboard/index.md):<br>View vulnerability findings in [dependency list](application_security/dependency_list/index.md) **(ULTIMATE)** | | | ✓ | ✓ | ✓ | -| [Terraform](infrastructure/index.md):<br>Read Terraform state | | | ✓ | ✓ | ✓ | -| [Terraform](infrastructure/index.md):<br>Manage Terraform state | | | | ✓ | ✓ | -| [Test cases](../ci/test_cases/index.md):<br>Archive | | ✓ | ✓ | ✓ | ✓ | -| [Test cases](../ci/test_cases/index.md):<br>Create | | ✓ | ✓ | ✓ | ✓ | -| [Test cases](../ci/test_cases/index.md):<br>Move | | ✓ | ✓ | ✓ | ✓ | -| [Test cases](../ci/test_cases/index.md):<br>Reopen | | ✓ | ✓ | ✓ | ✓ | +| Action | Guest | Reporter | Developer | Maintainer | Owner | +|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|----------|-----------|------------|----------| +| [Analytics](analytics/index.md):<br>View issue analytics | ✓ | ✓ | ✓ | ✓ | ✓ | +| [Analytics](analytics/index.md):<br>View [merge request analytics](analytics/merge_request_analytics.md) | ✓ | ✓ | ✓ | ✓ | ✓ | +| [Analytics](analytics/index.md):<br>View value stream analytics | ✓ | ✓ | ✓ | ✓ | ✓ | +| [Analytics](analytics/index.md):<br>View [DORA metrics](analytics/ci_cd_analytics.md) | | ✓ | ✓ | ✓ | ✓ | +| [Analytics](analytics/index.md):<br>View [CI/CD analytics](analytics/ci_cd_analytics.md) | | ✓ | ✓ | ✓ | ✓ | +| [Analytics](analytics/index.md):<br>View [code review analytics](analytics/code_review_analytics.md) | | ✓ | ✓ | ✓ | ✓ | +| [Analytics](analytics/index.md):<br>View [repository analytics](analytics/repository_analytics.md) | | ✓ | ✓ | ✓ | ✓ | +| [Application security](application_security/index.md):<br>View licenses in [dependency list](application_security/dependency_list/index.md) | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| [Application security](application_security/index.md):<br>Create and run [on-demand DAST scans](application_security/dast/index.md#on-demand-scans) | | | ✓ | ✓ | ✓ | +| [Application security](application_security/index.md):<br>Manage [security policy](application_security/policies/index.md) | | | ✓ | ✓ | ✓ | +| [Application security](application_security/index.md):<br>View [dependency list](application_security/dependency_list/index.md) | | | ✓ | ✓ | ✓ | +| [Application security](application_security/index.md):<br>View [threats list](application_security/threat_monitoring/index.md#threat-monitoring) | | | ✓ | ✓ | ✓ | +| [Application security](application_security/index.md):<br>Create a [CVE ID Request](application_security/cve_id_request.md) | | | | ✓ | ✓ | +| [Application security](application_security/index.md):<br>Create or assign [security policy project](application_security/policies/index.md) | | | | | ✓ | +| [Clusters](infrastructure/clusters/index.md):<br>View [pod logs](project/clusters/kubernetes_pod_logs.md) | | | ✓ | ✓ | ✓ | +| [Clusters](infrastructure/clusters/index.md):<br>View clusters | | | ✓ | ✓ | ✓ | +| [Clusters](infrastructure/clusters/index.md):<br>Manage clusters | | | | ✓ | ✓ | +| [Container Registry](packages/container_registry/index.md):<br>Create, edit, delete cleanup policies | | | ✓ | ✓ | ✓ | +| [Container Registry](packages/container_registry/index.md):<br>Push an image to the Container Registry | | | ✓ | ✓ | ✓ | +| [Container Registry](packages/container_registry/index.md):<br>Pull an image from the Container Registry | ✓ (*20*) | ✓ (*20*) | ✓ | ✓ | ✓ | +| [Container Registry](packages/container_registry/index.md):<br>Remove a Container Registry image | | | ✓ | ✓ | ✓ | +| [GitLab Pages](project/pages/index.md):<br>View Pages protected by [access control](project/pages/introduction.md#gitlab-pages-access-control) | ✓ | ✓ | ✓ | ✓ | ✓ | +| [GitLab Pages](project/pages/index.md):<br>Manage | | | | ✓ | ✓ | +| [GitLab Pages](project/pages/index.md):<br>Manage GitLab Pages domains and certificates | | | | ✓ | ✓ | +| [GitLab Pages](project/pages/index.md):<br>Remove GitLab Pages | | | | ✓ | ✓ | +| [Incident Management](../operations/incident_management/index.md):<br>View [alerts](../operations/incident_management/alerts.md) | | ✓ | ✓ | ✓ | ✓ | +| [Incident Management](../operations/incident_management/index.md):<br>Assign an alert | ✓ | ✓ | ✓ | ✓ | ✓ | +| [Incident Management](../operations/incident_management/index.md):<br>View [incident](../operations/incident_management/incidents.md) | ✓ | ✓ | ✓ | ✓ | ✓ | +| [Incident Management](../operations/incident_management/index.md):<br>Create [incident](../operations/incident_management/incidents.md) | (*16*) | ✓ | ✓ | ✓ | ✓ | +| [Incident Management](../operations/incident_management/index.md):<br>View [on-call schedules](../operations/incident_management/oncall_schedules.md) | | ✓ | ✓ | ✓ | ✓ | +| [Incident Management](../operations/incident_management/index.md):<br>Participate in on-call rotation | ✓ | ✓ | ✓ | ✓ | ✓ | +| [Incident Management](../operations/incident_management/index.md):<br>View [escalation policies](../operations/incident_management/escalation_policies.md) | | ✓ | ✓ | ✓ | ✓ | +| [Incident Management](../operations/incident_management/index.md):<br>Manage [on-call schedules](../operations/incident_management/oncall_schedules.md) | | | | ✓ | ✓ | +| [Incident Management](../operations/incident_management/index.md):<br>Manage [escalation policies](../operations/incident_management/escalation_policies.md) | | | | ✓ | ✓ | +| [Issues](project/issues/index.md):<br>Add Labels | ✓ (*15*) | ✓ | ✓ | ✓ | ✓ | +| [Issues](project/issues/index.md):<br>Assign | ✓ (*15*) | ✓ | ✓ | ✓ | ✓ | +| [Issues](project/issues/index.md):<br>Create (*18*) | ✓ | ✓ | ✓ | ✓ | ✓ | +| [Issues](project/issues/index.md):<br>Create [confidential issues](project/issues/confidential_issues.md) | ✓ | ✓ | ✓ | ✓ | ✓ | +| [Issues](project/issues/index.md):<br>View [Design Management](project/issues/design_management.md) pages | ✓ | ✓ | ✓ | ✓ | ✓ | +| [Issues](project/issues/index.md):<br>View related issues | ✓ | ✓ | ✓ | ✓ | ✓ | +| [Issues](project/issues/index.md):<br>Set weight | ✓ (*15*) | ✓ | ✓ | ✓ | ✓ | +| [Issues](project/issues/index.md):<br>View [confidential issues](project/issues/confidential_issues.md) | (*2*) | ✓ | ✓ | ✓ | ✓ | +| [Issues](project/issues/index.md):<br>Close / reopen (*19*) | | ✓ | ✓ | ✓ | ✓ | +| [Issues](project/issues/index.md):<br>Lock threads | | ✓ | ✓ | ✓ | ✓ | +| [Issues](project/issues/index.md):<br>Manage related issues | | ✓ | ✓ | ✓ | ✓ | +| [Issues](project/issues/index.md):<br>Manage tracker | | ✓ | ✓ | ✓ | ✓ | +| [Issues](project/issues/index.md):<br>Move issues (*14*) | | ✓ | ✓ | ✓ | ✓ | +| [Issues](project/issues/index.md):<br>Set issue [time tracking](project/time_tracking.md) estimate and time spent | | ✓ | ✓ | ✓ | ✓ | +| [Issues](project/issues/index.md):<br>Upload [Design Management](project/issues/design_management.md) files | | | ✓ | ✓ | ✓ | +| [Issues](project/issues/index.md):<br>Delete | | | | | ✓ | +| [License Compliance](compliance/license_compliance/index.md):<br>View allowed and denied licenses | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| [License Compliance](compliance/license_compliance/index.md):<br>View License Compliance reports | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| [License Compliance](compliance/license_compliance/index.md):<br>View License list | | ✓ | ✓ | ✓ | ✓ | +| [License Compliance](compliance/license_compliance/index.md):<br>Manage license policy | | | | ✓ | ✓ | +| [Merge requests](project/merge_requests/index.md):<br>Assign reviewer | | ✓ | ✓ | ✓ | ✓ | +| [Merge requests](project/merge_requests/index.md):<br>See list | | ✓ | ✓ | ✓ | ✓ | +| [Merge requests](project/merge_requests/index.md):<br>Apply code change suggestions | | | ✓ | ✓ | ✓ | +| [Merge requests](project/merge_requests/index.md):<br>Approve (*8*) | | | ✓ | ✓ | ✓ | +| [Merge requests](project/merge_requests/index.md):<br>Assign | | | ✓ | ✓ | ✓ | +| [Merge requests](project/merge_requests/index.md):<br>Create (*17*) | | | ✓ | ✓ | ✓ | +| [Merge requests](project/merge_requests/index.md):<br>Add labels | | | ✓ | ✓ | ✓ | +| [Merge requests](project/merge_requests/index.md):<br>Lock threads | | | ✓ | ✓ | ✓ | +| [Merge requests](project/merge_requests/index.md):<br>Manage or accept | | | ✓ | ✓ | ✓ | +| [Merge requests](project/merge_requests/index.md):<br>Manage merge approval rules (project settings) | | | | ✓ | ✓ | +| [Merge requests](project/merge_requests/index.md):<br>Delete | | | | | ✓ | +| [Metrics dashboards](../operations/metrics/dashboards/index.md):<br>Manage user-starred metrics dashboards (*6*) | ✓ | ✓ | ✓ | ✓ | ✓ | +| [Metrics dashboards](../operations/metrics/dashboards/index.md):<br>View metrics dashboard annotations | | ✓ | ✓ | ✓ | ✓ | +| [Metrics dashboards](../operations/metrics/dashboards/index.md):<br>Create/edit/delete metrics dashboard annotations | | | ✓ | ✓ | ✓ | +| [Package registry](packages/index.md):<br>Pull a package | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| [Package registry](packages/index.md):<br>Publish a package | | | ✓ | ✓ | ✓ | +| [Package registry](packages/index.md):<br>Delete a package | | | | ✓ | ✓ | +| [Package registry](packages/index.md):<br>Delete a file associated with a package | | | | ✓ | ✓ | +| [Project operations](../operations/index.md):<br>View [Error Tracking](../operations/error_tracking.md) list | | ✓ | ✓ | ✓ | ✓ | +| [Project operations](../operations/index.md):<br>Manage [Feature Flags](../operations/feature_flags.md) | | | ✓ | ✓ | ✓ | +| [Project operations](../operations/index.md):<br>Manage [Error Tracking](../operations/error_tracking.md) | | | | ✓ | ✓ | +| [Projects](project/index.md):<br>Download project | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| [Projects](project/index.md):<br>Leave comments | ✓ | ✓ | ✓ | ✓ | ✓ | +| [Projects](project/index.md):<br>Reposition comments on images (posted by any user) | ✓ (*9*) | ✓ (*9*) | ✓ (*9*) | ✓ | ✓ | +| [Projects](project/index.md):<br>View Insights | ✓ | ✓ | ✓ | ✓ | ✓ | +| [Projects](project/index.md):<br>View [releases](project/releases/index.md) | ✓ (*5*) | ✓ | ✓ | ✓ | ✓ | +| [Projects](project/index.md):<br>View Requirements | ✓ | ✓ | ✓ | ✓ | ✓ | +| [Projects](project/index.md):<br>View [time tracking](project/time_tracking.md) reports | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| [Projects](project/index.md):<br>View [wiki](project/wiki/index.md) pages | ✓ | ✓ | ✓ | ✓ | ✓ | +| [Projects](project/index.md):<br>Create [snippets](snippets.md) | | ✓ | ✓ | ✓ | ✓ | +| [Projects](project/index.md):<br>Manage labels | | ✓ | ✓ | ✓ | ✓ | +| [Projects](project/index.md):<br>View [project traffic statistics](../api/project_statistics.md) | | ✓ | ✓ | ✓ | ✓ | +| [Projects](project/index.md):<br>Create, edit, delete [milestones](project/milestones/index.md). | | | ✓ | ✓ | ✓ | +| [Projects](project/index.md):<br>Create, edit, delete [releases](project/releases/index.md) | | | ✓ (*12*) | ✓ (*12*) | ✓ (*12*) | +| [Projects](project/index.md):<br>Create, edit [wiki](project/wiki/index.md) pages | | | ✓ | ✓ | ✓ | +| [Projects](project/index.md):<br>Enable Review Apps | | | ✓ | ✓ | ✓ | +| [Projects](project/index.md):<br>View project [Audit Events](../administration/audit_events.md) | | | ✓ (*10*) | ✓ | ✓ | +| [Projects](project/index.md):<br>Add deploy keys | | | | ✓ | ✓ | +| [Projects](project/index.md):<br>Add new team members | | | | ✓ | ✓ | +| [Projects](project/index.md):<br>Change [project features visibility](../public_access/public_access.md) level | | | | ✓ (*13*) | ✓ | +| [Projects](project/index.md):<br>Configure [webhooks](project/integrations/webhooks.md) | | | | ✓ | ✓ | +| [Projects](project/index.md):<br>Delete [wiki](project/wiki/index.md) pages | | | ✓ | ✓ | ✓ | +| [Projects](project/index.md):<br>Edit comments (posted by any user) | | | | ✓ | ✓ | +| [Projects](project/index.md):<br>Edit project badges | | | | ✓ | ✓ | +| [Projects](project/index.md):<br>Edit project settings | | | | ✓ | ✓ | +| [Projects](project/index.md):<br>Export project | | | | ✓ | ✓ | +| [Projects](project/index.md):<br>Manage [project access tokens](project/settings/project_access_tokens.md) (*11*) | | | | ✓ | ✓ | +| [Projects](project/index.md):<br>Manage [Project Operations](../operations/index.md) | | | | ✓ | ✓ | +| [Projects](project/index.md):<br>Rename project | | | | ✓ | ✓ | +| [Projects](project/index.md):<br>Share (invite) projects with groups | | | | ✓ (*7*) | ✓ (*7*) | +| [Projects](project/index.md):<br>View 2FA status of members | | | | ✓ | ✓ | +| [Projects](project/index.md):<br>Administer project compliance frameworks | | | | | ✓ | +| [Projects](project/index.md):<br>Archive project | | | | | ✓ | +| [Projects](project/index.md):<br>Change project visibility level | | | | | ✓ | +| [Projects](project/index.md):<br>Delete project | | | | | ✓ | +| [Projects](project/index.md):<br>Disable notification emails | | | | | ✓ | +| [Projects](project/index.md):<br>Transfer project to another namespace | | | | | ✓ | +| [Repository](project/repository/index.md):<br>Pull project code | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| [Repository](project/repository/index.md):<br>View project code | ✓ (*1*) | ✓ | ✓ | ✓ | ✓ | +| [Repository](project/repository/index.md):<br>View a commit status | | ✓ | ✓ | ✓ | ✓ | +| [Repository](project/repository/index.md):<br>Add tags | | | ✓ | ✓ | ✓ | +| [Repository](project/repository/index.md):<br>Create new branches | | | ✓ | ✓ | ✓ | +| [Repository](project/repository/index.md):<br>Create or update commit status | | | ✓ (*4*) | ✓ | ✓ | +| [Repository](project/repository/index.md):<br>Force push to non-protected branches | | | ✓ | ✓ | ✓ | +| [Repository](project/repository/index.md):<br>Push to non-protected branches | | | ✓ | ✓ | ✓ | +| [Repository](project/repository/index.md):<br>Remove non-protected branches | | | ✓ | ✓ | ✓ | +| [Repository](project/repository/index.md):<br>Rewrite or remove Git tags | | | ✓ | ✓ | ✓ | +| [Repository](project/repository/index.md):<br>Enable or disable branch protection | | | | ✓ | ✓ | +| [Repository](project/repository/index.md):<br>Enable or disable tag protection | | | | ✓ | ✓ | +| [Repository](project/repository/index.md):<br>Manage [push rules](project/repository/push_rules.md) | | | | ✓ | ✓ | +| [Repository](project/repository/index.md):<br>Push to protected branches (*4*) | | | | ✓ | ✓ | +| [Repository](project/repository/index.md):<br>Turn on or off protected branch push for developers | | | | ✓ | ✓ | +| [Repository](project/repository/index.md):<br>Remove fork relationship | | | | | ✓ | +| [Repository](project/repository/index.md):<br>Force push to protected branches (*3*) | | | | | | +| [Repository](project/repository/index.md):<br>Remove protected branches (*3*) | | | | | | +| [Requirements Management](project/requirements/index.md):<br>Archive / reopen | | ✓ | ✓ | ✓ | ✓ | +| [Requirements Management](project/requirements/index.md):<br>Create / edit | | ✓ | ✓ | ✓ | ✓ | +| [Requirements Management](project/requirements/index.md):<br>Import / export | | ✓ | ✓ | ✓ | ✓ | +| [Security dashboard](application_security/security_dashboard/index.md):<br>Create issue from vulnerability finding | | | ✓ | ✓ | ✓ | +| [Security dashboard](application_security/security_dashboard/index.md):<br>Create vulnerability from vulnerability finding | | | ✓ | ✓ | ✓ | +| [Security dashboard](application_security/security_dashboard/index.md):<br>Dismiss vulnerability | | | ✓ | ✓ | ✓ | +| [Security dashboard](application_security/security_dashboard/index.md):<br>Dismiss vulnerability finding | | | ✓ | ✓ | ✓ | +| [Security dashboard](application_security/security_dashboard/index.md):<br>Resolve vulnerability | | | ✓ | ✓ | ✓ | +| [Security dashboard](application_security/security_dashboard/index.md):<br>Revert vulnerability to detected state | | | ✓ | ✓ | ✓ | +| [Security dashboard](application_security/security_dashboard/index.md):<br>Use security dashboard | | | ✓ | ✓ | ✓ | +| [Security dashboard](application_security/security_dashboard/index.md):<br>View vulnerability | | | ✓ | ✓ | ✓ | +| [Security dashboard](application_security/security_dashboard/index.md):<br>View vulnerability findings in [dependency list](application_security/dependency_list/index.md) | | | ✓ | ✓ | ✓ | +| [Terraform](infrastructure/index.md):<br>Read Terraform state | | | ✓ | ✓ | ✓ | +| [Terraform](infrastructure/index.md):<br>Manage Terraform state | | | | ✓ | ✓ | +| [Test cases](../ci/test_cases/index.md):<br>Archive | | ✓ | ✓ | ✓ | ✓ | +| [Test cases](../ci/test_cases/index.md):<br>Create | | ✓ | ✓ | ✓ | ✓ | +| [Test cases](../ci/test_cases/index.md):<br>Move | | ✓ | ✓ | ✓ | ✓ | +| [Test cases](../ci/test_cases/index.md):<br>Reopen | | ✓ | ✓ | ✓ | ✓ | <!-- markdownlint-disable MD029 --> @@ -243,33 +251,33 @@ More details about the permissions for some project-level features follow. - [Pipeline visibility](../ci/enable_or_disable_ci.md#enable-cicd-in-a-project): When set to **Everyone with Access**, gives access to certain CI/CD "view" features to *non-project* members. -| Action | Non-member | Guest | Reporter | Developer | Maintainer | Owner | -|-----------------------------------------------------------------------------------|------------|---------|----------|-----------|------------|-------| -| See that artifacts exist | ✓ (*3*) | ✓ (*3*) | ✓ | ✓ | ✓ | ✓ | -| View a list of jobs | ✓ (*1*) | ✓ (*2*) | ✓ | ✓ | ✓ | ✓ | -| View and download artifacts | ✓ (*1*) | ✓ (*2*) | ✓ | ✓ | ✓ | ✓ | -| View [environments](../ci/environments/index.md) | ✓ (*3*) | ✓ (*3*) | ✓ | ✓ | ✓ | ✓ | -| View job logs and job details page | ✓ (*1*) | ✓ (*2*) | ✓ | ✓ | ✓ | ✓ | -| View pipeline details page | ✓ (*1*) | ✓ (*2*) | ✓ | ✓ | ✓ | ✓ | -| View pipelines page | ✓ (*1*) | ✓ (*2*) | ✓ | ✓ | ✓ | ✓ | -| View pipelines tab in MR | ✓ (*3*) | ✓ (*3*) | ✓ | ✓ | ✓ | ✓ | -| [View vulnerabilities in a pipeline](application_security/security_dashboard/index.md#view-vulnerabilities-in-a-pipeline) | | ✓ (*2*) | ✓ | ✓ | ✓ | ✓ | -| Cancel and retry jobs | | | | ✓ | ✓ | ✓ | -| Create new [environments](../ci/environments/index.md) | | | | ✓ | ✓ | ✓ | -| Delete job logs or job artifacts | | | | ✓ (*4*) | ✓ | ✓ | -| Run CI/CD pipeline for a protected branch | | | | ✓ (*5*) | ✓ (*5*) | ✓ | -| Stop [environments](../ci/environments/index.md) | | | | ✓ | ✓ | ✓ | -| View a job with [debug logging](../ci/variables/index.md#debug-logging) | | | | ✓ | ✓ | ✓ | -| Use pipeline editor | | | | ✓ | ✓ | ✓ | -| Add specific runners to project | | | | | ✓ | ✓ | -| Clear runner caches manually | | | | | ✓ | ✓ | -| Enable shared runners in project | | | | | ✓ | ✓ | -| Manage CI/CD settings | | | | | ✓ | ✓ | -| Manage job triggers | | | | | ✓ | ✓ | -| Manage project-level CI/CD variables | | | | | ✓ | ✓ | -| Run [interactive web terminals](../ci/interactive_web_terminal/index.md) | | | | | ✓ | ✓ | -| Use [environment terminals](../ci/environments/index.md#web-terminals-deprecated) | | | | | ✓ | ✓ | -| Delete pipelines | | | | | | ✓ | +| Action | Non-member | Guest | Reporter | Developer | Maintainer | Owner | +|---------------------------------------------------------------------------------------------------------------------------|------------|---------|----------|-----------|------------|-------| +| See that artifacts exist | ✓ (*3*) | ✓ (*3*) | ✓ | ✓ | ✓ | ✓ | +| View a list of jobs | ✓ (*1*) | ✓ (*2*) | ✓ | ✓ | ✓ | ✓ | +| View and download artifacts | ✓ (*1*) | ✓ (*2*) | ✓ | ✓ | ✓ | ✓ | +| View [environments](../ci/environments/index.md) | ✓ (*3*) | ✓ (*3*) | ✓ | ✓ | ✓ | ✓ | +| View job logs and job details page | ✓ (*1*) | ✓ (*2*) | ✓ | ✓ | ✓ | ✓ | +| View pipeline details page | ✓ (*1*) | ✓ (*2*) | ✓ | ✓ | ✓ | ✓ | +| View pipelines page | ✓ (*1*) | ✓ (*2*) | ✓ | ✓ | ✓ | ✓ | +| View pipelines tab in MR | ✓ (*3*) | ✓ (*3*) | ✓ | ✓ | ✓ | ✓ | +| [View vulnerabilities in a pipeline](application_security/security_dashboard/index.md#view-vulnerabilities-in-a-pipeline) | | ✓ (*2*) | ✓ | ✓ | ✓ | ✓ | +| Cancel and retry jobs | | | | ✓ | ✓ | ✓ | +| Create new [environments](../ci/environments/index.md) | | | | ✓ | ✓ | ✓ | +| Delete job logs or job artifacts | | | | ✓ (*4*) | ✓ | ✓ | +| Run CI/CD pipeline for a protected branch | | | | ✓ (*5*) | ✓ (*5*) | ✓ | +| Stop [environments](../ci/environments/index.md) | | | | ✓ | ✓ | ✓ | +| View a job with [debug logging](../ci/variables/index.md#debug-logging) | | | | ✓ | ✓ | ✓ | +| Use pipeline editor | | | | ✓ | ✓ | ✓ | +| Run [interactive web terminals](../ci/interactive_web_terminal/index.md) | | | | ✓ | ✓ | ✓ | +| Add specific runners to project | | | | | ✓ | ✓ | +| Clear runner caches manually | | | | | ✓ | ✓ | +| Enable shared runners in project | | | | | ✓ | ✓ | +| Manage CI/CD settings | | | | | ✓ | ✓ | +| Manage job triggers | | | | | ✓ | ✓ | +| Manage project-level CI/CD variables | | | | | ✓ | ✓ | +| Use [environment terminals](../ci/environments/index.md#web-terminals-deprecated) | | | | | ✓ | ✓ | +| Delete pipelines | | | | | | ✓ | <!-- markdownlint-disable MD029 --> @@ -288,20 +296,20 @@ More details about the permissions for some project-level features follow. This table shows granted privileges for jobs triggered by specific types of users: -| Action | Guest, Reporter | Developer | Maintainer| Administrator | -|---------------------------------------------|-----------------|-----------|-----------|---------------| -| Run CI job | | ✓ | ✓ | ✓ | -| Clone source and LFS from current project | | ✓ | ✓ | ✓ | -| Clone source and LFS from public projects | | ✓ | ✓ | ✓ | -| Clone source and LFS from internal projects | | ✓ (*1*) | ✓ (*1*) | ✓ | -| Clone source and LFS from private projects | | ✓ (*2*) | ✓ (*2*) | ✓ (*2*) | -| Pull container images from current project | | ✓ | ✓ | ✓ | -| Pull container images from public projects | | ✓ | ✓ | ✓ | -| Pull container images from internal projects| | ✓ (*1*) | ✓ (*1*) | ✓ | -| Pull container images from private projects | | ✓ (*2*) | ✓ (*2*) | ✓ (*2*) | -| Push container images to current project | | ✓ | ✓ | ✓ | -| Push container images to other projects | | | | | -| Push source and LFS | | | | | +| Action | Guest, Reporter | Developer | Maintainer | Administrator | +|----------------------------------------------|-----------------|-----------|------------|---------------| +| Run CI job | | ✓ | ✓ | ✓ | +| Clone source and LFS from current project | | ✓ | ✓ | ✓ | +| Clone source and LFS from public projects | | ✓ | ✓ | ✓ | +| Clone source and LFS from internal projects | | ✓ (*1*) | ✓ (*1*) | ✓ | +| Clone source and LFS from private projects | | ✓ (*2*) | ✓ (*2*) | ✓ (*2*) | +| Pull container images from current project | | ✓ | ✓ | ✓ | +| Pull container images from public projects | | ✓ | ✓ | ✓ | +| Pull container images from internal projects | | ✓ (*1*) | ✓ (*1*) | ✓ | +| Pull container images from private projects | | ✓ (*2*) | ✓ (*2*) | ✓ (*2*) | +| Push container images to current project | | ✓ | ✓ | ✓ | +| Push container images to other projects | | | | | +| Push source and LFS | | | | | 1. Only if the triggering user is not an external one. 1. Only if the triggering user is a member of the project. @@ -327,7 +335,7 @@ to learn more. ### Value stream analytics permissions Find the current permissions on the value stream analytics dashboard, as described in -[related documentation](analytics/value_stream_analytics.md#permissions). +[related documentation](analytics/value_stream_analytics.md#access-permissions-for-value-stream-analytics). ### Issue board permissions @@ -361,64 +369,64 @@ The following table lists group permissions available for each role: <!-- Keep this table sorted: first, by minimum role, then alphabetically. --> -| Action | Guest | Reporter | Developer | Maintainer | Owner | -|--------------------------------------------------------------------------|-------|----------|-----------|------------|-------| -| Browse group | ✓ | ✓ | ✓ | ✓ | ✓ | -| Pull a container image using the dependency proxy | ✓ | ✓ | ✓ | ✓ | ✓ | -| View Contribution analytics | ✓ | ✓ | ✓ | ✓ | ✓ | -| View group epic **(PREMIUM)** | ✓ | ✓ | ✓ | ✓ | ✓ | -| View group wiki pages **(PREMIUM)** | ✓ (6) | ✓ | ✓ | ✓ | ✓ | -| View Insights **(ULTIMATE)** | ✓ | ✓ | ✓ | ✓ | ✓ | -| View Insights charts **(ULTIMATE)** | ✓ | ✓ | ✓ | ✓ | ✓ | -| View Issue analytics **(PREMIUM)** | ✓ | ✓ | ✓ | ✓ | ✓ | -| View value stream analytics | ✓ | ✓ | ✓ | ✓ | ✓ | -| Create/edit group epic **(PREMIUM)** | | ✓ | ✓ | ✓ | ✓ | -| Create/edit/delete epic boards **(PREMIUM)** | | ✓ | ✓ | ✓ | ✓ | -| Manage group labels | | ✓ | ✓ | ✓ | ✓ | -| Publish [packages](packages/index.md) | | | ✓ | ✓ | ✓ | -| Pull [packages](packages/index.md) | | ✓ | ✓ | ✓ | ✓ | -| Delete [packages](packages/index.md | | | | ✓ | ✓ | -| Pull a Container Registry image | ✓ (7) | ✓ | ✓ | ✓ | ✓ | -| Remove a Container Registry image | | | ✓ | ✓ | ✓ | -| View Group DevOps Adoption **(ULTIMATE)** | | ✓ | ✓ | ✓ | ✓ | -| View metrics dashboard annotations | | ✓ | ✓ | ✓ | ✓ | -| View Productivity analytics **(PREMIUM)** | | ✓ | ✓ | ✓ | ✓ | -| Create and edit group wiki pages **(PREMIUM)** | | | ✓ | ✓ | ✓ | -| Create project in group | | | ✓ (3)(5) | ✓ (3) | ✓ (3) | -| Create/edit/delete group milestones | | | ✓ | ✓ | ✓ | -| Create/edit/delete iterations | | | ✓ | ✓ | ✓ | -| Create/edit/delete metrics dashboard annotations | | | ✓ | ✓ | ✓ | -| Enable/disable a dependency proxy | | | ✓ | ✓ | ✓ | -| Purge the dependency proxy for a group | | | | | ✓ | -| Use security dashboard **(ULTIMATE)** | | | ✓ | ✓ | ✓ | -| View group Audit Events | | | ✓ (7) | ✓ (7) | ✓ | -| Create subgroup | | | | ✓ (1) | ✓ | -| Delete group wiki pages **(PREMIUM)** | | | ✓ | ✓ | ✓ | -| Edit epic comments (posted by any user) **(ULTIMATE)** | | | | ✓ (2) | ✓ (2) | -| List group deploy tokens | | | | ✓ | ✓ | -| Manage [group push rules](group/index.md#group-push-rules) **(PREMIUM)** | | | | ✓ | ✓ | -| View/manage group-level Kubernetes cluster | | | | ✓ | ✓ | -| Administer project compliance frameworks | | | | | ✓ | -| Create/Delete group deploy tokens | | | | | ✓ | -| Change group visibility level | | | | | ✓ | -| Delete group | | | | | ✓ | -| Delete group epic **(PREMIUM)** | | | | | ✓ | -| Disable notification emails | | | | | ✓ | -| Edit group settings | | | | | ✓ | -| Edit SAML SSO **(PREMIUM SAAS)** | | | | | ✓ (4) | -| Filter members by 2FA status | | | | | ✓ | -| Manage group level CI/CD variables | | | | | ✓ | -| Manage group members | | | | | ✓ | -| Share (invite) groups with groups | | | | | ✓ | -| View 2FA status of members | | | | | ✓ | -| View Billing **(FREE SAAS)** | | | | | ✓ (4) | -| View Usage Quotas **(FREE SAAS)** | | | | | ✓ (4) | -| Manage runners | | | | | ✓ | +| Action | Guest | Reporter | Developer | Maintainer | Owner | +|-----------------------------------------------------------------------------------------|-------|----------|-----------|------------|-------| +| Browse group | ✓ | ✓ | ✓ | ✓ | ✓ | +| Pull a container image using the dependency proxy | ✓ | ✓ | ✓ | ✓ | ✓ | +| View Contribution analytics | ✓ | ✓ | ✓ | ✓ | ✓ | +| View group [epic](group/epics/index.md) | ✓ | ✓ | ✓ | ✓ | ✓ | +| View [group wiki](project/wiki/group.md) pages | ✓ (6) | ✓ | ✓ | ✓ | ✓ | +| View [Insights](project/insights/index.md) | ✓ | ✓ | ✓ | ✓ | ✓ | +| View [Insights](project/insights/index.md) charts | ✓ | ✓ | ✓ | ✓ | ✓ | +| View [Issue analytics](analytics/issue_analytics.md) | ✓ | ✓ | ✓ | ✓ | ✓ | +| View value stream analytics | ✓ | ✓ | ✓ | ✓ | ✓ | +| Create/edit group [epic](group/epics/index.md) | | ✓ | ✓ | ✓ | ✓ | +| Create/edit/delete [epic boards](group/epics/epic_boards.md) | | ✓ | ✓ | ✓ | ✓ | +| Manage group labels | | ✓ | ✓ | ✓ | ✓ | +| Publish [packages](packages/index.md) | | | ✓ | ✓ | ✓ | +| Pull [packages](packages/index.md) | | ✓ | ✓ | ✓ | ✓ | +| Delete [packages](packages/index.md) | | | | ✓ | ✓ | +| Pull a Container Registry image | ✓ (7) | ✓ | ✓ | ✓ | ✓ | +| Remove a Container Registry image | | | ✓ | ✓ | ✓ | +| View [Group DevOps Adoption](group/devops_adoption/index.md) | | ✓ | ✓ | ✓ | ✓ | +| View metrics dashboard annotations | | ✓ | ✓ | ✓ | ✓ | +| View [Productivity analytics](analytics/productivity_analytics.md) | | ✓ | ✓ | ✓ | ✓ | +| View Usage quota data | | ✓ | ✓ | ✓ | ✓ | +| Create and edit [group wiki](project/wiki/group.md) pages | | | ✓ | ✓ | ✓ | +| Create project in group | | | ✓ (3)(5) | ✓ (3) | ✓ (3) | +| Create/edit/delete group milestones | | | ✓ | ✓ | ✓ | +| Create/edit/delete iterations | | | ✓ | ✓ | ✓ | +| Create/edit/delete metrics dashboard annotations | | | ✓ | ✓ | ✓ | +| Enable/disable a dependency proxy | | | ✓ | ✓ | ✓ | +| Purge the dependency proxy for a group | | | | | ✓ | +| Use [security dashboard](application_security/security_dashboard/index.md) | | | ✓ | ✓ | ✓ | +| View group Audit Events | | | ✓ (7) | ✓ (7) | ✓ | +| Create subgroup | | | | ✓ (1) | ✓ | +| Delete [group wiki](project/wiki/group.md) pages | | | ✓ | ✓ | ✓ | +| Edit [epic](group/epics/index.md) comments (posted by any user) | | | | ✓ (2) | ✓ (2) | +| List group deploy tokens | | | | ✓ | ✓ | +| Manage [group push rules](group/index.md#group-push-rules) | | | | ✓ | ✓ | +| View/manage group-level Kubernetes cluster | | | | ✓ | ✓ | +| Administer project compliance frameworks | | | | | ✓ | +| Create/Delete group deploy tokens | | | | | ✓ | +| Change group visibility level | | | | | ✓ | +| Delete group | | | | | ✓ | +| Delete group [epic](group/epics/index.md) | | | | | ✓ | +| Disable notification emails | | | | | ✓ | +| Edit group settings | | | | | ✓ | +| Edit [SAML SSO](group/saml_sso/index.md) | | | | | ✓ (4) | +| Filter members by 2FA status | | | | | ✓ | +| Manage group level CI/CD variables | | | | | ✓ | +| Manage group members | | | | | ✓ | +| Share (invite) groups with groups | | | | | ✓ | +| View 2FA status of members | | | | | ✓ | +| View [Billing](../subscriptions/gitlab_com/index.md#view-your-gitlab-saas-subscription) | | | | | ✓ (4) | +| View [Usage Quotas](usage_quotas.md) Page | | | | ✓ | ✓ (4) | +| Manage runners | | | | | ✓ | <!-- markdownlint-disable MD029 --> -1. Groups can be set to [allow either Owners or Owners and - Maintainers to create subgroups](group/subgroups/index.md#creating-a-subgroup) +1. Groups can be set to allow either Owners, or Owners and users with the Maintainer role, to [create subgroups](group/subgroups/index.md#create-a-subgroup). 2. Introduced in GitLab 12.2. 3. Default project creation role can be changed at: - The [instance level](admin_area/settings/visibility_and_access_controls.md#define-which-roles-can-create-projects). @@ -437,7 +445,7 @@ permission level from the parent group(s). This model allows access to nested groups if you have membership in one of its parents. To learn more, read through the documentation on -[subgroups memberships](group/subgroups/index.md#membership). +[subgroups memberships](group/subgroups/index.md#subgroup-membership). ## External users **(FREE SELF)** @@ -478,7 +486,7 @@ An administrator can flag a user as external by either of the following methods: 1. On the left sidebar, select **Overview > Users** to create a new user or edit an existing one. There, you can find the option to flag the user as external. -Additionally users can be set as external users using: +Additionally, users can be set as external users using: - [SAML groups](../integration/saml.md#external-groups). - [LDAP groups](../administration/auth/ldap/ldap_synchronization.md#external-groups). @@ -543,7 +551,7 @@ with the permissions described on the documentation on [auditor users permission ## Users with minimal access **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/40942) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.4. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/40942) in GitLab 13.4. Owners can add members with a "minimal access" role to a parent group. Such users don't automatically have access to projects and subgroups underneath. Owners must explicitly add these "minimal access" users to the specific subgroups and diff --git a/doc/user/profile/account/delete_account.md b/doc/user/profile/account/delete_account.md index c116b1fc00d..3f5554b80ac 100644 --- a/doc/user/profile/account/delete_account.md +++ b/doc/user/profile/account/delete_account.md @@ -5,7 +5,7 @@ group: Authentication and Authorization info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Deleting a User account **(FREE)** +# Deleting a user account **(FREE)** Users can be deleted from a GitLab instance, either by: @@ -15,7 +15,7 @@ Users can be deleted from a GitLab instance, either by: NOTE: Deleting a user deletes all projects in that user namespace. -## As a user +## Delete your own account As a user, to delete your own account: @@ -24,7 +24,7 @@ As a user, to delete your own account: 1. On the left sidebar, select **Account**. 1. Select **Delete account**. -## As an administrator **(FREE SELF)** +## Delete users and user contributions **(FREE SELF)** As an administrator, to delete a user account: @@ -32,55 +32,40 @@ As an administrator, to delete a user account: 1. On the left sidebar, select **Overview > Users**. 1. Select a user. 1. Under the **Account** tab, select: - - **Delete user** to delete only the user but maintain their - [associated records](#associated-records). - - **Delete user and contributions** to delete the user and - their associated records. + - **Delete user** to delete only the user but maintain their [associated records](#associated-records). You can't use this option if + the selected user is the sole owner of any groups. + - **Delete user and contributions** to delete the user and their associated records. This option also removes all groups (and + projects within these groups) where the user is the sole direct Owner of a group. Inherited ownership doesn't apply. WARNING: -Using the **Delete user and contributions** option may result -in removing more data than intended. Please see [associated records](#associated-records) -below for additional details. +Using the **Delete user and contributions** option may result in removing more data than intended. See +[associated records](#associated-records) for additional details. ### Associated records -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/7393) for issues in GitLab 9.0. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10467) for merge requests, award emoji, notes, and abuse reports in GitLab 9.1. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/10273) hard deletion from abuse reports and spam logs in GitLab 9.1. -> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/11853) hard deletion from the API in GitLab 9.3. - -There are two options for deleting users: - -- **Delete user** -- **Delete user and contributions** - -When using the **Delete user** option, not all associated records are deleted with the user. -Here's a list of things created by the user that are **not** deleted: - -- Abuse reports -- Award emoji -- Epics -- Issues -- Merge requests -- Notes - -Instead of being deleted, these records are moved to a system-wide -user with the username Ghost User, whose sole purpose is to act as a container -for such records. Any commits made by a deleted user still display the -username of the original user. - -When using the **Delete user and contributions** option, **all** associated records -are removed. This includes all of the items mentioned above including issues, -merge requests, notes/comments, and more. Consider -[blocking a user](../../admin_area/moderate_users.md#block-a-user) -or using the **Delete user** option instead. - -When a user is deleted from an [abuse report](../../admin_area/review_abuse_reports.md) -or spam log, these associated -records are not ghosted and are removed, along with any groups the user -is a sole owner of. Administrators can also request this behavior when -deleting users from the [API](../../../api/users.md#user-deletion) or the -Admin Area. +When deleting users, you can either: + +- Delete just the user. Not all associated records are deleted with the user. Instead of being deleted, these records + are moved to a system-wide user with the username Ghost User. The Ghost User's purpose is to act as a container for + such records. Any commits made by a deleted user still display the username of the original user. +- Delete the user and their contributions, including: + - Abuse reports. + - Award emojis. + - Epics. + - Groups of which the user is the only user with the Owner role. + - Issues. + - Merge requests. + - Notes and comments. + - Personal access tokens. + - Snippets. + +An alternative to deleting is [blocking a user](../../admin_area/moderate_users.md#block-a-user). + +When a user is deleted from an [abuse report](../../admin_area/review_abuse_reports.md) or spam log, these associated +records are always removed. + +The deleting associated records option can be requested in the [API](../../../api/users.md#user-deletion) as well as +the Admin Area. ## Troubleshooting diff --git a/doc/user/profile/img/personal_readme_setup_v14_5.png b/doc/user/profile/img/personal_readme_setup_v14_5.png Binary files differindex 92d8e0ec936..6a776506ac6 100644 --- a/doc/user/profile/img/personal_readme_setup_v14_5.png +++ b/doc/user/profile/img/personal_readme_setup_v14_5.png diff --git a/doc/user/profile/notifications.md b/doc/user/profile/notifications.md index acbaf62579f..4c9622810b2 100644 --- a/doc/user/profile/notifications.md +++ b/doc/user/profile/notifications.md @@ -184,6 +184,7 @@ Users are notified of the following events: | Password changed | User | Security email, always sent when user changes their own password. | | Password changed by administrator | User | Security email, always sent when an administrator changes the password of another user. | | Personal access tokens expiring soon | User | Security email, always sent. | +| Personal access tokens have been created | User | Security email, always sent. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/337591) in GitLab 14.9. | | Personal access tokens have expired | User | Security email, always sent. | | Project access level changed | User | Sent when user project access level is changed. | | SSH key has expired | User | Security email, always sent. _[Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/322637) in GitLab 13.12._ | diff --git a/doc/user/profile/personal_access_tokens.md b/doc/user/profile/personal_access_tokens.md index 10d718fdf1a..1fbbe438370 100644 --- a/doc/user/profile/personal_access_tokens.md +++ b/doc/user/profile/personal_access_tokens.md @@ -25,8 +25,8 @@ Personal access tokens are: - Used with a GitLab username to authenticate with GitLab features that require usernames. For example, [GitLab managed Terraform state backend](../infrastructure/iac/terraform_state.md#using-a-gitlab-managed-terraform-state-backend-as-a-remote-data-source) and [Docker container registry](../packages/container_registry/index.md#authenticate-with-the-container-registry), -- Similar to [project access tokens](../project/settings/project_access_tokens.md), but are attached - to a user rather than a project. +- Similar to [project access tokens](../project/settings/project_access_tokens.md) and [group access tokens](../group/settings/group_access_tokens.md), but are attached + to a user rather than a project or group. NOTE: Though required, GitLab usernames are ignored when authenticating with a personal access token. diff --git a/doc/user/profile/preferences.md b/doc/user/profile/preferences.md index 52baf5189e1..48160bb97ac 100644 --- a/doc/user/profile/preferences.md +++ b/doc/user/profile/preferences.md @@ -39,7 +39,7 @@ The default theme is Indigo. You can choose between 10 themes: ## Dark mode -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28252) in GitLab 13.1 as an Alpha release. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28252) in GitLab 13.1 as an [Alpha](../../policy/alpha-beta-support.md#alpha-features) release. GitLab has started work on dark mode! The dark mode Alpha release is available in the spirit of iteration and the lower expectations of diff --git a/doc/user/project/badges.md b/doc/user/project/badges.md index 79d395d51c3..2f9e04fb828 100644 --- a/doc/user/project/badges.md +++ b/doc/user/project/badges.md @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w Badges are a unified way to present condensed pieces of information about your projects. They consist of a small image and a URL that the image points to. Examples for badges can be the [pipeline status](../../ci/pipelines/settings.md#pipeline-status-badge), -[test coverage](../../ci/pipelines/settings.md#test-coverage-report-badge), or ways to contact the +[test coverage](../../ci/pipelines/settings.md#test-coverage-report-badge), [latest release](../../ci/pipelines/settings.md#latest-release-badge), or ways to contact the project maintainers.  diff --git a/doc/user/project/canary_deployments.md b/doc/user/project/canary_deployments.md index 77cf04cc7eb..344caed7449 100644 --- a/doc/user/project/canary_deployments.md +++ b/doc/user/project/canary_deployments.md @@ -6,8 +6,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Canary Deployments **(FREE)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/1659) in [GitLab Premium](https://about.gitlab.com/pricing/) 9.1. -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/212320) to GitLab Free in 13.8. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/1659) in GitLab 9.1. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/212320) from GitLab Premium to GitLab Free in 13.8. A popular [Continuous Deployment](https://en.wikipedia.org/wiki/Continuous_deployment) strategy, where a small portion of the fleet is updated to the new version of @@ -35,7 +35,7 @@ your pods fleet and watch their behavior as a percentage of your user base visits the temporarily deployed feature. If all works well, you can deploy the feature to production knowing that it shouldn't cause any problems. -Canary deployments are also especially useful for backend refactors, performance +Canary deployments are also especially required for backend refactors, performance improvements, or other changes where the user interface doesn't change, but you want to make sure the performance stays the same, or improves. Developers need to be careful when using canaries with user-facing changes, because by default, @@ -50,7 +50,7 @@ this document. Canary deployments require that you properly configure deploy boards: 1. Follow the steps to [enable deploy boards](deploy_boards.md#enabling-deploy-boards). -1. To track canary deployments you need to label your Kubernetes deployments and +1. To track canary deployments you must label your Kubernetes deployments and pods with `track: canary`. To get started quickly, you can use the [Auto Deploy](../../topics/autodevops/stages.md#auto-deploy) template for canary deployments that GitLab provides. @@ -60,19 +60,19 @@ Any other track label is considered `canary` (temporary). This allows GitLab to discover whether a deployment is stable or canary (temporary). Once all of the above are set up and the pipeline has run at least once, -navigate to the environments page under **Pipelines > Environments**. +Go to the environments page under **Pipelines > Environments**. As the pipeline executes, deploy boards clearly mark canary pods, enabling -quick and easy insight into the status of each environment and deployment. +quick and clear insight into the status of each environment and deployment. Canary deployments are marked with a yellow dot in the deploy board so that you -can easily notice them. +can quickly notice them.  ### Advanced traffic control with Canary Ingress (DEPRECATED) -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/215501) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.6. -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/212320) to Free in GitLab 13.8. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/215501) in GitLab 13.6. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/212320) from GitLab Premium to GitLab Free in 13.8. > - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. WARNING: @@ -82,7 +82,7 @@ Canary deployments can be more strategic with [Canary Ingress](https://kubernete which is an advanced traffic routing service that controls incoming HTTP requests between stable and canary deployments based on factors such as weight, sessions, cookies, and others. GitLab uses this service in its [Auto Deploy architecture](../../topics/autodevops/upgrading_auto_deploy_dependencies.md#v2-chart-resource-architecture) -to let users easily and safely roll out their new deployments. +to let users quickly and safely roll out their new deployments. #### How to set up a Canary Ingress in a canary deployment @@ -115,13 +115,13 @@ Here's an example setup flow from scratch: #### How to change the traffic weight on a Canary Ingress -You can change the traffic weight within your environment's deploy board by using [GraphiQL](../../api/graphql/getting_started.md#graphiql), +You can change the traffic weight in your environment's deploy board by using [GraphiQL](../../api/graphql/getting_started.md#graphiql), or by sending requests to the [GraphQL API](../../api/graphql/getting_started.md#command-line). To use your [deploy board](../../user/project/deploy_boards.md): -1. Navigate to **Deployments > Environments** for your project. -1. Set the new weight with the dropdown on the right side. +1. Go to **Deployments > Environments** for your project. +1. Set the new weight with the dropdown list on the right side. 1. Confirm your selection. Here's an example using [GraphiQL](../../api/graphql/getting_started.md#graphiql): diff --git a/doc/user/project/clusters/add_eks_clusters.md b/doc/user/project/clusters/add_eks_clusters.md index e14a71a7e10..82106c9d1a9 100644 --- a/doc/user/project/clusters/add_eks_clusters.md +++ b/doc/user/project/clusters/add_eks_clusters.md @@ -10,7 +10,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. WARNING: -This feature was deprecated in GitLab 14.5. Use [Infrastructure as Code](../../infrastructure/iac/index.md#create-a-new-cluster-through-iac-deprecated) +This feature was deprecated in GitLab 14.5. Use [Infrastructure as Code](../../infrastructure/iac/index.md#create-a-new-cluster-through-iac) to create new clusters. Through GitLab, you can create new clusters and add existing clusters hosted on Amazon Elastic @@ -19,11 +19,11 @@ Kubernetes Service (EKS). ## Connect an existing EKS cluster If you already have an EKS cluster and want to connect it to GitLab, -use the [GitLab Agent](../../clusters/agent/index.md). +use the [GitLab agent](../../clusters/agent/index.md). ## Create a new EKS cluster -To create a new cluster from GitLab, use [Infrastructure as Code](../../infrastructure/iac/index.md#create-a-new-cluster-through-iac-deprecated). +To create a new cluster from GitLab, use [Infrastructure as Code](../../infrastructure/iac/index.md#create-a-new-cluster-through-iac). ### How to create a new cluster on EKS through cluster certificates (DEPRECATED) diff --git a/doc/user/project/clusters/add_existing_cluster.md b/doc/user/project/clusters/add_existing_cluster.md index 0c3827bcbb1..f2d537513b7 100644 --- a/doc/user/project/clusters/add_existing_cluster.md +++ b/doc/user/project/clusters/add_existing_cluster.md @@ -10,7 +10,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w WARNING: This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. -To connect your cluster to GitLab, use the [GitLab Agent](../../clusters/agent/index.md) +To connect your cluster to GitLab, use the [GitLab agent](../../clusters/agent/index.md) instead. If you have an existing Kubernetes cluster, you can add it to a project, group, @@ -69,8 +69,8 @@ To add a Kubernetes cluster to your project, group, or instance: 1. Project's **{cloud-gear}** **Infrastructure > Kubernetes clusters** page, for a project-level cluster. 1. Group's **{cloud-gear}** **Kubernetes** page, for a group-level cluster. 1. **Menu > Admin > Kubernetes** page, for an instance-level cluster. -1. Click **Add Kubernetes cluster**. -1. Click the **Add existing cluster** tab and fill in the details: +1. On the **Kubernetes clusters** page, select the **Connect with a certificate** option from the **Actions** dropdown menu. +1. On the **Connect a cluster** page, fill in the details: 1. **Kubernetes cluster name** (required) - The name you wish to give the cluster. 1. **Environment scope** (required) - The [associated environment](multiple_kubernetes_clusters.md#setting-the-environment-scope) to this cluster. diff --git a/doc/user/project/clusters/add_gke_clusters.md b/doc/user/project/clusters/add_gke_clusters.md index 99edddeb3e9..cb8d04e0e28 100644 --- a/doc/user/project/clusters/add_gke_clusters.md +++ b/doc/user/project/clusters/add_gke_clusters.md @@ -19,7 +19,7 @@ hosted on Google Kubernetes Engine (GKE). ## Connect an existing GKE cluster If you already have a GKE cluster and want to connect it to GitLab, -use the [GitLab Agent](../../clusters/agent/index.md). +use the [GitLab agent](../../clusters/agent/index.md). ## Create a new GKE cluster from GitLab diff --git a/doc/user/project/clusters/add_remove_clusters.md b/doc/user/project/clusters/add_remove_clusters.md index a0fca517f2e..a4f6dc325c8 100644 --- a/doc/user/project/clusters/add_remove_clusters.md +++ b/doc/user/project/clusters/add_remove_clusters.md @@ -10,7 +10,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w WARNING: This feature was [deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 14.0. -To create a new cluster use [Infrastructure as Code](../../infrastructure/iac/index.md#create-a-new-cluster-through-iac-deprecated). +To create a new cluster use [Infrastructure as Code](../../infrastructure/iac/index.md#create-a-new-cluster-through-iac). NOTE: Every new Google Cloud Platform (GCP) account receives @@ -29,7 +29,7 @@ in a few clicks. > [Deprecated](https://gitlab.com/gitlab-org/gitlab/-/issues/327908) in GitLab 14.0. -As of GitLab 14.0, use [Infrastructure as Code](../../infrastructure/iac/index.md#create-a-new-cluster-through-iac-deprecated) +As of GitLab 14.0, use [Infrastructure as Code](../../infrastructure/iac/index.md#create-a-new-cluster-through-iac) to **safely create new clusters from GitLab**. Creating clusters from GitLab using cluster certificates is still available on the @@ -49,7 +49,7 @@ supports connecting existing clusters using the certificate-based connection met ## Add existing cluster -As of GitLab 14.0, use the [GitLab Agent](../../clusters/agent/index.md) +As of GitLab 14.0, use the [GitLab agent](../../clusters/agent/index.md) to connect your cluster to GitLab. Alternatively, you can [add an existing cluster](add_existing_cluster.md) @@ -57,7 +57,7 @@ through the certificate-based method, but we don't recommend using this method f ## Configure your cluster -As of GitLab 14.0, use the [GitLab Agent](../../clusters/agent/index.md) +As of GitLab 14.0, use the [GitLab agent](../../clusters/agent/index.md) to configure your cluster. ## Disable a cluster diff --git a/doc/user/project/clusters/cluster_access.md b/doc/user/project/clusters/cluster_access.md index 4e00ae0dd07..8ff0a275649 100644 --- a/doc/user/project/clusters/cluster_access.md +++ b/doc/user/project/clusters/cluster_access.md @@ -11,7 +11,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w WARNING: This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. -To connect your cluster to GitLab, use the [GitLab Agent](../../clusters/agent/index.md) +To connect your cluster to GitLab, use the [GitLab agent](../../clusters/agent/index.md) instead. When creating a cluster in GitLab, you are asked if you would like to create either: diff --git a/doc/user/project/clusters/deploy_to_cluster.md b/doc/user/project/clusters/deploy_to_cluster.md index e89ce12b35e..c1cdf754c11 100644 --- a/doc/user/project/clusters/deploy_to_cluster.md +++ b/doc/user/project/clusters/deploy_to_cluster.md @@ -10,8 +10,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w WARNING: This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. -To connect your cluster to GitLab, use the [GitLab Agent](../../clusters/agent/index.md). -To deploy with the Agent, use the [CI/CD Tunnel](../../clusters/agent/ci_cd_tunnel.md). +To connect your cluster to GitLab, use the [GitLab agent](../../clusters/agent/index.md). +To deploy with the agent, use the [CI/CD workflow](../../clusters/agent/ci_cd_tunnel.md). A Kubernetes cluster can be the destination for a deployment job. If @@ -143,6 +143,6 @@ Reasons for failure include: NOTE: Project-level clusters upgraded from GitLab 12.0 or older may be configured -in a way that causes this error. Ensure you deselect the +in a way that causes this error. Ensure you clear the [GitLab-managed cluster](gitlab_managed_clusters.md) option if you want to manage namespaces and service accounts yourself. diff --git a/doc/user/project/clusters/gitlab_managed_clusters.md b/doc/user/project/clusters/gitlab_managed_clusters.md index 686b3026b2c..9c5cc14f720 100644 --- a/doc/user/project/clusters/gitlab_managed_clusters.md +++ b/doc/user/project/clusters/gitlab_managed_clusters.md @@ -12,7 +12,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w WARNING: This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. -To connect your cluster to GitLab, use the [GitLab Agent](../../../user/clusters/agent/index.md). +To connect your cluster to GitLab, use the [GitLab agent](../../../user/clusters/agent/index.md). To manage applications, use the [Cluster Project Management Template](../../../user/clusters/management_project_template.md). You can choose to allow GitLab to manage your cluster for you. If your cluster diff --git a/doc/user/project/clusters/index.md b/doc/user/project/clusters/index.md index dc37060593c..f89d863e83b 100644 --- a/doc/user/project/clusters/index.md +++ b/doc/user/project/clusters/index.md @@ -12,7 +12,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w WARNING: This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. To connect clusters to GitLab, use the -[GitLab Agent](../../clusters/agent/index.md). +[GitLab agent](../../clusters/agent/index.md). [Project-level](../../infrastructure/clusters/connect/index.md#cluster-levels-deprecated) Kubernetes clusters allow you to connect a Kubernetes cluster to a project in GitLab. diff --git a/doc/user/project/clusters/multiple_kubernetes_clusters.md b/doc/user/project/clusters/multiple_kubernetes_clusters.md index a56eaa9b953..149df5248c8 100644 --- a/doc/user/project/clusters/multiple_kubernetes_clusters.md +++ b/doc/user/project/clusters/multiple_kubernetes_clusters.md @@ -13,7 +13,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w WARNING: Using multiple Kubernetes clusters for a single project **with cluster certificates** was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. -To connect clusters to GitLab, use the [GitLab Agent](../../../user/clusters/agent/index.md). +To connect clusters to GitLab, use the [GitLab agent](../../../user/clusters/agent/index.md). You can associate more than one Kubernetes cluster to your project. That way you can have different clusters for different environments, diff --git a/doc/user/project/clusters/protect/container_host_security/index.md b/doc/user/project/clusters/protect/container_host_security/index.md index a0297a7a86f..f6f31ee8f36 100644 --- a/doc/user/project/clusters/protect/container_host_security/index.md +++ b/doc/user/project/clusters/protect/container_host_security/index.md @@ -1,7 +1,7 @@ --- stage: Protect group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Container Host Security **(FREE)** diff --git a/doc/user/project/clusters/protect/container_host_security/quick_start_guide.md b/doc/user/project/clusters/protect/container_host_security/quick_start_guide.md index 4e56b7e5140..25e2f6ddb91 100644 --- a/doc/user/project/clusters/protect/container_host_security/quick_start_guide.md +++ b/doc/user/project/clusters/protect/container_host_security/quick_start_guide.md @@ -1,7 +1,7 @@ --- stage: Protect group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Getting started with Container Host Security **(FREE)** diff --git a/doc/user/project/clusters/protect/container_network_security/index.md b/doc/user/project/clusters/protect/container_network_security/index.md index 7176a1cf1b9..eeaf7e82ef4 100644 --- a/doc/user/project/clusters/protect/container_network_security/index.md +++ b/doc/user/project/clusters/protect/container_network_security/index.md @@ -1,7 +1,7 @@ --- stage: Protect group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Container Network Security **(FREE)** diff --git a/doc/user/project/clusters/protect/container_network_security/quick_start_guide.md b/doc/user/project/clusters/protect/container_network_security/quick_start_guide.md index e6c91302d7b..43f4ea6c326 100644 --- a/doc/user/project/clusters/protect/container_network_security/quick_start_guide.md +++ b/doc/user/project/clusters/protect/container_network_security/quick_start_guide.md @@ -1,7 +1,7 @@ --- stage: Protect group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Getting started with Container Network Security **(FREE)** diff --git a/doc/user/project/clusters/protect/index.md b/doc/user/project/clusters/protect/index.md index 473195f4c17..3a80132fb50 100644 --- a/doc/user/project/clusters/protect/index.md +++ b/doc/user/project/clusters/protect/index.md @@ -1,7 +1,7 @@ --- stage: Protect group: Container Security -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Protecting your deployed applications **(FREE)** diff --git a/doc/user/project/code_owners.md b/doc/user/project/code_owners.md index eb18834cc6b..fefc27063a6 100644 --- a/doc/user/project/code_owners.md +++ b/doc/user/project/code_owners.md @@ -124,7 +124,7 @@ Only one CODEOWNERS pattern can match per file path. ### Organize Code Owners by putting them into sections -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12137) in GitLab Premium 13.2 behind a feature flag, enabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12137) in GitLab 13.2 behind a feature flag, enabled by default. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/42389) in GitLab 13.4. You can organize Code Owners by putting them into named sections. @@ -170,7 +170,7 @@ entries under **Database**. The entries defined under the sections **Documentati ### Make a Code Owners section optional -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/232995) in GitLab Premium 13.8. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/232995) in GitLab 13.8. You can designate optional sections in your Code Owners file. Prepend the section name with the caret `^` character to treat the entire section as optional. diff --git a/doc/user/project/deploy_boards.md b/doc/user/project/deploy_boards.md index 15490ab0b94..567c15c7d5f 100644 --- a/doc/user/project/deploy_boards.md +++ b/doc/user/project/deploy_boards.md @@ -7,8 +7,8 @@ type: howto, reference # Deploy boards (DEPRECATED) **(FREE)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/1589) in [GitLab Premium](https://about.gitlab.com/pricing/) 9.0. -> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/212320) to GitLab Free in 13.8. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/1589) in GitLab 9.0. +> - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/212320) from GitLab Premium to GitLab Free in 13.8. > - In GitLab 13.5 and earlier, apps that consist of multiple deployments are shown as > duplicates on the deploy board. This is [fixed](https://gitlab.com/gitlab-org/gitlab/-/issues/8463) > in GitLab 13.6. @@ -20,7 +20,7 @@ type: howto, reference WARNING: This feature was [deprecated](https://gitlab.com/groups/gitlab-org/configure/-/epics/8) in GitLab 14.5. [An epic exists](https://gitlab.com/groups/gitlab-org/-/epics/2493) -to add this functionality to the [Agent](../index.md). +to add this functionality to the [agent](../index.md). GitLab deploy boards offer a consolidated view of the current health and status of each CI [environment](../../ci/environments/index.md) running on [Kubernetes](https://kubernetes.io), displaying the status diff --git a/doc/user/project/deploy_keys/img/public_deploy_key_v13_0.png b/doc/user/project/deploy_keys/img/public_deploy_key_v13_0.png Binary files differdeleted file mode 100644 index 2f708555af1..00000000000 --- a/doc/user/project/deploy_keys/img/public_deploy_key_v13_0.png +++ /dev/null diff --git a/doc/user/project/deploy_keys/index.md b/doc/user/project/deploy_keys/index.md index 57f6efa3092..b7674be2fa0 100644 --- a/doc/user/project/deploy_keys/index.md +++ b/doc/user/project/deploy_keys/index.md @@ -2,186 +2,145 @@ stage: Release group: Release info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments -type: howto, reference --- # Deploy keys **(FREE)** -Deploy keys allow read-only or read-write access to your -repositories by importing an SSH public key into your GitLab instance. +Use deploy keys to access repositories that are hosted in GitLab. In most cases, you use deploy keys +to access a repository from an external host, like a build server or Continuous Integration (CI) server. -Deploy keys streamline interactions between your GitLab repository and another -machine. For example, setting up a deploy key allows secure cloning of your -repositories to a Continuous Integration (CI) server without setting up a fake -user account. +Depending on your needs, you might want to use a [deploy token](../deploy_tokens/) to access a repository instead. -There are two types of deploy keys: +| Attribute | Deploy key | Deploy token | +|------------------|-------------|--------------| +| Sharing | Shareable between multiple projects, even those in different groups. | Belong to a project or group. | +| Source | Public SSH key generated on an external host. | Generated on your GitLab instance, and is provided to users only at creation time. | +| Validity | Valid as long as it's registered and enabled. | Can be given an expiration date. | +| Registry access | Cannot access a package registry. | Can read from and write to a package registry. | -- [Project deploy keys](#project-deploy-keys) -- [Public deploy keys](#public-deploy-keys) +## Scope -## Key details on deploy keys +A deploy key has a defined scope when it is created: -Deploy keys allow a remote machine (VM, physical, and so on) to access a GitLab -repository with just a few steps. If you want a remote machine to interact with a GitLab -repository in automation, it's a simple solution. +- **Project deploy key:** Access is limited to the selected project. +- **Public deploy key:** Access can be granted to _any_ project in a GitLab instance. Access to each + project must be [granted](#grant-project-access-to-a-public-deploy-key) by a user with at least + the Maintainer role. -A drawback is that your repository could become vulnerable if a remote machine is compromised -by a hacker. You should limit access to the remote machine before a deploy key is -enabled on your repository. A good rule to follow is to provide access only to trusted users, -and make sure that the allowed users have at least the Maintainer role -in the GitLab project. +You cannot change a deploy key's scope after creating it. -If this security implication is a concern for your organization, -[Deploy Tokens](../deploy_tokens/index.md) works as an alternative, but with more -security control. +## Permissions -## Deploy keys permissions +A deploy key is given a permission level when it is created: -You can choose the access level of a deploy key when you enable it on a project: +- **Read-only:** A read-only deploy key can only read from the repository. +- **Read-write:** A read-write deploy key can read from, and write to, the repository. -- `read-only`: The deploy key can read a repository. -- `read-write`: The deploy key can read a repository and write to it. +You can change a deploy key's permission level after creating it. Changing a project deploy key's +permissions only applies for the current project. -Project maintainers and owners can activate and deactivate deploy keys. -They can also add their own deploy keys and enable them for this project. +When a read-write deploy key is used to push a commit, GitLab checks if the creator of the +deploy key has permission to access the resource. -When a `write-access` deploy key is used to push a commit, GitLab checks if -the **creator** of the deploy key has permission to access the resource. For example: +For example: - When a deploy key is used to push a commit to a [protected branch](../protected_branches.md), - the **creator** of the deploy key must have access to the branch. -- When a deploy key is used to push a commit that triggers a CI/CD pipelines, the **creator** of - the deploy key must have access to the CI/CD resources (like protected environments, secret variables, and so on). -- If the **creator** of a deploy key does not have permissions to read a project's - repository, the deploy key _might_ encounter an error during the process. + the _creator_ of the deploy key must have access to the branch. +- When a deploy key is used to push a commit that triggers a CI/CD pipeline, the _creator_ of the + deploy key must have access to the CI/CD resources, including protected environments and secret + variables. -## Differences between deploy keys and deploy tokens +## View deploy keys -Both deploy keys and [deploy tokens](../deploy_tokens/index.md#deploy-tokens) can -help you access a repository, but there are some notables differences between them: - -- Deploy keys are shareable between projects that are not related or don't even - belong to the same group. Deploy tokens belong to either a project or - [a group](../deploy_tokens/index.md#group-deploy-token). -- A deploy key is an SSH key you generate on the **remote machine**. A deploy - token, on the other hand, is generated by your **GitLab instance**, and is - provided to users only once (at creation time). -- A deploy key is valid as long as it's registered and enabled. Deploy tokens - can be time-sensitive, as you can control their validity by setting an - expiration date to them. -- You can't log in to a registry with deploy keys, or perform read / write operations - on it, but this [is possible with deploy tokens](../deploy_tokens/index.md#gitlab-deploy-token). -- You need an SSH key pair to use deploy keys, but not deploy tokens. - -## How to enable deploy keys - -### Project deploy keys - -[Project maintainers and owners](../../permissions.md#project-members-permissions) -can add or enable a deploy key for a project repository: +To view the deploy keys available to a project: 1. On the top bar, select **Menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Deploy keys**. -1. Specify a title for the new deploy key and paste your public SSH key. -1. Optional. To allow `read-write` access, select the **Grant write permissions to this key** checkbox. Leave it unchecked for `read-only` access. -There are three lists of project deploy keys: +The deploy keys available are listed: -- Enabled deploy keys -- Privately accessible deploy keys -- Public accessible deploy keys +- **Enabled deploy keys:** Deploy keys that have access to the project. +- **Privately accessible deploy keys:** Project deploy keys that don't have access to the project. +- **Public accessible deploy keys:** Public deploy keys that don't have access to the project. - +## Create a project deploy key -After you add a key, it's enabled for this project by default and it appears -in the **Enabled deploy keys** tab. +Prerequisites: -In the **Privately accessible deploy keys** tab, you can enable a private key which -has been already imported in a different project. If you have access to these keys, -it's because you have either: +- You must have at least the Maintainer role for the project. +- [Generate an SSH key pair](../../../ssh/index.md#generate-an-ssh-key-pair). Put the private SSH + key on the host that requires access to the repository. -- Previously uploaded the keys yourself in a different project. -- You are a maintainer or owner of the other project where the keys were imported. - -In the **Publicly accessible deploy keys** tab, you can enable -keys that were [made available to your entire GitLab instance](#public-deploy-keys). +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. +1. Expand **Deploy keys**. +1. Complete the fields. +1. Optional. To grant `read-write` permission, select the **Grant write permissions to this key** + checkbox. -After a key is added, you can edit it to update its title, or switch between `read-only` -and `read-write` access. +A project deploy key is enabled when it is created. You can modify only a project deploy key's +name and permissions. -NOTE: -If you have enabled a privately or publicly accessible or deploy key for your -project, and if you then update the access level for this key from `read-only` to -`read-write`, the change is only for the **current project**. +## Create a public deploy key -### Public deploy keys +Prerequisites: -Public deploy keys allow `read-only` or `read-write` access to any repository in -your GitLab instance. This allows for the integration of your repositories to -secure, shared services, such as CI/CD. +- You must have administrator access. +- [Generate an SSH key pair](../../../ssh/index.md#generate-an-ssh-key-pair). Put the private SSH + key on the host that requires access to the repository. -Instance administrators can add public deploy keys: +To create a public deploy key: 1. On the top bar, select **Menu > Admin**. 1. On the left sidebar, select **Deploy Keys**. 1. Select **New deploy key**. +1. Complete the fields. + - Use a meaningful description for **Name**. For example, include the name of the external host + or application that will use the public deploy key. -Make sure your new key has a meaningful title. This title is the primary -way for project maintainers and owners to identify the correct public deploy key -to add to a repository or project. For example, the key you set up might be -intended to give access to a SaaS CI/CD instance. In this case use the name of -that service in the key title if that is all the key is used for. +You can modify only a public deploy key's name. - +## Grant project access to a public deploy key -After adding a key, it's available to any shared system. Users with a maintainer role or -higher can [authorize a public deploy key](#project-deploy-keys) to start using -it with the project. +Prerequisites: -NOTE: -The **Publicly accessible deploy keys** tab in a Project's CI/CD -settings only appears if there is at least one Public deploy key configured. +- You must have at least the Maintainer role for the project. -Public deploy keys can provide greater security compared to project deploy keys. -This is because the administrator of the target integrated system is the only -entity who needs to know or configure the key value. +To grant a public deploy key access to a project: -When creating a Public deploy key, consider what scope and permissions are -required for it across the entire GitLab instance. For very narrow usage, such -as a single specific service, a `read-only` deploy key tied to this service is -best. If the service entails broader usage across the instance, a -deploy key with full `read-write` access is more appropriate. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Repository**. +1. Expand **Deploy keys**. +1. Select **Publicly accessible deploy keys**. +1. In the key's row, select **Enable**. +1. To grant read-write permission to the public deploy key: + 1. In the key's row, select **Edit** (**{pencil}**). + 1. Select the **Grant write permissions to this key** checkbox. + +## Revoke project access of a deploy key -WARNING: -Adding a public deploy key **does not** immediately expose any repository -to the remote machine. Access to a project is only given when a project -maintainer chooses to make use of a deploy key in the project's -configuration. +To revoke a deploy key's access to a project, you can disable it. Any service that relies on +a deploy key stops working when the key is disabled. -## How to disable deploy keys +Prerequisites: -[Project maintainers and owners](../../permissions.md#project-members-permissions) -can remove or disable a deploy key for a project repository: +- You must have at least the Maintainer role for the project. + +To disable a deploy key: 1. On the top bar, select **Menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Repository**. 1. Expand **Deploy keys**. 1. Select **Disable** (**{cancel}**). -NOTE: -Any service that relies on a deploy key stops working after that key is removed. - -If the key is **publicly accessible**, it is removed from the project, but can -still be found under **Publicly accessible deploy keys**. - -If the key is **privately accessible** and only in use by this project, it is -deleted entirely from GitLab on removal. +What happens to the deploy key when it is disabled depends on the following: -If the key is **privately accessible** and also in use by other projects, it is -removed from the project, but still available under **Privately accessible -deploy keys**. +- If the key is publicly accessible, it is removed from the project but still available in the + **Publicly accessible deploy keys** tab. +- If the key is privately accessible and only in use by this project, it is deleted. +- If the key is privately accessible and also in use by other projects, it is removed from the + project, but still available in the **Privately accessible deploy keys** tab. ## Troubleshooting @@ -192,7 +151,7 @@ branch](../protected_branches.md). - The owner associated to a deploy key does not have access to the protected branch. - The owner associated to a deploy key does not have [membership](../members/index.md) to the project of the protected branch. -- **No one** is selected in [the "Allowed to push" section](../protected_branches.md#configure-a-protected-branch) of the protected branch. +- **No one** is selected in [the **Allowed to push** section](../protected_branches.md#configure-a-protected-branch) of the protected branch. All deploy keys are associated to an account. Since the permissions for an account can change, this might lead to scenarios where a deploy key that was working is suddenly unable to push to a protected branch. diff --git a/doc/user/project/file_lock.md b/doc/user/project/file_lock.md index 9911c90863d..4810fb96ed3 100644 --- a/doc/user/project/file_lock.md +++ b/doc/user/project/file_lock.md @@ -25,10 +25,10 @@ GitLab supports two different modes of file locking: - [Exclusive file locks](#exclusive-file-locks) for binary files: done **through the command line** with Git LFS and `.gitattributes`, it prevents locked - files from being modified on any branch. **(FREE)** + files from being modified on any branch. - [Default branch locks](#default-branch-file-and-directory-locks): done **through the GitLab UI**, it prevents locked files and directories being - modified on the default branch. **(PREMIUM)** + modified on the default branch. ## Permissions @@ -40,7 +40,7 @@ users are prevented from modifying locked files by pushing, merging, or any other means, and are shown an error like: `The path '.gitignore' is locked by Administrator`. -## Exclusive file locks **(FREE)** +## Exclusive file locks This process allows you to lock single files or file extensions and it is done through the command line. It doesn't require GitLab paid subscriptions. @@ -192,7 +192,7 @@ Suggested workflow for shared projects: ## Default branch file and directory locks **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/440) in GitLab Enterprise Edition 8.9. Available in [GitLab Premium](https://about.gitlab.com/pricing/). +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/440) in GitLab 8.9. This process allows you to lock one file at a time through the GitLab UI and requires access to [GitLab Premium](https://about.gitlab.com/pricing/) diff --git a/doc/user/project/highlighting.md b/doc/user/project/highlighting.md index 728f51a8062..ef0c787b9d3 100644 --- a/doc/user/project/highlighting.md +++ b/doc/user/project/highlighting.md @@ -7,61 +7,67 @@ type: reference # Syntax Highlighting **(FREE)** -GitLab provides syntax highlighting on all files through the [Rouge](https://rubygems.org/gems/rouge) Ruby gem. It attempts to guess what language to use based on the file extension, which most of the time is sufficient. +GitLab provides syntax highlighting on all files through the +[Rouge](https://rubygems.org/gems/rouge) Ruby gem. It attempts to guess what language +to use based on the file extension, which most of the time is sufficient. + +The paths here are Git's built-in [`.gitattributes` interface](https://git-scm.com/docs/gitattributes). NOTE: The [Web IDE](web_ide/index.md) and [Snippets](../snippets.md) use [Monaco Editor](https://microsoft.github.io/monaco-editor/) for text editing, which internally uses the [Monarch](https://microsoft.github.io/monaco-editor/monarch.html) library for syntax highlighting. -<!-- vale gitlab.Spelling = NO --> - -If GitLab is guessing wrong, you can override its choice of language using the -`gitlab-language` attribute in `.gitattributes`. For example, if you are working in a Prolog -project and using the `.pl` file extension (which would normally be highlighted as Perl), -you can add the following to your `.gitattributes` file: +## Override syntax highlighting for a file type -<!-- vale gitlab.Spelling = YES --> - -``` conf -*.pl gitlab-language=prolog -``` +NOTE: +The Web IDE [does not support `.gitattribute` files](https://gitlab.com/gitlab-org/gitlab/-/issues/22014). -<!-- vale gitlab.Spelling = NO --> +To override syntax highlighting for a file type: -When you check in and push that change, all `*.pl` files in your project are highlighted as Prolog. +1. If a `.gitattributes` file does not exist in the root directory of your project, + create a blank file with this name. +1. For each file type you want to modify, add a line to the `.gitattributes` file + declaring the file extension and your desired highlighting language: -<!-- vale gitlab.Spelling = YES --> + ```conf + # This extension would normally receive Perl syntax highlighting + # but if we also use Prolog, we may want to override highlighting for + # files with this extension: + *.pl gitlab-language=prolog + ``` -The paths here are Git's built-in [`.gitattributes` interface](https://git-scm.com/docs/gitattributes). So, if you were to invent a file format called a `Nicefile` at the root of your project that used Ruby syntax, all you need is: +1. Commit, push, and merge your changes into your default branch. -``` conf -/Nicefile gitlab-language=ruby -``` +After the changes merge into your [default branch](repository/branches/default.md), +all `*.pl` files in your project are highlighted in your preferred language. -To disable highlighting entirely, use `gitlab-language=text`. Lots more fun shenanigans are available through common gateway interface (CGI) options, such as: +You can also extend the highlighting with common gateway interface (CGI) options, such as: ``` conf -# json with erb in it +# JSON file with .erb in it /my-cool-file gitlab-language=erb?parent=json -# an entire file of highlighting errors! +# An entire file of highlighting errors! /other-file gitlab-language=text?token=Error ``` -These configurations only take effect when the `.gitattributes` -file is in your [default branch](repository/branches/default.md). +## Disable syntax highlighting for a file type -NOTE: -The Web IDE does not support `.gitattribute` files, but it's [planned for a future release](https://gitlab.com/gitlab-org/gitlab/-/issues/22014). +To disable highlighting entirely for a file type, follow the instructions for overriding +the highlighting for a file type, and use `gitlab-language=text`: -## Configure maximum file size for highlighting +```conf +# Disable syntax highlighting for this file type +*.module gitlab-language=text +``` -You can configure the maximum size of the file to be highlighted. +## Configure maximum file size for highlighting -The file size is measured in kilobytes, and is set to a default of `512 KB`. Any file _over_ the file size is rendered in plain text. +By default, GitLab renders any file larger than 512 KB in plain text. To change this value: -1. Open the [`gitlab.yml`](https://gitlab.com/gitlab-org/gitlab-foss/blob/master/config/gitlab.yml.example) configuration file. +1. Open the [`gitlab.yml`](https://gitlab.com/gitlab-org/gitlab-foss/blob/master/config/gitlab.yml.example) + configuration file for your project. 1. Add this section, replacing `maximum_text_highlight_size_kilobytes` with the value you want. @@ -72,3 +78,5 @@ The file size is measured in kilobytes, and is set to a default of `512 KB`. Any ## https://docs.gitlab.com/ee/user/project/highlighting.html maximum_text_highlight_size_kilobytes: 512 ``` + +1. Commit, push, and merge your changes into your default branch. diff --git a/doc/user/project/img/labels_sort_label_priority.png b/doc/user/project/img/labels_sort_label_priority.png Binary files differdeleted file mode 100644 index faf629ac61d..00000000000 --- a/doc/user/project/img/labels_sort_label_priority.png +++ /dev/null diff --git a/doc/user/project/img/labels_sort_priority.png b/doc/user/project/img/labels_sort_priority.png Binary files differdeleted file mode 100644 index a6b5fca26f4..00000000000 --- a/doc/user/project/img/labels_sort_priority.png +++ /dev/null diff --git a/doc/user/project/img/labels_subscriptions_v13_5.png b/doc/user/project/img/labels_subscriptions_v13_5.png Binary files differdeleted file mode 100644 index a2a4e9e50cc..00000000000 --- a/doc/user/project/img/labels_subscriptions_v13_5.png +++ /dev/null diff --git a/doc/user/project/import/bitbucket.md b/doc/user/project/import/bitbucket.md index 62495872659..8c0d9fc422b 100644 --- a/doc/user/project/import/bitbucket.md +++ b/doc/user/project/import/bitbucket.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Import your project from Bitbucket Cloud to GitLab **(FREE)** NOTE: -The Bitbucket Cloud importer works only with Bitbucket.org, not with Bitbucket +The Bitbucket Cloud importer works only with [Bitbucket.org](https://bitbucket.org/), not with Bitbucket Server (aka Stash). If you are trying to import projects from Bitbucket Server, use [the Bitbucket Server importer](bitbucket_server.md). @@ -31,25 +31,37 @@ When importing: - Repository public access is retained. If a repository is private in Bitbucket, it's created as private in GitLab as well. -## Requirements +## Prerequisite for GitLab self-managed To import your projects from Bitbucket Cloud, the [Bitbucket Cloud integration](../../../integration/bitbucket.md) -must be enabled. Ask your GitLab administrator to enable this if it isn't already enabled. +must be enabled. If it isn't enabled, ask your GitLab administrator to enable it. By default it's +enabled on GitLab.com. ## How it works -When issues/pull requests are being imported, the Bitbucket importer tries to find -the Bitbucket author/assignee in the GitLab database using the Bitbucket `nickname`. -For this to work, the Bitbucket author/assignee should have signed in beforehand in GitLab -and **associated their Bitbucket account**. Their `nickname` must also match their Bitbucket -`username`. If the user is not found in the GitLab database, the project creator -(most of the times the current user that started the import process) is set as the author, -but a reference on the issue about the original Bitbucket author is kept. +When issues/pull requests are being imported, the Bitbucket importer uses the Bitbucket nickname of +the author/assignee and tries to find the same Bitbucket identity in GitLab. If they don't match or +the user is not found in the GitLab database, the project creator (most of the times the current +user that started the import process) is set as the author, but a reference on the issue about the +original Bitbucket author is kept. The importer will create any new namespaces (groups) if they don't exist or in the case the namespace is taken, the repository will be imported under the user's namespace that started the import process. +## Requirements for user-mapped contributions + +For user contributions to be mapped, each user must complete the following before the project import: + +1. Verify that the username in the [Bitbucket account settings](https://bitbucket.org/account/settings/) + matches the public name in the [Atlassian account settings](https://id.atlassian.com/manage-profile/profile-and-visibility). + If they don't match, modify the public name in the Atlassian account settings to match the + username in the Bitbucket account settings. + +1. Connect your Bitbucket account in [GitLab profile social sign-in](https://gitlab.com/-/profile/account). + +1. [Set your public email](../../profile/index.md#set-your-public-email). + ## Import your Bitbucket repositories 1. Sign in to GitLab. @@ -69,9 +81,35 @@ namespace that started the import process. ## Troubleshooting -If you have more than one Bitbucket account, be sure to sign in to the correct account. +### If you have more than one Bitbucket account + +Be sure to sign in to the correct account. + If you've accidentally started the import process with the wrong account, follow these steps: 1. Revoke GitLab access to your Bitbucket account, essentially reversing the process in the following procedure: [Import your Bitbucket repositories](#import-your-bitbucket-repositories). 1. Sign out of the Bitbucket account. Follow the procedure linked from the previous step. + +### User mapping fails despite matching names + +[For user mapping to work](#requirements-for-user-mapped-contributions), +the username in the Bitbucket account settings must match the public name in the Atlassian account +settings. If these names match but user mapping still fails, the user may have modified their +Bitbucket username after connecting their Bitbucket account in the +[GitLab profile social sign-in](https://gitlab.com/-/profile/account). + +To fix this, the user must verify that their Bitbucket external UID in the GitLab database matches their +current Bitbucket public name, and reconnect if there's a mismatch: + +1. [Use the API to get the currently authenticated user](../../../api/users.md#list-current-user-for-normal-users). + +1. In the API's response, the `identities` attribute contains the Bitbucket account that exists in + the GitLab database. If the `extern_uid` doesn't match the current Bitbucket public name, the + user should reconnect their Bitbucket account in the [GitLab profile social sign-in](https://gitlab.com/-/profile/account). + +1. Following reconnection, the user should use the API again to verify that their `extern_uid` in + the GitLab database now matches their current Bitbucket public name. + +The importer must then [delete the imported project](../../project/working_with_projects.md#delete-a-project) +and import again. diff --git a/doc/user/project/import/clearcase.md b/doc/user/project/import/clearcase.md index 120c64e00f2..867477c83b2 100644 --- a/doc/user/project/import/clearcase.md +++ b/doc/user/project/import/clearcase.md @@ -51,4 +51,4 @@ are some useful links to get you started: - [ClearCase to Git](https://therub.org/2013/07/19/clearcase-to-git/) - [Dual syncing ClearCase to Git](https://therub.org/2013/10/22/dual-syncing-clearcase-and-git/) - [Moving to Git from ClearCase](https://sateeshkumarb.wordpress.com/2011/01/15/moving-to-git-from-clearcase/) -- [ClearCase to Git webinar](https://www.brighttalk.com/webcast/11817/162473/clearcase-to-git) +- [ClearCase to Git webinar](https://www.brighttalk.com/webcast/11817/162473) diff --git a/doc/user/project/import/cvs.md b/doc/user/project/import/cvs.md index 55c5feff1f0..8bb716d8122 100644 --- a/doc/user/project/import/cvs.md +++ b/doc/user/project/import/cvs.md @@ -71,6 +71,6 @@ Here's a few links to get you started with the migration: - [Migrate using the `cvs-fast-export` tool](https://gitlab.com/esr/cvs-fast-export) - [Stack Overflow post on importing the CVS repository](https://stackoverflow.com/a/11490134/974710) -- [Convert a CVS repository to Git](https://www.techrepublic.com/blog/linux-and-open-source/convert-cvs-repositories-to-git/) +- [Convert a CVS repository to Git](http://www.techrepublic.com/article/convert-cvs-repositories-to-git/) - [Man page of the `git-cvsimport` tool](https://mirrors.edge.kernel.org/pub/software/scm/git/docs/git-cvsimport.html) - [Migrate using `reposurgeon`](http://www.catb.org/~esr/reposurgeon/repository-editing.html#conversion) diff --git a/doc/user/project/import/gitlab_com.md b/doc/user/project/import/gitlab_com.md index bcbc6e09f1b..6913cda0cd5 100644 --- a/doc/user/project/import/gitlab_com.md +++ b/doc/user/project/import/gitlab_com.md @@ -14,7 +14,7 @@ mind that it is possible only if GitLab.com integration is enabled on your GitLa To get to the importer page you need to go to "New project" page. NOTE: -If you are interested in importing Wiki and Merge Request data to your new instance, +If you are interested in importing Wiki and merge request data to your new instance, you'll need to follow the instructions for [exporting a project](../settings/import_export.md#export-a-project-and-its-data)  diff --git a/doc/user/project/import/jira.md b/doc/user/project/import/jira.md index 5f7475eac36..a44669e738e 100644 --- a/doc/user/project/import/jira.md +++ b/doc/user/project/import/jira.md @@ -32,8 +32,7 @@ imported into the GitLab issue's description as plain text. Our parser for converting text in Jira issues to GitLab Flavored Markdown is only compatible with Jira V3 REST API. -There is an [epic](https://gitlab.com/groups/gitlab-org/-/epics/2738) tracking the addition of -items, such as issue assignees, comments, and much more. These are included in the future +There is an [epic](https://gitlab.com/groups/gitlab-org/-/epics/2738) tracking the addition of issue assignees, comments, and much more in the future iterations of the GitLab Jira importer. ## Prerequisites @@ -60,7 +59,7 @@ Importing large projects may take several minutes depending on the size of the i To import Jira issues to a GitLab project: -1. On the **{issues}** **Issues** page, click **Import Issues** (**{import}**) **> Import from Jira**. +1. On the **{issues}** **Issues** page, select **Import Issues** (**{import}**) **> Import from Jira**.  @@ -68,20 +67,20 @@ To import Jira issues to a GitLab project: The following form appears. If you've previously set up the [Jira integration](../../../integration/jira/index.md), you can now see - the Jira projects that you have access to in the dropdown. + the Jira projects that you have access to in the dropdown list.  -1. Click the **Import from** dropdown and select the Jira project that you wish to import issues from. +1. Click the **Import from** dropdown list and select the Jira project that you wish to import issues from. In the **Jira-GitLab user mapping template** section, the table shows to which GitLab users your Jira users are mapped. - When the form appears, the dropdown defaults to the user conducting the import. + When the form appears, the dropdown list defaults to the user conducting the import. -1. To change any of the mappings, click the dropdown in the **GitLab username** column and +1. To change any of the mappings, click the dropdown list in the **GitLab username** column and select the user you want to map to each Jira user. - The dropdown may not show all the users, so use the search bar to find a specific + The dropdown list may not show all the users, so use the search bar to find a specific user in this GitLab project. 1. Click **Continue**. You're presented with a confirmation that import has started. diff --git a/doc/user/project/integrations/custom_issue_tracker.md b/doc/user/project/integrations/custom_issue_tracker.md index d155f91e40b..71ca9f4f640 100644 --- a/doc/user/project/integrations/custom_issue_tracker.md +++ b/doc/user/project/integrations/custom_issue_tracker.md @@ -8,7 +8,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w You can integrate an [external issue tracker](../../../integration/external-issue-tracker.md) with GitLab. If your preferred issue tracker is not listed in the -[integrations list](../../../integration/external-issue-tracker.md#integration), +[integrations list](../../../integration/external-issue-tracker.md#configure-an-external-issue-tracker), you can enable a custom issue tracker. After you enable the custom issue tracker, a link to the issue tracker displays diff --git a/doc/user/project/integrations/gitlab_slack_application.md b/doc/user/project/integrations/gitlab_slack_application.md index 0a685ad0efb..7e8de776619 100644 --- a/doc/user/project/integrations/gitlab_slack_application.md +++ b/doc/user/project/integrations/gitlab_slack_application.md @@ -32,15 +32,15 @@ where you can select a project to enable the GitLab Slack application for. Alternatively, you can configure the Slack application with a project's integration settings. -Keep in mind that you need to have the appropriate permissions for your Slack -team in order to be able to install a new application, read more in Slack's -docs on [Adding an app to your workspace](https://slack.com/help/articles/202035138-Add-apps-to-your-Slack-workspace). +Keep in mind that you must have the appropriate permissions for your Slack +team to be able to install a new application, read more in Slack's +documentation on [Adding an app to your workspace](https://slack.com/help/articles/202035138-Add-apps-to-your-Slack-workspace). To enable the GitLab service for your Slack team: 1. Go to your project's **Settings > Integration > Slack application** (only visible on GitLab.com). -1. Click **Add to Slack**. +1. Select **Add to Slack**. That's all! You can now start using the Slack slash commands. @@ -49,11 +49,11 @@ That's all! You can now start using the Slack slash commands. To create a project alias on GitLab.com for Slack integration: 1. Go to your project's home page. -1. Navigate to **Settings > Integrations** (only visible on GitLab.com) -1. On the **Integrations** page, click **Slack application**. +1. Go to **Settings > Integrations** (only visible on GitLab.com) +1. On the **Integrations** page, select **Slack application**. 1. The current **Project Alias**, if any, is displayed. To edit this value, - click **Edit**. -1. Enter your desired alias, and click **Save changes**. + select **Edit**. +1. Enter your desired alias, and select **Save changes**. Some Slack commands require a project alias, and fail with the following error if the project alias is incorrect or missing from the command: diff --git a/doc/user/project/integrations/harbor.md b/doc/user/project/integrations/harbor.md new file mode 100644 index 00000000000..d66e2222538 --- /dev/null +++ b/doc/user/project/integrations/harbor.md @@ -0,0 +1,50 @@ +--- +stage: Ecosystem +group: Integrations +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments +--- + +# Harbor container registry integration **(FREE)** + +Use Harbor as the container registry for your GitLab project. + +[Harbor](https://goharbor.io/) is an open source registry that can help you manage artifacts across cloud native compute platforms, like Kubernetes and Docker. + +This integration can help you if you need GitLab CI/CD and a container image repository. + +## Prerequisites + +In the Harbor instance, ensure that: + +- The project to be integrated has been created. +- The signed-in user has permission to pull, push, and edit images in the Harbor project. + +## Configure GitLab + +GitLab supports integrating Harbor projects at the group or project level. Complete these steps in GitLab: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Settings > Integrations**. +1. Select **Harbor**. +1. Turn on the **Active** toggle under **Enable Integration**. +1. Provide the Harbor configuration information: + - **Harbor URL**: The base URL of Harbor instance which is being linked to this GitLab project. For example, `https://harbor.example.net`. + - **Harbor project name**: The project name in the Harbor instance. For example, `testproject`. + - **Username**: Your username in the Harbor instance, which should meet the requirements in [prerequisites](#prerequisites). + - **Password**: Password of your username. + +1. Select **Save changes**. + +After the Harbor integration is activated: + +- The global variables `$HARBOR_USER`, `$HARBOR_PASSWORD`, `$HARBOR_URL`, and `$HARBOR_PROJECT` are created for CI/CD use. +- The project-level integration settings override the group-level integration settings. + +## Secure your requests to the Harbor APIs + +For each API request through the Harbor integration, the credentials for your connection to the Harbor API use +the `username:password` combination. The following are suggestions for safe use: + +- Use TLS on the Harbor APIs you connect to. +- Follow the principle of least privilege (for access on Harbor) with your credentials. +- Have a rotation policy on your credentials. diff --git a/doc/user/project/integrations/img/failed_badges.png b/doc/user/project/integrations/img/failed_badges.png Binary files differnew file mode 100644 index 00000000000..d44415a8687 --- /dev/null +++ b/doc/user/project/integrations/img/failed_badges.png diff --git a/doc/user/project/integrations/img/failed_banner.png b/doc/user/project/integrations/img/failed_banner.png Binary files differnew file mode 100644 index 00000000000..ba40c1301d6 --- /dev/null +++ b/doc/user/project/integrations/img/failed_banner.png diff --git a/doc/user/project/integrations/img/failed_banner_error.png b/doc/user/project/integrations/img/failed_banner_error.png Binary files differnew file mode 100644 index 00000000000..8f4dabbf8c2 --- /dev/null +++ b/doc/user/project/integrations/img/failed_banner_error.png diff --git a/doc/user/project/integrations/mattermost_slash_commands.md b/doc/user/project/integrations/mattermost_slash_commands.md index b317e65bdf2..768acb02ee6 100644 --- a/doc/user/project/integrations/mattermost_slash_commands.md +++ b/doc/user/project/integrations/mattermost_slash_commands.md @@ -32,7 +32,7 @@ on your configuration: If Mattermost is installed on the same server as GitLab, the configuration process can be done for you by GitLab. -Go to the Mattermost Slash Command service on your project and click **Add to Mattermost** button. +Go to the Mattermost Slash Command service on your project and select **Add to Mattermost**. ## Manual configuration @@ -52,13 +52,13 @@ installations from source. To enable custom slash commands from the Mattermost administrator console: 1. Sign in to Mattermost as a user with administrator privileges. -1. Next to your username, click the **{ellipsis_v}** **Settings** icon, and +1. Next to your username, select the **{ellipsis_v}** **Settings** icon, and select **System Console**. 1. Select **Integration Management**, and set these values to `TRUE`: - **Enable Custom Slash Commands** - **Enable integrations to override usernames** - **Enable integrations to override profile picture icons** -1. Click **Save**, but do not close this browser tab, because you need it in +1. Select **Save**, but do not close this browser tab, because you need it in a later step. ### Get configuration values from GitLab @@ -85,9 +85,9 @@ the previous step: 1. In the Mattermost tab you left open when you [enabled custom slash commands](#enable-custom-slash-commands), go to your team page. -1. Click the **{ellipsis_v}** **Settings** icon, and select **Integrations**. +1. Select the **{ellipsis_v}** **Settings** icon, and select **Integrations**. 1. In the left menu, select **Slash commands**. -1. Click **Add Slash Command**: +1. Select **Add Slash Command**:  1. Provide a **Display Name** and **Description** for your new command. @@ -101,7 +101,7 @@ the previous step: [viewed configuration values](#get-configuration-values-from-gitlab). 1. For all other values, you may use the suggestions from GitLab or use your preferred values. -1. Copy the **Token** value, as you need it in a later step, and click **Done**. +1. Copy the **Token** value, as you need it in a later step, and select **Done**. ### Provide the Mattermost token to GitLab @@ -116,7 +116,7 @@ provide to GitLab:  -1. Click **Save changes** for the changes to take effect. +1. Select **Save changes** for the changes to take effect. Your slash command can now communicate with your GitLab project. @@ -165,4 +165,4 @@ different types of notifications. All events are sent to the specified channel. ## Further reading - [Mattermost slash commands documentation](https://docs.mattermost.com/developer/slash-commands.html) -- [Omnibus GitLab Mattermost](https://docs.gitlab.com/omnibus/gitlab-mattermost/) +- [Omnibus GitLab Mattermost](../../../integration/mattermost/) diff --git a/doc/user/project/integrations/overview.md b/doc/user/project/integrations/overview.md index 8ecc16050be..2cb62b8924e 100644 --- a/doc/user/project/integrations/overview.md +++ b/doc/user/project/integrations/overview.md @@ -43,6 +43,7 @@ Click on the service links to see further configuration instructions and details | [Flowdock](../../../api/integrations.md#flowdock) | Send notifications from GitLab to Flowdock flows. | **{dotted-circle}** No | | [GitHub](github.md) | Obtain statuses for commits and pull requests. | **{dotted-circle}** No | | [Google Chat](hangouts_chat.md) | Send notifications from your GitLab project to a room in Google Chat.| **{dotted-circle}** No | +| [Harbor](harbor.md) | Use Harbor as the container registry. | **{dotted-circle}** No | | [irker (IRC gateway)](irker.md) | Send IRC messages. | **{dotted-circle}** No | | [Jenkins](../../../integration/jenkins.md) | Run CI/CD pipelines with Jenkins. | **{check-circle}** Yes | | JetBrains TeamCity CI | Run CI/CD pipelines with TeamCity. | **{check-circle}** Yes | diff --git a/doc/user/project/integrations/webhook_events.md b/doc/user/project/integrations/webhook_events.md index 9d8b98edba1..d37196ec114 100644 --- a/doc/user/project/integrations/webhook_events.md +++ b/doc/user/project/integrations/webhook_events.md @@ -202,6 +202,10 @@ The available values for `object_attributes.action` in the payload are: The `assignee` and `assignee_id` keys are deprecated and contain the first assignee only. +The `escalation_status` and `escalation_policy` fields are +only available for issue types which support escalations, +such as incidents. + Request header: ```plaintext @@ -271,6 +275,12 @@ Payload example: "url": "http://example.com/diaspora/issues/23", "state": "opened", "action": "open", + "severity": "high", + "escalation_status": "triggered", + "escalation_policy": { + "id": 18, + "name": "Engineering On-call" + }, "labels": [{ "id": 206, "title": "API", @@ -771,7 +781,8 @@ Payload example: Merge request events are triggered when: - A new merge request is created. -- An existing merge request is updated, approved, unapproved, merged, or closed. +- An existing merge request is updated, approved (by all required approvers), unapproved, merged, or closed. +- An individual user adds or removes their approval to an existing merge request. - A commit is added in the source branch. - All threads are resolved on the merge request. @@ -783,6 +794,8 @@ The available values for `object_attributes.action` in the payload are: - `update` - `approved` - `unapproved` +- `approval` +- `unapproval` - `merge` Request header: @@ -1294,7 +1307,7 @@ Payload example: "id": 3, "name": "User", "email": "user@gitlab.com", - "avatar_url": "http://www.gravatar.com/avatar/e32bd13e2add097461cb96824b7a829c?s=80\u0026d=identicon", + "avatar_url": "http://www.gravatar.com/avatar/e32bd13e2add097461cb96824b7a829c?s=80\u0026d=identicon" }, "commit": { "id": 2366, diff --git a/doc/user/project/integrations/webhooks.md b/doc/user/project/integrations/webhooks.md index ace5783c852..e823391401d 100644 --- a/doc/user/project/integrations/webhooks.md +++ b/doc/user/project/integrations/webhooks.md @@ -62,7 +62,8 @@ You can configure a webhook for a group or a project. ## Test a webhook -You can trigger a webhook manually, to ensure it's working properly. +You can trigger a webhook manually, to ensure it's working properly. You can also send +a test request to re-enable a [disabled webhook](#re-enable-disabled-webhooks). For example, to test `push events`, your project should have at least one commit. The webhook uses this commit in the webhook. @@ -72,6 +73,8 @@ To test a webhook: 1. Scroll down to the list of configured webhooks. 1. From the **Test** dropdown list, select the type of event to test. +You can also test a webhook from its edit page. +  ## Create an example webhook receiver @@ -143,7 +146,32 @@ GitLab webhooks, keep in mind the following: GitLab assumes the hook failed and retries it. Most HTTP libraries take care of the response for you automatically but if you are writing a low-level hook, this is important to remember. -- GitLab ignores the HTTP status code returned by your endpoint. +- GitLab usually ignores the HTTP status code returned by your endpoint, + unless the [`web_hooks_disable_failed` feature flag is set](#failing-webhooks). + +### Failing webhooks + +> - Introduced in GitLab 13.12 [with a flag](../../../administration/feature_flags.md) named `web_hooks_disable_failed`. Disabled by default. +> - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/329849) in GitLab 14.9. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available, +ask an administrator to [enable the feature flag](../../../administration/feature_flags.md) named `web_hooks_disable_failed`. +The feature is not ready for production use. + +If a webhook fails repeatedly, it may be disabled automatically. + +Webhooks that return response codes in the `5xx` range are understood to be failing +intermittently, and are temporarily disabled. This lasts initially +for 10 minutes. If the hook continues to fail, the back-off period is +extended on each retry, up to a maximum disabled period of 24 hours. + +Webhooks that return failure codes in the `4xx` range are understood to be +misconfigured, and these are disabled until you manually re-enable +them. These webhooks are not automatically retried. + +See [troubleshooting](#troubleshoot-webhooks) for information on +how to see if a webhook is disabled, and how to re-enable it. ## How image URLs are displayed in the webhook body @@ -186,6 +214,13 @@ To view the table: 1. In your project, on the left sidebar, select **Settings > Webhooks**. 1. Scroll down to the webhooks. +1. Each [failing webhook](#failing-webhooks) has a badge listing it as: + + - **Failed to connect** if it is misconfigured, and needs manual intervention to re-enable it. + - **Fails to connect** if it is temporarily disabled and will retry later. + +  + 1. Select **Edit** for the webhook you want to view. The table includes the following details about each request: @@ -216,8 +251,8 @@ If the endpoint doesn't send an HTTP response in those 10 seconds, GitLab may assume the webhook failed and retry it. If your webhooks are failing or you are receiving multiple requests, -you can try changing the default timeout value. -In your `/etc/gitlab/gitlab.rb` file, uncomment or add the following setting: +an administrator can try changing the default timeout value +by uncommenting or adding the following setting in `/etc/gitlab/gitlab.rb`: ```ruby gitlab_rails['webhook_timeout'] = 10 @@ -233,3 +268,17 @@ determined by [CAcert.org](http://www.cacert.org/). If that is not the case, consider using [SSL Checker](https://www.sslshopper.com/ssl-checker.html) to identify faults. Missing intermediate certificates are common causes of verification failure. + +### Re-enable disabled webhooks + +If a webhook is failing, a banner displays at the top of the edit page explaining +why it is disabled, and when it will be automatically re-enabled. For example: + + + +In the case of a failed webhook, an error banner is displayed: + + + +To re-enable a failing or failed webhook, [send a test request](#test-a-webhook). If the test +request succeeds, the webhook is re-enabled. diff --git a/doc/user/project/issue_board.md b/doc/user/project/issue_board.md index 71440298d85..d731ceb5aca 100644 --- a/doc/user/project/issue_board.md +++ b/doc/user/project/issue_board.md @@ -53,7 +53,7 @@ the issue board feature. > - Multiple issue boards per project [moved](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/53811) to GitLab Free in 12.1. > - Multiple issue boards per group are available in GitLab Premium. -Multiple issue boards allow for more than one issue board for a given project **(FREE)** or group **(PREMIUM)**. +Multiple issue boards allow for more than one issue board for a given project in GitLab Free or group in GitLab Premium and higher tiers. This is great for large projects with more than one team or when a repository hosts the code of multiple products. Using the search box at the top of the menu, you can filter the listed boards. @@ -275,7 +275,7 @@ Users on GitLab Free can use a single group issue board. ### Assignee lists **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5784) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.0. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5784) in GitLab 11.0. As in a regular list showing all issues with a chosen label, you can add an assignee list that shows all issues assigned to a user. @@ -295,7 +295,7 @@ To remove an assignee list, just as with a label list, click the trash icon. ### Milestone lists **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/6469) in [GitLab Premium](https://about.gitlab.com/pricing/) 11.2. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/6469) in GitLab 11.2. You're also able to create lists of a milestone. These are lists that filter issues by the assigned milestone, giving you more freedom and visibility on the issue board. To add a milestone list: @@ -333,7 +333,7 @@ to and from a iteration list to manipulate the iteration of the dragged issues. ### Group issues in swimlanes **(PREMIUM)** -> - Grouping by epic [introduced](https://gitlab.com/groups/gitlab-org/-/epics/3352) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.6. +> - Grouping by epic [introduced](https://gitlab.com/groups/gitlab-org/-/epics/3352) in GitLab 13.6. > - Editing issue titles in the issue sidebar [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/232745) in GitLab 13.8. > - Editing iteration in the issue sidebar [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/290232) in GitLab 13.9. @@ -356,7 +356,7 @@ appears on the right. There you can see and edit the issue's: - Title - Assignees -- Epic **(PREMIUM)** +- [Epic](../group/epics/index.md) - Milestone - Time tracking value (view only) - Due date @@ -506,14 +506,14 @@ You can filter by the following: - Assignee - Author -- Epic -- Iteration +- [Epic](../group/epics/index.md) +- [Iteration](../group/iterations/index.md) - Label - Milestone - My Reaction - Release - Type (issue/incident) -- Weight +- [Weight](issues/issue_weight.md) #### Filtering issues in a group board @@ -536,7 +536,7 @@ changing a label. A typical workflow of using an issue board would be: -1. You have [created](labels.md#label-management) and [prioritized](labels.md#label-priority) +1. You [create](labels.md#create-a-label) and [prioritize](labels.md#set-label-priority) labels to categorize your issues. 1. You have a bunch of issues (ideally labeled). 1. You visit the issue board and start [creating lists](#create-a-new-list) to diff --git a/doc/user/project/issues/csv_export.md b/doc/user/project/issues/csv_export.md index 947fbdcc2d1..e5d698fa97a 100644 --- a/doc/user/project/issues/csv_export.md +++ b/doc/user/project/issues/csv_export.md @@ -52,7 +52,7 @@ export, after which the email is prepared. ## Sorting -Exported issues are always sorted by `Issue ID`. +Exported issues are always sorted by `Title`. ## Format @@ -63,11 +63,11 @@ the values: | Column | Description | |------------------------------------------|-----------------------------------------------------------| +| Title | Issue `title` | +| Description | Issue `description` | | Issue ID | Issue `iid` | | URL | A link to the issue on GitLab | -| Title | Issue `title` | | State | `Open` or `Closed` | -| Description | Issue `description` | | Author | Full name of the issue author | | Author Username | Username of the author, with the `@` symbol omitted | | Assignee | Full name of the issue assignee | @@ -85,6 +85,10 @@ the values: | [Epic](../../group/epics/index.md) ID | ID of the parent epic, introduced in 12.7 | | [Epic](../../group/epics/index.md) Title | Title of the parent epic, introduced in 12.7 | +In GitLab 14.7 and earlier, the first two columns were `Issue ID` and `URL`, +which [caused an issue](https://gitlab.com/gitlab-org/gitlab/-/issues/34769) +when importing back into GitLab. + ## Limitations - Export Issues to CSV is not available at the Group's Issues List. diff --git a/doc/user/project/issues/issue_data_and_actions.md b/doc/user/project/issues/issue_data_and_actions.md index 66ca29c094e..e9f3f4be1c3 100644 --- a/doc/user/project/issues/issue_data_and_actions.md +++ b/doc/user/project/issues/issue_data_and_actions.md @@ -6,4 +6,6 @@ remove_date: '2022-02-24' This file was moved to [another location](index.md). <!-- This redirect file can be deleted after <2022-02-24>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/project/issues/managing_issues.md b/doc/user/project/issues/managing_issues.md index 155d6260a5c..58591129d97 100644 --- a/doc/user/project/issues/managing_issues.md +++ b/doc/user/project/issues/managing_issues.md @@ -19,7 +19,7 @@ You can create an issue in many ways in GitLab: - [From a project](#from-a-project) - [From a group](#from-a-group) -- [From another issue](#from-another-issue) +- [From another issue or incident](#from-another-issue-or-incident) - [From an issue board](#from-an-issue-board) - [By sending an email](#by-sending-an-email) - [Using a URL with prefilled values](#using-a-url-with-prefilled-values) @@ -70,9 +70,10 @@ The newly created issue opens. The project you selected most recently becomes the default for your next visit. This can save you a lot of time and clicks, if you mostly create issues for the same project. -### From another issue +### From another issue or incident -> New issue becoming linked to the issue of origin [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/68226) in GitLab 14.3. +> - New issue becoming linked to the issue of origin [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/68226) in GitLab 14.3. +> - **Relate to…** checkbox [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/198494) in GitLab 14.9. You can create a new issue from an existing one. The two issues can then be marked as related. @@ -83,10 +84,10 @@ Prerequisites: To create an issue from another issue: 1. In an existing issue, select the vertical ellipsis (**{ellipsis_v}**). -1. Select **New issue**. +1. Select **New related issue**. 1. Complete the [fields](#fields-in-the-new-issue-form). - The new issue's description is prefilled with `Related to #123`, where `123` is the ID of the - issue of origin. If you keep this mention in the description, the two issues become + The new issue form has a **Relate to issue #123** checkbox, where `123` is the ID of the + issue of origin. If you keep this checkbox checked, the two issues become [linked](related_issues.md). 1. Select **Create issue**. @@ -160,6 +161,9 @@ To regenerate the email address: ### Using a URL with prefilled values +> - Ability to use both `issuable_template` and `issue[description]` in the same URL [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/80554) in GitLab 14.9. +> - Ability to specify `add_related_issue` [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/198494) in GitLab 14.9. + To link directly to the new issue page with prefilled fields, use query string parameters in a URL. You can embed a URL in an external HTML page to create issues with certain fields prefilled. @@ -168,9 +172,10 @@ HTML page to create issues with certain fields prefilled. | -------------------- | --------------------- | ------------------------------------------------------------------------------------------------------------------------------- | | Title | `issue[title]` | Must be [URL-encoded](../../../api/index.md#namespaced-path-encoding). | | Issue type | `issue[issue_type]` | Either `incident` or `issue`. | -| Description template | `issuable_template` | Cannot be used at the same time as `issue[description]`. Must be [URL-encoded](../../../api/index.md#namespaced-path-encoding). | -| Description | `issue[description]` | Cannot be used at the same time as `issuable_template`. Must be [URL-encoded](../../../api/index.md#namespaced-path-encoding). | +| Description template | `issuable_template` | Must be [URL-encoded](../../../api/index.md#namespaced-path-encoding). | +| Description | `issue[description]` | Must be [URL-encoded](../../../api/index.md#namespaced-path-encoding). If used in combination with `issuable_template` or a [default issue template](../description_templates.md#set-a-default-template-for-merge-requests-and-issues), the `issue[description]` value is appended to the template. | | Confidential | `issue[confidential]` | If `true`, the issue is marked as confidential. | +| Relate to… | `add_related_issue` | A numeric issue ID. If present, the issue form shows a [**Relate to…** checkbox](#from-another-issue-or-incident) to optionally link the new issue to the specified existing issue. | Adapt these examples to form your new issue URL with prefilled fields. To create an issue in the GitLab project: @@ -264,7 +269,7 @@ When bulk editing issues in a project, you can edit the following attributes: - [Notification](../../profile/notifications.md) subscription - [Iteration](../../group/iterations/index.md) -### Bulk edit issues from a group +### Bulk edit issues from a group **(PREMIUM)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7249) in GitLab 12.1. > - Assigning epic [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/210470) in GitLab 13.2. @@ -573,13 +578,11 @@ To copy the issue's email address: > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/17589) in GitLab 13.3. Disabled by default. > - [Enabled on GitLab.com](https://gitlab.com/gitlab-com/gl-infra/production/-/issues/3413) in GitLab 13.9. > - [Enabled on self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/17589) in GitLab 14.5. - -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature per project or for your entire instance, ask an administrator to -[disable the feature flags](../../../administration/feature_flags.md) named `real_time_issue_sidebar` and `broadcast_issue_updates`. -On GitLab.com, this feature is available. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/17589) in GitLab 14.9. Feature flags `real_time_issue_sidebar` and `broadcast_issue_updates` removed. Assignees in the sidebar are updated in real time. +When you're viewing an issue and somebody changes its assignee, +you can see the change without having to refresh the page. ## Assignee diff --git a/doc/user/project/issues/multiple_assignees_for_issues.md b/doc/user/project/issues/multiple_assignees_for_issues.md index f957d701a3b..279f297d016 100644 --- a/doc/user/project/issues/multiple_assignees_for_issues.md +++ b/doc/user/project/issues/multiple_assignees_for_issues.md @@ -39,4 +39,4 @@ to assign the issue to.  -To remove an assignee, deselect them from the same dropdown menu. +To remove an assignee, clear them from the same dropdown menu. diff --git a/doc/user/project/issues/related_issues.md b/doc/user/project/issues/related_issues.md index f83ebc5e8a8..b8151ac873a 100644 --- a/doc/user/project/issues/related_issues.md +++ b/doc/user/project/issues/related_issues.md @@ -75,4 +75,9 @@ Access our [permissions](../../permissions.md) page for more information. When you [add a linked issue](#add-a-linked-issue), you can show that it **blocks** or **is blocked by** another issue. -Issues that block other issues have an icon (**{issue-block}**) shown in the issue lists and [boards](../issue_board.md). +Issues that block other issues have an icon (**{issue-block}**) next to their title, shown in the +issue lists and [boards](../issue_board.md). +The icon disappears when the blocking issue is closed or their relationship is changed or +[removed](#remove-a-linked-issue). + +If you try to close a blocked issue using the "Close issue" button, a confirmation message appears. diff --git a/doc/user/project/issues/sorting_issue_lists.md b/doc/user/project/issues/sorting_issue_lists.md index 329f65bfacd..9311ef590df 100644 --- a/doc/user/project/issues/sorting_issue_lists.md +++ b/doc/user/project/issues/sorting_issue_lists.md @@ -49,7 +49,7 @@ Ties are broken arbitrarily. Only the highest prioritized label is checked, and labels with a lower priority are ignored. For more information, see [issue 14523](https://gitlab.com/gitlab-org/gitlab/-/issues/14523). -To learn more about priority labels, read the [Labels](../labels.md#label-priority) documentation. +To learn how to change label priority, see [Label priority](../labels.md#set-label-priority). ## Sorting by last updated @@ -98,7 +98,9 @@ When you sort by **Priority**, the issue order changes to sort in this order: 1. Issues with a higher priority label. 1. Issues without a prioritized label. -To learn more about priority, read the [Labels](../labels.md#label-priority) documentation. +Ties are broken arbitrarily. + +To learn how to change label priority, see [Label priority](../labels.md#set-label-priority). ## Sorting by title diff --git a/doc/user/project/labels.md b/doc/user/project/labels.md index 7ccc39eeb8b..18197cd860f 100644 --- a/doc/user/project/labels.md +++ b/doc/user/project/labels.md @@ -6,145 +6,270 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Labels **(FREE)** -As your count of issues, merge requests, and epics grows in GitLab, it's more and more challenging +As your count of issues, merge requests, and epics grows in GitLab, it gets more challenging to keep track of those items. Especially as your organization grows from just a few people to -hundreds or thousands. This is where labels come in. They help you organize and tag your work -so you can track and find the work items you're interested in. +hundreds or thousands. With labels, you can organize and tag your work, and track the work items +you're interested in. Labels are a key part of [issue boards](issue_board.md). With labels you can: -- Categorize epics, issues, and merge requests using colors and descriptive titles like +- Categorize [epics](../group/epics/index.md), issues, and merge requests using colors and descriptive titles like `bug`, `feature request`, or `docs`. -- Dynamically filter and manage epics, issues, and merge requests. +- Dynamically filter and manage [epics](../group/epics/index.md), issues, and merge requests. - [Search lists of issues, merge requests, and epics](../search/index.md#search-issues-and-merge-requests), as well as [issue boards](../search/index.md#issue-boards). -## Project labels and group labels +## Types of labels -There are two types of labels in GitLab: +You can use two types of labels in GitLab: - **Project labels** can be assigned to issues and merge requests in that project only. -- **Group labels** can be assigned to issues and merge requests in any project in - the selected group or its subgroups. - - They can also be assigned to epics in the selected group or its subgroups.**(ULTIMATE)** +- **Group labels** can be assigned to issues, merge requests, and [epics](../group/epics/index.md) + in any project in the selected group or its subgroups. ## Assign and unassign labels > Unassigning labels with the **X** button [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/216881) in GitLab 13.5. -Every issue, merge request, and epic can be assigned any number of labels. The labels are -managed in the right sidebar, where you can assign or unassign labels as needed. +You can assign labels to any issue, merge request, or epic. To assign or unassign a label: -1. In the **Labels** section of the sidebar, click **Edit**. +1. In the **Labels** section of the sidebar, select **Edit**. 1. In the **Assign labels** list, search for labels by typing their names. You can search repeatedly to add more labels. The selected labels are marked with a checkmark. -1. Click the labels you want to assign or unassign. +1. Select the labels you want to assign or unassign. 1. To apply your changes to labels, select **X** next to **Assign labels** or select any area outside the label section. -Alternatively, to unassign a label, click the **X** on the label you want to unassign. +Alternatively, to unassign a label, select the **X** on the label you want to unassign. -You can also assign a label with the `/label` [quick action](quick_actions.md), -remove labels with `/unlabel`, and reassign labels (remove all and assign new ones) with `/relabel`. +You can also assign and unassign labels with [quick actions](quick_actions.md): -## Label management +- Assign labels with `/label`. +- Remove labels with `/unlabel`. +- Remove all labels and assign new ones with `/relabel`. -Users with a [permission level](../permissions.md) of Reporter or higher are able to create -and edit labels. +## View available labels -### Project labels +### View project labels -> Showing all inherited labels [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/241990) in GitLab 13.5. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/241990) in GitLab 13.5: the label list in a project also shows all inherited labels. -To view a project's available labels, in the project, go to **Project information > Labels**. -Its list of labels includes both the labels defined at the project level, and -all labels defined by its ancestor groups. For each label, you can see the -project or group path from where it was created. You can filter the list by -entering a search query in the **Filter** field, and then clicking its search -icon (**{search}**). +To view the **project's labels**: -To create a new project label: +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Project information > Labels**. -1. In your project, go to **Project information > Labels**. -1. Select the **New label** button. +Or: + +1. View an issue or merge request. +1. On the right sidebar, in the **Labels** section, select **Edit**. +1. Select **Manage project labels**. + +The list of labels includes both the labels created in the project and +all labels created in the project's ancestor groups. For each label, you can see the +project or group path where it was created. + +### View group labels + +To view the **group's labels**: + +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Group information > Labels**. + +Or: + +1. View an epic. +1. On the right sidebar, in the **Labels** section, select **Edit**. +1. Select **Manage group labels**. + +The list includes all labels created only in the group. It does not list any labels created in +the group's projects. + +## Create a label + +Prerequisites: + +- You must have at least the Reporter role for the project or group. + +### Create a project label + +To create a project label: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Project information > Labels**. +1. Select **New label**. +1. In the **Title** field, enter a short, descriptive name for the label. You + can also use this field to create [scoped, mutually exclusive labels](#scoped-labels). +1. Optional. In the **Description** field, enter additional + information about how and when to use this label. +1. Optional. Select a color by selecting from the available colors, or enter a hex color value for + a specific color in the **Background color** field. +1. Select **Create label**. + +### Create a project label from an issue or merge request + +You can also create a new project label from an issue or merge request. +Labels you create this way belong to the same project as the issue or merge request. + +Prerequisites: + +- You must have at least the Reporter role for the project. + +To do so: + +1. View an issue or merge request. +1. On the right sidebar, in the **Labels** section, select **Edit**. +1. Select **Create project label**. +1. Fill in the name field. You can't specify a description if creating a label this way. + You can add a description later by [editing the label](#edit-a-label). +1. Optional. Select a color by selecting from the available colors, or enter a hex color value for + a specific color. +1. Select **Create**. + +### Create a group label + +To create a group label: + +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Group information > Labels**. +1. Select **New label**. 1. In the **Title** field, enter a short, descriptive name for the label. You can also use this field to create [scoped, mutually exclusive labels](#scoped-labels). -1. Optional. In the **Description** field, you can enter additional +1. Optional. In the **Description** field, enter additional information about how and when to use this label. -1. Optional. Select a background color for the label by selecting one of the - available colors, or by entering a hex color value in the **Background color** - field. +1. Optional. Select a color by selecting from the available colors, or enter a hex color value for + a specific color in the **Background color** field. 1. Select **Create label**. -You can also create a new project label from within an issue or merge request. In the -label section of the right sidebar of an issue or a merge request: +### Create a group label from an epic **(PREMIUM)** + +You can also create a new group label from an epic. +Labels you create this way belong to the same group as the epic. + +Prerequisites: + +- You must have at least the Reporter role for the group. -1. Click **Edit**. -1. Click **Create project label**. - - Fill in the name field. Note that you can't specify a description if creating a label - this way. You can add a description later by editing the label (see below). - - Optional. Select a color by clicking on the available colors, or input a hex - color value for a specific color. -1. Click **Create**. +To do so: -To edit a label after you create it, select (**{pencil}**). +1. View an epic. +1. On the right sidebar, in the **Labels** section, select **Edit**. +1. Select **Create group label**. +1. Fill in the name field. You can't specify a description if creating a label this way. + You can add a description later by [editing the label](#edit-a-label). +1. Optional. Select a color by selecting from the available colors,enter input a hex color value + for a specific color. +1. Select **Create**. -To delete a project label, select (**{ellipsis_v}**) next to the **Subscribe** button -and select **Delete** or select **Delete** when you edit a label. +## Edit a label + +Prerequisites: + +- You must have at least the Reporter role for the project or group. + +### Edit a project label + +To edit a **project** label: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Project information > Labels**. +1. Next to the label you want to edit, select **Edit** (**{pencil}**). + +### Edit a group label + +To edit a **group** label: + +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Group information > Labels**. +1. Next to the label you want to edit, select **Edit** (**{pencil}**). + +## Delete a label WARNING: -If you delete a label, it is permanently deleted. All references to the label are removed from the system and you cannot undo the deletion. +If you delete a label, it is permanently deleted. All references to the label are removed from the +system and you cannot undo the deletion. + +Prerequisites: -#### Promote a project label to a group label +- You must have at least the Reporter role for the project. + +### Delete a project label + +To delete a **project** label: + +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Project information > Labels**. +1. Either: + + - Next to the **Subscribe** button, select (**{ellipsis_v}**). + - Next to the label you want to edit, select **Edit** (**{pencil}**). + +1. Select **Delete**. + +### Delete a group label + +To delete a **group** label: + +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Group information > Labels**. +1. Either: + + - Next to the **Subscribe** button, select (**{ellipsis_v}**). + - Next to the label you want to edit, select **Edit** (**{pencil}**). + +1. Select **Delete**. + +## Promote a project label to a group label > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/231472) in GitLab 13.6: promoting a project label keeps that label's ID and changes it into a group label. Previously, promoting a project label created a new group label with a new ID and deleted the old label. -If you previously created a project label and now want to make it available for other -projects within the same group, you can promote it to a group label. +You might want to make a project label available for other +projects in the same group. Then, you can promote the label to a group label. If other projects in the same group have a label with the same title, they are all merged with the new group label. If a group label with the same title exists, it is also merged. -All issues, merge requests, issue board lists, issue board filters, and label subscriptions -with the old labels are assigned to the new group label. +WARNING: +Promoting a label is a permanent action and cannot be reversed. -The new group label has the same ID as the previous project label. +Prerequisites: -WARNING: -Promoting a label is a permanent action, and cannot be reversed. +- You must have at least the Reporter role for the project. +- You must have at least the Reporter role for the project's parent group. To promote a project label to a group label: -1. Navigate to **Project information > Labels** in the project. -1. Click on the three dots (**{ellipsis_v}**) next to the **Subscribe** button and +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Project information > Labels**. +1. Next to the **Subscribe** button, select the three dots (**{ellipsis_v}**) and select **Promote to group label**. -### Group labels +All issues, merge requests, issue board lists, issue board filters, and label subscriptions +with the old labels are assigned to the new group label. + +The new group label has the same ID as the previous project label. -To view the group labels list, navigate to the group and click **Group information > Labels**. -The list includes all labels that are defined at the group level only. It does not -list any labels that are defined in projects. You can filter the list by entering -a search query at the top and clicking search (**{search}**). +## Generate default project labels -To create a **group label**, navigate to **Group information > Labels** in the group and -follow the same process as [creating a project label](#project-labels). +If a project or its parent group has no labels, you can generate a default set of project +labels from the label list page. -#### Create group labels from epics **(ULTIMATE)** +Prerequisites: -You can create group labels from the epic sidebar. The labels you create -belong to the immediate group to which the epic belongs. The process is the same as -creating a [project label from an issue or merge request](#project-labels). +- You must have at least the Reporter role for the project. +- The project must have no labels present. -### Generate default labels +To add the default labels to the project: -If a project or group has no labels, you can generate a default set of project or group -labels from the label list page. The page shows a **Generate a default set of labels** -button if the list is empty. Select the button to add the following default labels -to the project: +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Project information > Labels**. +1. Select **Generate a default set of labels**. + +The following labels are created: - `bug` - `confirmed` @@ -157,131 +282,143 @@ to the project: ## Scoped labels **(PREMIUM)** -Scoped labels allow teams to use the label feature to annotate issues, merge requests -and epics with mutually exclusive labels. This can enable more complicated workflows -by preventing certain labels from being used together. - -A label is scoped when it uses a special double-colon (`::`) syntax in the label's -title, for example: +Teams can use scoped labels to annotate issues, merge requests, and epics with mutually exclusive +labels. By preventing certain labels from being used together, you can create more complex workflows.  -An issue, merge request or epic cannot have two scoped labels, of the form `key::value`, -with the same `key`. Adding a new label with the same `key`, but a different `value` -causes the previous `key` label to be replaced with the new label. +A scoped label uses a double-colon (`::`) syntax in its title, for example: `workflow::in-review`. -For example: +An issue, merge request, or epic cannot have two scoped labels, of the form `key::value`, +with the same `key`. If you add a new label with the same `key` but a different `value`, +the previous `key` label is replaced with the new label. -1. An issue is identified as being low priority, and a `priority::low` project - label is added to it. -1. After more review the issue priority is increased, and a `priority::high` label is - added. -1. GitLab automatically removes the `priority::low` label, as an issue should not - have two priority labels at the same time. +<i class="fa fa-youtube-play youtube" aria-hidden="true"></i> +For a video overview, see [Scoped Labels Speed Run](https://www.youtube.com/watch?v=ebyCiKMFODg). ### Filter by scoped labels > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/12285) in GitLab 14.4. -To filter issue, merge request, or epic lists for ones with labels that belong to a given scope, enter +To filter issue, merge request, or epic lists by a given scope, enter `<scope>::*` in the searched label name. For example, filtering by the `platform::*` label returns issues that have `platform::iOS`, `platform::Android`, or `platform::Linux` labels. NOTE: -This is not available on the [issues or merge requests dashboard pages](../search/index.md#search-issues-and-merge-requests). +Filtering by scoped labels not available on the [issues or merge requests dashboard pages](../search/index.md#search-issues-and-merge-requests). + +### Scoped labels examples + +**Example 1.** Updating issue priority: + +1. You decide that an issue is of low priority, and assign it the `priority::low` label. +1. After more review, you realize the issue's priority is higher increased, and you assign it the + `priority::high` label. +1. Because an issue shouldn't have two priority labels at the same time, GitLab removes the + `priority::low` label. + +**Example 2.** You want a custom field in issues to track the operating system platform +that your features target, where each issue should only target one platform. + +You create three labels: + +- `platform::iOS` +- `platform::Android` +- `platform::Linux` + +If you assign any of these labels to an issue automatically removes any other existing label that +starts with `platform::`. + +**Example 3.** You can use scoped labels to represent the workflow states of your teams. -### Workflows with scoped labels +Suppose you have the following labels: -Suppose you wanted a custom field in issues to track the operating system platform -that your features target, where each issue should only target one platform. You -would then create three labels `platform::iOS`, `platform::Android`, `platform::Linux`. -Applying any one of these labels on a given issue would automatically remove any other -existing label that starts with `platform::`. +- `workflow::development` +- `workflow::review` +- `workflow::deployed` -The same pattern could be applied to represent the workflow states of your teams. -Suppose you have the labels `workflow::development`, `workflow::review`, and -`workflow::deployed`. If an issue already has the label `workflow::development` -applied, and a developer wanted to advance the issue to `workflow::review`, they -would simply apply that label, and the `workflow::development` label would -automatically be removed. This behavior already exists when you move issues -across label lists in an [issue board](issue_board.md#create-workflows), but -now, team members who may not be working in an issue board directly would still -be able to advance workflow states consistently in issues themselves. +If an issue already has the label `workflow::development` and a developer wants to show that the +issue is now under review, they assign the `workflow::review`, and the `workflow::development` label +is removed. -This functionality is demonstrated in a video regarding -[using scoped labels for custom fields and workflows](https://www.youtube.com/watch?v=4BCBby6du3c). +The same happens when you move issues across label lists in an +[issue board](issue_board.md#create-workflows). With scoped labels, team members not working in an +issue board can also advance workflow states consistently in issues themselves. -### Scoped labels with nested scopes +For a video explanation, see: + +<div class="video-fallback"> + See the video: <a href="https://www.youtube.com/watch?v=4BCBby6du3c">Use scoped labels for custom fields and custom workflows</a>. +</div> +<figure class="video-container"> + <iframe src="https://www.youtube.com/embed/4BCBby6du3c" frameborder="0" allowfullscreen="true"> </iframe> +</figure> + +### Nested scopes You can create a label with a nested scope by using multiple double colons `::` when creating it. In this case, everything before the last `::` is the scope. -For example, `workflow::backend::review` and `workflow::backend::development` are valid -scoped labels, but they **can't** exist on the same issue at the same time, as they -both share the same scope, `workflow::backend`. +For example, if your project has these labels: -Additionally, `workflow::backend::review` and `workflow::frontend::review` are valid -scoped labels, and they **can** exist on the same issue at the same time, as they -both have different scopes, `workflow::frontend` and `workflow::backend`. +- `workflow::backend::review` +- `workflow::backend::development` +- `workflow::frontend::review` -## Subscribing to labels +An issue **can't** have both `workflow::backend::review` and `workflow::backend::development` +labels at the same time, because they both share the same scope: `workflow::backend`. -From the project label list page and the group label list page, you can click **Subscribe** -to the right of any label to enable [notifications](../profile/notifications.md) for that -label. You are notified whenever the label is assigned to an epic, -issue, or merge request. +On the other hand, an issue **can** have both `workflow::backend::review` and `workflow::frontend::review` +labels at the same time, because they both have different scopes: `workflow::frontend` and `workflow::backend`. -If you are subscribing to a group label from within a project, you can select to subscribe -to label notifications for the project only, or the whole group. +## Receive notifications when a label is used - +You can subscribe to a label to [receive notifications](../profile/notifications.md) whenever the +label is assigned to an issue, merge request, or epic. -## Label priority +To subscribe to a label: -Labels can have relative priorities, which are used in the **Label priority** and -**Priority** sort orders of issues and merge request list pages. Prioritization -for both group and project labels happens at the project level, and cannot be done -from the group label list. +1. [View the label list page.](#view-available-labels) +1. To the right of any label, select **Subscribe**. +1. Optional. If you are subscribing to a group label from a project, select either: + - **Subscribe at project level** to be notified about events in this project. + - **Subscribe at group level**: to be notified about events in the whole group. -NOTE: -Priority sorting is based on the highest priority label only. [This discussion](https://gitlab.com/gitlab-org/gitlab/-/issues/14523) considers changing this. +## Set label priority -From the project label list page, star a label to indicate that it has a priority. - - +Labels can have relative priorities, which are used when you sort issue and merge request lists +by [label priority](issues/sorting_issue_lists.md#sorting-by-label-priority) and [priority](issues/sorting_issue_lists.md#sorting-by-priority). -Drag starred labels up and down the list to change their priority, where higher in the list -means higher priority. +When prioritizing labels, you must do it from a project. +It's not possible to do it from the group label list. - +NOTE: +Priority sorting is based on the highest priority label only. +[This discussion](https://gitlab.com/gitlab-org/gitlab/-/issues/14523) considers changing this. -On the merge request and issue list pages (for both groups and projects) you -can sort by `Label priority` or `Priority`. +Prerequisites: -If you sort by `Label priority`, GitLab uses this sort comparison order: +- You must have at least the Reporter role for the project. -1. Items with a higher priority label. -1. Items without a prioritized label. +To prioritize a label: -Ties are broken arbitrarily. Note that only the highest prioritized label is checked, -and labels with a lower priority are ignored. See this [related issue](https://gitlab.com/gitlab-org/gitlab/-/issues/14523) -for more information. +1. On the top bar, select **Menu > Projects** and find your project. +1. On the left sidebar, select **Project information > Labels**. +1. Next to a label you want to prioritize, select the star (**{star-o}**). - + -If you sort by `Priority`, GitLab uses this sort comparison order: +This label now appears at the top of the label list, under **Prioritized Labels**. -1. Items with milestones that have due dates, where the soonest assigned [milestone](milestones/index.md) - is listed first. -1. Items with milestones with no due dates. -1. Items with a higher priority label. -1. Items without a prioritized label. +To change the relative priority of these labels, drag them up and down the list. +The labels higher in the list get higher priority. -Ties are broken arbitrarily. + - +To learn what happens when you sort by priority or label priority, see +[Sorting and ordering issue lists](issues/sorting_issue_lists.md). ## Troubleshooting diff --git a/doc/user/project/members/index.md b/doc/user/project/members/index.md index 8be2ade3f2f..c3d5dca0675 100644 --- a/doc/user/project/members/index.md +++ b/doc/user/project/members/index.md @@ -10,16 +10,43 @@ Members are the users and groups who have access to your project. Each member gets a role, which determines what they can do in the project. +Project members can: + +1. Be [direct members](#add-users-to-a-project) of the project. +1. [Inherit membership](#inherited-membership) of the project from the project's group. +1. Be a member of a group that was [shared](share_project_with_groups.md) with the project. +1. Be a member of a group that was [shared with the project's group](../../group/index.md#share-a-group-with-another-group). + +```mermaid +flowchart RL + subgraph Group A + A(Direct member) + B{{Shared member}} + subgraph Project A + H(1. Direct member) + C{{2. Inherited member}} + D{{4. Inherited member}} + E{{3. Shared member}} + end + A-->|Direct membership of Group A\nInherited membership of Project A|C + end + subgraph Group C + G(Direct member) + end + subgraph Group B + F(Direct member) + end + F-->|Group B\nshared with\nGroup A|B + B-->|Inherited membership of Project A|D + G-->|Group C shared with Project A|E +``` + ## Add users to a project > - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/247208) in GitLab 13.11 from a form to a modal window [with a flag](../../feature_flags.md). Disabled by default. > - Modal window [enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/247208) in GitLab 14.8. - -FLAG: -On self-managed GitLab, by default the modal window feature is available. -To hide the feature, ask an administrator to [disable the feature flag](../../../administration/feature_flags.md) -named `invite_members_group_modal`. -On GitLab.com, this feature is available. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/352526) in GitLab 14.9. + [Feature flag `invite_members_group_modal`](https://gitlab.com/gitlab-org/gitlab/-/issues/352526) removed. Add users to a project so they become members and have permission to perform actions. @@ -52,12 +79,8 @@ using the email address the invitation was sent to. > - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/247208) in GitLab 13.11 from a form to a modal window [with a flag](../../feature_flags.md). Disabled by default. > - Modal window [enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/247208) in GitLab 14.8. - -FLAG: -On self-managed GitLab, by default the modal window feature is available. -To hide the feature, ask an administrator to [disable the feature flag](../../../administration/feature_flags.md) -named `invite_members_group_modal`. -On GitLab.com, this feature is available. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/352526) in GitLab 14.9. + [Feature flag `invite_members_group_modal`](https://gitlab.com/gitlab-org/gitlab/-/issues/352526) removed. When you add a group to a project, each user in the group gets access to the project. Each user's access is based on: @@ -68,6 +91,7 @@ Each user's access is based on: Prerequisite: - You must have the Maintainer or Owner role. +- Sharing the project with other groups must not be [prevented](../../group/index.md#prevent-a-project-from-being-shared-with-groups). To add groups to a project: @@ -98,11 +122,11 @@ To import users: 1. On the top bar, select **Menu > Projects** and find your project. 1. On the left sidebar, select **Project information > Members**. -1. On the **Invite member** tab, at the bottom of the panel, select **Import**. +1. Select **Import from a project**. 1. Select the project. You can view only the projects for which you're a maintainer. 1. Select **Import project members**. -A success message is displayed and the new members are now displayed in the list. +After the success message displays, refresh the page to view the new members. ## Inherited membership @@ -139,7 +163,7 @@ To remove a member from a project: 1. On the top bar, select **Menu > Projects** and find your project. 1. On the left sidebar, select **Project information > Members**. -1. Next to the project member you want to remove, select **Remove member** **{remove}**. +1. Next to the project member you want to remove, select **Remove member**. 1. Optional. In the confirmation box, select the **Also unassign this user from related issues and merge requests** checkbox. 1. To prevent leaks of sensitive information from private projects, verify the diff --git a/doc/user/project/members/share_project_with_groups.md b/doc/user/project/members/share_project_with_groups.md index 4e3bae2dc30..02a9b76ce38 100644 --- a/doc/user/project/members/share_project_with_groups.md +++ b/doc/user/project/members/share_project_with_groups.md @@ -15,10 +15,14 @@ Groups are used primarily to [create collections of projects](../../group/index. take advantage of the fact that groups define collections of _users_, namely the group members. -## Sharing a project with a group of users +## Share a project with a group of users -NOTE: -In GitLab 13.11, you can [replace this form with a modal window](#share-a-project-modal-window). +> - [Changed](https://gitlab.com/gitlab-org/gitlab/-/issues/247208) in GitLab 13.11 from a form to a modal + window [with a flag](../../feature_flags.md). Disabled by default. +> - Modal window [enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/issues/247208) + in GitLab 14.8. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/352526) in GitLab 14.9. + [Feature flag `invite_members_group_modal`](https://gitlab.com/gitlab-org/gitlab/-/issues/352526) removed. The primary mechanism to give a group of users, say 'Engineering', access to a project, say 'Project Acme', in GitLab is to make the 'Engineering' group the owner of 'Project @@ -28,9 +32,9 @@ This is where the group sharing feature can be of use. To share 'Project Acme' with the 'Engineering' group: 1. For 'Project Acme' use the left navigation menu to go to **Project information > Members**. -1. Select the **Invite group** tab. +1. Select **Invite a group**. 1. Add the 'Engineering' group with the maximum access level of your choice. -1. Optional. Select an expiration date. +1. Optional. Select an **Access expiration date**. 1. Select **Invite**. After sharing 'Project Acme' with 'Engineering': @@ -45,51 +49,19 @@ You can share a project only with: Administrators can share projects with any group in the system. -### Share a project modal window - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/247208) in GitLab 13.11. -> - [Deployed behind a feature flag](../../feature_flags.md), disabled by default. -> - Enabled on GitLab.com. -> - Recommended for production use. -> - Replaces the existing form with buttons to open a modal window. -> - To use in GitLab self-managed instances, ask a GitLab administrator to [enable it](#enable-or-disable-modal-window). - -WARNING: -This feature might not be available to you. Check the **version history** note above for details. - -In GitLab 13.11, you can optionally replace the sharing form with a modal window. -To share a project after enabling this feature: - -1. Go to your project's page. -1. On the left sidebar, go to **Project information > Members**, and then select **Invite a group**. -1. Select a group, and select a **Max role**. -1. Optional. Select an **Access expiration date**. -1. Select **Invite**. - -### Enable or disable modal window **(FREE SELF)** - -The modal window for sharing a project is under development and is ready for production use. It is -deployed behind a feature flag that is **disabled by default**. -[GitLab administrators with access to the GitLab Rails console](../../../administration/feature_flags.md) -can enable it. - -To enable it: - -```ruby -Feature.enable(:invite_members_group_modal) -``` +## Maximum access level -To disable it: +In the example above, the maximum access level of 'Developer' for members from 'Engineering' means that users with higher access levels in 'Engineering' ('Maintainer' or 'Owner') only have 'Developer' access to 'Project Acme'. -```ruby -Feature.disable(:invite_members_group_modal) -``` +### Share a project with a subgroup -## Maximum access level +You can't share a project with a group that's an ancestor of a [subgroup](../../group/subgroups/index.md) the project is +in. That means you can only share down the hierarchy. For example, `group/subgroup01/project`: -In the example above, the maximum access level of 'Developer' for members from 'Engineering' means that users with higher access levels in 'Engineering' ('Maintainer' or 'Owner') only have 'Developer' access to 'Project Acme'. +- Can not be shared with `group`. +- Can be shared with `group/subgroup02` or `group/subgroup01/subgroup03`. -## Sharing public project with private group +## Share public project with private group When sharing a public project with a private group, owners and maintainers of the project see the name of the group in the `members` page. Owners also have the possibility to see members of the private group they don't have access to when mentioning them in the issue or merge request. diff --git a/doc/user/project/merge_requests/approvals/rules.md b/doc/user/project/merge_requests/approvals/rules.md index fa447ea35af..01772e59127 100644 --- a/doc/user/project/merge_requests/approvals/rules.md +++ b/doc/user/project/merge_requests/approvals/rules.md @@ -175,7 +175,7 @@ granting them push access: 1. [Create a new group](../../../group/index.md#create-a-group). 1. [Add the user to the group](../../../group/index.md#add-users-to-a-group), and select the Reporter role for the user. -1. [Share the project with your group](../../members/share_project_with_groups.md#sharing-a-project-with-a-group-of-users), +1. [Share the project with your group](../../members/share_project_with_groups.md#share-a-project-with-a-group-of-users), based on the Reporter role. 1. Go to your project and select **Settings > General**. 1. Expand **Merge request (MR) approvals**. diff --git a/doc/user/project/merge_requests/approvals/settings.md b/doc/user/project/merge_requests/approvals/settings.md index 1e1c8ccb241..0a7fbc9ee95 100644 --- a/doc/user/project/merge_requests/approvals/settings.md +++ b/doc/user/project/merge_requests/approvals/settings.md @@ -140,9 +140,7 @@ To learn more, see [Coverage check approval rule](../../../../ci/pipelines/setti > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/285410) in GitLab 14.4. [Deployed behind the `group_merge_request_approval_settings_feature_flag` flag](../../../../administration/feature_flags.md), disabled by default. > - [Enabled by default](https://gitlab.com/gitlab-org/gitlab/-/issues/285410) in GitLab 14.5. - -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature per group, ask an administrator to [disable the feature flag](../../../../administration/feature_flags.md) named `group_merge_request_approval_settings_feature_flag`. On GitLab.com, this feature is available. +> - [Feature flag `group_merge_request_approval_settings_feature_flag`](https://gitlab.com/gitlab-org/gitlab/-/issues/343872) removed in GitLab 14.9. You can also enforce merge request approval settings: diff --git a/doc/user/project/merge_requests/cherry_pick_changes.md b/doc/user/project/merge_requests/cherry_pick_changes.md index 15ba6e9de98..fb41ec3eff6 100644 --- a/doc/user/project/merge_requests/cherry_pick_changes.md +++ b/doc/user/project/merge_requests/cherry_pick_changes.md @@ -66,11 +66,8 @@ git cherry-pick -m 2 7a39eb0 > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/21268) in GitLab 13.11 behind a [feature flag](../../feature_flags.md), disabled by default. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/324154) in GitLab 14.0. -WARNING: -This feature might not be available to you. Check the **version history** note above for details. - -You can use the GitLab UI to cherry-pick merge requests into a project, even if the -merge request is from a fork: +You can cherry-pick merge requests from the same project, or forks of the same +project, from the GitLab user interface: 1. In the merge request's secondary menu, click **Commits** to display the commit details page. 1. Click on the **Options** dropdown and select **Cherry-pick** to show the cherry-pick modal. diff --git a/doc/user/project/merge_requests/code_quality.md b/doc/user/project/merge_requests/code_quality.md index d735ce0ef91..10fc778d5ae 100644 --- a/doc/user/project/merge_requests/code_quality.md +++ b/doc/user/project/merge_requests/code_quality.md @@ -592,17 +592,17 @@ plugins: If your merge requests do not show any code quality changes when using a custom tool, ensure that the line property is an `integer`. -### Code Quality CI job with Code Climate plugins enabled fails with error "engine <plugin_name> ran for 900 seconds and was killed" +### Code Quality CI job with Code Climate plugins enabled fails with error -If you enabled any of the Code Climate plugins, and the Code Quality CI job fails with the error below, -it's likely the job takes longer than the default timeout of 900 seconds. +If you enabled any of the Code Climate plugins, and the Code Quality CI job fails with the error +below, it's likely the job takes longer than the default timeout of 900 seconds: ```shell error: (CC::CLI::Analyze::EngineFailure) engine pmd ran for 900 seconds and was killed Could not analyze code quality for the repository at /code ``` -To work around this problem, set `TIMEOUT_SECONDS` to a higher value in your `.gitlab.-ci.yml` file. +To work around this problem, set `TIMEOUT_SECONDS` to a higher value in your `.gitlab.-ci.yml` file. For example: diff --git a/doc/user/project/merge_requests/commit_templates.md b/doc/user/project/merge_requests/commit_templates.md index 0cc18d2117b..b9b65d759dc 100644 --- a/doc/user/project/merge_requests/commit_templates.md +++ b/doc/user/project/merge_requests/commit_templates.md @@ -69,6 +69,7 @@ GitLab creates a squash commit message with this template: > - [Added](https://gitlab.com/gitlab-org/gitlab/-/issues/346805) `first_commit` and `first_multiline_commit` variables in GitLab 14.6. > - [Added](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75639) `url`, `approved_by`, and `merged_by` variables in GitLab 14.7. > - [Added](https://gitlab.com/gitlab-org/gitlab/-/issues/20421) `co_authored_by` variable in GitLab 14.7. +> - [Added](https://gitlab.com/gitlab-org/gitlab/-/issues/26303) `all_commits` variable in GitLab 14.9. Commit message templates support these variables: @@ -81,15 +82,20 @@ Commit message templates support these variables: | `%{description}` | Description of the merge request. | `Merge request description.`<br>`Can be multiline.` | | `%{reference}` | Reference to the merge request. | `group-name/project-name!72359` | | `%{first_commit}` | Full message of the first commit in merge request diff. | `Update README.md` | -| `%{first_multiline_commit}` | Full message of the first commit that's not a merge commit and has more than one line in message body. Merge Request title if all commits aren't multiline. | `Update README.md`<br><br>`Improved project description in readme file.` | +| `%{first_multiline_commit}` | Full message of the first commit that's not a merge commit and has more than one line in message body. Merge request title if all commits aren't multiline. | `Update README.md`<br><br>`Improved project description in readme file.` | | `%{url}` | Full URL to the merge request. | `https://gitlab.com/gitlab-org/gitlab/-/merge_requests/1` | -| `%{approved_by}` | Line-separated list of the merge request approvers. This value is not updated until the first page refresh after an approval. | `Approved-by: Sidney Jones <sjones@example.com>` <br> `Approved-by: Zhang Wei <zwei@example.com>` | +| `%{approved_by}` | Line-separated list of the merge request approvers. | `Approved-by: Sidney Jones <sjones@example.com>` <br> `Approved-by: Zhang Wei <zwei@example.com>` | | `%{merged_by}` | User who merged the merge request. | `Alex Garcia <agarcia@example.com>` | | `%{co_authored_by}` | Names and emails of commit authors in a `Co-authored-by` Git commit trailer format. Limited to authors of 100 most recent commits in merge request. | `Co-authored-by: Zane Doe <zdoe@example.com>` <br> `Co-authored-by: Blake Smith <bsmith@example.com>` | +| `%{all_commits}` | Messages from all commits in the merge request. Limited to 100 most recent commits. Skips commit bodies exceeding 100KiB and merge commit messages. | `* Feature introduced` <br><br> `This commit implements feature` <br> `Changelog:added` <br><br> `* Bug fixed` <br><br> `* Documentation improved` <br><br>`This commit introduced better docs.`| Any line containing only an empty variable is removed. If the line to be removed is both preceded and followed by an empty line, the preceding empty line is also removed. +After you edit a commit message on an open merge request, GitLab will +not automatically update the commit message again. +To restore the commit message to the project template, reload the page. + ## Related topics - [Squash and merge](squash_and_merge.md). diff --git a/doc/user/project/merge_requests/commits.md b/doc/user/project/merge_requests/commits.md index 0014c1ba994..20e0d3a1f5b 100644 --- a/doc/user/project/merge_requests/commits.md +++ b/doc/user/project/merge_requests/commits.md @@ -31,15 +31,7 @@ To seamlessly navigate among commits in a merge request: > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/29274) in GitLab 13.12 [with a flag](../../../administration/feature_flags.md) named `context_commits`. Enabled by default. > - [Enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/320757) in GitLab 14.8. - -WARNING: -This feature is in [beta](../../../policy/alpha-beta-support.md#beta-features) -and is [incomplete](https://gitlab.com/groups/gitlab-org/-/epics/1192). - -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, -ask an administrator to [disable the feature flag](../../../administration/feature_flags.md) named `context_commits`. -On GitLab.com, this feature is available. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/320757) in GitLab 14.9. [Feature flag `context_commits`](https://gitlab.com/gitlab-org/gitlab/-/issues/320757) removed. When reviewing a merge request, it helps to have more context about the changes made. That includes unchanged lines in unchanged files, and previous commits diff --git a/doc/user/project/merge_requests/csv_export.md b/doc/user/project/merge_requests/csv_export.md index 6dbbab316a0..df527ec6ebf 100644 --- a/doc/user/project/merge_requests/csv_export.md +++ b/doc/user/project/merge_requests/csv_export.md @@ -18,11 +18,11 @@ The following table shows what attributes will be present in the CSV. | Column | Description | |--------------------|--------------------------------------------------------------| +| Title | Merge request title | +| Description | Merge request description | | MR ID | MR `iid` | | URL | A link to the merge request on GitLab | -| Title | Merge request title | | State | Opened, Closed, Locked, or Merged | -| Description | Merge request description | | Source Branch | Source branch | | Target Branch | Target branch | | Source Project ID | ID of the source project | @@ -39,6 +39,10 @@ The following table shows what attributes will be present in the CSV. | Created At (UTC) | Formatted as `YYYY-MM-DD HH:MM:SS` | | Updated At (UTC) | Formatted as `YYYY-MM-DD HH:MM:SS` | +In GitLab 14.7 and earlier, the first two columns were `MR ID` and `URL`, +which [caused an issue](https://gitlab.com/gitlab-org/gitlab/-/issues/34769) +when importing back into GitLab. + ## Limitations - Export merge requests to CSV is not available at the Group's merge request list. diff --git a/doc/user/project/merge_requests/fast_forward_merge.md b/doc/user/project/merge_requests/fast_forward_merge.md index cd65fe20e66..dc13b270f17 100644 --- a/doc/user/project/merge_requests/fast_forward_merge.md +++ b/doc/user/project/merge_requests/fast_forward_merge.md @@ -43,7 +43,7 @@ You can also rebase without running a CI/CD pipeline. The rebase action is also available as a [quick action command: `/rebase`](../../../topics/git/git_rebase.md#rebase-from-the-gitlab-ui). - + If the target branch is ahead of the source branch and a conflict free rebase is not possible, you need to rebase the diff --git a/doc/user/project/merge_requests/getting_started.md b/doc/user/project/merge_requests/getting_started.md index 8699a42dd5d..fd1751585d5 100644 --- a/doc/user/project/merge_requests/getting_started.md +++ b/doc/user/project/merge_requests/getting_started.md @@ -7,7 +7,7 @@ description: "Getting started with merge requests." # Getting started with merge requests **(FREE)** -A merge request (**MR**) is the basis of GitLab as a code +A merge request (**MR**) is the basis of GitLab as a tool for code collaboration and version control. When working in a Git-based platform, you can use branching @@ -161,7 +161,7 @@ enabled by default for all new merge requests, enable it in the [project's settings](../settings/index.md#merge-request-settings). This option is also visible in an existing merge request next to -the merge request button and can be selected or deselected before merging. +the merge request button and can be selected or cleared before merging. It is only visible to users with the Maintainer role in the source project. diff --git a/doc/user/project/merge_requests/img/ff_merge_rebase_v14_7.png b/doc/user/project/merge_requests/img/ff_merge_rebase_v14_7.png Binary files differdeleted file mode 100644 index 3c845d277e4..00000000000 --- a/doc/user/project/merge_requests/img/ff_merge_rebase_v14_7.png +++ /dev/null diff --git a/doc/user/project/merge_requests/img/ff_merge_rebase_v14_9.png b/doc/user/project/merge_requests/img/ff_merge_rebase_v14_9.png Binary files differnew file mode 100644 index 00000000000..f4330549a57 --- /dev/null +++ b/doc/user/project/merge_requests/img/ff_merge_rebase_v14_9.png diff --git a/doc/user/project/merge_requests/img/merge_request_diff_v12_2.png b/doc/user/project/merge_requests/img/merge_request_diff_v12_2.png Binary files differdeleted file mode 100644 index 7e23b7db309..00000000000 --- a/doc/user/project/merge_requests/img/merge_request_diff_v12_2.png +++ /dev/null diff --git a/doc/user/project/merge_requests/load_performance_testing.md b/doc/user/project/merge_requests/load_performance_testing.md index cc4ad186771..7861e1e28fc 100644 --- a/doc/user/project/merge_requests/load_performance_testing.md +++ b/doc/user/project/merge_requests/load_performance_testing.md @@ -92,7 +92,7 @@ template that is included with GitLab. NOTE: For large scale k6 tests you need to ensure the GitLab Runner instance performing the actual test is able to handle running the test. Refer to [k6's guidance](https://k6.io/docs/testing-guides/running-large-tests#hardware-considerations) -for spec details. The [default shared GitLab.com runners](../../../ci/runners/build_cloud/linux_build_cloud.md) +for spec details. The [default shared GitLab.com runners](../../../ci/runners/saas/linux_saas_runner.md) likely have insufficient specs to handle most large k6 tests. This template runs the diff --git a/doc/user/project/merge_requests/resolve_conflicts.md b/doc/user/project/merge_requests/resolve_conflicts.md deleted file mode 100644 index 611f37a949b..00000000000 --- a/doc/user/project/merge_requests/resolve_conflicts.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -redirect_to: 'conflicts.md' -remove_date: '2022-01-26' ---- - -This document was moved to [another location](conflicts.md). - -<!-- This redirect file can be deleted after <2022-01-26>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> diff --git a/doc/user/project/merge_requests/squash_and_merge.md b/doc/user/project/merge_requests/squash_and_merge.md index f90296e9626..a1d6959b75e 100644 --- a/doc/user/project/merge_requests/squash_and_merge.md +++ b/doc/user/project/merge_requests/squash_and_merge.md @@ -65,7 +65,7 @@ To configure the default squashing behavior for all merge requests in your proje 1. Expand **Merge requests**. 1. In the **Squash commits when merging** section, select your desired behavior: - **Do not allow**: Squashing is never performed, and the option is not displayed. - - **Allow**: Squashing is allowed, but deselected by default. + - **Allow**: Squashing is allowed, but cleared by default. - **Encourage**: Squashing is allowed and selected by default, but can be disabled. - **Require**: Squashing is always performed. While merge requests display the option to squash, users cannot change it. diff --git a/doc/user/project/merge_requests/test_coverage_visualization.md b/doc/user/project/merge_requests/test_coverage_visualization.md index bd54eda42f5..d7177208a6e 100644 --- a/doc/user/project/merge_requests/test_coverage_visualization.md +++ b/doc/user/project/merge_requests/test_coverage_visualization.md @@ -52,7 +52,12 @@ from any job in any stage in the pipeline. The coverage displays for each line: Hovering over the coverage bar provides further information, such as the number of times the line was checked by tests. -Uploading a test coverage report does not enable [test coverage results in Merge Requests](../../../ci/pipelines/settings.md#add-test-coverage-results-to-a-merge-request-deprecated) or [code coverage history](../../../ci/pipelines/settings.md#view-code-coverage-history). You must configure these separately. +Uploading a test coverage report does not enable: + +- [Test coverage results in merge requests](../../../ci/pipelines/settings.md#merge-request-test-coverage-results). +- [Code coverage history](../../../ci/pipelines/settings.md#view-code-coverage-history). + +You must configure these separately. ### Limits diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/dns_concepts.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/dns_concepts.md index 165131d50a4..3491346f7d9 100644 --- a/doc/user/project/pages/custom_domains_ssl_tls_certification/dns_concepts.md +++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/dns_concepts.md @@ -37,7 +37,7 @@ for the most popular hosting services: - [Cloudflare](https://support.cloudflare.com/hc/en-us/articles/201720164-Creating-a-Cloudflare-account-and-adding-a-website) - [cPanel](https://documentation.cpanel.net/display/84Docs/Edit+DNS+Zone) - [DigitalOcean](https://docs.digitalocean.com/products/networking/dns/how-to/manage-records/) -- [DreamHost](https://help.dreamhost.com/hc/en-us/articles/215414867-How-do-I-add-custom-DNS-records-) +- [DreamHost](https://help.dreamhost.com/hc/en-us/articles/360035516812) - [Gandi](https://docs.gandi.net/en/domain_names/faq/dns_records.html) - [Go Daddy](https://www.godaddy.com/help/add-an-a-record-19238) - [Hostgator](https://www.hostgator.com/help/article/changing-dns-records) diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md index ec66dce41f9..4d8919090a2 100644 --- a/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md +++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/index.md @@ -315,7 +315,7 @@ To enable this setting: 1. Tick the checkbox **Force HTTPS (requires valid certificates)**. If you use Cloudflare CDN in front of GitLab Pages, make sure to set the SSL connection setting to -`full` instead of `flexible`. For more details, see the [Cloudflare CDN directions](https://support.cloudflare.com/hc/en-us/articles/200170416-End-to-end-HTTPS-with-Cloudflare-Part-3-SSL-options#h_4e0d1a7c-eb71-4204-9e22-9d3ef9ef7fef). +`full` instead of `flexible`. For more details, see the [Cloudflare CDN directions](https://developers.cloudflare.com/ssl/origin-configuration/ssl-modes#h_4e0d1a7c-eb71-4204-9e22-9d3ef9ef7fef). <!-- ## Troubleshooting diff --git a/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md b/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md index 75fc407ce3f..f09aea3b02a 100644 --- a/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md +++ b/doc/user/project/pages/custom_domains_ssl_tls_certification/lets_encrypt_integration.md @@ -79,6 +79,8 @@ If you get an error **Something went wrong while obtaining the Let's Encrypt cer 1. Make sure [your domain is verified](index.md#1-add-a-custom-domain-to-pages). 1. Go to step 1. +Another possible cause of this error is the `_redirects` file because the current implementation relies on an HTTP ACME challenge. If you redirect the `.acme-challenge/` endpoint Let's Encrypt cannot validate the domain. Make sure you don't have a wildcard (`*`) redirect either as that too breaks validation. The problem with wildcard redirects is tracked in the [Wildcard redirects break Let's Encrypt integration](https://gitlab.com/gitlab-org/gitlab-pages/-/issues/649) issue. + ### Message "GitLab is obtaining a Let's Encrypt SSL certificate for this domain. This process can take some time. Please try again later." hangs for more than an hour If you've enabled Let's Encrypt integration, but a certificate is absent after an hour and you see the message, "GitLab is obtaining a Let's Encrypt SSL certificate for this domain. This process can take some time. Please try again later.", try to remove and add the domain for GitLab Pages again by following these steps: diff --git a/doc/user/project/pages/getting_started_part_one.md b/doc/user/project/pages/getting_started_part_one.md index 32826346eab..5773dd1f2c0 100644 --- a/doc/user/project/pages/getting_started_part_one.md +++ b/doc/user/project/pages/getting_started_part_one.md @@ -34,7 +34,7 @@ Pages domains are `*.gitlab.io`. | Project pages owned by a subgroup | `subgroup/projectname` | `http(s)://groupname.example.io/subgroup/projectname`| WARNING: -There are some known [limitations](introduction.md#limitations) +There are some known [limitations](introduction.md#subdomains-of-subdomains) regarding namespaces served under the general domain name and HTTPS. Make sure to read that section. diff --git a/doc/user/project/pages/introduction.md b/doc/user/project/pages/introduction.md index 65e1579001b..e4bc58854ef 100644 --- a/doc/user/project/pages/introduction.md +++ b/doc/user/project/pages/introduction.md @@ -76,17 +76,21 @@ website.  -## Limitations +## Subdomains of subdomains -When using Pages under the general domain of a GitLab instance (`*.example.io`), -you _cannot_ use HTTPS with sub-subdomains. That means that if your -username or group name contains a dot, for example `foo.bar`, the domain -`https://foo.bar.example.io` does _not_ work. This is a limitation of the -[HTTP Over TLS protocol](https://tools.ietf.org/html/rfc2818#section-3.1). -HTTP pages continue to work provided you don't redirect HTTP to HTTPS. +When using Pages under the top-level domain of a GitLab instance (`*.example.io`), you can't use HTTPS with subdomains +of subdomains. If your namespace or group name contains a dot (for example, `foo.bar`) the domain +`https://foo.bar.example.io` does _not_ work. -GitLab Pages [does **not** support group websites for subgroups](../../group/subgroups/index.md#limitations). -You can only create the highest-level group website. +This limitation is because of the [HTTP Over TLS protocol](https://tools.ietf.org/html/rfc2818#section-3.1). HTTP pages +work as long as you don't redirect HTTP to HTTPS. + +## GitLab Pages and subgroups + +You must host your GitLab Pages website in a project. This project can belong to a [group](../../group/index.md) or +[subgroup](../../group/subgroups/index.md). For +[group websites](../../project/pages/getting_started_part_one.md#gitlab-pages-default-domain-names), the group must be +at the top level and not a subgroup. ## Specific configuration options for Pages diff --git a/doc/user/project/pages/lets_encrypt_for_gitlab_pages.md b/doc/user/project/pages/lets_encrypt_for_gitlab_pages.md index 1f60aafe71b..7779f87b459 100644 --- a/doc/user/project/pages/lets_encrypt_for_gitlab_pages.md +++ b/doc/user/project/pages/lets_encrypt_for_gitlab_pages.md @@ -6,4 +6,6 @@ remove_date: '2022-03-14' This file was moved to [another location](custom_domains_ssl_tls_certification/lets_encrypt_integration.md). <!-- This redirect file can be deleted after <2022-03-14>. --> -<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/#move-or-rename-a-page --> +<!-- Redirects that point to other docs in the same project expire in three months. --> +<!-- Redirects that point to docs in a different project or site (for example, link is not relative and starts with `https:`) expire in one year. --> +<!-- Before deletion, see: https://docs.gitlab.com/ee/development/documentation/redirects.html --> diff --git a/doc/user/project/pages/redirects.md b/doc/user/project/pages/redirects.md index beafbc86cb5..cdae1d2f837 100644 --- a/doc/user/project/pages/redirects.md +++ b/doc/user/project/pages/redirects.md @@ -163,6 +163,12 @@ Splats also match empty strings, so the previous rule redirects ### Rewrite all requests to a root `index.html` +NOTE: +If you are using [GitLab Pages integration with Let’s Encrypt](custom_domains_ssl_tls_certification/lets_encrypt_integration.md), +you must enable it before adding this rule. Otherwise, the redirection breaks the Let's Encrypt +integration. For more details, see +[GitLab Pages issue 649](https://gitlab.com/gitlab-org/gitlab-pages/-/issues/649). + Single page applications (SPAs) often perform their own routing using client-side routes. For these applications, it's important that _all_ requests are rewritten to the root `index.html` so that the routing logic can be handled @@ -180,7 +186,7 @@ rule like: Use placeholders in rules to match portions of the requested URL and use these matches when rewriting or redirecting to a new URL. -A placehold is formatted as a `:` character followed by a string of letters +A placeholder is formatted as a `:` character followed by a string of letters (`[a-zA-Z]+`) in both the `from` and `to` paths: ```plaintext diff --git a/doc/user/project/protected_branches.md b/doc/user/project/protected_branches.md index 8bb18905222..292530e6c9c 100644 --- a/doc/user/project/protected_branches.md +++ b/doc/user/project/protected_branches.md @@ -29,7 +29,7 @@ When a branch is protected, the default behavior enforces these restrictions on ### Set the default branch protection level Administrators can set a default branch protection level in the -[Admin Area](../admin_area/settings/visibility_and_access_controls.md#protect-default-branches). +[Admin Area](../project/repository/branches/default.md#instance-level-default-branch-protection). ## Configure a protected branch @@ -41,7 +41,7 @@ To protect a branch: 1. Go to your project and select **Settings > Repository**. 1. Expand **Protected branches**. -1. From the **Branch** dropdown menu, select the branch you want to protect. +1. From the **Branch** dropdown list, select the branch you want to protect. 1. From the **Allowed to merge** list, select a role, or group that can merge into this branch. In GitLab Premium, you can also add users. 1. From the **Allowed to push** list, select a role, group, or user that can push to this branch. In GitLab Premium, you can also add users. 1. Select **Protect**. @@ -58,7 +58,7 @@ To protect multiple branches at the same time: 1. Go to your project and select **Settings > Repository**. 1. Expand **Protected branches**. -1. From the **Branch** dropdown menu, type the branch name and a wildcard. +1. From the **Branch** dropdown list, type the branch name and a wildcard. For example: | Wildcard protected branch | Matching branches | @@ -101,7 +101,7 @@ to a protected branch. This is compatible with workflows like the [GitLab workfl 1. Go to your project and select **Settings > Repository**. 1. Expand **Protected branches**. -1. From the **Branch** dropdown menu, select the branch you want to protect. +1. From the **Branch** dropdown list, select the branch you want to protect. 1. From the **Allowed to merge** list, select **Developers + Maintainers**. 1. From the **Allowed to push** list, select **No one**. 1. Select **Protect**. @@ -112,7 +112,7 @@ You can allow everyone with write access to push to the protected branch. 1. Go to your project and select **Settings > Repository**. 1. Expand **Protected branches**. -1. From the **Branch** dropdown menu, select the branch you want to protect. +1. From the **Branch** dropdown list, select the branch you want to protect. 1. From the **Allowed to push** list, select **Developers + Maintainers**. 1. Select **Protect**. @@ -128,27 +128,28 @@ key must have at least read access to the project. Prerequisites: -- The deploy key must be [enabled for your project](deploy_keys/index.md#how-to-enable-deploy-keys). -- The deploy key must have [write access](deploy_keys/index.md#deploy-keys-permissions) to your project repository. +- The deploy key must be enabled for your project. A project deploy key is enabled by default when + it is created. However, a public deploy key must be + [granted](deploy_keys/index.md#grant-project-access-to-a-public-deploy-key) access to the + project. +- The deploy key must have [write access](deploy_keys/index.md#permissions) to your project + repository. To allow a deploy key to push to a protected branch: 1. Go to your project and select **Settings > Repository**. 1. Expand **Protected branches**. -1. From the **Branch** dropdown menu, select the branch you want to protect. +1. From the **Branch** dropdown list, select the branch you want to protect. 1. From the **Allowed to push** list, select the deploy key. 1. Select **Protect**. -Deploy keys are not available in the **Allowed to merge** dropdown. +Deploy keys are not available in the **Allowed to merge** dropdown list. ## Allow force push on a protected branch > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15611) in GitLab 13.10 behind a disabled feature flag. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/323431) in GitLab 14.0. -WARNING: -This feature might not be available to you. Check the **version history** note above for details. - You can allow [force pushes](../../topics/git/git_rebase.md#force-push) to protected branches. @@ -156,7 +157,7 @@ To protect a new branch and enable force push: 1. Go to your project and select **Settings > Repository**. 1. Expand **Protected branches**. -1. From the **Branch** dropdown menu, select the branch you want to protect. +1. From the **Branch** dropdown list, select the branch you want to protect. 1. From the **Allowed to push** and **Allowed to merge** lists, select the settings you want. 1. To allow all users with push access to force push, turn on the **Allowed to force push** toggle. 1. To reject code pushes that change files listed in the `CODEOWNERS` file, turn on the @@ -169,12 +170,12 @@ To enable force pushes on branches that are already protected: 1. Expand **Protected branches**. 1. In the list of protected branches, next to the branch, turn on the **Allowed to force push** toggle. -When enabled, members who are can push to this branch can also force push. +Members who can push to this branch can now also force push. ## Require Code Owner approval on a protected branch **(PREMIUM)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13251) in GitLab Premium 12.4. -> - [In](https://gitlab.com/gitlab-org/gitlab/-/issues/35097) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.5 and later, users and groups who can push to protected branches do not have to use a merge request to merge their feature branches. This means they can skip merge request approval rules. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13251) in GitLab 12.4. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/35097) in GitLab 13.5, users and groups who can push to protected branches do not have to use a merge request to merge their feature branches. This means they can skip merge request approval rules. For a protected branch, you can require at least one approval by a [Code Owner](code_owners.md). @@ -182,7 +183,7 @@ To protect a new branch and enable Code Owner's approval: 1. Go to your project and select **Settings > Repository**. 1. Expand **Protected branches**. -1. From the **Branch** dropdown menu, select the branch you want to protect. +1. From the **Branch** dropdown list, select the branch you want to protect. 1. From the **Allowed to push** and **Allowed to merge** lists, select the settings you want. 1. Turn on the **Require approval from code owners** toggle. 1. Select **Protect**. @@ -221,7 +222,7 @@ Users with at least the Maintainer role can manually delete protected branches by using the GitLab web interface: 1. Go to **Repository > Branches**. -1. Next to the branch you want to delete, select the **Delete** button (**{remove}**). +1. Next to the branch you want to delete, select **Delete** (**{remove}**). 1. On the confirmation dialog, type the branch name and select **Delete protected branch**. Protected branches can only be deleted by using GitLab either from the UI or API. diff --git a/doc/user/project/protected_tags.md b/doc/user/project/protected_tags.md index 7d071a45d63..5df34fcdcc4 100644 --- a/doc/user/project/protected_tags.md +++ b/doc/user/project/protected_tags.md @@ -27,18 +27,18 @@ By default: ## Configuring protected tags -To protect a tag, you need to have at least the Maintainer role. +To protect a tag, you must have at least the Maintainer role. 1. Go to the project's **Settings > Repository**. -1. From the **Tag** dropdown menu, select the tag you want to protect or type and click **Create wildcard**. In the screenshot below, we chose to protect all tags matching `v*`: +1. From the **Tag** dropdown list, select the tag you want to protect or type and click **Create wildcard**. In the screenshot below, we chose to protect all tags matching `v*`:  -1. From the **Allowed to create** dropdown, select users with permission to create - matching tags, and click **Protect**: +1. From the **Allowed to create** dropdown list, select users with permission to create + matching tags, and select **Protect**: -  +  1. After done, the protected tag displays in the **Protected tags** list: @@ -61,7 +61,7 @@ Two different wildcards can potentially match the same tag. For example, In that case, if _any_ of these protected tags have a setting like **Allowed to create**, then `production-stable` also inherit this setting. -If you click on a protected tag's name, GitLab displays a list of +If you select a protected tag's name, GitLab displays a list of all matching tags:  diff --git a/doc/user/project/quick_actions.md b/doc/user/project/quick_actions.md index 8070c37db78..ae42d90af06 100644 --- a/doc/user/project/quick_actions.md +++ b/doc/user/project/quick_actions.md @@ -7,11 +7,11 @@ info: To determine the technical writer assigned to the Stage/Group associated w # GitLab quick actions **(FREE)** -> - Introduced in [GitLab 12.1](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/26672): +> - [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/merge_requests/26672) in GitLab 12.1: > once an action is executed, an alert appears when a quick action is successfully applied. -> - Introduced in [GitLab 13.2](https://gitlab.com/gitlab-org/gitlab/-/issues/16877): you can use +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16877) in GitLab 13.2: you can use > quick actions when updating the description of issues, epics, and merge requests. -> - Introduced in [GitLab 13.8](https://gitlab.com/gitlab-org/gitlab/-/issues/292393): when you enter +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/292393) in GitLab 13.8: when you enter > `/` into a description or comment field, all available quick actions are displayed in a scrollable list. > - The rebase quick action was [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/49800) in GitLab 13.8. @@ -49,15 +49,15 @@ threads. Some quick actions might not be available to all subscription tiers. | Command | Issue | Merge request | Epic | Action | |:-------------------------------------------------------------------------------------------------|:-----------------------|:-----------------------|:-----------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `/add_contacts [contact:email1@example.com] [contact:email2@example.com]` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add one or more [CRM contacts](../crm/index.md) ([introduced in GitLab 14.6](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/73413)). | +| `/add_contacts [contact:email1@example.com] [contact:email2@example.com]` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add one or more [CRM contacts](../crm/index.md) ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/73413) in GitLab 14.6). | | `/approve` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Approve the merge request. | | `/assign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Assign one or more users. | | `/assign me` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Assign yourself. | | `/assign_reviewer @user1 @user2` or `/reviewer @user1 @user2` or `/request_review @user1 @user2` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Assign one or more users as reviewers. | | `/assign_reviewer me` or `/reviewer me` or `/request_review me` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Assign yourself as a reviewer. | | `/award :emoji:` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Toggle emoji award. | -| `/child_epic <epic>` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Add child epic to `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic ([introduced in GitLab 12.0](https://gitlab.com/gitlab-org/gitlab/-/issues/7330)). | -| `/clear_health_status` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Clear [health status](issues/managing_issues.md#health-status) ([introduced in GitLab 14.7](https://gitlab.com/gitlab-org/gitlab/-/issues/213814)). | +| `/child_epic <epic>` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Add child epic to `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7330) in GitLab 12.0). | +| `/clear_health_status` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Clear [health status](issues/managing_issues.md#health-status) ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213814) in GitLab 14.7). | | `/clear_weight` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Clear weight. | | `/clone <path/to/project> [--with_notes]` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Clone the issue to given project, or the current one if no arguments are given ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9421) in GitLab 13.7). Copies as much data as possible as long as the target project contains equivalent labels, milestones, and so on. Does not copy comments or system notes unless `--with_notes` is provided as an argument. | | `/close` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Close. | @@ -68,47 +68,48 @@ threads. Some quick actions might not be available to all subscription tiers. | `/done` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Mark to do as done. | | `/draft` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Toggle the draft status. | | `/due <date>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set due date. Examples of valid `<date>` include `in 2 days`, `this Friday` and `December 31st`. | -| `/duplicate <#issue>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Close this issue and mark as a duplicate of another issue. **(FREE)** Also, mark both as related. | +| `/duplicate <#issue>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Close this issue and mark as a duplicate of another issue. Also, mark both as related. | | `/epic <epic>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add to epic `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic. | | `/estimate <time>` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Set time estimate. For example, `/estimate 1mo 2w 3d 4h 5m`. Learn more about [time tracking](time_tracking.md). | -| `/health_status <value>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set [health status](issues/managing_issues.md#health-status). Valid options for `<value>` are `on_track`, `needs_attention`, and `at_risk` ([introduced in GitLab 14.7](https://gitlab.com/gitlab-org/gitlab/-/issues/213814)). | +| `/health_status <value>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set [health status](issues/managing_issues.md#health-status). Valid options for `<value>` are `on_track`, `needs_attention`, and `at_risk` ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/213814) in GitLab 14.7). | | `/invite_email email1 email2` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add up to six email participants. This action is behind feature flag `issue_email_participants` and is not yet supported in issue templates. | -| `/iteration *iteration:"iteration name"` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set iteration. For example, to set the `Late in July` iteration: `/iteration *iteration:"Late in July"` ([introduced in GitLab 13.1](https://gitlab.com/gitlab-org/gitlab/-/issues/196795)). | +| `/iteration *iteration:"iteration name"` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set iteration. For example, to set the `Late in July` iteration: `/iteration *iteration:"Late in July"` ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/196795) in GitLab 13.1). | | `/label ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Add one or more labels. Label names can also start without a tilde (`~`), but mixed syntax is not supported. | | `/lock` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Lock the discussions. | | `/merge` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Merge changes. Depending on the project setting, this may be [when the pipeline succeeds](merge_requests/merge_when_pipeline_succeeds.md), or adding to a [Merge Train](../../ci/pipelines/merge_trains.md). | | `/milestone %milestone` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Set milestone. | -| `/move <path/to/project>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Move this issue to another project. Be careful when moving an issue to a project with different access rules. Before moving the issue, make sure it does not contain sensitive data. | -| `/parent_epic <epic>` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Set parent epic to `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic ([introduced in GitLab 12.1](https://gitlab.com/gitlab-org/gitlab/-/issues/10556)). | +| `/move <path/to/project>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Move this issue to another project. Be careful when moving an issue to a project with different access rules. Before moving the issue, make sure it does not contain sensitive data. | +| `/parent_epic <epic>` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Set parent epic to `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10556) in GitLab 12.1). | | `/promote` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Promote issue to epic. | -| `/promote_to_incident` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Promote issue to incident ([introduced in GitLab 14.5](https://gitlab.com/gitlab-org/gitlab/-/issues/296787)). | -| `/publish` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Publish issue to an associated [Status Page](../../operations/incident_management/status_page.md) ([Introduced in GitLab 13.0](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/30906)) | +| `/promote_to_incident` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Promote issue to incident ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/296787) in GitLab 14.5). | +| `/page <policy name>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Start escalations for the incident ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/79977) in GitLab 14.9). | +| `/publish` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Publish issue to an associated [Status Page](../../operations/incident_management/status_page.md) ([Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/30906) in GitLab 13.0) | | `/reassign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Replace current assignees with those specified. | | `/reassign_reviewer @user1 @user2` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Replace current reviewers with those specified. | | `/rebase` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Rebase source branch. This schedules a background task that attempts to rebase the changes in the source branch on the latest commit of the target branch. If `/rebase` is used, `/merge` is ignored to avoid a race condition where the source branch is merged or deleted before it is rebased. If there are merge conflicts, GitLab displays a message that a rebase cannot be scheduled. Rebase failures are displayed with the merge request status. | | `/relabel ~label1 ~label2` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Replace current labels with those specified. | | `/relate #issue1 #issue2` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Mark issues as related. | -| `/remove_child_epic <epic>` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Remove child epic from `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic ([introduced in GitLab 12.0](https://gitlab.com/gitlab-org/gitlab/-/issues/7330)). | -| `/remove_contacts [contact:email1@example.com] [contact:email2@example.com]` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove one or more [CRM contacts](../crm/index.md) ([introduced in GitLab 14.6](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/73413)). | +| `/remove_child_epic <epic>` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Remove child epic from `<epic>`. The `<epic>` value should be in the format of `&epic`, `group&epic`, or a URL to an epic ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7330) in GitLab 12.0). | +| `/remove_contacts [contact:email1@example.com] [contact:email2@example.com]` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove one or more [CRM contacts](../crm/index.md) ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/73413) in GitLab 14.6). | | `/remove_due_date` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove due date. | | `/remove_epic` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove from epic. | | `/remove_estimate` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Remove time estimate. | -| `/remove_iteration` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove iteration ([introduced in GitLab 13.1](https://gitlab.com/gitlab-org/gitlab/-/issues/196795)). | +| `/remove_iteration` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove iteration ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/196795) in GitLab 13.1). | | `/remove_milestone` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Remove milestone. | -| `/remove_parent_epic` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Remove parent epic from epic ([introduced in GitLab 12.1](https://gitlab.com/gitlab-org/gitlab/-/issues/10556)). | +| `/remove_parent_epic` | **{dotted-circle}** No | **{dotted-circle}** No | **{check-circle}** Yes | Remove parent epic from epic ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/10556) in GitLab 12.1). | | `/remove_time_spent` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Remove time spent. | -| `/remove_zoom` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove Zoom meeting from this issue ([introduced in GitLab 12.4](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/16609)). | +| `/remove_zoom` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Remove Zoom meeting from this issue ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/16609) in GitLab 12.4). | | `/reopen` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Reopen. | -| `/severity <severity>` | **{check-circle}** Yes | **{check-circle}** No | **{check-circle}** No | Set the severity. Options for `<severity>` are `S1` ... `S4`, `critical`, `high`, `medium`, `low`, `unknown`. [Introduced in GitLab 14.2](https://gitlab.com/gitlab-org/gitlab/-/issues/334045). | +| `/severity <severity>` | **{check-circle}** Yes | **{check-circle}** No | **{check-circle}** No | Set the severity. Options for `<severity>` are `S1` ... `S4`, `critical`, `high`, `medium`, `low`, `unknown`. [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334045) in GitLab 14.2. | | `/shrug <comment>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Append the comment with `¯\＿(ツ)＿/¯`. | | `/spend <time> [<date>]` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Add or subtract spent time. Optionally, specify the date that time was spent on. For example, `/spend 1mo 2w 3d 4h 5m 2018-08-26` or `/spend -1h 30m`. Learn more about [time tracking](time_tracking.md). | -| `/submit_review` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Submit a pending review ([introduced in GitLab 12.7](https://gitlab.com/gitlab-org/gitlab/-/issues/8041)). | +| `/submit_review` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Submit a pending review ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/8041) in GitLab 12.7). | | `/subscribe` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Subscribe to notifications. | | `/tableflip <comment>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Append the comment with `(╯°□°)╯︵ ┻━┻`. | | `/target_branch <local branch name>` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Set target branch. | | `/title <new title>` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Change title. | | `/todo` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Add a to-do item. | -| `/unapprove` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Unapprove the merge request. ([introduced in GitLab 14.3](https://gitlab.com/gitlab-org/gitlab/-/issues/8103) | +| `/unapprove` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Unapprove the merge request. ([introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/8103) in GitLab 14.3 | | `/unassign @user1 @user2` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Remove specific assignees. | | `/unassign` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Remove all assignees. | | `/unassign_reviewer @user1 @user2` or `/remove_reviewer @user1 @user2` | **{dotted-circle}** No | **{check-circle}** Yes | **{dotted-circle}** No | Remove specific reviewers. | @@ -118,7 +119,7 @@ threads. Some quick actions might not be available to all subscription tiers. | `/unlock` | **{check-circle}** Yes | **{check-circle}** Yes | **{dotted-circle}** No | Unlock the discussions. | | `/unsubscribe` | **{check-circle}** Yes | **{check-circle}** Yes | **{check-circle}** Yes | Unsubscribe from notifications. | | `/weight <value>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Set weight. Valid options for `<value>` include `0`, `1`, `2`, and so on. | -| `/zoom <Zoom URL>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add Zoom meeting to this issue ([introduced in GitLab 12.4](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/16609)). | +| `/zoom <Zoom URL>` | **{check-circle}** Yes | **{dotted-circle}** No | **{dotted-circle}** No | Add Zoom meeting to this issue ([introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/16609) in GitLab 12.4). | ## Commit messages diff --git a/doc/user/project/releases/index.md b/doc/user/project/releases/index.md index 8049ee9726d..26b36198bde 100644 --- a/doc/user/project/releases/index.md +++ b/doc/user/project/releases/index.md @@ -42,7 +42,7 @@ To view a list of releases: - On the left sidebar, select **Deployments > Releases**, or -- On the project's overview page, if at least one release exists, click the number of releases. +- On the project's overview page, if at least one release exists, select the number of releases.  @@ -52,10 +52,10 @@ To view a list of releases: ### Sort releases -To sort releases by **Released date** or **Created date**, select from the sort order dropdown. To +To sort releases by **Released date** or **Created date**, select from the sort order dropdown list. To switch between ascending or descending order, select **Sort order**. - + ## Create a release @@ -307,9 +307,9 @@ Read more about [Release permissions](#release-permissions). To edit the details of a release: 1. On the left sidebar, select **Deployments > Releases**. -1. In the top-right corner of the release you want to modify, click **Edit this release** (the pencil icon). +1. In the top-right corner of the release you want to modify, select **Edit this release** (the pencil icon). 1. On the **Edit Release** page, change the release's details. -1. Click **Save changes**. +1. Select **Save changes**. You can edit the release title, notes, associated milestones, and asset links. To change the release date use the @@ -330,16 +330,16 @@ the [Releases API](../../../api/releases/index.md#create-a-release). In the user interface, to associate milestones to a release: 1. On the left sidebar, select **Deployments > Releases**. -1. In the top-right corner of the release you want to modify, click **Edit this release** (the pencil icon). +1. In the top-right corner of the release you want to modify, select **Edit this release** (the pencil icon). 1. From the **Milestones** list, select each milestone you want to associate. You can select multiple milestones. -1. Click **Save changes**. +1. Select **Save changes**. On the **Deployments > Releases** page, the **Milestone** is listed in the top section, along with statistics about the issues in the milestones.  -Releases are also visible on the **Issues > Milestones** page, and when you click a milestone +Releases are also visible on the **Issues > Milestones** page, and when you select a milestone on this page. Here is an example of milestones with no releases, one release, and two releases, respectively. @@ -360,8 +360,8 @@ You can be notified by email when a new release is created for your project. To subscribe to notifications for releases: 1. On the left sidebar, select **Project information**. -1. Click **Notification setting** (the bell icon). -1. In the list, click **Custom**. +1. Select **Notification setting** (the bell icon). +1. In the list, select **Custom**. 1. Select the **New release** checkbox. 1. Close the dialog box to save. @@ -395,12 +395,12 @@ To set a deploy freeze window in the UI, complete these steps: 1. Sign in to GitLab as a user with the Maintainer role. 1. On the left sidebar, select **Project information**. -1. In the left navigation menu, navigate to **Settings > CI/CD**. +1. In the left navigation menu, go to **Settings > CI/CD**. 1. Scroll to **Deploy freezes**. -1. Click **Expand** to see the deploy freeze table. -1. Click **Add deploy freeze** to open the deploy freeze modal. +1. Select **Expand** to see the deploy freeze table. +1. Select **Add deploy freeze** to open the deploy freeze modal. 1. Enter the start time, end time, and time zone of the desired deploy freeze period. -1. Click **Add deploy freeze** in the modal. +1. Select **Add deploy freeze** in the modal. 1. After the deploy freeze is saved, you can edit it by selecting the edit button (**{pencil}**) and remove it by selecting the delete button (**{remove}**).  @@ -468,6 +468,28 @@ Each link as an asset has the following attributes: | `filepath` | The redirect link to the `url`. See [this section](#permanent-links-to-release-assets) for more information. | No | | `link_type` | The content kind of what users can download via `url`. See [this section](#link-types) for more information. | No | +##### Permanent link to latest release + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16821) in GitLab 14.9. + +Latest release page is accessible through a permanent URL. +GitLab will redirect to the latest release page URL when it is visited. + +The format of the URL is: + +```plaintext +https://host/namespace/project/-/releases/permalink/latest +``` + +We also support, suffix path carry forward on the redirect to the latest release. +Example if release `v14.8.0-ee` is the latest release and has a readable link `https://host/namespace/project/-/releases/v14.8.0-ee#release` then it can be addressed as `https://host/namespace/project/-/releases/permalink/latest#release`. + +Refer [permanent links to latest release assets](#permanent-links-to-latest-release-assets) section to understand more about the suffix path carry forward usage. + +###### Sorting preferences + +By default, GitLab fetches the release using `released_at` time. The use of the query parameter `?order_by=released_at` is optional, and support for `?order_by=semver` is tracked [in this issue](https://gitlab.com/gitlab-org/gitlab/-/issues/352945). + ##### Permanent links to release assets > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/27300) in GitLab 12.9. @@ -475,7 +497,7 @@ Each link as an asset has the following attributes: The assets associated with a release are accessible through a permanent URL. GitLab always redirects this URL to the actual asset location, so even if the assets move to a different location, you can continue -to use the same URL. This is defined during [link creation](../../../api/releases/links.md#create-a-link) or [updating](../../../api/releases/links.md#update-a-link). +to use the same URL. This is defined during [link creation](../../../api/releases/links.md#create-a-link) or [updating](../../../api/releases/links.md#update-a-link) using the `filepath` API attribute. The format of the URL is: @@ -503,6 +525,36 @@ https://gitlab.com/gitlab-org/gitlab-runner/-/releases/v11.9.0-rc2/downloads/bin The physical location of the asset can change at any time and the direct link remains unchanged. +##### Permanent links to latest release assets + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/16821) in GitLab 14.9. + +The `filepath` from [permanent links to release assets](#permanent-links-to-release-assets) can be used in combination with [permanent link to the latest release](#permanent-link-to-latest-release). It is useful when we want to link a permanant URL to download an asset from the *latest release*. + +The format of the URL is: + +```plaintext +https://host/namespace/project/-/releases/permalink/latest/downloads/:filepath +``` + +If you have an asset with [`filepath`](../../../api/releases/links.md#create-a-link) for the `v11.9.0-rc2` latest release in the `gitlab-org` +namespace and `gitlab-runner` project on `gitlab.com`, for example: + +```json +{ + "name": "linux amd64", + "filepath": "/binaries/gitlab-runner-linux-amd64", + "url": "https://gitlab-runner-downloads.s3.amazonaws.com/v11.9.0-rc2/binaries/gitlab-runner-linux-amd64", + "link_type": "other" +} +``` + +This asset has a direct link of: + +```plaintext +https://gitlab.com/gitlab-org/gitlab-runner/-/releases/permalink/latest/downloads/binaries/gitlab-runner-linux-amd64 +``` + ##### Link Types > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/207257) in GitLab 13.1. @@ -611,7 +663,7 @@ On [GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/releases), you can view t  The totals are displayed on [shields](https://shields.io/) and are generated per release by -[a Rake task in the `www-gitlab-com` repo](https://gitlab.com/gitlab-com/www-gitlab-com/-/blob/master/lib/tasks/update_gitlab_project_releases_page.rake). +[a Rake task in the `www-gitlab-com` repository](https://gitlab.com/gitlab-com/www-gitlab-com/-/blob/master/lib/tasks/update_gitlab_project_releases_page.rake). | Item | Formula | | ------ | ------ | @@ -634,7 +686,7 @@ This data is saved in a JSON file and called *release evidence*. The feature includes test artifacts and linked milestones to facilitate internal processes, like external audits. -To access the release evidence, on the Releases page, click the link to the JSON file that's listed +To access the release evidence, on the Releases page, select the link to the JSON file that's listed under the **Evidence collection** heading. You can also [use the API](../../../api/releases/index.md#collect-release-evidence) to @@ -720,7 +772,7 @@ When you create a release, if [job artifacts](../../../ci/yaml/index.md#artifact Although job artifacts normally expire, artifacts included in release evidence do not expire. -To enable job artifact collection you need to specify both: +To enable job artifact collection you must specify both: 1. [`artifacts:paths`](../../../ci/yaml/index.md#artifactspaths) 1. [`artifacts:reports`](../../../ci/yaml/index.md#artifactsreports) diff --git a/doc/user/project/repository/branches/default.md b/doc/user/project/repository/branches/default.md index d706a0e8f8a..f9fd1a48b9a 100644 --- a/doc/user/project/repository/branches/default.md +++ b/doc/user/project/repository/branches/default.md @@ -26,7 +26,7 @@ using the GitLab default only if no customizations are set: 1. A [project-specific](#change-the-default-branch-name-for-a-project) custom default branch name. 1. A [subgroup-level](#group-level-custom-initial-branch-name) custom default branch name. 1. A [group-level](#group-level-custom-initial-branch-name) custom default branch name. -1. An [instance-level](#instance-level-custom-initial-branch-name) custom default branch name. **(FREE SELF)** +1. An [instance-level](#instance-level-custom-initial-branch-name) custom default branch name. 1. If no custom default branch name is set at any level, GitLab defaults to: - `main`: Projects created with GitLab 14.0 or later. - `master`: Projects created before GitLab 14.0. @@ -81,13 +81,83 @@ overrides it. Users with at least the Owner role of groups and subgroups can configure the default branch name for a group: 1. Go to the group **Settings > Repository**. -1. Expand **Default initial branch name**. +1. Expand **Default branch**. 1. Change the default initial branch to a custom name of your choice. 1. Select **Save changes**. Projects created in this group after you change the setting use the custom branch name, unless a subgroup configuration overrides it. +## Protect initial default branches **(FREE SELF)** + +GitLab administrators and group owners can define [branch protections](../../../project/protected_branches.md) +to apply to every repository's [default branch](#default-branch) +at the [instance level](#instance-level-default-branch-protection) and +[group level](#group-level-default-branch-protection) with one of the following options: + +- **Not protected** - Both developers and maintainers can push new commits + and force push. +- **Protected against pushes** - Developers cannot push new commits, but are + allowed to accept merge requests to the branch. Maintainers can push to the branch. +- **Partially protected** - Both developers and maintainers can push new commits, + but cannot force push. +- **Fully protected** - Developers cannot push new commits, but maintainers can. + No one can force push. + +### Instance-level default branch protection **(FREE SELF)** + +This setting applies only to each repository's default branch. To protect other branches, +you must either: + +- Configure [branch protection in the repository](../../../project/protected_branches.md). +- Configure [branch protection for groups](../../../group/index.md#change-the-default-branch-protection-of-a-group). + +Administrators of self-managed instances can customize the initial default branch protection for projects hosted on that instance. Individual +groups and subgroups can override this instance-wide setting for their projects. + +1. On the top bar, select **Menu > Admin**. +1. On the left sidebar, select **Settings > Repository**. +1. Expand **Default branch**. +1. Select [**Initial default branch protection**](#protect-initial-default-branches). +1. To allow group owners to override the instance's default branch protection, select + [**Allow owners to manage default branch protection per group**](#prevent-overrides-of-default-branch-protection). +1. Select **Save changes**. + +#### Prevent overrides of default branch protection **(PREMIUM SELF)** + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/211944) in GitLab 13.0. + +Instance-level protections for default branches +can be overridden on a per-group basis by the group's owner. In +[GitLab Premium or higher](https://about.gitlab.com/pricing/), GitLab administrators can +disable this privilege for group owners, enforcing the instance-level protection rule: + +1. On the top bar, select **Menu > Admin**. +1. On the left sidebar, select **Settings > Repository**. +1. Expand the **Default branch** section. +1. Clear the **Allow owners to manage default branch protection per group** checkbox. +1. Select **Save changes**. + +NOTE: +GitLab administrators can still update the default branch protection of a group. + +### Group-level default branch protection **(PREMIUM)** + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/7583) in GitLab 12.9. +> - [Settings moved and renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/340403) in GitLab 14.9. + +Instance-level protections for [default branch](#default-branch) +can be overridden on a per-group basis by the group's owner. In +[GitLab Premium or higher](https://about.gitlab.com/pricing/), GitLab administrators can +[enforce protection of initial default branches](#prevent-overrides-of-default-branch-protection) +which locks this setting for group owners. + +1. On the top bar, select **Menu > Groups** and find your group. +1. On the left sidebar, select **Settings > Repository**. +1. Expand **Default branch**. +1. Select [**Initial default branch protection**](#protect-initial-default-branches). +1. Select **Save changes**. + ## Update the default branch name in your repository WARNING: diff --git a/doc/user/project/repository/forking_workflow.md b/doc/user/project/repository/forking_workflow.md index b085372c8f2..ddeef65a6b5 100644 --- a/doc/user/project/repository/forking_workflow.md +++ b/doc/user/project/repository/forking_workflow.md @@ -18,36 +18,24 @@ submit them through a merge request to the repository you don't have access to. ## Creating a fork -To fork an existing project in GitLab: - -1. On the project's home page, in the top right, click **{fork}** **Fork**. - -  - -1. Select the project to fork to: +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15013) a new form in GitLab 13.11 [with a flag](../../../user/feature_flags.md) named `fork_project_form`. Disabled by default. +> - [Enabled on GitLab.com and self-managed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/77181) in GitLab 14.8. Feature flag `fork_project_form` removed. - - Recommended method. Below **Select a namespace to fork the project**, identify - the project you want to fork to, and click **Select**. Only namespaces where you have - at least the Developer role for are shown. - -  - - - Experimental method. If your GitLab administrator has - enabled the experimental fork project form, read - [Create a fork with the fork project form](#create-a-fork-with-the-fork-project-form). - Only namespaces where you have at least the Developer role for are shown. - - NOTE: - The project path must be unique in the namespace. +To fork an existing project in GitLab: -GitLab creates your fork, and redirects you to the project page for your new fork. -The permissions you have in the namespace are your permissions in the fork. +1. On the project's home page, in the top right, select **{fork}** **Fork**: +  +1. Optional. Edit the **Project name**. +1. For **Project URL**, select the [namespace](../../group/index.md#namespaces) + your fork should belong to. +1. Add a **Project slug**. This value becomes part of the URL to your fork. + It must be unique in the namespace. +1. Optional. Add a **Project description**. +1. Select the **Visibility level** for your fork. For more information about + visibility levels, read [Project and group visibility](../../../public_access/public_access.md). +1. Select **Fork project**. -WARNING: -When a public project with the repository feature set to **Members Only** -is forked, the repository is public in the fork. The owner -of the fork must manually change the visibility. Issue -[#36662](https://gitlab.com/gitlab-org/gitlab/-/issues/36662) exists for this issue. +GitLab creates your fork, and redirects you to the new fork's page. ## Repository mirroring @@ -59,7 +47,7 @@ Without mirroring, to work locally you must use `git pull` to update your local with the upstream project, then push the changes back to your fork to update it. WARNING: -With mirroring, before approving a merge request, you are asked to sync. Because of this, automating it is recommended. +With mirroring, before approving a merge request, you are asked to sync. We recommend you automate it. Read more about [How to keep your fork up to date with its origin](https://about.gitlab.com/blog/2016/12/01/how-to-keep-your-fork-up-to-date-with-its-origin/). @@ -75,30 +63,9 @@ When creating a merge request, if the forked project's visibility is more restri  Then you can add labels, a milestone, and assign the merge request to someone who can review -your changes. Then click **Submit merge request** to conclude the process. When successfully merged, your +your changes. Then select **Submit merge request** to conclude the process. When successfully merged, your changes are added to the repository and branch you're merging into. ## Removing a fork relationship You can unlink your fork from its upstream project in the [advanced settings](../settings/index.md#removing-a-fork-relationship). - -## Create a fork with the fork project form **(FREE SELF)** - -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/15013) in GitLab 13.11 [with a flag](../../../administration/feature_flags.md) named `fork_project_form`. Disabled by default. -> - [Enabled on self-managed and GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/64967) in GitLab 13.8. - -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../../../administration/feature_flags.md) named `fork_project_form`. -On GitLab.com, this feature is available. - -This version of the fork project form is experimental: - - - -To use it, follow the instructions at [Creating a fork](#creating-a-fork) and provide: - -- The project name. -- The project URL. -- The project slug. -- Optional. The project description. -- The visibility level for your fork. diff --git a/doc/user/project/repository/gpg_signed_commits/img/profile_settings_gpg_keys_paste_pub.png b/doc/user/project/repository/gpg_signed_commits/img/profile_settings_gpg_keys_paste_pub.png Binary files differdeleted file mode 100644 index 6e2ff33eebb..00000000000 --- a/doc/user/project/repository/gpg_signed_commits/img/profile_settings_gpg_keys_paste_pub.png +++ /dev/null diff --git a/doc/user/project/repository/gpg_signed_commits/index.md b/doc/user/project/repository/gpg_signed_commits/index.md index f5cea8a8075..00998c9a227 100644 --- a/doc/user/project/repository/gpg_signed_commits/index.md +++ b/doc/user/project/repository/gpg_signed_commits/index.md @@ -6,129 +6,96 @@ info: To determine the technical writer assigned to the Stage/Group associated w # Signing commits with GPG **(FREE)** -You can use a GPG key to sign Git commits made in a GitLab repository. Signed -commits are labeled **Verified** if the identity of the committer can be -verified. To verify the identity of a committer, GitLab requires their public -GPG key. +You can sign the commits you make in a GitLab repository with a +GPG ([GNU Privacy Guard](https://gnupg.org/)) key. When you add a cryptographic +signature to your commit, you provide extra assurance that a commit +originated from you, rather than an impersonator. If GitLab can verify a commit +author's identity with a public GPG key, the commit is marked **Verified** in the +GitLab UI. You can then configure [push rules](../push_rules.md#enabling-push-rules) +for your project to reject individual commits not signed with GPG, or reject all +commits from unverified users. NOTE: -The term GPG is used for all OpenPGP/PGP/GPG related material and +GitLab uses the term GPG for all OpenPGP, PGP, and GPG-related material and implementations. -To view a user's public GPG key, you can: - -- Go to `https://gitlab.example.com/<username>.gpg`. -- Select **View public GPG keys** (**{key}**) in the top right of the user's profile. - -GPG verified tags are not supported yet. - -See the [further reading](#further-reading) section for more details on GPG. - -## How GitLab handles GPG - -GitLab uses its own keyring to verify the GPG signature. It does not access any -public key server. - -For a commit to be verified by GitLab: +For GitLab to consider a commit verified: - The committer must have a GPG public/private key pair. -- The committer's public key must have been uploaded to their GitLab - account. -- One of the emails in the GPG key must match a **verified** email address - used by the committer in GitLab. This address will be part of the public key. - If you want to keep this address private, use the automatically generated +- The committer's public key must be uploaded to their GitLab account. +- One of the email addresses in the GPG public key must match a **verified** email address + used by the committer in GitLab. To keep this address private, use the automatically generated [private commit email address](../../../profile/index.md#use-an-automatically-generated-private-commit-email) GitLab provides in your profile. - The committer's email address must match the verified email address from the GPG key. -## Generating a GPG key - -If you don't already have a GPG key, the following steps can help you get -started: - -1. [Install GPG](https://www.gnupg.org/download/index.html) for your operating system. - If your operating system has `gpg2` installed, replace `gpg` with `gpg2` in - the following commands. -1. Generate the private/public key pair with the command appropriate for your version - of `gpg`. This command spawns a series of questions: - - ```shell - # Use this command for the default version of gpg, including - # Gpg4win on Windows, and most macOS versions: - gpg --gen-key +GitLab uses its own keyring to verify the GPG signature. It does not access any +public key server. - # Use this command for versions of GPG later than 2.1.17: - gpg --full-gen-key - ``` +GPG verified tags are not supported. -1. The first question is which algorithm can be used. Select the kind you want - or press <kbd>Enter</kbd> to choose the default (RSA and RSA): +For more details about GPG, refer to the [related topics list](#related-topics). - ```plaintext - Please select what kind of key you want: - (1) RSA and RSA (default) - (2) DSA and Elgamal - (3) DSA (sign only) - (4) RSA (sign only) - Your selection? 1 - ``` +## View a user's public GPG key -1. The next question is key length. We recommend you choose `4096`: +To view a user's public GPG key, you can either: - ```plaintext - RSA keys may be between 1024 and 4096 bits long. - What keysize do you want? (2048) 4096 - Requested keysize is 4096 bits - ``` +- Go to `https://gitlab.example.com/<USERNAME>.gpg`. GitLab displays the GPG key, + if the user has configured one, or a blank page for users without a configured GPG key. +- Go to the user's profile (such as `https://gitlab.example.com/<USERNAME>`). In the top right + of the user's profile, select **View public GPG keys** (**{key}**). -1. Specify the validity period of your key. This is something - subjective, and you can use the default value, which is to never expire: +## Configure commit signing - ```plaintext - Please specify how long the key should be valid. - 0 = key does not expire - <n> = key expires in n days - <n>w = key expires in n weeks - <n>m = key expires in n months - <n>y = key expires in n years - Key is valid for? (0) 0 - Key does not expire at all - ``` +To sign commits, you must configure both your local machine and your GitLab account: -1. Confirm that the answers you gave were correct by typing `y`: +1. [Create a GPG key](#create-a-gpg-key). +1. [Add a GPG key to your account](#add-a-gpg-key-to-your-account). +1. [Associate your GPG key with Git](#associate-your-gpg-key-with-git). +1. [Sign your Git commits](#sign-your-git-commits). - ```plaintext - Is this correct? (y/N) y - ``` +### Create a GPG key -1. Enter your real name, the email address to be associated with this key - (should match a verified email address you use in GitLab) and an optional - comment (press <kbd>Enter</kbd> to skip): +If you don't already have a GPG key, create one: - ```plaintext - GnuPG needs to construct a user ID to identify your key. +1. [Install GPG](https://www.gnupg.org/download/) for your operating system. + If your operating system has `gpg2` installed, replace `gpg` with `gpg2` in + the commands on this page. +1. To generate your key pair, run the command appropriate for your version of `gpg`: - Real name: Mr. Robot - Email address: <your_email> - Comment: - You selected this USER-ID: - "Mr. Robot <your_email>" + ```shell + # Use this command for the default version of GPG, including + # Gpg4win on Windows, and most macOS versions: + gpg --gen-key - Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O + # Use this command for versions of GPG later than 2.1.17: + gpg --full-gen-key ``` -1. Pick a strong password when asked and type it twice to confirm. -1. Use the following command to list the private GPG key you just created: +1. Select the algorithm your key should use, or press <kbd>Enter</kbd> to select + the default option, `RSA and RSA`. +1. Select the key length, in bits. GitLab recommends 4096-bit keys. +1. Specify the validity period of your key. This value is subjective, and the + default value is no expiration. +1. To confirm your answers, enter `y`. +1. Enter your name. +1. Enter your email address. It must match a + [verified email address](../../../profile/index.md#change-the-email-displayed-on-your-commits) + in your GitLab account. +1. Optional. Enter a comment to display in parentheses after your name. +1. GPG displays the information you've entered so far. Edit the information or press + <kbd>O</kbd> (for `Okay`) to continue. +1. Enter a strong password, then enter it again to confirm it. +1. To list your private GPG key, run this command, replacing + `<EMAIL>` with the email address you used when you generated the key: ```shell - gpg --list-secret-keys --keyid-format LONG <your_email> + gpg --list-secret-keys --keyid-format LONG <EMAIL> ``` - Replace `<your_email>` with the email address you entered above. - -1. Copy the GPG key ID that starts with `sec`. In the following example, that's - `30F2B65B9246B6CA`: +1. In the output, identify the `sec` line, and copy the GPG key ID. It begins after + the `/` character. In this example, the key ID is `30F2B65B9246B6CA`: ```plaintext sec rsa4096/30F2B65B9246B6CA 2017-08-18 [SC] @@ -137,49 +104,46 @@ started: ssb rsa4096/B7ABC0813E4028C0 2017-08-18 [E] ``` -1. Export the public key of that ID (replace your key ID from the previous step): +1. To show the associated public key, run this command, replacing `<ID>` with the + GPG key ID from the previous step: ```shell - gpg --armor --export 30F2B65B9246B6CA + gpg --armor --export <ID> ``` -1. Finally, copy the public key and [add it in your user settings](#adding-a-gpg-key-to-your-account) +1. Copy the public key, including the `BEGIN PGP PUBLIC KEY BLOCK` and + `END PGP PUBLIC KEY BLOCK` lines. You need this key in the next step. -## Adding a GPG key to your account +### Add a GPG key to your account -NOTE: -After you add a key, you cannot edit it, only remove it. In case the paste -didn't work, you have to remove the offending key and re-add it. - -You can add a GPG key in your user settings: +To add a GPG key to your user settings: +1. Sign in to GitLab. 1. In the top-right corner, select your avatar. 1. Select **Edit profile**. -1. On the left sidebar, select **GPG Keys**. -1. Paste your _public_ key in the **Key** text box. - -  - -1. Select **Add key** to add it to GitLab. You can see the key's fingerprint, the corresponding - email address, and creation date. +1. On the left sidebar, select **GPG Keys** (**{key}**). +1. In **Key**, paste your _public_ key. +1. To add the key to your account, select **Add key**. GitLab shows the key's + fingerprint, email address, and creation date:  -## Associating your GPG key with Git +After you add a key, you cannot edit it. Instead, remove the offending key and re-add it. -After you have [created your GPG key](#generating-a-gpg-key) and [added it to -your account](#adding-a-gpg-key-to-your-account), it's time to tell Git which -key to use. +### Associate your GPG key with Git -1. Use the following command to list the private GPG key you just created: +After you [create your GPG key](#create-a-gpg-key) and +[add it to your account](#add-a-gpg-key-to-your-account), you must configure Git +to use this key: + +1. Run this command to list the private GPG key you just created, + replacing `<EMAIL>` with the email address for your key: ```shell - gpg --list-secret-keys --keyid-format LONG <your_email> + gpg --list-secret-keys --keyid-format LONG <EMAIL> ``` - Replace `<your_email>` with the email address you entered above. - -1. Copy the GPG key ID that starts with `sec`. In the following example, that's +1. Copy the GPG private key ID that starts with `sec`. In this example, the private key ID is `30F2B65B9246B6CA`: ```plaintext @@ -189,114 +153,103 @@ key to use. ssb rsa4096/B7ABC0813E4028C0 2017-08-18 [E] ``` -1. Tell Git to use that key to sign the commits: +1. Run this command to configure Git to sign your commits with your key, + replacing `<KEY ID>` with your GPG key ID: ```shell - git config --global user.signingkey 30F2B65B9246B6CA + git config --global user.signingkey <KEY ID> ``` - Replace `30F2B65B9246B6CA` with your GPG key ID. - -1. Optional. If Git is using `gpg` and you get errors like `secret key not available` - or `gpg: signing failed: secret key not available`, run the following command to - change to `gpg2`: +1. Optional. If Git uses `gpg` and you get errors like `secret key not available` + or `gpg: signing failed: secret key not available`, run this command to + use `gpg2` instead: ```shell git config --global gpg.program gpg2 ``` -## Signing commits +### Sign your Git commits -After you have [created your GPG key](#generating-a-gpg-key) and [added it to -your account](#adding-a-gpg-key-to-your-account), you can start signing your -commits: +After you [add your public key to your account](#add-a-gpg-key-to-your-account), +you can sign individual commits manually, or configure Git to default to signed commits: -1. Commit like you used to, the only difference is the addition of the `-S` flag: +- Sign individual Git commits manually: + 1. Add `-S` flag to any commit you want to sign: - ```shell - git commit -S -m "My commit msg" - ``` + ```shell + git commit -S -m "My commit message" + ``` -1. Enter the passphrase of your GPG key when asked. -1. Push to GitLab and check that your commits [are verified](#verifying-commits). + 1. Enter the passphrase of your GPG key when asked. + 1. Push to GitLab and check that your commits [are verified](#verify-commits). +- Sign all Git commits by default by running this command: -If you don't want to type the `-S` flag every time you commit, you can tell Git -to sign your commits automatically: + ```shell + git config --global commit.gpgsign true + ``` -```shell -git config --global commit.gpgsign true -``` +## Verify commits -## Verifying commits +You can review commits for a merge request, or for an entire project: -1. Within a project or [merge request](../../merge_requests/index.md), navigate to - the **Commits** tab. Signed commits show a badge containing either - **Verified** or **Unverified**, depending on the verification status of the GPG - signature. +1. To review commits for a project: + 1. On the top bar, select **Menu > Projects** and find your project. + 1. On the left sidebar, select **Repository > Commits**. +1. To review commits for a merge request: + 1. On the top bar, select **Menu > Projects** and find your project. + 1. On the left sidebar, select **Merge requests**, then select your merge request. + 1. Select **Commits**. +1. Identify the commit you want to review. Signed commits show either a **Verified** + or **Unverified** badge, depending on the verification status of the GPG + signature. Unsigned commits do not display a badge:  -1. By clicking on the GPG badge, details of the signature are displayed. +1. To display the signature details for a commit, select the GPG badge:  -  +  -## Revoking a GPG key +## Revoke a GPG key -Revoking a key **unverifies** already signed commits. Commits that were -verified by using this key changes to an unverified state. Future commits -stay unverified after you revoke this key. This action should be used -in case your key has been compromised. +If a GPG key becomes compromised, revoke it. Revoking a key changes both future and past commits: + +- Past commits signed by this key are marked as unverified. +- Future commits signed by this key are marked as unverified. To revoke a GPG key: 1. In the top-right corner, select your avatar. 1. Select **Edit profile**. -1. On the left sidebar, select **GPG Keys**. +1. On the left sidebar, select **GPG Keys** (**{key}**). 1. Select **Revoke** next to the GPG key you want to delete. -## Removing a GPG key +## Remove a GPG key + +When you remove a GPG key from your GitLab account: -Removing a key **does not unverify** already signed commits. Commits that were -verified by using this key stay verified. Only unpushed commits stay -unverified after you remove this key. To unverify already signed commits, you need -to [revoke the associated GPG key](#revoking-a-gpg-key) from your account. +- Previous commits signed with this key remain verified. +- Future commits (including any commits created but not yet pushed) that attempt + to use this key are unverified. To remove a GPG key from your account: 1. In the top-right corner, select your avatar. 1. Select **Edit profile**. -1. On the left sidebar, select **GPG Keys**. -1. Select the trash icon (**{remove}**) next to the GPG key you want to delete. - -## Rejecting commits that are not signed **(PREMIUM)** - -You can configure your project to reject commits that aren't GPG-signed -via [push rules](../push_rules.md). - -## GPG signing API - -Learn how to [get the GPG signature from a commit via API](../../../../api/commits.md#get-gpg-signature-of-a-commit). - -## Further reading - -For more details about GPG, see: - -- [Git Tools - Signing Your Work](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work) -- [Managing OpenPGP Keys](https://riseup.net/en/security/message-security/openpgp/gpg-keys) -- [OpenPGP Best Practices](https://riseup.net/en/security/message-security/openpgp/best-practices) -- [Creating a new GPG key with subkeys](https://www.void.gr/kargig/blog/2013/12/02/creating-a-new-gpg-key-with-subkeys/) (advanced) -- [Review existing GPG keys in your instance](../../../admin_area/credentials_inventory.md#review-existing-gpg-keys) - -<!-- ## Troubleshooting - -Include any troubleshooting steps that you can foresee. If you know beforehand what issues -one might have when setting this up, or when something is changed, or on upgrading, it's -important to describe those, too. Think of things that may go wrong and include them here. -This is important to minimize requests for support, and to avoid doc comments with -questions that you know someone might ask. - -Each scenario can be a third-level heading, e.g. `### Getting error message X`. -If you have none to add when creating a doc, leave this section in place -but commented out to help encourage others to add to it in the future. --> +1. On the left sidebar, select **GPG Keys** (**{key}**). +1. Select **Remove** (**{remove}**) next to the GPG key you want to delete. + +If you must unverify both future and past commits, +[revoke the associated GPG key](#revoke-a-gpg-key) instead. + +## Related topics + +- [Sign commits and tags with X.509 certificates](../x509_signed_commits/index.md) +- [Commits API](../../../../api/commits.md) +- GPG resources: + - [Git Tools - Signing Your Work](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work) + - [Managing OpenPGP Keys](https://riseup.net/en/security/message-security/openpgp/gpg-keys) + - [OpenPGP Best Practices](https://riseup.net/en/security/message-security/openpgp/best-practices) + - [Creating a new GPG key with subkeys](https://www.void.gr/kargig/blog/2013/12/02/creating-a-new-gpg-key-with-subkeys/) (advanced) + - [Review existing GPG keys in your instance](../../../admin_area/credentials_inventory.md#review-existing-gpg-keys) diff --git a/doc/user/project/repository/img/fork_form_v13_10.png b/doc/user/project/repository/img/fork_form_v13_10.png Binary files differdeleted file mode 100644 index 00c2f89a844..00000000000 --- a/doc/user/project/repository/img/fork_form_v13_10.png +++ /dev/null diff --git a/doc/user/project/repository/img/forking_workflow_choose_namespace_v13_10.png b/doc/user/project/repository/img/forking_workflow_choose_namespace_v13_10.png Binary files differdeleted file mode 100644 index 74f65cb663d..00000000000 --- a/doc/user/project/repository/img/forking_workflow_choose_namespace_v13_10.png +++ /dev/null diff --git a/doc/user/project/repository/jupyter_notebooks/index.md b/doc/user/project/repository/jupyter_notebooks/index.md index 5646f478d9f..cd0c6679d8d 100644 --- a/doc/user/project/repository/jupyter_notebooks/index.md +++ b/doc/user/project/repository/jupyter_notebooks/index.md @@ -25,11 +25,8 @@ GitLab. ## Cleaner diffs -> [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6589) in GitLab 14.5 [with a flag](../../../../administration/feature_flags.md) named `jupyter_clean_diffs`. Enabled by default. - -FLAG: -On self-managed GitLab, by default this feature is available. To hide the feature, ask an administrator to [disable the feature flag](../../../../administration/feature_flags.md) named `jupyter_clean_diffs`. -On GitLab.com, this feature is available. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/6589) in GitLab 14.5 [with a flag](../../../../administration/feature_flags.md) named `jupyter_clean_diffs`. Enabled by default. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75500) in GitLab 14.9. Feature flag `jupyter_clean_diffs` removed. When commits include changes to Jupyter Notebook files, GitLab: diff --git a/doc/user/project/repository/mirror/index.md b/doc/user/project/repository/mirror/index.md index cddbc6fd891..c9fa30e39a8 100644 --- a/doc/user/project/repository/mirror/index.md +++ b/doc/user/project/repository/mirror/index.md @@ -96,9 +96,7 @@ Prerequisite: 1. Select **Update now** (**{retry}**):  -## Mirror only protected branches **(PREMIUM)** - -> Moved to GitLab Premium in 13.9. +## Mirror only protected branches You can choose to mirror only the [protected branches](../../protected_branches.md) in the mirroring project, diff --git a/doc/user/project/repository/mirror/push.md b/doc/user/project/repository/mirror/push.md index 221616bd41c..ebc4430ac53 100644 --- a/doc/user/project/repository/mirror/push.md +++ b/doc/user/project/repository/mirror/push.md @@ -76,7 +76,7 @@ through the [remote mirrors API](../../../../api/remote_mirrors.md). To configure a mirror from GitLab to GitHub: -1. Create a [GitHub personal access token](https://docs.github.com/en/github/authenticating-to-github/keeping-your-account-and-data-secure/creating-a-personal-access-token) +1. Create a [GitHub personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) with `public_repo` selected. 1. Enter a **Git repository URL** with this format: `https://<your_access_token>@github.com/<github_group>/<github_project>.git`. diff --git a/doc/user/project/repository/reducing_the_repo_size_using_git.md b/doc/user/project/repository/reducing_the_repo_size_using_git.md index 37b16d90d93..83fafd409e8 100644 --- a/doc/user/project/repository/reducing_the_repo_size_using_git.md +++ b/doc/user/project/repository/reducing_the_repo_size_using_git.md @@ -13,8 +13,6 @@ Git repositories become larger over time. When large files are added to a Git re - They take up a large amount of storage space on the server. - Git repository storage limits [can be reached](#storage-limits). -Such problems can be detected with [git-sizer](https://github.com/github/git-sizer#getting-started). - Rewriting a repository can remove unwanted history to make the repository smaller. We **recommend [`git filter-repo`](https://github.com/newren/git-filter-repo/blob/main/README.md)** over [`git filter-branch`](https://git-scm.com/docs/git-filter-branch) and @@ -40,7 +38,8 @@ These refs are not automatically downloaded and hidden refs are not advertised, To purge files from a GitLab repository: -1. [Install `git filter-repo`](https://github.com/newren/git-filter-repo/blob/main/INSTALL.md) +1. Install either [`git filter-repo`](https://github.com/newren/git-filter-repo/blob/main/INSTALL.md) or + [`git-sizer`](https://github.com/github/git-sizer#getting-started) using a supported package manager or from source. 1. Generate a fresh [export from the @@ -63,31 +62,43 @@ To purge files from a GitLab repository: git clone --bare --mirror /path/to/project.bundle ``` -1. Navigate to the `project.git` directory: +1. Go to the `project.git` directory: ```shell cd project.git ``` -1. Using `git filter-repo`, purge any files from the history of your repository. Because we are +1. Using either `git filter-repo` or `git-sizer`, analyze your repository + and review the results to determine which items you want to purge: + + ```shell + # Using git filter-repo + git filter-repo --analyze + head .git/filter-repo/analysis/*-{all,deleted}-sizes.txt + + # Using git-sizer + git-sizer + ``` + +1. Proceed to purging any files from the history of your repository. Because we are trying to remove internal refs, we rely on the `commit-map` produced by each run to tell us which internal refs to remove. NOTE: - `git filter-repo` creates a new `commit-map` file every run, and overwrite the `commit-map` from + `git filter-repo` creates a new `commit-map` file every run, and overwrites the `commit-map` from the previous run. You need this file from **every** run. Do the next step every time you run `git filter-repo`. - To purge all files larger than 10M, the `--strip-blobs-bigger-than` option can be used: + To purge specific files, the `--path` and `--invert-paths` options can be combined: ```shell - git filter-repo --strip-blobs-bigger-than 10M + git filter-repo --path path/to/file.ext --invert-paths ``` - To purge specific large files by path, the `--path` and `--invert-paths` options can be combined. + To generally purge all files larger than 10M, the `--strip-blobs-bigger-than` option can be used: ```shell - git filter-repo --path path/to/big/file.m4v --invert-paths + git filter-repo --strip-blobs-bigger-than 10M ``` See the @@ -148,7 +159,7 @@ operations before continuing. To clean up a repository: 1. Go to the project for the repository. -1. Navigate to **Settings > Repository**. +1. Go to **Settings > Repository**. 1. Upload a list of objects. For example, a `commit-map` file created by `git filter-repo` which is located in the `filter-repo` directory. @@ -158,7 +169,7 @@ To clean up a repository: split -l 3000 filter-repo/commit-map filter-repo/commit-map- ``` -1. Click **Start cleanup**. +1. Select **Start cleanup**. This: @@ -229,7 +240,7 @@ Until `git gc` runs on the GitLab side, the "removed" commits and blobs still ex must be able to push the rewritten history to GitLab, which may be impossible if you've already exceeded the maximum size limit. -In order to lift these restrictions, the administrator of the self-managed GitLab instance must +To lift these restrictions, the Administrator of the self-managed GitLab instance must increase the limit on the particular project that exceeded it. Therefore, it's always better to proactively stay underneath the limit. If you hit the limit, and can't have it temporarily increased, your only option is to: diff --git a/doc/user/project/repository/x509_signed_commits/index.md b/doc/user/project/repository/x509_signed_commits/index.md index 27ef14d31c5..c9cddad1a91 100644 --- a/doc/user/project/repository/x509_signed_commits/index.md +++ b/doc/user/project/repository/x509_signed_commits/index.md @@ -20,15 +20,15 @@ The main difference is the way GitLab determines whether or not the developer's (A trust store is a repository of trusted security certificates.) Combined with any required intermediate certificates in the signature, the developer's certificate can be chained back to a trusted root certificate. -- For GPG, developers [add their GPG key](../gpg_signed_commits/index.md#adding-a-gpg-key-to-your-account) +- For GPG, developers [add their GPG key](../gpg_signed_commits/index.md#add-a-gpg-key-to-your-account) to their account. GitLab uses its own certificate store and therefore defines the -[trust chain](https://www.ssl.com/faqs/what-is-a-chain-of-trust/). +[trust chain](https://www.ssl.com/faqs/what-is-a-certificate-authority/). For a commit or tag to be *verified* by GitLab: - The signing certificate email must match a verified email address in GitLab. -- The GitLab instance must be able to establish a full [trust chain](https://www.ssl.com/faqs/what-is-a-chain-of-trust/) +- The GitLab instance must be able to establish a full trust chain from the certificate in the signature to a trusted certificate in the GitLab certificate store. This chain may include intermediate certificates supplied in the signature. You may need to add certificates, such as Certificate Authority root certificates, diff --git a/doc/user/project/requirements/index.md b/doc/user/project/requirements/index.md index d549a22910a..6abe9c08d28 100644 --- a/doc/user/project/requirements/index.md +++ b/doc/user/project/requirements/index.md @@ -10,8 +10,8 @@ info: To determine the technical writer assigned to the Stage/Group associated w NOTE: In 14.4, Requirements was moved under **Issues**. -> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2703) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 12.10. -> - The ability to add and edit a requirement's long description [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/224622) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.5. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2703) in GitLab 12.10. +> - The ability to add and edit a requirement's long description [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/224622) in GitLab 13.5. > - [Moved under Issues](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/70748) in 14.4 With requirements, you can set criteria to check your products against. They can be based on users, @@ -64,7 +64,7 @@ next to the requirement title. ## Edit a requirement -> The ability to mark a requirement as Satisfied [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218607) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.5. +> The ability to mark a requirement as Satisfied [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/218607) in GitLab 13.5. You can edit a requirement from the requirements list page. @@ -108,7 +108,7 @@ As soon as a requirement is reopened, it no longer appears in the **Archived** t ## Search for a requirement -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/212543) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.1. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/212543) in GitLab 13.1. > - Searching by status [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/224614) in GitLab 13.10. You can search for a requirement from the requirements list page based on the following criteria: @@ -131,8 +131,8 @@ You can also sort the requirements list by: ## Allow requirements to be satisfied from a CI job -> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2859) in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.1. -> - [Added](https://gitlab.com/gitlab-org/gitlab/-/issues/215514) ability to specify individual requirements and their statuses in [GitLab Ultimate](https://about.gitlab.com/pricing/) 13.2. +> - [Introduced](https://gitlab.com/groups/gitlab-org/-/epics/2859) in GitLab 13.1. +> - [Added](https://gitlab.com/gitlab-org/gitlab/-/issues/215514) ability to specify individual requirements and their statuses in GitLab 13.2. GitLab supports [requirements test reports](../../../ci/yaml/artifacts_reports.md#artifactsreportsrequirements) now. You can add a job to your CI pipeline that, when triggered, marks all existing diff --git a/doc/user/project/service_desk.md b/doc/user/project/service_desk.md index 1e5b818c99d..90732ba4fe2 100644 --- a/doc/user/project/service_desk.md +++ b/doc/user/project/service_desk.md @@ -41,7 +41,7 @@ Here's how Service Desk works for you: 1. You provide a project-specific email address to your paying customers, who can email you directly from the application. 1. Each email they send creates an issue in the appropriate project. -1. Your team members navigate to the Service Desk issue tracker, where they can see new support +1. Your team members go to the Service Desk issue tracker, where they can see new support requests and respond inside associated issues. 1. Your team communicates back and forth with the customer to understand the request. 1. Your team starts working on implementing code to solve your customer's problem. @@ -93,8 +93,8 @@ displayed in the information note. ### Using customized email templates -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/2460) in GitLab Premium 12.7. -> - Moved to GitLab Free in 13.2. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/2460) in GitLab 12.7. +> - Moved from GitLab Premium to GitLab Free in 13.2. An email is sent to the author when: @@ -153,7 +153,7 @@ To use a custom description template with Service Desk: 1. On the top bar, select **Menu > Projects** and find your project. 1. [Create a description template](description_templates.md#create-an-issue-template). 1. On the left sidebar, select **Settings > General > Service Desk**. -1. From the dropdown **Template to append to all Service Desk issues**, search or select your template. +1. From the dropdown list **Template to append to all Service Desk issues**, search or select your template. ### Using a custom email display name @@ -171,7 +171,7 @@ To edit the custom email display name: ### Using a custom email address **(FREE SELF)** -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/2201) in GitLab Premium 13.0. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/2201) in GitLab 13.0. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/issues/284656) in GitLab 13.8. It is possible to customize the email address used by Service Desk. To do this, you must configure @@ -190,13 +190,13 @@ you can customize the mailbox used by Service Desk. This allows you to have a separate email address for Service Desk by also configuring a [custom suffix](#configuring-a-custom-email-address-suffix) in project settings. -The `address` must include the `+%{key}` placeholder within the 'user' -portion of the address, before the `@`. This is used to identify the project +The `address` must include the `+%{key}` placeholder in the 'user' +portion of the address, before the `@`. The placeholder is used to identify the project where the issue should be created. NOTE: When configuring a custom mailbox, the `service_desk_email` and `incoming_email` -configurations must always use separate mailboxes. This is important, because +configurations must always use separate mailboxes. It's important, because emails picked from `service_desk_email` mailbox are processed by a different worker and it would not recognize `incoming_email` emails. @@ -241,7 +241,7 @@ The configuration options are the same as for configuring ##### Microsoft Graph -> Introduced in [GitLab 13.11](https://gitlab.com/gitlab-org/gitlab/-/issues/214900) +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/214900) in GitLab 13.11. Service Desk can be configured to read Microsoft Exchange Online mailboxes with the Microsoft Graph API instead of IMAP. Follow the [documentation in the incoming email section for setting up an OAuth2 application for Microsoft Graph](../../administration/incoming_email.md#microsoft-graph). @@ -267,7 +267,7 @@ The Microsoft Graph API is not yet supported in source installations. See [this #### Configuring a custom email address suffix -You can set a custom suffix in your project's Service Desk settings once you have configured a [custom mailbox](#configuring-a-custom-mailbox). +You can set a custom suffix in your project's Service Desk settings after you have configured a [custom mailbox](#configuring-a-custom-mailbox). It can contain only lowercase letters (`a-z`), numbers (`0-9`), or underscores (`_`). When configured, the custom suffix creates a new Service Desk email address, consisting of the @@ -281,7 +281,7 @@ For example, suppose the `mygroup/myproject` project Service Desk settings has t The Service Desk email address for this project is: `contact+mygroup-myproject-support@example.com`. The [incoming email](../../administration/incoming_email.md) address still works. -If you don't configure the custom suffix, the default project identification will be used for identifying the project. You can see that email address in the project settings. +If you don't configure the custom suffix, the default project identification is used for identifying the project. You can see that email address in the project settings. ## Using Service Desk @@ -292,6 +292,7 @@ In these issues, you can also see our friendly neighborhood [Support Bot](#suppo > Support for additional email headers [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/346600) in GitLab 14.6. > In earlier versions, the Service Desk email address had to be in the "To" field. + To create a Service Desk issue, an end user does not need to know anything about the GitLab instance. They just send an email to the address they are given, and receive an email back confirming receipt: @@ -327,7 +328,28 @@ You can read and write comments as you normally do in GitLab: Note that: - The project's visibility (private, internal, public) does not affect Service Desk. -- The path to the project, including its group or namespace, are shown in emails. +- The path to the project, including its group or namespace, is shown in emails. + +#### Issues created on someone's behalf + +To allow third party applications and ticketing systems to interface with Service Desk, +when the email contains the `Reply-To` email header, this email address is used as the address of the +issue author. + +Because the `Reply-To` header can be set to arbitrary values, do not blindly trust that an issue +created on behalf of `someone@example.com` was indeed created by the real owner of such email address. + +For example, an email with headers `To: support@example.com` and `Reply-To:someone@example.com` +creates an issue with the following note: + +> Created (…) by `support@example.com` (reply to: `someone@example.com`) (…) + +#### Privacy considerations + +Service Desk issues are confidential, but the project owner can +[make an issue public](issues/confidential_issues.md#modify-issue-confidentiality). +When a Service Desk issue becomes public, the issue creator's email address is disclosed +to everyone who can view the project. ### Support Bot user diff --git a/doc/user/project/settings/import_export.md b/doc/user/project/settings/import_export.md index 8cee567ae93..ae3decb8079 100644 --- a/doc/user/project/settings/import_export.md +++ b/doc/user/project/settings/import_export.md @@ -229,7 +229,7 @@ and the exports between them are compatible. ## Related topics - [Project import/export API](../../../api/project_import_export.md) -- [Project import/export administration Rake tasks](../../../administration/raketasks/project_import_export.md) **(FREE SELF)** +- [Project import/export administration Rake tasks](../../../administration/raketasks/project_import_export.md) - [Group import/export](../../group/settings/import_export.md) - [Group import/export API](../../../api/group_import_export.md) @@ -305,6 +305,9 @@ reduce the repository size for another import attempt: #### Workaround option 2 +NOTE: +This workaround requires access to the rails console, which isn't available to end-users on GitLab.com. + Rather than attempting to push all changes at once, this workaround: - Separates the project import from the Git Repository import @@ -351,32 +354,32 @@ Rather than attempting to push all changes at once, this workaround: git push -u origin ${COMMIT_SHA}:refs/heads/main done git push -u origin main - git push -u origin -—all - git push -u origin -—tags + git push -u origin --all + git push -u origin --tags ``` ### Manually execute export steps Exports sometimes fail without giving enough information to troubleshoot. In these cases, it can be -helpful to [execute the export process manually within rails](https://gitlab.com/gitlab-com/runbooks/-/blob/master/docs/uncategorized/project-export.md#export-a-project-via-rails-console). +helpful to [open a rails console session](../../../administration/operations/rails_console.md#starting-a-rails-console-session) +and loop through [all the defined exporters](https://gitlab.com/gitlab-org/gitlab/-/blob/b67a5b5a12498d57cd877023b7385f7251e57de8/app/services/projects/import_export/export_service.rb#L65). Execute each line individually, rather than pasting the entire block at once, so you can see any errors each command returns. ```shell +# User needs to have permission to export u = User.find_by_username('someuser') p = Project.find_by_full_path('some/project') e = Projects::ImportExport::ExportService.new(p,u) e.send(:version_saver).send(:save) -e.send(:avatar_saver).send(:save) -e.send(:project_tree_saver).send(:save) -e.send(:uploads_saver).send(:save) -e.send(:wiki_repo_saver).send(:save) -e.send(:lfs_saver).send(:save) -e.send(:snippets_repo_saver).send(:save) -e.send(:design_repo_saver).send(:save) +e.send(:repo_saver).send(:save) +## continue using `e.send(:exporter_name).send(:save)` going through the list of exporters +# The following line should show you the export_path similar to /var/opt/gitlab/gitlab-rails/shared/tmp/gitlab_exports/@hashed/49/94/4994.... s = Gitlab::ImportExport::Saver.new(exportable: p, shared:p.import_export_shared) + +# To try and upload use: s.send(:compress_and_save) s.send(:save_upload) ``` diff --git a/doc/user/project/settings/index.md b/doc/user/project/settings/index.md index 12d75551aac..342b8d80bcf 100644 --- a/doc/user/project/settings/index.md +++ b/doc/user/project/settings/index.md @@ -541,7 +541,7 @@ Configure [alert integrations](../../../operations/incident_management/integrati #### Alert integration -Automatically [create](../../../operations/incident_management/incidents.md#create-incidents-automatically), [notify on](../../../operations/incident_management/paging.md#email-notifications), and [resolve](../../../operations/incident_management/incidents.md#automatically-close-incidents-via-recovery-alerts) incidents based on GitLab alerts. +Automatically [create](../../../operations/incident_management/incidents.md#create-incidents-automatically), [notify on](../../../operations/incident_management/paging.md#email-notifications-for-alerts), and [resolve](../../../operations/incident_management/incidents.md#automatically-close-incidents-via-recovery-alerts) incidents based on GitLab alerts. #### PagerDuty integration @@ -555,11 +555,11 @@ Automatically [create](../../../operations/incident_management/incidents.md#crea Configure Error Tracking to discover and view [Sentry errors within GitLab](../../../operations/error_tracking.md). -### Jaeger tracing **(ULTIMATE)** +### Jaeger tracing Add the URL of a Jaeger server to allow your users to [easily access the Jaeger UI from within GitLab](../../../operations/tracing.md). -### Status Page +### Status Page **(ULTIMATE)** [Add Storage credentials](../../../operations/incident_management/status_page.md#sync-incidents-to-the-status-page) to enable the syncing of public Issues to a [deployed status page](../../../operations/incident_management/status_page.md#create-a-status-page-project). diff --git a/doc/user/project/settings/project_access_tokens.md b/doc/user/project/settings/project_access_tokens.md index aa30c576c7c..a78226ac2f8 100644 --- a/doc/user/project/settings/project_access_tokens.md +++ b/doc/user/project/settings/project_access_tokens.md @@ -24,6 +24,8 @@ Project access tokens are similar to [group access tokens](../../group/settings/ and [personal access tokens](../../profile/personal_access_tokens.md), except they are associated with a project rather than a group or user. +In self-managed instances, project access tokens are subject to the same [maximum lifetime limits](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-personal-access-tokens) as personal access tokens if the limit is set. + You can use project access tokens: - On GitLab SaaS if you have the Premium license tier or higher. Project access tokens are not available with a [trial license](https://about.gitlab.com/free-trial/). @@ -43,7 +45,8 @@ To create a project access token: 1. On the top bar, select **Menu > Projects** and find your project. 1. On the left sidebar, select **Settings > Access Tokens**. 1. Enter a name. The token name is visible to any user with permissions to view the project. -1. Optional. Enter an expiry date for the token. The token will expire on that date at midnight UTC. +1. Optional. Enter an expiry date for the token. The token expires on that date at midnight UTC. An instance-wide [maximum lifetime](../../admin_area/settings/account_and_limit_settings.md#limit-the-lifetime-of-personal-access-tokens) setting can limit the maximum allowable lifetime in self-managed instances. + 1. Select a role for the token. 1. Select the [desired scopes](#scopes-for-a-project-access-token). 1. Select **Create project access token**. diff --git a/doc/user/project/static_site_editor/index.md b/doc/user/project/static_site_editor/index.md index 072f5bf1927..7c74a625b92 100644 --- a/doc/user/project/static_site_editor/index.md +++ b/doc/user/project/static_site_editor/index.md @@ -18,18 +18,13 @@ WARNING: This feature is in its end-of-life process. It is [deprecated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/77246) for use in GitLab 14.7, and is planned for -[removal](https://gitlab.com/groups/gitlab-org/-/epics/7351) in GitLab 15.0. -Users should instead use the [Web Editor](../repository/web_editor.md) or [Web IDE](../web_ide/index.md). +[removal](https://gitlab.com/groups/gitlab-org/-/epics/7351) in GitLab 14.10. +Users should instead use the [Web Editor](../repository/web_editor.md) or [Web IDE](../web_ide/index.md). [Removal instructions](#remove-the-static-site-editor) for existing projects are included on this page. Static Site Editor (SSE) enables users to edit content on static websites without prior knowledge of the underlying templating language, site architecture, or Git commands. A contributor to your project can quickly edit a Markdown page -and submit the changes for review. - -## Use cases - -The Static Site Editor allows collaborators to submit changes to static site -files seamlessly. For example: +and submit the changes for review. For example: - Non-technical collaborators can edit a page directly from the browser. They don't need to know Git and the details of your project to contribute. @@ -37,6 +32,43 @@ files seamlessly. For example: - Temporary collaborators can jump from project to project and quickly edit pages instead of having to clone or fork every single project they need to submit changes to. +## Remove the Static Site Editor + +The Static Site Editor itself isn't part of your project. To remove the Static Site Editor +from an existing project, remove links that point back to the editor: + +1. Remove any links that use `edit_page_url` in your project. If you used the + **Middleman - Static Site Editor** project template, the only instance of this + helper is located in `/source/layouts/layout.erb`. Remove this line entirely: + + ```ruby + <%= link_to('Edit this page', edit_page_url(data.config.repository, current_page.file_descriptor.relative_path), id: 'edit-page-link') %> + ``` + +1. In `/data/config.yml`, delete the `repository` key / value pair: + + ```yaml + repository: https://gitlab.com/<username>/<myproject> + ``` + + - If `repository` is the only value stored in `/data/config.yml`, you can delete the entire file. +1. In `/helpers/custom_helpers.rb`, delete `edit_page_url()` and `endcode_path()`: + + ```ruby + def edit_page_url(base_url, relative_path) + "#{base_url}/-/sse/#{encode_path(relative_path)}/" + end + + def encode_path(relative_path) + ERB::Util.url_encode("master/source/#{relative_path}") + end + ``` + + - If `edit_page_url()` and `encode_path()` are the only helpers, you may delete + `/helpers/custom_helpers.rb` entirely. +1. Clean up any extraneous configuration files. +1. Commit and push your changes. + ## Requirements - In order use the Static Site Editor feature, your project needs to be diff --git a/doc/user/project/web_ide/index.md b/doc/user/project/web_ide/index.md index 4565b5f1c91..ff8a076465d 100644 --- a/doc/user/project/web_ide/index.md +++ b/doc/user/project/web_ide/index.md @@ -17,16 +17,16 @@ You can also open the Web IDE when viewing a file, from the repository file list and from merge requests: - *When viewing a file, or the repository file list* - - 1. In the upper right corner of the page, select **Edit in Web IDE** if it is visible. - 1. If **Edit in Web IDE** is not visible: + 1. In the upper right corner of the page, select **Open in Web IDE** if it is visible. + 1. If **Open in Web IDE** is not visible: 1. Select the **(angle-down)** next to **Edit** or **Gitpod**, depending on your configuration. - 1. Select **Edit in Web IDE** from the list to display it as the editing option. - 1. Select **Edit in Web IDE** to open the editor. + 1. Select **Open in Web IDE** from the list to display it as the editing option. + 1. Select **Open in Web IDE** to open the editor. - *When viewing a merge request* - 1. Go to your merge request, and select the **Overview** tab. 1. Scroll to the widgets section, after the merge request description. - 1. Select **Edit in Web IDE** if it is visible. - 1. If **Edit in Web IDE** is not visible: + 1. Select **Open in Web IDE** if it is visible. + 1. If **Open in Web IDE** is not visible: 1. Select the **(angle-down)** next to **Open in Gitpod**. 1. Select **Open in Web IDE** from the list to display it as the editing option. 1. Select **Open in Web IDE** to open the editor. @@ -145,7 +145,7 @@ Each schema entry supports two properties: ## Configure the Web IDE -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/23352) in GitLab Free 13.1. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/23352) in GitLab 13.1. The Web IDE supports configuration of certain editor settings by using [`.editorconfig` files](https://editorconfig.org/). When opening a file, the @@ -164,8 +164,8 @@ The Web IDE currently supports the following `.editorconfig` settings: ## Commit changes -> - Starting with [GitLab 12.7](https://gitlab.com/gitlab-org/gitlab/-/issues/33441), files are automatically staged. -> - In [GitLab 12.9](https://gitlab.com/gitlab-org/gitlab/-/issues/196609), support for staging files was removed to prevent loss of unstaged data. All of your current changes must be committed or discarded. +> - [Starting](https://gitlab.com/gitlab-org/gitlab/-/issues/33441) with GitLab 12.7, files are automatically staged. +> - In GitLab 12.9, support for staging files was [removed](https://gitlab.com/gitlab-org/gitlab/-/issues/196609) to prevent loss of unstaged data. All of your current changes must be committed or discarded. After making your changes, select **Commit** on the bottom-left to review the list of changed files. @@ -215,13 +215,13 @@ different branch. ## Markdown editing -> - Support for pasting images [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22822) in GitLab Free 13.1. -> - Side-by-side Markdown preview [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/68274) in GitLab Free 14.3. +> - Support for pasting images [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22822) in GitLab 13.1. +> - Side-by-side Markdown preview [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/68274) in GitLab 14.3. To edit Markdown files in the Web IDE: 1. Go to your repository, and navigate to the Markdown page you want to edit. -1. Select **Edit in Web IDE**, and GitLab loads the page in a tab in the editor. +1. Select **Open in Web IDE**, and GitLab loads the page in a tab in the editor. 1. Make your changes to the file. GitLab supports [GitLab Flavored Markdown](../../markdown.md#gitlab-flavored-markdown). 1. When your changes are complete, select **Commit** in the left sidebar. 1. Add a commit message, select the branch you want to commit to, and select **Commit**. @@ -286,10 +286,10 @@ An example `package.json`: ## Interactive Web Terminals for the Web IDE -> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/211685) to GitLab Free in 13.1. +> [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/211685) from GitLab Ultimate to GitLab Free in 13.1. WARNING: -Interactive Web Terminals for the Web IDE is currently in **Beta**. +Interactive Web Terminals for the Web IDE is currently in [**Beta**](../../../policy/alpha-beta-support.md#beta-features). GitLab.com shared runners [do not yet support Interactive Web Terminals](https://gitlab.com/gitlab-org/gitlab/-/issues/24674), so you must use your own private runner to make use of this feature. @@ -308,7 +308,7 @@ to work: This section requires at least a `session_timeout` value (which defaults to 1800 seconds) and a `listen_address` value. If `advertise_address` is not defined, `listen_address` is used. - If you are using a reverse proxy with your GitLab instance, web terminals must be - [enabled](../../../administration/integration/terminal.md#enabling-and-disabling-terminal-support). **(ULTIMATE SELF)** + [enabled](../../../administration/integration/terminal.md#enabling-and-disabling-terminal-support). If you have the terminal open and the job has finished with its tasks, the terminal blocks the job from finishing for the duration configured in @@ -391,7 +391,7 @@ click **Restart Terminal** to start a new terminal session. ### File syncing to web terminal -> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5276) in GitLab Ultimate 12.0. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/5276) in GitLab 12.0. > - [Moved](https://gitlab.com/gitlab-org/gitlab/-/issues/211686) from GitLab Ultimate to GitLab Free in 13.1. File changes in the Web IDE can be synced to a running web terminal. diff --git a/doc/user/project/wiki/group.md b/doc/user/project/wiki/group.md index 2c499af123c..37f2ef8fc6a 100644 --- a/doc/user/project/wiki/group.md +++ b/doc/user/project/wiki/group.md @@ -7,7 +7,7 @@ type: reference, how-to # Group wikis **(PREMIUM)** -> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13195) in [GitLab Premium](https://about.gitlab.com/pricing/) 13.5. +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/13195) in GitLab 13.5. If you use GitLab groups to manage multiple projects, some of your documentation might span multiple groups. You can create group wikis, instead of [project wikis](index.md), @@ -17,7 +17,7 @@ Group wikis are similar to [project wikis](index.md), with a few limitations: - [Git LFS](../../../topics/git/lfs/index.md) is not supported. - Group wikis are not included in [global search](../../search/advanced_search.md). - Changes to group wikis don't show up in the [group's activity feed](../../group/index.md#group-activity-analytics). -- Group wikis are enabled by default for **(PREMIUM)** and higher tiers. +- Group wikis are enabled by default for GitLab Premium and higher tiers. You [can't turn them off from the GitLab user interface](https://gitlab.com/gitlab-org/gitlab/-/issues/208413). For updates, follow [the epic that tracks feature parity with project wikis](https://gitlab.com/groups/gitlab-org/-/epics/2782). @@ -38,7 +38,7 @@ To access a group wiki: ## Export a group wiki -> Introduced in [GitLab 13.9](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/53247). +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/53247) in GitLab 13.9. Users with the Owner role in a group can [import and export group wikis](../../group/settings/import_export.md) when importing diff --git a/doc/user/project/working_with_projects.md b/doc/user/project/working_with_projects.md index 9cfcaf4ee81..19e77d18aca 100644 --- a/doc/user/project/working_with_projects.md +++ b/doc/user/project/working_with_projects.md @@ -68,9 +68,8 @@ If you're an instance administrator, you can administer all project topics from To create a project in GitLab: -1. On the top bar, select **Menu > Project**. -1. Select **Create new project**. -1. On the **New project** page, choose if you want to: +1. On the top bar, select **Menu > Project > Create new project**. +1. On the **Create new project** page, choose if you want to: - Create a [blank project](#create-a-blank-project). - Create a project from a: - [built-in template](#create-a-project-from-a-built-in-template). @@ -80,21 +79,20 @@ To create a project in GitLab: from a different repository. Contact your GitLab administrator if this option is not available. - [Connect an external repository to GitLab CI/CD](../../ci/ci_cd_for_external_repos/index.md). -NOTE: -For a list of words that can't be used as project names see +- For a list of words that you cannot use as project names, see [reserved project and group names](../../user/reserved_names.md). +- For a list of characters that you cannot use in project and group names, see +[limitations on project and group names](../../user/reserved_names.md#limitations-on-project-and-group-names). ## Create a blank project To create a blank project: -1. On the top bar, select **Menu > Project**. -1. Select **Create new project**. +1. On the top bar, select **Menu > Projects > Create new project**. 1. Select **Create blank project**. 1. Enter the project details: - - In the **Project name** field, enter the name of your project. You can use spaces, hyphens, - underscores, and emoji. You cannot use special characters. After you enter the name, - the **Project slug** populates. + - In the **Project name** field, enter the name of your project. You cannot use special characters at + the start or end of a project name. - In the **Project slug** field, enter the path to your project. The GitLab instance uses the slug as the URL path to the project. To change the slug, first enter the project name, then change the slug. @@ -121,17 +119,15 @@ Anyone can contribute a built-in template by following [these steps](https://abo To create a project from a built-in template: -1. On the top bar, select **Menu > Project**. -1. Select **Create new project**. +1. On the top bar, select **Menu > Projects > Create new project**. 1. Select **Create from template**. 1. Select the **Built-in** tab. 1. From the list of templates: - To view a preview of the template, select **Preview**. - To use a template for the project, select **Use template**. 1. Enter the project details: - - In the **Project name** field, enter the name of your project. You can use spaces, hyphens, - underscores, and emoji. You cannot use special characters. After you enter the name, - the **Project slug** populates. + - In the **Project name** field, enter the name of your project. You cannot use special characters at + the start or end of a project name. - In the **Project slug** field, enter the path to your project. The GitLab instance uses the slug as the URL path to the project. To change the slug, first enter the project name, then change the slug. @@ -149,17 +145,15 @@ Custom project templates are available at: - The [instance-level](../../user/admin_area/custom_project_templates.md) - The [group-level](../../user/group/custom_project_templates.md) -1. On the top bar, select **Menu > Project**. -1. Select **Create new project**. +1. On the top bar, select **Menu > Projects > Create new project**. 1. Select **Create from template**. 1. Select the **Instance** or **Group** tab. 1. From the list of templates: - To view a preview of the template, select **Preview**. - To use a template for the project, select **Use template**. 1. Enter the project details: - - In the **Project name** field, enter the name of your project. You can use spaces, hyphens, - underscores, and emoji. You cannot use special characters. After you enter the name, - the **Project slug** populates. + - In the **Project name** field, enter the name of your project. You cannot use special characters at + the start or end of a project name. - In the **Project slug** field, enter the path to your project. The GitLab instance uses the slug as the URL path to the project. To change the slug, first enter the project name, then change the slug. @@ -177,17 +171,15 @@ HIPAA Audit Protocol published by the U.S Department of Health and Human Service To create a project from the HIPAA Audit Protocol template: -1. On the top bar, select **Menu > Project**. -1. Select **Create new project**. +1. On the top bar, select **Menu > Projects > Create new project**. 1. Select **Create from template**. 1. Select the **Built-in** tab. 1. Locate the **HIPAA Audit Protocol** template: - To view a preview of the template, select **Preview**. - To use the template for the project, select **Use template**. 1. Enter the project details: - - In the **Project name** field, enter the name of your project. You can use spaces, hyphens, - underscores, and emoji. You cannot use special characters. After you enter the name, - the **Project slug** populates. + - In the **Project name** field, enter the name of your project. You cannot use special characters at + the start or end of a project name. - In the **Project slug** field, enter the path to your project. The GitLab instance uses the slug as the URL path to the project. To change the slug, first enter the project name, then change the slug. @@ -196,7 +188,7 @@ To create a project from the HIPAA Audit Protocol template: change the **Visibility Level**. 1. Select **Create project**. -## Push to create a new project +## Create a new project with Git push > [Introduced](https://gitlab.com/gitlab-org/gitlab-foss/-/issues/26388) in GitLab 10.5. @@ -218,7 +210,7 @@ Prerequisites: [added to your GitLab account](../../ssh/index.md#add-an-ssh-key-to-your-gitlab-account). - You must have permission to add new projects to a namespace. To check if you have permission: - 1. On the top bar, select **Menu > Project**. + 1. On the top bar, select **Menu > Projects**. 1. Select **Groups**. 1. Select a group. 1. Confirm that **New project** is visible in the upper right @@ -266,14 +258,14 @@ You can add a star to projects you use frequently to make them easier to find. To add a star to a project: -1. On the top bar, select **Menu > Project**. +1. On the top bar, select **Menu > Projects**. 1. Select **Your projects** or **Explore projects**. 1. Select a project. 1. In the upper right corner of the page, select **Star**. ## View starred projects -1. On the top bar, select **Menu > Project**. +1. On the top bar, select **Menu > Projects**. 1. Select **Starred projects**. 1. GitLab displays information about your starred projects, including: @@ -290,33 +282,33 @@ you can [enable delayed project removal](../group/index.md#enable-delayed-projec To delete a project: -1. On the top bar, select **Menu > Project**. +1. On the top bar, select **Menu > Projects**. 1. Select **Your projects** or **Explore projects**. 1. Select a project. 1. Select **Settings > General**. 1. Expand the **Advanced** section. 1. Scroll down to the **Delete project** section. -1. Select **Delete project** +1. Select **Delete project**. 1. Confirm this action by completing the field. -## Projects pending deletion **(PREMIUM)** +## View projects pending deletion **(PREMIUM)** > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/37014) in GitLab 13.3 for Administrators. > - [Tab renamed](https://gitlab.com/gitlab-org/gitlab/-/issues/347468) from **Deleted projects** in GitLab 14.6. > - [Available to all users](https://gitlab.com/gitlab-org/gitlab/-/issues/346976) in GitLab 14.8 [with a flag](../../administration/feature_flags.md) named `project_owners_list_project_pending_deletion`. Enabled by default. - -FLAG: -On self-managed GitLab, by default this feature is available to all users. To make it available for administrators only, -ask an administrator to [disable the feature flag](../../administration/feature_flags.md) named `project_owners_list_project_pending_deletion`. -On GitLab.com, this feature is available to all users. +> - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/351556) in GitLab 14.9. [Feature flag `project_owners_list_project_pending_deletion`](https://gitlab.com/gitlab-org/gitlab/-/issues/351556) removed. When delayed project deletion is [enabled for a group](../group/index.md#enable-delayed-project-deletion), -projects within that group are not deleted immediately, but only after a delay. To access a list of all projects that are pending deletion: +projects within that group are not deleted immediately, but only after a delay. + +To view a list of all projects that are pending deletion: 1. On the top bar, select **Menu > Projects > Explore projects**. -1. Select the **Pending deletion** tab (in GitLab 14.6 and later) or the **Deleted projects** tab (GitLab 14.5 and earlier). +1. Based on your GitLab version: + - GitLab 14.6 and later: select the **Pending deletion** tab. + - GitLab 14.5 and earlier: select the **Deleted projects** tab. -Listed for each project is: +Each project in the list shows: - The time the project was marked for deletion. - The time the project is scheduled for final deletion. @@ -326,7 +318,7 @@ Listed for each project is: To view the activity of a project: -1. On the top bar, select **Menu > Project**. +1. On the top bar, select **Menu > Projects**. 1. Select **Your projects** or **Explore projects**. 1. Select a project. 1. On the left sidebar, select **Project information > Activity**. @@ -334,12 +326,12 @@ To view the activity of a project: ## Leave a project -If you leave a project you are no longer a project +If you leave a project, you are no longer a project member and cannot contribute. To leave a project: -1. On the top bar, select **Menu > Project**. +1. On the top bar, select **Menu > Projects**. 1. Select **Your projects** or **Explore projects**. 1. Select a project. 1. Select **Leave project**. The **Leave project** option only displays @@ -481,3 +473,4 @@ download starts, the `insteadOf` configuration sends the traffic to the secondar - [Connect an external repository to GitLab CI/CD](../../ci/ci_cd_for_external_repos/index.md). - [Fork a project](repository/forking_workflow.md#creating-a-fork). - [Adjust project visibility and access levels](settings/index.md#sharing-and-permissions). +- [Limitations on project and group names](../../user/reserved_names.md#limitations-on-project-and-group-names) diff --git a/doc/user/reserved_names.md b/doc/user/reserved_names.md index 203b1c9e93a..33ecf252e43 100644 --- a/doc/user/reserved_names.md +++ b/doc/user/reserved_names.md @@ -4,7 +4,7 @@ group: Workspace info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- -# Reserved project and group names +# Reserved project and group names **(FREE)** Not all project & group names are allowed because they would conflict with existing routes used by GitLab. @@ -17,6 +17,13 @@ under the `TOP_LEVEL_ROUTES`, `PROJECT_WILDCARD_ROUTES` and `GROUP_ROUTES` lists - `PROJECT_WILDCARD_ROUTES`: are names that are reserved for child groups or projects. - `GROUP_ROUTES`: are names that are reserved for all groups or projects. +## Limitations on project and group names + +- Special characters are not permitted at the start or end of project or group names. They are permitted in any other location of the name. +- Project or group names cannot end in `.git` or `.atom`. +- Project or group names can only contain letters, digits, emojis, "_", ".", "+", dashes, or spaces. +- Paths can only contain letters, digits, "_", "-", and "." + ## Reserved project names It is currently not possible to create a project with the following names: @@ -45,7 +52,7 @@ It is currently not possible to create a project with the following names: ## Reserved group names -Currently the following names are reserved as top level groups: +Currently, the following names are reserved as top level groups: - `\-` - `.well-known` diff --git a/doc/user/search/advanced_search.md b/doc/user/search/advanced_search.md index 05579696d35..cb272b3feed 100644 --- a/doc/user/search/advanced_search.md +++ b/doc/user/search/advanced_search.md @@ -7,7 +7,7 @@ type: reference # GitLab Advanced Search **(PREMIUM)** -> - Moved to GitLab Premium in 13.9. +> Moved to GitLab Premium in 13.9. NOTE: This is the user documentation. To configure the Advanced Search, @@ -147,8 +147,8 @@ its performance: FLAG: On self-managed GitLab, by default this feature is not available. To make it available, - ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `prevent_abusive_searches`. - The feature is not ready for production use. +ask an administrator to [enable the feature flag](../../administration/feature_flags.md) named `prevent_abusive_searches`. +The feature is not ready for production use. To prevent abusive searches, such as searches that may result in a Distributed Denial of Service (DDoS), Global Search ignores, logs, and doesn't return any results for searches considered abusive according to the following criteria, if `prevent_abusive_searches` feature flag is enabled: diff --git a/doc/user/search/img/code_search.png b/doc/user/search/img/code_search.png Binary files differdeleted file mode 100644 index 7c62bb6921b..00000000000 --- a/doc/user/search/img/code_search.png +++ /dev/null diff --git a/doc/user/search/img/code_search_git_blame_v14_9.png b/doc/user/search/img/code_search_git_blame_v14_9.png Binary files differnew file mode 100644 index 00000000000..33d4e77e3f5 --- /dev/null +++ b/doc/user/search/img/code_search_git_blame_v14_9.png diff --git a/doc/user/search/index.md b/doc/user/search/index.md index 0a799843d3c..44327af380e 100644 --- a/doc/user/search/index.md +++ b/doc/user/search/index.md @@ -34,14 +34,15 @@ in the search field in the upper right corner: > - Filtering by epics was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/195704) in GitLab 12.9. > - Filtering by child epics was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/9029) in GitLab 13.0. -> - Filtering by iterations was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118742) in GitLab 13.6. Moved from GitLab Ultimate to Premium in 13.9. +> - Filtering by iterations was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/118742) in GitLab 13.6. +> - Filtering by iterations was moved from GitLab Ultimate to GitLab Premium in 13.9. > - Filtering by type was [introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/322755) in GitLab 13.10 [with a flag](../../administration/feature_flags.md) named `vue_issues_list`. Disabled by default. Follow these steps to filter the **Issues** and **Merge requests** list pages in projects and groups: -1. Click in the field **Search or filter results...**. -1. In the dropdown menu that appears, select the attribute you wish to filter by: +1. Select **Search or filter results...**. +1. In the dropdown list that appears, select the attribute you wish to filter by: - Assignee - Author - Confidential @@ -112,8 +113,8 @@ You can filter the **Issues** list to individual instances by their ID. For exam > Moved to GitLab Premium in 13.9. -To filter merge requests by an individual approver, you can type (or select from -the dropdown) **Approver** and select the user. +To filter merge requests by an individual eligible approver ([Codeowner](../project/code_owners.md)), you can type (or select from +the dropdown list) **Approver** and select the user.  @@ -123,29 +124,29 @@ the dropdown) **Approver** and select the user. > - Moved to GitLab Premium in 13.9. To filter merge requests already approved by a specific individual, you can type (or select from -the dropdown) **Approved-By** and select the user. +the dropdown list) **Approved-By** and select the user.  -### Filtering merge requests by reviewer **(FREE)** +### Filtering merge requests by reviewer > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/47605) in GitLab 13.7. To filter review requested merge requests for a specific individual, you can type (or select from -the dropdown) **Reviewer** and select the user. +the dropdown list) **Reviewer** and select the user. -### Filtering merge requests by environment or deployment date **(FREE)** +### Filtering merge requests by environment or deployment date > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/44041) in GitLab 13.6. To filter merge requests by deployment data, such as the environment or a date, -you can type (or select from the dropdown) the following: +you can type (or select from the dropdown list) the following: - Environment - Deployed-before - Deployed-after -When filtering by an environment, a dropdown presents all environments that +When filtering by an environment, a dropdown list presents all environments that you can choose from:  @@ -174,7 +175,7 @@ you must type at least `Sim` before autocomplete displays results. Search history is available for issues and merge requests, and is stored locally in your browser. To run a search from history: -1. In the top menu, click **Issues** or **Merge requests**. +1. In the top menu, select **Issues** or **Merge requests**. 1. To the left of the search bar, click **Recent searches**, and select a search from the list. ## Removing search filters @@ -193,7 +194,7 @@ Some filters can be added multiple times. These include but are not limited to a You can search your [To-Do List](../todos.md) by "to do" and "done". You can filter to-do items per project, author, type, and action. -Also, you can sort them by [**Label priority**](../../user/project/labels.md#label-priority), +Also, you can sort them by [**Label priority**](../../user/project/labels.md#set-label-priority), **Last created**, and **Oldest created**. ## Projects @@ -227,7 +228,7 @@ and sort them by **Last created**, **Oldest created**, **Last updated**, or **Ol From an [issue board](../../user/project/issue_board.md), you can filter issues by **Author**, **Assignee**, **Milestone**, and **Labels**. You can also filter them by name (issue title), from the field **Filter by name**, which is loaded as you type. -To search for issues to add to lists present in your issue board, click +To search for issues to add to lists present in your issue board, select the button **Add issues** on the top-right of your screen, opening a modal window from which you can, besides filtering them by **Name**, **Author**, **Assignee**, **Milestone**, and **Labels**, select multiple issues to add to a list of your choice: @@ -281,13 +282,13 @@ To search through code or other documents in a single project, you can use the search field on the top-right of your screen while the project page is open. Code search shows only the first result in the file. -#### Git blame from code search **(PREMIUM)** +#### Git blame from code search **(FREE)** > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/327052) in GitLab 14.7. You can access Git blame from any line that returned a result from the code search: - + ### SHA search @@ -307,7 +308,7 @@ GitLab instance. ## Search settings > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/292941) in GitLab 13.8 [with a flag](../../administration/feature_flags.md) named `search_settings_in_page`. Disabled by default. -> - [Added to Group, Administrator, and User settings](https://gitlab.com/groups/gitlab-org/-/epics/4842) in GitLab 13.9. +> - [Added](https://gitlab.com/groups/gitlab-org/-/epics/4842) to Group, Administrator, and User settings in GitLab 13.9. > - [Feature flag `search_settings_in_page` removed](https://gitlab.com/gitlab-org/gitlab/-/issues/294025) in GitLab 13.11. > - [Generally available](https://gitlab.com/gitlab-org/gitlab/-/issues/294025) in GitLab 13.11. diff --git a/doc/user/shortcuts.md b/doc/user/shortcuts.md index ebaa5da8e05..e807f251da1 100644 --- a/doc/user/shortcuts.md +++ b/doc/user/shortcuts.md @@ -17,10 +17,6 @@ following methods: - Press <kbd>?</kbd>. - In the Help menu in the top right of the application, select **Keyboard shortcuts**. -In [GitLab 12.8 and later](https://gitlab.com/gitlab-org/gitlab/-/issues/22113), -you can disable keyboard shortcuts by using the **Keyboard shortcuts** toggle -at the top of the keyboard shortcut window. - Although [global shortcuts](#global-shortcuts) work from any area of GitLab, you must be in specific pages for the other shortcuts to be available, as explained in each section. @@ -41,21 +37,22 @@ These shortcuts are available in most areas of GitLab: | <kbd>Shift</kbd> + <kbd>i</kbd> | Go to your Issues page. | | <kbd>Shift</kbd> + <kbd>m</kbd> | Go to your [Merge requests](project/merge_requests/index.md) page. | | <kbd>Shift</kbd> + <kbd>t</kbd> | Go to your To-Do List page. | -| <kbd>p</kbd> + <kbd>b</kbd> | Show or hide the Performance Bar. | -| <kbd>g</kbd> + <kbd>x</kbd> | Toggle between [GitLab](https://gitlab.com/) and [GitLab Next](https://next.gitlab.com/) (GitLab SaaS only). | -| <kbd>.</kbd> | Open the [Web IDE](project/web_ide/index.md). | +| <kbd>p</kbd> then <kbd>b</kbd> | Show or hide the Performance Bar. | +| <kbd>g</kbd> then <kbd>x</kbd> | Toggle between [GitLab](https://gitlab.com/) and [GitLab Next](https://next.gitlab.com/) (GitLab SaaS only). | +| <kbd>.</kbd> | Open the [Web IDE](project/web_ide/index.md). | Additionally, the following shortcuts are available when editing text in text fields (for example, comments, replies, issue descriptions, and merge request descriptions): -| Keyboard shortcut | Description | -|---------------------------------------------------------------------------|-------------| -| <kbd>↑</kbd> | Edit your last comment. You must be in a blank text field below a thread, and you must already have at least one comment in the thread. | -| <kbd>⌘</kbd> (Mac) / <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>p</kbd> | Toggle Markdown preview when editing text in a text field that has **Write** and **Preview** tabs at the top. | -| <kbd>⌘</kbd> (Mac) / <kbd>Control</kbd> + <kbd>b</kbd> | Bold the selected text (surround it with `**`). | -| <kbd>⌘</kbd> (Mac) / <kbd>Control</kbd> + <kbd>i</kbd> | Italicize the selected text (surround it with `_`). | -| <kbd>⌘</kbd> (Mac) / <kbd>Control</kbd> + <kbd>k</kbd> | Add a link (surround the selected text with `[]()`). | +| macOS shortcut | Windows shortcut | Description | +|----------------|------------------|-------------| +| <kbd>↑</kbd> | <kbd>↑</kbd> | Edit your last comment. You must be in a blank text field below a thread, and you must already have at least one comment in the thread. | +| <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>p</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>p</kbd> | Toggle Markdown preview when editing text in a text field that has **Write** and **Preview** tabs at the top. | +| <kbd>Command</kbd> + <kbd>b</kbd> | <kbd>Control</kbd> + <kbd>b</kbd> | Bold the selected text (surround it with `**`). | +| <kbd>Command</kbd> + <kbd>i</kbd> | <kbd>Control</kbd> + <kbd>i</kbd> | Italicize the selected text (surround it with `_`). | +| <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>s</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>s</kbd> | Strike through the selected text (surround it with `~~`). | +| <kbd>Command</kbd> + <kbd>k</kbd> | <kbd>Control</kbd> + <kbd>k</kbd> | Add a link (surround the selected text with `[]()`). | The shortcuts for editing in text fields are always enabled, even if other keyboard shortcuts are disabled. @@ -102,7 +99,7 @@ These shortcuts are available when viewing issues and [merge requests](project/m | <kbd>]</kbd> or <kbd>j</kbd> | Move to next file (merge requests only). | | <kbd>[</kbd> or <kbd>k</kbd> | Move to previous file (merge requests only). | | <kbd>b</kbd> | Copy source branch name (merge requests only). | -| <kbd>.</kbd> | Open the [Web IDE](project/web_ide/index.md). | +| <kbd>.</kbd> | Open the [Web IDE](project/web_ide/index.md). | ### Project files @@ -113,7 +110,7 @@ These shortcuts are available when browsing the files in a project (go to |-------------------|-------------| | <kbd>↑</kbd> | Move selection up. | | <kbd>↓</kbd> | Move selection down. | -| <kbd>enter</kbd> | Open selection. | +| <kbd>Enter</kbd> | Open selection. | | <kbd>Escape</kbd> | Go back to file list screen (only while searching for files, **Repository > Files**, then select **Find File**). | | <kbd>y</kbd> | Go to file permalink (only while viewing a file). | | <kbd>.</kbd> | Open the [Web IDE](project/web_ide/index.md). | @@ -122,15 +119,15 @@ These shortcuts are available when browsing the files in a project (go to These shortcuts are available when editing a file with the [Web IDE](project/web_ide/index.md): -| Keyboard shortcut | Description | -|------------------------------------------------------------|-------------| -| <kbd>⌘</kbd> (Mac) / <kbd>Control</kbd> + <kbd>p</kbd> | Search for, and then open another file for editing. | -| <kbd>⌘</kbd> (Mac) / <kbd>Control</kbd> + <kbd>Enter</kbd> | Commit (when editing the commit message). | +| macOS shortcut | Windows shortcut | Description | +|---------------------------------|---------------------|-------------| +| <kbd>Command</kbd> + <kbd>p</kbd> | <kbd>Control</kbd> + <kbd>p</kbd> | Search for, and then open another file for editing. | +| <kbd>Command</kbd> + <kbd>Enter</kbd> | <kbd>Control</kbd> + <kbd>Enter</kbd> | Commit (when editing the commit message). | ### Repository graph These shortcuts are available when viewing the project [repository graph](project/repository/index.md#repository-history-graph) -page (navigate to **Repository > Graph**): +page (go to **Repository > Graph**): | Keyboard shortcut | Description | |--------------------------------------------------------------------|-------------| @@ -151,63 +148,64 @@ This shortcut is available when viewing a [wiki page](project/wiki/index.md): ### Content editor -These shortcuts are available when editing a file with the [Content Editor](https://about.gitlab.com/direction/create/editor/content_editor/): +These shortcuts are available when editing a file with the +[Content Editor](https://about.gitlab.com/direction/create/editor/content_editor/): -| Keyboard shortcut | Description | -|-------------------|-------------| -| <kbd>⌘</kbd> + <kbd>C</kbd> (Mac) / <kbd>Control</kbd> + <kbd>C</kbd> | Copy | -| <kbd>⌘</kbd> + <kbd>X</kbd> (Mac) / <kbd>Control</kbd> + <kbd>X</kbd> | Cut | -| <kbd>⌘</kbd> + <kbd>V</kbd> (Mac) / <kbd>Control</kbd> + <kbd>V</kbd> | Paste | -| <kbd>⌘</kbd> + <kbd>Shift</kbd> + <kbd>V</kbd> (Mac) / <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>V</kbd> | Paste without formatting | -| <kbd>⌘</kbd> + <kbd>Z</kbd> (Mac) / <kbd>Control</kbd> + <kbd>Z</kbd> | Undo | -| <kbd>⌘</kbd> + <kbd>Shift</kbd> + <kbd>V</kbd> (Mac) / <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>V</kbd> | Redo | -| <kbd>Shift</kbd> + <kbd>Enter</kbd> | Add a line break | +| macOS shortcut | Windows shortcut | Description | +|----------------|------------------|-------------| +| <kbd>Command</kbd> + <kbd>C</kbd> | <kbd>Control</kbd> + <kbd>C</kbd> | Copy | +| <kbd>Command</kbd> + <kbd>X</kbd> | <kbd>Control</kbd> + <kbd>X</kbd> | Cut | +| <kbd>Command</kbd> + <kbd>V</kbd> | <kbd>Control</kbd> + <kbd>V</kbd> | Paste | +| <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>V</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>V</kbd> | Paste without formatting | +| <kbd>Command</kbd> + <kbd>Z</kbd> | <kbd>Control</kbd> + <kbd>Z</kbd> | Undo | +| <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>V</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>V</kbd> | Redo | +| <kbd>Shift</kbd> + <kbd>Enter</kbd> | <kbd>Shift</kbd> + <kbd>Enter</kbd> | Add a line break | #### Formatting -| Mac | Windows/Linux | Description | -|-----|---------------|-------------| -| <kbd>⌘</kbd> + <kbd>b</kbd> | <kbd>Control</kbd> + <kbd>b</kbd> | Bold | -| <kbd>⌘</kbd> + <kbd>i</kbd> | <kbd>Control</kbd> + <kbd>i</kbd> | Italic | -| <kbd>⌘</kbd> + <kbd>Shift</kbd> + <kbd>s</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>s</kbd> | Strikethrough | -| <kbd>⌘</kbd> + <kbd>e</kbd> | <kbd>Control</kbd> + <kbd>e</kbd> | Code | -| <kbd>⌘</kbd> + <kbd>Alt</kbd> + <kbd>0</kbd> | <kbd>Control</kbd> + <kbd>Alt</kbd> + <kbd>0</kbd> | Apply normal text style | -| <kbd>⌘</kbd> + <kbd>Alt</kbd> + <kbd>1</kbd> | <kbd>Control</kbd> + <kbd>Alt</kbd> + <kbd>1</kbd> | Apply heading style 1 | -| <kbd>⌘</kbd> + <kbd>Alt</kbd> + <kbd>2</kbd> | <kbd>Control</kbd> + <kbd>Alt</kbd> + <kbd>2</kbd> | Apply heading style 2 | -| <kbd>⌘</kbd> + <kbd>Alt</kbd> + <kbd>3</kbd> | <kbd>Control</kbd> + <kbd>Alt</kbd> + <kbd>3</kbd> | Apply heading style 3 | -| <kbd>⌘</kbd> + <kbd>Alt</kbd> + <kbd>4</kbd> | <kbd>Control</kbd> + <kbd>Alt</kbd> + <kbd>4</kbd> | Apply heading style 4 | -| <kbd>⌘</kbd> + <kbd>Alt</kbd> + <kbd>5</kbd> | <kbd>Control</kbd> + <kbd>Alt</kbd> + <kbd>5</kbd> | Apply heading style 5 | -| <kbd>⌘</kbd> + <kbd>Alt</kbd> + <kbd>6</kbd> | <kbd>Control</kbd> + <kbd>Alt</kbd> + <kbd>6</kbd> | Apply heading style 6 | -| <kbd>⌘</kbd> + <kbd>Shift</kbd> + <kbd>7</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>7</kbd> | Ordered list | -| <kbd>⌘</kbd> + <kbd>Shift</kbd> + <kbd>8</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>7</kbd> | Bullet list | -| <kbd>⌘</kbd> + <kbd>Shift</kbd> + <kbd>9</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>7</kbd> | Task list | -| <kbd>⌘</kbd> + <kbd>Shift</kbd> + <kbd>b</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>b</kbd> | Blockquote | -| <kbd>⌘</kbd> + <kbd>Alt</kbd> + <kbd>c</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>c</kbd> | Code block | -| <kbd>⌘</kbd> + <kbd>,</kbd> | <kbd>Control</kbd> + <kbd>,</kbd> | Subscript | -| <kbd>⌘</kbd> + <kbd>.</kbd> | <kbd>Control</kbd> + <kbd>,</kbd> | Superscript | -| <kbd>Tab</kbd> | | Indent list | -| <kbd>Shift</kbd> + <kbd>Tab</kbd> | | Outdent list | +| macOS shortcut | Windows/Linux shortcut | Description | +|----------------|------------------------|-------------| +| <kbd>Command</kbd> + <kbd>b</kbd> | <kbd>Control</kbd> + <kbd>b</kbd> | Bold | +| <kbd>Command</kbd> + <kbd>i</kbd> | <kbd>Control</kbd> + <kbd>i</kbd> | Italic | +| <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>x</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>x</kbd> | Strikethrough | +| <kbd>Command</kbd> + <kbd>e</kbd> | <kbd>Control</kbd> + <kbd>e</kbd> | Code | +| <kbd>Command</kbd> + <kbd>Alt</kbd> + <kbd>0</kbd> | <kbd>Control</kbd> + <kbd>Alt</kbd> + <kbd>0</kbd> | Apply normal text style | +| <kbd>Command</kbd> + <kbd>Alt</kbd> + <kbd>1</kbd> | <kbd>Control</kbd> + <kbd>Alt</kbd> + <kbd>1</kbd> | Apply heading style 1 | +| <kbd>Command</kbd> + <kbd>Alt</kbd> + <kbd>2</kbd> | <kbd>Control</kbd> + <kbd>Alt</kbd> + <kbd>2</kbd> | Apply heading style 2 | +| <kbd>Command</kbd> + <kbd>Alt</kbd> + <kbd>3</kbd> | <kbd>Control</kbd> + <kbd>Alt</kbd> + <kbd>3</kbd> | Apply heading style 3 | +| <kbd>Command</kbd> + <kbd>Alt</kbd> + <kbd>4</kbd> | <kbd>Control</kbd> + <kbd>Alt</kbd> + <kbd>4</kbd> | Apply heading style 4 | +| <kbd>Command</kbd> + <kbd>Alt</kbd> + <kbd>5</kbd> | <kbd>Control</kbd> + <kbd>Alt</kbd> + <kbd>5</kbd> | Apply heading style 5 | +| <kbd>Command</kbd> + <kbd>Alt</kbd> + <kbd>6</kbd> | <kbd>Control</kbd> + <kbd>Alt</kbd> + <kbd>6</kbd> | Apply heading style 6 | +| <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>7</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>7</kbd> | Ordered list | +| <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>8</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>8</kbd> | Bullet list | +| <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>9</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>9</kbd> | Task list | +| <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>b</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>b</kbd> | Blockquote | +| <kbd>Command</kbd> + <kbd>Alt</kbd> + <kbd>c</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>c</kbd> | Code block | +| <kbd>Command</kbd> + <kbd>,</kbd> | <kbd>Control</kbd> + <kbd>,</kbd> | Subscript | +| <kbd>Command</kbd> + <kbd>.</kbd> | <kbd>Control</kbd> + <kbd>.</kbd> | Superscript | +| <kbd>Tab</kbd> | <kbd>Tab</kbd> | Indent list | +| <kbd>Shift</kbd> + <kbd>Tab</kbd> | <kbd>Shift</kbd> + <kbd>Tab</kbd> | Outdent list | #### Text selection -| Keyboard shortcut | Description | -|-------------------|-------------| -| <kbd>⌘</kbd> + <kbd>a</kbd> (Mac) / <kbd>Control</kbd> + <kbd>a</kbd> | Select all | -| <kbd>Shift</kbd> + <kbd>←</kbd> | Extend selection one character to left | -| <kbd>Shift</kbd> + <kbd>→</kbd> | Extend selection one character to right | -| <kbd>Shift</kbd> + <kbd>↑</kbd> | Extend selection one line up | -| <kbd>Shift</kbd> + <kbd>↓</kbd> | Extend selection one line down | -| <kbd>⌘</kbd> + <kbd>Shift</kbd> + <kbd>↑</kbd> (Mac) / <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>↑</kbd> | Extend selection to the beginning of the document | -| <kbd>⌘</kbd> + <kbd>Shift</kbd> + <kbd>↓</kbd> (Mac) / <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>↓</kbd> | Extend selection to the end of the document | +| macOS shortcut | Windows shortcut | Description | +|----------------|------------------|-------------| +| <kbd>Command</kbd> + <kbd>a</kbd> | <kbd>Control</kbd> + <kbd>a</kbd> | Select all | +| <kbd>Shift</kbd> + <kbd>←</kbd> | <kbd>Shift</kbd> + <kbd>←</kbd> | Extend selection one character to left | +| <kbd>Shift</kbd> + <kbd>→</kbd> | <kbd>Shift</kbd> + <kbd>→</kbd> | Extend selection one character to right | +| <kbd>Shift</kbd> + <kbd>↑</kbd> | <kbd>Shift</kbd> + <kbd>↑</kbd> | Extend selection one line up | +| <kbd>Shift</kbd> + <kbd>↓</kbd> | <kbd>Shift</kbd> + <kbd>↓</kbd> | Extend selection one line down | +| <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>↑</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>↑</kbd> | Extend selection to the beginning of the document | +| <kbd>Command</kbd> + <kbd>Shift</kbd> + <kbd>↓</kbd> | <kbd>Control</kbd> + <kbd>Shift</kbd> + <kbd>↓</kbd> | Extend selection to the end of the document | ### Filtered search These shortcuts are available when using a [filtered search input](search/index.md): -| Keyboard shortcut | Description | -|--------------------------------------------------------|-------------| -| <kbd>⌘</kbd> (Mac) + <kbd>⌫</kbd> | Clear entire search filter. | -| <kbd>⌥</kbd> (Mac) / <kbd>Control</kbd> + <kbd>⌫</kbd> | Clear one token at a time. | +| macOS shortcut | Windows shortcut | Description | +|----------------------|----------------------------------------|-------------| +| <kbd>Command</kbd> | <kbd>Delete</kbd> | Clear entire search filter. | +| <kbd>Option</kbd> | <kbd>Control</kbd> + <kbd>Delete</kbd> | Clear one token at a time. | ## Epics **(PREMIUM)** @@ -218,3 +216,13 @@ These shortcuts are available when viewing [epics](group/epics/index.md): | <kbd>r</kbd> | Start writing a comment. Pre-selected text is quoted in the comment. Can't be used to reply in a thread. | | <kbd>e</kbd> | Edit description. | | <kbd>l</kbd> | Change label. | + +## Disable keyboard shortcuts + +> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/22113) in GitLab 12.8. + +To disable keyboard shortcuts: + +1. While viewing a page that supports keyboard shortcuts, and outside a text box, +press <kbd>?</kbd> to display the list of shortcuts. +1. Select **Toggle shortcuts**. diff --git a/doc/user/snippets.md b/doc/user/snippets.md index dc3ab35067e..1750e2c8d10 100644 --- a/doc/user/snippets.md +++ b/doc/user/snippets.md @@ -33,20 +33,18 @@ You can create snippets in multiple ways, depending on whether you want to creat 1. Select the kind of snippet you want to create: - **To create a personal snippet**: On the - [Snippets dashboard](https://gitlab.com/dashboard/snippets), click + [Snippets dashboard](https://gitlab.com/dashboard/snippets), select **New snippet**, or: - *If you're on a project's page,* select the plus icon (**{plus-square-o}**) in the top navigation bar, and then select **New snippet** from the - **GitLab** (GitLab SaaS) or **Your Instance** (self-managed) section - of the same dropdown menu. + **GitLab** section of the same dropdown list. - *For all other pages,* select the plus icon (**{plus-square-o}**) - in the top navigation bar, then select **New snippet** from the dropdown - menu. + in the top navigation bar, then select **New snippet** from the dropdown list. - If you installed the [GitLab Workflow VS Code extension](project/repository/vscode.md), use the [`Gitlab: Create snippet` command](https://marketplace.visualstudio.com/items?itemName=GitLab.gitlab-workflow#create-snippet). - **To create a project snippet**: Go to your project's page. Select the plus icon (**{plus-square-o}**), and then select **New snippet** from the - **This project** section of the dropdown menu. + **This project** section of the dropdown list. 1. Add a **Title** and **Description**. 1. Name your **File** with an appropriate extension, such as `example.rb` or `index.html`. Filenames with appropriate extensions display [syntax highlighting](#filenames). @@ -154,15 +152,14 @@ To delete a file from your snippet through the GitLab UI: 1. Go to your snippet in the GitLab UI. 1. Select **Edit** in the top right corner. -1. Select **Delete file** alongside the filename of each file -you wish to delete. +1. Select **Delete file** alongside the filename of each file you wish to delete. 1. Select **Save changes**. ## Clone snippets Instead of copying a snippet to a local file, you may want to clone a snippet to preserve its relationship with the repository, so you can receive or make updates -as needed. Select the **Clone** button on a snippet to display the URLs to clone with SSH or HTTPS: +as needed. Select **Clone** on a snippet to display the URLs to clone with SSH or HTTPS:  @@ -202,7 +199,7 @@ For example: ## Download snippets You can download the raw content of a snippet. By default, they download with Linux-style line endings (`LF`). If -you want to preserve the original line endings you need to add a parameter `line_ending=raw` +you want to preserve the original line endings you must add a parameter `line_ending=raw` (For example: `https://gitlab.com/snippets/SNIPPET_ID/raw?line_ending=raw`). In case a snippet was created using the GitLab web interface the original line ending is Windows-like (`CRLF`). diff --git a/doc/user/tasks.md b/doc/user/tasks.md index 5fde64e3578..8be7fbca213 100644 --- a/doc/user/tasks.md +++ b/doc/user/tasks.md @@ -9,7 +9,7 @@ info: To determine the technical writer assigned to the Stage/Group associated w > [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/334812) in GitLab 14.5 [with a flag](../administration/feature_flags.md) named `work_items`. Disabled by default. WARNING: -Tasks are in **Alpha**. Current implementation does not have any API requests and works with mocked data. +Tasks are in [**Alpha**](../policy/alpha-beta-support.md#alpha-features). Current implementation does not have any API requests and works with mocked data. For the latest updates, check the [Tasks Roadmap](https://gitlab.com/groups/gitlab-org/-/epics/7103). FLAG: diff --git a/doc/user/todos.md b/doc/user/todos.md index 2486db813ff..ec316d8ebe4 100644 --- a/doc/user/todos.md +++ b/doc/user/todos.md @@ -45,9 +45,30 @@ A to-do item is added to your To-Do List when: When several actions occur for the same user on the same object, GitLab displays the first action as a single to-do item. +To change this behavior, enable +[multiple to-do items per object](#multiple-to-do-items-per-object). To-do items aren't affected by [GitLab notification email settings](profile/notifications.md). +### Multiple to-do items per object **(FREE SELF)** + +<!-- When the feature flag is removed, integrate this topic into the one above. --> + +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/28355) in GitLab 13.8 [with a flag](../administration/feature_flags.md) named `multiple_todos`. Disabled by default. +> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/82470) in GitLab 14.9: only mentions create multiple to-do items. + +FLAG: +On self-managed GitLab, by default this feature is not available. To make it available per user, +ask an administrator to [enable the feature flag](../administration/feature_flags.md) named `multiple_todos`. +On GitLab.com, this feature is not available. +The feature is not ready for production use. + +When you enable this feature: + +- Every time you're mentioned, GitLab creates a new to-do item for you. +- Other [actions that create to-do items](#actions-that-create-to-do-items) + create one to-do item per action type on the issue, MR, and so on. + ## Create a to-do item You can manually add an item to your To-Do List. @@ -73,9 +94,7 @@ For example, in the following comment: - @carol can you please have a look? ->>> -@dan what do you think? ->>> +> @dan what do you think? @erin @frank thank you! ``` diff --git a/doc/user/usage_quotas.md b/doc/user/usage_quotas.md index 75d10084328..84a2449f481 100644 --- a/doc/user/usage_quotas.md +++ b/doc/user/usage_quotas.md @@ -2,7 +2,7 @@ type: howto stage: Fulfillment group: Utilization -info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#designated-technical-writers +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments --- # Storage usage quota **(FREE)** @@ -10,6 +10,9 @@ info: To determine the technical writer assigned to the Stage/Group associated w > - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/13294) in GitLab 12.0. > - Moved to GitLab Free. +NOTE: +Free tier namespaces on GitLab SaaS have a 5GB storage limit. This limit is not visible on the storage quota page nor currently enforced for users who exceed the limit. To learn more, visit our [pricing page](https://about.gitlab.com/pricing/). + A project's repository has a free storage quota of 10 GB. When a project's repository reaches the quota it is locked. You cannot push changes to a locked project. To monitor the size of each repository in a namespace, including a breakdown for each project, you can @@ -39,7 +42,7 @@ namespace to recalculate the storage. > - Enabled on self-managed in GitLab 14.5. > - [Feature flag removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/71270) in GitLab 14.5. -The following storage usage statistics are available to an owner: +The following storage usage statistics are available to a maintainer: - Total namespace storage used: Total amount of storage used across projects in this namespace. - Total excess storage used: Total amount of storage used that exceeds their allocated storage. diff --git a/doc/user/workspace/index.md b/doc/user/workspace/index.md index 0befcbe25d2..0ef3eb8b6fe 100644 --- a/doc/user/workspace/index.md +++ b/doc/user/workspace/index.md @@ -39,4 +39,4 @@ For a video introduction to the new hierarchy concept for groups and projects fo ## Related topics -- [Workspaces developer documentation](../../development/workspaces/index.md) +- [Workspace developer documentation](../../development/workspace/index.md) |
